Journal Articles

Questionnaire mode effects in interactive information retrieval experiments

---

The questionnaire is an important technique for gathering data from subjects during interactive information retrieval (IR)
experiments. Research in survey methodology, public opinion polling and psychology has demonstrated
a number of response biases and behaviors that subjects exhibit when responding to questionnaires.
Furthermore, research in human–computer interaction has demonstrated that subjects tend to inflate their ratings of systems when completing usability questionnaires. In this study we investigate the relationship between questionnaire mode and subjects’ responses to a usability questionnaire comprised of closed and open questions administered during an interactive IR experiment. Three questionnaire modes (pen-and-paper, electronic and interview) were explored with 51 subjects who used one of two information retrieval systems. Results showed that subjects’ quantitative evaluations of systems were significantly lower in the interview mode than in the electronic mode. With respect to open questions, subjects in the interview mode used significantly more words than subjects in the pen-and-paper or electronic modes to communicate their responses, and communicated a significantly higher number of response units, even though the total number of unique response units was roughly the same across condition. Finally, results showed that subjects in the pen-and-paper mode were the most efficient in communicating their responses to open questions. These results suggest that researchers should use the interview mode to elicit responses to closed questions from subjects and either pen-and-paper or electronic modes to elicit responses to open questions.

---

Availability

No copy data

Detail Information

Series Title

Information Processing & Management

Call Number

-

Publisher

s.l. : Elsevier., 2008

Collation

-

Language

English

ISBN/ISSN

0306-4573

Classification

-

Content Type

-

Media Type

-

Carrier Type

-

Edition

Vol. 44, No. 1, Page 122-141

Subject(s)

Interactive information retrieval experiment
Research methods
Questionnaires
Mode effects
Social desirability responding

Specific Detail Info

-

Statement of Responsibility

-

Other version/related

File Attachment

Comments

---

You must be logged in to post a comment