Today, survey data is collected using a variety of survey modes, including face-to-face mode with or without show cards, self-administered paper and pencil, web, and telephone, among others. Previous studies have shown that asking the same question in different modes can result in different survey responses, resulting in so called mode effects. Some theories have been suggested to explain how such differences may occur. For example, revealing socially undesirable information in a self-administered mode more than in an interviewer-administered mode is explained by embarrassment and the wish of a respondent to look better in front of an interviewer. Similarly, primacy effects are explained by processing first options in a visual mode deeper than other options. While many of these theoretical explanations are borrowed from Psychology, few of them were tested directly using survey questions.
Eye-tracking is becoming a popular testing tool to understand how different forms of asking questions influence respondents’ answers. Galesic et al. (2008) has successfully shown how primacy effects can be detected through an eye-tracker which measures the time of an eye fixation at different points of a question or response scale. Until now, due to the ease of eye-tracking on PC, this method was used to test questions only in web/PC mode. Our paper extends the application of eye-tracking to other modes with a visual component, including self-administered paper and pencil (PAPI) and face-to-face (CAPI) with show-cards. Unlike eye-tracking on PC, tracking eye movements in different modes requires using an innovative real world eye-tracker which enables following eye movements anywhere respondent may look. Using an ASL MobileEye eye-tracker, which is worn on the head of a respondent, we explore differences in the visual attention of a respondent when s/he answers questions in different modes.
Three important points are of interest in this study. First, can mobile eye-tracking detect what a respondent is looking at (including show-card, interviewer, question wording and separate question response categories) in each of the three modes? Second, are there detectable differences in visual attention between modes (e.g. differences in the time a respondent spends looking at the first response category in the three modes)? And, third, do differences in visual attention detected via the eye-tracker have the potential to explain previously documented mode effects? We report on results from 12 volunteers tested in a Psychology Lab in the three modes: CAPI with show cards, web, and PAPI. We conclude with a discussion of the potential for eye-tracking in testing survey response theories.

Reference: Galesic, Mirta, Roger Tourangeau, Mick P. Couper, and Frederick G. Conrad. 2008. “Eye-Tracking Data: New Insights on Response Order Effects and Other Cognitive Shortcuts in Survey Responding.” Public Opinion Quarterly 72(5): 892-913.