Data quality of questions sensitive to social-desirability bias in web surveys
Katja Lozar Manfreda, Nino Zajc, Nejc Berzelak, Vasja Vehovar
Building: Law Building
Room: Breakout 2 - Law Building, Room 026
Date: 2012-07-12 01:30 PM – 03:00 PM
Last modified: 2011-12-21
Abstract
Social desirability bias (SDB) in surveys refers to systematic error resulting from the desire of respondents to avoid embarrassment and project a favorable image to others. Respondents may choose responses they believe are more socially desirable or acceptable rather than choosing responses that are reflective of their true thoughts or feelings. This may occur more often by some respondents than others (this tendency is believed to be a personality trait based on the subject's need for approval), for some questions rather than others (experience show that is occurs more often when asking about voting behavior, behavior related to addiction, crimes, illnesses, sexual behavior, charity, financial matters, and being a well informed and cultured person) or for interviewer-administered rather than self-administered survey modes. For web surveys previous studies have shown that the problem of SDB is smaller than in other survey modes due to self-administration and perceived anonymity of the response process. Nevertheless, the presence of questions sensitive to SDB may negatively influence data quality in web surveys as well.
With our paper we will show in which cases the presence of questions sensitive to SDB negatively influences data quality. For this purpose we will study the behaviour of respondents in a sample of 50 surveys selected out of more than 2,000 web surveys, hosted by a web survey service at our University. We will explore how the following factors influence different indicators of data quality (item nonresponse, survey brake-off and response patterns):
- presence of questions sensitive to SDB (share among all questions, position in the questionnaire),
- reassuring subjects that their responses will be kept confidential or anonymous in the invitation page,
- characteristics of respondents.
The analysis will be performed at three levels – level of individual questions, level of questionnaire, level of respondent.
Studying several surveys with a meta-analytical approach and performing a multi-level statistical analysis is relatively rare approach when studying SDB. It provides precious insight into how presence of questions subject to SDB influences the behaviour. Based on that, new knowledge is generated about tailoring the questionnaire design and developing survey questions so to minimize nonresponse and social-desirability of answers.
With our paper we will show in which cases the presence of questions sensitive to SDB negatively influences data quality. For this purpose we will study the behaviour of respondents in a sample of 50 surveys selected out of more than 2,000 web surveys, hosted by a web survey service at our University. We will explore how the following factors influence different indicators of data quality (item nonresponse, survey brake-off and response patterns):
- presence of questions sensitive to SDB (share among all questions, position in the questionnaire),
- reassuring subjects that their responses will be kept confidential or anonymous in the invitation page,
- characteristics of respondents.
The analysis will be performed at three levels – level of individual questions, level of questionnaire, level of respondent.
Studying several surveys with a meta-analytical approach and performing a multi-level statistical analysis is relatively rare approach when studying SDB. It provides precious insight into how presence of questions subject to SDB influences the behaviour. Based on that, new knowledge is generated about tailoring the questionnaire design and developing survey questions so to minimize nonresponse and social-desirability of answers.