Online probability-based panels often apply two or more data collection modes to cover both online and offline populations. For example, in Australia, approximately one household out of every seven is without access to the internet, and that is why it is recommended to use an offline survey mode(s) to collect data from the offline population in online panel research. Generally speaking, one of the reasons why researchers choose a mix-mode approach is because it gives an opportunity to compensate for weaknesses of individual modes. However, when different respondent groups are collected data from using different survey modes, mode effects and differential measurement error may threaten the validity of results.

There are two specific hypotheses about the possible impact of shifting from one mode to another: social desirability response bias and satisficing. In addition to those two sources of measurement errors, differences in question format as a result of adjustments in different modes can add a net mode effect. While there has been a substantial body of research exploring mode effects of more traditional survey modes, there has been little research exploring mode effects in online panels, especially the probability-based ones.

In this study, I investigated mode effects between telephone, mail and online (panel) surveys through an experimental design controlling for sampling design; the administered surveys used either probability-based sampling methods or nonprobability-based sampling methods. The data analysed to explore the effects of mode and the effects of sampling design were the 2015 Australian Online Panels Benchmarking Study data and the Life in Australia panel Wave 2 data using the Health, Wellbeing and Technology Survey questionnaire. I primarily focused on measurement error dimensions such as primacy and recency, response non-differentiation, item non-response, straight-lining and social desirability. In addition to those, question format effect was considered as well.

The purpose of the presentation is to provide answers to the following research questions:
1. How significant are differences in distributions of response variables, associated with the mode of data collection?
2. How significant are differences in distributions of response variables, associated with the sampling design?
3. Which mode would be the right supplement to computer-assisted web interviewing, regarding mode effects, to collect data from the offline part of the population in probability-based online panel research?