Introducing the Total Survey Error (TSE) framework.
Background
Researchers are commonly aware of the limits of surveys in generating population estimates due to sampling error. However researchers are less aware of survey limits due to non-sampling errors.
Total survey error (TSE) is a term that is used to refer to all sources of bias (systematic error) and variance (random error) that may affect the quality of survey data.
The focus of this paper is on the use of the Total Survey Error paradigm as an overarching framework to guide researchers through the entire survey cycle from design through to the analysis and reporting of results.
Researchers who attend this session will have the TSE framework explained and be given practical examples as to the types of survey errors that can occur and how to minimise them.
Total Survey Quality
TSE part of a larger construct known as Total Survey Quality (see Figure 1, next page). Most national statistical offices, including Australia, have similar survey quality frameworks to determine whether or not a survey is fit-for-purpose within given budget constraints.
 
Figure 1: Common dimensions of Survey Quality Framework
Dimension Description
Accuracy Total survey error is minimised
Credibility Data are considered trustworthy by the survey community
Comparability Demographic, spatial and temporal comparison are valid
Usability / Interpretability Documentation is clear and metadata is well organised
Relevance Data satisfy user needs
Accessibility Access to the data is user friendly
Timeliness / Punctuality Data deliverables adhere to schedules
Completeness Data are rich enough to satisfy the analysis objectives without undue burden on respondents
Coherence Estimates from different sources can be reliably combined
Source: Public Opinion Quarterly 74 (5): 819, 2010
The TSE framework
As a subset of Total Survey Quality, the Total Survey Error (TSE) paradigm is advocated by many leading survey methodologists as an overarching framework to help focus attention on opportunities to reduce error. Broadly defined, the TSE framework, as depicted in Figure 2 (see next page) represents all the potential sources of error that can make the data collected and conclusions generated in research misleading at best, or false at worst. Figure 2 illustrates the TSE Framework and indicates potential types of errors that should be considered throughout the conduct of a survey.
It is the author’s experience the TSE framework provides a critical lens for researchers to evaluate all aspects survey cycle with a view to identifying, ands looking to minimise, possible sources of error.
 
Figure 2: Total Survey Error Framework

Adapted from Lavrakas (2013); and Groves, Fowler, Couper, Lepkowski, Singer and Tourangeau (2009), Survey Methodology (2nd edition); Wiley.
Common sources of survey error – further explored
Having introduced the TSE framework, each of the main sources of error (as depicted in Figure 2) are further explored and practical examples, and mitigation strategies, provided.
References
Biemer, Paul B. Lyberg Lars (Eds) Special Issue: Total Survey Error Public Opin Q (2010) 74 (5): 1045 doi:10.1093/poq/nft058
Groves Robert M., Fowler Floyd J., Jr., Couper Mick P., Lepkowski JM., Singer E, Tourangeau R 2009.Survey Methodology (2nd edition).Hoboken,NJ: John Wiley and Sons.
Lavrakas, Paul J. “Presidential Address: Applying a Total Error Perspective for Improving Research Quality in the Social, Behavioural, and Marketing Sciences.” Public Opin Q (2013) 77 (3): 831-850 doi:10.1093/poq/nft033