Telephone surveys using an overlapping dual frame designs—mobile phone random digit dialing (RDD) numbers and landline RDD phone number frames—are considered to be best practice in Australia because of the proportion of the population who are mobile-only phone users (around 36%) and the difference in dual phone users responses by frame. Because of the costs of conducting telephone surveys using overlapping RDD dual frame designs augmentation using non-probability sampling frames are being proposed – referred to as blended sampling designs. Blended sampling begins with a probability sample which complimented by some non-probability sampling from an unknown sample frame (e.g. volunteers who sign up for web-based surveys). As blended sampling is still quite experimental and there are still differences of opinion on the effectiveness of the weighting to adjust for the issues introduced from using non-probabilistic sample frames we conducted a literature review to identify current methods and assess their ability to produce robust estimates.

The only peer reviewed article that we identified that describes the blended method in detail was that of DiSogra et al., 2011. They used a sample of 105 respondents from the probability sample and 99 from the opt-in non-probability sample making a total blended sample of 174 participants. The process was: (i) weight the probability portion of the sample (probability of selection, overlap adjustment and benchmarking) using cell or propensity weighting (ii)scale the sample back to add to the sample number (referred to as the adjustment factors) and combine with the opt-in non-probability sample (All given weight of 1); adjust to the population benchmarks. This however produced higher estimates than what was obtained from the probability sample and so they adjusted the estimates accordingly. Yeager et al., 2011 and Valliant & Dever 2011 in their reviews of the literature concluded that ‘non-probability samples yield data that are neither as accurate as nor more accurate than data obtained from probability samples’ and ‘propensity modeling in blended samples does not correct bias'. Since 2001 there have been several conference presentations expanding on this method but, as far as I am aware from our literature search, there are no peer reviewed articles that prove that this blended methodology and proposed weighting methods are appropriate.

We therefore conclude that, although it is attractive to compliment the RDD mobile numbers with non-probability frame numbers to potentially save money, there is currently no evidence that these methods produce estimates equivalent to those obtained from probability sampling.

References
DiSogra C, Cobb C, Chan E & Dennis JM 2011, ‘Calibrating Non-Probability Internet Samples with Probability Samples Using Early Adopter Characteristics’. Section on Survey Research Methods – Journal of Survey Methods, 4501-4515.

Valliant R & Dever JA 2011, ‘Estimating propensity adjustments for volunteer web surveys’, Sociological Methods & Research, 40(1); 105-137.

Yeager DS, Krosnick JA, Chang L, Javitz HS, Levendusky MS, Simpser A, & Wang R 2011, ‘Comparing the accuracy of RDD telephone surveys and internet surveys conducted with probability and non-probability samples’. Public Opinion Quarterly, 75(4); 709-747.