When analysts estimate a multilevel model using survey data, they often use weighted procedures employing multilevel sampling weights to correct the effect of unequal probabilities of selection. This study addresses the impacts of including sampling weights and the consequences of ignoring them by assessing the performance of four approaches—the Multilevel Pseudo Maximum Likelihood (MPML), the Probability Weighted Iterative Generalized Least Square (PWIGLS), the naïve method (ignoring sampling weights), and the sample distribution method (Eideh and Nathan, 2009) for a linear random intercept model under a two-stage clustering sampling design.
Results show that the performance of each approach does vary according to the informativeness of a sampling design. None of methods is always superior to the others under all sampling designs. Ignoring informative sampling weights may not only have effects on the accuracy of the estimated intercept and the variance of random intercept, but it may also lead to incorrect inferences for all parameters. The sample distribution method produces less variable and more accurate estimates. However, when a sampling design is informative at the 2nd stage, it gives incorrect inferences for all parameters. The PWIGLS works well for all the fixed effects, but may not estimate the variance components accurately under an informative sampling design. Although the estimates of the fixed effects (except for the intercept and the variance of random intercept obtained by MPML) are close to unbiased, the performance is not satisfactory in terms of high mean square error and poor coverage rate.