Font Size:
Explaining Differential Item Functioning focusing on the crucial role of external information – an example from the measurement of adolescent mental health
Curt Hagquist, David Andrich
Building: Holme Building
Room: MacCallum Room
Date: 2018-12-13 03:30 PM – 05:00 PM
Last modified: 2018-10-17
Abstract
Background
In order to make valid comparisons between groups, a measurement instrument has to work invariantly across the groups. If an item of an instrument shows evidence of Differential Item Functioning (DIF), that DIF can be taken account of by resolving (splitting) the item into group specific items, rather than deleting the item. Resolving may improve fit and retain reliability but may affect the validity of the measure negatively if the source of the DIF is relevant for the content of the variable. This paper provides a concrete example of how external information can be used to study a hypothesis explaining the source of the DIF.
Methods
Swedish data from the international Health Behaviour in School-aged Children study collected in 2005/06, 2008/09 and 2013/14 were used. The sample comprised a total of 18 983 students in grades 5, 7 and 9, corresponding to 11, 13 and 15 years old students. A composite measure of psychosomatic problems was analysed for DIF with respect to gender using Rasch Measurement Theory. The specific substantive purpose of the study is to examine whether this gender DIF might be caused by a biological factor, i.e. the menstrual periods of girls. For this purpose analyses were disaggregated by grade and by whether or not premenstrual girls showed the same DIF compared to menstrual girls.
Results
The results confirm the hypothesis that the gender DIF for the item Stomach aches included in a psychosomatic problems scale is caused by the menstrual periods, i.e. a gender specific biological factor. Both the indirect and the direct approaches used in the study point in the same direction. Separate DIF analyses in each of the three grades show that the gender DIF increases by age. Also, girls who have had their first period show greater propensity for Stomach ache compared to those who haven’t, given the same location on the latent trait. Since this DIF is not constant along the trait the DIF is non-uniform.
Conclusions
External information about the menarche was required to carry out an in depth analyses of the source of the gender-DIF indicating that a biological factor was causing the gender DIF evident for the item Stomach ache. The consequence of this explanation is that the purpose of any comparisons between boys and girls on the psychosomatic measure governs the trade-off between fit/reliability and validity, i.e. whether the DIF-item should be resolved or not.
In order to make valid comparisons between groups, a measurement instrument has to work invariantly across the groups. If an item of an instrument shows evidence of Differential Item Functioning (DIF), that DIF can be taken account of by resolving (splitting) the item into group specific items, rather than deleting the item. Resolving may improve fit and retain reliability but may affect the validity of the measure negatively if the source of the DIF is relevant for the content of the variable. This paper provides a concrete example of how external information can be used to study a hypothesis explaining the source of the DIF.
Methods
Swedish data from the international Health Behaviour in School-aged Children study collected in 2005/06, 2008/09 and 2013/14 were used. The sample comprised a total of 18 983 students in grades 5, 7 and 9, corresponding to 11, 13 and 15 years old students. A composite measure of psychosomatic problems was analysed for DIF with respect to gender using Rasch Measurement Theory. The specific substantive purpose of the study is to examine whether this gender DIF might be caused by a biological factor, i.e. the menstrual periods of girls. For this purpose analyses were disaggregated by grade and by whether or not premenstrual girls showed the same DIF compared to menstrual girls.
Results
The results confirm the hypothesis that the gender DIF for the item Stomach aches included in a psychosomatic problems scale is caused by the menstrual periods, i.e. a gender specific biological factor. Both the indirect and the direct approaches used in the study point in the same direction. Separate DIF analyses in each of the three grades show that the gender DIF increases by age. Also, girls who have had their first period show greater propensity for Stomach ache compared to those who haven’t, given the same location on the latent trait. Since this DIF is not constant along the trait the DIF is non-uniform.
Conclusions
External information about the menarche was required to carry out an in depth analyses of the source of the gender-DIF indicating that a biological factor was causing the gender DIF evident for the item Stomach ache. The consequence of this explanation is that the purpose of any comparisons between boys and girls on the psychosomatic measure governs the trade-off between fit/reliability and validity, i.e. whether the DIF-item should be resolved or not.