Beyond Likert-Scale Survey Analysis: Using Open-Comments in Surveys to Depict Survey Users’ Context through Interpretation
Pamela J Theroux, Megan Tones
Building: Holme Building
Room: MacCallum Room
Date: 2014-12-10 09:00 AM – 10:30 AM
Last modified: 2014-10-31
Abstract
This session will present the findings from a longitudinal research project capturing “Closing the Gap” reforms in schools throughout Australia. As part of a larger federally-funded project, this session focuses on the survey data collected from school leaders and teachers over several years. The survey captures school staff perspectives on school reform initiatives and how these messages are translated into the schooling experience for staff and students. Using a quasi-experimental design, surveys were administered to school staff from schools participating in specified “Closing the Gap” school reform policy initiatives. These surveys were also administered to a control group of principals and teachers from matched schools not participating in the targeted reform effort to elicit their perceptions of this national policy initiative. The total sample represents almost 150 schools throughout Australia. Survey items were a mix of closed ended statements to record Likert-style responses of perceptions and practices, and open ended questions for leaders and teachers to record their own views via comments. Data were analysed via quantitative and qualitative analysis. Exploratory and confirmatory factor analysis were used to determine the underlying structure of school staff perceptions of the key reform messages-- such as high expectations and inclusivity of students from other cultures. School staff perceptions of these reform messages were then compared across reform and matched control schools. Qualitative coding of comments was also undertaken to identify major themes in leaders’ and teachers’ responses, and subsequently used to interpret findings from the quantitative survey results. This session presents a study that was unique in its design, scope and content giving priority to survey participant’s open-ended comments as a viable insight into school staff perception and practice of school reform policy initiatives.