Lucky guess?: applying Rasch measurement theory to grade 5 South African mathematics achievement data

SOURCE: Journal of Applied Measurement
OUTPUT TYPE: Journal Article
PUBLICATION YEAR: 2019
TITLE AUTHOR(S): S.Bansilal, C.Long, A.Juan
KEYWORDS: GRADE 5, MATHEMATICS ACHIEVEMENT, TIMSS, TRENDS IN INTERNATIONAL MATHEMATICS AND SCIENCE STUDY
DEPARTMENT: Education and Skills Development (ESD)
Print: HSRC Library: shelf number 10902

If you would like to obtain a copy of this Research Output, please contact Hanlie Baudin at researchoutputs@hsrc.ac.za.

Abstract

The use of multiple-choice items in assessments in the interest of increased efficiency brings associated challenges, notably the phenomenon of guessing. The purpose of this study is to use Rasch measurement theory to investigate the extent of guessing in a sample of responses taken from the Trends in International Mathematics and Science Study (TIMSS) 2015. A method of checking the extent of the guessing in test data, a tailored analysis, is applied to the data from a sample of 2188 learners on a subset of items. The analysis confirms prior research that showed that as the difficulty of the item increases, the probability of guessing also increases. An outcome of the tailored analysis is that items at the high proficiency end of the continuum, increase in difficulty. A consequence of item difficulties being estimated as relatively lower than they would be without guessing, is that learner proficiency at the higher end is under estimated while the achievement of learners with lower proficiencies are over estimated. Hence, it is important that finer analysis of systemic data takes into account guessing, so that more nuanced information can be obtained to inform subsequent cycles of education planning.