The good, the bad and the potential: unpacking TIMSS 2011

International studies of educational achieve-ment have been conducted since the 1960s. There are an increasing number of these kinds of studies and growing numbers of participating countries. Participation in international achievement studies allows for the comparison of performance with other countries and provides access to technical expertise in measurement and analysis, which can be shared and transferred. Vijay Reddy discusses the positives and negatives of such achievement studies, including the 2011 Trends in International Mathematics and Science Study (TIMSS).

Achievement studies are regularly conducted in South Africa. Mathematics and/or science assessments form part of the following comparative studies: Trends in International Mathematics and Science Study (TIMSS), Monitoring Learning Achievements (MLA), the Southern Africa Consortium for Monitoring Educational Quality (SACMEQ) initiated studies, and Performance in International Student Achievement (PISA).

South Africa participated in the TIMSS (1995, 1999, 2002 and 2011) and SACMEQ (2000, 2007) cross-national studies, with TIMSS 1995 providing the first national analysis of learner achievement. Participation in the four rounds of TIMSS has provided the country with systemic information, external benchmarking against other countries and a trend analysis of the mathematics performance since 1995. This has yielded significant insight into the well-being of the educational system. In addition to its participation in international studies, South Africa undertook systemic evaluations in 2002, 2004 and 2007. In 2011 the then-Department of Basic Education introduced the population based Annual National Assessments (ANAs) for grades 1 to 6 and grade 9.

Participating in international, cross-national achievement studies has both benefits and limitations. The main concerns about international comparative studies relate to:

  • Comparisons, or the league table presentation of the results, could take on a competitive edge with negative consequences;
  • TIMSS uses the curriculum as the major organising concept and a way to explain achievement, and this raises the concern of possible pressure for the gradual convergence of different curriculums;
  • Countries such as England and the United States are concerned about the possible negative effects of the TIMSS results trying to shape the national curriculums and push the curriculums to a “back to basics” approach at the expense of areas in which children are doing well; and
  • Although instruments are intended to be designed on the basis of consensus among countries, the instruments may be influenced by, and better suited to, the more influential countries. Large-scale assessment studies are expensive and need both financial and human resources. There are also opportunity costs linked to participating in such studies, especially in poorer countries.

Comparative achievement studies, whether loved or hated, catalyse debate when the results are published, which can benefit participating countries.

Benefits of participation in TIMSS

Lever of change

The publication of the 1999 TIMSS results in South Africa sparked a great deal of debate in different circles and was one of the events that catalysed an increased allocation of resources to science and mathematics at school level, thus acting as a lever of change for these two subjects.
TIMSS could have the potential to harness positive changes in countries where policy making may not be informed or influenced by key research, or in countries where there are no robust civil society structures to lobby for change. In countries with outdated curriculums and weak academic voices to campaign for improvements, the international agendas can sometimes be those that catalyse the change.

Providing a benchmark

In addition, the comparison of performance with countries of similar context and histories could provide a basis for benchmarking a country’s performance, thus exposing the strengths and weaknesses of its education system.

Benefit from international technical skills

Not all countries have the resources and capabilities to organise national studies, but international research organisations have a vast repertoire of technical skills to design and manage these surveys.

TIMSS: the origins

TIMSS is a project of the International Association for the Evaluation of Educational Achievement (IEA) that aims to provide trend information on learner achievement in mathematics and science. Boston College’s International Study Centre for TIMSS and PIRLS manages the international project activities. The other organisations that work closely with Boston College are Statistics Canada in Ottawa, the IEA Data Processing and Research Center in Hamburg (Germany) and Educational Testing Services in Princeton, New Jersey (USA).

South Africa participated in TIMSS 2011, and the articles that follow provide the first analysis of the data gathered. This analysis could provide insights to policy makers and practitioners for interventions that could contribute to improving the state of South African education.

Author: Dr Vijay Reddy, Executive Director, Education and Skills Development research programme, HSRC.

1 Programme d’Analyse des Systemes Educatifs des Pays de la Confemen (PASEC) is the French equivalent of SACMEQ.
2 PIRLS is the IEA’s Progress in International Reading Literacy Study.