Uprooting bad mathematical performance: Pilot study into roots of problems

South African students score poorly in mathematics and language tests when compared with students from other African countries and when compared with what should be expected 14 years after the achievement of democracy. But Martin CarnoyLinda Chisholm and Hlengani Baloyi believe it is critically important to go beyond simply measuring low achievement and to find ways of diagnosing learning problems with a view to enabling correction and remediation.


Some reasons for poor mathematics and language performance in schools may be evident, such as widespread agreement that the main challenge is the quality of education. Yet there is little empirical analysis that helps policy makers understand the low level of student performance in South African schools or how to improve it.

Pilot study: teachers' skills

As a first step toward unpacking the factors contributing to low levels of learning in South African schools, we engaged in a smallscale empirical pilot study that focuses on the role that teacher skills and practice play in South African students' learning within the socioeconomic and administrative conditions in those schools (and South African society more broadly). The main purpose of the pilot study was to test the instruments and assess the viability of our models.

The pilot was conducted on a sample of grade 6 mathematics lessons in 40 primary schools in Gauteng. Students, teachers and principals filled in questionnaires, students took tests at two points in the year to measure gains, and teachers' grade 6 mathematics classes were videotaped and analysed.

Students could choose to do the questionnaire and test in English, Afrikaans, or an African language. All chose English or Afrikaans. The teacher questionnaire included questions about mathematics teaching, specifically content and pedagogical content knowledge questions. Researchers provided additional notes about the general situation at the school. The information yielded is copious and the results instructive. There are a number of familiar and strikingly new findings.

What the results showed

The data revealed a primary school system characterised both by well-known low average levels of learner and teacher mathematical knowledge and by considerable inequality in the distribution of mathematical knowledge among those who teach students of lower and higher socioeconomic background.

Not surprisingly, results showed a high correlation between the average socioeconomic level of students in the school, the total mathematical content knowledge of teachers and the average student's mathematics test score in the school.

" Teachers' level of education matters much less than the type of teacher education institution attended "

Teachers' pedagogical content knowledge emerged as a critical issue in low achievement. It refers to the application of mathematical knowledge for teaching, especially young children. Examples include the powerful explanations teachers use to develop deep understanding of concepts that are part of the curriculum, the ways in which they draw linkages with other elements of mathematics, and the questions they pose to students. Teachers' reported level of education matters much less to student test scores and teacher content and pedagogical content knowledge than the type of teacher education institution attended - university, white, Indian, coloured or African urban or homeland college of education.

A day in a grade 6 maths class

Classes generally revolve around a considerable amount of teacherled presentation, with the teacher asking the students in the class to reply individually or in chorus to questions the teacher asks while making his or her presentation. This is usually followed by seatwork, in which the teacher circulates, checking students' work. Sometimes this is followed or preceded by students coming to the board and doing problems at the board (classified as recitation); in other classes, it is followed by more individual and chorus recitation in response to questions from the teacher.

As set out in Figure 1, a typical mathematics class in Gauteng's grade 6 is about one-third teacher-led, in which the teacher talks to the class, about 25% of class time is spent by the teacher asking questions, which are answered by individual students or in chorus, and about one-third of the time is dedicated to seat work.

Much of the recitation time (individual students and student chorus responding to the teacher) is mixed in with the teacher-led talking about the subject. In the most affluent schools, more time is spent on whole-class teacher presentations and on seatwork, and less on recitation.


Figure 1: Time spent on lesson

In the poorer classrooms, students are more likely to be seated with their desks grouped into 4 - 6 students facing each other, but when the students in such grouped situations are doing seatwork, it is almost entirely individual. That said, there are greater possibilities in a grouped situation of looking over at the other students' work, and students often do that. Actual work in groups uses only about 4% of class time. Although we could characterise typical lessons, there was large variation between lessons.

An important observation was the lack of coherence in a large percentage of the lessons. Teachers tend not to have a clear goal for the lesson. Some of the lessons started with a short mini-lesson on some topic and ended with an ‘activity' related to the topic, but unrelated to the mini-lesson.

Often the teacher does a mini-lesson but does not follow up with other activities. That is a big problem - lessons do not have sufficient substance to allow learners opportunities to consolidate what has been learned.

The other pattern observed was the lack of whole-class discussion on the activities or worksheets. The ‘discussion' is often just a chorus of agreement to given answers - or the completion of comments-prompted answers, that really give no indication of whether or not learners actually were able to give the answer themselves.

Our study tends to support empirically the claim that pedagogical content knowledge is important in improving student achievement, and that the mechanism by which this occurs is through the improved teaching of a subject by teachers who know more about the subject and how to teach it.

We cannot draw causal inferences from our results. This was a pilot study, so the empirical results, while important, are meant to provide direction for further research, but we are confident that our model goes far to explain why grade 6 students in other African countries seem to know so much more mathematics than students in South Africa.

The quality of teachers' training is probably a key variable (of why grade 6 students in other African countries seem to know so much more mathematics than students in South Africa) We would hypothesise from this study that the quality of teachers' training is probably a key variable in this explanation, and that we should find better teacher training reflected in higher teacher-measured content knowledge and pedagogical content knowledge in those other countries. In further studies we would also focus more energy on measuring opportunity to learn. Opportunity to learn is undoubtedly also an important factor in explaining student learning differences.

Martin Carnoy is Vida Jacks Professor of Education and Economics at Stanford University, and Linda Chisholm is a research director in the Education, Science and Skills Development programme at the HSRC. This article is based on a report, Towards understanding student academic performance in South Africa: A pilot study of grade 6 mathematics lessons in Gauteng Province, by Martin Carnoy, Linda Chisholm, et al (HSRC).

The full report can be downloaded here.