Improving the effectiveness of our schools

Aside from personal attributes, learning outcomes for any particular child depend primarily on home characteristics, and secondarily on the influence of all the teachers through whose hands the child passes. For this reason the National School Effectiveness Study aimed to identify lessons for policy and practice for government, principals, teachers and parents. The study was designed to enable the gain scores of a learner over any one-year period to be related to the practices followed by the teacher for the same year. The study followed a group of children for three years, starting with Grade 3 in 2007 and ending in Grade 5 in 2009. Around 16 000 children participated in each year of data gathering, during which a cohort of 8 383 was tracked over all three years. NICK TAYLOR reports. The articles under this theme, on pages 30 to 33, explore the process of research on a national scale, and its findings.

Analysis of the National School Effectiveness Study  

 
Designing research on a national level 
  National policy lessons arising from any research study are most powerful if they can be shown to apply to the whole school population. To address this consideration, a nationally representative sample of 268 schools was drawn for the study. All provinces were included in the sample except Gauteng, which was excluded when it was discovered that provincial tests were being written at the same time as the first round of data collection.

The study assessed learner performance by means of literacy and mathematics tests, which were administered to the learners at the end of each year. It is common in large-scale studies to collect data on educational activities by means of survey questionnaires. Such methods do not always provide the most valid kind of data, given the well-known tendency for principals and teachers to place their practices in a favourable light. Thus, the school and classroom data was collected by means of interviews and direct observations, using structured instruments and fieldworkers experienced in the work of schools.  

Due to budget limitations, we did not undertake classroom observations. This is a limitation of the study, given the importance of teaching quality to learner performance. However, we did assess teacher practices through an analysis of planning and assessment records, and undertook a detailed analysis of pupil writing in both maths and literacy by looking at all the exercise books of the best student in each class. We also administered a very short test in their respective subjects to maths and language teachers.  

We used an asset-based method for assessing both school and learner socioeconomic status, and learners completed a questionnaire to describe the educational practices they experience at home: reading, homework, and exposure to the language of instruction of the school.  

Multivariate modelling techniques (involving several statistical variables) provided the first level of analysis of the study data. This exercise investigated the strength of relationships between proxies for educational activities in homes, schools and classrooms on one hand, and test scores on the other.  

Regarding home factors, a consistent pattern that emerged was that greater exposure to English through speaking and hearing English on the television was associated with higher achievement when controlling for home language and poverty. Children who read frequently at home on their own also performed better.

 

Socioeconomic status


Regarding the influence of socioeconomic status (SES) and school type, the model indicates that African-language students in historically white schools enjoy a considerable performance advantage over those in historically black schools. This difference is statistically significant and large, especially so in the case of numeracy.

It is clear from this analysis that, although achievement is strongly connected to socioeconomic status at home, much of this connection has to do with the effectiveness of schools in which School resources located.

Weak evidence was found that school resources such as pupil to teacher ratios and school facilities are associated with student achievement. As other studies have argued, more important than the mere presence of resources is how well they are managed. No resource is more poorly used in South African schools than time, and a positive effect was obtained in our regression models for schools in which the principal and all teachers were present on the day of the survey. Another school management factor positively associated with better literacy scores is whether the school has an inventory for textbooks and readers that is up-to-date. 

Quality of teachers

At the teacher level, a positive effect on both literacy and maths was obtained for schools in which a curriculum plan for the whole year could be produced. Schools where more than two English marks were seen in teacher assessment records scored better on the literacy test. Similarly, where the quality of assessment records was very poor, scores on the maths test were worse than where records were present and up-to-date. Teacher scores on the literacy test were not strongly associated with learner performance in literacy, but there was a significant effect of teacher knowledge on the model for numeracy, although this was only apparent for learners whose teacher scored 100% on the maths test.

Curriculum and performance

 

There was also a reasonably large, positive and significant effect on maths associated with having covered more than 25 curriculum topics, as identified in student workbooks.

A large and statistically significant negative impact on literacy scores occurred when no paragraph-length writing had been undertaken over the year, while a positive effect was found for schools in which more than 27 writing exercises of all types were counted in students’ English workbooks.

The indicators of good management identified in this research should not be interpreted as more than exactly that: indicators that point to the characteristics typically exhibited by good managers, rather than levers to be manipulated by policy to achieve improved student outcomes. The correlations revealed by our modelling exercise in many instances provide only very blunt responses to questions like, ‘Is the presence of an annual curriculum plan associated with better learner test scores?’ The answer to this question was affirmative, but that tells us little about what is entailed in these planning practices.

 

It seems likely that curriculum planning is one element in a constellation of activities undertaken by effective school leaders and teachers in order to optimise learning. And the really useful knowledge that principals and policy makers need to understand is what that constellation of activities consists of and how it ranges across schools which produce stronger and weaker test performances.

 

We undertook a set of case studies to investigate this and a number of related questions concerning school leadership practices. Similarly, we drew on a variety of data sets, in combination with the data from this study to better profile teacher subject knowledge, to probe the role of writing in language learning, and to describe the actual mathematics curriculum to which learners have access in class.

 

School leadership and management

 

Our case studies reinforced the findings from other research in the area of school leadership and management. What the principal and staff do together in a well-functioning school is to build systems which drive the work of teaching and learning.

 

Parents are incorporated into an extended pedagogical team. A structured division of labour distributes functions and integrates curriculum delivery across the classroom, the school and the home. School-level systems regulating the flow of work are time management, curriculum planning, assessment, book procurement and retrieval, and teacher professional development.

 

While there certainly are standard features to these systems, in general innovative solutions need to be found to local manifestations of the problems endemic to poor communities: learner hunger, tardiness, shortages of books and classrooms, and home conditions not conducive to parental engagement. The case studies provide vivid examples of how enterprising principals deal with these issues.

 

The case studies also looked at the issue of professional behaviour at the levels of both individual teachers and the school. We investigated three elements of teacher professionalism: ethical comportment, understanding the importance of subject knowledge as the foundation for teaching, and a sense of intrinsic motivation. The last element, and to a large extent the second one too, appear to be missing among most teachers and principals interviewed. In the large majority of the eight case studies, supported by a survey of 65 schools, principals underestimated the subject knowledge needs of their teachers.

Furthermore, those few teachers who do realise their own shortcomings in this area have little sense of agency with regard to their own knowledge, falling back on a passive dependence on the district to provide training.

Writing in language classes

 

   
   

The power of writing comes from its ability to leave a permanent trace. This unique characteristic allows the writer to reflect upon what has been written, generating and refining ideas in the process. Moreover, it allows ideas and information to be detached from space and time, giving them a capacity to reach a wide audience across continents and generations. Even more important for the development of individual children, the academic literature has firmly established the centrality of writing in shaping the way we think, reason, and learn. As Langer and Applebee (1987) put it, “To improve the teaching of writing ... is also to improve the quality of thinking required of school children”.

 

While writing helps us remember and better understand ideas, information and experiences, not all types of writing tasks have the same effect on learning. Some tasks, like writing summaries or analytical essays, require a deeper level of processing than answering fill-in-the blanks or short-answer questions.

 

Studies have found that the degree to which information is reformulated or manipulated through writing has an impact on how well the information is integrated, learned, and retained. This finding would seem to favour analytical essays as the writing task of choice, since they tend to demand careful structuring of an extended argument, and evaluation and reformulation of the material. The NSES study reveals that such writing is very seldom undertaken in South African classes, and this must rank as one of the biggest shortcomings of the school system, particularly for children from poor homes. On average South African children perform writing of any kind in language classes once in about four days.

Number of exercises and frequency of writing in Grade 5:

The most common form of writing seen in children’s workbooks consists of single words, with an average of 22 exercises of this type written over the course of the year. The next most frequently observed exercises consist of isolated sentences, which reflect an average of 12 exercises over the year. Writing of paragraph length or longer is very infrequent in South African schools, occurring on average only once a quarter (3.6 times a year).

Average frequency of writing paragraphs – number of exercises over the year:

 

Most disturbing is the number of books in which no paragraph writing at all was done over the year, a phenomenon seen in 44% of Grade 4 and 32% of Grade 5 classes.

 

Opportunity to learn mathematics

 

As in the language classes, all the writing books of the best learner in each maths class in grades 4 (in 2008) and 5 (2009) were examined. Using a list of all the topics specified in the curriculum, fieldworkers noted each topic on which one or more written exercises had been completed.

 

For each topic, we then computed the mean percentage of classes that had completed at least one written exercise.

 

The results are aggregated by the learning outcome in the table on the following page.

Coverage of each of the five learning outcomes in grades 4 and 5:

On average, only 24% of topics are covered in both grades 4 and 5. Overall, 88% of teachers had covered no more than 35 of the 89 (40%) topics specified in the Grade 5 maths curriculum, and 58% had covered no more than 20 topics, which make up only 22% of the curriculum.

 

There were very few topics where at least half the sample had completed one exercise or more. These included only the simplest of topics: counting, writing numbers, the operations of addition, subtraction and multiplication, and rounding off numbers. More advanced topics, including those which constitute the building blocks for a deeper, conceptual understanding of the subject, are covered by very few teachers.

It is clear that the overwhelming majority of South African mathematics schoolteachers avoid topics that are in any way challenging. These are also the topics that build conceptual understanding. Only the simplest of topics are taught to learners, and then largely in a mechanical, procedural fashion.

Attendance and punctuality by principals and teachers, thorough planning, frequency of assessment, teacher knowledge and curriculum coverage vary substantially across South African schools and are strongly linked to educational achievement.

 

For example, our modelling exercise estimated that the national average for maths could be expected to improve from 34,2% to 42,3% in response to raising teacher knowledge and curriculum coverage across the system.