Benchmarking against the best in the world

"Instead of comparing ourselves with the past, we should compare ourselves with the best"              

by Michael Gove (2010)

Politicians and the media have long taken an interest in the results of the two main cross-national comparative studies of pupil achievement – The Programme for International Student Assessment (PISA, published 6 December 2016) and Trends in International Mathematics and Science Study (TIMSS, published 30 November 2016).

The Programme for International Student Assessment (PISA) is run by the Organisation for Economic Cooperation and Development (OECD) and takes place every three years. It is a sample survey that assesses 15–16 year olds in three areas: literacy, maths and science. Each wave of the study includes a particular focus on one of those areas to give more detailed results. Covering close to 80 school systems it is the largest international assessment. 

PISA claims to place less emphasis on whether a student can reproduce content and focuses more on their ability to apply knowledge to solve tasks. Pupils and head teachers also complete a survey, the idea being that scores can be linked with contextual data about a pupil's social background, attitudes towards learning, and the nature of the institutions they learn in. 

Trends in International Mathematics and Science Study (TIMSS) is run by the International Association for the Evaluation of Educational Achievement. It assesses 9-10 year olds and 13-14 year olds on their skills in both maths and science. TIMSS takes place every three years and more than 50 countries participate. It focuses broadly on curriculum and as a result tends to test pupil's content knowledge rather than their ability to apply it. 

UK politicians and media tend to give more prominence to PISA results than to TIMSS. 

Both PISA and TIMSS have been designed to study how different countries' educational systems are performing relative to one another and how this is changing over time. Countries are then ranked according to their students' performances. 

Supporters of this kind of benchmarking say it enables England, Northern Ireland, Scotland and Wales to get a sense of the potential for systems to improve and provides a peg for them to be measured against the best in the world. They say the exercise generates important data – providing insights that cannot be garnered solely from looking within the English system – and that it can provide a source of 'external challenge' to hidden assumptions within our own system that may be holding it back. 

Critics, on the other hand, say that league tables grossly oversimplify a set of complex variables to deliver simplistic judgements, which are then used to reform whole systems. The methodologies used in the sampling methods of international assessments have been criticised for being too small to reliably judge a whole system's performance – and for being open to countries 'gaming' the sample by excluding pupils who are likely to perform poorly. Comparing like with like they say is virtually impossible, and too often correlations are confused with causation. Furthermore, they say, country-specific factors – including the nature of curriculum, testing and teaching – can mean some pupils are better prepared for the format of these international assessments than others. 

In both the PISA and TIMSS rankings, the UK's performance between the cycles remains fairly similar. As countries move in and out of the assessment process, it is always difficult to compare country rankings and draw firm comparative conclusions.

Take a look at some of the following sources ...