The London Challenge

Gabriel Heller Sahlgren, PhD student at LSE and Political Science and Director of Research, Centre for the Study of Market Reform in Education (CMRE), shares his thoughts on one of the most talked about issues in education. 'The London Challenge' a policy designed to create a "step change" in the performance of London secondary schools. 

An important problem in education policy is that pundits and politicians tend to draw strong conclusions regarding the success or otherwise of specific policies, without having adequate research to back them up.

An important example of this is the London Challenge, which has been widely praised as a successful policy to promote better outcomes via school-to-school collaboration. The arguments in favour of the policy were almost always based on a simple before-after comparison, without adequate consideration of other things that were going on at the same time. There were a lot of changes in the schools landscape at the same time as the London Challenge was in operation – and improvements that were due to the other changes may then mistakenly be attributed to the London Challenge.

It is also important to notice that this problem cannot be remedied by qualitative research techniques. Last year, the CfBt and the Centre for London published a report using focus group interviews with teachers and other parties. This report highlighted a range of policy-relevant factors for the success of London schools, including the London Challenge.

But there is no way to answer the question of what caused the success of London schools with these methods. Researchers' observations and interviews can at best understand the motivations of actors, but they are insufficient for understanding what policies caused pupil performance to improve. Again, there are too many things going on at the same time.

This is evident when considering more quantitative investigations of the issue. One study by Simon Burgess at Bristol University found that the only reason London secondary schools are doing better is because of a more favourable ethnic composition. By merely holding constant the ethnic background of the pupils, the London advantage evaporates. London schools have been blessed with high-achieving Asian and Eastern European kids. The phenomenon is down to composition, not policy.

Also, even more recently, a group of economists showed that the actual improvements in London schools that have occurred over the years could be explained by changes in primary schools, where the London Challenge wasn't even implemented. The improvements also started in the mid-1990s, long before the policy was even rolled out.

The major point to take away from this is that mere before-after comparisons and qualitative research cannot be used to prove causality between policies or practices and pupil outcomes. There are so many other things going on at the same time, and it is impossible to find out what caused what. The idea that the London Challenge was behind the improvements in London schools was politically attractive, but never received rigorous support in the literature. It is a case study of the dangers of drawing policy conclusions from less rigorous research.