Professor John Jerrim
Between 2000 and 2012, PISA was carried out as a regular paper-based assessment. However, in 2015, pupils in the great majority of countries took the test on a computer instead. Since the change to computer-based assessment could affect pupil performance in itself – in ways that differ between countries – it has the potential to reduce the comparability of PISA test scores across countries and over time.
The researchers had access to data from three countries: Germany, Sweden and Ireland. In all three countries, the results show that pupils completing the computer-based test performed substantially worse than pupils completing the paper-based test. The difference is most pronounced in Germany (up to 26 PISA points), followed by Ireland (up to 18 PISA points) and Sweden (up to 15 PISA points).
Additionally, the researchers find little evidence of systematic gender differences in the impact of computers. The key conclusion is the adjustment made in PISA 2015 does not overcome all the potential challenges of switching to computer-based tests, but that it represents an improvement compared with not making any adjustment at all.
However, the results show that policymakers should take great care when comparing the results obtained via different modes, across and within countries.
In listing research, EdCentral makes no judgment or recommendation as to its quality, validity or methodology and none should be inferred. Through peer ratings left by education practitioners, EdCentral’s aim is to support the development of a repository of shared knowledge and experience.
* Please note that your reading list can only be saved permanently if you are logged into your account.
