A digital divide? Randomised evidence on the impact of computer-based assessment in PISA
EdCentral community rating

Add to my reading list
Professor John Jerrim

Between 2000 and 2012, PISA was carried out as a regular paper-based assessment. However, in 2015, pupils in the great majority of countries took the test on a computer instead. Since the change to computer-based assessment could affect pupil performance in itself – in ways that differ between countries – it has the potential to reduce the comparability of PISA test scores across countries and over time.

The researchers had access to data from three countries: Germany, Sweden and Ireland. In all three countries, the results show that pupils completing the computer-based test performed substantially worse than pupils completing the paper-based test. The difference is most pronounced in Germany (up to 26 PISA points), followed by Ireland (up to 18 PISA points) and Sweden (up to 15 PISA points).

Additionally, the researchers find little evidence of systematic gender differences in the impact of computers. The key conclusion is the adjustment made in PISA 2015 does not overcome all the potential challenges of switching to computer-based tests, but that it represents an improvement compared with not making any adjustment at all.

However, the results show that policymakers should take great care when comparing the results obtained via different modes, across and within countries.

In listing research, EdCentral makes no judgment or recommendation as to its quality, validity or methodology and none should be inferred. Through peer ratings left by education practitioners, EdCentral’s aim is to support the development of a repository of shared knowledge and experience.

*   Please note that your reading list can only be saved permanently if you are logged into your account.

Professor John Jerrim

Published by:
Centre for Education Economics

Date of publication:
January 2018

Country of origin:

CPD opportunities:

Of interest to those seeking to accurately compare performances in PISA 2015 tests between those taking a paper-based test and those taking a computer-based test.


Record ID:
R456 / 545
Rating Summary:

0 based on 0 votes

Start a discussion

Back to search results   |   New search   |   View my reading list   |   Print this page

EdCentral Logo