European Commission has published the final deliverables from the study on Successful PISA stories in the EU – How some Member states have been able to improve their performance over time. This study was commissioned by the Directorate-General for Education, Culture, Sport and Youth (DG EAC). It was carried out by PPMI in cooperation with the EQUALE research group from the University of Liege.
The study explored performance trends over the period from 2009 to 2018 (the last four OECD’s Programme for International Student Assessment (PISA) cycles), and education policy factors that might be associated with the positive trends observed. The quantitative and qualitative analysis focused on seven EU Member States (Bulgaria, Estonia, Latvia, Poland, Portugal, Slovenia and Sweden) that we're able to demonstrate, at least to some extent, long-term improvements in performance or equity indicators as measured by PISA.
Factors associated with improvements in PISA performance
The statistical analysis methods (descriptive statistics ad multivariate regression modelling) implemented in the study have confirmed the findings from the broader literature on the key individual and school level factors that significantly influence students’ academic success as measured by PISA. Indeed, the analysis finds that individual and family background characteristics (gender, socio-economic status of the family, migrant background and similar) explain a significant part of students’ performance. The academic path of students is also closely linked to their academic performance (e.g., those who choose the vocational education track or have to repeat a grade usually perform lower). Finally, multi-level regression run on the selected countries showed that school and instructional factors (having more than four periods of mathematics per week, disciplinary climate, perceived cooperation between students, support from teachers and similar factors) also tend to play an important role in determining academic success.
Policies associated with the improvement in academic performance
Analysis of PISA journeys and the accompanying reform processes in Bulgaria, Estonia, Latvia, Poland, Portugal, Slovenia and Sweden demonstrated that policy-makers have made, and are still making, significant attempts in their quest to improve educational outcomes and address school-level inequalities by implementing various sets of reforms. In general, such reforms target structural changes, the development and assessment of competencies, and the improvement of equity.
Rigorously assessing the extent to which particular reforms and interventions ‘work’ and achieve their intended outcomes proves challenging due to several reasons. Firstly, there is a lack of an evidence-based education policy tradition in Europe. While the movement toward the use of evidence in education in Europe appears to be gradually asserting itself through political intent and initiative, it is yet to be systematically incorporated into the practices of the various Member States. Secondly, there is limited cross-sectional data on student achievements, which means that isolating the impact of educational policy reforms on PISA performance through statistical modelling is a challenging undertaking. Finally, analysing data coming from different PISA cycles can prove challenging, due to changes in the methodologies used, available variables, and the computation process of various indicators between cycles.
Even though the available evidence does not allow causal inferences to be made regarding the impact of most of these reforms on student outcomes (as measured by PISA), the stakeholder consultations and qualitative analysis conducted for this study bring valuable insights into the way in which reforms in the selected countries are initiated and translated into practice, which can either facilitate or impede the achievement of the intended outcomes. The study found that the reform is likely to be perceived as more effective and translate into practice more smoothly if they are implemented as a part of a comprehensive complementary set of measures, are well-planned and consistent with long-term education objectives, focus on both equity and quality, build on the consensus of the range of actors, and are regularly monitored and adapted.
You can find a more detailed overview of PISA journeys and recommendations for future education policies in the final report of the study (in English) and executive summary (in English, German and French). An in-depth analysis of different reforms in selected countries is available as an annex.
