Foreword

Back in 2018, while working at the OECD Secretariat, the editors of this report were developing the first global assessment of students’ creative thinking. Countries around the world were also administering the latest cycle of the OECD’s Programme for International Student Assessment (PISA) tests, which for the first time included an assessment of global competence. These two “innovative domains” represented new frontiers for PISA. The global competence assessment aimed to provide comparative evidence on a multidimensional construct that included attitudinal and socio-emotional elements. The creative thinking assessment was also being developed to include open-ended items across four different domain areas, with complex scoring rubrics guiding human coders. These experiences made it clear that developing new assessments of complex skills was – unsurprisingly – a highly complex task, starting from how such constructs should be defined through to how they should be measured, validated and reported.

The editors then had an idea: what if PISA could bring together an international group of senior experts in educational measurement and assessment design to discuss the next generation of educational assessments? In 2019, the PISA Research and Innovation Group was created and thus held its first meeting to discuss a strategy for the next cycles of the PISA “innovative domains”, the purpose of which are to develop new measures of relevant 21st Century competencies in large-scale student assessment and to foster innovation in the way that students are assessed.

It soon became clear that the need for change in educational assessment reached beyond a single “innovative” PISA component. With that, the scope of the group’s work expanded: in 2020, PISA member countries decided to establish and fund a dedicated Research, Development and Innovation programme in PISA; and the OECD began to develop the Platform for Innovative Learning Assessments (PILA), a tool for prototyping innovative assessment tasks that can be used either for formative or summative purposes.

This report is the product of a collaborative, multi-year effort between the PISA Research and Innovation Group, various experts in the field of educational measurement and assessment design, and the OECD PISA Secretariat. It summarises much of the research work carried out by members of the group since its first meeting in 2019. This report aims to provide future directions for the development of innovative PISA assessments that focus on measuring complex skills. It also aims to support countries around the world who are engaged in reforming or expanding the scope of their national assessment systems. As more countries include skills like creative thinking, self-regulation and complex problem solving in their curricula goals, it is important that assessments evolve to better measure and support these skills in ways that are valid and instructionally relevant.

This report does not aim to provide a list of new assessments that PISA or national education systems should implement. Rather, it unpacks some of the key arguments in support of innovating assessments and the challenges facing assessment designers in doing so. To this end, the report presents design principles supporting the design of assessments that are fit for their purpose, some future directions based on promising examples – and some new questions that we need to address. We hope this report inspires ambition to pursue the goal of innovating assessments, both in PISA and beyond, while highlighting areas for future research and collaboration.