Foreword
Large-scale education surveys are a core asset for education policy making. They foster new analyses and findings about ways to improve the professionalism of teachers and school leaders, and the quality and equity of education around the world. The combination of several trends, including the increasing migration of surveys to digital devices and the technical progress made in data collection, storage and processing, are leading to unprecedented opportunities in the education sphere. Over the past two decades, the OECD Directorate for Education and Skills, together with its member and partner countries, has collected huge volumes of valid, reliable, comparable, rigorous, interpretable and policy-relevant data on teachers, school leaders and students around the world. Let us take a moment to zoom in on the two largest OECD education surveys to date.
The OECD Programme for International Student Assessment (PISA) provides the most comprehensive and rigorous international assessment of student learning outcomes to date. PISA also collects large amounts of data on students’ family backgrounds, schools, learning conditions and on students’ lives in general. Results from PISA indicate the quality and equity of cognitive and social-emotional outcomes attained around the world, and allow educators and policy makers to learn from the policies and practices applied in other countries. Today, PISA brings together more than 90 countries and economies, representing 80% of the world economy, in a global conversation about education.
The OECD Teaching and Learning International Survey (TALIS) is the largest international and periodic survey asking teachers and school leaders about their working conditions and learning environments. Based on the voices of more than 260 000 teachers representing more than 8 million teachers from around 50 countries and economies worldwide, TALIS 2018 offers orientations to help strengthen the knowledge and skills of the teaching workforce to support teaching and learning.
Now, imagine combining all these data into one, big dataset. Here is what happened: in 2018, nine countries and economies administered both TALIS and PISA to teachers, school leaders and students of the same schools. This has resulted in the creation of the TALIS-PISA link dataset, which comprises thousands of variables from more than 30 000 students and more than 15 000 teachers from more than 1 000 schools on four different continents. The OECD’s job is to make sense of it all: to understand “what the data say” and extract important patterns that can inform education policy.
To achieve this, analysis relies on the thorough conceptual process that has taken place prior to collecting this data. None of the variables in the TALIS and PISA datasets were included by chance! Every question in our questionnaires and assessments was devised, revised and refined by large international expert groups following a comprehensive iterative process; informed by previous research findings, conceptual frameworks, outcomes of survey piloting and field trialling; and in consultation with many stakeholders from the teaching profession and the education sector. All these data were collected with the view of shedding light on important education issues.
At the same time, the challenges related to the complexity and size of these TALIS-PISA link data required complementing this theoretical approach with data-driven techniques borrowed from the rapidly expanding field of machine or statistical learning. At the OECD, not only do we examine the role that artificial intelligence (AI) might have in improving education processes and preparing students for increasingly automated economies and societies – readers can look at our paper on Trustworthy artificial intelligence (AI) in education in support of the G20 artificial intelligence dialogue, for example – but we are also agents of this transformation. This report is the first of its kind to explore the potential of one particular form of AI – supervised statistical learning – to improve our understanding of the nexus between teaching and learning.
Yet, while applying statistical learning to the TALIS-PISA link data, the analysis never loses sight of theory and inputs from education stakeholders. This report maintains a steady dialogue between theory, fieldwork and data. It first addresses three questions: According to past research, what teacher and school factors matter the most for student learning outcomes? For student social-emotional development? For all students to thrive and fulfil their potential, regardless of their personal characteristics? Then the report turns to the data and tries to understand “what the data say” using statistical learning complemented with more standard statistical analyses. The analysis goes back and forth between data results and the available research literature. New statistical analyses can lead to unexpected findings. Yet, existing research studies can be used to validate, complement and put the new findings into perspective.
So what have we learned? If there was only one conclusion to take away from this report, it is that what teachers do in and outside the classroom matters the most – and the most directly – for the cognitive and social-emotional outcomes of the school’s students. Classroom practices that create opportunities to learn, teachers’ use of working time, as well as the well-being and job satisfaction of the teachers are among the most influential school factors. But this report does not only confirm the crucial role of teachers in young people’s development. It also sheds light on other actors. Students’ classmates and schoolmates, as well as the school’s culture and leadership (including the role parents play) are also found to matter a lot for student outcomes. All this suggests that any education reform or any change in education practice should tackle one or several of these dimensions first for students to be able to thrive and be active participants in our economies and societies.
A few months ago, we released the OECD Global Teaching Insights, our first large-scale video study of teaching, which provides a detailed account of how teaching and learning take place in the classrooms. For the first time, the OECD has implemented a study that combines the observation of lessons via video and a longitudinal design able to measure growth in student outcomes. With this report, Positive, High-achieving Students? What Schools and Teachers Can Do, the OECD is now trialling new research methods to shed light on the key factors that can be activated to raise student outcomes and reduce disparities deriving from personal characteristics and circumstances. And the OECD will continue to do so in the future. We will keep reflecting on, revising and refining our research and policy questions, our study designs and our research methods to reap the most of what teachers, school leaders and students tell us about teaching and learning. Stay tuned.
Director for Education and Skills
Special Advisor on Education Policy to the OECD Secretary-General