Chapter 4. Aligning school evaluation with its core purposes of accountability and improvement
This chapter looks at how the Republic of North Macedonia can align school evaluation with its core purposes of accountability and improvement. The country has developed a robust school evaluation framework, however it has not been fully implemented or appropriated by stakeholders. Rather than encouraging a culture of reflection in the country, school evaluation focuses largely on compliance. This is exacerbated by a useful, yet complicated evaluation framework, which inspectors and schools find difficult to apply, and little support to schools to use evaluation results to lead improvements. North Macedonia should take steps to bridge the gap between the purpose of school evaluation and its perception among stakeholders. Another key priority is to make the process more manageable and provide schools with greater support to ensure that they appropriate evaluation and direct improvement.
The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law.
Introduction
The core purpose of school evaluation is to help schools improve their practices and keep them accountable for the quality of the education that they provide to students. Over the past ten years, the Republic of North Macedonia (referred to as “North Macedonia” hereafter) has developed a school evaluation framework that covers the key areas that are important for an effective school evaluation system, in particular the quality of teaching and learning practices.
However, this framework has not been fully implemented or appropriated by stakeholders. Both external and self-evaluation focus largely on ensuring compliance with the framework, rather than encouraging a culture of reflection and improvement in schools. Fundamentally, this reflects a disconnect between the aims of the framework – to enhance school quality and school-led improvement – and the perception of evaluation among inspectors and schools as an administrative requirement. This is exacerbated by a useful, yet complicated evaluation framework, which inspectors and schools find difficult to apply, and by the lack of support to schools on how to use evaluation results to inform improvements efforts.
The State Education Inspectorate (SEI) needs to be urgently reformed to take on a role of leadership and responsibility for the quality of the country’s schools. This role will need to be matched by far greater scrutiny and accountability of the inspectorate, so that it is accountable for the quality of its work. Another priority is to better support schools ‒ through evaluation follow-up, training, data and adequate financial resources – so that they can appropriate evaluation to direct improvements. Finally, small changes to the evaluation framework and process will help to better orient evaluation towards improvement and create a more manageable process for schools and inspectors.
Key features of an effective school evaluation system
In most OECD countries, school evaluations ensure compliance with rules and procedures, and focus increasingly on school quality and improvement (see Figure 4.1). Another recent trend has been the development of school self-evaluation, which has become a central mechanism for encouraging school-led improvement and objective setting. Internationally, strengthened systems for external and school-level monitoring and evaluation are seen as essential complements to the increasing decentralisation of education systems to ensure local and school accountability for education quality.
Frameworks for school evaluation ensure transparency, consistency and focus on key aspects of the school environment
Frameworks for school evaluation should align with the broader aims of an education system. They should ensure that schools create an environment where all students can thrive and achieve national learning standards. As well as ensuring compliance with rules and procedures, effective frameworks focus on the aspects of the school environment that are most important for students’ learning and development. These include the quality of teaching and learning, support for teachers’ development, and the quality of instructional leadership (OECD, 2013[1]). Most frameworks also use a measure of students’ educational outcomes and progress according to national learning standards, such as assessments results or teachers’ reports.
A number of OECD countries have developed a national vision of a good school (OECD, 2013[1]). The vision guides evaluation, helping to focus on the ultimate purpose of ensuring that every school is good. Visions are often framed around learners, setting out how a good school supports their intellectual, emotional and social development.
Countries’ external evaluations balance accountability and improvement
The vast majority of OECD countries have external school evaluation (see Table 4.1). Schools tend to be evaluated on a cyclical basis, most commonly every three to five years. (OECD, 2015[2]). Within the broad purpose of evaluating school performance, some countries emphasise accountability for teaching quality and learning outcomes. In these countries, national assessment data, school ratings and the publication of evaluation reports play an important role. In contrast, in countries that place greater emphasis on improvement, evaluations tend to focus more on support and feedback to schools. They also place strong emphasis on helping schools develop their own internal evaluation and improvement processes.
Evaluations aim to establish a school-wide perspective on teaching and learning
Administrative information for compliance reporting is a standard source of information for evaluations, although it is now collected digitally in most countries (OECD, 2015[2]). This frees up time during school visits to collect evidence of school quality. Most evaluations are based on a school visit over multiple days. Visits frequently include classroom observations. Unlike for teacher appraisal, these observations do not evaluate individual teachers but rather aim to cover a sample of classes across different subjects and grades to establish a view of teaching and learning across the school. Inspectors also undertake interviews with school staff, students and sometimes collect the views of parents. Since much of this information is qualitative and subjective, making it difficult to reliably evaluate, countries develop significant guidance such as rubrics for classroom observations to ensure fairness and consistency.
Many countries have created school inspectorates in central government
External evaluations are led by national education authorities, frequently from central government (OECD, 2013[1]). Across Europe, most countries have created an inspectorate that is affiliated to, but frequently independent of government. This arrangement ensures integrity and enables the inspectorate to develop the significant professional expertise necessary for effective evaluation. School inspectors may be permanent staff or accredited experts contracted to undertake evaluations. The latter provides flexibility for countries, enabling them to meet the schedule of school evaluations and draw on a range of experience, without the costs of maintaining a large permanent staff. Inspectors across OECD countries are generally expected to have significant experience of the teaching profession.
Source: (OECD, 2015[2]), Education at a Glance 2015: OECD Indicators, OECD Publishing, Paris, https://dx.doi.org/10.1787/eag-2015-en.
The consequences of evaluations vary according to their purpose
To serve improvement purposes, evaluations must provide schools with clear, specific feedback in the school evaluation report, which helps them understand what is good in the school, and what they can do to improve. To follow-up and ensure that recommendations are implemented, countries often require schools to use evaluation results in their development plans. In some countries, local authorities also support evaluation follow-up and school improvement. Around half of OECD countries use evaluation results to target low-performing schools for more frequent evaluations (OECD, 2015[2]).
In most countries, evaluations also result in a rating that highlights excellent, satisfactory or underperforming schools. To support accountability, most OECD countries publish evaluation reports (OECD, 2015[2]). Public evaluation reports can generate healthy competition between schools and are an important source of information for students and parents in systems with school choice. However, publishing reports also risks distorting school-level practices such as encouraging an excessive focus on assessment results or preparation for evaluations. This makes it critical that evaluation frameworks emphasise the quality of school-level processes, and an inclusive vision of learning where all students, regardless of ability or background, are supported to do their best. Evaluation systems that emphasise decontextualised outcome data like assessment results are likely to unfairly penalise schools where students come from less advantaged backgrounds, since socio-economic background is the most influential factor associated with educational outcomes (OECD, 2016[3]).
Self-evaluation is an internal tool for improvement
Most OECD countries require schools to undertake self-evaluations annually or every two years (see Figure 4.2). Self-evaluations encourage reflection, goal setting and inform school development plans (OECD, 2013[1]). To be an effective source of school-led improvement, many countries encourage schools to appropriate self-evaluation as an internal tool for improvement rather than an externally imposed requirement. In some countries, schools develop their own frameworks for self-evaluation. In others, they use a common framework with external evaluation, but have the discretion to add or adapt indicators to reflect their context and priorities.
The relationship between external and internal evaluations varies across countries. In general, as systems mature, greater emphasis is placed on self-evaluation while external evaluation is scaled back. Most OECD countries now use the results from self-evaluations to feed external evaluations, with, for example, inspectors reviewing self-evaluation results as part of external evaluations. However, the relationship is also shaped by the degree of school autonomy – in centralised systems, external evaluations continue to have a more dominant role, while the reverse is true for systems that emphasise greater school autonomy.
Effective self-evaluation requires strong school-level capacity
Effective self-evaluation requires strong leadership, and strong processes for monitoring, evaluating and setting objectives (SICI, 2003[4]). Many OECD countries highlight that developing this capacity in schools is a challenge. This makes specific training for principals and teachers in self-evaluation – using evaluation results, classroom and peer observations, analysis of data and developing improvement plans – important (OECD, 2013[1]). Other supports include guidelines on undertaking self-evaluations and suggested indicators for self-evaluations.
While a principal’s leadership plays a critical role in self-evaluation, creating teams to share self-evaluation roles is also important. The most effective self-evaluation team involves a range of staff that are respected by their colleagues and have a clear vision of how self-evaluation can support school improvement. In order to support collective learning, self-evaluation should engage the whole school community. This includes students, who have a unique perspective on how schools and classrooms can be improved (Rudduck, 2007[5]). Students’ views also help to understand how the school environment impacts students’ well-being and their overall development. This is important for evaluating achievement of a national vision focused on learners.
Data systems provide important inputs for evaluation
Administrative school data – like the number of students, their background and teacher information – provides important contextual information for internal and external evaluators. Increasingly, countries use information systems that collect information from schools for multiple purposes including evaluation and policy making.
Most countries also collect information about school outcomes. Standardised assessments and national examinations provide comparative information about learning to national standards. Some countries also use this information to identify schools at risk of low performance and target evaluations (European Commission/EACEA/Eurydice, 2015[6]). However, since assessment results do not provide a full picture of a school, they are often complemented by other information like student retention and progression, student background, school financial information and previous evaluation results. A number of countries use this data to develop composite indicators of school performance. Indicators frequently inform evaluation and support school accountability.
Principals must be able to lead school improvement
Strong school leadership is essential for effective school self-evaluation, and school improvement more generally. Principals support evaluation and improvement through a number of leadership roles – defining the school’s goals, observing instruction, supporting teachers’ professional development and collaborating with teachers to improve instruction (Schleicher, 2015[7]). This diversity points to a major shift in the principal’s role in recent years, with principals increasingly leading instructional improvement.
Principals need a deep understanding of teaching and learning, and strong leadership skills to become instructional leaders
Most principals bring significant experience of the teaching profession – among the countries participating in the OECD Teacher and Learning International Survey (TALIS), the average principal has 21 years of teaching experience. Teaching experience alone however is not sufficient, and the ability to demonstrate strong leadership of the school community is particularly important. Nearly 80% of principals in TALIS participating countries reported that they received training in instructional leadership either before or after taking up their position, or both (OECD, 2014[8]).
Principals’ initial training must be complemented by opportunities for continued professional development once in post. One of the most effective types are collaborative professional learning activities, where principals work together to examine practices and acquire new knowledge (DuFour, 2004[9]). In countries where international assessment results suggest that learning levels are high like Australia, the Netherlands and Singapore, more than 80% of principals reported participating in these kinds of activities in the last 12 months (OECD, 2014[8]).
Professionalising school leadership – standards, selection and appraisal
Given the important role that principals occupy, OECD countries are taking steps to professionalise the role. A number of countries have developed professional principal standards that set out what a school leader is expected to know and be able to do. Principal standards should include how principals are expected to contribute to self-evaluation and improvement. Similar to teachers, principal standards guide the recruitment of principals, their training and appraisal.
Around half of OECD countries have legislated appraisal of school leaders (see Figure 4.3) (OECD, 2015[2]). These kinds of appraisals hold principals accountable for their leadership of the school, but also provide them with valuable professional feedback and support in their demanding role. Responsibility for principal appraisal varies. In some countries, it is led by central authorities, like the school inspectorate or the same body that undertakes external teacher appraisals. In others, it is the responsibility of a school-level body, like the school board. While the latter provides the opportunity to ensure that appraisal closely reflects the school context, boards need significant support to appraise principals competently and fairly.
Notes: Data for Lithuania are drawn from (European Commission/EACEA/Eurydice, 2015[6]) Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe, The European Union, Luxembourg, https://publications.europa.eu/en/publication-detail/-/publication/4a2443a7-7bac-11e5-9fae-01aa75ed71a1/language-en (accessed on 15 June 2018).
Sources: OECD (2015[2]), Education at a Glance 2015: OECD Indicators, OECD Publishing, Paris, https://doi.org/10.1787/eag-2015-en and European Commission/EACEA/Eurydice (2015) Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe, The European Union, Luxembourg, https://publications.europa.eu/en/publication-detail/-/publication/4a2443a7-7bac-11e5-9fae-01aa75ed71a1/language-en (accessed on 21 June 2018).
School governance in North Macedonia
The school leadership function is relatively underdeveloped in North Macedonia. Principals receive little training or guidance to lead the school, and limited support from school boards. In recent years the Ministry of Education and Science (MoES) has taken some steps to address these concerns. In 2016, a new training and licensing process was introduced to ensure that new principals receive minimum preparation in core aspects of school leadership. However, the principal role remains primarily administrative, and principals’ capacity to steer teaching and learning and set goals for improvement remain relatively limited. In addition, the politicisation of the appointment process and high turnover rate make it difficult to develop a corps of professional school principals with experience and competence.
Principals receive little training in instructional leadership and school management
As is the case in most European countries, school principals in North Macedonia are required to have a Bachelor’s degree and some teaching experience (at least five years) (Eurydice, 2013[10]). However, the Law on Principles envisages that principal candidates must undertake 192 hours of mandatory training over a year provided by the National Examination Centre (NEC) since 2016. Contrary to training programmes in many OECD and European countries, the training in North Macedonia is primarily theoretical and does not include practical training, where new school principals or candidates can observe the work of an experienced principal (Eurydice, 2013[10]). At the end of the training, candidates take the school principal certification and licensing examination, also administered by the NEC (60% in 2018). The examination is organised at least twice a year and consists of three parts: practical test of computer skills, test to assess the candidates’ theoretical knowledge and a presentation of a seminar paper, including a case study. After passing the certification exam, new principals need to find a school placement within five years.
Once in school, principals receive very limited in-service training (four days a year) and there are no specific modules for vocational education and training (VET) schools. The principals have to find and pay for the professional development programmes they undertake themselves.
The hiring of school principals is highly politicised
School boards are responsible for selecting principals. The board issues a call for application and selects a candidate from a pool of licensed principals. However, there are no national guidelines or criteria to guide the board in the selection process. Municipalities are responsible for validating the board’s choice for secondary education institutions. The municipality can only refuse a board’s nomination once. This rule was introduced in 2004 to try to curb municipal political interference in school appointments. However, it was reported to the OECD team that in practice, political influence means that principals often continue to be selected from candidates of the same party in power at the municipal level.
High turnover make it difficult to develop a professional school leadership body
The school board is also responsible for renewing principals’ appointments at the end of their four-year mandate, but most principals are not renewed and go back to being full-time teachers. For example, principals surveyed for this review reported being a principal for four years on average. This is short compared to the average years of experience that principals in OECD countries report - nine years) (OECD, 2014[8]). It was reported to the review team that one reason for such frequent change is that principals are rarely renewed when the party in power at the municipal level changes. In addition, the SEI, the body in charge of school external evaluation in North Macedonia, is sometimes requested to conduct ad hoc school inspections to justify a political decision to dismiss a principal.
School boards are involved in key strategic decisions, but lack training and independence
School boards are also responsible for validating school self-evaluation reports and schools’ annual work plans, and indicating the school’s budget needs to the municipality. The board comprises 9 members in primary schools and 12 members in secondary schools representing the teacher council, the parent association, students and the local community, elected for 4 years.
Despite their important responsibilities, boards receive limited national support. For example, there are no national guidelines for boards. While there are manuals and workshops available, the board members that the review team met had not received any training. Political interference in the boards’ decisions regarding principal hiring and firing, and their general functioning further hinder their capacity to steer decisions at the school level.
There are few formal school roles or bodies that support the principal
Principals are not supported by any administrative staff, and do not have a deputy to share leadership responsibilities. There is no formal practice of experienced teachers taking on leadership for their subject or for teaching more broadly across the school. While there is a proposal to define clearer pedagogical leadership for teachers as part of a differentiated career structure, this has yet to be implemented (see Chapter 3).
The main support for principals comes from the multi-actor team of a pedagogue, psychologist and special needs teacher. However, there are a number of issues which hinder the team’s capacity to support teaching and learning effectively. Notably, their lack of practical teaching experience and predominant focus on identifying special learning needs, rather than being driven by a more inclusive approach for adapting learning to the individual needs of all students (see Chapter 3).
School evaluation in North Macedonia
North Macedonia has well-established practices for school evaluation. In 2011, an external evaluation process, called the School Integral Evaluation carried out by the State Education Inspectorate, was introduced. The process aims to evaluate the quality of schools to inform improvement by providing formative feedback to stakeholders. Since 2008, schools have also been required to undertake self-evaluation every two years (Table 4.1). However, in reality, both school external and self-evaluation focus primarily on compliance with regulation and administrative processes, and do not give schools specific, high quality feedback to improve learning and teaching practices.
School performance quality indicators set expectations for school evaluation but gaps remain
The School Performance Quality Indicators (SPQI) framework developed in 2011, and refined in 2014, is the key reference document for school evaluation in North Macedonia. The SPQI framework was modelled on indicator frameworks in countries with long standing traditions of school evaluation such as Scotland in the United Kingdom and the Netherlands. It includes seven areas of evaluation, which are common to both the external evaluation carried by the State Education Inspectorate and schools’ self-evaluations (see Table 4.2).
These areas cover most of the main factors that research suggests are important for school quality such as teaching and learning practices, school environment, school planning and management. Each area includes several indicators to measure school performance. The descriptors include both qualitative descriptions of school practices like teaching and learning practices (see example in Table 4.3), and administrative compliance descriptors. However, there are some notable gaps such as the absence of indicators on school principals’ pedagogical leadership and the quality of school self-evaluation. For each indicator, schools receive a rating of: very good, good, satisfactory, or not satisfactory. They also receive an overall rating. It was reported to the OECD that most schools get a good rating, but less than 1% received a very good rating (MoES, 2018[11]).
While it has an established school evaluation framework, North Macedonia does not have a national vision of a good school. An increasing number of OECD countries have developed a vision of a good school to guide evaluation. The vision helps focus evaluation on accountability for school quality and improvement, to avoid that evaluation becomes a check box exercise. North Macedonia could build on the SPQI when developing its national vision. Countries frequently use national consultations including teachers and principals to develop the national vision, creating opportunities to build national understanding of the purpose of school evaluation for educational improvement. Countries may also choose to include additional criteria specific to vocational schools to acknowledge their different mandate.
School external evaluation does not fulfil its school improvement purpose
The School Integral Evaluation is carried out in all schools every three years, to evaluate the quality of school practices and inform improvement (Government of the Republic of North Macedonia, 2015[13]). During the evaluation, the inspection team collects data and school documents like the school plan, observes the classroom practices of all teachers and interviews school staff, the school board, representatives of the parent council and students. At the end of the visit, the inspectors discuss the results with the principal and submit a written report to the school within two weeks. The evaluated school then sets an action plan detailing how it intends to implement the recommendations from the evaluation, and submits it to the SEI (see Table 4.1).
The process and tools for evaluation do not reflect its intended purpose
There is a perception throughout the SEI that the primary role of evaluation is to monitor schools’ compliance with the law and regulation. Moreover, the tools for evaluation focus primarily on checking whether the school has documented its processes (e.g. recorded minutes of council meetings or the availability of a school plan), and not as extensively on a qualitative assessment of teaching and learning practices. This is apparent when looking at the limited time allocated for classroom observations (one-fifth of the evaluation visit time) which is mostly used to check classroom documents (e.g. student portfolios and teacher plans), leaving little time for observing classroom teaching and stakeholder interviews. In contrast, the quality of instruction is a central component of external evaluations in 22 out of 35 OECD countries (OECD, 2015[2]).
School integral evaluation reports are provided to the school and made public
Schools receive written reports detailing their strong points and areas for improvement across the seven areas of the school evaluation framework. While schools are given a descriptive mark overall and for each of the seven areas, these scores are not accompanied by descriptions of performance justifying the score. The Handbook for School Integral Evaluation provides detailed guidelines for inspection teams about how to draft the report, such as making it clear and writing in an accessible manner. The school reports are made public and available on the State Education Inspectorate’s website.
Evaluation is perceived to be high stakes by school actors
School staff reported to the OECD team feeling that the significant number of documents produced and kept for the integral evaluation distracts them from their core mission and responsibilities. This perception may be due to the fact that they often prepare all the documentation in few days, before the integral evaluation, rather than regularly throughout the year. Teachers and students reported feeling stressed and under pressure to perform well during the evaluations. The introduction in 2014 of individual teacher appraisals as part of integral evaluations might have contributed to teachers’ perception of evaluation as high stakes, as every teacher is scored. However, the grade has no impact on a teachers’ career and teachers do not receive written feedback on their performance (see Chapter 3).
The State Education Inspectorate lacks professional independence
The State Education Inspectorate that leads integral evaluations is a separate body, but remains part of the ministry. The SEI is strongly influenced by the ministry, which manages its budget annually and selects its director. On occasion, the SEI’s work has been subject to political influence, such as when the inspectorate has been requested to undertake inspections to justify principal dismissals. Unlike in many OECD and other European countries, integrity and professional independence are not sufficiently emphasised as expectations for the SEI’s director’s role, even though staff are required to adhere to a code of conduct.
Another reason that the SEI lacks professional independence is its limited accountability. Like the other ministry agencies in North Macedonia, there are few mechanisms to ensure that the inspectorate is effectively fulfilling its mandate. In contrast, most OECD countries have statutory requirements to keep inspectorates accountable for the quality of their work. This includes annual reports, parliamentary hearings, and performance reviews.
Inspectors receive little training or guidance on how to undertake evaluations
As is the case in most European countries, state inspectors in North Macedonia are former teachers with a minimum of five years of experience (European Commission/EACEA/Eurydice, 2015[6]). However, contrary to practices in most European countries, they receive little preparation – just three days ‒ for the role of inspector. This is far less than new inspectors commonly receive in most European countries, which varies from several months to one year (European Commission/EACEA/Eurydice, 2015[6]). In North Macedonia, there is also no regular professional development for inspectors once they are in post. Inspectors may occasionally take part in training on new reforms organised by non-governmental organisations (NGOs) and donor institutions. Unlike the practice in a number of OECD countries, the inspectorate relies solely on a corps of permanent inspectors.
The lack of preparation and training that inspectors receive significantly limits their capacity to provide meaningful feedback to schools. It also undermines their authority in schools.
The quality of school self-evaluation varies
Almost all (99%) of students participating in the OECD Programme for International Student Assessment (PISA) in 2015 were in schools that conduct self-evaluations (OECD, 2016[3]). During the preparatory phase of self-evaluation, the school board forms an ad hoc, self-evaluation committee that leads the evaluation. School principals are responsible for establishing the committees, but are not themselves a member of it. The self-evaluation committee includes teachers, students, school support staff, parents and the local and business community (European Commission/EACEA/Eurydice, 2015[6]). The committee assesses the school’s performance in the seven areas specified by the SPQI indicators and drafts a self-evaluation report that is submitted to the school board and principal. The reports are also made available online for the wider public.
Schools receive limited guidance and tools to undertake self-evaluations, resulting in considerable heterogeneity in practices and quality. A rulebook for self-evaluation defines the areas that schools need to look at, which are the same seven domains as in the SPQI framework. However, the rulebook does not define the indicators that schools should use to evaluate quality or possible sources of evidence. As evaluation systems mature, they frequently provide schools with considerable autonomy to determine self-evaluation procedures. However, at the beginning, when effective self-evaluation is still being established, some guidance is important. The absence of such guidance in North Macedonia risks that evaluations do not consistently address key issues of school quality, and that conclusions are not based on valid evidence. The review team analysed four school self-evaluation reports and found that while all four reports included an analysis of strengths and weaknesses, in most cases very little evidence was given to explain the choices of priority actions.
Self-evaluations may also not be consistently drawing on a broad range of evidence. Classroom observations, discussions with school staff, data analysis and discussions with students are all important sources of evidence to understand overall school quality. However, in only one of the four reports that the review team read were the opinions of students and parents reflected.
Schools’ capacity for self-evaluation is limited
School principals and other staff receive no mandatory training in school self-evaluation as part of their initial training or while in-service. In contrast, in most OECD countries, principals’ initial training include modules on school self-evaluation and planning (OECD, 2013[1]). Some NGOs and donor organisations have taken the initiative to provide training to improve the school capacity for self-evaluation, but this remains occasional.
Schools use evaluation results to draft their school action plan, which is developed every four years. The State Education Inspectorate uses self-evaluation results to inform external school evaluations. Although the inspectorate reviews the quality of the self-evaluations by reviewing the report and interviewing school staff, it does not systematically provide schools with feedback on the quality of their self-evaluations.
Other forms of school evaluation and quality assurance
In addition to integral evaluations, the SEI carries out frequent ad-hoc inspections of school practice or individual practices following a formal complaint from parents, parent councils, school staff or other citizens. Municipalities can also carry out audits of schools to monitor compliance with regulation and finances.
Policy issues
While North Macedonia has the foundations for an effective school evaluation system, a number of factors are currently preventing the latter from coming to fruition. A major issue that requires immediate attention is creating an inspectorate with the integrity, independence and capacity to act with professional authority. Simple changes to the evaluation framework and process will help to create a more manageable system that schools and inspectors can apply more easily to evaluate school quality. Finally, schools will need more direct support and resources, so that they can fully use external evaluation results, and adopt self-evaluation as an internal tool for their own improvement.
Policy issue 4.1. Professionalising the State Education Inspectorate
External school evaluation in North Macedonia does not yet fulfil its stated core functions of ensuring school accountability and helping them improve. The process is focused heavily on ensuring schools’ compliance with regulations and administrative processes, and the SEI lacks the technical capacity and independence to lead a meaningful school evaluation system.
To make external evaluation more effective, North Macedonia needs to invest in professionalising the SEI. The inspectorate needs to be reformed so that it undertakes its role with independence and integrity and is accountable for the quality of its work. This needs to be accompanied by building professional capacity for school evaluation within the SEI.
Recommendation 4.1.1. Guarantee the independence and integrity of the inspectorate
The stated purpose of evaluation in North Macedonia is to improve school quality. However, the overwhelming perception of evaluation – external and internal - as reported to the review team by the inspectorate, principals and teachers, was as a process to ensure compliance with regulations and the evaluation framework. Creating more meaningful school evaluation requires a shared national understanding of the important role that it is expected to play for school improvement. A number of steps are important to create this understanding.
First, the head of the SEI needs to be appointed based on demonstrated competence in school improvement. Second, increased professional independence of the SEI needs to be balanced by greater oversight of, and accountability for, the inspectorate’s work. Accompanying these measures with a national consultation to create a vision of a good school will encourage national understanding and ownership of evaluation as a means to support school improvement. Developing a national vision will also help to keep evaluation focused on its core purpose of school improvement.
Ensure the integrity and professional competence of the SEI’s director
Leadership of the SEI is key for shaping how staff within the SEI and schools understand the role of school evaluation. The SEI’s director must combine a deep understanding of school improvement, strong leadership skills and integrity. As well as being responsible for the quality of the country’s schools, the director should hold a senior leadership position within a country’s education system, regularly advising the ministry and minister directly, on issues of school quality.
At present however in North Macedonia, the director of the SEI is not expected to occupy this leadership role. The minimum eligibility requirements for the director are similar to those of other inspectors (e.g. five years of teaching experience, having at least a bachelor’s degree). In contrast, Education Scotland, a country where school evaluation has a major role in school improvement, defines the role of the Director of Inspection as “a member of the senior management team [of Education Scotland] with appropriate experience, stature and credibility in relation to Scottish education, quality evaluation and improvement” (Education Scotland, n.d.[14]).
The ministry should revise the selection criteria and recruitment process for the director of the SEI, to focus on demonstrating significant professional expertise in school improvement. The selection process should also require candidates to demonstrate their strong understanding of the role of evaluation, and how it impacts school quality.
Ensure that inspectors undertake their role with utmost integrity
One step to ensure that inspectors carry out their role with integrity and independence is to encourage the widespread use of the existing code of practice that sets out how inspectors are expected to perform their duties. North Macedonia’s codes of practice ‒ including one currently being developed ‒ should provide inspectors with a practical handbook that sets out the ethical values and principles they are expected to follow. It should also explicitly indicate practices which are considered unethical.
Another important step is to ensure, given the inspectorate’s influence, that all stakeholders – principals, teachers, students and parents – have clear and fair opportunities to redress any grievances. For example, all actors should clearly understand how to make a complaint about an evaluation. As well as having a clearly stipulated internal complaints and review process, most countries also have parliamentary ombudsmen, to deal fairly with complaints about public organisations.
Create a board to oversee the SEI’s work
Given the powerful influence that inspectorates have on schools’ work, a number of countries have boards, composed of respected educationalists that help to maintain the inspectorate’s independence and integrity. In North Macedonia, such a board could also play an important oversight role ‒ monitoring the work of the inspectorate to ensure that it is focused on school quality and improvement and does not veer into administrative compliance checks again.
The independent advisory board should be composed of education professionals with significant experience in school improvement. Given North Macedonia’s ethnic composition, the board might also include representatives from the country’s main communities, Macedonian, Albanian and possibly others. North Macedonia might also consider inviting one or more international representatives to provide an external perspective and guidance. One option is a representative from a country with an established tradition of school evaluation – such as the Netherlands or Scotland (United Kingdom). Alternatively a country that has relatively recently established school evaluation – such as Romania – could be invited to provide practical advice in addressing common challenges.
Make the SEI accountable for the quality of its work
The SEI is legally required to produce annual, public reports on the inspectorate’s work and the quality of the education system based on the integral evaluation results. To ensure that these reports produce valuable information for the system, North Macedonia can look to the many examples of inspectorate reports from other countries. In England for example, the state inspectorate Ofsted, produces an annual report on its performance, governance, and finances. In the report, Ofsted reports its performance against key objectives, including how far schools perceive the inspectorate to be a force for improvement, and ensuring that inspections focus on learning standards for all groups of students (Ofsted, 2018[15]).
In North Macedonia, as in other countries, the ministry should also consider requiring that the inspectorate’s annual report be debated in parliament. In England, the senior leadership of Ofsted is required to attend a hearing of the investigative parliamentary committee on education. This kind of public reporting and debate in North Macedonia will create impetus within the inspectorate to better understand its role, and the accountability to encourage each inspector to focus on achieving it. Over time, public reporting and debate will educate the wider education system and the public on the role of the inspectorate.
Develop a national vision of a “good school”
While the overall framework for school evaluation in North Macedonia focuses on many of the important aspects of school quality, it is not guided by an overall vision of schooling. This vision is important to avoid that evaluation becomes focused on mechanically complying with individual descriptors in the evaluation framework. Instead, it helps to focus teachers, schools and evaluators on the fundamental purpose of evaluation – to create schools where all students can learn and thrive.
In general, visions are often short and simple (see Box 4.1). Avoiding long, complicated descriptive text helps to ensure that schools and evaluators do not become distracted interpreting what the vision means. It also makes the vision easier for schools to appropriate as a goal that guides their own planning and self-evaluation, and for external evaluators to reflect upon during school evaluations.
Education systems develop a definition of a “good school” at the national level in order to provide standard quality criteria for the evaluation of educational processes and outcomes. This common definition of effectiveness often includes several characteristics, including the quality of teaching and learning, how teachers are developed and made more effective, the quality of instructional leadership, the use of assessment for learning, the rate and equity of student outcomes and progress, setting the school’s vision and expectations, self-evaluation practices and factors concerning the curriculum.
A shared, future-focused and compelling vision at the national level can provide direction and steering to an educational system, bringing key actors together to work towards achieving the vision. It should be shared across all levels of the education system, while allowing space for interpretation based on local or regional differences. A clearly communicated and shared vision can also help ensure reforms continue in the long-term, particularly when faced with challenges or obstacles.
-
Ontario’s (Canada) vision for education explicitly incorporates goals:
Ontario’s vision for education is focused on four core goals: achieving excellence, ensuring equity, promoting well-being and enhancing public confidence.
-
In 2008, the government of Japan developed the Basic Plan for the Promotion of Education, in which it set out a ten-year education vision:
1) To cultivate, in all children, the foundations for independence within society by the time they complete compulsory education.
2) To develop human resources capable of supporting and developing our society and leading the international society.
-
In Estonia, the Lifelong Learning Strategy 2020 guides the formal education system, as well as in-service, non-formal and informal education and retraining. The vision for 2020 is:
Learning is a lifestyle. Development opportunities are noticed and smart solutions are pursued.
In order to develop their national vision, many countries undertake a consultation process. Such a strategy helps to gather input, engage stakeholders and build consensus. Moreover, when education stakeholders, including teachers, support the vision it is more likely they will dedicate time and energy to their roles. Indeed, effective policy implementation requires a shared vision, and the acceptance, ownership and legitimacy of a policy’s plan and purpose and the process of change must be developed among actors in order to move toward the vision (Burns, Köster and Fuster, 2016[16]).
Sources: (OECD, 2013[1]), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, OECD Publishing Paris, https://dx.doi.org/10.1787/9789264190658-en; (Kitchen et al., 2017[17]), Romania 2017, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264274051-en; (OECD, 2018[18]), Developing Schools as Learning Organisations in Wales, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264307193-en; (Ontario Ministry of Education, n.d.[19]), Education in Ontario, https://www.ontario.ca/page/education-ontario; (MEXT, 2008[20]), Basic Plan for the Promotion of Education (Provisional translation), http://www.mext.go.jp/en/policy/education/lawandplan/title01/detail01/1373797.htm; (Ministry of Education and Research, n.d.[21]), The Estonian Lifelong Learning Strategy 2020, https://www.hm.ee/sites/default/files/estonian_lifelong_strategy.pdf; (OECD, 2018[22]), Education for a Bright Future in Greece, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264298750-en; (Burns, Köster and Fuster, 2016[16]), Education Governance in Action: Lessons from Case Studies, Educational Research and Innovation, https://dx.doi.org/10.1787/9789264262829-en.
The process of developing the national vision can be almost as important as the final vision itself. Many countries use its development as an opportunity to undertake a national consultation that involves students, teachers and schools, asking them what they consider the most important characteristics of a good school. This process would be particularly valuable in North Macedonia, since the school evaluation framework and national strategy were developed in the absence of wide consultation. An inclusive process, similar to that undertaken for the development of the Comprehensive Education Strategy 2018-25, would also help promote awareness of the role of evaluation in school quality, and create a sense of ownership of the standards among school actors.
Recommendation 4.1.2. Build the professional capacity of the State Education Inspectorate
Another issue that currently undermines the SEI professional authority is its lack of technical capacity. Inspectors’ initial training is too short and inadequate to enable them to carry out meaningful school evaluations. To build capacity for evaluations, the inspectorate should consider revising the content and length of inspectors’ training, and include other actors from outside the inspectorate in the evaluation teams.
Reinforce the training of school inspectors
The SEI first priority should be to ensure that its current staff have the necessary knowledge and skills to carry out the integral evaluations. Inspectors’ training needs to focus on the major knowledge gaps that are currently preventing inspectors from undertaking evaluations that support school improvement, such as understanding how evaluation supports school accountability and improvement. Training should provide inspectors with practical examples of how school evaluation supports these functions, illustrated by specific evaluation practices that best support accountability and improvement. Training should also help inspectors to identify evaluation practices that only focus on ensuring administrative compliance. Other important areas to address include how to evaluate teaching and learning and how to provide formative feedback.
The ministry should also ensure that the SEI has enough resources to cover regular training of its staff on key priority areas such as curriculum reform, student assessment, and inclusive education and any new reform affecting teaching and learning in schools.
An overriding concern in developing all training should be to ensure that it provides inspectors with practical learning opportunities. For example, inspectors should have the opportunity to try out new techniques and receive feedback, and to participate in an evaluation visit. Practical training on how to conduct meaningful classroom observations will also be essential. Such a model would be similar to the one currently in place in Lithuania, which has also recently introduced school evaluation. In Lithuania, external evaluators complete 80 hours of theoretical training and 45 hours of practical training (European Commission/EACEA/Eurydice, 2015[6]). This training should be provided to full-time inspectors and to the pool of licensed part-time evaluators.
To accommodate the above, the initial training period of inspectors needs to be increased. The current initial training lasts three days which is short compared to initial training of inspectors in other European countries, which in most cases lasts at least several months (European Commission/EACEA/Eurydice, 2015[6]). The limited time is primarily used to review the laws, regulations and the evaluation process. The inspectorate should consider increasing the initial training period.
Create a roster of licensed inspectors to undertake school integral evaluations
The SEI should consider training and licensing experts as external consultants that can join the evaluation teams on an ad hoc basis to contribute different experience and perspectives. This would also be an important learning opportunity – inspectors can learn from other educationalists with different skills and experience, while school leaders that participate in the evaluations of other schools can learn from their practices.
Several inspectorates in OECD countries use similar practices. School inspection teams in Scotland include full-time inspectors as well as high- performing school practitioners such as school principals and deputy school principals from other schools and other non-education profiles that are contracted as external experts for school evaluations (e.g. doctors, psychologists etc.) (European Commission/EACEA/Eurydice, 2015[6]). The SEI should consider recruiting external inspectors with the following profiles:
-
Experienced teachers from other schools: when the new teacher career development structure is implemented, the SEI can hire ad hoc inspectors from the pool of expert teachers. Expert teachers will have experience observing classroom practices and providing feedback to their peers (see Chapter 3). As the practice becomes more established, prior participation in an integral evaluation could be a pre-requisite for teachers who want to apply to become permanent inspectors.
-
Advisors from the Bureau for the Development of Education (BDE) and the Vocational Education and Training Centre (VETC): the Law on the State Education Inspectorate states that BDE advisors and VETC experts can be invited to join evaluations to lead the classroom observation component. Both agencies have some experience in classroom observation and provide closer pedagogical support to schools than the inspectorate. However in practice, the BDE and the VETC are rarely able to join the inspection teams. Including BDE and the VETC experts more regularly will help improve the quality of the classroom observations and ensure alignment in classroom observation practices between the agencies. Advisors from the BDE and the VETC might be given an explicit role to lead or co-ordinate classroom observations during evaluations.
The inspectorate might also invite other types of experts to join the inspection team based on the focus of the evaluation (e.g. school health, infrastructure, resources, etc.).
Policy issue 4.2. Ensuring that integral school evaluations focuses centrally on improving school quality
There are many aspects of North Macedonia’s school evaluation process that are positive, and reflect the practices used in many OECD countries. Professionalising the SEI and investing in its capacity will help to ensure that it is better equipped to undertake evaluations so that they reflect their intended purpose. However, this review suggests that this will be complemented by a few revisions to the school evaluation framework, so that it becomes a more manageable tool for inspectors to implement. Refocusing follow-up support, so that those schools in greatest need receive proportionally more support, will help to ensure a fairer, more efficient model of follow-up.
Recommendation 4.2.1. Revise school integral evaluation to focus more centrally on the quality of teaching and learning
While the framework for school evaluation in North Macedonia focuses on many important aspects of school quality, it can be difficult for inspectors to implement. With nearly 30 indicators, it can seem overwhelming for schools and inspectors. Revising the framework to prioritise core indicators will help better orient inspectors in their work. This will be complemented by one important change, which will be to give more space to meaningful evaluation of teaching and learning across the school.
Revise the School Performance Quality Indicators (SPQI) framework to focus on core teaching and learning areas
The School Performance Quality Indicators (SPQI) provides a relatively complete framework, including indicators of school quality and descriptors of the practices and behaviours expected from schools. It is also very positive that the majority of school principals interviewed by the review team were aware of the seven areas of evaluation, which shows that the SPQI is a well-established reference framework.
However, the framework is very dense compared to indicator frameworks used by OECD countries. The SPQI framework includes seven areas, 28 indicators and 99 parameters detailing further the indicators. In contrast, the indicator framework of Education Scotland, which was used as a model for the SPQI, only includes three areas – Leadership and Management, Learning Provision and Successes and Achievement – and 15 quality indicators (Education Scotland, 2015[23]). A long list of indicators can encourage evaluation to become a checkbox exercise. Internationally, as many countries have implemented their school evaluation frameworks, they have found that it has been important to simplify their frameworks to focus on key aspects of school quality. This is important to move evaluation from a checkbox exercise, to a more focused, in-depth review of the quality of school practices and how they can be improved.
Given that school evaluation in North Macedonia frequently emphasises compliance with descriptors, rather than evaluation of quality, reviewing the evaluation framework to simplify it would be helpful. The number of indicators in the framework should be reduced to around 10 to 15 indicators to make it more manageable for inspection teams and give them more time to focus on key indicators of teaching and learning quality.
The SPQI can be revised to distinguish between a set of core indicators evaluated in each integral evaluation and a set of secondary indicators evaluated on a rotating basis or when a problem arises. The indicators that have a direct and proven impact on improving learning and teaching in schools such as “teaching process” and “students’ learning experience” should be prioritised as core indicators. Indicators related to the quality of the school environment such as “health”, “school climate” and “accommodation and premises” could be evaluated on a rotating basis. The rotation of indicators will also allow the inspectorate to go deeper in investigating the non-core areas and produce thematic reports to inform national policies.
As part of this review, some gaps in the existing framework should also be addressed. The framework should include school pedagogical leadership as an indicator under teaching and learning. The SPQI framework should also include the quality of self-evaluation and schools’ capacity to reflect on its processes. This might be addressed in an indicator on schools’ capacity for improvement.
Streamline and reduce administrative reporting
Schools in North Macedonia spend considerable time preparing and submitting many administrative documents and data to the SEI. As well as distracting schools from their core role, some of the documents reported to the inspectorate are already reported to other parts of the ministry. For instance, schools provide inspection teams with data on retention rates and students’ attendance, information about the school principal and teacher. While this information is already available in the ministry’s Education Management and Information System (EMIS), but it is not shared with the SEI.
The SEI should simplify and digitalise the collection of administrative data as the majority of OECD countries have. (OECD, 2013[1]). The SEI should try to retrieve as much administrative data as it can (such as grade retention rates, school staff profiles and student attendance rates) directly from the EMIS database. The SEI should also work with the EMIS unit to ensure that administrative data needed for the evaluation are adequately reported and included in EMIS in the future (see Chapter 5). It is also recommended that the SEI stop requesting and collecting some documents produced specifically for the inspection and which do not provide valuable information about the quality of the school practices, such as minutes of school board meetings and teacher council meetings.
Revise classroom observations to focus on teaching and learning across the school
The individual teacher appraisals as part of school evaluations in North Macedonia play a limited role in supporting school improvement. Teachers do not receive written feedback or their scores from the classroom observation. Given the limited time available for observing all teachers, individual classroom observations are often very short, just ten minutes, during which inspectors will simply check documents such as students’ portfolios and lesson plans.
While the teacher appraisal component has no consequences for teachers’ career or salaries, teachers feel under pressure to perform well during the classroom observation. It even leads to some distortive teaching behaviour, such as only calling on the best students in the class to respond to questions, or providing students with the answers to the questions in advance.
The SEI should replace the individual teacher appraisals with more extended classroom observations of a sample of classrooms to gain a deeper understanding of instruction across the school. Inspectors should plan to visit a range of classrooms across different subjects and grades. The focus of these classroom observations would be to develop a general overview of teaching and learning across the school. Instead, individual teacher appraisals will be led by the school principal and the BDE (see Chapter 3).
Develop guidance on how to observe teacher practice and student-teacher interactions
The classroom observation protocol included in the evaluation handbook specifies expectations for inspectors’ conduct and the documents they need to look at. However, it does not help inspectors understand how to meaningfully observe teacher practice. The inspectorate should consider introducing a set of qualitative measures for classroom observations to help inspectors evaluate teaching practice in a structured way, since by its very nature, it is subjective and difficult to evaluate.
In this exercise, the inspectorate might draw on the classroom observation indicators developed by the International Comparative Analysis of Learning and Teaching (ICALT). The ICALT indicators are based on teaching and learning practices with a proven impact on student learning (see Box 4.2).
Guidelines should explain clearly the purpose of the classroom observation and list the indicators and descriptors that will be used. The International Comparative Analysis of Learning and Teaching (ICALT) was a collaboration among European external school evaluation bodies to develop an instrument to observe and analyse the quality of teaching and learning in primary schools.
The study found that the following five aspects could be compared in a reliable and valid way and that these were positively correlated with student involvement, attitude, behaviour and attainment: efficient classroom management, safe and stimulating learning climate, clear instruction, adaptation of teaching, and teaching-learning strategies. The final observation instrument was adopted for use by external school evaluation bodies in five European countries: Belgium (Flemish Community), Lower Saxony (Germany), the Netherlands, the Slovak Republic, and Scotland (United Kingdom). Below are a subset of the observation indicators:
Recommendation 4.2.2. Make sure that integral evaluations deliver constructive feedback to schools
For school evaluations to lead to school improvement, they need to help schools understand what they do well, and where improvements can be made. At present however, schools in North Macedonia do not feel that evaluations help them to do this. The review team’s interviews revealed that schools largely perceive evaluation to be an externally imposed process that is disassociated from their own planning and development efforts.
This concern will be addressed by enhancing the SEI’s professional authority and capacity, and re-orienting the process to focus more centrally on school quality (Recommendations 4.1.1., 4.1.2. and 4.2.1.). However, another important dimension is ensuring that evaluations result in more useful, actionable feedback, complemented by greater follow-up support where necessary.
Improve the quality of feedback to schools
Precise and actionable feedback to schools following evaluations is important to help them understand their strengths, where improvement is needed and how to take action to achieve this (Faubert, 2009[24]). This is even more important in education contexts like North Macedonia where school leaders receive limited support – in terms of training or other support roles across the school – to become instructional leaders.
According to the SEI’s guidelines, the school report should include feedback to schools on the seven areas evaluated, highlighting strengths and weakness, and providing examples of good practices to help schools improve. These guidelines are in line with the approach to reporting results of external evaluations in OECD countries (OECD, 2013[1]). However, teachers reported to the review that they found that the report’s recommendations on teaching processes of little value.
The SEI might review the feedback that is provided to schools, to ensure that it is easily understood and useful. This might include reviewing a sample of reports nationally, conducting interviews with schools and looking at international practices. The review would result in recommendations for improvements in written and oral feedback. In the future, schools should continue to be surveyed periodically to make sure that reports are useful, with revisions made to the school report format as necessary.
Create a more meaningful follow-up process, focused on schools in greatest need of improvement
The SEI should consider reviewing the evaluation follow-up process so that schools with a “non-satisfactory” rating receive sufficient support. Currently, follow-up mainly plays a compliance function – to make sure that schools have implemented the evaluation’s recommendations. However, no external support is provided to help schools implement the recommendations. Moreover, the timing of the follow-up visit (six months after an evaluation) does not leave enough time to schools to put in place significant changes that may be necessary. Instead, the SEI should consider:
-
Gradually introducing a risk-based approach to evaluation and follow-up: meaningful follow-up is demanding in terms of technical expertise and time-consuming. Given the limited resources available at the BDE and the municipal level, North Macedonia might consider introducing a risk-based approach to follow-up. Such an approach prioritises follow up with schools at greater risk of not meeting evaluation recommendations and low performance more generally. In North Macedonia, a result of “not satisfactory” in the integral evaluation might trigger follow-up for a school, while those that receive a “very good” grading do not. In the latter case, the SEI will simply check implementation of the recommendations during the next evaluation.
-
Replacing the follow-up visit with more continuous support: the SEI is too remote from schools to provide meaningful follow-up. In most OECD countries, follow-up to external evaluation is not carried out by the external evaluator but by agencies closer to the school (see examples in Box 4.3). While municipalities in some countries perform this role, the very limited capacity of municipalities in North Macedonia (1-2 education officers per municipality) and their large number (81) make it unfeasible to provide support within each municipality. Instead, North Macedonia might consider establishing support improvement officers that work across multiple municipalities; developing a separate unit dedicated to school improvement within the inspectorate.
Wales (United Kingdom)
External evaluation in schools is conducted by Estyn, the Office of Her Majesty's Inspectorate for Education and Training in Wales. When a school is found not to be performing at the level defined in Estyn’s standards, the school may be classified into one of four categories. In three of the categories, Estyn itself monitors and revisits the school. In the fourth category, the classification for schools of least concern, schools are monitored by the local authority.
The local authority meets with Estyn every term and produces a report on the school’s progress in improving and implementing Estyn’s inspection recommendations. Estyn uses the report to decide on the extent to which Estyn itself must monitor the school. When Estyn has serious concerns about a school, local authorities are expected to intervene, as they are responsible for standards. For example, local authorities might address issues of staff performance and professional competence, school governance, resources and training.
Scotland (United Kingdom)
Education Scotland is responsible for conducting inspections of schools in Scotland. A school that is inspected will receive one of four ratings - innovative practice, no continuing engagement, additional support and further inspection ‒ each of which includes follow-up by local authorities :
-
In the case where Education Scotland identifies “innovative practice” in the school, Education Scotland works with the local authority to create a record of and disseminate the practice.
-
When a school receives the “no continuing engagement” designation, Education Scotland conducts no further follow-up visits in relation to this inspection and the local authority reports to parents on the progress of the school.
-
An “additional support” designation engages Education Scotland in providing support alongside the local authorities to improve the school.
-
If a school is in need of “further inspection,” Education Scotland, via an Area Lead Officer who oversees all scrutiny and capacity building activity in a particular local authority, will work with the local authority to identify the supports needed to improve the school. In this case, the Area Lead Officer monitors progress via the local authority and Education Scotland returns at a later date to evaluate progress and improvement.
Source: (European Commission/EACEA/Eurydice, 2015[6]), Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe, The European Union, Luxembourg, https://publications.europa.eu/en/publication-detail/-/publication/4a2443a7-7bac-11e5-9fae-01aa75ed71a1/language-en.
Communicate and educate the school community on the purpose of school evaluation
Given the mistrust associated with school evaluation, and the distortive influence it has had on practices in some schools, the SEI should invest in communication materials to explain clearly its purpose teachers, other school staff, students and parents. For instance, the external school evaluation agency in England, United Kingdom, Ofsted, has developed a guideline called “Ofsted inspections: myths” that debunk some of the most common misconceptions about the inspection process (Ofsted, 2018[15]).
Similar brochures should be developed in North Macedonia and given to schools. They should address clearly a number of common concerns by clearly explaining for example that: school evaluation carries no consequences for individual teachers; no information on individual teachers, classes or students is made publicly available in the report; and that the school evaluation results cannot directly lead to a school principals’ dismissal.
Policy issue 4.3. Developing schools’ capacity to carry out meaningful self-evaluation
While most schools in North Macedonia undertake regular self-evaluations and develop school action plans, few have appropriated these processes as internal tools to improve the quality of their practices. An overriding reason is that the improvement function of school evaluation is not well embedded in the country. Steps to professionalise the inspectorate and create a national vision of a good school will engage schools in a conversation about what school evaluation means, and its role for school improvement (see Recommendation 4.1.1.).
The above will need to be accompanied by more practical steps to provide schools with greater support for self-evaluation. At present, school actors with a leading role in self-evaluation in schools do not receive any training or guidance to implement an effective self-evaluation process that is embedded in school planning activities. Steps also need to be taken to develop principals’ capacity to become instructional leaders in schools. Principals play an essential role in engaging the whole of a school community in self-evaluating and galvanising the school behind the self-evaluation process.
Recommendation 4.3.1. Provide support and training for school actors on self-evaluation
There needs to be greater national investment in improving school-level capacity for self-evaluation, and helping to develop a culture of improvement in schools. This will include more practical guidance for schools – through self-evaluation guidance and by promoting the exchange of good practices across schools. Actors with a key role in self-evaluation ‒ school principals, boards and teachers – need far more support and training so that they can undertake their roles effectively.
Revise school self-evaluation guidance
Schools in North Macedonia need clear guidance and direction to understand the process of school self-evaluation and embed it in their planning process. To do so, the SEI should consider revising the self-evaluation supporting documents so that self-evaluation becomes a more meaningful tool for schools. Options include:
-
Review schools’ experience of self-evaluation over the past ten years. The SEI should lead a review of this experience to understand how schools are using self-evaluation and what practical changes to the self-evaluation manual and process would help make it more useful for them. This can be done by focusing one three year round of integral evaluations on observing self-evaluation in schools.
-
Include guiding questions and a simplified list of indicators. Schools would benefit from a short list of simple guiding questions to drive their self-reflection and objective setting.
For instance, research and experience suggests that self-evaluations should aim to answer simple questions focused on improving teaching and learning, such as: “how good is our school”; “how can we make it better”; “are teachers’ skills being put to good use”; and “how good is learning and teaching in our school?” (Riley and Macbeath, 2000[25]). This list of questions should be complemented by a short list of examples of good practices to help schools understand how to make an educated judgement about their practices. The ministry can prioritise some indicators from the SPQI framework to include in the Manual.
In 2008, as part of the Education Modernisation Project, a school self-evaluation manual was developed to provide schools with guidance on how to implement the then new self-evaluation process. However, the schools visited by the review team were unaware of this resource and rely solely on the rulebook for school self-evaluation. The manual has not been updated over the past decade. Steps to encourage greater use of the manual include:
-
Develop a comprehensive self-evaluation manual and include examples of good self-evaluation practices: the self-evaluation manual should be updated to reflect new policies and practices. This development process could also draw on the school improvement manual, the multicultural education self-evaluation manual developed by the United States Agency for International Development (USAID) and UNICEF’s self-evaluation manual focused on efficiency and inclusiveness, developed as part of the Child Friendly School programme. The ministry should consider including examples of good school self-evaluation practices observed by the inspectorate in the self-evaluation manual.
-
Make sure that schools are aware of the manual and encourage its use: all actors responsible for school quality need to inform schools about the available manual. The SEI could check for example if schools are aware of these manual during the evaluations. Similarly, the BDE and the VETC advisors can inform schools about their availability.
Provide more training for school actors on self-evaluation
Developing school capacity for self-evaluation takes time and requires considerable external support, especially in the first years of implementation (OECD, 2016[26]). In North Macedonia, teachers, members of the professional support and school principals that are tasked with undertaking self-evaluation do not receive sufficient training in this area. In particular, school principals, who play a leadership role in self-evaluation in many OECD countries, do not receive any dedicated preparation for this role. Their initial training is short compared to other European and OECD countries. Principals’ initial training does not provide time to train principles in practices for school evaluation and classroom observation. The ministry might commission the BDE to provide more support to develop school-level capacity for self-evaluation by:
-
Creating a mandatory, practical module on self-evaluation in school principals’ initial education: as a core instructional leadership responsibility, school self-evaluation is included in school principals’ training in most OECD and European countries (OECD, 2013[1]; European Commission/EACEA/Eurydice, 2015[6]). As part of a longer initial training, including content on how to lead self-evaluations (see Chapter 3), principals could be given a case study about a school and asked to interpret and analyse evidence about the schools’ practices and provide recommendations for improvement.
-
Offering regular training to school staff involved in school self-evaluation: to make training in key areas of self-evaluation available (e.g. use and interpretation of data and classroom observation). Take-up can be incentivised by subsidising schools’ participation. Most European countries offer training for school principals already in service. In some countries, such training is mandatory.
-
Offering training to the school boards: regular training is made available to the school board in other countries where boards monitor the quality of school self-evaluations (European Commission/EACEA/Eurydice, 2015[6]). While school boards in North Macedonia are tasked with approving the results and findings of school self-evaluations, its members do not receive any training on how to interpret results and judge the quality of the process. In practice, this means that their role is limited to administrative validation. In 2012, USAID organised training for boards on how to interpret the results of school self-evaluations, but given the high turnover rates in board members (who are replaced every four years), such training needs to be more systematic and regular to have an impact.
-
Providing technical assistance to schools that struggle with undertaking a meaningful self-evaluation: the ministry might also consider training the BDE advisors to assist schools that are facing difficulties in evaluating the quality of their practices and identifying areas for improvement. These specialists can join school self-evaluation committees to help guide their work. Similar models are used in Poland and Slovak Republic to help schools improve their self-evaluation practices (European Commission/EACEA/Eurydice, 2015[6]). The ministry can also bring experts from other European countries to train this new corps on how to provide support to schools.
Review and provide feedback on self-evaluation during integral evaluations
School external evaluation is used in a growing number of OECD countries to monitor schools’ capacity for improvement and provide schools with feedback on how to improve their self-evaluation practices (OECD, 2013[1]). In North Macedonia, as part of integral evaluations, the State Education Inspectorate checks schools’ self-evaluation processes, but does not review the quality.
To make more effective use of external evaluation for building schools’ self-evaluation capacity, the inspectorate should consider introducing an indicator in the SPQI framework that looks at how far school self-evaluation practices are informing improvement (e.g. quality of reflection on practices, pertinence of recommendations and follow-up actions). This indicator should also include descriptors with schools’ expected behaviours and practices, including the diversity and quality of sources of information used in self-evaluations process, and engagement of the whole school community. In New Zealand for example, the Education Review Office describes a school with “very good” self-evaluation capacity as a school that convincingly demonstrates “a school-wide culture of rigorous critical reflection and self-review is contributing to sustaining the school’s positive performance and continuous improvement” (Nusche et al., 2012[27]).
In the medium to long term, as the quality of self-evaluation improves across schools, the inspectorate can start relying on self-evaluation reports as a key source of information on school performance and practices. SEI inspectors report that self-evaluation reports seldom reflect schools’ actual performance.
Create an online portal for schools to share their experience with self-evaluation
In addition to external support from experts, schools need more opportunities to learn from each other. Disseminating best practices, for example on engaging the whole of the school, undertaking classroom observations or analysing data provides schools with inspiration about how they can improve their practices. Showcasing good practices also provides important recognition to encourage schools. To support this, the ministry might consider providing specific recognition to schools that have exceptional self-evaluation practice, like a “good improver” title.
The ministry can develop an online platform where schools can exchange self-evaluation practices (e.g. how they conduct interviews and classroom observations), surveys and tools used to collect information. The platform might also provide the space for schools to ask questions to the platform community so that they can receive advice from other schools.
Recommendation 4.3.2. Develop school principals’ instructional leadership skills
As well as developing principal’s skills for self-evaluation, it is important to develop their overall instructional leadership capacity. This will help to ensure that school evaluation feeds into other aspects of school leadership such as setting a vision for the school and planning for improvement (Ingersoll, Sirinides and Dougherty, 2017[28]).
In North Macedonia, principals currently receive limited training and support for leadership tasks. Moreover, political interference and the high level of turnover among principals make it difficult to build a professional school principal corps that is recognised for their expertise. Instead, school leadership is perceived as a temporary occupation carried out by teachers for a short period.
Create a leadership academy for principals’ initial training and continuous professional development
Increasingly, OECD countries are establishing master’s programmes on school leadership or leadership academies to improve principals’ capacity (OECD, 2014[8]). In North Macedonia, school principals’ initial training is short, and once in-service there are limited programmes for regular principals’ professional development. In the majority of school principals in OECD countries (80%) participate annually in some form of professional development programme (i.e. courses, mentorship or professional network) (OECD, 2014[8]).
North Macedonia needs to make sure that school principals are adequately trained before entering schools, and that they are encouraged to continue pursing professional development throughout their career, for example by:
-
Creating a leadership academy for initial training and continuous professional development: the responsibility for training school principals should be moved to a dedicated body. The National Education Centre (NEC) does not have the experience or sufficient staff numbers to provide training to principals and needs to focus on its core mission of assessment and examinations. Moreover, the body responsible for teachers’ professional development, the Bureau for Development of Education (BDE), does not have the technical capacity or the financial resources to provide training to school principals.
Leadership academies in OECD countries develop tools and guidelines on school leadership and provide training to principals. They also contribute to improving research on school leadership and provide more visibility for the profession (see Box 4.4).
-
Introducing competency standards for school principals: North Macedonia is currently developing competency standards for school principals. This is an important effort and it will be key that such standards provide a clear list of the knowledge, skills and behaviour expected from school principals. Standards will help provide guide principals in their job. They should also inform the preparation, licensing and selection process by the school board. The SEI should also refer to the standards when evaluating school leadership during the integral evaluations. The content of standards varies across OECD countries but usually covers the core instructional leadership and managerial competencies (Pont, Nusche and Moorman, 2009[29]).
-
Introducing a mentorship programme: while it is not the majority, there are some experienced school principals who could be appointed to coach newly appointed principals during their first year on the job. They can provide support and guidance to new principals on how to meaningfully undertake their instructional leadership duties and provide regular feedback. Similar to teacher mentors, school principals’ mentorship role should be recognised and compensated and, selected mentors adequately trained on how to provide guidance and feedback. In Estonia, coaches are selected among school principals with at least five years of experience. They also need to demonstrate high level of motivation and pass a mandatory training course on communication, need analysis, coaching and feedback skills (Education and Training 2020 Working Group, 2018[30]).
-
Ensure the certification exam assesses principals’ management and leadership skills: these skills can be tested through case studies in which school principals need to demonstrate their capacity to use information about the school to develop an improvement plan.
-
Ensuring participation in quality and relevant professional development training for principals who wish to renew their mandate: the ministry should ensure that mandatory professional development is of high quality and relevant to those who wish to renew their mandate or apply to another school.
Austria
Founded in 2004, the Leadership Academy provides training to improve the qualifications of executive-level personnel in schools, targeting leaders, directors and managers of and within education institutions in Austria. The training focuses on several elements of leadership, including leadership for learning, dialogue, shared leadership, innovation and the capacity to improve the quality of education.
Every year a new cohort or “generation” participates in a series of four three-day fora and works toward meeting certain certification criteria in order to be admitted into the Leadership Academy network. These criteria include participating in the four fora and other regional network meetings; presenting development issues during collegial team coaching workshops; coaching colleagues; and leading a participative development project in the home education institution and documenting and presenting progress and results. Certification occurs during the fourth forum and membership in the Leadership Academy is decided upon by the Leadership Academy directors.
The first “generation” of the Leader Academy served 300 participants, and roughly 3 000 school leaders have participated as of 2017. The programme has been noted for its high degree of engagement among participants and its positive impact on leadership practice, particularly in the areas of providing direction, demonstrating strength of character and community-building and creating a culture of achievement.
Ireland
In September 2015, the Centre for School Leadership (CSL) was established under a partnership between the Department of Education and Skills (DES), the Irish Primary Principals’ Network and the National Association of Principals and Deputy Principals. A variety of functions and objectives were set forth for the CSL, including supporting, leading, co-ordinating and delivering leadership professional development programmes for primary and post primary schools, which includes a programme for newly appointed principals, coaching for active principals and other activities along a continuum of continuous professional development. Additionally, CSL was to develop a strategic framework for a continuum of leadership professional development and a quality assurance framework for its provision, as well as to advise DES on leadership professional development policy.
An evaluation of the CSL’s first two years of operation identified several of the organisation’s achievements. For example, the CSL has been able to open a discussion about the concept of school leadership, including its strategic role and importance. The evaluation also found that the comprehensive draft continuum of leadership professional development has been well-received by practitioners and providers, and while informed by similar frameworks from other countries, it has been designed for the context in Ireland. Furthermore, a mentoring programme developed for new principals was found to be rated highly by mentees and mentors alike, and a coaching programme for experienced teachers was also very highly regarded. While the evaluation found the pilot to be a success, it notes that the CSL, the leadership professional development system and associated policies will need to be further developed in order for the CSL and its programmes to become fully operational and deliver on its objectives in the long term.
Sources: (Pont, Nusche and Moorman, 2009[29]), Improving School Leadership, Volume 2: Case Studies on System Leadership, OECD Publishing, Paris, https://doi.org/10.1787/9789264039551-en; (BMBWF, 2018[31]), Leadership Academy, https://www.leadershipacademy.at/ (accessed on 14 April 2018); (Schratz and Hartmann, 2009[32]) Innovation an Schulen durch Professionalisierung von Führungskräften Studie zur Evaluation der Auswirkungen der Leadership Academy an Schulen, https://www.leadershipacademy.at/downloads/LEA_Kurzfassung_Studie.pdf; (Fitzpatrick Associates, 2018[33]), School Leadership in Ireland and the Centre for School Leadership: Research and Evaluation Final Report, https://cslireland.ie/images/downloads/Final_CSL_Research_and_Evaluation_Final_Report__Feb_2018_.pdf; (Education and Training 2020 Working Group, 2018[30]), Teachers and school leaders in schools as learning organisations Guiding Principles for policy development in school education, https://ec.europa.eu/education/sites/education/files/teachers-school-leaders-wg-0917_en.pdf.
Make school principals’ appointment transparent and solely based on merit
Improving instructional leadership in schools will also require making sure that the best candidates, according to their professional competence, are selected and that there is trust in the integrity of the appointment process. North Macedonia should consider revising the appointment process to make sure it is impermeable to political interference. School boards should be provided with clear guidelines on how to select suitable candidates. In their note on new principal appointments to the ministry, school boards should justify their selection of a candidate vis-à-vis nationally defined competence standards and minimum eligibility criteria. To limit political interference, municipalities’ role in validating school principals’ appointment and renewal might be abolished. It could be replaced by a validation process by the State Education Inspectorate (SEI) or another independent external body who would check that the selected candidate meets the minimum requirements for the position.
Ensure that school principals’ renewal and dismissal are based on independent and external appraisal
North Macedonia needs to depoliticise decisions about the renewal and dismissal of principals, while ensuring that principals are accountable through an independent process of performance appraisal. Unlike in most OECD countries that have processes of appraisal, school principals’ performance is only appraised in North Macedonia as the consequence of a formal complaint (OECD, 2013[1]).
North Macedonia should consider introducing an external appraisal for principals led by the inspectorate and based on the new principal standards. The principal appraisal system would play the dual role of keeping principals accountable for their performance, while identifying areas for professional development and improvement. The school board should then be required to justify decisions of dismissal based on the results of external appraisals. For this accountability process to work, the integrity and independence of the SEI needs to be guaranteed (see Recommendation 4.1.1.).
Policy issue 4.4. Providing schools with greater resources to enhance the quality and impact of school evaluation
Schools in North Macedonia will need more resources to appropriate evaluation as a tool to drive their own improvement. Better access to their own data will allow schools to analyse and monitor their performance and, compare themselves to others. Second, schools need predicable, adequate financial resources to introduce quality improvements. The current absence of a school funding formula – or any established funding rules – to distribute funds from municipalities to schools vastly limits schools’ ability to plan and implement changes.
Recommendation 4.4.1. Provide schools with indicators and tools to measure their performance
For school evaluation to inform improvement, it needs to provide a fair and comparative view of schools’ practices and outcomes, and how they compare to practices in other schools with similar contexts (MacBeath, 2008[34]). One source of data that many countries use to inform this are standardised assessment and examination results. At present in North Macedonia however, data from the national examination at the end of upper secondary – the state matura – are not used as a measure of student performance during evaluations, and there is no national assessment.
Strengthen the use of standardised measures of learning outcomes in school evaluations
The SPQI framework for school evaluation includes few indicators on student learning outcomes limiting schools’ and the inspectorate’s capacity to evaluate how far learning outcomes in a given school compare to national or local benchmarks.
In the immediate term, the State Education Inspectorate should ensure that results from the State Matura examination are used as a measure of learning outcomes in the SPQI framework. The ministry and the NEC should also make sure that schools have access to national and local averages to benchmark their performance (see Chapter 5). In the medium term, schools and the inspectorate can use results from the planned national student assessment as a reliable measure of learning during schooling (see Chapter 5).
Rethink plans to rank schools based on students’ performance in the national assessment
To monitor learning outcomes nationally, the ministry is planning to introduce a new national assessment. One planned use of the results that was under discussion was to develop a school ranking system. Staff in schools at the top of the ranking will receive financial bonuses. While the introduction of a national assessment is a great step forward in improving monitoring of learning outcomes in North Macedonia, using assessment results alone to rank schools and reward certain teachers is unfair and does not accurately reflect school quality. Assessment results need to be contextualised to avoid unfairly rewarding schools with students from more advantaged backgrounds.
The ministry should reconsider the planned school ranking. Integrating the assessment results in the SPQI framework will ensure that via self-evaluation, assessment results are used to encourage schools to reflect on how they are supporting students’ learning outcomes. While using the results in external evaluations will ensure that they support school accountability.
In the medium term, once the EMIS system is further developed and data collection improved, the ministry could consider developing a school index to provide schools with a simple tool to compare their performance to that of other with similar levels of inputs and socio-economic contexts. The ministry could use the Romanian school efficiency index presented in Box 4.5 as an example. Romania’s school efficiency index looks at both the inputs and outputs of a school and is used to inform external evaluation and internal school evaluations (Kitchen et al., 2017[17]). The index will complement the qualitative school evaluation system. Qualitative measures of school quality should continue to be given priority as they capture better the processes and practices associated with quality learning and teaching.
In 2009, Romania’s Agency for Quality Assurance in Pre-University Education (ARACIP) began developing a contextualised attainment model to take account of the factors that may have a strong influence on students’ learning outcomes. The data are collected directly from the schools by ARACIP, and since 2014, the collection has taken place exclusively on line. The index was first piloted in 2011 in 1 023 schools across all levels - kindergartens, primary, gymnasium and high school. The methodology was further revised and the index was applied to another 1 300 schools in 2014. In 2016, a ministerial decision confirmed the intention to extend the index to the remaining schools that have not yet been included in the pilot index.
The index is calculated at the school level and currently incorporates input indicators for:
-
Family background: for example, the percentage of children from families with low income, parents’ education in number of years and the average commute time between home and school.
-
Education environment: for example, if the school is located in a socio-economically disadvantaged area, the number of school shifts and the average number of pupils per class.
-
Infrastructure: for example, the availability of basic utilities such as water and electricity, and the availability of classroom furniture such as desks.
-
Equipment and teaching aids: for example, the number of books in the school library, the number of computers, and the number of computers with Internet connection per 100 pupils.
-
Information communication technology (ICT): the level of ICT use in the school.
-
Human resources: for example, the percentage of qualified teachers, new teachers, and the average teacher-pupil ratio.
Expected and actual results are measured through the following indicators:
-
Participation: the average number of absences per pupil, the percentage of students dropping out and the percentage of students repeating a grade.
-
Results: the distribution of average classroom assessment marks at the end of the school year, the average results in the grade 8 and baccalaureate national examinations, and average results in the competence certification exam for vocational schools.
When a school receives its quality certificate following an ARACIP evaluation, it also receives its performance against the efficiency index criteria and its overall “index” of efficiency. Where this value is higher than 1, it indicates that a school is achieving better results than other schools functioning in similar conditions and with similar resources.
Source: (Kitchen et al., 2017[17]), OECD Reviews of Evaluation and Assessment in Education: Romania, OECD Publishing, Paris, https://doi.org/10.1787/9789264274051-en.
Simplify schools’ access to data and indicators
The ministry should simplify schools’ access to data so that they can monitor their learning outcomes, student completion and drop-out rates. Currently, schools need to take the initiative and contact two different units in the ministry to receive national or regional data on student retention, enrolment and examination results, which they can use as benchmarks in their self-evaluations. The review team’s discussions with the EMIS unit indicate that few schools, if any, request this data.
As the ministry develops its EMIS system (Chapter 5), it should consider including a school “view” that captures all the key data on individual schools, and provides national and regional benchmarks. The ministry can also choose to make this information available to the larger public to educate parents and citizens about what school quality means and create a demand for improvement. As mentioned above, such information needs to be presented in a way that takes into account school context.
Recommendation 4.4.2. Provide schools with adequate financial resources to implement their improvement plans
While schools in North Macedonia have significant discretionary power over the use of their resources, decades of underfunding and lack of transparency in the distribution of financial resources mean that they have few financial resources to implement improvements. The ministry needs to revise how funds are distributed to schools to ensure transparency, fairness and efficient use of resources. The ministry also needs to ensure that all schools have minimum funding levels so that they can act upon the recommendations of school evaluations.
Ensure that school resources are allocated fairly and transparently
In North Macedonia, school funding is distributed from the ministry to municipalities according to a clear funding formula. However, municipalities are not required to follow any specific approach or formula when they allocate central funds to schools. A few municipalities, including the City of Skopje, use a funding formula to distribute funding to schools. Most municipalities use a historic funding model and there is a general lack of transparency in how funds are distributed. In some municipalities, funds are withheld from schools for political reasons or to pay the debts that they have built up (see Chapter 1). This means that schools may not receive funding that is adequate for their needs and they have limited ability to plan ahead.
Most OECD countries use funding formulas to ensure that schools receive funding that is adequate to implement their programmes. Funding formulas also ensure transparency and fairness. Formulas are often based on per student funding and control for contextual variables such as the socio-economic background of students, the localisation of the school (i.e. rural or urban) and other relevant equity variables for the country (i.e. ethnicity or language of instruction) to make sure that schools with a large share of disadvantaged students receive additional support (OECD, 2017[35]).
The ministry should consider making it mandatory for municipalities to use funding formulas and provide clear guidelines on the variables to be included. For instance, given the high level of disparities in learning outcomes factors like urban and rural schools and students’ socio-economic background should be taken into account when determining funding levels. As discussed in Chapter 5, oversight of municipal use of funds also needs to be significantly strengthened.
Provide schools with discretionary funds to implement their school plan
Improvements in transparency and fairness of funding will however not be enough. Schools in North Macedonia need access to additional funds to implement their school plans. Current school budgets are insufficient to cover recurring costs, leaving school with little additional budget to introduce new projects or programmes (World Bank, forthcoming[36]). This makes it almost impossible for many schools to set and implement meaningful plans for improvement. The ministry should consider providing schools with small discretionary grants directly from central funding for professional development or projects in their action plan. This would help to incentivise schools to take the improvement function of school self-evaluation seriously. The use of this grant can be monitored by the municipal audit inspection, which already oversees school spending of funds.
To make such funds available for schools, the ministry will need to profoundly review the use and allocation of resources to free up funds for school improvement. As shown in Chapter 1, considerable gains are possible from rethinking the school network.
Conclusion
North Macedonia has invested significantly in developing a modern framework for school evaluation, based on research about what matters for school quality. A decade after it was implemented however, this framework has had limited impact. Professionalising the inspectorate, providing greater support for school level capacity, and making small changes so that the evaluation framework is more operational could result in a major change. The result will be a school evaluation framework that provides far more effective support to help schools raise the quality of their learning environments.
Professionalising the State Education Inspectorate
4.1.1 Guarantee the independence and integrity of the SEI. The appointment requirements for the SEI’s director should focus on demonstrated competence in school improvement and a strong understanding of how evaluation impacts school quality. Adherence to national codes of conduct should also be enforced, with any violations resulting in dismissal.
Increased professional independence of the SEI will need to be balanced by greater oversight of, and accountability for, its work. All stakeholders should have clear and fair opportunities to redress any grievances. A board of respected national education experts could also be formed to help maintain the SEI’s independence and oversee the quality of its work. The role and prominence of the SEI’s annual report could also be reinforced by focusing explicitly on the quality of the organisation’s work (and not merely reporting activities undertaken), and debating its contents in parliament.
Accompanying the above measures with a national consultation to determine a shared vision of “a good school” in North Macedonia would help in developing greater national understanding and ownership of the role of evaluation in supporting school improvement and better student outcomes.
4.1.2 Build the professional capacity of the SEI. New inspectors currently receive only three days of initial training, which is inadequate to develop the skills needed to evaluate schools in a way that is consistent and valid. As an immediate priority, the SEI should design and deliver a set of training courses for existing inspectors with a focus on explaining the purpose of school evaluation and developing key evaluation competencies, such as how to conduct a classroom observation and report back to schools. The training should provide inspectors with practical opportunities to try out new techniques and receive feedback, and to participate in an evaluation visit. In the medium term, the ministry will need to revise inspectors’ initial training to bring it more into line with the duration, structure and depth of well-established programmes in OECD and EU countries.
To expand the breadth and depth of the SEI’s experience and expertise, it should consider training and licensing experts as external consultants that can join the evaluation teams on an ad hoc basis. Possible external consultants might include experienced teachers from other schools and advisors from the BDE and the Vocational Education and Training Centre (VETC).
Ensuring that integral school evaluation focuses centrally on improving school quality
4.2.1 Revise school integral evaluation to focus more centrally on the quality of teaching and learning. The indicators in the School Performance Quality Indicators (SPQI) framework that guide evaluation should be reduced to around 10 to 15 and revised to distinguish between a set of core indicators to be evaluated in every cycle and secondary indicators to be evaluated on a rotating/discretionary basis. This will make the framework more manageable for inspection teams and give them more time to focus on key indicators of teaching and learning. There are also gaps in the existing framework that need to be addressed. For example, indicators on school pedagogical leadership, the quality of self-evaluation and schools’ capacity to reflect on its practices should be included as part of the core indicators evaluated by the SEI.
In order to create more time to meaningfully evaluate teaching and learning, the individual teacher appraisals that are currently part of the integral evaluation should be replaced with more extended classroom observations of a sample of classrooms to gain a deeper understanding of instruction in the school. Individual teacher appraisals should instead be led by the school principal and the BDE (Recommendation 3.2.1). To reduce the administrative burden that integral evaluations place on schools, the SEI should simplify and digitalise the collection of administrative data.
4.2.2 Make sure that integral evaluations deliver constructive feedback to schools. The SEI needs to make sure that recommendations in the school evaluation reports are clear, specific and actionable. The SEI can start by reviewing a sample of national reports and interviewing schools to understand how feedback is used. It should also look at international practices on reporting. Low-performing schools will likely require additional assistance to act on feedback, and the ministry should explore ways to concentrate external support on where it can make the most difference. Towards this end, the country should consider gradually introducing a risk-based approach to school evaluation and follow-up that prioritises schools at risk on core indicators. As part of this new model, the follow-up visits by inspectors to all schools might be replaced by more sustained support for select schools that is led by the regional or municipal level. North Macedonia will need to consider the best structure to provide such support – for example, whether to create a body of school improvement officers that work across multiple municipalities or develop a separate improvement unit within the SEI.
Developing schools’ capacity to carry out meaningful self-evaluation
4.3.1 Provide support and training for school actors on self-evaluation, by considering the following actions:
-
Revising self-evaluation guidance. As a first step, the country might review schools’ experience of self-evaluation to understand what a new comprehensive self-evaluation manual should include.
-
Providing more training for school actors on self-evaluation. A mandatory, practical module on self-evaluation could be added to school principals’ initial preparation. Regular training should be offered to school staff involved in self-evaluation and school boards. Schools that struggle the most with undertaking meaningful self-evaluation might be offered technical assistance from BDE advisors.
-
Reviewing and providing feedback on self-evaluation during integral evaluations, by adding an indicator on the quality of self-evaluation practices to the integral evaluation framework.
-
Creating an online portal for schools to share their experience with self-evaluation. The ministry might also recognise schools with exceptional self-evaluation practices through a title like “learning organisation” or “excellence in using evidence” and share these examples of good practice on the online portal.
4.3.2 Develop school principals’ instructional leadership skills. North Macedonia should consider creating a leadership academy to help professionalise the principal role. This academy would be in charge of providing initial preparation and in-service professional development for principals. As a first step, the academy should co-ordinate the finalization and introduction of competency standards for principals, which would be used to inform their selection and training, and the evaluation of school leadership during school evaluations.
Protecting the principal appointment process from political interference will be essential to ensure that principals are selected based on their professional competence. Ways to achieve this include providing school boards with clear selection guidelines and ending municipalities’ role in validating principal appointments and renewals. The municipalities’ validation process should be replaced by validation by an external and independent body, such as the SEI. Decisions about principal renewal and dismissal should also be depoliticised by introducing a principal performance appraisal to provide a fair and independent measure of performance.
Providing schools with greater resources to strengthen the quality and impact of school evaluation
4.4.1 Provide schools with indicators and tools to measure their performance, for example, by using results from the State Matura as a measure of students’ learning outcomes in the school evaluation framework. The ministry should also make data more accessible to schools so that they can monitor key outcome indicators such as students’ learning outcomes, completion and drop-out rates. One option is to develop a school portal or “view” on EMIS that gives individual schools access to their own data and provides national and regional benchmarks.
North Macedonia’s plans to introduce a national assessment are very positive and will provide essential data for monitoring learning outcomes nationally (see Policy issue 5.2). However, the intention to use the results for school ranking should be reconsidered. Using assessment results alone to rank schools and reward certain teachers is unfair as it does not control for the school’s socio-economic profile. Instead, the assessment results can be included in the school evaluation framework to encourage schools to reflect on how they support students’ learning outcomes and school accountability. In the medium term, the ministry might consider developing a school index that contextualises school performance including factors such as financial inputs and socio-economic context.
4.4.2 Provide schools with adequate financial resources to implement their improvement plans. The ministry should consider making it compulsory for municipalities to use funding formulas and provide clear guidelines on the variables to be included (e.g. urban/rural, students’ socio-economic background). The ministry should also consider providing schools with small discretionary grants from central funding for professional development or implementing projects under their improvement plan.
References
[31] BMBWF (2018), Leadership Academy, Bundesministerium für Bildung, Wissenschaft und Forschung, https://www.leadershipacademy.at/ (accessed on 7 March 2019).
[16] Burns, T., F. Köster and M. Fuster (2016), Education Governance in Action: Lessons from Case Studies, Educational Research and Innovation, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264262829-en.
[9] DuFour, R. (2004), “What Is a Professional Learning Community?”, Educational Leadership, Vol. 61/8, pp. 6-11, http://www.ascd.org/publications/educational-leadership/may04/vol61/num08/What-Is-a-Professional-Learning-Community%C2%A2.aspx.
[30] Education and Training 2020 Working Group (2018), Teachers and school leaders in schools as learning organisations Guiding Principles for policy development in school education, The European Commission, Brussels, https://ec.europa.eu/education/sites/education/files/teachers-school-leaders-wg-0917_en.pdf (accessed on 7 September 2018).
[23] Education Scotland (2015), How good is our school? (4th edition), http://www.educationscotland.gov.uk/resources/h/hgios4/ (accessed on 23 August 2018).
[14] Education Scotland (n.d.), Framework document, https://education.gov.scot/Documents/EducationScotlandFramework.pdf (accessed on 27 March 2019).
[6] European Commission/EACEA/Eurydice (2015), Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe, Publications Office of the European Union, Luxembourg, http://eacea.ec.europa.eu/education/eurydice/documents/thematic_reports/178en.pdf.
[10] Eurydice (2013), Eurydice Highlights: Key Data on Teachers and School Leaders in Europe, http://eacea.ec.europa.eu/educa (accessed on 7 February 2018).
[24] Faubert, V. (2009), “School Evaluation: Current Practices in OECD Countries and a Literature Review”, OECD Education Working Papers, No. 42, OECD Publishing, Paris, https://doi.org/10.1787/218816547156.
[33] Fitzpatrick Associates (2018), School Leadership in Ireland and the Centre for School Leadership: Research and Evaluation Final Report, https://cslireland.ie/images/downloads/Final_CSL_Research_and_Evaluation_Final_Report__Feb_2018_.pdf (accessed on 7 March 2019).
[13] Government of the Republic of North Macedonia (2015), Law on educational inspection (“Official Gazette of the Republic of Macedonia“ No. 52/2005, 81/2008, 148/2009, 57/2010, 51/2011, 24/2013, 137/2013, 164/2013, 41/2014, 33/2015 and 145/2015), http://www.mon.gov.mk/images/documents/zakoni/zakon-za-prosvetna-inspekcija-2015.pdf (accessed on 27 March 2019).
[28] Ingersoll, R., P. Sirinides and P. Dougherty (2017), “School Leadership, Teachers’ Roles in School Decisionmaking, and Student Achievement”, CPRE Working Papers, Consortium for Policy Research and Education, https://repository.upenn.edu/cpre_workingpapers/15 (accessed on 5 September 2018).
[17] Kitchen, H. et al. (2017), Romania 2017, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264274051-en.
[34] MacBeath, J. (2008), “Leading learning in the self-evaluating school”, School Leadership and Management, Vol. 28/4, pp. 385-399.
[20] MEXT (2008), Basic Plan for the Promotion of Education（Provisional translation), http://www.mext.go.jp/en/policy/education/lawandplan/title01/detail01/1373797.htm.
[21] Ministry of Education and Research (n.d.), The Estonian Lifelong Learning Strategy 2020, https://www.hm.ee/sites/default/files/estonian_lifelong_strategy.pdf.
[11] MoES (2018), Republic of North Macedonia - Country Background Report, Ministry of Education and Science, Skopje.
[27] Nusche, D. et al. (2012), OECD Reviews of Evaluation and Assessment in Education: New Zealand, https://www.oecd-ilibrary.org/docserver/9789264116917-en.pdf?expires=1535964711&id=id&accname=guest&checksum=548FB833921B2C149A6722114C665722 (accessed on 3 September 2018).
[18] OECD (2018), Developing Schools as Learning Organisations in Wales, Implementing Education Policies, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264307193-en.
[22] OECD (2018), Education for a Bright Future in Greece, Reviews of National Policies for Education, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264298750-en.
[35] OECD (2017), The Funding of School Education: Connecting Resources and Learning, OECD Reviews of School Resources, OECD Publishing, Paris, https://doi.org/10.1787/9789264276147-en.
[3] OECD (2016), PISA 2015 Results (Volume II): Policies and Practices for Successful Schools, PISA, OECD Publishing, Paris, https://doi.org/10.1787/9789264267510-en.
[26] OECD (2016), What makes a school a learning organisation?, http://www.oecd.org/education/school/school-learning-organisation.pdf (accessed on 24 October 2018).
[2] OECD (2015), Education at a Glance 2015: OECD Indicators, OECD Publishing, Paris, https://dx.doi.org/10.1787/eag-2015-en.
[8] OECD (2014), TALIS 2013 Results: An International Perspective on Teaching and Learning, TALIS, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264196261-en.
[1] OECD (2013), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264190658-en.
[15] Ofsted (2018), Ofsted inspections: myths, https://www.gov.uk/government/publications/school-inspection-handbook-from-september-2015/ofsted-inspections-mythbusting (accessed on 24 August 2018).
[19] Ontario Ministry of Education (n.d.), Education in Ontario, https://www.ontario.ca/page/education-ontario.
[29] Pont, B., D. Nusche and H. Moorman (2009), Improving School Leadership: policy and practice, OECD Publishing, Paris, http://www.sourceoecd.org/education/9789264044678 (accessed on 7 September 2018).
[25] Riley, K. and J. Macbeath (2000), Putting self-evaluation in place, The World Bank Group, Washington DC, http://www.woddbank.orq (accessed on 31 August 2018).
[5] Rudduck, J. (2007), “Student Voice, Student Engagement, And School Reform”, in International Handbook of Student Experience in Elementary and Secondary School, Springer Netherlands, Dordrecht, https://doi.org/10.1007/1-4020-3367-2_23.
[7] Schleicher, A. (2015), Schools for 21st-Century Learners: Strong Leaders, Confident Teachers, Innovative Approaches, International Summit on the Teaching Profession, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264231191-en.
[32] Schratz, M. and M. Hartmann (2009), Innovation an Schulen durch Professionalisierung von Führungskräften Studie zur Evaluation der Auswirkungen der Leadership Academy an Schulen, Bundesministerium für Unterricht, Kunst und Kultur, https://www.leadershipacademy.at/downloads/LEA_Kurzfassung_Studie.pdf.
[4] SICI (2003), Effective School Self-Evaluation, Standing International Conference of Inspectorates, Belfast.
[12] State Education Inspectorate (2014), Indicators of school quality, http://dpi.mon.gov.mk/images/pravilnici/IKRU-MAK.pdf.
[36] World Bank (forthcoming), North Macedonia: Public Finance Review, World Bank, Washington D.C.