5. Promoting the use of evidence in Austria’s education system
This section analyses the results of the online survey among key decision makers in the Austrian education system. The survey gauged the presence of efforts and opportunities that empirical research finds to promote decision makers’ use of evidence. The section is structured around decision makers in policy (executives in the federal ministry and education directorates), quality assurance (school quality managers) and schools (school leaders).
The Federal Ministry for Education, Science and Research (Bundesministerium für Bildung, Wissenschaft und Forschung, BMBWF), with the support of a group of graduate students of the Vienna University of Economics and Business, carried out an online survey among key decision makers in the Austrian education system in November 2019. The survey is adapted for use in the Austrian context from the Knowledge Governance module of the OECD’s policy toolkit for strategic education governance. The OECD, in cooperation with the BMBWF, developed questionnaire adaptations for four different target groups. Two first target groups for the survey were decision makers at the BMBWF itself and in the education directorate in each province. Education directorates are carrying out jointly the executive responsibilities of the provinces and the federal ministry in education. School quality managers (SQMs) were a third target group. They are part of the education directorate in each province. Each province is divided in one or more Education Regions. SQMs take the place of former school supervision and take a regional focus. SQMs are responsible for optimising and coordinating the education offer within a given Education Region. They support schools in their quality assurance based on a quality management system common to all school types and exert a supervisory function in relation to school leaders. A fourth target group were school leaders. Regarding this most numerous group, the 2017 Education Reform Act increased autonomy of schools to better cater to the local needs of students (BMBWF, 2019[1]).
The different adaptations for these groups comprised about 25 questions each. They were divided into the five areas that promote the capability, motivation, and opportunity to use evidence in decision making:
The skills to access and make sense of evidence.
Making relevant evidence conveniently available to decision makers.
Fostering the organisational processes and structures that encourage use of evidence.
Fostering the exchange among decision makers and their exchange with evidence producers, and
Promoting use of evidence as a principle of good decision-making, building a shared understanding on what constitutes fit-for-purpose evidence, as well as how and when evidence should be used.
The survey included further introductory questions that relate to current evidence use of the respective target group and closed with an open question for further suggestions and innovative examples. The questionnaires were sent out to 72 executives of the federal ministry, 27 executives from the education directorates, 163 school quality managers (SQM) and 5 745 Austrian school leaders. Overall, around half of all decision makers asked to participate completed the questionnaire (47%).
Half of executives of the BMBWF (35 of 72, 49%) and two-thirds of education directors (17 of 27, 63%) responded to their respective adaption of the questionnaire. The education directorates are a new administrative authority for the education sector in any one province, bringing together the previously dispersed administrative tasks of the federal Government and the given province (Land). The education directorates are responsible for the overall implementation of school legislation. This includes quality assurance through school supervision and school development in the form of school quality managers (SQM). The Federal Ministry of Education, Science and Research (BMBWF) is set to oversee external school evaluation/ inspection, which is to be distinct from tasks carried out by school quality managers (school supervision) (BMBWF, 2019[1]).
Decision makers in the federal ministry (BMBWF) overwhelmingly consider use of evidence to be not yet as extensive in the education system as it could be. Education directors are likewise keen to expand the use of evidence in the system. In terms of using evidence, virtually all decision makers in the federal ministry and education directorates rely frequently or regularly on evidence in their work. They use varied resources to access relevant evidence. This includes dedicated departments within the federal ministry and education directorate, respectively. Education directorates work closely with the federal ministry directly to access relevant evidence. Decision makers carry out their own research, as well as make use of out-contracted research and external experts. Resources also include national institutes (such as the Federal Institute for Educational Research, Innovation and Development BIFIE)1 and international organisations (such as OECD, EU, and Non-Governmental Organisations). Decision makers also reach out to universities and, to a lesser extent, university colleges of teacher education to inform their decisions.
Building the skills for effective evidence use at the BMBWF and education directorates
Developing the skills to make sense of evidence are crucial to consider evidence systematically in daily decision making. Decision makers in the federal ministry rely on exchange with colleagues and report (external) professional development to help them acquire skills to use evidence effectively. To develop their skills to better instruct their staff in the use of evidence, decision makers in education directorates and the federal ministry report (external) professional development as the most widely available opportunity.
Around a third of decision makers who completed the survey in federal ministry and education directorates is aware of professional development opportunities tailored to their role as supervisors to build these skills. Another third is not aware of any opportunities to acquire skills to better make use of evidence or supervise the use of evidence. This may suggest that respective opportunities are not systematically available for all decision makers or that their availability is not systematically communicated. Regarding efforts to build the skills to access and make sense of evidence, one senior decision maker at the federal ministry highlights:
“The department is developing a training programme on data literacy. In this area, there is a need for many more offers for in-house staff and for the education directorates, especially for SQM [school quality managers]”.
Organisational resources are crucial to help senior decision makers guide and instruct their staff’s use of evidence and encourage staff to acquire skills relevant to make better use of evidence in decision making. This includes access and use of competency frameworks, instructions/guidance on how to carry out staff reviews to foster evidence use, and staff development measures to encourage staff to develop their skills to use evidence. Decision makers in the federal ministry appear to be not systematically aware of or able to draw on resources to help guide their staff in the use of evidence. Only a quarter of decision makers in the federal ministry is aware of any one measure to support them in this task. More than 40% of decision makers are not aware of any resources to guide their staff in the use of evidence.
Making evidence available
Untargeted communication of evidence – or access to evidence, for instance through databases – can make it difficult for the audience of decision makers to identify the most relevant elements for their practice. Beyond targeting groups of decision makers, tailoring evidence to decision makers’ specific preferences and work habits can further increase convenience and thus improve the uptake.
The federal ministry does not systematically target evidence to different groups. While half of senior decision makers who completed the questionnaire reported that the federal ministry mostly targets different groups, a third of respondents reports that the ministry does little to target its evidence to specific groups. Where the BMBWF tailors evidence to the needs and habits of its audience, it largely focuses on topics/content, communication channels and, to a lesser extent, different levels of experience with using evidence.
In contrast, adapting evidence to user needs is high on the agenda for education directorates, with over two-thirds of decision makers reporting to this effect. Well over half of decision makers in education directorates who completed the survey report to tailor evidence to target groups such as school quality managers, school leaders, teachers or the general public. Education directorates report to tailor evidence largely along different levels of experience in using evidence and interest in different topics among target groups.
Consulting the audience in preparing evidence can help improve its relevance, as well as the motivation and ease of using evidence. This may include consultation over relevant topics and preferred communication channels and type of preparation (such as databases, newsletters, information material, or advisory services). Equally, involving stakeholders in the preparation of evidence can include soliciting content (such as writing articles for a publication). Stakeholders may also be included in evaluation efforts. Overall, education directorates involve others to a greater extent in their preparation of evidence. However, the BMBWF puts greater emphasis on involving teachers in evaluation efforts related to preparing its evidence (Figure 5.1).
Note: Follow-up question drawing on positive responses to question “The [ministry] [education directorate] involves the users of evidence in the preparation of evidence”. Responses to the question “With regard to which aspects does the [ministry] [education directorate] involve the listed user groups?” N=22 (ministry), N=14 (education directorates).
Organisational processes
Organisational processes are relevant to efforts to promote the use of evidence mainly by encouraging its use. This includes promoting transparency of decision-making processes (for instance, through publishing decision-making procedures or direct exchange with stakeholders to make decisions plain) and involving different perspectives and stakeholders in decision making. Compared to the BMBWF, education directorates are found to engage more frequently in measures to create transparency around decision processes and involving internal and external stakeholders.
Efforts to clarify decisions and decision-making processes focus on internal exchange
The BMBWF and the education directorates work to make decisions plain and decision-making processes transparent mainly through internal exchange. Within the federal ministry, three-quarters of decision makers who completed the survey reported so. Virtually all education directors engage in exchange within their directorate to clarify decisions. Engaging with other stakeholders to this effect is likewise relatively widespread in the federal ministry and the education directorates, with close to two-thirds of decision makers engaging with (external) stakeholders. Decision makers in the BMBWF and education directorates also mention (publicly) available documents to clarify decision making. Making use of published documents to create transparency is more widespread in education directorates (60% vs. 35%).
Involving different perspectives in decision making can encourage greater use of evidence; by opening the decision-making process to stakeholders or by being encouraged to consider their perspectives in greater detail. Overall in the federal ministry and education directorates, diverse perspectives are more frequently involved in substantive decision-making areas of preparing, implementing and evaluating measures. In comparison with each other, education directorates put greater emphasis on internal processes of staff and organisational development than the ministry. However, the ministry involves different perspectives in evaluation more frequently than education directorates (Figure 5.2).
Note: N=34 (ministry), N=17 (education directorates). Answer to question "In which processes of the [ministry] [education directorate] are different perspectives (internal/external) taken into account or explicitly included in decision making?".
Awareness of knowledge management systems could be more systematic
Knowledge management systems help contextualise evidence and link knowledge across and within organisations for access by decision makers where needed. This can include general data management, a collection of organisational procedures, or other knowledge relevant to the functioning of the organisation and the decision-making processes at hand. Today, knowledge management systems are predominantly computer-based but likewise include printed material.
The focus of knowledge management is on systems to exchange and contextualise information within the ministry and education directorates. Online project tools, cloud services and an office suite are specifically identified as knowledge management systems. To make the best use of knowledge management systems, respective systems should be available to all decision makers and all decision makers should be aware of available systems. Decision makers across ministry and education directorates appear not systematically aware of available knowledge management systems. While a quarter of decision makers identified knowledge management available to them across organisations, a third of decision makers who completed the survey were not aware of any knowledge management system at their disposal, suggesting that decision makers do not have a common understanding of knowledge management systems.
Exchanging with colleagues to promote evidence use in decision making
Half of decision makers in the federal ministry and the education directorates who completed the survey report regular opportunities to exchange informally with colleagues regarding experiences and methods around using evidence. However, 14 of the 34 (40%) decision makers in the ministry who completed the survey, report not to have opportunities to exchange with colleagues informally or in organisationally established exchanges about using evidence in their work. Education directors report opportunities more frequently (75%) and fewer decision makers report to be without opportunities to exchange with colleagues about their experiences and methods to use evidence for their work (17%).
Exchanges with colleagues cover a wide range of topics. Topics include encouraging greater consideration of evidence, developing common standards for considering evidence in decision making, and improving quality or preparation of evidence. Of those in the ministry who report exchanges with colleagues around methods and experience around using evidence, 45% emphasise that these exchanges focus on strengthening the use of evidence in decision making and improving the preparation of evidence. Similar to the ministry, education directors emphasise in exchanges with colleagues how to encourage the use of evidence. Half of education directors who reported opportunities to exchange with colleague emphasise the development of common standards for using evidence as topic of these discussions.
Developing common standards for using evidence
Efforts to raise awareness of evidence use as a principle of good decision making are relatively widespread. This can include distributing information material on the merits and importance of considering evidence in decision making as well as events and initiatives to this effect. It may also include advisory services that include elements to raise awareness, or dedicated advisory for school quality managers (SQM) and school leaders on how to raise awareness among those with which they work. For SQMs this would include raising awareness for the merits of using evidence among schools; for school leaders this pertains to raising respective awareness among their teaching staff. Raising awareness is high on the agenda. As one education director highlights,
Evidence-based work is important, but in the future it needs an even stronger focus and awareness to see this as a basis for everyday work.
Distribution of information material is the most popular means to raise awareness followed by events and initiatives. Overall, education directorates engage more systemically in efforts to raise awareness, in particular in terms of providing advice and organising events for SQMs and school leaders to raise awareness and help them in turn to raise awareness (Figure 5.3).
Note: Follow-up question drawing on positive responses to question “In your opinion, does the ministry take measures to raise awareness of the use of evidence as a basis for decision making?”. Answers to question “Which awareness-raising measures are implemented for which target groups?” Multiple answers possible. N=21 (ministry), N=14 (education directorates).
Emergent efforts to develop a common understanding how evidence should be used in specific decision-making areas
Efforts to create a common understanding how evidence should be used in concrete decision-making areas and respective challenges are emergent. About half of decision makers in the ministry who completed the survey are not aware of any efforts to this effect or feel they cannot judge whether such efforts exist. Individual decision makers already engage in respective efforts in various decision-making areas. This includes decisions in preparing, implementing and evaluating measures, as well as staff development and governance decisions. Within these areas, efforts to develop common standards how to use evidence focus on governance decisions as one of the central responsibilities of the ministry and education directorates (Figure 5.4).
Note: N=34 (BMBWF), N=17 (education directorates). Answer to question “Are there efforts within the [ministry] [education directorates] to develop a common understanding of how evidence should be used in concrete decision-making situations?” Multiple answers possible.
Almost two-thirds (64%, 104 of 163) of school quality managers completed the survey across 31 education regions. Administrative arrangements are diverse regarding number of Education Regions in each province and the number of school quality managers overseeing education regions in any one province (Table 5.1). More than half (57%) of the responding SQMs work as school supervisors between two and ten years. More than 30% have even been working in this position for more than ten years. 12% of SQMs worked less than two years in their current position.
Developing skills to use evidence for quality assurance and supporting schools
The skills to access and make sense of evidence are crucial to consider evidence systematically in daily decision making. Overall, opportunities to build skills to access and make sense of evidence are widely available to school quality managers. Opportunities focus on diverse training offers and exchange with colleagues over mentoring/coaching and e-learning/learning platforms. School quality managers rely on previous studies, ongoing studies as well as individual coursework at universities to ensure they have the requisite skills to access and make sense of evidence for their daily work. They engage with regional university colleges of teacher education and individual university staff to build their skills.
What school quality managers frequently report lacking are opportunities to build skills to guide and instruct evidence use of their staff and of the schools with which they work. SQMs with the requisite skills to guide and instruct the use of evidence can motivate their staff as well as schools to make better use of evidence and to acquire relevant skills. Overall, close to a third of school quality managers report that there are no opportunities to build these skills specifically. On the upper end, this is the case for 10 out of the 17 school quality managers in Vienna. Reflecting this scarcity, individual school quality managers specifically pointed out they rely on their own initiative to build these skills, for example through engaging in self-study.
Varied resources to guide and instruct the use of evidence – with regional differences
Important for effective instruction and supervision of evidence use is that school quality managers can rely on organisational resources to support them in this task. This includes education and training requirements for staff, instructions for conducting reviews, and the possibility to initiate related development measures (Langer, Tripney and Gough, 2016[2]; Gray et al., 2012[3]). School quality managers report varied such resources available, though with substantial regional differences in which resources are available and in the degree to which these are systematically available to SQMs. School quality managers centrally guide and instruct school leaders in the use of evidence. SQM further oversee their staff’s use of evidence within their respective responsibilities. The SQM team in an education region includes diversity managers as well as administrative staff.
About half of all school quality managers report education and training requirements that include competencies related to evidence use to support them in instructing use of evidence. SQMs report to a similar degree that they have access to guidelines on how to conduct staff reviews with the aim to foster the use of evidence. About a third of school quality managers report that they have the possibility to initiate staff development measures related to use of evidence.
Competency development requirements appear systematically available and known to SQMs in Burgenland and Styria with over 80% of SQMs reporting them as available. Guidelines to make use of staff appraisal to promote use of evidence are systematically available to SQMs in Styria and Vorarlberg. Despite at a lower degree, still over half of all SQMs in Salzburg and Tyrol report both competency development requirements guidelines for staff appraisal as resources available to them. In the remaining provinces, resources appear unsystematically available, with only 50% or fewer SQMs reporting any one instrument available to them. In particular, in Vienna and Upper Austria, over a third of SQMs report that no resources are available to support them in instructing and guiding their staff’s use of evidence.
As part of their school supervision function, school quality managers have the task of guiding and instructing schools in the use of evidence. The school-internal evaluation is a legal obligation of the school which includes the use of the data for school development. In the context of performance measurements, it is the task of school leaders to define the development areas. The use of evidence, the school development measures derived from it and their results are important topics in the performance review and objective-setting discussions (BZG) between SQM and school leaders.2
To help school quality managers guide and instruct the use of evidence by the schools with which they work, school quality managers highlight the SAND tool (Schulentwicklung durch Analyse und Nutzung von Daten; School development through analysis and use of data). The tool and respective trainings present a resource to guide and instruct use of evidence. SQMs also report their engagement with Schulentwicklungsberater/-innen (school development advisors) and Rückmeldemoderatoren/-innen (feedback moderators) as an important resource helping them guide and instruct schools in their use of evidence for quality development. Moreover, school quality managers note the Bilanz- und Zielvereinbarungsgespräch (BZG, performance review and objective-setting discussion) as an instrument to support them in guiding and instructing schools in the use of evidence.
Making evidence available for decisions at the intersection of schools and regional level
School quality managers (SQM) work at the intersection of supporting schools and regional development. On the one hand, this requires work with evidence close to schools to support their quality development. This includes support of schools’ and their staff’s professional development as well as organisational development. On the other, school quality managers require evidence to optimise the education and support measures offered to students in the region, and allocate resource across their region.
School quality managers largely consider evidence as well prepared for their work but wish for more involvement to improve preparation and provision of evidence
Making evidence available pertains to supporting SQMs through providing relevant and adequately prepared evidence for their work at this intersection. This should entail BMBWF, BIFIE and other institutions working together with SQMs to identify which evidence is most relevant and in which form it is most user-friendly. On the other hand, this means supporting and encouraging SQMs to engage on their own initiative in bringing together and preparing evidence for their work.
SQMs regard the availability and user-friendliness of evidence with which they are provided largely positive. Overall, around 60% of SQMs largely or fully agree that evidence is well prepared for their work. Around a third of SQMs praise the evidence prepared by research institutions, such as BIFIE and university colleges of teacher education as highly user-friendly. Around half of SQMs perceive the evidence provided BMBWF and education directorates as largely user-friendly.
Given requisite capabilities, local decision makers are best placed to identify which evidence in which form can help them best to develop actionable knowledge for decision-making challenges at hand in the specific local context (Langer, Tripney and Gough, 2016[2]; Burns, Köster and Fuster, 2016[4]; Shewbridge and Köster, 2019[5]). Moreover, school quality managers face a change in their responsibilities from the 2017 governance reform. Effective and efficient work processes and habits take time to emerge. Needs regarding evidence are not fully known from the beginning and emerge over time in the process of day-to-day decision making.
Note: N=104. Share of answers to question “As School Quality Manager, how do you access the evidence relevant to you?”. Multiple answers possible. Categories pertaining to evidence on SQMs initiative in light blue. ‘Other’ includes school administrative data, direct engagement with schools and school leaders, interviews with school graduates, parents, students, teachers and others.
School quality managers would like to be involved to a greater extend in the provision and preparation of evidence. Only 30% of SQMs largely or fully agree to feeling adequately involved in providing and preparing evidence. However, regional differences are pronounced. In Carinthia and Vorarlberg, no SQM feels more than marginally involved3. On the other end, in Salzburg and Burgenland, over six out of ten SQM largely or fully agree to being sufficiently involved. In particular, SQMs wish for more involvement in deciding on relevant topics as well adequate communication channels. SQMs in Upper Austria show the greatest demand for more involvement over all. Here, half of all SQM wish for more involvement in defining topics, improving communication channels, effective formats, as well as the evaluation of evidence provision.
Beyond evidence made available to them, SQMs engage with evidence on their own initiative. More than 75% of SQMs engage with research to create or revise their knowledge, such as through working with research literature. Over 15% of SQMs report to engage in bringing together evidence themselves. This includes evidence from schools such as school administrative data and direct engagement with schools and school leaders as well as interviews with school graduates, parents, students, and teachers (Figure 5.5).
At the intersection between school support and regional development, school quality managers engage in their own preparation evidence for their work. Preparing evidence to support schools in their quality development is more widespread. Three-quarters of SQMs work up evidence themselves to support schools’ development and professionalising schools and their staff. Around 40% engage in working up evidence related to regional development (Figure 5.6).
Note: N=104. Share of answers to question "Do you prepare evidence yourself?”. Multiple answers possible. Categories pertaining to evidence to support schools’ development in light blue. ‘Other’ includes preparing performance review and objective-setting discussions (Bilanz- und Zielvereinbarungsgespräche, BZG), analysing educational opportunities, resource planning, evaluating development plans.
Organisational processes to foster the use of evidence in school quality assurance
Organisational processes within the education directorates and the Education Regions can encourage the use of evidence in decision making in school quality assurance. This pertains to clarifying decisions and how these were reached, inviting additional perspectives into decision making as well as being able to rely on knowledge management systems that help contextualise evidence to develop actionable knowledge.
School quality managers clarify how decisions were reached to stakeholders and – with regional differences – involve diverse perspectives in decision making
Around 70% of school quality managers report at least one measure to make decisions-making processes plain and make clear how decisions were reached. Most widespread are exchange formats within the Education Region. Over half of school quality managers are involved in respective exchanges. This can include exchanges with schools and stakeholders, as well as colleagues within the Education Region. Similar exchange processes with other colleagues within the education directorate are less common. Respective processes are reported by about one-third of school quality managers. Only about 10% of school quality managers report efforts to make decisions plain through (publicly) available documentation of the process behind decisions.
Overall, school quality managers involve or explicitly consider perspectives of different internal or external stakeholders mainly in substantive areas of decision-making (preparing, implementing and evaluating activities). Involvement is slightly lower in internal processes, in particular when concerning staff development. School quality managers’ use of evidence across areas of decision-making follows a similar pattern (Figure 5.7).
Note: N=104. Share of answers to questions "In which processes in the Education Region are different perspectives (internal/external) taken into account or involved in decision-making?” and “As School Quality Manager, how often do you use evidence in the following areas?” (Cumulated answers Regularly/Frequently). Multiple answers possible.
In some provinces, school quality managers prioritise individual areas of decision making in which they involve or explicitly consider diverse perspectives. Contrary to the average, SQMs in Burgenland emphasise organisational development and staff development. Here, all six school quality managers who answered the survey, report to involve diverse perspectives frequently or regularly in respective decisions. In Vorarlberg and Vienna, SQMs emphasise the preparation of measures. In either province, over 80% of SQMs report to involve or explicitly consider diverse perspectives frequently or regularly in respective decision-making, compared to around 50% across the remaining decision-making areas.
Limited awareness of knowledge management systems among school quality managers
Knowledge management systems help contextualise evidence and link knowledge across and within organisations for access by decision makers where needed. This can include general data management, a collection of organisational procedures, or other knowledge relevant to the functioning of the organisation and the decision-making processes at hand. Today, knowledge management systems are predominantly computer-based but likewise include printed material. To make best use of knowledge management systems, respective systems should be available to all decision makers and all decision makers should be aware of available systems.
Among the 104 school quality managers who filled out the survey, about 70% report to have access to some form of knowledge management. This includes knowledge management systems to support SQMs in exchanging knowledge across the directorate and its (potentially) multiple Education Regions; systems to share knowledge within one Education Region; and systems allowing to link and exchange knowledge across different organisations. Awareness of available knowledge management systems varies greatly across provinces. In Salzburg, Tyrol and Styria virtually all school quality managers report at least one available knowledge management system. In Carinthia, Upper and Lower Austria, and Vienna, only around 60% of SQMs report access to at least one knowledge management system. Within provinces, knowledge of available knowledge management systems appears unsystematic. Overall, only 30%-50% of SQMs report any one individual type of knowledge management system, indicating that there may be no common understating of knowledge management systems.
Exchange with evidence providers
Variation in how systematically school quality managers exchange with evidence providers
On average, school quality managers in Salzburg, Styria and Vienna interact similarly frequently with evidence producers as in Burgenland. However, interaction is carried out less systematically, with some school quality managers reporting no engagement of quality management with evidence providers. In Burgenland, SQMs report systematically to interact about once per semester with evidence providers. In the remaining provinces, exchange between school quality managers and evidence providers appears relatively unsystematic. Here, some SQMs report to engage with evidence providers once a year or once per semester; in some cases, SQMs interact with evidence providers up to once a month as well as based on immediate needs. However, across provinces, between 25% and 60% of SQMs report no interaction with evidence providers. In eight provinces, the purpose of exchange between and SQMs and evidence providers systematically aims at improving the quality and preparation of the provided evidence. SQMs in Vorarlberg report a more ad-hoc and needs-based approach to exchange. Consistent with this approach, SQMs in Vorarlberg report the aim of improving the quality to a lesser extent.
Supporting the exchange between evidence providers and SQMs organisationally is important to promote the opportunity for exchange beyond individually initiated exchange. Overall, the exchange of SQMs with evidence providers appears not systematically supported. Organisational support in four out of the nine provinces is fragmented with some SQM reporting high and others reporting low levels of organisational support for their exchange with evidence providers. However, SQMs in a number of provinces report organisational support more systematically. This includes Burgenland, Tyrol, and Vienna, and, with lower frequency, Upper Austria and Vorarlberg.
Fostering the use evidence through collegial exchange
Exchange with colleagues can promote use of evidence through various channels. This includes encouraging use of evidence in decision-making through building awareness of its importance and fostering the motivation and capability to use evidence. Exchange among colleagues offers the opportunity to develop a common understanding for using evidence in decision-making. It further can be used to collaboratively identify and articulate issues with quality and preparation of evidence available to decision makers to improve communication and quality of evidence.
Over 80% of school quality managers engage in exchanges with colleagues working on school quality management to discuss experiences and methods related to using evidence. This includes encouraging the use of evidence, and exchanging insights about how preparation and quality of evidence could be improved. It also includes developing standards of how evidence should be used in decision-making (Figure 5.8). School quality managers set different emphases in their exchanges related to sharing experiences and methods of using evidence. SQMs in Tyrol and Lower Austria appear to emphasise improving preparation. In Burgenland and Upper Austria, exchanges focus on encouraging use of evidence. Exchanges among colleagues predominantly occur both in work meetings and in (regular) exchanges with colleagues that are more informal.
On average, only one in four school quality managers report as a topic of discussion in exchanges with colleagues the development of common standards regarding how evidence should be used in decision making (Figure 5.8). Contrary to the average, collegial exchanges among SQMs in Salzburg (80%) and Styria (50%) focus on developing common standards on how to use evidence as well as improving the quality of evidence.
Note: Follow-up question drawing on positive responses to question “Are there opportunities to exchange with colleagues specifically regarding experiences and methods of using evidence?” Responses to question “Which aims do collegial exchanges pursue?”. Multiple answers possible. N=86. Category aimed at developing common understanding of evidence (standards of how to use evidence) in light blue. ‘Other’ includes developing school and class teaching, and training/instruction in using SAND tool.
Similarly, efforts by school quality managers (SQM) to build a common understanding of how evidence should be used are relatively few. Across the various areas of decision making, around one-third of SQMs report respective efforts in substantive decision-making processes. The area of staff development, as an internal decision-making process, is reported slightly lower with about one in four school quality managers reporting respective efforts (Figure 5.9).
Note: N=104. Answers to question “Are there (initial) efforts in the education region to develop a common understanding of how evidence should be used in concrete decision-making situations?” Multiple answers possible.
Raising awareness of the merit and importance of using evidence in decision-making
More than 90% of school quality managers (97 out of 104) engage in efforts to build awareness for using evidence as a principle of good decision-making. SQMs engage school leaders, school staff responsible for quality assurance, as well as teachers in varied measures to build awareness for evidence. This includes work meetings, dedicated events and initiatives, offering advice on how to build awareness, and distributing information material. SQMs work primarily with school leaders, which is reflected in a clear focus of SQM to engage with school leaders to build awareness, in particular through work meetings. In addition, about six in ten school quality managers report to engage teaching staff in efforts to build awareness. This engagement of teachers focuses on distribution of information material. However, efforts of a quarter of SQMs include (also) events and initiatives aimed to build awareness (Figure 5.10).
Note: Follow-up question drawing on positive responses to question “As school quality manager, do you take measures to create awareness for the use of evidence as a basis for decisions?” Responses to question "Which measures do you initiate to develop awareness for evidence as basis for decision-making? For which groups do you initiate measures?” Multiple answers possible. N=97.
Overall, 46% of school leaders (2 651 of 5 745) completed the questionnaire. Some differences between school types are noticeable. Highest response rates are observed in lower secondary schools (NMS) and upper secondary general education schools (AHS). In terms of response rates, special-needs schools (ASO), lower and upper secondary vocational schools (BMHS) and part-time vocational schools of the dual system (BS) bring up the rear (Table 5.2). School size as reported by school leaders varies greatly across school types (Figure 5.11).
Note: N=2651. School type abbreviations refer to primary schools (Volksschule, VS), lower secondary schools (Neue Mittelschule, NMS), upper secondary general education schools (Allgemein bildende höhere Schule, AHS), lower and upper secondary vocational schools (Berufsbildende mittlere und höhere Schule, BMHS) pre-vocational schools (Polytechnische Schule, PTS), part-time vocational schools of the dual system (Berufsschule, BS) and special-needs schools (Sonderschule, ASO).
In the survey, most school leaders have between two and ten years of professional experience in their current position. More than a third of (38%) of school leaders have more than 10 years of professional experience. In contrast, only 15% state that they have less than two years of professional experience.
Of the 2 651 school leaders who completed the survey, 264 took the opportunity to provide additional comments in the last question. The most commonly raised concern (25%, 66 out of 264) is that school leaders often do not have enough time and resources (such as organisational staff for administrative tasks) to prepare and use evidence. Other common barriers brought forward pertain to a lack of practice-oriented preparation of evidence, and a shortage of guidance to use evidence (16%, 41 and 42 out of 264 respectively). 14% of school leaders who responded to the question, highlighted that evidence for them pertains not only to quantitative data, in particular results from standardised tests, but also to qualitative evidence. 14% wish for additional/ other offers of evidence to support school leaders, while 6% of responses highlight a lack of common understanding of evidence to hinder the evidence use. Furthermore, 13% expressed their wish for more exchange, both with colleagues but also with different levels of decision-making in the education system (Figure 5.12).
Note: Common themes among responses to question "Are there any further suggestions/ innovative examples from your professional experience of using evidence?” Multiple themes in one response possible. N=264. Category ‘other remarks’ includes issues such as too high formality/standardisation, culture and staff related issues, administrative hierarchy.
Schools developing the skills to use evidence for autonomous quality development
School leaders require the individual skills to access and make sense of evidence. This pertains to the skills to locate, appraise, and synthesise evidence and integrate it with other information and particular needs. This can include identifying relevant evidence sources for their school, such as new studies, and the cooperation with university colleges of teacher education and other research institutions. It also includes identifying and adapting useful parts of available evidence for the needs of their specific school context. Importantly, school leaders also should be able to guide and instruct teachers and other staff in their school to build the skills to use evidence effectively in their daily work. This includes school leaders’ building the respective skills as well as being able to rely on resources helping them guide and instruct the use of evidence by their staff.
School leaders emphasise the need for own initiative to acquire skills relevant to using evidence. Collegial exchange is the main source for schools leaders in this respect.
Most commonly, school leaders learn from other colleagues about how to access and make sense of evidence. Overall, eight in ten schools leaders report that they acquire skills for evidence use through exchanging with colleagues. Slightly less widespread are training courses with seven out of ten school leaders reporting respective opportunities available to them. Mentoring and coaching as well as e-learning solutions are less common. More than half of the 61 school leaders who provided additional remarks on this issue, highlighted the need for self-initiative to acquire skills for evidence use. This includes own initiative in finding relevant literature and material, finding and paying for private coaching, and cooperating with a private institution.
There are notable differences between school types. Virtually all other school types (between 88% and 97%) report at least some opportunity to acquire skills relevant to access and make sense of evidence. However, almost a third of school leaders in schools of the vocational dual system (BS) report that they do not see any possibilities to acquire such skills. Moreover, in these schools only six in ten school leaders report to exchange with colleagues to build respective skills.
Beyond being able to use evidence individually to help them develop the quality at their school, school leaders should have the skills to guide and instruct the use of evidence by school staff. Close to seven in ten school leaders report opportunities to acquire these skills through training courses. However, only a third reports so with respect to trainings tailored to the supervisory capacity of school leaders. Four in ten school leaders report that they can rely on advisory services to help them guide and instruct the use of evidence by teaching staff. This includes in particular Rückmeldemoderatoren/-innen (feedback moderators)4. Moreover, about a quarter of school leaders reports access to mentoring or coaching offers with the aim to promote skills to use evidence. Opportunities to build respective skills through e-learning solutions, such as learning platforms, are less common (17%). Vorarlberg presents a provincial outlier. Here, a third of schools leaders report not to have any suitable opportunity in this respect, more than twice than in other provinces.
Of the 56 school leaders who provided additional remarks, 60% highlight the need of self-initiative or emphasise the importance of collegial exchange to acquire the skills to guide and instruct the use of evidence of school staff. Factors arising from school leaders’ professional context are a specific concern in acquiring the skills to use evidence for decisions at the school level and instructing and guiding the use of evidence by teachers. As one school leader highlights:
“As [school leader] of a small school with a high teaching commitment, I would be very happy if seminars and other support services for [school leaders] were offered in the afternoon/ weekends. Almost all seminars […] are full-day offers. However, I cannot attend a seminar in the morning as I have to teach in the morning […].”
Organisational resources support school leaders to instruct and guide teaching staff to use evidence in their practice as well as acquire respective skills. This includes education and training requirements for teaching staff, instructions for conducting reflection discussions, and the possibility to initiate professional development measures helping teaching staff to build the skills to use evidence (Langer, Tripney and Gough, 2016[2]; Gray et al., 2012[3]). Guidelines and advice related to reflection and planning discussions with teaching staff can help school leaders to encourage teachers’ use of evidence. Staff development measures allow school leaders to refer to regulations that require teachers to pursue professional development in how to use evidence for decision-making. Other examples mentioned to help school leaders to guide and instruct the use of evidence at their school are the quality management system for general education and associated methods and tools for systematic quality assurance (Schulqualität Allgemeinbildung, SQA), the SAND tool5, and reviews of education standards (Bildungsstandards-Überprüfung, BIST-Ü). About half of school leaders report that they can rely on education and training requirements to encourage teachers to acquire the skills to use evidence. Four in ten school leaders report they can rely on guidelines for teacher reflection discussions and have professional development measures for teachers at their disposal.
Making evidence conveniently available to schools
Evidence needs to be communicated and conveniently accessible, to increase its use in school leaders’ decision-making. Evidence can be communicated directly, for example through newsletters, publications, handouts, research teasers or research summaries. Evidence providers may make evidence available also through other means of access, such as through databases or evidence repositories.
Schools make greatest use of evidence gathered at the school level and rely on quality development staff and school quality managers for support
The vast majority of school leaders produce and use school-level evidence from standardised student testing (82%) and internal evaluations (87%). This includes in particular, reviews of education standards (Bildungsstandards-Überprüfung, BIST-Ü) and standardised graduation and diploma examination results (Standardisierte Reife und Diplom Prüfung, SRDP/SRP); evidence gathered through tools and methods aligned to quality management systems (SQA and QIBB, respectively)6; and results from diagnostic tools, such as informal competence assessment (Informelle Kompetenzmessung, IKM). More than half of school leaders who completed the survey, report to engage with research evidence to update and create new knowledge. A large majority of school leaders (84%) reports that school staff prepares evidence themselves.
Many school leaders took the opportunity of the survey to provide additional comments on what kind of evidence they prepare for their school. Examples include self-prepared statistical evidence within schools, such as school-developed competency catalogues, competence checks via the school’s own learning platform, annual student and teacher surveys and statistical information of registration and graduate numbers. Many examples pertain to qualitative evidence. This includes regular and intensive discussions with the teaching staff at the school, feedback from students and parents tailored to the school, and networking with school-leader peers. Moreover, some school leaders also provided examples of successful collaborations with universities. One example is a cooperation between a school and the University of Vienna’s Centre for Teacher Education, which helps choosing annual evaluations on varying topics that the school picks to make progress in school development. At the intersection between acquiring skills using evidence effectively and accessing evidence, six in ten school leaders frequently or regularly work with quality development staff (SQA/QIBB) to support use of evidence in their school. Around 50% of school leaders report that they work frequently or regularly with school quality managers/ school supervision to support them in using evidence effectively. However, this is subject to pronounced provincial differences (Figure 5.13).
Note: N=2651. Answer to question “How often do you use the listed facilities to support you in the effective use of evidence? [school supervision]” Multiple answers possible.
Feedback moderators (Rückmeldemoderatoren/-innen, RMM) help schools to process, analyse and interpret results from reviews of education standards (Bildungsstandards-Überprüfung, BIST-Ü). Practical support of subsequent school and instructional development processes is the task of school development advisory services (Entwicklungsberatung in Schulen, EBIS), a support initiative within SQA. Contrary to the widespread use of school-internal support and the work with school quality managers, three-quarters of schools rarely or never consult external advisory services, such as EBIS and quality process managers at the federal or provincial level (Bundes-qualitätsprozessmanager/-innen, BQPM; Landes-qualitätsprozessmanager/-innen, LQPM). Similarly, 85% of schools rarely or never consult feedback moderators (RMM) to support them in the effective use of (specific) evidence (Figure 5.14). Of the 264 school leaders who provided additional comments, over 25% highlight that they have capacity issues preventing them to prepare and use evidence. Especially in primary education (Volksschule), school leaders identify teaching responsibilities and a lack of administrative staff as bottlenecks for systematic evidence use.
Note: N=2651. Answer to question “How often do you use the listed facilities to support you in the effective use of evidence? Multiple answers possible.
Schools’ impressions are mixed regarding externally provided evidence. School leaders wish for greater involvement to help tailor provided evidence to their context
While useful to some, others do not consider evidence provided by the BMBWF as prepared in a user-friendly manner. Overall, only 41% of school leaders consider the BMBWF/ education directorates as largely or always interested in providing schools with evidence that is user-friendly. However, another 40% of school leaders considers this to be the case little or not at all. This differs by school type. Over half of school leaders of upper secondary schools (AHS) and half of school leaders at schools for secondary and higher vocational education (BMHS) report positively when asked about user-friendly preparation of evidence provided by the BMBWF. In contrast, only 30% of school leaders of pre-vocational schools (PTS) report positively. On average, school leaders report evidence provided by research institutions, such as BIFIE and university colleges of teacher education, overall as more usefully prepared for their needs.
A minority of school leaders report that evidence providers appear to consider different experiences in dealing with evidence (25%) and different interests in topics (35%) to tailor evidence to the needs arising from work processes and context of school leaders. The tailoring of evidence based on language needs is infrequently observed by school leaders (9%). Moreover, close to 15% of the 264 additional comments highlight a lack of user-friendliness or practice-orientation. As one school leader illustrates,
“Evidence should be more user-friendly. I understand the intention to present surveys as scientifically as possible (e.g. language use), but this makes them difficult to use. If one has to read through such evidence (explanations) several times to understand their purpose and sense, one will not fall back on such evidence knowledge in the future.”
A number of school leaders identify concrete measures to help them make better use of evidence in their specific context. This includes tools for evidence-based work through SQA; evaluation tools adaptable to specific projects or locations (tools previously used may not be available anymore (e.g. Tevalo) because of new data privacy regulations (DSGVO); evidence repository from which schools could autonomously choose what to use and through which (primary) schools can access site-specific evidence points.
Consulting schools and school leaders in making evidence available provides an opportunity for evidence providers to shape communication techniques, modes of access and presentation of evidence in a manner that is most relevant for schools’ needs. For instance, school leaders may be invited to give feedback on the content and communication of certain types of evidence, or to contribute content by sharing practices from their local context. On average, over a third of school leaders report that school leaders are not involved in the provision and communication of evidence (28% feel unable to assess this). Where school leaders are consulted, they report this to be the case mainly within evaluations (23%). Other forms are very rare, such as consultation on relevant topics, contributing to content, improving communication channels or type of preparation.
There are notable differences across school types in how involved school leaders feel specifically regarding consultation on relevant topics of external evidence. Almost a third of schools for secondary and higher vocational education (BMHS) (27%) report that school leaders are consulted about the topics relevant to them, but only 8% of school leaders at pre-vocational schools (PTS) report so. In comparison, 16-20% of school leaders from all other school types report so.
Organisational processes to encourage evidence use in schools
Organisational processes and structures can encourage schools to use evidence and reduce respective barriers. For schools, this includes making decisions and decision-making processes plain, such as exchanging with external stakeholders and school partners. It also includes inviting diverse perspectives in different areas of decision making, for example through processes of shared leadership. Moreover, knowledge management systems help contextualise evidence to develop actionable knowledge. Such systems include online platforms and databases, as well as offline collections, for instance in the form of printed handbooks, organisational documents, or teaching materials.
Schools focus on internal exchange over involvement with external partners to increase clarity around decisions and decision-making processes.
Schools’ efforts to make transparent how decisions were reached emphasise the exchange among teachers. Overall, nine out of ten school leaders use exchange formats with teachers to clarify decisions and processes. The observation is uniform across school types and school sizes. Around four in ten school leaders engage in exchange externally with school partners to increase clarity around decisions and their decisions-making processes. This is relatively more popular among very large schools (with more than 20 classes) and among upper secondary vocational and general education schools (Allgemeine höhere Schule, AHS; Berufsbildende mittlere und höhere Schule, BMHS).
Overall, a quarter of schools makes use of (publicly available) documents to make decisions and decision making processes transparent and comprehensible. Vocational schools put relatively more emphasis on this particular measure than general education schools (Figure 5.15). A number of school leaders took the opportunity to share examples of how their school seeks to increase clarity around school decisions and processes. This includes inter-school student councils and approaches of democratic leadership in the school.
Note: N=2651. Answers to question “Are there efforts to make decision-making processes at your school clear and comprehensible?” Multiple answers possible.
Schools involve diverse perspectives mainly to help develop classroom teaching but vocational and special-needs schools also emphasise staff development
Around three-quarters of school leaders often or regularly involve different perspectives in decisions. This includes perspectives of parents, students, teachers, as well as external school partners. Overall, efforts to include diverse perspectives in developing classroom teaching are most common (73%). Decisions around organisational development (66%) are less common. Decisions regarding staff development (62%) are the least frequent.
Larger schools have a slight tendency to involve diverse perspectives more often. In particular, very large schools emphasise involving diverse perspectives in organisational development over developing classroom teaching. Accordingly, upper secondary general education schools (AHS) (75% have over 20 classes) and part-time vocational schools (BS) (90% have over 20 classes) frequently involve diverse perspectives in organisational development. Notably, vocational schools (BS, BMHS) and special-needs schools (ASO), emphasise involving different perspectives in staff development compared to other school types (Figure 5.16).
N=2651. Answers to question “In which processes at your school are different perspectives (e.g. parents, teachers, students) involved in decision-making?” Multiple answers possible.
Schools are not systematically aware of knowledge management systems
Knowledge management systems link knowledge across and within organisations for access by decision makers where needed. By doing so, they help contextualise evidence. This can include general information management, a collection of organisational procedures, or other knowledge relevant to the functioning of the organisation and the decision-making processes at hand. Today, knowledge management systems are often computer-based but likewise include printed material. Individual school leaders may have clear ideas of the information to which they wish access in form of a knowledge management system, making it worthwhile to consult them in making available respective systems.
Two-thirds of school leaders report that school staff can rely on internal knowledge management systems to organise and share knowledge within the individual school. This is particularly the case in vocational schools (BMHS, 72%; BS, 75%), but less the case in pre-vocational schools (PTS) (53%). Knowledge management systems that allow organising and linking knowledge across the Education Region varies across provinces. In Burgenland and Tyrol, 58% and 44% of schools report to use regional knowledge management systems, while only a quarter of schools in Vienna, Vorarlberg, Styria and Carinthia report to do so. To make best use of knowledge management systems, respective systems should be available to all decision makers and all decision makers should be aware of available systems. However, school leaders appear to lack a common understanding of knowledge management systems. Among others, school leaders identify school supervision, training and support offers as knowledge management systems.
Schools’ exchange with evidence providers and exchange among school leaders
An important aspect to promote the systematic use of evidence in schools is to build trusted relationships and facilitate social influence through interaction with those who are engaged in evidence production and experienced in using evidence. Accordingly, central to fostering the use of evidence is facilitating and encouraging the exchange among schools and schools’ exchange with providers of evidence. In Austria, providers of evidence are for example the federal ministry, education directorates, the national institute for education research (Bundesinstitut für Bildungsforschung, Innovation & Entwicklung, BIFIE), university colleges of teacher education (Pedagogische Hochschulen, PH) and other organisations such as the national statistics bureau Statistik Austria.
School leaders exchange with evidence providers but organisational support may be lacking
Overall, about four in ten school leaders report they engage in a structured exchange with evidence providers at least once a year. There are notable differences between provinces and school types. Over 50% of school leaders in secondary general education (AHS, NMS) and vocational schooling (BMHS) engage in such exchanges. Special-needs schools (ASO), pre-vocational schools (PTS) and part-time vocational schooling of the dual (apprenticeship-combined) system (BS) engage less frequently. Around seven in ten schools of these school types do not regularly engage in structured exchange with evidence providers. About eight in ten schools that exchange with evidence providers report that this is at least most of the time carried out to improve the preparation or quality of evidence. School leaders report less frequently (50%) adequate organisational support of such exchanges, for instance, through time or human resources.
Note: N=2651. Answers to question “Is there a structured exchange between the providers of evidence (e.g. BIFIE, university colleges of teacher education) and you as a decision maker at the school?”
Schools in Burgenland stand out as exchanging with evidence providers particularly frequently. Here, three in ten Schools exchange with evidence providers about once per semester (Figure 5.17). Schools in Burgenland and Salzburg further report greater organisational support to carry out these exchanges. Here, about two-thirds of school leaders report organisational resources to be most of the time or always available to support them, compared to half of school leaders overall (Figure 5.18).
Note: Follow-up question drawing on positive responses to question “Is there a structured exchange between the providers of evidence (e.g. BIFIE, PH) and you as a decision maker at the school?” Answer to question “The exchange is supported organisationally (e.g. in terms of time/staff resources)”. N=1064.
School leaders, who provided additional comments on their exchange with evidence providers highlighted that they exchange as necessary, but not regularly. According to these comments, typical occasions for such an exchange on demand are school supervision, performance review and objective-setting discussion (Bilanz- und Zielvereinbarungsgespräche, BZG), within informal competence checks (Informelle Kompetenzmessung, IKM) or, formerly, education standards review (Bildungsstandards-Überprüfung, BIST-Ü), within the project securing basic competencies (Grundkompetenzen absichern, GRUKO) and in school-based professional development (Schulinterne Fort- und Weiterbildung, SCHILF). Performance review and objective-setting discussions (BZG) are part of SQA and QIBB. They regularly take place between managers of adjacent levels of governance and are based on the current development plan of the subordinate level.
School leaders frequently exchange with colleagues informally and in work meetings regarding methods and experiences of using evidence
Overall, nine in ten school leaders exchange with colleagues regarding methods and experiences related to using evidence in schools’ quality development. Discussion of evidence is carried out predominantly informal and frequently takes places within work meetings, rather than institutionalised, for instance through dedicated network meetings. As other opportunities to exchange with colleagues, school leaders further identify the (discontinued) Leadership Academy (LEA)7, SQA, and events within school-based professional development (SCHILF).
There is high variation across provinces and across school types in how systematically school leaders discuss evidence in collegial exchange. School leaders in Burgenland, Salzburg, and Styria report systemically to exchange with each other about using evidence in their practice, with only 2-7% of school leaders reporting no exchange with colleagues (Figure 5.19). Within these exchanges, they discuss research findings, reviews of education standards (Bildungsstandards-Überprüfung, BIST-Ü), or standardised students test results (SDRP) more systematically than their colleagues in other provinces. There are notable differences across different school types. Around 20% of school leaders at vocational schools of the dual (apprenticeship-combined) system (Berufsschule, BS) never engage with colleagues to exchange about methods and experiences using evidence. School leaders of pre-vocational schools report similarly little exchange with colleagues. In contrast, 10% school leaders in primary and secondary general education and vocational/technical schools (AHS, BMHS, NMS and VS) report so.
Note: N=2651. Answer to question “Are there opportunities specifically to exchange with colleagues about experiences and methods of using evidence?” Multiple answers possible.
Raising awareness of evidence and developing a common understanding how to use it
To promote the use of evidence for decision making in schools, school leaders and teaching staff need to be mindful of evidence use as a principle of good decision making. Schools need to adjust to greater autonomy and potentially new areas of decision making. Central to using evidence for new or changed responsibilities depends centrally on developing a shared understanding of what constitutes fit-for-purpose evidence, and how evidence should be used best in specific situations of decision making, such as decisions about developing classroom teaching or school development. Important elements to raise awareness include events and advisory services, information and training material. School leaders additionally mention informal exchanges, work meetings, printed material and events related to SQA and pedagogical conferences as means to develop awareness for the merits and importance of using evidence in day-to-day decisions as well as for strategic quality development.
Half of the school leaders who completed the survey, report the use of information material to raise awareness of the use of evidence as a basis for decision making at the school. One-fifth of schools report such efforts through consulting or events offered by education directorates and the BMBWF. Across school types, 44% of pre-vocational schools report that there are no efforts at the school to raise awareness. In contrast, the share of school leaders without any means at their disposal to raise respective awareness amounts to 25% to 35% among other school types. More than half of school leaders report efforts at their school to develop a common understanding of how to use evidence in concrete decision-making situations within school development and development of classroom teaching. A minority of schools report such efforts in staff development (20%) and organisational processes (35%). As one school leader illustrates,
"All measures concerning evidence use are currently only carried out by me as the school leader. There is probably still a long way to go to create an understanding of evidence use, so that it becomes practically apparent that very valuable development processes can arise from it."
Close to nine in ten school leaders report efforts to create a common understanding of what constitutes fit-for-purpose evidence for specific decision-making challenges. A notable difference is Vorarlberg, where a quarter of school leaders report no such efforts, compared to under 15% of school leaders across other provinces. Efforts to build a common understanding of what constitutes fit-for-purpose evidence as well as how evidence should be used in specific decision-making areas include discussions of standardised graduation and diploma examination results (Standardisierte Reife und Diplom Prüfung, SRDP/SRP), school-based professional development (Schulinterne Fort- und Weiterbildung, SCHILF), meetings, conferences and joint discussions on the review of education standards (Bildungsstandards-Überprüfung, BIST-Ü).
References
[1] BMBWF (2019), Steuerung des Schulsystems in Österreich: Weissbuch [Governance of the Education System in Austria: White Paper], Bundesministerium für Bildung, Wissenschaft und Forschung, Abt. III/3, http://www.bmbwf.gv.at.
[6] BMBWF (2017), Zahlenspiegel 2017: Statistiken im Bereich Schule und Erwachsenenbildung in Österreich [Statistical data 2017: Statistics in the area of school and adult education in Austria], Bundesministerium für Bildung Wissenschaft und Forschung (BMBWF), Vienna, https://bildung.bmbwf.gv.at/schulen/bw/ueberblick/grunddaten.html.
[4] Burns, T., F. Köster and M. Fuster (2016), Education Governance in Action: Lessons from Case Studies, Educational Research and Innovation, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264262829-en.
[3] Gray, M. et al. (2012), “Implementing evidence-based practice”, Research on Social Work Practice, Vol. 23/2, pp. 157-166, https://doi.org/10.1177/1049731512467072.
[2] Langer, L., J. Tripney and D. Gough (2016), The Science of Using Science - Researching the Use of Research Evidence in Decision-Making, EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London, http://eppi.ioe.ac.uk/cms/Default. (accessed on 15 January 2018).
[5] Shewbridge, C. and F. Köster (2019), Strategic Education Governance - Project Plan and Organisational Framework, http://www.oecd.org/education/ceri/SEG-Project-Plan-org-framework.pdf.
Notes
Notes
← 1. The Federal Institute for Educational Research, Innovation and Development (Bundesinstitut für Bildungsforschung, Innovation und Entwicklung des österreichischen Schulwesens, BIFIE) has since been restructured to form the Institute of the Federal Government for Quality Assurance in the Austrian School System (Institut des Bundes für Qualitätssicherung im österreichischen Schulwesen, IQS). The change took effect on 1 July 2020.
← 2. In addition, results of external school evaluation (when implemented) are set to be essential evidence that will form a basis for individual schools’ quality development.
← 3. In Vorarlberg, two School Quality Managers feel they cannot assess whether they are adequately involved. This may be related to the little time in this position. In Vorarlberg, 50% of the eight SQMs who have completed the survey have less than 2 years of experience in this role, compared to ca. 12% across all provinces.
← 4. School leaders can request feedback moderators (Rückmeldemoderatoren/-innen) via the university colleges of teacher education. Feedback moderators advise schools on the analysis and interpretation of the results of the education-standard reviews and support them in processing the results. School leaders and teaching staff initiate and see through the subsequent school and instructional development processes.
← 5. The Federal Institute for Educational Research, Innovation and Development (BIFIE) offers the SAND as tool for school development. The tool allows analysing and comparing data individual and across school locations.
← 6. SQA and QIBB are the respective quality management systems for general education (Schulqualität Allgemeinbildung, SQA) and vocational education and training (QualitätsInitiative BerufsBildung, QIBB). Both systems include tools and methods for systematic quality assurance. With the 2017 governance reform they are superseded by a quality management system common to all schools.
← 7. The Leadership Academy (LEA) network comprised decision makers across levels and areas of the Austrian education system who participated in seminars over the course of one year. LEA seminars were carried out between 2004 and 2018.
