Annex A. Stakeholder engagement

As part of the project, the OECD team organised several stakeholder engagement activities to ensure that the views of key higher education stakeholder organisations and institutions in Hungary were taken into account for the development of the analysis, recommendations and policy options included in this report. An overview of the different stakeholder engagement activities organised as part of the project is presented in Table A.1. The research tools and stakeholders to be engaged in each activity were developed in close consultation with the Hungarian Ministry of Culture and Innovation (KIM) and the Hungarian Accreditation Committee (MAB). They are presented in the remainder of this Annex.

Table A.1. Overview of stakeholder engagement activities
Stakeholder engagement activity	Timing
Individual and institutional stakeholder interviews
Stakeholder interviews (online)	3-15 February 2022
Virtual site visits to higher education institutions (online)	16-25 March 2022
Stakeholder consultation on “Policy Options for Hungary to Assure the Quality of Digital Higher Education” (online)	11-25 November 2022
Peer learning events and activities
Launch event “Supporting Quality Digital Higher Education in Hungary: Findings to Date and What’s Next” (online)	18 November 2021
National roundtable “Best Practices and New Policies for Quality Digital Higher Education in Hungary” (online)	31 May 2022
International conference “International Quality Assurance Standards, Practices and Supports for Digital Higher Education” (online)	14 June 2022
National roundtable “Key Lessons Hungary Might Learn from Other Systems” (Budapest, Hungary)	4 October 2022
Final conference “Ensuring Quality Digital Higher Education in Hungary: The Path to Modernisation” (Györ, Hungary)	29 March 2023

Individual and institutional stakeholder interviews

Stakeholder interviews (online)

In February 2022, the OECD review team carried out interviews with a sample of higher education institutions and stakeholder organisations to gain an in-depth understanding of the key challenges and potential approaches to monitoring and improving the quality of the digital education offered by higher education institutions (HEIs) in Hungary. Stakeholders were also invited to share their views on how they think the existing QA standards, practices and supports could or should be adapted in order to be “fit for purpose” for digital teaching and learning in higher education. They were also invited to share examples of institutional best practice for the QA of digital higher education which they might be familiar with in Hungary.

Table A.2 provides an overview of the higher education stakeholder organisations and institutions interviewed by the OECD review team in February 2022. The list of stakeholder organisations interviewed is based on advice provided by KIM and MAB. In total, the OECD review team interviewed 19 stakeholders across all these organisations.

Table A.2. Overview of higher education stakeholder organisations interviewed by the OECD review team
Higher education stakeholder organisations interviewed by the OECD review team in February 2022
Ministry of Culture and Innovation (KIM) Educational Authority (OH) Hungarian Accreditation Committee (MAB) Tempus Public Foundation (TPF) Hungarian Rectors’ Conference (MRK) National Doctoral Council European Institute for Innovation and Technology (EIT) Association of Hungarian PhD and DLA Candidates Digital Success Nonprofit Ltd., Digital Higher Education Competence Centre (DHECC) The University of Debrecen

The interviews were structured along the following four main areas of inquiry:

• Key challenges and priorities facing higher education training and digital provision in Hungary today

• Policies and practices carried out by stakeholder organisations interviewed

• Stakeholder views on the quality assurance standards and procedures currently used by MAB and HEIs for the QA of digital study programmes

• Recommendation which the OECD might usefully develop to help improve the quality of digital higher education in Hungary

A. Key challenges and priorities for enhancing the quality of digital higher education in Hungary

In this part, we would like to hear your views on the key challenges facing higher education training and digital provision in Hungary today and, more specifically, in relation to the quality of teaching and learning offered in fully online and/or hybrid courses and programmes.

1. 1. What do you think are the key challenges facing higher education training and digital provision in Hungary today?

   1. a. Please highlight challenges related to the higher education system in general, as well as the higher education training and digital provision system more specifically, elaborating on any contextual factors and/or factors that may help explain these challenges.

   2. b. Please highlight challenges related to the quality of teaching and learning more specifically, both in in-person and in online/hybrid courses and programmes. Reflect in particular on the digital readiness of institutions, instructors and students, and the supports and incentives that are currently available to them in order to ensure high-quality teaching and learning.

   3. c. Which of these do you see as being the most urgent priority areas to be addressed?

2. 2. What do you think is the role or potential of digitalisation in overcoming (some of) these challenges and, more specifically, in improving the quality of their teaching and learning currently offered by HEIs in Hungary?

   1. a. What do you think might be some of the key benefits of digitalisation for tackling systemic challenges and improving the quality of higher education in Hungary?

   2. b. What do you think might be some of the main risks of digitalisation for the future development of Hungary’s higher education system and the quality of teaching and learning?

   3. c. What do you think is the role or importance of quality assurance in overcoming (some of) these challenges, and more specifically, in improving the quality of the teaching and learning currently offered by HEIs in Hungary?

   4. d. Key challenges and priorities for enhancing the quality of digital higher education in Hungary

   5. e. Policies and practices for supporting the quality of digital higher education

   6. f. Standards and practices for the quality assurance of digital higher education

   7. g. Recommendations

      1. i. The role of external quality assurance (as carried out by MAB) to ensure quality digital higher education

      2. ii. The role of HEIs for monitoring and supporting the quality of their digital provision

      3. iii. The role played by the higher education policy field and other actors in the system to support institutions to improve the quality of their digital provision

B. Policies and practices for supporting the quality of digital higher education

1. a. Please consider the following areas when describing your organisation’s activities

[Title of policy/practice/initiative]

What challenges do you seek to address?

[Please describe]

Which policies, activities or projects are you carrying out to tackle these challenges? What are the (expected) outcomes?

[Please describe]

What are some of the main challenges you are experiencing in relation to digital education?

[Please describe]

1. 1. Please describe relevant policies, practices, or actions of other organisations which you are aware of (e.g., international, European, national, institutional) that you think have been important or could be helpful to improve the quality of digital higher education in Hungary going forward.

   1. a. Please consider the following areas when describing the organisation’s activities

[Title of organisation/policy/practice/initiative for Hungary]

What challenges do they seek to address?

[Please describe]

Which policies, activities or projects are they carrying out to tackle these challenges? What are the (expected) outcomes?

[Please describe]

What are some of the main challenges they are experiencing in relation to digital education?

[Please describe]

1. 2. What do you think are missing actions, policies or initiatives that could be implemented, supported, or strengthened to improve the quality of digital higher education in Hungary?

   1. a. What areas are currently not being addressed and you think are important?

   2. b. How do you think your organisation could contribute to addressing these gaps?

   3. c. How do you think other organisations (e.g., government, Hungarian Accreditation Committee, sector organisations, representative bodies such as Hungarian Rectors’ Conference, the National Union of Students in Hungary, the Association of Hungarian PhD, and DLA Students HEIs themselves, etc.) could contribute to addressing these gaps?

C. Standards and practices for the quality assurance of digital higher education

In this part, we would like to hear your views on how the quality assurance of higher education in Hungary takes place today, and what you see as being helpful/unhelpful processes or gaps in the standards and practices currently implemented by (1) the policy field, (2) the Hungarian Accreditation Committee (MAB) and (3) HEIs themselves for the monitoring and evaluation of the quality of digital higher education. How could they be modified in order to better assess and support the quality of digital higher education?

1. 1. How do you think the standards and procedures used by MAB could be revised in order to make them “fit for purpose” to monitor and evaluate the quality of digital higher education?

   1. a. What do you think would be helpful (minimum) standards or criteria for MAB to adopt to monitor the quality of HEIs’ digital provision? How relevant/useful are the current standards?

   2. b. What do you think would be relevant or helpful procedures to monitor/evaluate these minimum standards? What data would be needed? How relevant/useful are MAB’s current procedures?

   3. c. What are the main challenges experienced by MAB for carrying out assessments of the quality of digital higher education? What additional resources or supports (e.g., funding, human resources, expertise, etc.) do you think they need to effectively carry out quality assessments?

2. 2. How do you think the standards and procedures used by HEIs could be revised in order to make them “fit for purpose” to monitor and evaluate the quality of their digital provision?

   1. a. What do you think are key areas for HEIs to monitor/evaluate to ensure minimum standards of quality are being met? (e.g. quality of teachers, student satisfaction, graduate outcomes, …?)

   2. b. What do you think would be useful procedures for HEIs to monitor/evaluate the quality of their courses, programmes, or teachers? (e.g. surveys, administrative data, self-assessments, …?)

   3. c. What are the main challenges experienced by HEIs for carrying out assessments of the quality of their online provision? What additional resources or supports do you think they would need in order to put in place effective quality assessments and enhancement practices?

3. 3. Going beyond external and internal quality assurance, what other supports and incentives do you feel are important to put in place to support HEIs to improve the quality of digital higher education?

4. 4. How do you think the higher education policy field (Ministry) can support the common framework for quality assurance in Hungarian higher education?

5. 5. How can the current external and internal QA policies be improved, what measures are missing in your view?

D. Recommendations

1. 1. What do you think would be helpful lines of recommendation for the OECD to develop as part of this project to help improve the quality of digital higher education in Hungary?

2. 2. Are there any other questions or initiatives which have not been covered in the interview which you would like to draw the OECD’s attention to?

Virtual site visits to higher education institutions (online)

In March 2022, the OECD review team carried out virtual site visits to six HEIs in Hungary, as part of which a sample of instructors, students and administrators was interviewed in each institution. The purpose of these visits was the following:

• Understand in greater depth the views and experiences of HEIs regarding: (1) the quality of digital provision in its online and hybrid forms as it is currently being offered by HEIs in Hungary; (2) key challenges or barriers experienced by current government regulation, external quality assurance standards and procedures for effectively managing the quality of their digital provision; and (3) supports or incentives at institutional, sectoral or national level deemed relevant to support HEIs to more effectively manage and continuously improve the quality of their digital provision;

• Collect feedback on the key questions emerging from the analysis to date;

• Identify examples of best practice and gaps across institutions in Hungary on how the quality of digital provision can be effectively monitored and supported;

• Identify gaps across institutions in Hungary in terms of how they are currently monitoring and/or supporting the quality of their digital provision (in order to fill these gaps with relevant and targeted examples of international practice); and

• Promote practice sharing and peer learning among HEI administrators, students, and instructors on the quality of their digital provision, and how it can or should be further improved.

Table A.3 presents an overview of the six HEIs that participated in the virtual site visit programme organised by the OECD review team in March 2022. The selected sample of HEIs is based on advice provided by KIM and MAB. Table A.4 presents the total number of higher education stakeholders interviewed by the OECD review team as part of the virtual site visits.

Table A.3. Participating institutions in the virtual site visits organised by the OECD review team
Institution	Date of site visit	Status	Classification	Number of Students (2021-2022)	Location	Training Profile
Eötvös Loránd University (ELTE)	16 March 2022	Public	University	31 775	Budapest	Comprehensive, research-focused
Károli Gáspár University of the Reformed Church (KRE)	21 March 2022	Church-owned	University	8 049	Budapest	Comprehensive, research-focused
University of Debrecen (DE)	22 March 2022	Foundation	University	29 954	Debrecen	Comprehensive, research-focused
Budapest Metropolitan University (METU)	23 March 2022	Private	University of Applied Sciences	6 378	Budapest	Specialised, applied sciences
Tomori Pál College (TPF)	24 March 2022	Private	College	443	Budapest	Specialised, applied sciences
University of Szeged (SZTE)	25 March 2022	Foundation	University	20 813	Szeged	Comprehensive, research-focused

Table A.4. Number and type of stakeholders participating in virtual site visits
Higher education institution	Administrators	Instructors	Students	Total
Eötvös Loránd University (ELTE)	12	5	2	19
Károli Gáspár University of the Reformed Church (KRE)	13	6	2	21
University of Debrecen (DE)	7	5	6	18
Budapest Metropolitan University (METU)	12	5	4	21
Tomori Pál College (TPF)	5	4	5	14
University of Szeged (SZTE)	8	3	3	14
Total	57	28	22	107

Table A.5 presents a template with sessions which HEIs were asked to prepare for the OECD review team’s virtual visit to their institution. In function of each HEI’s individual context, policies, and practices, some of the sessions proposed below were removed or replaced by other sessions or focus on areas deemed to be most relevant by the institution and OECD team.

Table A.5. Agenda template for virtual site visits to HEIs
Timing	Topic
Session 1: Institutional strategy, vision, and mission for digitalisation
09:00 – 09:40	In this session, the OECD would like the institution to give a short presentation of its vision and strategy for digitalisation, as well as highlight some of the key challenges observed in relation to the quality of teaching and learning offered in digital settings. The institution should also present the actions it is (or is planning to) undertake in order to improve the quality of its digital provision. Short presentation prepared and delivered by institution (10-15 minutes) Discussion with OECD (30 minutes) Possible stakeholders to attend: Rector, staff responsible for HEI’s overall or specific digital strategy, and other relevant staff responsible for institutional or digital strategy.
09:40 – 09:45	Break
Session 2: Monitoring and evaluating the quality of teaching and learning in online courses
09:45 – 10:25	In this session, the OECD would like the office or administrator(s) responsible for monitoring and evaluating the quality of teaching and learning in the institution, and for reporting to MAB, to give a key presentation of its internal QA processes. They should highlight key challenges/barriers observed in relation to (1) the quality of teaching and learning offered in digital settings in the institution and (2) the standards and procedures to be followed by MAB for the quality assessment of digital provision. Short presentation prepared and delivered by institution (10-15 minutes) Discussion with OECD (30 minutes) Possible stakeholders to attend: Staff members of the office responsible for liaison with MAB and carrying out institutional evaluation and monitoring of the quality of teaching and learning (e.g., carrying out or analysing student, staff or course satisfaction surveys).
10:25 – 10:30	Break
Session 3: Technical, pedagogical and/or other supports provided by the institution
10:30 – 11:10	In this session, the OECD would like the institution to give a short presentation of any more specific technical, pedagogical, or other supports provided to instructors and learners to ensure quality online teaching and learning, followed by a discussion on what more could or should be done in this area, and what supports they would need. Short presentation prepared and delivered by institution (10-15 minutes) Discussion with OECD (30 minutes) Possible stakeholders to attend: Staff members of HEI’s teaching and learning centre (if it exists), of centres or staff providing pedagogical and/or technical support to staff, or academic, technical and/or socio-emotional support to students.
11:10 – 11:15	Break
Session 4: The instructors’ perspective
11:15 – 11:55	In this session, the OECD would like to meet with a sample of 3-5 instructors with (1) more and (2) less experience of teaching in digital settings. The goal is to find out more about the types of technical and/or pedagogical supports needed by instructors to support the development of quality teaching and learning in online courses. Possible stakeholders to attend: A sample of 3-5 instructors, ideally from a variety of disciplines and with experience teaching in online settings or developing fully online or hybrid courses. Representative of staff association (if it exists) could also participate in this session.
11:55 – 12:00	Break
Session 5: The students’ perspective
12:00 – 12:40	In this session, the OECD would like to speak with a sample of 3-5 students to hear what they see as being the key challenges to attending and successfully completing online courses, and which supports they feel would be useful to help them overcome some of these challenges. Possible stakeholders to attend: A sample of 3-5 students, as well as a representative of the university’s student office or association (if it exists).
12:40 – 12:45	End of the site visit

Stakeholder consultation “Policy Options for Hungary to Assure the Quality of Digital Higher Education” (online)

Between 11 and 25 November 2022, the OECD review team invited a carefully selected number of higher education stakeholders (both international experts and experts from Hungary) to anonymously review and comment on an internal Consultation Document “Policy Options for Hungary to Assure the Quality of Digital Higher Education”, developed by the OECD review team. The feedback received from stakeholders on this document has been used to refine the findings and recommendations included in this report. The instructions shared with the stakeholders are presented in Box A.1 below.

Box A.1. Instructions for stakeholder consultation

Invited stakeholders were given the following instructions for their review of the internal Consultation Document “Policy Options for Ensuring Quality Digital Higher Education in Hungary”:

Purpose and focus of your review

The document shared with you presents draft final recommendations and policy options for Hungary to assure the quality of digital higher education and should not be shared with third parties. They are based on the presentations and feedback received from stakeholders during a workshop (see “Lectures for 2022” on MAB’s website) organised on 4 October 2022 in Budapest, Hungary, as well as various stakeholder consultation activities, national and international desk research carried out as part of the project over the past year. The purpose of your review is to ensure that the OECD’s recommendations meet the challenges and needs of the higher education sector and will help to build the capacity of HEIs in Hungary to effectively manage the quality of fully online and hybrid study programmes. You are invited to read the document by highlighting:

• Any factual inaccuracies or errors that you believe need to be corrected by the OECD;

• Any important additional (quality) considerations which you believe are important to reflect; and

• Which recommendations are the most urgent/important for Hungary to implement or support?

Timing for submitting your review

Please return the document with your feedback (in track changes and comments) no later than Friday 25 November 2022 to François Staring ([email protected]). Please note that any feedback or comments you submit will be treated fully confidentially by the OECD team and not be shared with others.

Peer learning events and activities

Launch event “Supporting Quality Digital Higher Education in Hungary: Key Findings and What’s Next?”

On 18 November 2022, the OECD review team supported KIM to organise an online webinar to conclude the project “Supporting the Digital Transformation of Higher Education in Hungary”, which took place between July 2020 and October 2021, and introduce the new project “Ensuring quality digital higher education in Hungary”. The event aimed to:

• Share the findings and recommendations of the digital transformation project;

• Outline the objectives and methods of the quality assurance for digital higher education project;

• Present the perspectives and current efforts of higher education stakeholders to support the digitalisation of Hungarian higher education.

Table A.6 presents the agenda for the event, which was organised via Zoom videoconference.

Table A.6. Agenda launch event (18 November 2022)
Timing	Topic and speakers
9:00 – 9:15	Welcome and purpose of meeting Dr. Balázs Hankó, Deputy State Secretary for Higher Education, Hungarian Ministry for Innovation and Technology (MIT) Ágota Kovács, Policy Officer, Directorate-General for Structural Reform Support, European Commission Paulo Santiago, Head of the Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
9:15 – 9:55 9:15 – 9:30 9:30 – 9:55	Session 1: International projects on digital higher education in Hungary Presentations Supporting the digital transformation of higher education in Hungary: findings and recommendations Patricia Mangeol, Analyst, Higher Education Policy, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD Ensuring quality digital higher education in Hungary: objectives, key milestones and methods François Staring, Analyst, Higher Education Policy, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD Discussion and Q&A Moderation: Thomas Weko, Head of the Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
9:55 – 10:00	Break
10:00 – 10:55 10:00 – 10:40 10:40 – 10:55	Session 2: National initiatives on digital higher education in Hungary Presentations Supporting the digital transformation of higher education in Hungary Dr. János Setényi, expert on behalf of the Digital Higher Education Competence Centre, Digital Success Nonprofit Ltd. Viktória Lilla Pató, Vice-president of international affairs, Association of Hungarian PhD and DLA Candidates Ensuring the quality of digital higher education Péter Levente Lakatos, Deputy Director, Secretariat, Hungarian Accreditation Committee Dr. Péter Ekler, Associate Professor, Department of Automation and Applied Informatics, Budapest University of Technology and Economics Discussion and Q&A Moderation: Paulo Santiago, Head of the Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
10:55 – 11:00	Closing Dr. Laura Sinóros-Szabó, Head of Department for Strategy and Institutional Development in Higher Education, Hungarian Ministry for Innovation and Technology Thomas Weko, Head of the Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD

National roundtable “Best Practices and New Policies for Digital Higher Education in Hungary” (online)

On 31 May 2022, the OECD team supported KIM to organise a public roundtable event on “Best Practices and New Policies for Digital Higher Education in Hungary”. The purpose of this event was to provide HEIs and key stakeholder organisations in Hungary with an opportunity to share and discuss:

• best practices of higher education institutions with respect to managing and supporting quality digital higher education; and

• how public policies and external quality assurance can support the offer of quality digital higher education by Hungarian higher education institutions.

Table A.7 presents the agenda for the national roundtable event, which was organised via Zoom videoconference and attended by 48 stakeholders from 21 different organisations.

Table A.7. Agenda national roundtable event (31 May 2022)
Timing	Topic and speakers
09:30 – 09:40	Welcome and introduction Dr. Balázs Hankó, State Secretary for Innovation and Higher Education, Hungarian Ministry for Culture and Innovation (KIM) Prof Dr Valéria Csépe, President, Hungarian Accreditation Committee (MAB) Ms Ágota Kovács, Policy Officer, European Commission’s Directorate-General for Structural Reform Support Mr Paulo Santiago, Head of the Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
09:40 – 09:45	Project overview and purpose of the event Mr François Staring, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
Session 1 – PLENARY DISCUSSIONS: What are higher education institutions in Hungary doing to ensure the quality of digital higher education?
09:45 – 10:15	Presentations: Institutional vision, mission and measuring digitalisation Representatives from two HEIs in Hungary will share their vision, mission and approach to measuring the quality of digital higher education as part of their overall internal quality assurance system. Dr István Vilmos Kovács, Vice-Rector for International Academic Relations, E-Learning Strategy and Internal Quality Assurance at the Budapest Metropolitan University of Applied Sciences Dr Elek Bartha, Vice-Rector for Educational Affairs, Internal Quality Assurance at Debrecen University Moderation: Mr François Staring, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
10:15 – 10:45	Presentations: Internal quality assurance and support for digital teaching and learning Representatives from two HEIs in Hungary will share their approach to supporting students and teachers for digital higher education as part of internal quality assurance strategies. Dr Ida Dringó-Horváth, Associate Professor and Head of ICT Research Centre, Károli Gáspár University of the Reformed Church (KRE), ICT, Teaching and Learning Centres in Hungary and Activities of KRE’s ICT Research Centre Dr Péter Balkany, Senior Lecturer, Corvinus University, Internal Quality Assurance for Digital Higher Education at Corvinus University of Budapest and Activities of the Centre for Educational Quality Enhancement and Methodology Moderation: Mr François Staring, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
10:45 – 11:15	Discussion: External evaluation and accreditation of digital higher education Representatives from three HEIs in Hungary will reflect on how MAB could revise or adapt its current standards and procedures for institutional and programme accreditation to support institutions in expanding and improving the quality of their digital education offer. Mr Péter Szakál, Director of Academic Affairs, University of Szeged Dr Péter Baldy, Deputy-Director of Academic Affairs, Eötvös Loránd University Dr Levente Kiss, Director of Centre for Educational Development, Methodology and Organisation, Semmelweis University Moderation: Mr Thomas Weko, Head of the Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
11:15 – 11:30	Break
Session 2 – BREAKOUT GROUPS: What should higher education institutions and MAB do to ensure quality digital higher education?
11:30 – 12:00	Purpose of the breakout discussions Participants are invited to join breakout groups, to discuss the questions put to them, and to prepare an answer to be shared with the plenary session. Groups 1 and 2: What are 3-4 ways that the accreditation standards and methods implemented by the Hungarian Accreditation Committee (MAB) may be revised to support the wider offer of high-quality digital study programmes? Groups 3, 4, and 5: What are 4-5 useful indicators of quality that higher education institutions should use to monitor and evaluate the quality of their digital study programmes?
	Group 1: Higher education leadership Moderation: Mr Simon Roy, Policy Analyst, Higher Education Policy Team, Directorate for Education and Skills, OECD
	Group 2: Staff responsible for institutional quality assurance, monitoring, and evaluation Moderation: Mr Thomas Weko, Head of the Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
	Group 3: Staff responsible for supporting students and teachers Moderation: Mr François Staring, Policy Analyst, Higher Education Policy Team, Directorate for Education and Skills, OECD
	Group 4: Instructors Moderation: Ms Andrea-Rosalinde Hofer, Policy Analyst, Higher Education Policy Team, Directorate for Education and Skills, OECD
	Group 5: Students Moderation: Dr Gillian Golden, Policy Analyst, Higher Education Policy Team, Directorate for Education and Skills, OECD
Session 3 – PLENARY REPORTING: Key take-aways from the group discussions
12:00 – 12:15	Key lessons learned by OECD moderators from breakout groups Presenters: OECD moderators of breakout groups Moderation: Mr Paulo Santiago, Head of the Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
Session 4 – LOOKING AHEAD: Supporting the quality of digital higher education in Hungary
12:15 – 12:45	Reflections from national-level higher education stakeholders National-level higher education stakeholders in Hungary will be invited to share what they are doing to support the quality of digital higher education in Hungary, and their reflections on the national roundtable. Prof. Dr Valéria Csépe, President, Hungarian Accreditation Committee (MAB) Dr Péter Levente Lakatos, Deputy Director, Hungarian Accreditation Committee (MAB) Dr Gábor Mészáros, Senior Counsellor, Higher Education Training Department, Ministry for Innovation and Technology László Murai, President, National Union of Students (HOÖK) Dr Zoltán Dubéczi, Secretary General, Hungarian Rectors’ Conference (MRK) Moderation: Mr Paulo Santiago, Head of the Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
12:45 – 12:50	Closing and next steps Dr Laura Sinóros-Szabó, Head of Department for Strategy and Institutional Development, Hungarian Ministry for Culture and Innovation (KIM)

International conference “International Quality Assurance Standards, Practices and Supports for Digital Higher Education” (online)

On 14 June 2022, the OECD team organised an online conference on “International Quality Assurance Standards, Practices and Supports for Digital Higher Education”. The event was aimed at:

• sharing international examples of best practice for the quality assurance of digital higher education with higher education stakeholders in Hungary; and

• providing an opportunity to delegates from the OECD’s Group of National Experts for Higher Education (GNE-HE) to attend as observers and learn more about the quality assurance of digital higher education.

Table A.8 provides presents the agenda for the international event, which was organised via Zoom videoconference.

Table A.8. Agenda international conference (14 June 2022)
Timing	Topic and speakers
13:00 – 13:15	Welcome and introduction Ms Ágota Kovács, Policy Officer, European Commission’s Directorate-General for Structural Reform Support (DG REFORM) Prof. Dr Valéria Csépe, President, Hungarian Accreditation Committee (MAB) Moderation: Mr Paulo Santiago, Head of the Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
Session 1 – How are quality assurance agencies evaluating and supporting the quality of digital higher education?
13:15 – 13:30	Setting the scene: Professor Mark Brown, Director, National Institute for Digital Learning (NIDL), Dublin City University (Ireland)
13:30 – 14:10	A closer look at the approach of three European quality assurance agencies Mr Walter Balfe, Head of the Quality Assurance Unit, Quality and Qualifications Ireland (QQI), Statutory Quality Assurance Guidelines for Providers of Blended Learning Programmes (2018) Dr Liia Lauri, Assessment Director, Estonian Quality Agency for Higher and Vocational Education (EKKA), E-Learning Guidelines and E-Learning Quality Label Dr Ailsa Crum, Director of Membership, Quality Enhancement & Standards, The UK Quality Assurance Agency for Higher Education (United Kingdom), Digitalisation Advice and Guidance for UK Higher Education Providers
14:10 – 14:30	Reflections from Hungary and Q&A with audience Interlocutor: Dr Péter Levente Lakatos, Deputy Director, Hungarian Accreditation Committee (MAB) Moderator: Mr Paulo Santiago, Head of the Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
14:30 – 14:45	Break
Session 2 – How are higher education institutions managing the quality of their digital courses and programmes?
14:45 – 15:00	Setting the scene: Dr Paul Bacsich, CEO, Dualversity (United Kingdom)
15:00 – 15:30	A closer look at the approach of two European higher education institutions Dr Panayiotis Angelides, Vice Rector for Academic Affairs and Professor of Inclusion for School Improvement (University of Nicosia) and Prof Christina Hadjisoteriou, Associate Professor School of Education (University of Nicosia), Quality Assurance at the University of Nicosia, Cyprus Dr Martin Stabauer, Deputy Head of the Institute for Digital Business, (Johannes Kepler University Linz), Internal Quality Assurance at Johannes Kepler University Linz, Austria
15:30 – 15:50	Reflections from Hungary and Q&A with audience Interlocutor: Dr Eszter Lukács, Vice-Rector for International Affairs, Széchenyi István University of Győr, Hungary Moderation: Mr Thomas Weko, Head of the Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
15:50 – 16:00	Break
Session 3 – How to support higher education institutions to deliver high-quality digital education?
16:00 – 16:10	Setting the scene: Professor Mark Brown, Director, National Institute for Digital Learning (NIDL), Dublin City University (Ireland)
16:10 – 16:45	Presentations: Supporting higher education institutions to deliver high-quality digital education Ms Ivana Juraga, Policy Officer, European Commission (DG EAC), Digital Education Action Plan (2021-2027) Ms Johanna de Groot, Programme Manager, SURF (Netherlands), Acceleration Plan and Digitalisation Impulse for Higher Education Institutions in the Netherlands Dr Bethany Simunich, Director of Research and Innovation, Quality Matters (United States), Quality Matters Rubrics for Programme and Course Evaluation
16:45 – 17:00	Reflections from Hungary and Q&A with audience Interlocutor: Dr Laura Sinóros-Szabó, Head of Department, Hungarian Ministry for Culture and Innovation (KIM) Moderator: Mr Paulo Santiago, Head of the Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
17:00	Thanking participants and closing of the conference Mr Paulo Santiago, Head of the Policy Advice and Implementation Division, Directorate for Education and Skills, OECD

National roundtable “Policy Options for Hungary to Assure the Quality of Digital Higher Education” (Budapest, Hungary)

On 4 October 2022, the OECD supported the Hungarian Deputy State Secretariat for Higher Education at the Ministry of Culture and Innovation (KIM) and the Hungarian Accreditation Committee (MAB) in organising a one-day national roundtable. The roundtable event was hosted by Hungarian University of Sports Science. The purpose of this event was to:

• present an initial diagnosis and discuss preliminary recommendations with key stakeholders from the higher education sector in Hungary about revisions to external QA standards, institutional quality management practices, and supports.

• provide Hungarian HEIs and other stakeholders the opportunity to test, debate, validate or contest the OECD review team’s initial findings and diagnosis, and to provide expert feedback on the preliminary recommendations.

Table A.9 presents the agenda for the national roundtable event, which was organised in person in Budapest, Hungary and was attended by 51 stakeholders from 27 different organisations.

Table A.9. Agenda national roundtable event (4 October 2022)
Timing	Topic and speakers
09:30 – 10:00	Welcome coffee
10:00 – 10:15	Welcome and introduction Dr Balázs Hankó, Deputy State Secretary for Higher Education, Hungarian Ministry of Culture, and Innovation (KIM) Prof Dr Valéria Csépe, President, Hungarian Accreditation Committee (MAB) Ágota Kovács, Policy Officer, European Commission’s Directorate-General for Structural Reform Support (DG REFORM) Prof Dr Tamás Sterbenz, Rector, Hungarian University of Sports Science
Session 1 – External quality assurance: Policy options for adapting the Hungarian external quality assurance framework for higher education to digital education
10:15 – 10:35 [20 minutes]	External quality assurance of digital higher education – International practices and policy options for Hungary Presentation of OECD’s initial findings from its analysis of external quality assurance practices for digital higher education internationally, and policy options Hungary might consider. Presentation: François Staring, Policy Analyst, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
10:35 – 11:15 [40 minutes]	Discussion with stakeholders Stakeholders are invited to react to the OECD’s presentation. Invited reaction [5 minutes]: Dr Péter Levente Lakatos (MAB), Director of International Affairs, Hungarian Accreditation Committee (MAB) Moderated discussion [35 minutes]: Thomas Weko, Team Leader, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
11:15 – 11:30	Comfort break
Session 2 – Internal quality management: Policy options to support greater HEI responsibility for the internal quality management of digital education
11:30 – 11:50	Internal quality management of digital higher education – International practices and policy options for Hungary Presentation of the OECD’s initial findings from its analysis of internal quality management practices for digital higher education internationally, and policy options Hungary might consider supporting greater institutional responsibility. Presentation: François Staring, Policy Analyst, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
11:50 – 12:30	Discussion with stakeholders Stakeholders are invited to react to the OECD’s presentation. Moderated discussion [40 minutes]: Thomas Weko, Team Leader, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
12:30 – 13:30	Lunch break
Session 3 – Institutional support: Policy options to support Hungarian HEIs in enhancing the quality of their digital higher education offer
13:30 – 13:45	Supporting institutions to enhance the quality of digital teaching and learning – International practices and policy options for Hungary Presentation of OECD’s initial findings from its analysis of institutional supports for digital higher education internationally, and policy options Hungary might consider. Presentation: François Staring, Policy Analyst, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
13:45 – 14:15	Discussion with stakeholders Stakeholders are invited to react to the presentation. Invited reactions [5 minutes]: Dr Laura Sinóros-Szabó, Head of Department for Strategy and Institutional Development, Hungarian Ministry of Culture and Innovation (KIM) Moderated discussion [25 minutes]: Thomas Weko, Team Leader, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
14:15 – 14:30	Comfort break
Session 4 – Potential standards and indicators: Potential standards and associated indicators for the quality assurance of digital higher education in Hungary
14:30 – 14:50	Presentation of potential standards and associated indicators for the quality assurance of digital higher education in Hungary Presentation of a list of potential standards and associated indicators for the quality assurance of digital higher education in Hungary. Presentation [15 minutes]: Prof Mark Brown, Director, National Institute for Digital Learning (NIDL), Dublin City University (Ireland)
14:50 – 15:20	Discussion with stakeholders Stakeholders are invited to react to the presentation. Presentation [10 minutes]: Prof Dr Valéria Csépe, President, Hungarian Accreditation Committee (MAB) Moderated discussion [20 minutes]: Thomas Weko, Team Leader, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
15:20 – 15:30	Closing of the event and next steps Dr Laura Sinóros-Szabó, Head of Department for Strategy and Institutional Development, Hungarian Ministry of Culture and Innovation (KIM)

Final conference “Ensuring Quality Digital Higher Education in Hungary” (Györ, Hungary)

On 29 March 2023, the OECD team organised a final project conference in Györ (Hungary), hosted by Széchenyi István University (SZE), and organised in cooperation with KIM and MAB. The purpose of the conference was to:

• share the findings and recommendations of the project with the higher education sector in Hungary;

• discuss concrete steps on how to implement the project recommendations in Hungary.

Table A.10 presents the agenda for the final conference, as agreed between KIM and MAB in February 2023.

Table A.10. Agenda final conference (29 March 2023)
Timing	Topic and speakers
09:30 – 10:00	Welcome coffee
10:00 – 10:20	Welcome and introduction Prof Dr László Palkovics, Chair of Board of Trustees, Széchenyi István University Dr Balázs Hankó, State Secretary for Higher Education and Innovation, Hungarian Ministry of Culture and Innovation (KIM) Ágota Kovács, Policy Officer, European Commission’s Directorate-General for Structural Reform Support (DG REFORM) Andreas Schleicher, Director, Directorate for Education and Skills, OECD
Session 1 – Global and European Trends: Global and European Trends in Higher Education Quality Assurance
10:20 – 10:35	Presentation International expert presentation on global trends affecting the quality assurance of higher education in the European Higher Education Area (EHEA). Elena Cirlan, Senior Policy and Project Coordinator, European Association of Quality Assurance in Higher Education (ENQA)
10:35 – 11 05	Panel discussion Moderated panel discussion on global trends affecting the quality assurance of higher education in the European Higher Education Area (EHEA). Dr Esther Huertas Hidalgo, Head of Quality Assurance Department, AQU Catalunya, Spain Diane Freiberger, Managing Director, Foundation for International Business Administration Accreditation (FIBAA), Germany Elena Cirlan, Senior Policy and Project Coordinator, European Association of Quality Assurance in Higher Education (ENQA)
11:05 – 11:15	Questions from audience
Session 2 – Quality Assurance in Hungary: Policy Options for Ensuring Quality Digital Higher Education in Hungary
11:15 – 11:35	Presentation Launch of the OECD publication Ensuring Quality Digital Higher Education in Hungary and presentation of key findings and recommendations. François Staring, Analyst, Higher Education Policy Team, Policy Advice and Implementation Division, Directorate for Education and Skills, OECD
11:35 – 11:50	Reflections from the Hungarian Accreditation Committee (MAB) Reflections from MAB on the findings and recommendations presented by the OECD and plans for the modernisation of higher education quality assurance in Hungary. Prof Dr Valéria Csépe, President, Hungarian Accreditation Committee (MAB)
11:50 – 12:00	Questions from audience
12:00 – 13:00	Lunch break
Session 3 – Supporting Institutional Reform: Institutional Practices and Supports for the Effective Quality Management of Digital Higher Education
13:00 – 13:15	Presentation International expert presentation on key challenges and inspiring practices for the development of digitally enhanced learning and teaching in higher education. Michael Gaebel, Director of Higher Education Policy, European University Association (EUA)
13:15 – 13:45	Panel discussion Moderated panel discussion on institutional quality management and support for the development of high-quality digital higher education. Prof Dr Petra Aczél, Full Professor of Communication and Rhetoric Moholy-Nagy University of Art and Design (MOME), Hungary Prof Dr Levente Kovács, Rector and Professor at John von Neumann Faculty of Informatics, Obuda University (OE), Hungary Michael Gaebel, Director of Higher Education Policy, European University Association (EUA)
13:45 – 14:00	Questions from audience
Session 4 – Reflections for the Future: Emergence of Alternative Credentials in Higher Education and Quality Assurance
14:00 – 14:20	Presentation International expert presentation on the emergence of alternative credentials in higher education, and key considerations for quality assurance. Professor Mark Brown, Director of the National Institute for Digital Learning (NIDL), Dublin City University (DCU), Ireland Professor Josep M. Duart, President of EDEN Digital Learning Europe and Full Professor at the Faculty of Psychology and Education Sciences of the Open University of Catalunya (UOC), Spain
14:20 – 14:30	Reflections from the Hungarian Ministry of Culture and Innovation (KIM) Reflections from KIM on the international expert presentation and plans to support the quality enhancement of digital higher education in Hungary. Dr Laura Sinóros-Szabó, Head of Department for Strategy and Institutional Development, Hungarian Ministry of Culture and Innovation (KIM)
14:30 – 14:40	Questions from audience
14:40 – 14:45	Closing Dr Balázs Hankó, State Secretary for Higher Education and Innovation, Hungarian Ministry of Culture and Innovation (KIM)