2. Regulation and external quality assurance of digital higher education
This project, led and implemented by the Organisation for Economic Co-operation and Development (OECD), was carried out with financial support provided by the European Commission’s Directorate-General for Structural Reform Support (DG REFORM), and in close collaboration with the Hungarian Ministry of Culture and Innovation (KIM) and the Hungarian Accreditation Committee (MAB).
This chapter provides an analysis of Hungary’s regulatory framework and external quality assurance system for higher education, and provides recommendations on how they can be modified to support the further development and quality enhancement of digital higher education.
This section analyses Hungary’s higher education regulation and external quality assurance (QA) system, and identifies two key barriers for the further development and quality of digital higher education.
Regulatory framework for digital higher education in Hungary
This section starts by describing Hungary’s institutional landscape and recent legislative changes affecting the overall governance and funding structure of higher education institutions (HEIs). It then analyses the existing regulation on programme and study formats, and how this impact on the development of digital higher education, programme innovation and study flexibility.
Institutional landscape and recent legislative changes
The Hungarian higher education system is comprised of 64 accredited HEIs (Educational Authority, 2021[1]). Table 2.1 provides an overview of the number and different types of HEIs operating in the country, broken down by educational profile and type of provider. The higher education law distinguishes between three types of institutions: universities (egyetem), universities of applied sciences (UAS) (alkalmazott tudományok egyeteme) and university colleges (főiskola). HEIs also differ from each other depending on whether they are state-owned or non-state operated. The latter are private entities operated by churches, business organisations or public interest trust foundations (DSN/DHECC, 2020[2]).
Minimum operating requirements for higher education institutions (HEIs)
Table 2.2 provides an overview of the minimum operating requirements for HEIs in Hungary, which include the minimum number of academic staff that should hold a doctoral qualification or above, and the minimum number of bachelor’s and master’s programmes to be offered for recognition as either a university, a UAS or a university college. Besides these minimum requirements, which take into account differences between institutions based on their educational profile, all HEIs in Hungary must be accredited by the Educational Authority (OH) at institution and programme level to be allowed to operate. HEIs in Hungary are not required to meet any specific criteria related to their capacity to offer flexible or digital study programmes, the only two exceptions to this rule being the requirement for libraries of public universities to “offer conventional and virtual learning environments” (Government of Hungary, 2011a[3]) and – since March 2020, in response to the COVID-19 pandemic – for all HEIs to have in place a virtual learning environment (VLE) or learning management system (LMS) that can support the flexible planning and organisation of student learning, the delivery of digital programmes, and the evaluation and recording of student learning.
Organisation, management and funding of higher education institutions (HEIs)
Since 2011, the government has taken several steps to introduce a foundation management model of HEIs to ensure a more modern and competitive operation of HEIs that is adjusted to the needs of the modern economy (KIM, 2020[5]; Vida, 2021[6]). The stated rationale for the change also includes increasing HEIs’ responsibility and accountability for assuring the quality of their teaching, learning and research activities, measured in terms of direct economic benefits.
Hungary’s recent institutional landscape reform included the following three phases:
Introduction of a dual management model in public institutions. In 2014, Hungary introduced a “dual management” model in public HEIs to tackle the practice of HEIs appointing rectors with an outstanding academic track record, but limited managerial, organisational or financial skills or experience. As a result, each state-owned HEI in Hungary is now led by both a Rector and a Chancellor. The Rector chairs the Senate and is responsible for teaching and research matters whereas the Chancellor chairs the Consistory and oversees operational, financial and strategic matters. However, as the Chancellor, Rector and three members of the Consistory are directly appointed by the Ministry, this provides Hungarian government with a potentially high degree of influence over how teaching and learning takes place in public HEIs.
Establishment of institutions as Public Trust Foundations. Public Trust Foundations were introduced by the Ministry in 2018, starting with the “model change” of Corvinus University. At its core, the model change involves changing the maintenance and governance model of HEIs from a public status into a private charitable organisation. The public property of these HEIs (such as historical buildings) passes from public to foundation ownership. Permanently appointed employees also lose their civil service rights and benefits granted to them in the National Act on Civil Servants and State Employees (Government of Hungary, 1992[7]).
Introduction of a performance-based funding model. In 2021-22, Hungary introduced a 3-to-5-year performance-based financing system, using performance indicators agreed between the government and individual HEIs. The aim is that, by 2024-25, 50% of all funding of foundation institutions will be based on a set of nationally agreed key performance indicators (KPIs), many of which include a focus on the outcomes of HEIs’ educational offer (see Table 2.3), to incentivise greater institutional attention to quality enhancement and labour market alignment.
Some higher education stakeholders interviewed by the OECD review team expressed concern that the introduction of a labour market and performance-oriented management and funding model would diminish the priority of academic excellence in higher education. Stakeholders also underlined that HEIs would require additional resources and support from the government to meet the additional quality expectations, and that the implementation and monitoring of performance indicators should accommodate the diversity of institutions, programmes, and modes of instruction (Vida, 2021[6]). For example, stakeholders felt that fully online programmes should not be assessed against the same performance criteria as in-person or hybrid study programmes, as evidence shows that there are higher risks of non-completion for students enrolled in fully online or distance learning programmes. As discussed in this section, adult learners are most likely to enrol in distance learning programmes, as this allows them to combine work and studies. These additional commitments, however, mean that they are at higher risk of dropping out than “regular” daytime students.
The wider impact of these legislative changes on the development and quality of HEIs’ internal operations and the quality of teaching and learning is yet to be seen. Actors at government and institutional level have different views on the expected benefits and perceived risks associated with the model change process, with some strongly opposed to its implementation (Derényi, 2020[8]). Table 2.4 provides an overview of the expected benefits and risks perceived by governmental and institutional stakeholders.
Study formats in Hungarian higher education
Hungary has adopted the three-cycle bachelor’s, master’s, and doctoral degree structure, thereby following the official three-cycle qualifications framework in the European Higher Education Area (EHEA) (EHEA, 2005[9]). The European Credit Transfer System (ECTS) is used to define the average number of study hours and semesters for each level of education, with one ECTS credit equalling an average of 30 hours of study. In addition, the Hungarian Central Statistical Office (KSH, 2011[10]) indicates the level to which each programme corresponds using the International Standard of Classification of Education (ISCED). In addition to bachelor’s, master’s and doctoral programmes, HEIs in can offer three other types of programmes: higher vocational education and training (VET) programmes, single-cycle long programmes and postgraduate specialisation programmes (see Table 2.5).
Higher VET programmes serve primarily as a bridge between secondary and tertiary education and are a rather recent initiative in the Hungarian higher education system, the first of these programmes being launched in 2013. Most higher VET programmes are four semesters in length and worth 120 ECTS credits. Upon completion, students receive a certificate that can provide access to bachelor’s programmes. Single-cycle long programmes are different to the three-cycle structure and have kept their original (pre-Bologna) structure. They are linked to, and typically regulated by, the respective profession such as medicine, dentistry, forestry or law. Upon completing these programmes, students receive a master’s degree. Professional specialisation programmes do not lead to a higher-level qualification. They are aimed at training the workforce in a specific professional field after having completed higher education degree.
Regulation on the study format of higher education programmes
Within the overarching three-cycle Bologna structure, higher education law in Hungary strictly regulates the study formats that HEIs may use to offer degree programmes and courses. According to Article 17 of the National Act on Higher Education (Government of Hungary, 2011a[3]), HEIs can offer study programmes as full-time, part-time or distance learning programmes according to the provisions of the training and outcome requirements. Each of these has strict requirements on the minimum/maximum number of contact hours per semester (study intensity) as well as when (i.e. evening/daytime, weekdays/weekend) and how (i.e. online/in-person) instruction is to be delivered (study mode). The definition of distance learning and contact hours in Hungarian higher education law is presented in Box 2.1. An overview of the requirements for the delivery of instruction is included in Table 2.6.
In Hungarian higher education law, distance learning and a contact hour are defined as follows:
Distance learning is defined as “a form of training in which the theoretical training knowledge is taught within a digital curriculum, and in co-operation of the teacher and the student in a closed distance virtual learning environment or learning management system (VLE/LMS) via the internal IT network of the higher education institution (internet, intranet). Within this VLE/LMS, the instructor, the computer and the IT network, as well as the VLE/LMS and the study system, are the common means of communication between the education organiser and the student or person participating in the training”.
One contact hour is defined as “a session (lecture, seminar, practice session, consultation) with a duration of not less than 45 and not more than 60 minutes, where the personal contribution of a lecturer or professor is needed for the fulfilment of the academic requirements laid down in the curriculum”.
Source: Government of Hungary (2011a[3]), Act CCIV of 2011 on National Higher Education, Government of Hungary, Budapest, https://net.jogtar.hu/jogszabaly?docid=A1100204.TV.
Several stakeholders from HEIs interviewed by the OECD review team highlighted that under current study format rules, HEIs are not authorised to offer hybrid study programmes. However, the COVID-19 pandemic has led to case-by-case derogations that have permitted HEIs to offer “regular” programmes as fully online and hybrid study programmes, and several HEIs are continuing to do so, albeit without legal background. The current rules reduce the flexibility for learners to organise their studies in line with their individual needs and interests (Tolnai, 2021[11]). However, in the case of postgraduate training programmes, HEIs only need to register their programmes with the OH and are not required to go through ex ante programme accreditation.
Hungarian law1 also distinguishes between highly theory-oriented, theory-oriented, balanced, practice-oriented and highly practice-oriented programmes (Government of Hungary, 2011a[3]). According to the administrative data system for higher education (Educational Authority, 2022a[12]), in 2021-22 there were 515 different programmes in Hungarian higher education (excluding PhD programmes and postgraduate specialisation programmes). Of these 515 programmes, 4 (1%) were highly theory-oriented, 80 (16%) theory-oriented, 264 (51%) balanced, 134 (26%) practice-oriented and 33 (6%) highly practice-oriented. In practice, however, teaching in Hungarian higher education is primarily lecture-based. According to a recent study comparing the teaching approaches of Hungarian and Finnish academics, the least characteristic teaching approach of Hungarian lecturers was practice-based teaching, focused on combining theory and practice and connecting the content of a course to practical exercises (Kálmán, Tynjälä and Skaniakos, 2020[13]). These results are confirmed by a study commissioned by the European Commission. Around 60% of higher education leaders interviewed as part of this study stated that lecture-based teaching is the most common teaching method in their institutions (OECD/EU, 2017[14]).
Regulation on the content of higher education programmes
Regulation stipulates that HEIs can only launch new programmes in registered fields of study. Applications for new fields of study must be evaluated by the Hungarian Accreditation Committee (MAB) as an expert body and subsequently approved by the OH and the Ministry of Culture and Innovation (KIM). Table 2.7 provides an overview of the criteria applied by MAB in the evaluation of applications for the establishment of programmes in new fields of study. Applications consist of two parts: part one asks institutions to justify the establishment of a programme in a new field of study in the context of the existing higher education offer in Hungary and internationally; part two relates to the new field of study’s proposed education plan and learning outcomes. Approved applications are included in the official Higher Education Qualifications Register.2
Higher education stakeholders interviewed by the OECD review team highlighted that the Higher Education Qualifications Register is rarely reviewed and is therefore not aligned with the latest developments in their research field or the labour market, which hinders programme innovation. In practice, however, as there is no ex post programme review procedure in Hungary, institutions and instructors are able to deviate from the national content requirements once a programme has been launched. While some instructors saw this flexibility as beneficial, as it allows them to ensure the relevance of the content delivered to their students, others felt that the lack of a regular programme review procedure leads to disparities in the quality of teaching and learning across higher education in Hungary, and does not sufficiently incentivise institutions or instructors to take responsibility for assuring the quality of instruction and student learning outcomes.
Regulation on student admission, course selection and progression, and the recognition of courses and degree programmes
The enrolment capacity of HEIs is set by the OH based upon an assessment of HEIs’ instructional sites, computers, library spaces, and student accommodation, as well as their student and career counselling services and available sports facilities (Educational Authority, 2022b[15]). Based on this assessment, the institutions themselves are responsible for defining the maximum student numbers and admission criteria for each programme. Admission criteria typically include applicants’ previous academic performance, the student capacity of the selected programme and the order of preference indicated by applicants. The OH’s higher education admissions and information website Felvi.hu provides information for applicants on the maximum student capacity and admission requirements for each programme (Educational Authority, n.d.[16]).
Upon enrolling, students must choose one of the five legally authorised study formats. Based on the selected study mode, HEIs provide students with a recommended curriculum from which they can create their own study plans. For full-time study programmes, the curricula proposed by HEIs typically recommend 30 ECTS credits per semester. To retain their scholarship, state-funded students must have completed at least 18 ECTS credits in each of their previous two semesters and obtain a minimum weighted grade point average (GPA).3 The GPA requirement differs depending on the discipline (see Table 2.8). When composing their individual curricula, students can typically select courses from other study programmes at their home institution or at another HEI in Hungary (as guest students), provided that these courses relate to their field of study.
In principle, it is not possible for students to select courses from programmes taking place at different times (e.g. selecting courses from evening study programmes as a full-time daytime student), or to follow a course organised in a different study mode (e.g. choosing courses from a distance learning programme as a full-time student), as the programme intensity and mode of study is strictly regulated at national level, and often also at institutional level. In practice, however, higher education stakeholders interviewed by the OECD review team mentioned that students and institutions are trying to find “loopholes” in the legislation to give students more flexibility. For example, in some institutions it is possible for students to enrol for the same programme twice (e.g. as a full-time day student and as a part-time evening student), and submit a credit transfer form to have courses completed in the part-time evening programme recognised for the completion of their full-time day programme (or vice-versa).
For the recognition of courses and degree programmes completed by students at other institutions, higher education law recommends that HEIs verify a 75% match in student learning outcomes (Government of Hungary, 2011a[3]). This assessment is typically carried out by an institutional Credit Transfer Committee, which is also responsible for the recognition of prior non-formal and informal learning, as well as work experience. Higher education stakeholders interviewed by the OECD review team noted that staff working in such committees typically focus on comparing the content of courses rather than students’ learning outcomes, for which evidence is often lacking. Credit Transfer Committees often do not have sufficient information on courses and programmes offered at other institutions, as not all HEIs in Hungary publish regular and up-to-date information on the content and learning outcomes of their study programmes online. This often leads to the non-recognition of courses or full degrees that have been successfully completed by students at other institutions, and students having to take up additional courses at their home institution to replace non-recognised courses. This significantly increases their study load for some students, which negatively impacts their higher education experience, and increases the risk of drop-out.
Finally, higher education stakeholders interviewed by the OECD review team noted that a major barrier to the further development and internationalisation of higher education in Hungary is that the law still prescribes paper-based administration for several procedures. For example, Government Decree 87/2015 (IV. 9.) specifies that “enrolment can be initiated by filling in and signing the enrolment form”, and that diplomas can only be awarded on paper. Article 39/A states that non-Hungarian nationality students can start their studies in distance learning format by sending their enrolment form electronically to the institution (Government of Hungary, 2015[18]). Article 12 (5) of Government Decree 423/2012 (XII. 29.) specifies that students are required to present original, paper-based documents upon enrolment, prior to starting their degree. By contrast, distance learning and correspondence students are given the flexibility to present these documents in person only when they arrive at the institution for their first lecture or consultation (Government of Hungary, 2011a[3]).
The higher education stakeholders interviewed by the OECD review team shared the following reflections related to the existing regulations on student admission and enrolment, course selection and progression, and the recognition of courses and degrees:
Regulation on student admission and enrolment. Higher education stakeholders felt the current student admission and selection criteria are too strict, and have discouraged student applications and enrolments, especially among socio-economically disadvantaged groups. They also felt that the practice of regulating the maximum student capacity of HEIs based on their physical infrastructure, staff and available support services might need to be revised to take into account the specific types of digital equipment and supports needed to ensure quality and inclusive teaching and learning in fully online and hybrid study programmes.
Regulation on course selection and progression. Higher education stakeholders pointed out that the current regulation on course selection and progression limits students’ flexibility to choose what, when (e.g. daytime, evening) and from where (e.g. online, in person) to study. Making course and programme selection requirements more flexible and supporting institutions to develop hybrid flexible or “hy-flex” programmes4 were mentioned as options that could help Hungary move towards a more student-centred, modern, flexible and inclusive higher education system.
Regulation on the recognition of courses and degree programmes. Stakeholders highlighted the need to support and monitor the application of the learning outcomes approach by recognition officers, as well as the need for greater flexibility in the application of recognition procedures by institutions to expand (virtual) student mobility and encourage students to explore courses from other institutions and disciplines, thereby promoting inter-disciplinary teaching and learning approaches, and inter-institutional co-operation, both nationally and internationally. They also highlighted the importance of ensuring that all institutions publish reliable and up-to-date information on their courses online, including details on the study materials, teaching methods and assessment practices used to develop student learning outcomes, to facilitate the work of recognition officers. In this context, the use of digitalisation (e.g. block-chain technology) for the reliable and secure exchange of student and course information was highlighted as having the potential to transform the quality, fairness and efficiency of recognition practices.
Impact of regulation on the development and quality of digital higher education
The introduction of a state of “epidemiological preparedness” (Government of Hungary, 2011a[3]) by KIM in response to the COVID-19 pandemic prompted many HEIs to rapidly develop fully online and hybrid study programmes, outside of the existing regulation on study formats, and for public authorities to grant exceptional approval – derogations – to authorise their initiatives (see Table 2.9).
While some form of digital education is now offered across all Hungarian HEIs, it is difficult to reliably identify the exact number of fully online and hybrid study programmes currently on offer in Hungary, because national-level data collection by the OH is still based on the legal categories of full-time, part-time and distance learning (Educational Authority, 2019[19]), meaning only distance learning programmes delivered in their traditional form can be counted. In September 2021, 45 distance learning programmes were on offer at nine institutions.5 As the total number of programmes offered in Hungary that year was 11 246, officially accredited distance learning programmes represented only a very small proportion (0.004%) of the higher education offer in Hungary (Educational Authority, 2021[20]).
Higher education stakeholders interviewed by the OECD review team indicated that one of the main reasons for the low number of officially accredited distance learning programmes in Hungary may be the fact that the public authorities view “full-time daytime study” as the preferred mode of study. Another reason could be the strict requirements for launching programmes in distance learning format, which are discussed further in this section. Figure 2.1 shows that, between 2011 and 2020, the total number of applicants and enrolment in distance learning programmes dropped from 2 219 (applicants) and 1 202 (enrolments) in 2011 to 653 (applications) and 251 (enrolments) in 2020. In 2021, however, student demand for distance learning programmes slightly increased again to 1 055 (applications) and 452 (enrolments), perhaps as a result of the COVID-19 pandemic.
Source: FELVI (2021[20]), Statistics from the past years of applications and acceptance (2001-2021), Educational Authority (OH), Budapest, https://www.felvi.hu/felveteli/ponthatarok_statisztikak/elmult_evek/!ElmultEvek/index.php/elmult_evek_statisztikai/munkarendenkent.
Another factor contributing to the low number of officially accredited distance learning programmes in Hungary may be the longstanding status of regular full-time programmes, which have higher completion rates. Across all levels of education, evidence shows that distance learning students are at higher risk of dropping out than students enrolled in full-time, correspondence or evening education.6 Table 2.10 shows that in 2011-12, 54.3% of bachelor’s students enrolled in a distance learning programme had dropped out, compared to 31.5% of full-time students. Moreover, evidence shows that students from a lower socio-economic background, students from rural areas and adult learners are at higher risk of dropping out than younger students from a more socio-economically advantaged and urban background. However, as adult learners are most likely to enrol in distance learning, evening or correspondence programmes, which allow them to combine work and studies, much of the observed difference in drop-out rates might be the result of student characteristics, rather than study modes (Vida, 2021[6]).
All stakeholders interviewed by the OECD review team highlighted the need to expand and increase the quality of digital higher education in Hungary as a key priority for the future, especially to tackle major challenges related to demographics and skills. Digital higher education can play a role in upskilling and reskilling the active workforce. This is important, as studies show a low uptake of lifelong learning among the active labour force in Hungary. For example, in 2019 only 5.8% of Hungarian adults were participating in formal education or training courses, which was well below the EU average of 10.8% (European Commission, 2020[21]). Digital higher education can also be an important lever to increase tertiary education participation and attainment rates, especially among students from disadvantaged socio-economic backgrounds and international students. Retaining students after they graduate, however, is a wider systemic challenge facing Hungary that goes beyond higher education policy alone. Hungary is one of the few countries across the OECD where those with high levels of educational attainment are more likely to emigrate than those with lower levels of educational attainment (European Commission, 2020[21]), (Hárs, 2019[22]). A recent report by the Hungarian State Audit office noted that up to 14% of students in tertiary education hope that their degree will allow them to gain employment abroad (Vida, 2021[6]).
External quality assurance of digital higher education in Hungary
This section starts by describing the overall structure and governance of Hungary’s external QA system for higher education. It then focuses more specifically on the role and activities carried out by MAB as the independent expert body tasked with ensuring the quality of teaching, learning, research and artistic activities in Hungarian higher education, and the extent to which the standards and procedures implemented by MAB reflect specific considerations for digital education. First, recent (international) developments driving MAB’s procedures are reviewed. Next, as per the analytical framework presented in Table 1.1 (Chapter 1), this section describes and analyses how MAB ensures the quality of (digital) higher education in Hungary through both formal quality assurance and institutional quality enhancement.
The review of standards and indicators as part of MAB’s formal quality assurance procedures is carried out as follows:
Number of indicators. For each procedure, the total number of indicators for which institutions are required to provide evidence is set out.
Level and focus of indicators. For each indicator, an assessment is made as to whether it focuses on requirements at the institution, programme, course, or individual student/instructor level, as well as whether it focusses on the inputs, processes or outputs of education, and includes any specific considerations or requirements for digital education.
Evidence. For each indicator, an assessment is made as to whether it requires HEIs to provide quantitative or qualitative evidence, or a mix of both.
Structure and governance of external quality assurance
A report published by the OH on the Hungarian Qualifications Framework states that a “multi-level and multifunctional accreditation system is operated in Hungarian higher education linked with licensing procedures” (Szlamka, 2015, p. 5[24]). This means that the external QA of higher education teaching, learning, research and artistic activities in Hungary is ensured through inter-related processes of regulation (set by the Minister responsible for higher education), evaluation (carried out by MAB, based on the quality standards embedded in the regulation) and licensing (granted by the OH, based on MAB expert reports):
Evaluation. MAB is responsible for carrying out ex ante evaluations of applications for the establishment of new HEIs, higher VET, bachelor’s and master’s programmes, as well as the establishment of foreign HEIs and new doctoral schools at universities. It also carries out ex post reviews of the operations of HEIs and doctoral schools in five-year cycles. In addition to this, with the involvement of Hungarian and international reviewers, MAB evaluates the educational and scientific/artistic performance of applicants for university professor positions, based on specific and publicly available criteria. MAB carries out its evaluations following a formal request from the OH and, based upon the results of its reviews, develops and submits reports to the OH.
Regulation. KIM is the authority with “second instance competence” (appellate forum)7 for the external QA of higher education. In addition to being responsible for setting the overall regulation governing the overall structure and operations of HEIs, the Minister responsible for higher education also acts as a partner of the OH (or the HEI, in the case of voluntary requests from institutions to have specific programmes evaluated) in requesting MAB to carry out evaluations of specific training programmes, institutions or university professors, and to submit an expert report to the OH.
Licensing. The OH is “the body designated by the Government for the performance of certain tasks falling within the sphere of the public education responsibilities of the Minister” (Government of Hungary, 2011a[3]). This means it is a body operating at arms’ length of the Ministry to support the implementation of all regulation pertaining to education. With regards to the external QA of higher education, the OH is the institution with “first instance competence”8 to license, register and grant permissions to HEIs and their programmes to operate by “issuing formal approval (in the form of regulatory acts) for the operation of higher education institutions and individual […] programmes” (Government of Hungary, 2011a[3]). The OH orders MAB to carry out institutional or programme evaluations, bases its decisions on their expert reports, and also makes the final decision on university professor applications. If requested by the Minister, the OH can participate in inspections carried out by MAB.
MAB was established in 1993 together with the country’s first higher education law. Figure 2.2 provides an overview of MAB’s organisational structure. MAB is an independent higher education QA agency, participating as an expert body in assuring and reviewing the quality of HEIs and their operations. KIM exercises legal supervision over MAB’s activities and provides budget support for the performance of its public tasks. MAB’s budget is under the control of the agency’s President, who is supported by the Board of Financial Supervisors and appointed directly by the Ministry. The bulk of the organisation’s expenditure goes is on personnel (wages of Board members and MAB staff, including site visit teams), followed by social contributions and material expenses (MAB, 2018[25]). MAB performs its role as the provider of expert evaluations through its Discipline-Specific Expert Committees, as well as several additional Advisory and Ad Hoc Expert Committees. In addition to carrying out reviews of institutions, the senior academic experts (both Hungarian and foreign experts) included in these committees are responsible for reviewing the quality of study programmes and university professor applications, as well as advising MAB on the preparation and implementation of QA decisions and reforms.
Source: Adapted from MAB (2022a[26]), A MAB [About MAB], https://www.mab.hu/mab/.
International drivers for external quality assurance in Hungary
In recent years, MAB has taken several steps to increase its compliance with international standards and practices for the external QA of higher education, and succeeded in raising the international profile and engagement of Hungarian higher education. MAB has embedded the European Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) (ENQA, 2015[27]) in its accreditation procedures as well as increased compliance with other international standards and practices, such as the standards of the World Federation for Medical Education (WFME) (MAB, 2021b[28]). It is also active in various international networks and projects related to higher education QA, and there are plans to grant accredited institutions self-accreditation status to independently launch new master’s level programmes.
Increasing compliance with the European Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG)
The use of the ESG is a key requirement for membership in the European Association for Quality Assurance in Higher Education (ENQA). MAB has therefore taken several steps to embed the ESG across its accreditation procedures, starting in 2017 with the introduction of the accreditation of institutions based on the ESG. Prior to this, there had been a five-yearly institutional accreditation procedure in Hungary, but this focused primarily on technical requirements, with limited attention to teaching and learning processes, outcomes and internal QA practices. In September 2019, MAB then introduced the accreditation of doctoral schools in five-year cycles based on the ESG. More recently, upon the request of KIM, MAB has started a project – in collaboration with the OH and the Hungarian Rectors’ Conference (MRK) – aimed at reflecting on how to embed the ESG standards and principles in programme accreditation, as well as how to strengthen the capacity of HEIs to take responsibility and ownership for the quality enhancement of their (digital) teaching and learning offerings (see Box 2.2), as the ESG sate that “higher education institutions have primary responsibility for the quality of their provision and its assurance” (ENQA, 2015, p. 8[27]).
In April 2022, Hungary started a project on the modernisation of higher education and accreditation, funded by KIM and implemented in collaboration with the OH and MRK. Among other objectives, the project seeks to address longstanding challenges in Hungarian higher education and support the development of quality teaching and learning. Longstanding challenges in Hungary include:
Institutions’ and instructors’ strong attachment to discipline-specific knowledge transfer
Limited focus on transversal skills development, experimentation and innovation
Limited labour market and societal relevance of higher education programmes.
In line with leading international practice across the OECD, options the project is exploring include:
The introduction of a self-accreditation status for HEIs with demonstrated capacity to manage programmes of high quality
The introduction of an ex post programme review procedure focused on the education and labour market outcomes of courses and programmes
Simplification of the current two-stage ex ante programme accreditation procedure.
Source: Based on stakeholder interviews with MAB as well as (2022f[29]), “Why and how to change the program accreditation system in Hungary”, National Roundtable on Policy Options for Hungary to Assure the Quality of Digital Higher Education, Presentation by Prof Dr Valéria Csépe on 4 October 2022, Budapest, https://www.mab.hu/en/publications/.
MAB has been an official member of ENQA since 2002 and has undergone regular external evaluations to ensure it complies with Parts 2 and 3 of the ESG. Following ENQA’s latest external review of its activities, MAB received official re-confirmation of its membership on 13 September 2018. In its evaluation report (ENQA, 2015[27]), the ENQA panel found MAB to be fully compliant with nine of the ESG Part 2 and Part 3 standards, substantially compliant with four, and partially compliant with one. In preparation for the next ENQA review (in 2023), MAB was asked to submit a follow-up report in 2020, setting out planned and completed actions to address ENQA’s recommendations. MAB submitted the report to ENQA in October 2020, followed by two international experts from the ENQA review panel conducting a (virtual) visit to MAB on 27 January 2021, to discuss MAB’s planned and completed actions in response to ENQA’s review (MAB, 2021d, p. 10[30]). Table 2.11 outlines the recommendations included in ENQA’s external evaluation report in relation to the five standards with which MAB was found to be partially and substantially compliant, as well as the actions taken by MAB to improve compliance with them, is presented.
Introduction of self-accreditation of master’s programmes for accredited institutions
International and regional bodies active in the field of (higher) education and QA, such as the International Network for Quality Assurance Agencies in Higher Education (INQAAHE), ENQA and the European Commission have been calling upon higher education systems to move towards the introduction of self-accreditation for HEIs, to further enhance their responsibility for quality. On 13 April 2022, the EU adopted a Council Recommendation on building bridges for effective European higher education co-operation, in which it called upon EU Member States to “move further towards the use of institutional-based external quality assurance” and “consider the possibility of allowing for self-accreditation of programmes to underpin the self-responsibility of higher education institutions” (Council of the European Union, 2022a, p. 12[37]).
In Hungary, the Parliament adopted a package of legislative changes which will make it possible for all accredited HEIs in Hungary to independently launch new programmes at master’s level in disciplines in which they are already offering programmes (see Box 2.3). Higher education stakeholders interviewed by the OECD team mentioned that they expect this will be a major game changer for how HEIs in Hungary perceive external accreditation as well as their role in QA. Stakeholders expect this change to have the potential to contribute to the quality enhancement of teaching and learning in higher education.
On 20 December 2022, a new package of legislative changes was adopted by the Hungarian Parliament and introduced to the National Act on Higher Education of 2011. One of these is the possibility for higher education institutions with valid institutional accreditation by MAB to independently launch new master’s programmes in disciplines within which they have previously obtained the right to offer bachelor’s, master’s or single-cycle long programmes.
The legislative change allows Hungarian HEIs to freely design the curriculum and learning outcomes of new master’s programmes, and to register them directly with the OH, without first having to apply for MAB accreditation. The only requirement checked by the OH is that the name of the proposed new programme cannot be confused with the names of other already existing study programmes. Teacher training programmes and master’s programmes in the field of political science are excluded from this rule, and still require ex ante accreditation by MAB.
Source: Government of Hungary (2011a[3]), Act CCIV of 2011 on National Higher Education, Government of Hungary, Budapest, https://net.jogtar.hu/jogszabaly?docid=A1100204.TV.
Increasing compliance with international quality standards and practices
Following ENQA’s confirmation of its full compliance with the ESG, MAB applied for listing in the European Register of Quality Assurance Agencies in the European Higher Education Area (EQAR) and was admitted as a full member (“substantially compliant with the ESG”) for the first time in April 2019. MAB’s membership will remain valid until 30 September 2023 and has led to participation in several international projects aimed at further strengthening the international relevance of MAB’s accreditation procedures and the quality of Hungarian higher education. Participation in these international projects is seen by MAB as “necessary to strengthen the organisation’s reaction capabilities and to incorporate the new European trends that are useful for the country’s higher education” (MAB, 2021d, p. 9[30]). Some of the main actions taken by MAB to increase compliance with international quality standards are described below.
Application of international quality standards. The quality of medical education in Hungary is receiving increasing international recognition. For example, in 2020 the national QA body of Kazakhstan asked MAB to provide medical experts to participate in their external quality reviews. To further support the quality enhancement of medical education in Hungary, MAB has started the implementation of an ex post evaluation procedure for medical training programmes based on the standards of the WFME (MAB, 2021b[28]), and was recently recognised by the WFME.
Participation in international quality assurance events. MAB staff members regularly attend international workshops and conferences to stay up to date of the latest international developments in higher education QA. For example, in 2020 and 2021 MAB staff members attended a range of international conferences and events organised by international bodies active in the area of higher education QA, such as ENQA, the European University Association (EUA), or the European Quality Assurance Forum (EQAF) (MAB, 2021d, pp. 9-13[30]).
Regional co-operation on quality assurance. MAB is very active in transnational and regional collaboration on higher education QA. Examples include the following:
MAB is a founding member of the Central and Eastern European Network of Quality Assurance Agencies in Higher Education (CEENQA). The network assembles 27 QA agencies that follow internationally recognised standards and guidelines for QA in higher education such as the ESG (ENQA, 2015[27]), the INQAAHE Guidelines of Good Practice (INQAAHE, 2018[38]) and the ECA Code of Good Practice (ECA, n.d.[39]).
On 7 October 2021, the leaders of the higher education QA agencies of the four Visegrád countries (Czech Republic, Hungary, Poland and Slovakia) signed a memorandum of understanding, valid for five years (NAB, Czechia>;PKA, Poland; SAAHE; MAB, Hungary, 2021[40]). This has led to the establishment of the Visegrád Four Quality Assurance Forum (V4QA Forum), aimed at facilitating regional collaboration and exchange between MAB and the QA agencies in the Czech Republic, Poland, and Slovakia, to develop joint policy proposals on higher education and QA in the EHEA.
In August 2021, MAB visited Romania’s QA agency for higher education (ARACIS) (MAB, 2021d, p. 11[30]). Following this visit, a memorandum of collaboration was signed on 15 December 2021, in which both agencies agreed to “participate in joint projects, organise professional exchange programmes, publish in each other’s publications and support each other’s work through the regular exchange of experience” (MAB, 2021a[41]).
Participation in international projects. MAB also participates in several international projects on higher education QA. Examples include the following:
As part of the DEQAR CONNECT project (EQAR, n.d.[42]), MAB has been uploading its agency review reports to the EQAR Database of External Quality Assurance Results (DEQAR) to help EQAR expand DEQAR’s coverage to currently under-represented countries (EQAR, n.d.[43]).
Between 2018 and 2022, MAB took part in the MICROBOL Working Group on the Quality Assurance of Micro-Credentials. The discussions of this Working Group fed into the publication of a Common Framework for Micro-Credentials in the EHEA, in March 2022. Micro-credentials are “certified small volumes of learning”, often offered in online or hybrid formats, targeting the working adult population in search of upskilling or reskilling to meet rapidly changing skills and labour market demands. The report recommends that “the focus of external QA should be on the institutional approach to micro-credentials and their explicit inclusion in existing or new processes” (MICROBOL, 2022, p. 7[44]). The report also suggests that setting up a register of trustworthy (or accredited) higher education providers that are allowed to offer micro-credentials could be a good way of both promoting and ensuring the quality of micro-credentials. At a webinar organised by MAB, in co-operation with DEQAR, on 16 February 2022, MAB underlined the importance of opening up Hungary’s higher education system to alternative providers and making changes to existing regulations to make it possible for providers to offer micro-credentials. HEIs would, however, need specific guidance, and regulations on programme types would need to be made more flexible (MAB, 2022e[45]).
Quality assurance of higher education in Hungary
This section analyses the formal QA procedures for which MAB is responsible, including the standards underpinning their implementation. For each set of procedures and standards, there is analysis of their relevance and impact on the development of digital higher education and institutional quality management. The standards and procedures for the formal QA of higher education are defined by Government Decree 19/2012 on higher education QA and enhancement (Government of Hungary, 2012a[46]) and government Decree 387/2012 on doctoral schools (Government of Hungary, 2012b[47]).
Procedures for the accreditation of higher education in Hungary
Table B.1 (Annex B) provides an overview of the external QA processes for which MAB is responsible. This includes both ex ante (i.e. prior to operation) and ex post (i.e. in operation) procedures at institutional, programme and individual instructor level. While each procedure differs in terms of the specific steps underpinning its implementation, as well as which actors are involved in the process, both types of procedure largely adhere to the following steps (see Figure 2.3).
Ex ante accreditation (to establish a new institution, programme or doctoral school) is initiated by HEIs petitioning the OH. The OH then formally appoints MAB to undertake an independent evaluation of the application documents submitted by the HEI, carried out by an independent expert committee of national and international experts in relevant discipline(s). Based on their evaluation of the documentation submitted by the institution, the expert committee prepares a report, which is reviewed by the MAB Secretariat. Based on this review, MAB makes an accreditation decision and communicates this decision to the OH. The OH then reviews MAB’s expert report and makes a final decision, informing the Ministry and relevant HEI of the outcome, registering the institution or new (doctoral programme), giving it the official license to operate.
Ex post accreditation is carried out at institutional and doctoral schools every five years, and every eight years for medical training programmes. As part of this process, institutions are not required to petition the OH. MAB is directly responsible for contacting institutions that are up for review, asking them to submit relevant documentation and to prepare a self-evaluation report. This is followed by an institutional site visit carried out by an independent expert committee. Based on the written documentation and evidence collected through the site visit, the expert committee prepares a report which is reviewed by the MAB Secretariat. MAB’s accreditation decision is then communicated directly to the HEI.
Note: This figure represents a high-level overview of MAB’s accreditation and evaluation processes only. Each individual procedure may include more or fewer steps and/or actors in the evaluation or accreditation process. For example, while the establishment of new HEIs, programmes and the appointment of university professors requires specific ministerial approval, this is not the case for the ex post accreditation of institutions, doctoral schools and medical training programmes, which is led by MAB. MAB is also only appointed by the OH for the ex ante evaluation of new (doctoral) programmes; all other processes are initiated by MAB (for ex post review) and the HEIs themselves (for university professor applications). Institutional site visits and the preparation of a self-evaluation report by HEIs is also only part of institutional, doctoral schools and medical training accreditation. Full details on each accreditation procedure are presented in Table B.1 (Annex B).
Source: Adapted from MAB (2022b[48]), MAB Eljárások (MAB Procedures), https://www.mab.hu/eljarasok/.
A first observation made by higher education stakeholders interviewed by the OECD review team is that the various steps underpinning MAB’s accreditation procedures are a significant administrative burden for all actors involved. The two-stage ex ante programme accreditation process (requiring institutions to obtain separate study field and programme accreditation) was highlighted as the process most in need of simplification. The accreditation procedures in general also require multiple interactions between the OH, MAB, HEIs and the Ministry, making this a burdensome process. As stated earlier in this section, MAB is keen to simplify the existing programme accreditation process and to make better use of digital technology to enhance the efficiency of QA procedures in general. As mentioned by MAB’s President, Prof Dr Valéria Csépe, at a national roundtable event which took place on 31 May 2022 as part of this project, MAB wants to develop modern QA processes that are “digital, well-organised and supportive”. MAB has recently started to develop a new information system (TIR2) that will allow institutions to submit all accreditation documents in one integrated online platform.
“We would like to have a digital, well-organised and supportive QA system” (Prof Dr Valéria Csépe, President of MAB, national roundtable, 31 May 2022)
A second observation made by higher education stakeholders interviewed by the OECD team is that MAB’s programme accreditation procedures are characterised by low success rates. A thematic review of MAB’s operations9 between 2017 and 2019 (PwC, 2020[32]) found that the success rate of new study field and programme launch applications were 56% and 53% respectively. An analysis of new study field and programme launch applications between 2018 and 2021 shows that MAB evaluated 69 new study field applications, of which 33 were approved and 36 were rejected. MAB also evaluated 459 applications for the launch of new programmes, of which 237 were approved and 222 were rejected (see Table B.2, Annex B). As a consequence, MAB is required to ask almost half of all institutions to revise and re-submit their programme accreditation application documents, adding to the already very lengthy and administratively burdensome two-stage ex ante programme accreditation process. A small number of higher education stakeholders interviewed by the OECD review team questioned the motivation of reviewers, speculating that they would reject some programme applications to hamper programmatic competition.
Table 2.12 presents an overview of the main reasons for rejection of new study field and programme launch applications. This shows that the ex ante programme accreditation process puts a strong focus on ensuring the quality of programme content and inputs. The evaluation of applications for the launch of programmes in new study fields, for example, primarily consists of assessing the relevance and demand for the proposed new programme against the – rarely updated – education and learning outcome content requirements included in the Higher Education Qualifications Register (Government of Hungary, 2011a[3]).10 The second stage consists of assessing programmes against 24 requirements (see Table 2.15), of which 20 focus on the proposed inputs for programme delivery (e.g. infrastructure, qualifications of teaching staff, educational content). The template only includes three process indicators (e.g. the proposed student support services or teaching and assessment practices) and one output indicator (publications of proposed teaching staff in the scientific discipline).
Stakeholders felt that the strong focus on programme inputs, and the lack of an ex post programme review procedure are hindering the development of institutional quality cultures in Hungary. In the past, MAB has attempted to carry out ex post reviews of study programmes in disciplinary clusters. MAB has assessed bachelor’s and master’s programmes in Economics in 32 institutions between 2017 and 2019. However, this process was discontinued as MAB did not have sufficient capacity to carry out such reviews on a more regular basis for more study fields, and there were no regulatory framework or standards to conduct ex post programme review.
The third observation made by higher education stakeholders interviewed by the OECD review team is that the recommendations emerging from the accreditation of institutions and doctoral schools, based on the ESG, are seen as highly relevant to supporting institutional quality enhancement. Stakeholders explained that both the self-evaluation reports and the site visits undertaken as part of these reviews constitute good learning experiences and an opportunity to engage the entire institutional stakeholder community in quality discussions. They felt that it would be helpful if all MAB procedures followed the ESG approach and focused more on processes and outputs.
Higher education stakeholders mentioned the introduction of accreditation based on the ESG as an important driver for directing institutions’ attention to the quality of their pedagogical practices and student support mechanisms. Dr Levente Kiss, who presented at a national roundtable event organised on 31 May 2022 as part of this project, said “MAB is our ally, as it stresses that education is important”, and thereby redirects institutions’ and instructors’ attention from their historic primary focus on research.
“MAB is our ally, as it stresses that education is important” (Dr Levente Kiss, Semmelweis University, national roundtable, 31 May 2022)
Standards and indicators for the ex post accreditation of institutions and doctoral schools
Table 2.13 and Table 2.14 provide an overview of standards used by MAB for the ex post accreditation of institutions and doctoral schools. Each standard is accompanied by a list of indicators for which HEIs are required to provide evidence in their self-evaluation report. For institutional accreditation, the template covers three parts: the general situation of the HEI (Part 1), the actions taken to increase compliance with the ESG (Part 2), and a description of the scientific, academic and educational activities of the HEI (Part 3). In the case of doctoral schools, the focus of parts 1 and 2 is the same, although the exact number and type of indicators differs. Part 3 of the template asks doctoral schools to provide miscellaneous information such as an updated list of doctoral school members, certified by the Rector, or statistical information on completion and degree award rates from the last 14 academic years.
The following observations can be made on the indicators covered by each of the templates:
Number of indicators. For institutional accreditation, HEIs are required to provide evidence on 93 indicators and doctoral schools on 36 indicators. Stakeholders interviewed by the OECD review team mentioned that the amount of evidence to be provided in the evaluation template, while relevant, is highly time-consuming. They therefore recommended that MAB considers simplifying the template by reducing the total number of indicators and focus areas, especially for those institutions that have already obtained positive accreditation.
Level and focus of indicators. The majority of the template (80 indicators for institutional accreditation; 28 for doctoral schools accreditation) focuses on actions taken by the HEI to increase compliance with the ESG. The areas assessed by MAB in this part of the template are comprehensive, including input, process and output indicators at the institution, programme, course and individual instructor/learner level. However, except for ESG standards 1.7 (Information management) and 1.8 (Public information), the standards do not include any specific e-learning considerations. The reason for this is that the ESG – which are used by MAB as a guideline – have been designed with broad applicability to “all higher education offered in the EHEA regardless of the mode of study or place of delivery” (ENQA, 2015, p. 9[27]).
Evidence. The evidence MAB asks institutions to provide in their self-evaluation report is primarily qualitative in nature. Institutions are only asked under ESG standards 1.2 and 1.9 to specify the number of courses that are reviewed per semester and study cycle. However, HEIs have the option to submit additional data to MAB to supplement their self-evaluation report.
Stakeholders interviewed by the OECD review team explained that compliance with ESG Standard 1.1 (Policy for quality assurance) is the only mandatory requirement for institutions to obtain accreditation. In cases where institutional QA policies exist but are deemed insufficiently comprehensive (e.g. an overall QA system is in place, but there are insufficient policies to support teaching staff or students), an institution can be “accredited with monitoring arrangements”. This means that, during its five-year accreditation period, the institution will be required to undergo an interim evaluation by MAB.
Standards and indicators for the ex ante accreditation of programmes
Table 2.15 presents the standards applied by MAB for the accreditation of new bachelor’s and master’s programmes in already established study fields (i.e. the second stage of the programme accreditation process). With the exception of Part III (Sufficient scientific expertise), which only applies to master’s programmes, both bachelor’s and master’s programmes are required to broadly meet the same requirements – although there are some subject-specific differences in the accreditation templates for different disciplines (e.g. History or Economics).
The following observations can be made on the indicators covered by the templates:
Number of indicators. The application template includes 34 requirements that must be met before institutions can launch a new master’s programme, or 32 in the case of bachelor’s programmes. For certain disciplines, the application template includes additional requirements in relation to the content of the study programme. Higher education stakeholders interviewed by the OECD review team commented that the template is difficult to complete and the type of information to be provided is often unclear. As a result, many applications are rejected by MAB (as discussed earlier in this section). Stakeholders mentioned better guidance and a simplification of the ex ante programme accreditation requirements as potential options to make it easier for HEIs to launch new study programmes and remain competitive in an increasingly international higher education landscape.
Level and focus of indicators. The application template focuses primarily on input indicators, such as the proposed programme content (Part I), infrastructure (Part IV) or the qualifications of teaching staff (Part II). The template only includes one output criterion, which relates to the scientific output of the proposed teaching staff for master’s programmes (Part III). Finally, only three process indicators under Part I (Programme content) ask institutions to describe how the programme will ensure the implementation of effective and varied teaching practices, as well as high-quality practical teaching and student evaluation. In Part VII (Special provisions for distance learning), four process indicators seek to ensure that institutions adopt tailored academic models, teaching resources, grading and student evaluation protocols for the delivery of distance learning programmes.
Evidence. While most of the template asks institutions to provide qualitative information on the programme content, policies and processes, several more quantitative indicators seek to verify that the institution has a sufficient number of qualified teaching and administrative staff, as well as realistic expectations on the number of students in the programme.
Table 2.16 presents the personnel requirements for programme management and delivery. Of note is that the template includes no specific requirements on student-teacher ratios. For example, there are no upper or lower limits provided for the requirements to ensure “sufficient numbers of teaching and support personnel” and “locally-based teaching staff”. Instead, institutions have to specify the maximum number of students they will accept in the programme and, based on this estimate, justify the proposed number of administrative and teaching staff. By contrast, for distance learning programmes there is a specified maximum of 50 students per instructor. In the case of programmes delivered fully asynchronously and online, stakeholders felt that this upper limit might be too low and might therefore be limiting the further development of digital higher education in Hungary.
Special provisions for the ex ante accreditation of distance learning programmes
Institutions that wish to offer programmes in distance learning format must meet several requirements in addition to those that apply to in-person study programmes. Table 2.17 provides an overview of these special provisions, with a more detailed description of each indicator as follows:
Indicator 1: Content and unit responsible for managing the distance learning programme. Under this indicator, institutions are asked to explain the organisational structure, logistics and processes used to manage the distance learning programme (e.g. the instructional technology and LMS/VLE used, the student supports provided). Institutions also need to submit an adapted curriculum for distance learning students, as well as explain the process for ongoing curriculum development and renewal. Students should also be provided with a study guide for the entire duration of the distance learning programme, including semester-based guidelines that indicate mandatory and optional (printed and online) study content and media.
Indicator 2: Teaching resources. This indicator asks institutions to submit one sample online module per course plus sample course guidelines, as well as explain how the institution will ensure ongoing access to teaching materials. Some stakeholders interviewed by the OECD review team highlighted that this requirement is too demanding, as it is not always possible for HEIs to have developed digital educational content for all courses before they start. Often, instructors develop the content of their courses on a rolling basis, throughout the academic year and based on feedback from students on their specific learning needs.
Indicator 3: Grading and student evaluation. Here, institutions are asked to describe how they will ensure trusted and authentic remote (online) assessment. In line with national regulation, student assessment should form an integral part of the curriculum and be adjusted to meet individual learning needs (i.e. a mix of formative and summative assessment). The final exam should take place in person at the institution and the examination committee should include an external and reputable member that does not have a legal relationship with the institution. Typically, this is an expert from another Hungarian HEI. Stakeholders interviewed by the OECD review team said that the requirement for students to take the final exam in person was a significant barrier to the further development of fully online study programmes in Hungary and to attracting remote international students. However, institutions require guidance on how to effectively conduct student assessments online.
Indicator 4: Academic consultations. This indicator asks institutions to explain how distance learning students will be provided with opportunities to consult with academic staff during their studies (e.g. through a consultation centre or regular contact hours established in the distance learning curriculum).
Indicators 5-7: Teaching staff. Three indicators focus on the qualifications and responsibilities of distance learning teaching staff. First, a dedicated full-time or part-time staff member should be appointed to oversee the content of the entire distance learning programme. Distance learning programmes should also be managed by a staff member with at least five years of distance learning experience. For institutions that are just starting to introduce digital education, the vast majority of stakeholders interviewed by the OECD review team saw this requirement as almost impossible to meet. For many HEIs in Hungary, the COVID-19 pandemic was the first time they had started experimenting with online and hybrid education, meaning very few HEIs have staff that meet this requirement. Finally, instructors cannot be responsible for more than 50 students or more than three courses per semester. This requirement was felt to be inappropriate for fully online or hybrid programmes where the online components are delivered asynchronously, as asynchronous online instruction allows courses to be opened up to a much higher number and more diverse range of students.
Indicators 8-9: Digital infrastructure. Under these indicators, institutions should provide details on the (digital) infrastructure used to deliver the distance learning programme, as well as how it will be reviewed and developed. However, few details are included on the type(s) of digital tools and technologies that institutions should consider implementing or supporting. More guidance on good quality digital tools and resources that are secure and compatible with the existing institution and national-level infrastructure were highlighted as important by higher education stakeholders.
Indicator 10: Consultation centre. Finally, institutions that wish to launch a distance learning programme need to have in place a dedicated consultation centre for distance learning students that will provide them with access to technical support, teaching materials and any other supports they might need to complete their programme at a distance.
Higher education stakeholders interviewed by the OECD review team said that on the one hand, some of the distance learning indicators are too demanding for institutions (e.g. the requirement to present a sample online module for each course of the distance learning programme, or the requirement for distance learning programme managers to have five years’ distance learning experience). On the other hand, some are not detailed enough (e.g. the digital infrastructure and student support requirements). Others were felt to be inappropriate or limiting (e.g., the threshold of 50 students and three courses for distance learning teaching staff). They also underlined that the current provisions only apply to fully online study programmes, and that there is a need to revise the existing standards to also reflect the specificities of hybrid education.
Quality enhancement of higher education in Hungary
In addition to accrediting institutions and programmes based on the ESG (ENQA, 2015[27]) and WFME (MAB, 2021b[28]) standards, in line with international best practice across the OECD, MAB has also started to implement a range of quality enhancement-oriented activities to more actively support institutions to build their capacity for the internal quality management of their (digital) education offerings.
Collection and dissemination of best practices
In line with international practice, MAB has been publishing all its accreditation reports and decisions on its website since 2006. In addition to increasing the transparency of its procedures, MAB stakeholders interviewed by the OECD review team explained that the publication of these reports serves as a tool for HEIs to learn about each other’s internal QA systems. However, the higher education stakeholders interviewed by the OECD review team noted that few practitioners consult the accreditation reports from other institutions. They felt that it might be more helpful to have guidelines and best practices distilled from accreditation reports, based on a transversal thematic analysis of institutional quality management practices, co-ordinated by MAB in collaboration with HEIs and external experts.
In 2020, MAB launched the Hungarian Accreditation Review, an online journal published twice a year with the aim of more regularly informing institutions on MAB’s activities and international QA developments (MAB, 2022c[33]). As an example of content, the first issue explains how MAB’s procedures for the accreditation of institutions and doctoral schools work, as well as the timing and process for submitting applications for university professor status. It also explains the international QA landscape within which higher education in Hungary functions (e.g. the ECTS credit system, ENQA, the Bologna process) as well as key findings from PwC’s thematic review of MAB’s activities between 2017 and 2019 (PwC, 2020[32]). Stakeholders interviewed by the OECD review team felt that more regular engagement by MAB (with the support of external experts and HEIs) in thematic analyses such as these, including on the topic of digitalisation, would be beneficial to support them.
Training and peer learning
During the COVID-19 pandemic, MAB organised several online knowledge-sharing webinars for HEIs, focused on topics relevant to the sector. Examples include online webinars organised with ENQA (27 January 2021) and DEQAR (16 February 2022), as well as a webinar focused on QA in the European Universities Initiative (EUI) (9 March 2022). As part of the current project, two online webinars were organised on the QA of digital higher education in Hungary (31 May 2022) and internationally (14 June 2022), as well as a national roundtable in Budapest to discuss policy options for the QA of digital higher education in Hungary (4 October 2022).
MAB has also been involved in supporting Hungarian HEIs to join the EUI. For example, following the successful application of 11 universities during the first EUI call, MAB started negotiations on the QA of these new joint programmes in 2019. In February 2020, Tempus Public Foundation, in collaboration with the higher education policy field, organised a workshop for institutions taking part in the first and second call of the EUI, which also involved MAB: “the main focus was on bridging the Hungarian legislative restrictions and the flexible approaches needed for the international university model” (MAB, 2021d, p. 9[30]).
Specific quality enhancement for digital higher education
However, compared with other QA agencies in the OECD, the majority of MAB’s activities are QA-oriented (i.e. focused on checking that institutions and programmes meet minimum requirements laid out in national regulation). Furthermore, with the exception of the events organised as part of the current OECD project, none of the QE-oriented activities carried out to date focus specifically on the topic of digitalisation. One of the reasons for this might be the lack of in-house expertise on digitalisation as well as a lack of capacity for MAB to organise such activities, due to the large volume of QA activities it is responsible for.
Table 2.18 compares the QA and QE activities implemented by MAB with those of the Quality Assurance Agency (QAA) in the United Kingdom (UK) and the Quality Agency for Higher and Vocational Education in Estonia (HAKA).
Key barriers for the further development and quality enhancement of digital higher education in Hungary
Based the analysis and stakeholder consultations conducted by the OECD review team, two key barriers for the further development and quality enhancement of digital higher education in Hungary emerge:
The existing set of study format hinders the development of digital higher education; and
There is a lack of up-to-date definitions, standards, and indicators for digital higher education.
Existing set of study formats hinders the development of digital higher education
A first key barrier to the further development of digital higher education in Hungary is the existing categorisation of study formats. These do not reflect an up-to-date understanding of how teaching and learning takes place in today’s digital world. Digitally savvy secondary school graduates who have lived through remote instruction during the COVID-19 pandemic, as well as adult learners in search of flexible (and often online) opportunities for upskilling and reskilling, are entering higher education with expectations of increased flexibility to decide on what, how, where, and when to study. They also expect – and deserve – to receive the same quality of instruction and support, regardless of their chosen study mode.
As evidenced by the name – “regular training” -- Hungary’s study format regulations are based on the view that full-time study on weekdays, during the day, and on a face-to-face basis, is the normative or default study mode. Part-time and distance forms of education are, according to this view, to be offered exceptionally to learners who are unable to study on a “regular” basis, while hybrid study programmes do not even fall within the range of permissible study formats.
As a result, the total share of accredited distance learning programmes in Hungary has remained low. In addition, prior to the COVID-19 pandemic, the (effective) use of digital technologies by instructors was very limited in Hungary (Eurydice/EACEA/EC, 2019[57]; Hülber, Papp-Danka and Dringó-Horváth, 2020[58]). However, this picture has changed, though, and today digital education has emerged across all HEIs in Hungary. Although it is difficult to define the While precise figures on the full offer exact number of online and hybrid study programmes available in Hungary is lacking, there is evidently a need for this calls for a deep reconceptualisation of how higher education study is organised and regulated.
Lack of up-to-date definitions, standards, and indicators for digital higher education
A second key barrier is the near absence of digital considerations in the minimum operating requirements for HEIs as well as the standards and indicators employed by MAB for the external QA of higher education providers and programmes. With the exception of the March 2020 requirement that HEIs should have a VLE/LMS in place, the minimum operating requirements for universities, UAS and university colleges do not otherwise include any specific requirements related to their capacity to deliver digital education. The ESG, which Hungarian HEIs are required to follow for the development of their internal quality management policies and processes, and which are used by MAB for the external QA of HEIs and doctoral schools, also do not include any specific education indicators. The guidelines apply broadly to “all higher education offered in the EHEA regardless of the mode of study or place of delivery” (KIM, 2016[59]).
Specific standards for digital education can only be found in MAB’s procedures for the accreditation of distance learning programmes. Institutions that wish to offer distance learning programmes are required to meet ten criteria (or, “special provisions”) in addition to those that apply to regular programmes. These criteria are used by MAB as part of ex ante programme accreditation. Stakeholders from HEIs interviewed by the OECD review team felt that the distance learning criteria used by MAB are sometimes either too burdensome (e.g. institutions are required to present a sample online module for each course of the distance learning programme, distance learning programme managers must have five years’ distance learning experience), or too limiting (e.g. maximum of 50 students per distance learning programme, three courses per distance learning teaching staff), while in other instances they provide less guidance than is necessary (e.g. on digital infrastructure and student support requirements).
If institutions in Hungary are to expand their digital education offers and deliver high-quality digital education, there will need to be significant modernisation to enhance how teaching and learning in general takes place in Hungarian higher education. This will require Hungary to revise its overarching regulatory and external QA systems for higher education, to ensure that they provide institutions and instructors with the flexibility they need to develop innovative and digitally enhanced study programmes that permit students to more flexibly choose when, where and how to study, and allow academic instructors to make better use of the potential of digital technologies. It will also be necessary to ensure that the QA framework for higher education sets relevant and up-to-date quality standards that reflect specific considerations for digital education.
This section presents examples of international practice from which Hungary could take inspiration, as well as two proposed policy recommendations Hungary should consider adopting as a matter of priority to boost study flexibility, innovation and digitalisation in its higher education system.
Revise study format regulations to increase the flexibility and diversity of study modes and the provision of digital education
If Hungary wishes to expand its digital higher education offer, it will be necessary to update its definition and conceptualisation of digital education in the categorisation of higher education study formats. At present, digital higher education is narrowly understood as distance (or fully online) education, and entirely different or separate to in-person forms of study. While digital education requires different methodological considerations, such a definition of digital education is problematic, as it suggests a binary opposition between online and in-person learning. As mentioned in the introduction of this report, more often than not the two modes are combined in practice, and there is – or soon will be – no fully in-person instruction that is not supported in some way by digital technologies, such as a VLE/LMS or Open Education Resources (OER) (Gourlay, 2021[60]), (D’Agostino, 2022[61]). As outlined in the introduction of this report, there are three broad categories of digital education:
Blended education refers to a study mode where courses are intentionally designed to harness the capacities of digital technology, using it to enrich rather than substitute in-person instruction. For example, a language or mathematics course delivered on campus might use learning analytics to adapt problem sets to learner abilities. Importantly, most instruction continues to take place on a physical campus.
Hybrid education refers to a study mode where instruction involves a mix of on-campus and off-campus instruction. Learners have some flexibility regarding the location in which they complete their study. For example, learners might complete laboratory segments of an engineering course on campus, while participating in lecture-based course segments through live web streaming.
Online education refers to a study mode where instruction is delivered off campus, either synchronously or asynchronously, or a combination of both. Students complete their course or programme of study at a distance, without the need for on-campus instruction.
To achieve flexibility and diversity of provision, Hungary should decouple study mode (i.e. online, hybrid, blended) and study intensity (i.e. full-time, part-time) in any revised categorisation of study formats. A decision will need to be taken on how much flexibility to allow students with regard to enrolment intensity – i.e. whether learners may study at any pace they wish – as there is evidence that studying on a less than a half-time basis can lead to higher non-completion rates (OECD, 2021a[62]). Box 2.4 provides examples of how study intensity is managed in different OECD jurisdictions.
Flemish Community of Belgium
The Flemish Community of Belgium operates a highly flexible enrolment system for higher education. Students can choose to enrol in a full degree programme (“diploma contract”), selected courses (“credit contract”) or (“exam contract”). Students can also enrol in different courses, degree programmes and exams at the same time, either at the same institution or at different institutions (Flemish Department of Education and Training, 2022[63]). However, the Flemish higher education system has high rates of initial re-orientation, slow progression and drop-out due to its system of open access to higher education (i.e. anyone who has successfully completed secondary education can enter higher education, albeit with some specific entry requirements for certain disciplines such as medicine), even in comparison to other OECD jurisdictions with similar open access and entry systems, such as the Netherlands or Austria (OECD, 2021a, p. 43[62]).
United States
In Community of Belgium, most institutions set upper and lower enrolment limits for full-time and part-time programmes, based on students’ performance. For example, Pennsylvania State University has recently raised the maximum number of credits for undergraduate programmes from 19 to 24 credits per semester.1 Students that wish to exceed the recommended credit load of 15-19 credits per semester are advised to consult with their designated academic adviser. Newly admitted students, transfer students and students not meeting a cumulative minimum GPA, cannot exceed 19 credits per semester (PennState, 2022[64]).
United Kingdom and Ireland
The United Kingdom and Ireland have strict national definitions for full-time and part-time study, including strict entry requirements for higher education, and relatively structured study paths to mitigate the risk of study delays and student drop-out (OECD, 2021a[62]).
← 1. One US credit point equals two ECTS credits. The typical “full course load” at an American university implies 15 US credits per semester, which is equal to 30 ECTS credits at a European university.
Source: Based on a review of emerging quality standards, practices and supports for digital higher education in Staring et al. (2022[56]), “Digital Higher Education: Emerging Quality Standards, Practices and Supports”, OECD Education Working Papers, No. 281, OECD Publishing, Paris, https://www.oecd-ilibrary.org/education/digital-higher-education_f622f257-en.
Institutions in Hungary will also need to decide whether to introduce restrictions to the development of fully online and hybrid courses and programmes (i.e. whether to allow fully online education for all types of students and study fields, or whether to set some limits or access requirements), as international evidence indicates that students with poor academic backgrounds and other risk factors may struggle to complete fully online courses if they are insufficiently prepared or supported (Baum and Mcpherson, 2019[65]), (Staring et al., 2022[56]). Not all courses and programmes – especially those with a higher proportion of practical components – can be moved fully online as easily or at the same level of quality (Study International, 2020[66]). Box 2.5 presents examples of measures introduced by institutions in various OECD jurisdiction to mitigate the risk of drop-out and non-completion in fully online and hybrid courses and study programmes.
Several institutions across the OECD have introduced the successful completion of preparatory courses and/or digital skills assessments as an entry requirement for online learning, as they have sought to mitigate the risk of study delays and non-completion while expanding their digital provision. Some institutions have also experimented with the opportunities offered by digital technologies to enhance the overall online learning experience for students with the aim of lowering the risk of drop-out.
Digital skills assessment
International best practice shows that well-developed digital competencies and self-directed learning skills are crucial for students to mitigate the risk of study delays and non-completion, and ensure the successful completion of online or hybrid courses. Several institutions provide students with (online) training prior to entering fully online or hybrid courses, or assess their (digital) skills upon entry into higher education. Examples of preparatory training courses can be found at Athabasca University in Canada (Athabasca University, 2022[67]) or Dublin City University in Ireland (FutureLearn, 2022[68]). The University of Tasmania in Australia is an example of an institution that has developed an interactive online digital skills self-assessment tool for students (University of Tasmania, 2022[69]). The tool assesses seven key competencies for online learning included in a Digital Capabilities Framework for Students, developed by the institution in 2020 based on Jisc’s digital capability framework (Jisc, 2022a[70]). Based on their result, students are directed to specific training materials and courses to develop their digital skills and competencies.
Enhancing students’ (online) learning experience
Other institutions, especially in the United States, are experimenting with the potential offered by artificial intelligence (AI), virtual reality (VR) and augmented reality (AR) to create digital twin “metaversities” to offer online students an almost identical online learning experience as on campus students, including for practical seminars (D’Agostino, 2022[71]; Paykmian, 2022[72]). A ‘metaversity’ is a “portmanteau of ‘metaverse’ and ‘universities’ […] an immersive virtual reality platform where remote faculty and students don VR headsets and meet synchronously as they would on a physical campus” (D’Agostino, 2022[71]). By creating an almost identical campus experience for fully remote online and on campus students, these institutions aim to mitigate the potential risk of non-completion and drop-out due to a poorer learning experience. Some university leaders believe that “the vast majority of the schools that are going to close in the next 10 years are going to be schools […] that pay no attention to the student-life experience” (Hatch, 2022[73]).
Source: Based on a review of emerging quality standards, practices and supports for digital higher education in Staring et al. (2022[56]), “Digital Higher Education: Emerging Quality Standards, Practices and Supports”, OECD Education Working Papers, No. 281, OECD Publishing, Paris, https://www.oecd-ilibrary.org/education/digital-higher-education_f622f257-en.
A recommendation for Hungary related to embedding flexibility and digitalisation in its higher education teaching and learning architecture is as follows.
Recommendation 1: Consider allowing institutions to offer programmes in three study modes, with some limits on study intensity
In consultation with HEIs and based on the definition of digital higher education presented above, Hungary should revise the categorisation of study formats in Article 17 of the National Act on Higher Education, to clearly distinguish between three modes of study (i.e. online, hybrid and in-person/blended) and two types of study intensity (i.e. full-time, part-time). Institutions should have full autonomy to decide whether to offer courses or programmes in the online, hybrid or in-person/blended study mode, and whether to offer them on a full-time or part-time basis (within agreed definitions of full-time and part-time study).
If institutions (and students) are given greater flexibility to offer (and choose between) full-time or part-time programmes in fully online, hybrid and blended formats, institutions will need to strengthen their student support services to inform student choice and support students in successfully navigating and completing an increasingly diverse and flexible higher education offer.
In consultation with HEIs, Hungary should consider whether fully online and asynchronous online delivery in certain “high stakes” disciplines (such as medicine) or for the delivery of certain learning outcomes or courses as part of programmes (such as practical skills) is advisable, to ensure that learners continue to meet the required learning outcomes. The burden of proof for disallowing a fully online offer should rest with those proposing its exclusion. At their discretion, individual HEIs should have the opportunity to introduce additional entry requirements or measures to mitigate the risk of study delays and drop-outs, such as a requirement for students to complete a digital skills assessment or training course prior to enrolment in a fully online course, or a requirement for hybrid programmes to contain a minimum amount of on-campus instruction.
Table 2.19 provides a potential model for the revised categorisation of study modes in Hungarian higher education.
Develop specific standards and indicators for digital education, and embed them in existing quality assurance frameworks
An international mapping of emerging quality standards, practices and supports for digital higher education carried out as part of this project (Staring et al., 2022[56]) shows that, so far, only a limited number of QA agencies across the OECD and EHEA have developed specific quality standards or guidance for digital higher education and integrated them into their existing QA frameworks and procedures There appear to be two approaches to the challenge of embedding these quality standards into existing QA frameworks and procedures:
The first approach consists of embedding the specific standards for digital education as an additional set of criteria to be met by higher education providers of digital education, in addition to those that apply to traditional study modes. For example:
Campus Alberta’s Quality Council (CACQ) in Canada has developed Additional Quality Assessment Standards for Programs Delivered in Blended, Distributed or Distance Modes (CAQC, 2011[74]). Since 2021, institutions offering programmes in either of these study modes are required to meet these additional standards in addition to those that apply to in-person study modes (CAQC, 2021[75]).
Romania follows a similar approach and has developed additional standards for fully online (ARACIS, 2020[76]) and hybrid programmes (ARACIS, 2022[77]) in addition to those that apply to in-person study modes.
In some jurisdictions, for example Estonia and Spain, the specific standards for digital higher education are used for a voluntary quality review process of digital courses and programmes. In these systems, HEIs have the option to apply (and pay) for an external review of their digital course offer by an external team of digitalisation experts and receive a “quality label” upon successful assessment, but this is not mandatory (HAKA, 2020[78]; ANECA, 2022[79]).
The second approach consists of systematically integrating specific criteria for digital education across the standards included in accreditation frameworks used for in person study modes. For example:
Estonia has revised its Guidelines for Institutional Accreditation (HAKA, 2022b[80]) by including specific guidance for the implementation of the quality standards in digital contexts. Every seven years, institutions are evaluated against these standards as part of institutional accreditation.
In Australia, specific guidance on how to implement the Higher Education Standards (HES) Framework (Threshold Standards) 2021 in a digital context is provided in a separate Guidance Note on Technology-Enhanced Learning (TEQSA, 2019[81]). While the Guidance Note is not binding for institutions or formally checked as part of accreditation, the note provides a list of “risks to quality” in technology-enhanced learning (TEL), linked to the relevant HES standards. As part of institutional accreditation, the Tertiary Education and Quality Standards Agency (TEQSA) includes digital education experts in site visit teams (TEQSA, 2022[82]), and institutions are required to demonstrate how they ensure the implementation of HES standards in TEL settings.
Malta uses a similar approach. Each of the eight standards included in the national Guidelines for the Quality Assurance of Online Learning Providers (MFHEA, 2021[83]) provides an explicit link to the overarching national standards for institutional accreditation.
International and regional quality organisations, such as ENQA or the International Network of Quality Assurance Agencies in Higher Education (INQAAHE), recommend the second approach. Namely that instead of developing separate standards or procedures for the accreditation of digital higher education, QA agencies should develop and integrate specific quality indicators for digital education across the standards applied for in-person education, to make them more “multidimensional” and “multifunctional” (Staring et al., 2022[56]). The advantage of adopting such an approach, in their view, is that one common set of standards and procedures applies to all types of provision, but the standards are enhanced to reflect the specific methodological considerations for ensuring quality in digital settings. An integrated approach also recognises that, as stated earlier in this section, all instruction will (in future) at least to some extent make use of digital technology.
Researchers and practitioners from a wide range of private, non-profit, non-governmental and academic organisations active in the field of QA and (digital) education, have been fast-moving to develop quality frameworks, specifically designed to support QA agencies and HEIs with the development of specific considerations for digital higher education. An overview of such quality frameworks, which have been primarily developed to inform the institutional self-assessment of digital learning by HEIs, can be found in publications by Esfijani (2018[84]), the International Council for Open and Distance Education (ICDE) (Ossiannilsson et al., 2015[85]), and the EUA (Volungevičienė et al., 2021[86]). However, as stated by Staring et al. (2022[56]), “since the principal responsibility for quality rests with HEIs, and national standards should be informed by the work of HEIs, the standards and indicators included in these frameworks can be used as a basis by QA agencies to develop evidence- and practice-based digital education standards, to be integrated in existing QA frameworks” (Staring et al., 2022, p. 26[56]).
Several of these frameworks include a specific focus on the European context, taking into consideration the ESG, and might therefore be particularly relevant to inform the development of specific digital education standards and indicators in Hungary. In addition to this, any national guidance or standards for digital education should also take into consideration the guidance developed by institutions in Hungary.
Guidance developed by ENQA. Between 2016 and 2018, ENQA co-ordinated a Working Group to assess the relevance of the ESG for digital education. This led to the publication of the report Considerations for the quality assurance of e-learning provision, which provides a list of 36 indicators for digital education, mapped across the ESG (see Box 2.6). Importantly, the Working Group report advises that “external quality assurance considers the characteristics of e-learning in regular procedures” (Huertas et al., 2018, p. 18[87]). Among other suggestions, it recommends that QA agencies ensure institutions make specific reference to e-learning in their self-assessment reports, that site visits take place at the location where most of the institution’s technical infrastructure is located, that QA agencies include e-learning competence in the selection process of peer review experts, and that they provide training to experts prior to conducting institutional reviews. ENQA has now embarked on a revision of the ESG and, as part of this process, will build on the 2018 ENQA Working Group report to ensure that the revised set of standards and guidelines includes specific considerations for digital education.
Guidance developed with financial support from the European Commission. In recent years, the European Commission has funded several organisations to develop specific frameworks to support the QA of digital (higher) education. This includes the E-xcellence (EADTU, 2016[88]), DigCompOrg (Kampylis et al., 2015[89]) and DigCopmEdu frameworks (Redecker and Punie, 2017[90]). A more recent framework, which includes a list of considerations for assuring the quality of hybrid courses and programmes, is the European Maturity Model for Blended Education (EMBED) (Goeman, Poelmans and Van Rompaey, 2018[91]).
Guidance developed by Hungarian HEIs. In 2020, digital education experts from four HEIs in Hungary11 developed a handbook to promote and support the use of digital tools among Hungarian higher education instructors (Dringó-Horváth et al., 2020[92]), following the six domains included in the EU’s DigiCompEdu framework frameworks (Redecker and Punie, 2017[90]). The publication is available in English and Hungarian, and is the result of an annual conference series on digitalisation in higher education, launched in November 2020 and co-ordinated by the ICT Research Centre and the Centre for Continuing Education in Educational Informatics at Károli Gáspár University of the Reformed Church (Pintér, 2021[93]; KRE, 2021[94]).
Part I of the ESG includes a set of ten standards and guidelines that can be used by external QA agencies operating in the EHEA to guide their development of national standards for institutions’ internal QA processes. Across these ten standards, the ENQA Working Group report (Huertas et al., 2018[87]). provides 36 indicators for the QA of digital education:
ESG 1.1 Policies for quality assurance. Seven indicators are outlined under this standard, for example the inclusion of e-learning in the institution’s overall strategy and the involvement of remote learners in the internal QA system.
ESG 1.2 Design and approval of programmes. This standard covers six indicators, including “the institution has a clear strategy for digital innovation… E-learning programmes are aligned with the institutional mission… [and] Curricula design reflects pedagogical practices and innovation” (Huertas et al., 2018[87]). The report also recommends checking that the people involved in designing, developing and evaluating e-learning have the required academic and technical expertise, and that teaching staff are made aware of the challenges and opportunities of developing e-learning programmes. Finally, students are mentioned as key stakeholders to be consulted when developing e-learning curricula.
ESG 1.3 Student-centred learning, teaching and assessment. Nine indicators are proposed for this standard. Under this standard, the report recommends that QA agencies check the chosen teaching and learning processes, learning materials and technical infrastructure meet the aim of achieving learning outcomes, allow for e-assessment, facilitate student learning and are regularly reviewed and updated. QA agencies are also advised to check if students are made aware of e-assessment processes and plagiarism rules, and advised on how to appropriately work with online materials and behave in online environments.
ESG 1.4 Student admission, progression, recognition and certification. The three indicators proposed for this standard are: (1) (prospective) students are informed about the equipment, e-learning, digital skills and knowledge requirements; (2) students are informed about the workload and pedagogical model and (3) there is an institutional policy and procedure in place to recognise prior learning.
ESG 1.5 Teaching staff. Eight indicators are covered under this standard, including: “The teaching staff is trained and proficient in the use of learning technologies and e-assessment methods… The institution has developed procedures to identify the support requirements of the teaching staff… [and] Technological and pedagogical support services for teachers are adequate, accessible, and timely” (Huertas et al., 2018[87]). The report also recommends that QA agencies assess whether institutions monitor student-staff ratio to keep teachers’ workload manageable, as well as assessing staff hiring and recruitment procedures.
ESG 1.6 Learning resources and student support. Five indicators are outlined under this standard, including: “The VLE supports a variety of methods and tools … The technical infrastructure ensures the accessibility of the e-learning programme by students with special educational needs … [or] The institution provides students with an adequate e-library and virtual labs” (Huertas et al., 2018[87]).
ESG 1.7 Information management. The four indicators proposed under this standard recommend QA agencies to check whether institutions adequately collect and use data to evaluate the quality of e-learning programmes, including learning analytics to track students’ performance in real time. The HEI should also have information management systems that include “relevant, updated, and reliable information concerning the institution and its programmes” and policies that consider “ethical norms and government policy with respect to data protection and the privacy of students” (Huertas et al., 2018[87]).
ESG 1.8 Public information. This standard includes four indicators. They focus on making sure that institutions publish reliable, complete and up-to-date information on: (1) study programmes, (2) technical supports, (3) technical requirements to use the system and (4) completion rates, pass rates and drop-out rates.
ESG 1.9 Ongoing monitoring and periodic review of programmes. The four indicators under this standard advise QA agencies to assess whether: e-learning programmes are regularly reviewed, updated and improved; pedagogical developments are aligned with institutional strategy; information and communication technology (ICT) and pedagogy developments are analysed and implemented; and the internal quality assurance system takes into account feedback from key stakeholders (especially students).
ESG 1.10 Cyclical external quality assurance. The report recommends including the assessment of e-learning in external QA procedures in the same way as for provision through other means. It recommends institutions contact their respective QA agencies regarding their e-learning provision and start a process of exchange of information and collaboration for the development of sector-wide accepted standards and processes for the QA of digital education.
Source: Adapted from Huertas et al. (2018[87]), Considerations for Quality Assurance of E-Learning Provision, European Network for Quality Assurance in Higher Education (ENQA), Brussels, https://www.aqu.cat/elButlleti/butlleti91/articles2_en.html#.YGY_R5NKhTZ.
In addition to considering how to embed specific standards for fully online and hybrid education in existing QA frameworks, higher education systems across the OECD are also reflecting on how to embed micro-credentials in national QA frameworks. Micro-credentials are “increasingly recognised by institutions as a means to deliver more flexible and personalised pathways for learners to upskill and reskill throughout life” and are often offered as fully online courses or programmes (OECD, 2021, p. 13[95]). While an in-depth analysis on the current state of micro-credentials in Hungary, including how to embed them in the existing higher education and QA systems was outside of the scope of this project, the OECD’s 2021 Economic Survey of Hungary highlighted that HEIs in Hungary are not widely involved in adult learning, and few of them offer alternative credentials. To stimulate the development of alternative credentials, the report recommended “funding and deregulation measures” as well as “incorporating shorter learning programmes into the existing higher education framework” (OECD, 2021b, p. 86[96]).
On 16 June 2022, the EU adopted a Council Recommendation on a European approach to micro-credentials for lifelong learning and employability (Council of the European Union, 2022b[97]). In this recommendation, the EU proposes ten principles for the QA of micro-credentials and recommends EU Member States consider “integrating micro-credentials in national qualifications frameworks and systems” and assure their quality using the same standards and principles that apply to other programmes. Box 2.7 describes emerging approaches to the regulation and QA of micro-credentials in three OECD jurisdictions.
Ireland
The Irish Higher Education Authority has funded the development of micro-credentials through its Human Capital Initiative (HEA, 2020[98]). Micro-credentials are defined by Quality and Qualifications Ireland (QQI) as “minor, special purpose or supplemental award-types” that may be used by individuals “to gain exemptions from parts of, and advanced entry to, programmes leading to NFQ qualification” and to “record the acquisition of specific skills needed by individuals, e.g. for work” (QQI, 2021a, p. 5[99]). Micro-credentials are seen, in Ireland, as alternative credentials oriented to both the labour market and educational advancement. While this definition does not include a clear upper or lower limit for micro-credentials, their value typically ranges between 10 and 30 ECTS credits (QQI, 2021b[100]).
New Zealand
In 2018, the New Zealand Qualifications Authority (NZQA) created a QA system for micro-credentials, by defining them in specific regulations and setting quality standards (New Zealand Qualifications Authority, 2018[101]). In 2019, the New Zealand Tertiary Education Commission started providing funding to higher education providers for the development and delivery of micro-credentials. Micro-credentials range in size between 5 and 40 credits (equivalent to 2.5-20 ECTS credits) and serve to reskill and upskill the labour force. They require compulsory employer involvement and, to obtain recognition by the NZQA, HEIs need to demonstrate that they do not duplicate an existing programme offer (OECD, 2021[95]).
Australia
Australia adopted a National Microcredentials Framework in March 2022 to guide learners, instructors and providers in the development and delivery of micro-credentials. The Framework defines microcredentials as “a certification of assessed learning or competency, with a minimum volume of learning of one hour and less than an AQF award qualification, that is additional, alternate, complementary to or a component part of an AQF award qualification” (Government of Australia, 2022, p. 9[102]). Among other elements, the Framework establishes critical information requirements, and outlines a minimum standard for providers to apply as they develop and deliver micro-credentials that will sit on the Microcredentials Marketplace. The Microcredentials Marketplace, released as MicroCred Seeker in December 2022, is a nationally consistent platform that allows student to search and compare higher education micro-credentials and understand how they can be stacked and used for credit towards a complete qualification. The Marketplace connects providers with learners, employers and industry groups to facilitate lifelong learning and meet emerging workforce demands.
Source: Adapted from OECD (2021[95]), Quality and value of micro-credentials in higher education : Preparing for the future, OECD Publishing, Paris, https://www.oecd-ilibrary.org/education/quality-and-value-of-micro-credentials-in-higher-education_9c4ad26d-en.
A recommendation for Hungary to related to the adaptation of its existing accreditation and QA frameworks to digital education is as follows.
Recommendation 2: Develop specific indicators for digital education and embed them in existing accreditation frameworks by systematically integrating them across all standards
Develop a Working Group of national and international digital higher education experts, responsible for the development of revised assessment frameworks to be used by MAB for its accreditation procedures. The Working Group should consist of experts representing as wide a range of higher education training profiles and disciplines as possible, as well as representatives from national-level higher education, stakeholder representatives and supporting organisations (e.g. national student union, Erasmus+ national agency, academies of sciences, etc.). The same group of experts could – in future – be appointed as external members of MAB (appointed for a specific cycle) and be involved on a regular basis in Disciplinary Committees or site visit teams for the accreditation of institutions, doctoral schools and study programmes.
In collaboration with HEIs, the Working Group on Digital Higher Education analyses the standards and indicators included in international quality frameworks for digital higher education, especially those identified in the paper Digital Higher Education: Emerging Quality Standards, Practices and Supports (Staring et al., 2022[56]) developed as part of this project, the ESG (ENQA, 2015[27]), and the existing frameworks for institution and programme accreditation used by MAB. Based on this analysis, the Working Group identifies relevant standards and indicators for the QA of digital education in Hungary at institution, programme, course and individual learner/instructor level, and advises on how they can be embedded in the existing frameworks.
Prior to finalising these standards and indicators, MAB could conduct pilot reviews of a small sample of fully online and hybrid study programmes, as well as institutions with a high number of fully online and hybrid courses and programmes, to assess the suitability of the updated assessment frameworks and make adjustments where necessary prior to rolling them out across all accreditation procedures.
Potential standards and indicators for the quality assurance of digital higher education providers in Hungary
This section illustrates how the existing assessment frameworks used by MAB could be revised to reflect specific considerations for digital education. As demonstrated below, such a revision does not necessarily require major changes. As well as adding a limited number of indicators for digital education, small revisions to the phrasing or wording of the existing standards and indicators can be sufficient to reflect the specificities of digital education. It is important to note that the additional and revised standards and indicators presented in this section are indicative only and should be used as a starting point for a more comprehensive revision, led by a dedicated Working Group of experts (as per Recommendation 2).
Options for embedding specific considerations for digital education in the minimum operating requirements of higher education institutions in Hungary
Higher education providers in Hungary are not currently expected to meet any specific minimum requirements related to their capacity to deliver digital education. To address this gap, one option for Hungary is to develop an additional requirement or standard related to HEIs’ capacity for digital delivery, pedagogical innovation and study flexibility, consisting of three indicators (see Table 2.20).
Institutional capacity for digital delivery: The first indicator consists of ensuring that HEIs have the required digital learning resources and virtual learning environments in place (e.g., institution-wide VLE/LMS or electronic access to digital library resources) to support the type(s) of digital courses and study programmes they wish to offer (i.e. online, hybrid and/or in person/blended).
Institutional capacity for pedagogical innovation: The second indicator focuses on instructors’ pedagogical skills and institutional supports to build the capacity of instructors and students to effectively use digital technologies for pedagogical innovation.
Institutional capacity for flexible delivery: The third indicator seeks to ensure that HEIs have a flexible and adapted (digital) course offer that meets the needs of its targeted student population.
Options for embedding specific considerations for digital education in the institutional accreditation template in Hungary
Building on the review of standards and indicators for the QA of digital higher education included in Staring et al. (2022[56]), Table 2.21 presents a potential model of embedding specific indicators for digital education across the institutional accreditation template currently being used by MAB. 24 additional indicators are proposed, as well as small revisions to the wording of existing indicators across all parts of the template.
The general situation of the institution (Part I): This part of the framework could be enhanced by including two additional indicators for digital education, drawn from ENQA’s Considerations for the quality assurance of e-learning provision (Huertas et al., 2018[87]). The first proposed indicator recognises the importance of alignment between digital capacity and the institution’s mission and overall strategy. The second emphasises the crucial role of leadership and management in developing strategic plans, defining performance indicators and influencing the overall quality culture across the institution.
Compliance with the ESG (Part II): This part of the framework already lists over 80 elements, meaning the scope to add a comprehensive list of additional requirements specific to digital education is limited, and this needs to be weighed up against the additional cost and time required to be compliant (for both HEIs and MAB). However, an analysis of the indicators included in the current framework reveals several significant gaps in relation to digital education. Table 2.21 illustrates how some of these gaps could be addressed with the inclusion of 24 additional quality indicators, as well as rewording some of the existing indicators (the proposed revisions to existing indicators is emphasised In bold and italics).
The academic, scientific and educational activities of the institution (Part III): In this part of the template, one additional indicator is proposed, which recognises institutional engagement in professional bodies, partnerships and educational alliances that help to benchmark best practice in digital higher education.
References
[79] ANECA (2022), Quality Label for Distance Learning and Hybrid Education, National Agency for Quality Assessment and Accreditation (ANECA), Madrid, http://www.aneca.es/eng/Evaluation-and-reports/Programme-evaluation-procedure/SIC/By-Distance-Learning-and-Hybrid-Education-modality (accessed on 19 December 2022).
[77] ARACIS (2022), Standarde de calitate cu privire la modul de desfăşurare a activităţilor de predare, învăţare, cercetare, aplicații practice şi evaluare1 , la forma de învățământ cu frecvență, prin utilizarea unor resurse electronice, informatice şi de comunicaţii sincrone specifice [Quality standards on how to carry out teaching, learning, research, practical application and evaluation in the form of education, through the use of electronic, IT and other resources, specific to synchronous instruction], Romanian Agency for Quality Assurance in Higher Education (ARACIS), Bucharest, https://www.aracis.ro/wp-content/uploads/2022/08/Standarde-de-calitate-modul-mixt-05.08.pdf (accessed on 19 December 2022).
[76] ARACIS (2020), Methodology and Guidelines on External Quality Evaluation in Higher Education in Romania. Part VI: Specific Standards and Guidelines on External Evaluation of the Quality of Distance Learning (DL) and Part-Time Learning (PTL) Degree Programmes, Romanian Agency for Quality Assurance in Higher Education (ARACIS), Bucharest, https://www.aracis.ro/wp-content/uploads/2021/11/Result-1.-Part-VI-METHODOLOGY-DISTANCE-LEARNING-EN.pdf (accessed on 19 December 2022).
[67] Athabasca University (2022), Learning to Learn Online, http://www.ltlo.ca/ (accessed on 19 December 2022).
[65] Baum, S. and M. Mcpherson (2019), “The Human Factor: The Promise & Limits of Online Education”, American Academy of Arts & Sciences, pp. 235-254, https://doi.org/10.1162/DAED_a_01769 (accessed on 19 December 2022).
[75] CAQC (2021), Handbook: Quality assessment and quality assurance., Campus Alberta Quality Council, Edmonton, https://caqc.alberta.ca/media/6122/m-alcp-caqc-publications-caqc-handbook-caqc-handbook-2020-handbook_with-revisions-to-june-2021.pdf (accessed on 19 December 2022).
[74] CAQC (2011), Additional quality assessment standards for programs delivered in blended, distributed or distance modes, Campus Alberta Quality Council , Edmonton, https://caqc.alberta.ca/media/1092/caqc_distance_program_standards.pdf (accessed on 25 July 2022).
[37] Council of the European Union (2022a), Council Recommendation on building bridges for effective European higher education cooperation, Official Journal of the European Union, Brussels, https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32022H0413(01)&from=EN (accessed on 19 December 2022).
[97] Council of the European Union (2022b), Council recommends European approach to micro-credentials - Consilium, Brussels, Council of the European Union, https://www.consilium.europa.eu/en/press/press-releases/2022/06/16/council-recommends-european-approach-to-micro-credentials/ (accessed on 19 December 2022).
[71] D’Agostino, S. (2022), “College in the metaverse is here. Is higher ed ready?”, Inside Higher Ed, https://www.insidehighered.com/news/2022/08/03/college-metaverse-here-higher-ed-ready?utm_source=Inside+Higher+Ed&utm_campaign=99353111a4-DNU_2021_COPY_02&utm_medium=email&utm_term=0_1fcbc04421-99353111a4-236921490&mc_cid=99353111a4&mc_eid=30e212460c (accessed on 19 December 2022).
[61] D’Agostino, S. (2022), Online learning leaders think fully in-person will be a rarity, https://www.insidehighered.com/news/2022/08/09/online-learning-leaders-think-fully-person-will-be-rarity (accessed on 19 December 2022).
[23] Demcsákné Ódor, Z. and P. Huszárik (2020), Lemorzsolódási vizsgálatok a felsőoktatásban: Összefoglaló tanulmány [Attrition studies in higher education: a synthesis study], Educational Authority (OH), Budapest, https://www.oktatas.hu/pub_bin/dload/felsooktatas/projektek/fir/EFOP345_FIR_LEMORZSOLODAS_VIZSGALAT_tanulmany.pdf (accessed on 19 December 2022).
[8] Derényi, A. (2020), “Az intézményi működési keretek átalakítási kísérletei a magyar felsőoktatásban [Attempts to transform the institutional operating framework in Hungarian higher education]”, Opus et Educatio, Vol. 29/1, pp. 64-77, http://epa.oszk.hu/01500/01551/00111/pdf/EPA01551_educatio_2020_01_064-077.pdf (accessed on 19 December 2022).
[92] Dringó-Horváth, I. et al. (2020), Az oktatásinformatika módszertana a felsőoktatásban (Educational Technology in Higher Education – Methodological Considerations), Károli Gáspár Református Egyetem IKT Kutatóközpontja, Budapest, https://btk.kre.hu/images/ikt/oktatasinformatika_a_felsooktatasban.pdf (accessed on 13 July 2022).
[2] DSN/DHECC (2020), Position Paper on Digitalisation of Hungarian Higher Education, Digital Success Nonprofit Ltd. (DSN)/Digital Higher Education Competence Centre (DHECC), Budapest, document provided to OECD for the project "Supporting the Digital Transformation of Higher Education in Hungary".
[88] EADTU (2016), E-xcellence. Quality Assessment for E-Learning: A Benchmarking Approach., European Association for Distance Teaching Universities (EADTU), Brussels, https://e-xcellencelabel.eadtu.eu/e-xcellence-review/manual (accessed on 19 December 2022).
[39] ECA (n.d.), Code of Good Practice, The European Consortium for Accreditation in Higher Education (ECA), The Hague, https://ecahe.eu/archive/code-of-good-practice/ (accessed on 19 December 2022).
[1] Educational Authority (2021), Államilag elismert magyar felsőoktatási intézmények (Hungarian higher education institutions recognised by the state), Felsőoktatási Információs Rendszer [Higher Education Information System], Budapest, https://firgraf.oh.gov.hu/tematikus-lista/magyar-felsooktatasi-intezmenyek/html/page/2/pageCount/50/orderBy/-/direction/ASC. (accessed on 19 December 2022).
[20] Educational Authority (2021), Statistics from the past years of applications and acceptence (2001-2021), Educational Authority (OH), Budapest, https://www.felvi.hu/felveteli/ponthatarok_statisztikak/elmult_evek/!ElmultEvek/index.php/elmult_evek_statisztikai/munkarendenkent (accessed on 19 December 2022).
[19] Educational Authority (2019), Felsőoktatási statisztikák 1.9 A hallgatók száma összes képzési szinten munkarend szerint (Statistics on higher education 1.9 Number of students at all levels of education by work system), Educational Authority (OH), Budapest, https://dari.oktatas.hu/fir_stat_pub (accessed on 19 December 2022).
[15] Educational Authority (2022b), Felsőoktatási intézmény működési engedélyének módosítása [Amendment to the operating licence of a higher education institution], Educational Authority (OH), Budapest, https://www.oktatas.hu/felsooktatas/hatosagi_ugyintezes/foi_intezmenyek_ugyintezes/hallgatoi_letszam/kapacitas_megallapitasa (accessed on 19 December 2022).
[16] Educational Authority (n.d.), Felvi.hu, https://www.felvi.hu/ (accessed on 19 December 2022).
[12] Educational Authority (2022a), FIRGRÁF, Educational Authority (OH), Budapest, https://firgraf.oh.gov.hu/ (accessed on 19 December 2022).
[9] EHEA (2005), The framework of qualifications for the European Higher Education Area, European Higher Education Area, Brussels, http://ehea.info/media.ehea.info/file/WG_Frameworks_qualification/85/2/Framework_qualificationsforEHEA-May2005_587852.pdf (accessed on 19 December 2022).
[31] ENQA (2018), Report of the panel of the external review of the HAC (Hungarian Accreditation Committee), European Association for Quality Assurance in Higher Education (ENQA), Brussels, https://www.mab.hu/wp-content/uploads/HAC_REVIEW_REPORT_Final_7_30_2018.pdf (accessed on 19 December 2022).
[27] ENQA (2015), Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG), European Association for Quailty Assurance in Higher Education (ENQA), Brussels, http://www.enqa.eu/wp-content/uploads/2015/11/ESG_2015.pdf (accessed on 19 December 2022).
[42] EQAR (n.d.), Database of External Quality Assurance Results (DEQAR), European Quality Assurance Register for Higher Education (EQAR), Brussels, https://www.eqar.eu/qa-results/search/by-institution/ (accessed on 19 December 2022).
[43] EQAR (n.d.), Enhancing the Coverage and Connectivity of QA in the EHEA (DEQAR CONNECT), European Quality Assurance Register for Higher Education (EQAR), Brussels, https://www.eqar.eu/about/projects/deqar-connect/ (accessed on 19 December 2022).
[84] Esfijani, A. (2018), “Measuring Quality in Online Education: A Meta-synthesis”, American Journal of Distance Education, Vol. 32/1, pp. 57-73, https://doi.org/10.1080/08923647.2018.1417658 (accessed on 19 December 2022).
[21] European Commission (2020), Education and Training Monitor 2020, Publications Office of the European Union, Luxembourg, https://op.europa.eu/webpub/eac/education-and-training-monitor-2020/countries/hungary.html (accessed on 19 December 2022).
[57] Eurydice/EACEA/EC (2019), Digital Education at School in Europe, Publications Office of the European Union, Luxembourg, https://op.europa.eu/en/publication-detail/-/publication/d7834ad0-ddac-11e9-9c4e-01aa75ed71a1/language-en (accessed on 19 December 2022).
[63] Flemish Department of Education and Training (2022), Inschrijving en contracten - voor studenten, https://onderwijs.vlaanderen.be/nl/studenten/toelating-en-inschrijving/inschrijving-en-contracten (accessed on 19 December 2022).
[68] FutureLearn (2022), A Digital Edge: Essentials for the Online Learner, https://www.futurelearn.com/courses/a-digital-edge-essentials-for-the-online-learner (accessed on 19 December 2022).
[91] Goeman, K., S. Poelmans and V. Van Rompaey (2018), Research report on state of the art in blended learning and innovation. European Maturity Model for Blended Education (EMBED), European Association of Distance Teaching Universities (EADTU), Maastricht, https://www.researchgate.net/project/EMBED-European-Maturity-model-for-Blended-Education-EMBED (accessed on 19 December 2022).
[60] Gourlay, L. (2021), “There Is No ’Virtual Learning’: The Materiality of Digital Education”, Journal of New Approaches in Educational Research, Vol. 10/1, p. 57, https://doi.org/10.7821/naer.2021.1.649 (accessed on 19 December 2022).
[102] Government of Australia (2022), National Microcredentials Framework, Australian Department of Education, Melbourne, https://www.education.gov.au/higher-education-publications/resources/national-microcredentials-framework (accessed on 19 December 2022).
[18] Government of Hungary (2015), 87/2015. (IV. 9.) Korm. rendelet a nemzeti felsőoktatásról szóló 2011. évi CCIV. törvény egyes rendelkezéseinek végrehajtásáról [Government Decree on the implementation of certain provisions of Act CCIV of 2011 on National Act on Higher Education], Government of Hungary, Budapest, https://net.jogtar.hu/jogszabaly?docid=a1500087.kor (accessed on 19 December 2022).
[7] Government of Hungary (1992), Kjt. - 1992. évi XXXIII. törvény a közalkalmazottak jogállásáról [Act on the legal status of civil servants], Government of Hungary, Budapest, https://net.jogtar.hu/jogszabaly?docid=99200033.tv (accessed on 19 December 2022).
[46] Government of Hungary (2012a), 19/2012. (II. 22.) Korm. rendelet a felsőoktatási minőségértékelés és -fejlesztés egyes kérdéseiről [Government Decree on certain issues of quality assessment and development in higher education], Government of Hungary, Budapest, https://net.jogtar.hu/jogszabaly?docid=a1200019.kor (accessed on 8 June 2022).
[47] Government of Hungary (2012b), 387/2012. (XII. 19.) Korm. rendelet a doktori iskolákról, a doktori eljárások rendjéről és a habilitációról [Government Decree on doctoral schools, doctoral procedures and habilitation], Government of Hungary, Budapest, https://net.jogtar.hu/jogszabaly?docid=a1200387.kor (accessed on 19 December 2022).
[3] Government of Hungary (2011a), Act CCIV of 2011 on National Higher Education, Government of Hungary, Budapest, https://net.jogtar.hu/jogszabaly?docid=A1100204.TV (accessed on 19 December 2022).
[4] Government of Hungary (2011b), Áht. - 2011. évi CXCV. törvény az államháztartásról [Law on Public Finances], Government of Hungary, Budapest, https://net.jogtar.hu/jogszabaly?docid=a1100195.tv (accessed on 19 December 2022).
[78] HAKA (2020), E-kursuse kvaliteedimärk [E-course quality label], Estonian Quality Agency for Vocational and Higher Education (HAKA), Tallinn, https://ekka.edu.ee/e-kursuse-kvaliteedimark/ (accessed on 19 December 2022).
[55] HAKA (2022a), Estonian Quality Agency for Education (HAKA), previously Estonian Quality Agency for Higher and Vocational Education (EKKA), Estonian Quality Agency for Education (HAKA), Tallinn, https://ekka.edu.ee/en/ (accessed on 19 December 2022).
[80] HAKA (2022b), Guidelines for Institutional Accreditation, Estonian Quality Agency for Vocational and Higher Education (HAKA), Tallinn, https://ekka.edu.ee/wp-content/uploads/IA_Guidelines.pdf (accessed on 19 December 2022).
[22] Hárs, Á. (2019), “Increasing outward migration – opportunities, hopes and labour market impacts”, in Tóth, I. (ed.), Hungarian Social Report 2019, TÁRKI Social Research Institute, Budapest, https://www.tarki.hu/sites/default/files/2019-02/137_159_Hars_elvandorlas.pdf (accessed on 19 December 2022).
[73] Hatch, B. (2022), “Why One College Is Hiring a ‘Vibrant-Campus-Community Coordinator’”, The Chronicle of Higher Education, https://www.chronicle.com/article/why-one-college-is-hiring-a-vibrant-campus-community-coordinator?cid=gen_sign_in (accessed on 4 August 2022).
[98] HEA (2020), Minister Harris announces 22 innovative projects to be funded under Human Capital Initiative | News | Higher Education Authority, Irish Higher Education Authority (HEA), Dublin, https://hea.ie/2020/10/05/minister-harris-announces-22-innovative-projects-to-be-funded-under-human-capital-initiative/ (accessed on 19 December 2022).
[87] Huertas, E. et al. (2018), Considerations for quality assurance of e-learning provision. Report from the ENQA Working Group VIII on Quality Assurance and E-Learning., European Association for Quality Assurance in Higher Education (ENQA), Brussels, https://www.enqa.eu/wp-content/uploads/Considerations-for-QA-of-e-learning-provision.pdf (accessed on 19 December 2022).
[58] Hülber, L., A. Papp-Danka and I. Dringó-Horváth (2020), “A felsőoktatás digitális oktatási kultúrájának korrajza szakértői interjúk alapján [Era of digital education culture in higher education based on expert interviews]”, Opus et Educatio, Vol. 7, pp. 302-330, https://doi.org/10.3311/ope.401. (accessed on 19 December 2022).
[38] INQAAHE (2018), Guidelines of Good Practice - Procedural Manual, International Network for Quality Assurance Agencies in Higher Education (INQAAHE), Barcelona, https://www.inqaahe.org/sites/default/files/GGP-Procedural-Manual-2018.pdf (accessed on 10 January 2022).
[70] Jisc (2022a), Building digital capabilities: The six elements defined, Joint Information Systems Committee (JISC), Bristol, https://repository.jisc.ac.uk/6611/1/JFL0066F_DIGIGAP_MOD_IND_FRAME.PDF (accessed on 20 June 2022).
[13] Kálmán, O., P. Tynjälä and T. Skaniakos (2020), “Patterns of university teachers’ approaches to teaching, professional development and perceived departmental cultures”, Teaching in Higher Education, Vol. 25/5, pp. 595-614, https://doi.org/10.1080/13562517.2019.1586667 (accessed on 19 December 2022).
[89] Kampylis, P. et al. (2015), Promoting effective digital-age learning: a European framework for digitally-competent educational organisations, Publications Office of the European Union, Luxembourg, https://joint-research-centre.ec.europa.eu/european-framework-digitally-competent-educational-organisations-digcomporg_en (accessed on 19 December 2022).
[5] KIM (2020), National Digitalisation Strategy 2021-2031, Ministry of Culture and Innovation (KIM), Budapest, https://2015-2019.kormany.hu/download/f/58/d1000/NDS.pdf (accessed on 19 December 2022).
[59] KIM (2016), Shifting of Gears in Higher Education Mid-Term Policy Strategy 2016 - Action Plan 2016-2020, Hungarian Ministry of Culture and Innovation (KIM), Budapest, https://2015-2019.kormany.hu/download/9/19/d1000/Hungarian%20Higher%20Education%20Mid-Term%20Policy%20Strategy%20-%20Action%20Plan%202016-2020.pdf (accessed on 19 December 2022).
[94] KRE (2021), Oktatásinformatika a felsőoktatásban II. – A digitális oktatás átmentett értékei 2021. október 22. [Educational Technology in Higher Education II - The Transferable Values of Digital Education 22 October 2021], Károli Gáspár University of the Reformed Church (KRE), Budapest, https://btk.kre.hu/konf/oktinfkonf_felsooktatas/2021/fooldal (accessed on 19 December 2022).
[10] KSH (2011), Az oktatási programok egységes nemzetközi osztályozási rendszere [A uniform international classification system for educational programmes], KSH, Budapest, https://www.ksh.hu/docs/osztalyozasok/isced/isced_2011_tartalom.pdf (accessed on 19 December 2022).
[25] MAB (2018), Hungarian Accreditation Committee External Evaluation 2018 - Self-Assessment Report, Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/Self-Assessment_Report.pdf (accessed on 19 December 2022).
[26] MAB (2022a), A MAB [About MAB], https://www.mab.hu/mab/ (accessed on 19 December 2022).
[41] MAB (2021a), ARACIS – MAB Cooperation, Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/en/aracis-mab-cooperation/ (accessed on 19 December 2022).
[28] MAB (2021b), Az orvosképzés akkreditációs eljárásainak dokumentumai - Értékelő lap [Documents on accreditation procedures for medical training - Evaluation sheet], Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/eljarasok/ (accessed on 19 December 2022).
[34] MAB (2020a), Complaint Management Policy of the Hungarian Accreditation Committee, Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/2020/10/HAC_Complaint-management-policy_2020_EN.pdf (accessed on 19 December 2022).
[50] MAB (2021c), Doktori akkreditációs útmutató: Önértékelési szempontrendszer [Doctoral accreditation guide: self-evaluation criteria], Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/eljarasok/ (accessed on 19 December 2022).
[30] MAB (2021d), Hungarian Accreditation Committee - Report for the year 2020 until September 2021, Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/MAB-Annual-Report-2021_v2..pdf (accessed on 19 December 2022).
[36] MAB (2020b), Hungarian Accreditation Committee Follow-up Report to the Recommendations of the Panel of the External Review of the HAC of May 2018, Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/HAC_Followup-report-2020-2.pdf (accessed on 19 December 2022).
[48] MAB (2022b), MAB Eljárások [MAB Procedures], https://www.mab.hu/eljarasok/ (accessed on 19 December 2022).
[33] MAB (2022c), MAB Publications, https://www.mab.hu/en/publications/ (accessed on 19 December 2022).
[54] MAB (2022d), Magyar Felsőoktatási Akkreditációs Bizottság [The Hungarian Accreditation Committee], Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/en/home-page/ (accessed on 19 December 2022).
[49] MAB (2021e), Önértékelési útmutató (Institutional accreditation), Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/OnertUtmut_Intakkr2021.pdf (accessed on 19 December 2022).
[51] MAB (2017a), SZAKMAI BÍRÁLATI SZEMPONTJAI (SzBSz) (osztott és osztatlan) mesterképzési szak / szakirány*, tanárszak indításának véleményezésében [PROFESSIONAL JUDGEMENT POINTS (MAB) in the assessment of the start of a Master’s degree programme (split and undivided], Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/MA_I_b%C3%ADr%C3%A1lati-szempontok.pdf (accessed on 19 December 2022).
[52] MAB (2017b), SZAKMAI BÍRÁLATI SZEMPONTJAI (SzBSz) alapképzési szak/szakirány indításának véleményezésében [COMMITTEE OF EXAMINERS OF PROFESSIONAL EXAMINATION (CEAS) for the opinion on the opening of a bachelor’s degree course/sub-discipline], Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/BA_I_b%C3%ADr%C3%A1lati-szempontok.pdf (accessed on 19 December 2022).
[17] MAB (2017c), SZAKMAI BÍRÁLATI SZEMPONTJAI (SzBSz) mesterképzési szak létesítésének, képzési és kimeneti követelményeinek (KKK) véleményezésében [Sectoral Judgment Points (SJP) on the establishment, training and outcome requirements) of a master’s degree], Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/MA_L_b%C3%ADr%C3%A1lati-szempontok.pdf (accessed on 19 December 2022).
[45] MAB (2022e), Universities in the DEQAR system, Online Webinar Series - Universities in the DEQAR system, https://www.mab.hu/en/universities-in-the-deqar-system/ (accessed on 19 December 2022).
[29] MAB (2022f), Why and how to change the program accreditation system in Hungary, Presentation by Prof Dr Valéria Csépe at the National Roundtable on Policy Options for Hungary to Assure the Quality of Digital Higher Education, https://www.mab.hu/en/publications/ (accessed on 19 December 2022).
[35] Magyar Felsőoktatási Akkreditációs Bizottság (2022), Nyilvános adatok - Testület, MAB Titkársági Információs Rendszer v3.07.3, http://tir.mab.hu/index.php?pid=830 (accessed on 7 June 2022).
[83] MFHEA (2021), Guidelines for Quality Assurance - For Online Learning Providers in Malta, Malta Further and Higher Education Authority (MFHEA), Valletta, https://mfhea.mt/wp-content/uploads/2021/10/Guidelines-for-FHEI-V1.pdf (accessed on 19 December 2022).
[44] MICROBOL (ed.) (2022), Common Framework for Micro-credentials in the EHEA, https://microcredentials.eu/wp-content/uploads/sites/20/2022/03/Micro-credentials_Framework_final-1.pdf (accessed on 19 December 2022).
[103] Milman Natalie, V. et al. (2020), “7 Things You Should Know About the HyFlex Course Model”, EDUCAUSE Learning Iniviative (ELI), https://library.educause.edu/resources/2020/7/7-things-you-should-know-about-the-hyflex-course-model (accessed on 19 December 2022).
[40] NAB, Czechia; PKA, Poland; SAAHE; MAB, Hungary (2021), Memorandum of understanding, Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/Memorandum-of-Understanding-V4QA-Forum.pdf (accessed on 19 December 2022).
[101] New Zealand Qualifications Authority (2018), Guidelines for applying for approval of a training scheme or a micro-credential, New Zealand Qualifications Authority, Wellington, https://www.nzqa.govt.nz/providers-partners/approval-accreditation-and-registration/micro-credentials/guidelines-training-scheme-micro-credential/ (accessed on 19 December 2022).
[95] OECD (2021), “Quality and value of micro-credentials in higher education : Preparing for the future”, OECD Education Policy Perspectives, Vol. 40, https://www.oecd-ilibrary.org/education/quality-and-value-of-micro-credentials-in-higher-education_9c4ad26d-en (accessed on 18 March 2022).
[96] OECD (2021b), OECD Economic Surveys: Hungary 2021, OECD Publishing, Paris, https://doi.org/10.1787/1d39d866-en (accessed on 19 December 2022).
[62] OECD (2021a), Resourcing Higher Education in the Flemish Community of Belgium, Higher Education, OECD Publishing, Paris, https://doi.org/10.1787/3f0248ad-en (accessed on 19 December 2022).
[14] OECD/EU (2017), Supporting Entrepreneurship and Innovation in Higher Education in Hungary, OECD Skills Studies, OECD Publishing, Paris/European Union, Brussels, https://doi.org/10.1787/9789264273344-en (accessed on 19 December 2022).
[85] Ossiannilsson, E. et al. (2015), The ICDE reports series Quality models in online and open education around the globe: State of the art and recommendations, International Council for Open and Distance Education (ICDE), Oslo, http://www.icde.orgCompletereport,ExecutivesummaryandAppendices:http://icde.typepad.com/quality_models/ (accessed on 19 December 2022).
[72] Paykmian, B. (2022), “10 Universities Plan ‘Digital Twin’ Metaversities for Fall”, Government Technology, https://www.govtech.com/education/higher-ed/10-universities-to-launch-digital-twin-metaversities (accessed on 19 December 2022).
[64] PennState (2022), Undergraduate students reminded of change in credit limit policy | Penn State University, https://www.psu.edu/news/office-undergraduate-education/story/undergraduate-students-reminded-change-credit-limit-policy/ (accessed on 19 December 2022).
[93] Pintér, T. (2021), “Oktatásinformatikai helyzetkép a magyarországi felsőoktatásban [State of play of educational technology in higher education in Hungary]”, Új Pedagógiai Szemle [New Pedagogical Review], Vol. 71/3-4, pp. 54-74, https://upszonline.hu/index.php?article=710304009 (accessed on 19 December 2022).
[32] PwC (2020), Thematic review of activities (2017–2020), Carried out for the Hungarian Accreditation Committee. Hungarian Accreditation Committee (MAB), Budapest, https://www.mab.hu/wp-content/uploads/Thematic-review-of-HAC-activities_deliverable.pdf (accessed on 19 December 2022).
[53] QAA (2022), The Quality Assurance Agency for Higher Education, Quality Assurance Agency for Higher Education (QAA), Gloucester, https://www.qaa.ac.uk/ (accessed on 19 December 2022).
[99] QQI (2021a), QQI early exploration into Micro-credentials in Higher Education, 2014-2020, QQI, Dublin, https://www.qqi.ie/sites/default/files/2021-10/early-exploration-into-micro-credentials-in-higher-education-2014-20.pdf (accessed on 19 December 2022).
[100] QQI (2021b), The Boom in Micro-Credentials, https://www.qqi.ie/news/the-boom-in-micro-credentials (accessed on 12 October 2022).
[90] Redecker, C. and Y. Punie (2017), European Framework for the Digital Competence of Educators. DigCompEdu., Publications Office of the European Union, Luxembourg, https://publications.jrc.ec.europa.eu/repository/handle/JRC107466 (accessed on 19 December 2022).
[56] Staring, F. et al. (2022), “Digital Higher Education: Emerging Standards, Practices, and Supports”, OECD Education Working Papers , No. 281, OECD Publishing, Paris, p. 97, https://www.oecd-ilibrary.org/education/digital-higher-education_f622f257-en.
[66] Study International (2020), Courses that are extremely difficult or downright impossible to be taught online, https://www.studyinternational.com/news/in-person-classes/ (accessed on 19 December 2022).
[24] Szlamka, E. (2015), Referencing and Self-certification Report of the Hungarian Qualifications Framework to the EQF and to the QF-EHEA, Educational Authority (OH), Budapest, https://www.oktatas.hu/pub_bin/dload/LLL/HuQF/HuQF_referencing_report.pdf (accessed on 19 December 2022).
[82] TEQSA (2022), TEQSA Register of External Experts as at 1 May 2022, Tertiary Education Quality and Standards Agency (TEQSA), Melbourne, https://www.teqsa.gov.au/information-teqsa-experts#register (accessed on 19 December 2022).
[81] TEQSA (2019), Guidance note: technology-enhanced learning. V.1, Tertiary Education Quality and Standards Agency (TEQSA), Melbourne, https://www.teqsa.gov.au/latest-news/publications/guidance-note-technology-enhanced-learning (accessed on 19 December 2022).
[11] Tolnai, Á. (2021), Quality assurance issues of blended learning courses, J. Selye University, Komárno, Slovakia, pp. 171-177, https://doi.org/10.36007/4133.2022.171 (accessed on 19 December 2022).
[69] University of Tasmania (2022), Digital skills profile - Digital Skills for Study, University of Tasmania, Tasmania, https://utas.libguides.com/digital_skills/digital_profile (accessed on 19 December 2022).
[6] Vida, C. (2021), Elemzés: Felsőoktatás a változások tükrében – verseny, minőség, teljesítmény [Analysis: Higher education in the face of change - competition, quality, performance], Állami Számvevőszék, Budapest, https://www.asz.hu/storage/files/files/elemzesek/2021/felsooktatas_valtozasok_tukreben_20210406.pdf (accessed on 19 December 2022).
[86] Volungevičienė, A. et al. (2021), Developing a High-Performance Digital Education Ecosystem: Institutional Self-Assessment Instruments, European University Association (EUA), Brussels, https://eua.eu/downloads/publications/digi-he%20desk%20research%20report.pdf (accessed on 19 December 2022).
Notes
← 1. Government Decree No. 18/2016 (VIII. 5.).
← 2. Government Decree No. 139/2015. (VI. 9.).
← 3. Government Decree No. 87/2015 (IV. 9).
← 4. The hybrid flexible or “hy-flex” education model is “an instructional approach that combines face-to-face (F2F) and online learning. Each class session and learning activity is offered in-person, synchronously online, and asynchronously online. Students can decide how to participate” (Milman Natalie et al., 2020[103]).
← 5. Eszterházy Károly Catholic University (BSc in Business Administration and Management); Gábor Dénes College (BSc in Tourism and Catering); Kodolányi János University (BSc in Human Resources BSc); Széchenyi István University (BSc in Transportation Engineering); University of Szeged (business administration and management BSc); University of Miskolc (Higher VET in Information Technology Engineering); University of Pécs (Higher VET programme in Law); University of Pannonia (MA in Educational Sciences); and Sárospatak Reformed Theological Academy (MA in Theology).
← 6. Quantifying student drop-out in Hungarian higher education is complicated, as there is no officially agreed definition on what constitutes dropping out. Evidence is also primarily collected in ad-hoc reports and research papers, which use different methodologies (Kálmán, Tynjälä and Skaniakos, 2020[13]).
← 7. “Second instance competence” refers to the authority responsible for deciding on appeals made against decisions made by the authority with first instance competence.
← 8. “First instance competence” refers to the authority acting as the first instance in the administrative/judicial procedure.
← 9. The study focused on MAB’s procedures for institutional accreditation, programme launch and establishment.
← 10. Appendix to the Government Decree No. 139/2015. (VI. 9.).
← 11. Károli Gáspár University of the Reformed Church, Budapest Business School, the University of Pécs and the Hungarian Dance Academy.