Chapter 2. Quality and relevance of higher education in Kazakhstan

This chapter focuses on the quality and relevance of higher education in Kazakhstan. It deals with the quality of student and faculty qualifications, faculty workloads and professional development, pedagogy, curriculum design and regulatory processes. It also discusses the overall outputs of higher education in light of the needs of the 21st century economy. Priority areas identified for Kazakhstan include resolving the barriers and implementation gaps in the Bologna Process and targeting inefficiencies in the current quality assurance system. Faculty development opportunities are scarce and instructional methods require improvement, while curriculum and the processes that support curricular design are not yet structured well enough to generate academic programmes of consistently high quality. The chapter stresses that the available data on student learning and the labour market outcomes of students are not sufficiently detailed to permit an extensive analysis of the quality of higher education outputs and outcomes.

Kazakhstan’s aspirations, challenges and achievements

Kazakhstan’s aspirations for its education system are clearly reflected in a number of policy statements that link education to the broader goal of becoming a world-leading nation. Three of these statements have direct implications on the way Kazakhstan conceives higher education quality:

The 2050 Strategy, which highlights the critical role higher education plays in preparing a skilled workforce.
The State Programme for Education Development (SPED) for 2011-2020 and the State Programme for Education and Science Development 2016-2019 MESRK (2010), which stress (among many things) the need to prepare undergraduate and graduate students to meet the demands of industrial-innovative development; the importance of independent assessment of the qualifications of graduates; and the value of integration into the European Higher Education Area.
The Plan of the Nation: The 100 Concrete Steps. From the perspective of this chapter, highlights of that document include the creation of a group of ten leading higher education institutions which receive extra resources and autonomy, with a view towards eventual transfer of their experiences to other institutions; the gradual devolution of the currently centralised control of education; and the gradual adoption of English as a widely used language of instruction.

The targets outlined in these documents are ambitious, but often primarily favour the “excellence” dimension of quality. They include an aspiration to be ranked among “the top 30 countries” in the Global Competitiveness Index (GCI) and to have two higher education institutions appear in the top tier of international university rankings. While such targets define quality narrowly, they may still be useful in helping countries identify weaknesses and enhance their focus on inputs and processes that require improvement.

In the 2016 GCI, Kazakhstan was ranked 42nd out of 140 countries. It stood in 60th place on the index’s fifth pillar (related to higher education and training) and 67th on the quality of the education system indicator. Elsewhere, an “inadequately educated workforce” was identified as the fifth most problematic factor for doing business in the country (World Economic Forum, 2016). These rankings suggest gaps in the alignment between academic programmes, graduate outcomes and the needs of the labour market.

The Quacquarelli Symonds World University Rankings use six performance indicators (academic reputation, employer reputation, faculty/student ratios, international faculty, international students and citations per faculty) to rank universities worldwide. In the 2016 QS World University Rankings, nine Kazakhstani universities appeared in the list of the top 800 universities. These rankings use six performance indicators (academic reputation, employer reputation, faculty/student ratios, international faculty, international students and citations per faculty) to rank universities worldwide. The best-performing Kazakhstani university in 2016 was Al Farabi Kazakh National University, which ranked 275th. Again these rankings can point to specific areas for improvement – in particular towards potential to further:

develop and deliver academic programmes that satisfy employers and thus bolster institutional reputation
significantly increase faculty members’ research output in international indexed journals
increase the presence of international faculty and students on campus.

However, Chapter 6 of this review cautions against overreliance on the comparative dimension of international rankings: quality improvements should be measured primarily in absolute terms. Furthermore, improving performance on these and similar metrics requires overcoming significant challenges, some of which are beyond the control of higher education institutions.

Improvement means, for instance, enhancing a variety of inputs and processes; raising the performance of educational institutions that feed students into higher education; and reforming a heavily centralised, statist approach to governance that relies too much on control mechanisms to quality. Improvement also means building the capacity of academics and academic leaders to foster learning and to run institutions; reconceptualising curricula to emphasise the development of the cognitive, social and emotional skills that students and employers both value; and teaching and assessing in ways that encourage active student engagement, shifting away from the simple memorisation of facts towards the development and application of cognitive and social skills to existing and emerging problems.

Since the 2007 OECD/World Bank Review of Higher Education in Kazakhstan, and in the face of persistent challenges, Kazakhstan has made some noteworthy strides towards meeting strategic targets for education quality. For instance:

In 2010, the country became a signatory to the Bologna Process, indicating its willingness to reform the structure of its higher education institutions to conform to European standards. Since then, it has implemented a number of initiatives to bring practices in line with Bologna standards.
It has expanded quality assurance practices. The National Registry for Accreditation Agencies now includes two national agencies that perform reviews of institutions and programmes (see Figure 2.1), and eight international agencies that review programmes.
It has begun a gradual shift toward institutional autonomy. Ten universities have been given greater leeway in designing curricula and in training doctoral students.

A summary of the recommendations made in the 2007 OECD/World Bank review and their implementation status is presented in Figure 2.1.

The quality of educational inputs

In Kazakhstan’s highly centralised governance system, the Law on Education (2007) and associated regulations play a major role in determining the inputs and processes of higher education. These include for instance levels of funding; who accesses higher education institutions; which students receive state grants; and which higher education institutions can offer doctoral level programmes to how many students.

The government has taken steps towards the devolution of power from the Ministry of Education and Science of the Republic of Kazakhstan (MESRK) to higher education institutions. For example, supervisory boards have been established at some institutions; the degree of required adherence to state standards for curriculum design has been reduced (especially at National Research Universities); and the National Accreditation Center has been reorganised in two new independent centres, the Bologna Process and Academic Mobility Center, and the independent National Center for Education Statistics and Evaluation (NCESE). Those bodies take on functions that formerly belonged to the MESRK. However, the devolution of control has been slow due to regulatory constraints that are linked to issues of funding and governance (see Chapters 6 and 7) as well as to perceived gaps in institutions’ capacity to assume full responsibility for curriculum and programme design. For example plans made in 2010, which would have given full autonomy to the National Research Universities and replaced state attestation by 2015, have not been realised.

As was reported to the OECD review team during the meetings with senior university administrators, higher education institutions would like to acquire full autonomy. This desire is also indicated by the large number of Kazakhstani higher education institutions (66) which are now signatories to the Magna Charta Universitatum. The Magna Charta is an internationally supported document that promotes good governance of universities, and is predicated on the principles of academic freedom and institutional autonomy. By signing on to this charter, higher education institutions express their commitment to academic values and purposes that are held in high international regard.

Delays in regulatory change draw attention to the interdependence between inputs, processes and outputs/outcomes in higher education. If higher education institutions lack real autonomy, they can neither be fully responsible nor fully accountable for their overall performance. In particular, if institutions are expected to perform at internationally benchmarked levels – generating outputs and outcomes that are comparable to those of high-performing global peers – then in the areas of inputs and processes, they need to be on a level playing field with these peers.

Students

Knowledge has replaced physical capital as the most important resource for advanced economies; it has become a critical driver of economic performance and of the competitiveness of nations (OECD, 2008; The Task Force on Higher Education, 2000). The “knowledge revolution” makes it imperative that individuals engage in lifelong learning so that they remain current and are able to succeed in a changing economic environment. Participation in the knowledge economy requires not just up-to-date knowledge of specific domains, but also a broader set of skills.

The preparation, performance and outcomes of students who enrol in higher education – for example, their ability to apply higher-order thinking skills such as problem solving and analytical thinking – will depend in part on the extent to which, throughout their education, these skills have been emphasised and developed. It also depends on the extent to which they perceive that higher education, and society more generally, values these skills.

All levels of education in Kazakhstan need to break away from an emphasis on memorisation and rote learning, and move towards an approach that fosters the knowledge and skills that enable graduates to be innovative and independent thinkers, problem solvers, and collaborators. The most recent performance of 15-year-olds shows that there is still some progress to be made when it comes to basic skills (see Chapter 1). Kazakhstan’s 15-year-olds score much lower than the OECD average and lower than most PISA-participating countries when it comes to translating a problem into a form that is amenable to mathematical treatment; employing mathematical concepts, facts, procedures and reasoning; and interpreting, applying and evaluating mathematical outcomes (OECD, 2014a). In fact, PISA reports that 45% of Kazakhstani students are unable to understand and solve simple math problems.

PISA results on reading show a similar level of underperformance (OECD, 2014a). Almost six out of every ten students lack basic reading skills. Kazakhstani youth are able to “follow and understand continuous text” better than non-continuous text, and are on average only able to “locate one or more independent pieces of explicitly stated information” or make “simple connections.” They perform better when dealing with traditional texts, and less well when relating their own experiences to the text. As in the case of math results, this suggests a disconnect between what students learn and their ability to apply knowledge in real-life situations (Inoue and Gortazar, (2014). These findings are consistent with the review team’s observation that, both at the higher education level and in the years that lead up to entry to that level, the development of broad cognitive skills is less of a focus than the memorisation of facts.

At the end of grade 11, students are assessed using the Unified National Test (UNT). This high stakes assessment serves as a school-leaving exam, but is also the pathway of entry to higher education for the majority of students who have completed grade 11. Importantly, it also determines who is eligible to receive a state grant to study.

The UNT’s impact on the quality of students admitted to higher education institutions, and on student readiness to engage in learning, is a relevant factor affecting the overall quality of higher education. The test is a powerful driver of learning and teaching behaviour. However, the type of learning it values does not reflect what is expected of students at higher education institutions across the world that are increasingly oriented towards producing graduates who can excel in evolving labour markets where “book learning” is no longer enough.

Reports from the JSC Information-Analytic Center (IAC) and the media, combined with input provided by students and faculty members during interviews with the OECD review team, indicate that UNT test items primarily assess students’ ability to memorise facts. However they do not appropriately measure the knowledge, skills and competencies that are outlined for example by the European Qualifications Framework, and that comparable exams such as the Irish Leaving Certificate Examination and the German Abitur successfully measure (OECD and the World Bank 2007). Teaching and learning efforts during the last two years of high school appear to be very much oriented towards preparing students to pass the UNT (OECD, 2014b). This means that valuable time that could be spent developing a host of cognitive, emotional and social skills is lost to the students.

Moreover, the UNT’s predictive value for academic success, and its validity as an instrument to match students to a programme of study that corresponds with their interests and background knowledge, are other points of weakness. The OECD review team was able to consult a study on financial aid that was carried out by one of Kazakhstan’s higher education institutions using a relatively small sample of 529 students. Among other things, this study looked at the correlation between UNT scores and students’ grade point average (GPA) in their second year of study. The correlation between GPA and UNT scores was only modest (p = 0.32). One of the conclusions of this study was that “a 10% increase in the UNT score increases the probability of a better GPA by two hundredths of a percent to three thousandths of a percent” (Taylor, L. personal communication, February 12, 2016). Other tests, such as the Scholastic Aptitude Test (SAT) report a higher correlation with educational outcomes (e.g. GPA, principled reasoning, career success) that is in the range of p = 0.85 − 0.95 (Gibbs, 2010; The Economist, 2015). To be meaningful, a high-stakes test such as the UNT would require, at the very minimum, rigorous analysis using cross-sectional data to ascertain its psychometric properties, including its predictive value for academic success.

Kazakhstan’s higher education drop-out rates also potentially point to the inadequacy of the UNT in predicting academic success. In 2014-15, almost 60 000 students (12%) dropped out of their respective programmes. Of these, 24% dropped out either voluntarily or because of poor academic performance, and 29% transitioned to other forms of education. These percentages are high enough to warrant further in-depth exploration of the reasons that lead to drop-out, for example, financial barriers, poor academic preparation or low student motivation (MESRK,2014-2015). In any case, UNT scores do not seem to appropriately model how students will persist in their studies once admitted to higher education.

It is fair to conclude that the current UNT may be a good measure of students’ ability to recognise correct answers and to memorise facts. However, its role as the primary gateway controlling higher education access requires careful review (OECD, 2014b; The World Bank, 2012; Winter et al., 2014a, 2014b). At a minimum, the test needs to be adjusted to ensure that it measures the knowledge and skills that are valued in modern economies and societies.

Faculty qualifications

Highly qualified and motivated faculty members are critical for the quality of higher education at the undergraduate and, in particular, the graduate level. Faculty with higher qualifications typically have more in-depth domain knowledge, and are better able to keep abreast of advances in their field. Advanced research training, which is typically acquired through doctoral-level studies, develops the ability to foster an inquiry-based approach to learning – an approach that is instrumental in building transversal skills. Well-qualified faculty tend to be more adept at the use of innovative pedagogical approaches, particularly with respect to course design, active engagement of students in the learning process and assessment (Christensen Hughes and Mighty (eds.), 2010; Svinicki and McKeachie, 2014; Weimer, 2013).

The Task Force on Higher Education and Society, convened by the World Bank and UNESCO in 2000, highlighted faculty qualifications as a serious problem in developing countries. It attributed deficiencies to:

inadequate graduate-level training
a tendency to use approaches that reinforce rote and passive learning
inadequate remuneration, coupled with incentive structures that reward years of service rather than performance.

Statistics on faculty and teaching staff suggest that these problems also prevail in Kazakhstani higher education institutions, despite several initiatives aimed at bolstering the credentials and professional capacity of faculty.

Faculty appointments in Kazakhstan are made at the institutional level, although salary scales are regulated centrally. A minimum degree-level is not required as part of the academic appointment process. Appointments can be either full- or part-time, and faculty members may be assigned to teach either face-to-face or distance education courses. Despite the Minister’s Order according to which faculty must hold at least a master’s degree, it seems that some of them who hold less than a master’s degree could be simultaneously enrolled as graduate students. A number of higher education institutions rely on visiting international faculty to bolster the overall quality of their teaching force.

Of the 40 320 individuals who made up the core teaching staff in 2014-15, nearly half had an advanced degree equivalent to a level 8 of UNESCO’s International Standard Classification of Education, 2011 (ISCED). These include holders of a PhD or a specialised PhD, the Doctor of Science or the Candidate of Science degrees. Nearly a quarter of the core teaching staff (24%) held a master’s degree as their highest credential. The exact qualifications distribution of the remaining 27% was not clear to the OECD review team, but this group likely includes many individuals who are in the process of obtaining their master’s degree (Yergebekov and Temirbekova, 2012).

When holders of master’s degrees and those with lesser qualifications are put together (Figure 2.2), the higher education system’s gaps in the area of qualifications of academics become more apparent. Only 12% of faculty members hold a degree equivalent to a PhD, and the majority (51%) hold qualifications equivalent to a master’s degree or less. The pattern of distribution of faculty by qualification is similar in public and private institutions (Figure 2.3).

The distribution of faculty members’ qualifications varies significantly by region, though, with implications for the uniformity of quality of instruction in Kazakhstan. Three oblasts (Aktobe, Atyrau and South Kazakhstan) have the highest percentage of faculty who hold less than a master’s degree. On the other hand, Almaty, Pavlodar and Zhymbal have the lowest percentage of such faculty. North Kazakhstan, Kyzylorda and Mangystau have the lowest percentage of faculty with a PhD or a Dr. of Science degree. Astana City and Almaty City have the highest proportion of such faculty members (Figure 2.4).

Representatives of Kazakhstan’s accreditation agencies reported to the OECD review team that inadequate faculty qualifications are one of the consistent weaknesses identified by the accreditation process. In light of these shortfalls, it has been reported that higher education institutions sometimes make temporary appointments of individuals with higher qualifications so that they can present themselves in a better light for accreditation (Yergebekov and Temirbekova, 2012).

As Kazakhstan moves to improve the performance of its higher education system, it will be important to ensure a supply of potential faculty members who are adequately prepared to advance the core mission of teaching and learning. This can only happen if the training of master’s and doctoral-level students is made a top priority, and if the obstacles that currently impede access to graduate education and its completion are removed. For example, the requirement that students have one publication in international indexed scholarly journals before receiving a PhD degree represents an obstacle – in particular given that mastery of English in Kazakhstan is still very low (Chapter 5 explores this question in more detail). It is important to recognise, though, that any increase in the number of graduate students would also require a greater number of faculty members with doctoral degrees to provide good supervision.

Quality in educational processes

Appropriate process indicators need to take into account difference in the contexts in which higher education institutions operate and the diversity of their missions. Only indicators for which the OECD review team was able to compile evidence are discussed in this section of the report. These include two regulatory processes (i.e. the adoption of the Bologna Framework, and the implementation of accreditation and quality assurance processes) and processes related to the delivery of education (i.e. the provision of student learning experiences, the curriculum and its complementary processes, and faculty and academic development).

Regulatory Processes

The Bologna Framework and its implementation in Kazakhstan

The Bologna Process was initiated to foster co-operation, support student mobility and employability and strengthen the competitiveness of European higher education. To these ends, it has worked to:

implement a three-cycle system of credentials (bachelor’s/master’s/doctorate)
improve quality assurance
facilitate the recognition of qualifications and periods of study.

Bologna calls for signatory nations to develop their national qualifications framework in ways that correspond to the overarching Framework for Qualifications of the European Higher Education Area, and to adopt standards that embody that framework’s principles.

When Kazakhstan became a signatory to the Bologna Process in 2010, it indicated its willingness to reform the structure of its higher education system to conform to European standards. Since then, it has gradually been implementing changes to better adhere to Bologna principles. Notable advances include:

The implementation of a system for translating national Kazakhstani credits into European Credit Transfer and Accumulation System (ECTS) credits, and changes to the duration of the bachelor, master’s and doctoral degrees to make them into four-two and three-year cycles respectively. This work has been carried out with support from the European Commission’s Tempus Programme (Education, Audiovisual and Culture Agency, 2010).
Increased engagement of Kazakhstani students and faculty members in mobility activities – including through enhanced support for travel both within and outside Kazakhstan. In 2014 about 1.6% of students studied abroad for at least one semester. Kazakhstan has also supported mobility for international students and faculty coming to the country.

Despite this progress, major challenges still persist in the implementation of Bologna principles. These are apparent in the delays in building a coherent national qualifications framework and in granting full autonomy to institutions.

In interviews with faculty, the OECD review team heard both positive and negative comments about the implementation of Bologna. Younger faculty typically had more positive attitudes towards Bologna, and reported that it had forced them to focus on learning outcomes rather than on simply teaching “content”. However, the team also heard that Kazakhstan’s adhesion to Bologna has not led to substantive changes, because the education system continues to be heavily centralised.

Other studies have similarly observed that Kazakhstan’s embrace of Bologna and its values has sometimes been more one of form than of substance (Soltys, 2014; Yergebekov and Temirbekova, 2012). The lack of substantive change has for instance been attributed to the legacy of centralised planning, which favours control and monitoring as the primary mechanism for quality control. The absence of a strong civil society has been cited as another reason why real change has not happened.

An illustrative example of the implementation gap can be observed in Kazakhstan’s deployment of the European Credit Transfer and Accumulation System (ECTS). There is a substantial difference between the principles behind the system and the way it actually works in Kazakhstan. An approach which is meant to offer students flexibility in course selection appears to be dysfunctional in Kazakhstan because the Law on Education (2007) and associated regulations, impede students from freely selecting courses or instructors.

Another implementation gap in implementing Bologna can be observed: only one of the two country’s national accreditation agencies has been unable to acquire “full member” status with the European Association for Quality assurance in Higher education. This reflects the fact that the principles underlying governance in Kazakhstani higher education institutions do not entirely conform to those advocated in the European Higher Education Area, and which are outlined in the 2015 Standards and Guidelines for Quality Assurance in the European Higher Education Area. Points of divergence between the standards and the current situation in Kazakhstan include (but are not limited to):

levels of academic integrity and freedom
the processes by which programmes are designed and approved, their linkage with a Kazakhstani qualifications framework for higher education, and ultimately their linkage with the Framework for Qualifications of the European Higher Education Area (EHEA)
the extent of student-centred learning
teaching and assessment practices
the qualifications of teaching staff
the ongoing monitoring and cyclical review of programmes.

Even though Kazakhstan’s adoption of Bologna is incomplete, the OECD review team heard concerns about some of the unintended effects that certain elements of Bologna are having on quality. For instance, it was reported to the review team that reductions in the number of years of study (e.g. to conform to the Bologna model of an undergraduate education) had harmed higher education quality. Interview participants cited mathematics and engineering as examples of where reductions in programme duration (down from five years to four) have led, in their view, to a more superficial treatment of subject matter. This complaint likely reflects a combination of student-preparation and curricular-design issues: better-prepared students, and better-designed curricula could help overcome it.

The team also observed an apparent tension regarding the value that is attributed to newly created academic credentials. Specifically, that Kandidat Nauk (Candidate of Science) and Doktor Nauk (Doctor of Science) obtained in the Soviet tradition are now perceived, at least by some, to be of a lesser value than the new PhD degree. Senior faculty appeared to be particularly sensitive to this issue. Such sentiments have also been reflected in the literature (Pak, 2010).

The tensions brought on by Bologna related-reforms, and the assertions in the academic literature about the dangers of “artificial or hasty institutionalisation” (Satpayev, 2014), suggest that the MESRK should as soon as possible undertake a careful analysis of potential barriers to the full implementation of Bologna and unintended consequences of current implementation efforts. The Ministry should then design interventions to address both sorts of issue. One possible strategy might involve marshalling compelling and reliable data to better communicate to the public the positive impact of the Bologna reforms. Otherwise, there is the danger that an already fragile public trust may further erode and thus endanger progress towards full reform.

Accreditation and quality assurance processes

The higher education landscape has changed dramatically in the last few decades. Governments’ interest in quality assurance measures, and in the potential of these to monitor, maintain and improve quality, have been heightened by a number of trends: mass participation in higher education; increased pressure on national budgets; greater emphasis on accountability, deregulation and expanding the range of providers; concerns about the relevance of education to the needs of a knowledge economy; growing internationalisation; and challenges to social cohesion (OECD, 2008).

Organisations such as the International Network for Quality Assurance Agencies in Higher Education (INQAAHE), the European Association for Quality Assurance in Higher Education (ENQA), the United Kingdom’s Quality Assurance Agency for Higher Education (QAA), and Australia’s Tertiary Education Quality and Standards Agency (TEQSA) have made significant contributions to systematising processes and developing quality assurance standards in ways that support both accountability and improvement. The “improvement” function underscores the value of the quality assurance process as a formative and developmental exercise. These processes will vary, of course, recognising how institutions, traditions and cultural norms differ across countries. A typology of quality assurance activities and their scope (Figure 2.5), and recommendations for designing a quality assurance framework (Box 2.1), are provided here as a reference (OECD, 2008).

Box 2.1. Designing a good quality assurance framework

Design a quality assurance framework consistent with the goals of higher education.
Build consensus on clear goals and expectations of the quality assurance system.
Ensure that quality assurance serves both the improvement and accountability purposes.
Combine internal and external quality assurance mechanisms.
Build capacity and secure legitimacy.
Make stakeholders such as students, graduates, and employers visible in the evaluation procedures.
Increase focus on student outcomes.
Enhance the international comparability of the quality assurance framework.

For internal evaluation

Develop a strong quality culture in the system.
Put more emphasis on internal quality assurance mechanisms.
Ensure that internal accountability is guided by key principles.
Undertake the external validation of internal quality assurance systems.

For external evaluation

Commit external quality assurance to an advisory role as the system gains maturity but retain strong external components in certain contexts.
Implement adequate follow-up procedures and view quality assurance as a continuous process.
Allow for selected assessment to be initiated by an external quality assurance agency.
Avoid direct links between assessment results and public funding decisions.

Practical arrangements for quality assurance systems

Avoid fragmentation of the quality assurance organisational structure.
Avoid excessive costs and burdens.
Improve information base about quality.
Improve information dissemination.

Sources: OECD (2008), Tertiary Education for the Knowledge Society, Volume 1, Special Features: Governance, Funding, Quality, OECD Publishing, Paris, www.oecd.org/education/skills-beyond-school/41266690.pdf.

Kazakhstan established its two national accreditation agencies, the “Independent Kazakh Agency for Quality Assurance in Education (IQAA)” and the “Independent Agency for Accreditation Rating (IAAR)” in 2008 and 2011 respectively. On 30 November 2006, the IAAR was granted membership of the European Association for Quality Assurance in Higher Education. Their mandate is to accredit both higher education institutions and academic programmes. While the titles of both agencies qualify them as “independent”, in practice they remain dependent agencies of the government. The MESRK remains the authority which stands behind accreditation decisions and the consequences of these decisions.

International accrediting agencies are also active in Kazakhstan, although their remit is limited to the accreditation of individual programmes. Eight are currently listed in the National Register for Accreditation Agencies (Kalanova, 2015). Professional associations, comparable to professional accrediting or licensing bodies in western countries, are not developed or organised enough to play a role in accrediting programmes for licensure purposes.

Accreditation in Kazakhstan stands in contradistinction to the “attestation” process, which is an older and more rigid form of quality assurance. Accreditation is performed on a voluntary basis using an institution’s agency of choice. Attestation consists of reviews that are conducted at least every five years to control quality on the basis of state standards, and on the basis of which higher education institutions are authorised to operate (see Chapter 7). Attestation lacks the formative dimension of accreditation.

Based on data available at the time this report was being prepared, the two national accreditation agencies have accredited 78 higher education institutions – that is roughly 60% of all higher education institutions in Kazakhstan. Of these, 47 are public and 31 are private. The two agencies have also accredited just over 2 000 programmes, with four-fifths of these in public institutions. Also, by 2014, 139 undergraduate and graduate programmes in 22 Kazakhstani higher education institutions had been accredited by the registered international agencies.

The OECD review team heard reports about the composition of accreditation teams. These are said to be typically made up of six academics, four representatives of the Chamber of Commerce, one student and one international member. However, in 2012, the European Commission observed that neither international experts nor students had representation on IQQA accreditation teams. The reports made in late 2015 to the OECD review team may reflect a positive change since 2012. However, the team was not able to verify that international disciplinary experts are now a standard feature of accreditation panels nor that student representatives are able to convey student points of view in confidence and without fear of reprisal.

Kazakhstani higher education institutions also undertake their own internal quality assurance activities. These primarily involve the preparation of self-studies in preparation for accreditation and attestation. Given the limited information available to it, the OECD review team was unable to discern the extent to which these activities have actually made a difference in the quality of education – other than perhaps raising some awareness in the academic community, and informing the members of that community about how they might better present themselves for accreditation. It was reported to the team that pre-attestation self-studies may simply be bureaucratic exercises.

The review team found no evidence that higher education institutions have embraced and internalised a culture of quality assurance, or that they have mechanisms in place to improve inputs, processes, and ultimately outputs and outcomes. The prevalence of a monitoring culture in Kazakhstan, coupled with the number of other concurrent activities undertaken by the MESRK to verify compliance (licensing, the still active attestation process, recurring inspections, ad hoc inspections, the EASA External assessment of students achievement test, etc.), may have undermined the potential of quality assurance to drive institutional improvement processes.

Furthermore, the large number of programmes and institutions that have undergone formal accreditation by the two national agencies in a relatively short period of time raises concern about the thoroughness of the process, given the limited number of faculty in Kazakhstan who have the expertise needed to serve on review panels. If senior and highly qualified faculty members are being seconded constantly to the accreditation agencies for this purpose, higher education institutions themselves may be deprived of one of their most important quality assurance resources. If less qualified individuals are being invited to join accreditation teams, this raises a question about their ability to express an informed opinion about quality, that is, one that goes beyond simply verifying inputs named on a checklist.

The OECD review team was also told of other capacity constraints within the accreditation process. For instance, national accreditation agencies are said to lack sufficient resources to undertake the review of research programmes which are an important part of the functions of an accreditation agency.

Finally, based on reports the OECD review team heard during its interviews, international accreditation team members appear much of the time to be participating only “at a distance”. This raises questions about the extent to which Kazakhstan’s accreditation process goes beyond merely the verification of compliance with standards. The ability to interact with students, faculty and academic administrators, and to directly share an outsider’s perspective about programme elements and institutional policies and practices, are important advantages of on-site international participation in accreditation visits. International team members also help informally benchmark internal practices with international norms. Kazakhstan appears to be not fully benefitting from this potential.

Clearly accreditation is still developing in Kazakhstan, and it is limited in its development by a persistent “control” approach to quality assurance. Given this, the OECD review team is concerned that the plan to have accreditation replace the old state attestation has been postponed. This postponement suggests that full trust in the work of designated accreditation agencies is still lacking. Until such trust is developed, it will be difficult to meet the very first standard of quality assurance outlined by European Association for Quality Assurance in Higher Education, which relates to the trust that institutions and the public place in accreditation agencies.

Putting greater emphasis on the improvement dimension of quality assurance through internal mechanisms will be important in developing a strong quality culture and commitment to uphold its principles. Ensuring that Kazakhstan’s quality assurance framework is aligned with the goals of higher education institutions and their diverse missions, and that institutions develop the capacity to credibly engage in quality assurance, will also be important steps towards fostering a culture of quality and trust.

Student learning experiences

Many factors contribute to successful student learning. While all are important, three strike the OECD review team as especially relevant in the Kazakhstani context: student-focused approaches to learning; measures that promote active student engagement; and exposure to authentic contexts where students can apply theoretical knowledge.

Student-focused approaches to learning

The educational literature makes a distinction between a “surface” and “deep” approach to learning. This distinction separates learning where learners memorise and reproduce content from learning where learners internalise content. A surface approach yields very limited learning, whereas a deep approach fosters the kind of learning that lasts a lifetime (Gibbs, 2010; Marton and Säljö, 1976).

Students’ approaches to learning are context-dependent, and are motivated by their perception of what the education system and society at large value. This makes it important that learning environments be designed in ways that foster not just the acquisition of facts and theories but also the application of knowledge. These environments also need to develop cognitive skills such as problem solving, critical and analytical thinking, reasoning; social and emotional skills such as teamwork, leadership, and conflict resolution; and competencies that combine a variety of skillsets, such as entrepreneurship.

As this chapter has already observed, Kazakhstan’s regulatory context and its high stakes UNT-based admission process push students to focus on memorisation long before they enter higher education institutions. This approach continues to be reinforced in higher education through an emphasis on knowledge retention, and by the ways in which the fourth-year VODE test (the “External Test of Academic Achievements”) measures the quality of education. Like the UNT, the VODE test items normally have a multiple-choice format and assess factual knowledge related to the subject matter. The results of the VODE do not directly affect students themselves, but have important consequences for higher education institutions themselves (e.g. as part of the attestation process), and so shape instructional processes. When a high-stakes test such as the VODE focuses on the retention of facts rather than on higher-order thinking skills, it is unlikely to encourage deep learning.

On a more positive note, tests like the VODE can be adapted to shift learning approaches from surface to deep. This requires re-orienting questions to tap into higher-order learning, including advanced cognitive skills. Such changes should not, of course, be limited to just summative exams such as the VODE. They should be applied to all assessments, including assignments and classroom tests, as the higher education culture shifts to one of deep learning.

The OECD review team had a positive impression of the various social and sports clubs available to students in Kazakhstan, and of the emphasis that administrators place on the overall well-being of students. However, the team did not develop sufficient evidence to allow conclusions about the extent of active learning practices or the quality of student engagement.

The National Survey of Student Engagement (NSSE), conducted primarily in the United States, is one approach to tracking student engagement in learning. The survey seeks to determine how much time and effort students put in their studies and other complementary activities, and whether institutional support and resources are invested in activities that are known to foster student learning. Institutions use the results of this survey to develop new policies, practices and resources to make their institutional climate more conducive to student learning (Figure 2.6).

Active student engagement

Student engagement indicators used by NSSE point to ways in which undergraduate education can be improved. While some of the included indicators such as “study abroad” fit into current national strategic directions and institutional objectives, other indicators used by NSSE could inspire positive change in Kazakhstan.

Authentic work-related experiences

Structured learning experiences that expose students to real life situations, and encourage them to apply theoretical and conceptual knowledge to concrete problems, can go a long way towards developing lifelong learners. Some effective programmes from around the world make formal, supervised opportunities a required component of the curriculum and ensure that all students are exposed to these experiences. This can be accomplished through a variety of approaches such as internships, co-op programmes, project and problem-based learning, and research projects (Donovan et al.,1999; Sawyer, 2006) (see Box 2.2 on examples of programmes).

Box 2.2. Effective approaches to work-integrated learning

Waterloo University’s co-op programmes

Co-op options include study/work sequences starting either in the first or second year of study. They typically involves four months of study on campus, followed by four to eight months working full time in a job related to area of study. This option is available for a range of programmes offered by the faculties of Arts, Applied Health Sciences, Engineering, Mathematics, and Science. It allows students to see how classroom learning can be applied in the workplace – thus enriching both classroom and workplace learning. It helps students develop certain skills (e.g. negotiation, conflict resolution) which are comparatively hard to teach in the classroom.

Community Service Work Study at the Massachusetts Institute of Technology

This programme provides MIT students with the opportunity to earn a pay cheque while giving back to the community. Students who qualify for Federal work-study are able to add to their work experience, hone their skills, and explore a career while helping non-profit organisations find creative solutions to the problems they face.

At the national level, the United States’ Community Service Federal Work-Study Program supports learning experience like that at MIT at a wide range of institutions.

Sources: Priscilla King Gray Public Center (n.d.), http://web.mit.edu/mitpsc/whatwedo/work-study/; University of Waterloo, https://uwaterloo.ca/find-out-more/co-op/study-work-sequences.

In principle, all higher education students in Kazakhstan have work experiences as part of their programmes. This is a positive feature of higher education in Kazakhstan, giving students the chance to see how coursework can link up to the world of work. Internships in Kazakhstan are often facilitated through satellite offices that academic departments set up in enterprises.

Nonetheless, in interviews with the OECD review team, students sometimes expressed a desire to have more opportunities to gain practical experience through work placements. This suggests that there may be room to improve the work experience process. Reports from students and faculty members led the OECD review team to conclude that on-site educational activities, while widespread, are likely to differ in quality. Internships are varied and last anywhere between a few weeks to several months. Students in the second year of their programme, for instance, often get short placements where they primarily “observe” the workplace; it is unclear how much value is derived from this low-contact approach, nor how active supervision is. In other cases reported to the team, students appeared be engaged in what might primarily be characterised as seasonal labour. In upper years, though, student work is organised around a final project that may provide a good deal of work-related experience.

The OECD review team did not come across any evidence of co-op or study/work options where formal periods of study alternate with paid internships. Many countries are experimenting with approaches to work-integrated learning using insights about how experience in the workplace can best be harnessed to develop a range of technical and transversal skills. Kazakhstan may wish to explore models from other countries as it seeks to modernise its approval to work-integrated learning. Box 2.3 outlines examples of international good practice.

Box 2.3. Good practices in work-based learning and the Canadian Association for Co-operative Education

Principles of the Canadian Association for Co-operative Education

Among the principles that the Canadian Association for Co-operative Education lays out for effective co-operative work-integrated learning are the following:

Each work situation needs to be developed or approved by the university or college as a suitable learning situation.
The co-op student needs to be engaged in productive work rather than merely observing.
The co-op student needs to receive remuneration for the work performed.
The co-op student’s progress on the job needs to monitored by the university or college.
The co-op student’s performance on the job needs to be supervised and evaluated by the student’s employer.

Best international practices in work-based learning in VET

The OECD recommends that work-based learning in vocational education and training should be:

Systematic and mandatory: work-based learning has to be an integrated part of the vocational programme, and needs to build a culture of partnership between training providers and employers. Teachers need to work more closely with employers and help design the curriculum locally.
Quality-assured and credit bearing: quality standards and a clear legal framework are necessary to encourage work placement arrangements. Quality assurance should play a decisive role in the accreditation of new programmes. Placements should be as useful as possible for both vocational programmes and employers, and they should be closely linked to learning outcomes.

These recommendations apply equally well to higher education.

Sources: OECD (2014), Skills Beyond School: Synthesis Report, OECD Reviews of Vocational Education and Training; www.cafce.ca/_Library/_documents/coopmanual.pdf.

Research experience opportunities for students have become institutionalised in many higher education institutions across the world, in part because of recommendations made by the Boyer Commission’s Report on Reinventing Undergraduate Education in 1998 (see Box 2.4). Benefits of research experience for undergraduate students include greater confidence in understanding research-related issues and increased interest in careers in research and the natural sciences (Jenkins, 2004). In interviews with the OECD review team, students in some programmes (e.g. in medicine) expressed the desire to gain more research experience during their studies. It does not appear that research involvement at the undergraduate level is a deliberate focus of higher education institutions in Kazakhstan (although it may happen in some final projects). However, if Kazakhstan were to seize more extensively on the benefits of this approach, it would be important to ensure a sufficient supply of faculty members who are actively engaged in research (see Chapter 5).

Box 2.4. The Boyer Commission’s recommendations for changing undergraduate education in research universities

The Boyer Commission was created in 1995 (under the auspices of the Carnegie Foundation for the Advancement of Teaching) to examine the undergraduate education offered in research universities in light of changes to the United States’ higher educational landscape. The Commission began with an acknowledgement that the American higher education system had become less elite, and that there were greater demands of accountability on the part of students and parents. It also observed that the number of undergraduate professional degrees had expanded widely, and that the freshman year was often reduced to a repetition of high school curricula.

The Boyer Commission argued that unless research universities made an explicit effort to introduce students to inquiry-based learning, they would be denying them the education that they were promised. The Commission’s recommendations were to:

Make research-based learning the standard.
Construct an inquiry-based first year of study.
Build on the first-year foundation.
Remove barriers to interdisciplinary education.
Link communication skills and course work.
Use information technology creatively.
Culminate with a capstone experience.
Educate graduate students as apprentice teachers.
Change faculty reward systems.
Cultivate a sense of community.

Sources: www.as.wvu.edu/~lbrady/boyer-report.htm.

Curriculum design and complementary processes

High-quality programmes intentionally develop outcomes-based curricula that follow specifications outlined in national qualifications frameworks; they design learning environments that seek to develop these outcomes. Qualifications frameworks themselves are typically built with input from the labour market, and they target knowledge and skills at different levels of education. One of their roles is to specify how different types of educational institutions articulate with each other in an education system. They thus help to determine equivalencies for the purposes of credit transfer and to identify work-related experiences for which credit can be granted.

The National Qualifications Framework

Kazakhstan’s National Qualifications Framework is the blueprint that should guide the development of curricula. Kazakhstan is to be commended for taking initial steps to develop and formalise the Qualifications Framework – but this work remains in its very early stages. For instance, specific frameworks for sectors, and the standards and competencies that correspond to these, have yet to be developed.

The European Commission’s Tempus review, conducted in 2012, gave Kazakhstan a score of 1 out of 5 for its work on the Qualifications Framework, noting that the process had just started. The recent first volume of the OECD Multi-dimensional Review of Kazakhstan also concludes that: “there is little indication to suggest that the National Qualifications Framework is now being used systematically to guide the design of curricula and to align graduate outcomes with employer needs, both in higher education institutions and VETs” (OECD, 2016).

The OECD review team was also unable to find evidence of progress. In interviews with the team, faculty members, academic administrators and students never spontaneously raised the issue of the Qualifications Framework. It seems likely that the state standards for the mandatory portion of the curricula have not followed Qualifications Framework guidelines, or that perhaps the existence of the Qualifications Framework has not been publicised widely in academic circles. It could also be that faculty simply lack the pedagogical know-how required to design outcomes-based curricula and courses. Finally, it is possible that the format of Kazakhstan’s National Qualifications Framework, which clusters knowledge, skills and competencies together rather than presenting them separately (and thus differs from the European Qualifications Framework), is not clear enough to help programmes target specific outcomes.

Certain other aspects of the Qualifications Framework also appear to undermine its potential to support high-quality education. A comparison of Kazakhstan’s framework with the European Qualifications Framework reveals important differences. Level 4 of the Kazakhstani National Qualifications Framework emphasises “ability to take directions”. In contrast, the European framework refers to items such as “cognitive and practical skills”, “problem solving” and “autonomy” from as early as Level 21 , and adds more complex skills and competencies at each subsequent level. The higher-order skills that are expected at Level 6 (comparable to the first cycle of higher education) include “advanced knowledge of a field”, “critical understanding of theories and principles”, “advanced skills demonstrating innovative approaches to solving unpredictable problems”, “reflection” and “self-regulation”. None are targeted directly in Kazakhstan’s Framework. Clearly, much work remains to be done to bring the Qualifications Framework to a level that would be comparable to the European framework, and thus enable it to serve as the primary blueprint in developing academic curricula, courses and assessment.

Since March 2016, within the context of the State Program on Education and Science Development 2016-2019, the government approved a new National Qualifications Framework. In this framework, reference is made to concepts such as complexity, autonomy and creativity, rendering it better aligned with the European Qualifications Framework. Since these amendments were implemented after the team’s visit, it is difficult to appreciate how the framework is being received and used by various stakeholders.

Curriculum design processes

The 2007 OECD/World Bank review observed the negative impact that a centralised and highly regulated education system has on curriculum design and delivery. This remains a problem, although institutions now have somewhat more freedom to deviate from state standards in curriculum design. The amount of leeway varies by institutional type: the ten institutions recently designated as National Research Universities are reportedly allowed 70% freedom in designing their bachelor’s level curriculum (i.e. only 30% of the curriculum is centrally prescribed). The remaining higher education institutions have control over 55% of their curriculum. Regulation is looser at the master’s level (where only 30% of the curriculum is prescribed) and at the PhD level (just 10% is prescribed).

The OECD review team was interested in finding out whether institutions are taking advantage of their new freedom to introduce varied and innovative elements into their programme content. To this end, the team sought – but was unable to acquire – substantial documentation that would show how different institutions are structuring the “same” programme (e.g. education programmes) across the country. Nonetheless, based on the very limited information that the team was able to access, and consistent with reports during interviews, the team concludes that there is as of yet little deviation from state standards. Higher education institutions are apparently not yet making extensive use of their limited freedom to orient curricula to better reflect their mission, to meet the diverse needs of their student body, and to be more responsive to labour market needs. If this is indeed the case, the team surmises that breaking with a structured and controlled framework is a difficult step for higher education institutions to make. It seems likely that a long-standing habit of being told what and how to teach may not have permitted the development of sufficient internal expertise in curricular design.

The review team did hear a number of concerns about the curriculum that go beyond those, mentioned earlier, that were related to the effects of the Bologna Process. It was reported that Kazakhstan’s higher education curriculum has a very rigid structure; that programmes have heavy (some said “excessive”) credit loads; that alternative pathways to programme completion are not available; and that students have little opportunity to select their courses (electives). Many of these problems could be addressed by taking advantage of the flexibility that Bologna promotes through a carefully developed national qualifications framework. In addition, universities around the world demonstrate other potential approaches to building flexible curricula that respond to students’ unique contexts (see Box 2.5 for the example in Canada).

Box 2.5. McGill University’s flexible and responsive bachelor’s programme in Arts

Students in McGill University’s Bachelor of Arts degree are normally admitted to a four-year programme requiring the completion of 120 credits. However, “advanced standing” of up to 30 credits may be granted if satisfactory results are obtained in Québec’s Diploma of Collegial Studies, the International Baccalaureate, the French Baccalaureate, Advanced Levels or Advanced Placement exams.

To recognise the diversity of student backgrounds and interests, the Faculty of Arts offers a 90-credit multi-track system that includes a major concentration complemented by at least one minor concentration. It may be completed in one of the following ways:

Major Concentration (36) + Minor Concentration (18) + 36 credits of electives
Major Concentration (36) + Minor Concentration (36) + 18 credits of electives
Major Concentration (36) + Minor Concentration (18) + Minor Concentration (18) + 18 credits of electives

Sources: www.mcgill.ca/study/2015-2016/faculties/arts/undergraduate/ug_arts_program_reqs#topic_C1AC51F9F8024453B327E7BEBD67FEE9.

The academic literature provides insights to support the development of a coherent and relevant 21st century curriculum. Key considerations include:

Co-ordinate efforts with the labour market to align curriculum with the requirements of the range of professions the programme is intended to serve as well as secure supervised internships for graduate students in the private sector.
Specify programme outcomes with respect to knowledge, competencies and skills as well as academic and professional ethics.
Increase the use of English in courses offered at the graduate level to prepare graduates for a global context and to attract and retain international talent.
Detail supervisory and mentorship responsibilities of faculty and offer structured mentorship in teaching for graduate students headed towards academic careers. (Douglass, 2015; Nerad and Evans, 2014).

Employer involvement in curriculum development

The OECD review teams spoke with employers and Chamber of Commerce representatives about the extent to which higher education institutions have ties with the labour market. The conversations confirm what has also been observed in the literature: such ties do exist, but not yet on a national, co-ordinated scale. They most often develop on an ad hoc basis and involve local industries (Sagintayeva et al., 2015). Some employers reported a substantial gap between student knowledge and the curriculum on the one hand, and the expectations of the labour market on the other. This is explored in more detail in the next section of this chapter.

Collaboration between employers and higher education has recently been reinforced by increased representation of the private sector on the supervisory boards of higher education institutions (i.e. at the minority of institutions where these have been implemented). It was reported to the review team that this governance change has led to the design of “over 3 000 programmes” using input from the private sector; that it has created supervised internship positions, using industry-based laboratories and equipment; and that it has generated opportunities for invited lectureships. These types of collaboration need to be tracked and evaluated to establish effectiveness – so that best practices can be replicated across the system (see Box 2.6 for two examples of effective collaboration between education and employers).

Box 2.6. Effective collaboration between employers and education

The Business Higher Education Round Table (B/HERT) in Australia

Since 1990, the B/HERT has stressed the importance of knowledge exchange and cross-sector collaboration in the knowledge-based economy. It has strengthened the relationship between business and the tertiary education sector by establishing strategic partnerships to develop programmes that advance education, research and innovation.

The B/HERT’s main activities revolve around promoting policy debate and discussion on issues such as the training agenda as well as the importance of innovation and entrepreneurship. It also produces publications that reach a wide audience of thought leaders in education, science, research, business and civil society.

The Sectors Skills Council (SSCs) in the United Kingdom

The SSCs are employer-led organisations working with over 550 000 employers. They are licensed by the government through the United Kingdom Commission for Employment and Skills. By covering specific industries, they contribute to the development of National Occupational Standards, the design and approval of apprenticeship frameworks and the New Apprenticeship Standards, and Sector Qualification Strategies.

The SSCs’ main goals are to support employers in developing and managing apprenticeship standards; to reduce skills gaps and shortages and improve productivity; to boost the skills of their sector workforces; and to improve learning supply. There are currently 21 SSCs in the United Kingdom covering in total approximately 90% of the United Kingdom workforce.

Sources: Business Higher Education Round Table, (n.d.), www.bhert.com/about.html; Federation for Industry Sector, (n.d.), http://fisss.org/sector-skills-council-body/directory-of-sscs/.

The Bologna Framework itself encourages collaboration among public authorities, universities, teachers, students, stakeholder associations, employers and quality assurance agencies. This level of harmonisation is critical for the delivery of education that is relevant to the labour market. In addition to participation of employers on boards, the development of strong national-level professional bodies is a promising approach that Kazakhstan could take to ensure that higher education programmes reflect required professional competencies.

Faculty workloads, pedagogy and professional development

Faculty workloads

Faculty members play an important role in each of the three core missions of higher education: teaching, advancing knowledge through research, and service and community engagement. The percentage of time that faculty are expected to spend on each function depends on institutional type and mission. There can be wide variation even within the same country. For example, in North American research-intensive universities, teaching and graduate supervision typically account for 40-50% of academic responsibilities (around three or four courses per year or 135-180 classroom contact hours), with the remainder of faculty time devoted to research or, to a lesser extent, academic service and community engagement. However, in universities primarily focused on teaching, faculty might be expected to teach as many as eight courses per year, that is, four courses per term, which means 12 classroom contact hours per week.

In the United States, as in many other countries, the main mechanism for assessing faculty performance and productivity is typically an annual review. In some contexts, evidence of academic performance (e.g. student course ratings and teaching portfolios, evidence of pedagogical innovations, awards and patents, research productivity, service contributions, etc.) can also serve as the basis for calculating annual salary increases.

In Kazakhstan, faculty members are fittingly referred to as “teaching staff”, since most of their time is in fact dedicated to instruction. The formal teaching load is calculated on the basis of the number of students that a faculty member is responsible for in his/her academic setting. In practice, this load can translate into to anywhere between 20 and 25 hours of contact time per week. Classes last between 50-110 minutes, and the term is 15 weeks long. Faculty members who have higher degrees or who are engaged in research are typically assigned a lighter teaching load (between 20%-50% lower). Teaching loads can thus range from approximately 400 to 750 hours per year.

A higher education institution’s mission quite naturally determines both the distribution of faculty members’ workload and the key performance indicators to assess faculty performance. Teaching loads of 20-25 hours per week should normally require at least an equal amount of time for preparation, grading and student engagement. This would thus effectively leave no additional time for other academic activities such as research – or it would lead to faculty members short-changing the instructional function and thus reducing the quality of student learning.

In addition to their teaching and other academic functions, faculty members are required to spend time regularly preparing monitoring reports about their activities and performance. For instance, one university rector with whom the OECD review team spoke reported requiring full weekly reports from his division heads; the work-generating effects of these reports presumably cascade down through the faculty. In interviews, the review team heard that reporting takes up a large amount of a faculty member’s time, but generates little benefit for instructional quality. The 2007 OECD/World Bank review recommended reducing the reporting burden, but it appears that Kazakhstan has made little progress in this regard.

Kazakhstan needs to think carefully about its expectations of faculty members, and how these relate to its strategic objectives for higher education. For example, if the intent is to raise the research productivity of faculty, then faculty members will need time, tools and resources to engage in publishable research. High teaching loads or added responsibilities that do not directly advance the mission of the institution – for instance record-keeping requirements that generate little incremental benefit and could be effectively replaced with an annual reporting system – should be reviewed in light of Kazakhstan’s overarching goals for a high-quality, responsive system of higher education.

Approaches to teaching

Around the world, greater access to higher education has resulted in more diversity in the student body. At the same time, there has been a shift in what is expected of graduates as they enter a world of employment which is characterised by rising uncertainty, speed, risk and complexity – and where interdisciplinary collaboration is increasingly the norm (Fadel, 2014; Hénard, 2010).

Effective teaching is no longer defined in terms of competent use of didactic approaches, for example, an ability to deliver well-organised lectures. Modern teaching requires innovative and multiple approaches that support different learning styles and engage all students in active learning and develop a range of skills. It also requires ongoing reflection that uses feedback from students and peers (Schön, 1983, 1987).

Higher education institutions around the world have recognised the challenges of “learning and teaching for the 21st century”, and are moving to ensure that faculty members and students have the supports and experiences they need. The OECD/IMHE’s survey of 29 higher education institutions in 20 countries gives a broad perspective on the main kinds of commitments that higher education is making to enhance the quality of learning and teaching (Hénard, 2010). These include:

structures and support units (dedicated teaching and learning units; programme leaders; curriculum officers)
incentives (teaching excellence awards; opportunities for personal and professional development)
curriculum-related projects (fast track programmes; instruction delivered in foreign languages)
quality assurance (course ratings; cyclical reviews; internal quality assurance for teaching and learning effectiveness)
other innovations (e-learning platforms; learning communities; conducive learning environments).

The OECD review team was able to speak with a wide range of people (students, faculty members, employers and administrators) at over 20 higher education institutions, although it did not have the opportunity to observe actual classes is session. Based on its interviews with many students and faculty members, the team concludes that the typical higher education teaching approach in Kazakhstan remains primarily didactic, relying mainly on a lecture format. Interactive participation occurs in study groups that follow the lectures and is often led by assistants. While these groups provide an opportunity for students to process material differently, there is a risk (given what the team observed regarding professional development) that assistants lack the pedagogical fluency to foster deeper learning. Moreover, despite the OECD review team’s inquiries, no specific examples were reported that would suggest that technologies are being used innovatively to support teaching that fosters deep learning, self-regulation, reflection and independence.

Teaching appears to be evaluated primarily by means of student course ratings although neither the students nor faculty were able to explain the consequences of unsatisfactory ratings. There is however an annual recognition process for excellent teaching. Each year, the “200 best teachers” across the system receive a monetary award of up to KZT 4 million. For these and other practices to have greater impact on quality, there needs to be follow-up in the form of teaching development initiatives to redress poor evaluations and to track the impact of teaching awards. For instance, empirical evidence suggests that relatively small investments in the form of grants targeted for teaching development can have a direct impact on improving the quality of curricula and teaching, and help develop a community of practice (Hum et al., 2015). Some of the funding required to support such an approach might reasonably be identified by reducing the size of the annual teaching awards.

Academic development opportunities

To remain relevant in a changing world, higher education institutions need to ensure that faculty and academic leaders have the ability to make ongoing changes to curricula and instructional processes. Dedicated teaching and learning development units based within higher education institutions are the primary supports for academic development. The academic literature explores models of both centralised and decentralised units, and examines the expertise required to foster quality learning and teaching (Saroyan and Freney (eds.), 2010). There is also much to be learned both from successful national programmes and teaching development units within institutions (see Box 2.7).

Box 2.7. National and institutional activities targeting professional development

Ireland’s National Forum for the Enhancement of Teaching and Learning in Higher Education offers a good example of promising practices at the national level. It was established by Ireland’s Minister for Education and Skills to bring together leaders, administrators and teachers and to draw on their expertise to shape best practices across all higher education institutions.

The Forum’s activities include for instance “Pre-Specified Nationally Coordinated projects” which address specific sectoral-level priorities. Four projects are currently underway or planned on higher education infrastructure, data mining, digital education and open access education. Other activities include seminars, professional development, awards and a fund (the “Teaching and Learning Enhancement Fund”) that seeks to build capacity to transform higher education.

The University of Sydney’s Institute for Teaching and Learning provides professional learning opportunities which are based on an Academic Professional Learning and Development Framework, and are delivered through online programmes and courses. The Institute also provides a range of e-learning services to teaching staff. These include supports for communication, assessment and collaboration, as well as learning materials and objects. Faculty members can also share practical advice on teaching with colleagues through a portal “Teaching Insights”.

Sources: The National Forum for the Enhancement of Teaching and Learning in Higher Education, (n.d.), www.teachingandlearning.ie; Higher Education Research and Development Society of Australasia, (n.d.), http://herdsa.org.au/; The University of Sydney, (n.d.), www.itl.usyd.edu.au/.

Academic development is typically delivered within institutions or by a cluster of institutions that have similar faculty profiles, share the same mission and are located close to each other. This allows interventions to target the competencies that are most relevant to specific educational contexts. Sustainable and nimble models focus on developing a particular expertise within a group, and using members of that group as trainers who go on to train others.

In Kazakhstan, faculty members are required by law to participate in professional development activities every five years. Such development does not take place within higher education institutions themselves, though. Rather, it is co-ordinated or offered centrally by the National Center for Professional Development (ORLEU). The standard training for higher education faculty includes four modules on innovative methods, independent learning for students, information technology, and management and new criteria for assessment. The courses are offered on-line for their first two weeks, followed by a test. The remainder of the programme is offered face-to-face. Trainers are typically Kazakhstanis, though occasionally international experts are used. ORLEU also partners with a number of foreign universities and organisations – including the University of Valencia, New Castle University and Pearson Publishing – to deliver academic development to faculty in international settings; 933 individuals have benefited from this opportunity.

The number of individuals who have benefited from ORLEU training programmes was reported to the team as being between 4 000 and “more than 7 000”. In other words, between 9% and 17% of current or recent faculty members may have benefited from some academic development activity. In addition, the Graduate School of Education of Nazarbayev University is also involved in academic development, in particular of leadership capacity in senior academic management. The scope of this project, and its impact on quality, were not evident to the review team.

Making continuing academic development mandatory is an important achievement, and represents a potentially powerful mechanism for ensuring much-needed professional capacity in academics, teachers and academic leaders. However, Kazakhstan’s centralised model, offering a fixed set of training modules, may not be optimal. Faculty members in Kazakhstani higher education institutions are very diverse with respect to their academic and pedagogical qualifications. This means that they will often require different types of interventions. In the face of the urgent challenge of raising overall levels of faculty proficiency and qualifications, a centralised model – which lacks the multiplier effect of a “teach trainers to train others” approach – is unlikely to be an adequate response.

Higher education and Kazakhstan’s school teachers

A related issue is the quality of the “pre-service” and “in-service” training that Kazakhstani higher education provides to primary and secondary school teachers. Roughly one-quarter of all higher education students are enrolled in education programmes, and the majority of these are in teacher education programmes. Previous OECD research has raised concerns with respect to teacher education in Kazakhstan (OECD and the World Bank, 2015; OECD, 2014c). While the present review did not analyse in depth teacher education programmes, various factors signal that the professionalisation of teachers would likely require an overhaul in their design and delivery.

Currently, the lack of national standards for teachers - an important element of professionalisation - presents an obstacle to high and consistent quality in initial and continuous teacher education in Kazakhstan. Having clear and concise standards would certainly help to clarify expectations and serve as a framework for the selection of candidates, enable better judgement of competences and provide guidelines for professional development and career progression. It seems that the current framework for teacher professional development is not in phase with teachers’ needs. This is due to the high level of specialisation of degrees limiting the flexibility of the teacher on the labour market, the lack of assessments identifying the competences to teach, and the limited autonomy of institutions designing their teacher education programmes.

One way to free up considerable resources, which might then be invested in modern programmes and the professionalisation of teachers, would be for education programmes to stop providing coursework in disciplinary areas (e.g. math, science, Kazakh language). Based on its observations, it is the opinion of the OECD review team that this coursework risks being an inferior replication of similar programmes offered in the more comprehensive universities. Instead, education programmes could take in students who already have a degree in a subject area, and enrol them in a one- or two-year graduate-level programme whose sole focus would be developing professional and pedagogical capacities. Greater time and emphasis on developing teachers’ pedagogical skills will be critical to effecting a transition towards more student-focused approaches to teaching and learning. Alternatively, education programmes could team up with faculties at other institutions to offer a joint programme that might lead to three kinds of degree: an undergraduate degree, a Bachelor of Education degree, and a professional teacher education which entitles a graduate to teach in schools (see Box 2.8 for an example from Ontario).

Box 2.8. The Ontario Institute for the Study of Education’s Teacher Education Undergraduate Program

The Concurrent Teacher Education Program is a five-year programme in which students complete the requirements for an undergraduate degree, a Bachelor of Education degree and a professional teacher qualification simultaneously. The undergraduate degree programme provides students with the necessary knowledge for their teaching areas, along with several introductory education-focused courses and field experiences. The Bachelor of Education degree is focused on developing essential skills to effectively apply the subject-specific knowledge gained in the undergraduate degree programme to the teaching profession. It connects theory to practice.

Sources: Ontario Institute for Studies in Education (n.d.), Program Strucuture, www.oise.utoronto.ca/BEdConcurrent/About_CTEP/Program_Structure.html.

Quality of outputs and outcomes

There are multiple possible indicators of quality outputs and outcomes. These might include completion and dropout rates, time to completion, measures of skills gained, graduate employability, graduate labour market outcomes (including employer satisfaction measures) and graduate satisfaction. Indicators can also be linked to each other. For instance, linkages between input and output indicators can be used to measure efficiency. If data is reliable and its time series are robust, many output and outcomes indicators can be quantified and tracked over time to explore trends and changes in patterns.

In this section, one key outcome will be explored: graduate employment (which itself presupposes the instructional skills output, “graduate employability”).

Overall outputs of higher education in Kazakhstan

Among others, policy makers face two key questions when evaluating higher education systems:

Is the system producing a sufficient number of graduates overall?
Does the mix of graduates correspond to the skills needs of the economy?

The mix of enrolments by field of study is one aspect of a response to the latter question – although analysis also needs to take into account the mix of (and level of) cognitive, social and emotional skills developed in students. In the absence of reliable measures of these transversal skills in Kazakhstan, the OECD review team has focused its analysis on the issue of disciplinary mix.

In the 2014-2015 academic year, a total of approximately 475 000 students enrolled in higher education institutions. As Chapter 3 explains in greater detail, government-issued grants (provided to roughly one-quarter of students) are the mechanism used to draw students into disciplines where there is projected need for growth.

If the government’s intent is not merely to ensure a minimum number of graduates in certain fields, but rather to encourage more students to enrol in these disciplines through signals about their relative importance, then it is unclear that it is succeeding (Figure 2.7). The fields where state grants are most numerous are not necessarily the fields of highest enrolment.

However, even assuming the public policy intention is merely to ensure that there are a minimum number of graduates in any given field, the mechanism by which that State determines these quotas, and the actual quota mechanism itself, face several challenges. Leaving aside issues raised in Chapter 3 (regarding the equity implications of the state grant) and Chapter 6 (regarding the deadweight losses that the grant system incurs, since it presumably pays individuals to do something they would themselves do without incentive), if the State Order approach is to be effective in a market or mixed market (rather than a planned) economy, it would need to be based on exceptionally good labour intelligence about future labour markets.

To a certain extent, the State does have some intelligence – but only (and even here, imperfectly) on the demand side, that is, regarding its own hiring decisions and its planning assumptions. It is the OECD review team’s understanding that the State Order is based largely on input from various ministries across the government who are using information derived from their own business plans, any assumptions they might make about the effects of large-scale public initiatives (e.g. an industrialisation programme), and perhaps (to an unclear extent), input from the sectors that they must often work with.

The risks of this approach are fourfold and interrelated:

Insofar as it can forecast labour market needs, the model can primarily only forecast them in the shorter term – whereas higher education trains students (at least youth, who make up most of the Kazakhstani student body) to prepare them for a life of work.
The model is focused on an instrumentalist view of higher education disciplines – e.g. embodying the assumption that certain occupations neatly correspond to certain fields of study. In truth, though, in many OECD countries even graduates from highly “specialised” fields of study, such as engineering, end up working in a wide range of occupations. This is because individual fields of study build a range of skills beyond narrow technical ones.
The supply side of the labour market – i.e. who will be available to take up what kind of job – cannot be modelled by ministries based on their information about their own demand, and their related sector’s demand, for labour. There is a constant churn in labour markets as individuals (not just new graduates) enter and exit them.
There appears to be a lack of data in Kazakhstan to reliably model private sector (i.e. market) needs.

This latter point is worth some elaboration. A limited argument might be made for planning a State Order of some student spaces based on recent trends in employment outcomes data, if students are not acting on this information. For instance, if the relative wages of graduates of one discipline are consistently high, or consistently rising, that could be a market signal that such graduates are in demand (whether or not they are working in an occupation which “matches” their field of study). Such a signal would then, in theory, be picked up by students entering higher education, who would then “vote with their feet” for the most desirable disciplines (at least until these disciplines became saturated, at which point returns to their investments would begin to fall). Of course, markets do not typically work in this idealised way: students will lack information or may, for other reasons, show little interest in certain “in demand” fields of study. In that case, a limited argument might be made for public intervention (e.g. via grants) to address the market failure by encouraging enrolments.

The OECD review team also learned of work to develop a labour market anticipation model in Kazakhstan. The status of this model, and its inner workings, were not clear to the team. It is the review team’s understanding, though, that this model is not used to inform the state order of higher education spaces.

Looking forward, then, Kazakhstan needs to ensure that it invests in the kinds of labour market data that can help it understand how employers are using, and demanding, the skills graduates. Such data would include indicators such as employment and wages – but would also need to look at self-employment income and, ideally, at indicators in areas such as job quality and skills match. Surveys of employers to determine their level of satisfaction with the quality of graduates would also be very valuable were they to be systematised – as would be surveys of recent graduates. Such surveys, if conducted regularly, can provide ongoing and up-to-date input to institutions about their curricula and their quality of teaching. They can also provide governments and individuals with information about current and emerging labour market demands.

In the end, good information about graduate labour market outcomes should have students themselves, not governments, as its primary customer. Some version of a “State Order” (i.e. public support targeted at specific fields) will doubtless always be necessary to ensure that market failures do not distort the production of graduates – and that the upfront costs of certain less well-remunerated occupations are low enough to allow students to expect to make a reasonable return on their educational investment. Over time, though, Kazakhstan needs to shift towards a model where informed students are able to make good decisions about their studies based on reliable and well understood labour market information.

The needs of 21st century economies

As noted above, the state order system appears to be based on an instrumentalist view of higher education that focuses on vocational and professional knowledge and skills. Employers, however, are focused on a broader range of skills. For instance, using the database “Labor Insights”, Georgetown University’s Center for Continuing and Professional Education (CCPE) had identified the 15 key skills sought by employers. The top six are communication skills, organisational skills, writing, customer service, information processing and problem solving (CCPE, 2016). Similar findings are reported by surveys in a variety of countries. All this explains the growing focus of governments and higher education on ensuring that higher education students develop key cognitive, social and emotional skills that are transferrable across jobs. It is thus a matter of concern that important subjects which contribute to the development of imagination, creativity and collaborative skills do not figure after 7th grade in the Kazakhstani secondary curriculum, and are not being emphasised in higher education curricula (OECD, 2014c).

The 2016 OECD Multi-dimensional Review of Kazakhstan points to similar gaps. It argues that, despite Kazakhstan’s high literacy rates, individuals entering the workforce often do not possess the types of skills and competencies that would help propel growth in a range of sectors. It also highlights the need for an approach to skills development that takes into account what is needed by small and medium-sized enterprises – suggesting that such a strategy could also reduce brain drain and attract talent to the country (OECD, 2016).

In this light, a survey of employers carried out in 2014 by the “Sange” Research Center on Competitiveness (Sange Research Center, 2014) is revealing. It found that employers’ perceptions of the quality of education were not particularly positive. The general view was that programmes had a limited applied component, that there was a gap between the knowledge of students and curricula and the expectations of the labour market, and that graduates had to be retrained once they were hired fresh out of Kazakhstani higher education institutions. Employers stressed underdeveloped leadership qualities, poor communication skills especially in English, limited analytic and problem-solving abilities, and shortfalls in the ability to think and work independently and to take risks. They acknowledged, though, that graduates of certain institutions consistently stood out as being much better prepared. It should be noted that Kazakhstan is not alone here. According to the World Bank Business Environment and Enterprise Performance (BEEPS) Survey in 2008, more than 40% of firms were dissatisfied with the availability of skilled workers in the middle-income CIS countries (World Bank, 2012).

In the 2014 “Sange” survey, employers also pointed to attitudinal issues affecting graduates: unrealistic expectations in terms of position and pay vis-à-vis their qualifications and competencies; an expectation to be constantly awarded and acknowledged for the work they do; and overconfidence bred in them by their institutions about the types of employment they were likely to secure following graduation. Some suggested that shortcomings were partly due to low levels of co-operation between employers and higher education institutions, and the unwillingness of higher education institutions to receive feedback from employers. The group suggested that by investing time in building relationships with employers and creating networks of alumni, higher education institutions could significantly enhance quality.

The findings of this survey are in many respects consistent with what the OECD review team heard when meeting with employers (who were convened by higher education institutions themselves) across the country. It would be fair to report that many of the Kazakhstani employers indicated they were happy with the technical skills of graduates, if not necessarily with the other “transversal” skills identified above. Informants who represented international firms were rather less sanguine: they expressed concerns about levels of both technical and transversal skills.

Reports from international firms as employers do suggest that higher education is not training graduates to internationally benchmarked levels of skills – and that Kazakhstani employers do not necessarily expect or demand these levels of skills. This in turn suggests the danger of Kazakhstan falling into a “low-skills equilibrium”, where low expectations become a self-fulfilling prophecy. This would impede the further productivity gains that Kazakhstan needs to meet its ambitious targets for economic growth.

The shortcomings highlighted above once again underscore that outcomes are very much dependent on processes. The targeting and intentional development of cognitive, social and emotional skills at all levels of education stands out as one area of some urgency.

The clear message from employers is that they want graduates who not only have good domain knowledge, but can apply their knowledge and lifelong learning skills, can use language and technology effectively, can innovate and think outside the box, and can learn on the job (see Box 2.9).

Box 2.9. What skills outcomes to target?

Drawing on decades of educational, psychological, sociological and economic research, policy-oriented organisations such as the OECD, The Conference Board of Canada, Canada’s National Research Council, and the United States’ National Center on Education and the Economy have recommended that educational institutions foster the development of cognitive competencies (e.g. reasoning, creativity, intellectual openness); fundamental skills (e.g. the ability to use tools such as language and technology effectively, information management, the ability to use numbers, thinking and problem solving); team work skills (e.g. interacting in heterogeneous groups, participating in projects and tasks, leadership); and personal management skills (e.g. acting autonomously and conscientiously, demonstrating positive attitudes, being responsible, being adaptable, learning continuously and working safely). They have also recommended that curricula and complementary educational experiences, teaching approaches and pedagogies, and assessments all be aligned to foster these learning outcomes.

Sources: Conference Board of Canada (2000), www.conferenceboard.ca/topics/education/learning-tools/employability-skills.aspx (accessed 25 April 20106).

Pellegrino, J. (2006), “Rethinking and Redesigning Curriculum, Instruction, and Assessment: What Contemporary Research and Theory Suggests”, National Centre on Education and Economy for the New Commission on the Skills of the American Workforce, Pennsylvania.

Pellegrino, J. and M. Hilton (eds.) (2012), Education for Life and Work: Desirable Knowledge and Skills for the 21st Century, National Research Council and National Academic Press, Washington DC.

Pellegrino, J., N. Chudowsky and R. Glaser (eds.) (2001), Knowing what students know: the science and design of educational assessment, National Academic Press, Washington DC.

OECD (2008), Tertiary Education for the Knowledge Society, Volume 1, Special Features: Governance, Funding, Quality, www.oecd.org/education/skills-beyond-school/41266690.pdf.

Recommendations

This review recommends that Kazakhstan:

Place greater emphasis on “21st century” graduate outcomes. Make systematic modifications to the inputs that higher education uses and to the processes it employs, in order to help students better develop the skills, knowledge and competencies that prepare them for lifelong learning, and that help them succeed in a changing world.

Actively orient academic programmes towards the knowledge, skills and competencies that will be relevant in the face of continuous social and economic change in the 21st century (i.e. not just specific vocational skills). Ensure that curriculum and course content, teaching approaches and assessment methods are aligned to produce these outcomes. Build approaches that develop analysis, application, creativity and divergent thinking into all programmes.
Make certain that complementary learning experiences, such as internships and extra-curricular activities, also help students acquire a broad range of skills.
Fully develop a useful, modern and easy-to-use National Qualifications Framework – with employer input playing an important role in this process. Align the specifications of knowledge, skills and competencies for each level of the framework with international benchmarks (e.g. those found in the European Qualifications Framework). Institutions should rigorously apply the framework when designing and implementing the curriculum, and when assessing and determining credit equivalencies.
Orient the UNT towards assessing specified outcomes that correspond to those outlined in the modern National Qualifications Framework. Supplement the UNT with other tools (e.g. a separate entrance examination, grade point averages at the secondary level, personal statements) that are appropriate for controlling student admissions into higher education.
Enhance flexibility in course selection and encourage cross-disciplinary course choices.

Reinforce linkages between higher education institutions and the employer community.

Ensure that higher education programmes and institutions – and the higher education system as a whole – further develop their collaborative links with employers. Move from the current local ad hoc approach of employer engagement to a more structured one.
Ensure that internships and other work-study opportunities actively expose students to authentic work-related situations. Explore alternative approaches (e.g. co-ops). Evaluate the impact of work experiences.
Consider offering undergraduate students formal research and inquiry approach experiences.
Reinforce faculty members’ linkages to the labour market, so that they keep abreast of current practices and needs. Encourage team teaching that pairs academics with practitioners.

Put in place decentralised structures and mechanisms that efficiently and effectively enhance the qualifications and the professional experiences of academics, teachers and academic leaders.

Provide academic and professional development opportunities locally for all core academic staff and academic leaders. Use qualified experts to deliver this training. Focus in particular on developing expertise in programme and course design, student-centred teaching approaches, and the assessment of cognitive, social and emotional skills.
Formalise mechanisms to systematically transfer any existing faculty development approaches employed at Nazarbayev University and the national universities to other higher education institutions. Evaluate the impact of these approaches.
Ensure that faculty members with the highest qualifications are well distributed across the higher education system.
Review the current faculty workload structure, ensuring that total classroom contact hours are not so high that they detract from faculty members’ performance of other instructional and research duties.
Conduct a careful analysis of teacher education programmes, with a view towards modernising their curricula and design, and thus better professionalising future and current teachers. Consider the option of offering a teacher education programme at the graduate level.

Put in place robust mechanisms to facilitate continuous quality improvement.

At the system level:

Create conditions that will enable higher education institutions, the accreditation process, and other quality assurance processes to conform fully to Bologna principles and standards.
1. – Shift from monitoring for control to a formative evaluation approach that supports improvement.
2. – Engender trust in processes such as accreditation that have been put in place for quality assurance: ensure that these processes are reliable, and that the responsibilities and accountabilities of the entities that oversee them are clear.
Set achievable and realistic targets and metrics for assessing the quality of different types of higher education institutions. Reward improved performance.
Embrace evidence-based practice. Use data analytics to make informed policy decisions. Ensure that data and their interpretation are reliable and publicly available.

At the institutional level:

Strengthen internal quality assurance. Conduct cyclical reviews of academic programmes, engaging faculty in the peer review process. Reduce reliance on rote reporting.
Actively use feedback from student course ratings as well as other inputs (e.g. self-reflection, teaching portfolios, etc.) to improve teaching and learning.
Benchmark programme content and outcomes against peer programmes at an international level.

Support higher education with a strong and well-disseminated system of labour market information that reports on the outcomes of higher education graduates.

Over time, shift away from a central planning approach (embodied in the current state order system), and move towards one that provides students with sophisticated labour market information that empowers them to make choices that reflect economic demand for a broad range of skills.
Retain, as necessary, targeted funding approaches that specifically address labour market failures. Move away, though, from approaches that merely fund students to do what they otherwise would have done on their own initiative.

References

Boyer Commission on Educating Undergraduates in the Research University (1998), Reinventing Undergraduate Education: A Blueprint for America’s Research Universities, Carnegie Foundation for the Advancement of Teaching, US, www.adelaide.edu.au/rsd/evidence/related-articles/Boyer_Report.pdf (accessed 10 February, 2016).

Business Higher Education Round table (n.d.), www.bhert.com/about.html

Christensen Hughes, J. and E. Mighty (eds.) (2010), Taking Stock: Research on Teaching and Learning in Higher Education, School of Policy Studies, Queen’s University, McGill-Queen’s University Press, Kingston, Canada.

Conference Board of Canada (2000), www.conferenceboard.ca/topics/education/learning-tools/employability-skills.aspx (accessed 25 April 2016).

Donovan, S., J. Bransford and J. Pellegrino (1999), How People Learn: Bridging Research and Practice, National Academy of Sciences, Washington DC.

Douglass, J. (ed.), (2016), The New Flagship University: Changing the Paradigm from Global Ranking to National Relevancy, Palgrave MacMillan, New York, https://doi.org/10.1057/9781137500496.

EACEA (2010), State Program of Education Development in the Republic of Kazakhstan for 2011-2020, European Commission, Tempus, http://eacea.ec.europa.eu/tempus/participating_countries/overview/Kazakhstan.pdf. (accessed 25 April 2016).

European Association for Quality Assurance in Higher Education (ENQA) et al. (2015), Standards and Guidelines for Quality Assurance in the European Higher Education Area, Brussels, www.enqa.eu/wp-content/uploads/2015/11/ESG_2015.pdf.

Fadel, C. (2014), “21st century education”, presentation made at the OECD “What Works” Conference on fostering quality teaching in higher education, Paris, 8-9 April.

Georgetown University’s Center for Continuing and Professional Education (CCPE, 2016): http://scs.georgetown.edu/departments/5/center-for-continuing-and-professional-education/news/5504/top-skills-job-seekers-in-2016

Gibbs, G. (2010), Dimensions of Quality, The Higher Education Academy, York, United Kingdom.

Hénard, F. (2010), Learning our Lesson: Review of Quality Teaching in Higher Education, OECD Publishing, Paris. https://doi.org/10.1787/9789264079281-en

Hum, G., C. Amundsen and E. Emmioglu (2015), “Evaluating a scholarship of teaching and learning grants program: Our framework, process, initial findings, and reflections”, Studies in Educational Evaluation, Vol. 46, pp. 29-38.

Inoue, K. and L. Gortazar (2014), Strengthening Kazakhstan’s education systems: An analysis of PISA 2009 and 2012, World Bank Group, Washington DC, http://documents.worldbank.org/curated/en/453571468050713745/Strengthening-Kazakhstans-education-systems-an-analysis-of-PISA-2009-and-2012.

Jenkins, A. (2004), A Guide to the Research on Teaching-Research Relations, Higher Education Academy, York, United Kingdom.

Kalanova, S. (2015), “Kazakhstan: A higher education system opening to Europe”, presentation made at a seminar organised by the Agencia de Calidad y Prospectiva Universitaria de Aragon, 17 April, Zaragoza, Spain.

Law on Education (2007), Law on Education of the Republic of Kazakhstan, of 27 July 2007, No.319 –III ZRK, About education, last edition of 09 April 2016.

Marton, F. and R. Säljö (1976), “On Qualitative Differences in Learning: 1- Outcome and Process”, British Journal of Educational Psychology, Vol. 46, pp. 4-11.

MCGILL University’s (2015), www.mcgill/study/2015-2016/faculties/arts/undergraduate/ug-arts-program.

MESRK(2016), The State Programme for Education and Science Development in the Republic of Kazakhstan 2016-19, RK Presidential decree as of 1 March, 2016.

MESRK (2014-2015), Statistics, Ministry of Education and Science of the Republic of Kazakhstan, http://edu.gov.kz.

MESRK (2010), The State Program for Education Development in the Republic of Kazakhstan 2011-2020, RK Presidential decree as of 7 December, 2010, Number 1118.

MNERK (2014-2015), Ministry of National Economy of the Republic of Kazakhstan, Committee on Statistics, www.stat.gov.

National Center on Education and the Economy (2008), Tough Choices or Tough Times: The Report of the New Commission on the Skills of the American Workforce, Jossey-Bass, San Francisco.

NSSE (2015), “Engagement Indicators and High-Impact Practices”, National Survey of Student Engagement, http://nsse.indiana.edu/pdf/EIs_and_HIPs_2015.pdf.

Nazarbayev University Graduate School of Education (2014), Development of Strategic Directions for Education Reforms in Kazakhstan for 2015-2020, Diagnostic Report, Indigo Print, Astana.

Nerad, M. and B. Evans (eds.) (2014), Globalization and Its Impacts on the Quality of PhD Education: Forces and Forms in Doctoral Education, Global Perspectives on Higher Education, Sense Publishers, Rotterdam, https://doi.org/10.1007/978-94-6209-569-4.

OECD (2016), Multi-dimensional Review of Kazakhstan: Volume 1. Initial Assessment, OECD Development Pathways, OECD Publishing, Paris, https://doi.org/10.1787/9789264246768-en.

OECD (2014), Skills Beyond School: Synthesis Report, OECD Publishing, Paris, https://doi.org/10.1787/9789264214682-en.

OECD (2014a), PISA 2012 Results: What Students Know and Can Do: Student Performance in Mathematics, Reading and Science (Volume I, Revised edition, February 2014), PISA, OECD Publishing, Paris, https://doi.org/10.1787/9789264208780-en.

OECD (2014b), Reviews of National Policies for Education: Secondary Education in Kazakhstan, OECD Publishing, Paris, https://doi.org/10.1787/9789264205208-en.

OECD (2012), Better Skills, Better Jobs, Better Lives: A Strategic Approach to Skills Policies, OECD Publishing, Paris, https://doi.org/10.1787/9789264177338-en.

OECD (2008), Tertiary Education for the Knowledge Society, Volume 1, Special Features: Governance, Funding, Quality, OECD Publishing, Paris, www.oecd.org/education/skills-beyond-school/41266690.pdf.

OECD/The World Bank (2015), OECD Reviews of School Resources: Kazakhstan 2015, OECD Publishing, Paris, https://doi.org/10.1787/9789264245891-en.

OECD/The World Bank (2007), Reviews of National Policies for Education: Higher Education in Kazakhstan, https://doi.org/10.1787/9789264033177-en.

Pak, N. (2010), “Integration of the Republic of Kazakhstan into Global Educational Scenario: from Reforms in Education to Quality Assurance Enhancement and World Ranking Participation”, Global Management Journal, Vol. 3, No. 2, http://globalmj.eu/?s=Natalya+Pak (accessed 20 February 2016).

Pellegrino, J. and M. Hilton (eds.) (2012), Education for Life and Work: Desirable Knowledge and Skills for the 21st Century, National Research Council and National Academic Press, Washington DC.

Pellegrino, J., N. Chudowsky and R. Glaser (eds.) (2001), Knowing what students know: the science and design of educational assessment, National Academic Press, Washington DC.

Priscilla King Gray Public Center (n.d.), http://web.net.edu/mitpsc/whatwedo/work-study/.

Sagintayeva, A. et al. (2015), “University-Industry Partnerships in Kazakhstan as a Strategy for Higher Education Reform”, presentation at the European Educational Research Conference, Berlin, 8-11 September.

Sange Research Center (2014), “Education and Research Assessment in Kazakhstan through GCI indicators” (in Kazakh).

Saroyan, A. and M. Frenay (eds.) (2010), Building Teaching Capacity in Universities: From Faculty Development to Educational Development, Stylus, Sterling, VA.

Satpayev, D. (2014), “Corruption in Kazakhstan and the Quality of Governance”, IDE Discussion Paper No. 475, Institute of Developing Economies, Chiba, Japan, www.ide.go.jp/English/Publish/Download/Dp/pdf/475.pdf (accessed 30 January 2016).

Sawyer, K. (2006), The Cambridge Handbook of the Learning Sciences, Cambridge University Press, New York.

Schön, D.A. (1983), The Reflective Practitioner: How Professionals Think in Action, London, Temple Smith, London.

Schön, D.A. (1987), Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions, Jossey-Bass Publishers, London.

Soltys, D. (2014), “Similarities, divergence, and incapacity in the Bologna Process reform implementation by the former socialist countries: the self-defeat of state regulations”, Comparative Education, Vol. 51, No. 2, pp. 179-195. https://doi.org/10.1080/03050068.2014.957908.

State program on Education and Science Development 2016-2019 (SPESD), http://adilet.zan.kz/rus/docs/U1600000205/.

Svinicki, M. and W. McKeachie (2014), McKeachie’s Teaching Tips, 14th ed. Wadsworth Cengage Learning, Belmont, CA.

Task Force on Higher Education and Society, (2000), Higher Education in Developing Countries: Peril and Promise, The World Bank, Washington DC.

The Economist (2015), “The value of University: Where’s the best?” www.economist.com/news/united-states/21677231-new-federal-data-reveal-which-colleges-do-most-their-graduates-pay-packets-they-are.

University of waterloo, https://uwaterloo.ca/find-out-more/co-op/.

Weimer, M. (2013), Learner-Centered Teaching: Five Key Changes to Practice, 2nd ed., Jossey-Bass, San Francisco.

Winter, L. et al. (2014a), “The Culture and Practice of Assessment in Kazakhstan: The Unified National Test, Past and Present”, in Educational Reform and Internationalisation: The Case of School Reform in Kazakhstan (pp. 106-132), Cambridge University Press, Cambridge.

Winter, L. et al. (2014b), The Culture and Practice of Assessment in Kazakhstan: An Alternative Model and the Future”, in Educational Reform and Internationalisation: The Case of School Reform in Kazakhstan (pp. 133-151), Cambridge University Press, Cambridge.

World Bank (2012), Kazakhstan - Student Assessment, Saber Country Report, www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2013/08/12/000356161_20130812163634/Rendered/PDF/799380WP0SABER0Box0379795B00PUBLIC0.pdf. (accessed 25 April, 2016).

World Bank (2002), Constructing Knowledge Societies: New Challenges for Tertiary Education, The World Bank, Washington DC.

World Economic Forum (2016), The Global Competitiveness Report, 2015-2016, Geneva, www3.weforum.org/docs/gcr/2015-2016/Global_Competitiveness_Report_2015-2016.pdf. (accessed on 25 April 2016).

Yergebekov, M. and Z. Temirbekova (2012), “The Bologna Process and Problems in Higher Education System of Kazakhstan,” Procedia - Social and Behavioral Sciences, Vol. 47, pp. 1473-1478.

Note

← 1. This refers to students below level 2 of proficiency in mathematics according to the PISA 2012 framework.