4. How do enterprises make decisions about training?
Enterprises invest in training because they believe that skilling their workforce helps them adapt to technological change, stay competitive, integrate new recruits and ensure health and safety of staff. To maximise these results, enterprises need to make the right decisions about training. This chapter presents existing and new evidence on how enterprises make decisions about training. It highlights where enterprises fall short of effective practices identified in the academic literature, thereby pointing to a need to build the capacity of enterprises and employees to make better training decisions.
How training is designed, delivered and evaluated matters for maximising its impact on employees, enterprises and societies as a whole. The academic literature suggests that making effective training decisions depends on implementing well-designed processes and practices, but provides limited empirical evidence. The case studies show that having a (general) HR function has no correlation with the generosity of training provided. However, firms that appoint a dedicated training manager/specialist provide more training. If they have a say, employees influence decisions directly, rather than through employee representatives, although there are cross-country differences.
Training needs assessment is a key tool for enterprises to understand their human resource requirements and develop plans to meet these through training. However, empirical evidence on the adoption and effectiveness of needs assessment is limited. New evidence from the case studies shows that enterprises use a variety of methods and approaches to assess their training needs. Based on the methods used, it seems that needs assessment is more reactive than strategic in most firms. Market analysis, foresight and other future-oriented methodologies are rarely applied. Training needs are typically assessed by HR and management functions, while employees are involved in around half of the enterprises included in the sample. As above, the involvement of employees is usually direct, with employee representatives playing a limited role.
Enterprises are confronted with make or buy decisions when delivering training. Existing research has analysed this trade-off mainly at the theoretical level. New evidence from the case studies suggests that enterprises make decisions to outsource training based on the availability of expertise in-house, costs and the possibility to customise training opportunities. Programmes targeting soft skills and language courses are typically outsourced. Evidence from the case studies also suggests that most enterprises do not have a structured process for selecting external providers.
Understanding how individuals gain access to training is crucial, given the large inequalities observed in training participation among employees. The existing literature provides some theoretical considerations on the issue, but empirical evidence is scarce. New evidence from the case studies shows that line managers are often gatekeepers when it comes to giving individuals access to training. It also shows that enterprises apply three types of approaches when giving access to training: Training participation is imposed top-down in one in three enterprises in the sample, most frequently when it comes to obligatory training and in low-tech manufacturing and less knowledge intensive services. One in four enterprises adopts a balanced approach, in which managers and employees negotiate access. The remaining enterprises apply a bottom-up approach that allows employees to access training on their own accord. There is evidence that some firms with a high-skilled workforce are empowering employees through individual training budgets and self-directed online training.
Enterprises use only basic methods to assess the outcomes of training, despite spending significant amounts of time and resources on its delivery. New evidence from the case studies suggests that most enterprises rely on informal feedback, employee surveys or observation at work to assess the results of training. Obstacles to training evaluation include the difficulties of measuring skills and lack of time. Some firms make a deliberate choice not to evaluate training, as they see a trade-off between monitoring and motivating employees.
Organisations invest in training because they believe that skilling their workforce helps them adapt to technological change, stay competitive, integrate new recruits and ensure health and safety of staff (Chapter 3). Yet, to achieve these outcomes, enterprises and employees need to make the right decisions about training.
This requires them to adopt well-designed internal processes and practices to maximise efficiency and effectiveness at all stages of organisational decision-making. These steps include assessing training needs and developing training plans, deciding if training should be delivered internally or externally, followed by selecting individuals into training and evaluating the outcomes of training for individuals, teams and the enterprises as a whole (Figure 4.1). The research literature shows that the way training is designed, delivered and implemented matters to the impact it will have on employees, enterprises and economies as a whole (Salas et al., 2012[1]).
This chapter presents existing and new evidence on how enterprises make decisions about training. It highlights where enterprises fall short of the good practice identified in the academic literature, thereby pointing to a need to build the capacity of enterprises and employees to make better training decisions. It investigates i) who makes decisions about training in enterprises; ii) how enterprises assess their training needs and plan training; iii) when enterprises use external providers to deliver training; iv) how they make choices about which individuals receive training; and v) to what extent enterprises evaluate the results of training.
Training in enterprises is a shared responsibility of employees and employers. Involving both actors in organisational decision-making on training helps balance enterprise needs, which are often profit-related, and individual needs, which are typically career or life-course related.
On the employer side, it is intuitive that having dedicated management functions, such as human resource managers responsible for the planning, implementation and evaluation of training improves the training offer in enterprises. In fact, the lack of such functions in small and medium enterprises is frequently seen as responsible for the absence of strategic attention paid to training in SMEs (Cardon and Valentin, 2017[2]). Yet, evidence on the effect of such functions on training is limited. Looking at the role of employees, there is a well-developed body of research suggesting that employee involvement in organisational decision-making in general has a positive effect on work-place well-being and company performance (Eurofound and Cedefop, 2020[3]). Evidence on the effect of employee involvement on training is more limited. Recent studies found a positive relationship between different kinds of employee involvement – direct, indirect (through representative bodies) and mixed – and training provision (OECD, 2019[4]).
New data from the enterprises case studies can help shed light on these evidence gaps. The interviews provide detailed insights about who makes decisions about training in enterprises and if decision-making structures bear any relationship to training outcomes.
Existing evidence on who makes training decisions
Dedicated human and financial resources
On average across the EU-27, less than two in five enterprises (39%) have a specific person or unit responsible for organising training (Figure 4.3). There are large differences between countries. While 56% of all enterprises in the United Kingdom have such structures, only 11% of enterprises in Latvia do. Within countries, smaller enterprises are less likely to have a specific person or unit responsible for organising training, and decisions are instead made by the owner or CEO (Cardon and Valentin, 2017[2]). Across the EU-27, 32% of small, 51% of medium and 80% of large enterprises have dedicated training functions on average. Differences between sectors are limited (not displayed in graph).
Even where organisational structures responsible for training exist, these are not always equipped with specific financial resources (Figure 4.3). Only one in four enterprises across the EU-27 (24%) has a specific budget for training. Again, there is significant variation between countries and differently sized enterprises. The shares of enterprises in a country that dedicate specific human and financial resources to training are strongly correlated (r=0.72).
Role and extent of employee involvement
Involving employees in decision making about training can increase their motivation to train and help firms to better identify individual and collective skill needs. This can maximise the effectiveness of training. There are two key ways in which employees can voice their views and participate in organisational decision-making around training (Bryson and Zimmermann, 2020[5]; Eurofound and Cedefop, 2020[3]): participation can take place directly, for example through meetings and discussions with line managers, cross-division or company-wide meetings on training matters. Participation can also take place indirectly in the form of workplace social dialogue, where employee representation structures engage in decision-making about training on behalf of employees. Such structures include works councils, trade union delegations or ad-hoc consultative committees.
Data from the CVTS is limited to indirect involvement and shows that the involvement of employee representatives in training-decisions varies between countries, size of enterprises and between the specific aspects of organisational decision-making. Overall, however, the involvement of employee representatives in organisational decision-making on training is limited. On average across the EU-27, employee representatives participate in setting overall training objectives in 6% of enterprises, in defining the type of training in 5% of enterprises and in the selection of training participants in 4% of enterprises, according to CVTS data.
Considering the involvement of employee representatives in objective setting as an example, illustrates the strong variation across countries and company size classes (Figure 4.4). Involvement of employee representatives is highest in Norway and the United Kingdom by a vast margin. Latvia and Poland display the lowest share of enterprises with employee involvement. Typically, the share of enterprises involving employee representatives in objective setting increases by company size, sometimes dramatically so. In the case of France, for example, this is due the fact that a training commission (commission de la formation) with the involvement of employee representatives is obligatory in enterprises with 300 employees or more.
New evidence on who makes training decisions
Evidence from the case studies corroborates these findings and provides some more detailed insights on who makes training decisions in enterprises. In particular, it offers new information on the relationship between the existence of HR and training managers and the generosity of training, as well as the extent of direct and indirect employee involvement in decision-making on training.
Existence, characteristics and role of human resource and training function
The vast majority of enterprises interviewed for this study have a human resource function (>80%), but their capacity varies by company size. In medium-sized enterprises, company owners or directors take on HR responsibilities as part of their job. As enterprises get larger, they tend to upscale their HR functions. Enterprises with more than 1 000 employees often have large HR departments, some with multiple training managers. The typical HR representative interviewed in the context of this study holds a tertiary qualification and has a positive attitude to life-long learning, generally demonstrated through his or her own participation in learning. Most enterprises without HR function are medium-sized. Where companies have no HR function, this is typically due to high associated costs (‘cannot afford HR given the company size’), having HR responsibilities at a corporate level or having outsourced HR functions to external providers.
Around one in four enterprises included in the case studies employ a person with dedicated responsibilities for training within this HR function. Where they exist, these are referred to as ‘training managers’, ‘training specialists’ or ‘head of learning’. Their responsibilities typically span the complete decision-making process of training from planning, organising the implementation and evaluating the training, as exemplified by an Estonian enterprise providing knowledge-intensive services.
A training specialist is responsible for collecting and analysing information on the training needs of employees and for organizing trainings. The specialist reviews information from an employee survey and from individual development conversations to identify training need at individual, departmental and company level. If the training need seems to apply to a larger group, it is organised by the training specialist.
The role of the HR (or the training specialist) function in organisational decision-making varies across contexts. According to evidence from the case studies, it is relatively rare that the HR function alone holds the full responsibility for training, from determining the training needs of the enterprise, over organising trainings, to sharing information about training opportunities with employees and having the final word on their participation. More often, HR is a broker and co-ordinator of various demands on the training-system made by different stakeholders in the organisation, be this management, finance, department managers and employees themselves (see also detailed description on different decision-making steps in subsequent sections).
Another level of complexity exists in enterprises or establishments that are part of a larger, often global, group. In these cases, not only does HR play a co-ordinating role internally, but also aligns training-decisions with HR functions at the central level. The case of a medium-sized Austrian branch of a large global computer manufacturer illustrates that the role of local HR can be much reduced in this set-up.
Training is not a central issue for the Austrian establishment. Training planning happens in the respective divisions at international level, which each have its own HR people and specific training strategies. These functions are based outside of Austria. The Austrian HR department focuses on salaries, agreements, labour law and recruiting and is involved when it comes to the training that applies to all employees in Austria (e.g. data protection, competition law, compliance).
Medium-sized manufacturer of computers or electronic products, Austria
Relationship between presence of training function and generosity of training
Evidence from the case studies can give an indication if the existence of an HR function and/or training manager is positively associated with the generosity of training provided. In this study, the generosity of training is defined as the coverage of the training offer – including all, the majority or the minority of employees – and the number of hours of training provided per employee and year.
When it comes to the existence of a (general) HR function and the generosity of training, results are mixed. While enterprises with an HR function are substantially more likely to offer training to all or most of their employees than those without (75% vs. 60%), they are actually less generous when it comes to training hours per employee and year (50% vs. 60%).
However, having a dedicated training manager/specialist seems to have a clear positive correlation with the generosity of training provided. Enterprises in the sample that have a dedicated training manager are substantially more likely to provide training to all or the majority of its employees than enterprises without such function (85% vs. 60%). Moreover, they are substantially more likely to provide above average or at least average number of training hours per year than enterprises without dedicated training managers/specialists (60% vs. 50%).
These relationships should not be interpreted as causal and are based on a small number of sampled enterprises (n=100). However, this finding adds to the limited empirical evidence on the subject by confirming what seems to be intuitive: enterprises that invest resources in having a dedicated training manager/specialist also invest more resources in training their employees. No clear patterns emerge on the relationship between the characteristics of the individual responsible for training in the enterprise, e.g. their gender, age, educational background or attitude towards life-long learning, and the generosity of the training offer in the enterprise.
Role of direct and indirect employee involvement in decision-making
In line with the literature, both direct and indirect employee involvement in decision-making about training can be observed in the case studies, with the role of direct involvement being relatively more common. Direct employee involvement can be formal, for example structured feedback provided in appraisal interviews, or informal, for example continuous exchange or one-to-one feedback. Indirect employee involvement is typically formal (Mowbray, Wilkinson and Tse, 2015[6]). Table 4.1 summarises the different types of involvement detected in the case studies and at what stage of organisational decision-making on training these are observed.
The role of direct employee involvement in decision-making is most strongly observed when it comes to selecting individuals into training. In many cases employee and line managers jointly discuss individual training needs, which can then feed into the assessment of the overall training needs of the enterprise (see sub-chapter on training needs assessment). Direct employee involvement also plays a role in the evaluation of training, as verbal feedback and employee surveys are the most common way of evaluating the outcomes of training (see sub-chapter on evaluating training). Econometric analysis on ECS data shows that enterprises using high-performance work practices, enterprises in the information and communication sectors and digitalised enterprises are more likely to facilitate direct employee involvement in training (Box 4.1).
Formal structures of indirect employee involvement and their prevalence vary across countries. For example, only one in four enterprises in Ireland interviewed in this study had a works council, compared to all but one enterprise interviewed in France. However, even where formal structures for representation exist, they played a limited role in organisational decision-making about training. Employee representatives contribute to the assessment of the training needs of the enterprise and the development of training plans in only one in five enterprises in the sample. The role of employee representation in identifying individual training needs is mostly limited to advocacy on behalf of a limited number of individuals who are in disagreement with the enterprise on training decisions.
In only a handful of enterprises in the sample is training a subject of negotiated agreements. Notable exceptions exist primarily in France and Italy where employee representation seems to play a greater role in organisational decision-making on training. In France, several case studies include descriptions of the role of the Social and Economic Committee (CSE), the employee representative body, in decision-making on training. Generally, company-training plans are discussed with the CSE and other union representatives. In some cases, the CSE has a sub-committee that deals with training decisions.
There are Social and Economic Committee (CSE) and union representatives within the company […] In order not to overload the CSE meetings, there is an Employment and Training Commission that reports to the CSE. […] Now, with the development of e-learning, employees’ representatives are oftentimes in a negotiation process because they do not always find it relevant. They are fighting to keep some training actions in the face-to-face mode.
Negotiated provisions may include, for example, the minimum number of training days per employee or other commitments to training in the enterprise. A high-tech manufacturer in Italy illustrates how such agreements look like in practice:
[…] in May 2020 the trade unions and the enterprise signed an agreement to try to overcome the crisis that is affecting the world industrial system and in particular the tension that is characterizing the Oil and Gas markets involving all the competences within the enterprise. The agreement is based on the following key concepts: (i) enrichment of the technical / professional background of all workers by extending executive skills through an incremental acquisition of skills; the increasing internalisation of knowledge and competences against externalisation in order to provide an added value to the technical process in the enterprise.
Econometric analysis of data from the European Company Survey 2019 (see Annex B for an illustration of the results and the methodology) provides additional insights on where direct employee involvement in training is more frequently observed. It shows that:
There is no relationship between company size and the influence of employee involvement.
Less hierarchical companies are less likely to involve employees in decision-making. This may seem counter-intuitive, but could be because these enterprises tend to be badly managed or because they rely on a different, predominately informal, learning model that requires less formal decision-making.
Some forms of high performance work practices, notably performance pay and worker autonomy are positively related to involving individuals in decision-making on training. This meets expectations, as employee involvement is part of the suite of HPWP.
Notably, the adoption of technology – with the exception of robots – is related positively to employees having an influence on training decisions.
Megatrends such as technological change, globalisation, ageing societies and the transition to a low-carbon economy are rapidly changing the skills enterprises need to achieve positive business outcomes. To keep abreast with these changes, enterprises need to increasingly engage in assessing current and future human resource requirements and in developing plans to meet these (Sparkmann, 2018[7]). Training needs assessment or analysis (TNA) is a key tool for companies in this context. At the most basic level, TNA is a gap analysis. It involves enterprises asking themselves i) what skills they need to achieve their objectives; ii) to what extent the existing workforce has these skills, and iii) if the gap between both is best addressed through training (Ferreira, Da Silva Abbad and Mourao, 2015[8]).
While the existing literature provides some theoretical insights to training needs analysis, it has only limited information on how it works in practice. Evidence from the enterprise case studies addresses this gap, by painting a comprehensive picture of how TNA takes place in enterprises and highlighting where enterprises may need support to prepare for current and future skill challenges.
Existing evidence on training needs assessment
The theoretical literature on TNA concerns itself with two issues. Firstly, it defines what is meant by training needs and, secondly, it develops different models that can be used to analyse these needs. Some academics criticise that models of TNA have become so ubiquitous that the term can now mean almost anything (Leigh et al., 2000[9]). Others criticise that one-size-fits-all models for TNA may not be appropriate for individual companies (Cotes and Ugarte, 2021[10]). A systematic literature review by Salas et al. (2012[1]) suggests that, practically, TNA can be broken-down into three components (see also McGhee and Thayer (1961[11])):
Job-task analysis: an analysis of job functions, task requirements and associated competences;
Organisational analysis: the analysis of strategic priorities of the organisation and its environment;
Person analysis: the assessment of individual skills and the type of training individuals needs.
There is only limited research on the application of TNA in practice. The bulk of empirical research on TNA has taken place in the health sector, followed by business administration (Ferreira, Da Silva Abbad and Mourao, 2015[8]). Overall, the evidence suggests that enterprises rarely take a systematic approach to establishing their training needs (Cotes and Ugarte, 2021[10]; Arthur et al., 2003[12]). Small and medium enterprises in particular seldom carry out TNA (Cardon and Valentin, 2017[2]; Macmahon and Murphy, 1999[13]). Despite the positive effects ascribed to TNA by the literature, empirical research on the effects of conducting TNA on training effectiveness and organisational performance is scant. A meta-analysis of the training and development literature from 1960 to 2000 failed to identify a positive association between needs assessment and training effectiveness, but was based on a small number of studies (Arthur et al., 2003[12]).
Firm-level survey data paint a slightly more optimistic picture, at least of the extent to which enterprises assess their future skill and competence needs (Figure 4.5).1 On average across the EU-27, 24% of enterprises regularly assess their skill and competence needs as part of the overall planning process in the enterprise. Consistent with the pattern, larger enterprises with more than 250 employees do so more frequently (56%) than smaller enterprises with 10-49 employees (21%). Among the countries for which data is available, enterprises in the United Kingdom most frequently assess their skill needs (49%), followed by Spain (40%) and France (37%). At the other end of the spectrum, less than 10% of enterprises regularly assess their skill needs in Bulgaria, Poland and Romania. In addition to the regular assessment of skills needs, 29% of enterprises conduct irregular assessments of future skill and competence needs on average across the EU-27 (not displayed in graph).
Note: Training refers to CVT and other training; enterprises for which an assessment of skills and competence need is part of the overall planning process in the enterprise.
Source: Eurostat, CVTS, 2015, [trng_cvt_33s].
Note: Excluding firms with less than ten employees; countries are ranked by their average degree of alignment; the degree of alignment is calculated as the overlap between the top three development priorities of firms and the top three training priorities (in terms of training hours). Each firm can score either zero (i.e. no overlap), low (i.e. one development priority is also a training priority), fair (i.e. two development priorities are also training priorities) or full alignment (i.e. complete overlap between development and training priorities).
Source: CVTS 2015, from OECD (2019[14]), Getting Skills Right: Future-ready adult learning systems, https://doi.org/10.1787/9789264311756-en.
Assessing future skill needs does not automatically imply training employees in line with these needs, as skill needs may be satisfied through recruiting new staff, for example. In fact, there seems to be limited overlap between the identified skill needs of enterprises and the training actually offered (OECD, 2019[15]). Only in 13% of enterprises in European OECD countries is there a complete overlap between the specific future skill needs identified and the focus of the training offered; an additional 30% of enterprises display a fair amount of overlap. Training for the future skill needs identified seems to be most common in Estonia, Ireland and Norway and least common in Spain and the United Kingdom (Figure 4.6).
New evidence on training needs assessment
The qualitative case studies provide richer information on the extent to which enterprises engage in strategic workforce planning more generally, which actors are involved and how training needs assessments are conducted in practice.
Extent of strategic workforce planning
About half of all enterprises included in the sample have a workforce strategy, which sets out how the organisation plans to prepare its workforce to deliver its business strategy. Such strategies include strategic considerations on the recruitment process, induction of new employees and training provision. In line with expectations and the existing evidence, larger enterprises in the sample are much more likely to engage in workforce planning. Close to all enterprises in the sample with more than 1000 employees have a workforce strategy, while the same is only true for 60% of enterprises with 250-499 employees and 20% of enterprises with 50-99 employees.
Workforce strategies are most common in high-tech manufacturing companies and least common in low-tech manufacturing. There is limited cross-country variation, with only Austrian enterprises in the sample being somewhat more likely to have a workforce strategy. Notably, all French companies with 300 or more employees have a workforce strategy, due to a legal requirement to engage in strategic workforce planning (la gestion prévisionnelle de l’emploi et des compétences, GPEC) every three years. The aim of the regulation is to avoid unforeseen skill needs and subsequent restructuring.
Where they exist, the planning horizon of the workforce strategies varies widely between one and ten years, with most enterprises having either annual or 5-year workforce strategies. Enterprises that are part of an international group may follow a workforce strategy that is set at the international level and adapt it to the local context.
Going beyond a workforce strategy, approximately 50% of enterprises in the sample have a training plan or training catalogue that outlines and lists available training opportunities. Not all enterprises that have a workforce strategy also have a training plan and vice versa. Training plans are typically updated on a yearly basis, or even more frequently if needed. One large Italian publishing company highlighted that the training plan for IT staff was updated more often that the plan for regular employees.
The training plan usually has a duration of one year. Only the plan for the IT division is shorter […] six months […]. Because of the pandemic, the IT department is practically indispensable, consequently having a continuous update of skills is very important.
Some enterprises, in particular in France, define training pathways for specific trades. These are long-term plans of continuing professional development, which have a modular structure. They can lead to a higher professional qualifications and salary increases.
Approaches used in training needs assessment
The case studies provide a number of new insights on how enterprises approach training needs assessment. There is no evidence that companies follow the three components of training needs assessment outlined in the literature, i.e. job-task, organisational and person analysis. Job-task analysis in particular is not commonly described by the enterprises interviewed, while aspects of organisational and personal analysis can be identified in the case studies. At least one in ten enterprises included in this study do not conduct any analysis of their training needs.
Enterprises use a variety of methods and approaches to assess their training needs, which fall in two categories: Top-down approaches – based on an assessment of business, client or regulatory needs – or bottom-up approaches – based on employee feedback or existing skills of the workforce. Table 4.2 provides an overview of the approaches to training needs analysis observed in the sample.
The use of bottom-up approaches is somewhat more common. A heavy reliance on employee requests can mean that training planning is more reactive than strategic. The HR manager of an Estonian manufacturer of electrical equipment, for example, draws information from the appraisal review process to assess individual training need.
The HR manager receives input from a software programme, which includes training requests from developmental conversations. If a number of people request a similar training (e.g. developing computer skills, training to communicate with problematic colleagues) and it is considered a valid request, the HR department starts planning the training.
Market analysis, foresight or other future-oriented strategic considerations are only mentioned as tools for TNA by a small number of enterprises interviewed. There are some notable exceptions of more innovative ways to assess training needs, such as foresight exercises or focus groups, as exemplified by an Italian service provider:
The new workforce strategy is in preparation. It aims to identify the unique selling point of each sector of activity. This means identifying which are the distinctive elements that each sector of activity would like to work towards in the long term. To this aim, the enterprise started to organise internal foresight workshops involving both the HR department and line managers.
However, much of how training needs are determined in enterprises remains a black box. For example, one in four enterprises state that they use a competence assessment, database, framework, matrix or mapping to determine training needs. Enterprises using these tools are primarily active in knowledge-intensive services or high-tech manufacturing. It is however unclear what lies behind these concepts and how this assessment of employee competences takes place in practice, given the notorious challenges competence assessment poses for enterprises. Frequently, enterprises simply state that an assessment of competences is carried out by teams and managers, without providing further details of how this takes places. The example of one rural manufacturer of chemical products in Italy illustrates this point:
The “Employee License” is an instrument to map and internally “certify” the competences and skills of a specific individual, to increase the visibility of skills and to keep track of training and courses that the individual has attended. The License is not a job profile (normative), but more of a descriptive tool. The Employee License was introduced in the past 1-2 years and is a tool to collect and make explicit the different competences, experiences and strengths that each worker has developed and achieved. It is also linked to the frequent delivery of tests and assessments related to different abilities and contents. The enterprise hopes that this tool will become an important tool in the assessment and sharing of competences and abilities, to further support personal and professional growth.
Actors involved in assessing enterprise training needs
In the enterprises included in the case studies, decisions on training needs are primarily made by HR and management functions (Table 4.3). Key players in the decision-making process are human resources – involved in three out of four enterprises – and top management, be this CEOs, managing directors or management boards – involved in two out of three enterprises. Some enterprises involve lower levels of management, such as division or department managers, when defining companies training needs. This is somewhat more common in large firms, where top-management may be further removed from training needs on the ground. In rare cases, finance departments are involved in decision-making, notably in the manufacturing sector. Where specific training functions exist in enterprises, they are involved in TNA, but such functions are rare as described above.
In practice, none of the actors outlined above assesses the training needs of the enterprise alone. The HR function typically co-ordinates training needs assessment, which involves multiple employees in management positions. This can be illustrated using the example of an Estonian manufacturer of electrical equipment included in the sample. In this Estonian firm, the HR manager compiles an initial training plan based on information from individual developmental conversations, i.e. the appraisal process. This draft training plan is then shared and discussed with line-managers, who identify gaps, assess at what level training should take place and decide if all training is strictly necessary. Following these exchanges, a revised plan is discussed with heads of departments, who will take a longer-term perspective and define when the trainings will be most relevant in the coming 1-2 years. Finally, a corporate steering group approves the training plan and then communicates it to employees.
In around half of the enterprises included in the sample, employees have a voice in assessing the training needs of the enterprise. In most cases, this input is made directly, i.e. by individuals themselves rather than employee representative bodies. It typically takes the form of discussions in the context of appraisal interviews, employee surveys and informal conversations. The example of a large Irish restaurant business illustrates why this direct input of employees is valued by enterprises.
Management believes that employees should have active roles in the business, as they are in contact with customers, clients, providers and competitors. As such, they are seeing trends and preferences that bring ideas to the business. Management welcomes their input and encourages all staff to come forward informally but also through formal avenues and meetings to contribute to the strategic direction and operational aspects of the division.
In the sample, a small number of enterprises had ad-hoc working groups on training, which provide an alternative channel for direct employee involvement. Their role is typically advisory. An example of such an approach is illustrated by a rural food manufacturer in Italy, who implements a consultation model called ‘philosophical practices’.
The workforce strategy has been based on the organisation of group-based training sessions called “philosophical practices” – focus groups – during which workers are asked to reflect on their training needs and identify methods to tackle these needs.
Elected employee representatives play a more limited role in assessing and defining enterprise training needs. They provide input to the definition of company training needs only in around one in five enterprises interviewed in this study. There is some cross-country variation and enterprises in France stand out in having strong involvement in the process. The strategic workforce planning for companies with more than 300 employees (GPEC) involves the social partners typically via the Social and Economic Committee (CSE). Employee representatives discuss the proposals for the competence and training needs assessment made by the management of enterprises. Some employee representatives interviewed suggested that these plans are often a formality and lack true strategic vision and are not accompanied by appropriate actions.
Reasons for not engaging in strategic workforce planning
One in three enterprises in the sample do not engage in workforce planning in a structured way, having neither a workforce strategy nor a training plan. The case studies provide some limited evidence of why this is the case. Reasons cited by enterprises include a lack of capacity, for example due to the lack of formal HR function; strategic planning not being part of the culture of the enterprise, due to it being a family business; and transitioning from being a small into a larger business. An interviewee from a medium-sized Italian wholesale company summarises how strategic planning may lag behind the reality of expanding a business.
The company has enjoyed rapid growth in the past years, doubling the number of employees in about five years. This growth has not been supported by specific strategies, tools or structures and looking forward – as the company continues to grow – interviewees do not report specific strategies to manage the workforce.
Several enterprises in the sample take a very conscious decision not to engage in workforce planning to maintain flexibility. These are often enterprises whose business models are based on quickly adapting to client-demand. New contracts may require staff to learn new skills on an ad-hoc basis. An interviewee from an industrial laundry service company in Austria illustrates this challenge.
Enterprises are faced with make or buy decisions when delivering training. This means that they must make a decision on delivering training in-house or outsource it to external providers. Existing research has paid limited attention on how companies arrive at this decision. Information from the case studies provides new evidence on what factors drive the choice of outsourcing training, which type of training are most likely to be outsourced and how enterprises select external training providers.
Existing evidence on training outsourcing
The existing research has analysed the problem of training outsourcing at a theoretical level as a typical make or buy decisions that enterprises need to take. Table 4.4 summarises the reasons for insourcing and outsourcing training discussed in the literature.
In theory, enterprises insource training when they have sufficient expertise in house or when training is important to sustain their competitive advantage, whereas they outsource when expertise is lacking and there are training providers available on the market. This is line with the resource-base-view of the firm, which has gained traction in the management literature over the past two decades. Consistent with transaction cost economics, enterprises should also insource training whenever the transaction costs of managing the relationship with external providers increase. This might happen because training is provided frequently or because the requirements of training programmes are difficult to specify, making it more challenging to secure effective co-operation.
Firm size ought to play a crucial role in the outsourcing decision. Smaller firms might face higher transaction costs in dealing with external providers, because they typically have less negotiation power. More importantly, firm size should determine whether outsourcing reduces delivery costs. Outsourcing can lead to lower costs of provision for smaller firms, which may benefit from economies-of-scale effects achieved by their providers. As firms increase in size, they will tend to insource training, as they can fully realise such economies-of-scale effects in-house.
Lastly, both outsourcing and insourcing may contribute to increasing the quality of training. On the one hand, delivering training externally can lead to a professionalisation of the training function, increasing access to best practices and timeliness. On the other hand, delivering training internally can result in programmes that are more tailored to the needs to the enterprise and generate a higher level of employee engagement, as well as supporting internal bonds and communication.
Research has so far failed to gather extensive empirical evidence on these theoretical predictions (Chaudhuria and Bartlett, 2014[16]). Using a sample of 100 Greek enterprises, the work by Galanaki, Bourantas and Papalexandris (2008[15]) is an exception. They reach three key conclusions: First, the outsourcing decision seems to be driven by the absence or availability of expertise rather than by cost considerations. Second, large companies are less inclined to rely on external providers, in line with the theoretical predictions on the economies-of-scale effects. Third, enterprises that provide training frequently or that have very specific training needs are more reluctant to engage with external providers, consistent with the theoretical predictions from transaction costs economics.
Firm-level surveys provide some evidence to corroborate and complement the theoretical predictions and findings from the HR research. According to CVTS data, enterprises in Europe are more likely to offer external rather than internal CVT courses, across all size classes and sectors (Figure 4.7). However, consistent with the literature, as enterprises grow in size, they tend to develop some internal training provision. Large enterprises are almost equally likely to offer internal (81%) and external (88%) training, whereas small enterprises rely mainly on external (50%) rather than internal (29%) courses. When looking at different industries, enterprises in financial, and information and communication (IC) services seem to be more inclined to deliver at least some training internally, relative to enterprises in other sectors.
Most enterprises, across all size classes and sectors, rely on private sector delivery, via training providers or other private companies, such as suppliers, customers or partner companies for external training provision. Publicly funded training providers do not seem to be a common choice. Their importance is similar across sectors, ranging from 10% in financial and IC services to 14% in industry, but is higher among large (21%) than medium (16%) and small (12%) enterprises. The relevance of employers’ associations varies across both sectors and size classes. They play a more important role among large (36%) and medium (32%) than small (24%) enterprises, and they seem to be more common in industry (29%) than financial and IC services (23%) or LKIS (24%) (not displayed in graph).
Note: Internal CVT courses are designed and managed by the enterprise itself, whereas external CVT courses are designed by external providers, such as education institutions, private training companies or chambers of commerce.
Source: Eurostat, CVTS 2015, [trng_cvt_01s , trng_cvt_01n2].
New evidence on training outsourcing
The HR literature provides some theoretical considerations of the drivers of outsourcing decisions, but there is a lack of robust empirical evidence applicable to most countries and sectors. The CVTS makes it possible to build a good understanding of how the reliance and the type of training providers varies across sector, size classes and countries. However, it is not well suited to provide information on what types of training are more likely to be outsourced and how enterprises select training providers. Insights from the qualitative case studies can shed light on the questions i) what factors drive the choice of outsourcing training for enterprises; ii) what types of training are more likely to be outsourced; and iii) how do enterprises select external training providers.
Factors driving the outsourcing decision
Almost all enterprises in the sample claimed that they rely on a mix of internal and external delivery, but large enterprises reported outsourcing training less frequently, in line with the existing evidence from both the CVTS and HR research. Other enterprise characteristics, such as sector or product market strategy, do not seem to play an important role in driving the outsourcing decision.
Three factors seem to explain these patterns: the availability of expertise, the reduction of costs and the customisation of training opportunities. As suggested in the literature, the availability or absence of expertise is a crucial driver of the decision to outsource training. Enterprises generally deliver training internally if they have knowledge on the subject, whereas they outsource it if they do not. This attitude was succinctly summarised by an interviewee in a large Italian low-tech manufacturing enterprise.
Training is partly delivered through internal trainers, and partly through external providers. Whenever specific knowledge of the internal processes and operations is needed, internal trainers are used. On other specific subjects, external trainers are selected.
Large enterprises tend to rely less frequently on outsourcing, because they have a larger knowledge pool they can draw upon, when organising training activities. For example, in a large multinational enterprise in Austria most courses are organised in-house and delivered by employees of the global group, because they have knowledge across many specific areas.
Reducing costs by exploiting economies of scale also plays an important role in the outsourcing decision. In line with the theoretical predictions from the research, large enterprises often deliver training internally, because it is less expensive than relying on external providers. For instance, a large KIS enterprise in France reports delivering most training internally because it is more cost-effective. Conversely, medium-sized enterprises are more likely to rely on external providers to benefit from economies-of-scale effects, as suggested the HR representative of a medium Estonian enterprise below.
Open courses by external providers are preferred because the training groups are not large and so it is more efficient to use open courses available for anyone rather than buy a custom-made course into the company.
Yet, even medium enterprises can insource some training to reduce costs, if it is delivered frequently. For example, a medium-sized enterprise providing industrial laundry services in Austria has a dedicated employee for hygiene training, because it needs to be delivered multiple times during the year.
Large enterprises go as far as setting up their own in-house training centres or separate training subsidiaries. This is driven by both costs and quality considerations. Establishing training centres can contribute to develop training programmes that are more specific or customised, as pointed out, for example, by the HR director of a large high-tech manufacturing French enterprise.
The Group has its own corporate training centre and a clear preference for their own trainers. The enterprise operates in a top-level field and few trainers could teach these matters.
Similarly, a large Italian KIS enterprise owns a training centre, chaired by the HR director, to deliver tailored programmes for different job profiles and to better anticipate skills needs according to the enterprise’s strategic orientations.
Patterns in outsourcing for different types of training
Overall, there is substantial variation in outsourcing patterns across types of training identified in Chapter 2 (Table 4.5). External providers generally deliver programmes targeting soft skills and foreign languages, whereas the induction of new employees is always delivered in-house. Other types of training are delivered either in-house or externally.
These patterns can be explained by the same two factors that were found to drive the choice of the mode of delivery in Chapter 2: fostering the transfer of training and respecting regulatory requirements. This is intuitive, as outsourced training is typically classroom based, while on-the-job training is typically internal. Enterprises rely on external providers for more transversal skills and knowledge areas, such as conflict management or foreign languages, which might be best delivered by external subject experts. For instance, a large Estonian low-tech manufacturing enterprise uses external training providers to train for soft skills, because it believes that there is no sufficient capability in-house. Conversely, they deliver training internally when it involves job- or firm-specific skills and knowledge, such as those covered in the induction of new employees.
However, in the case of technical skills or health and safety in the workplace, enterprises sometimes need to rely on external providers, because the workers need to obtain a certification to be able to complete tasks or processes. For example, a manufacturing enterprise in Ireland reports having to offer a one-day training whenever new equipment is installed to comply with health and safety regulations. The importance of these two factors was best summarised by interviewees in a French high-tech manufacturing enterprise.
The enterprise does not have the in-house competences to train for transversal or soft competences, the safety and security training – which leads to a licence to practice (e.g. electricity, fire-fighting). The training actions that are delivered in-house are developed by experts and HR. They are about enterprise-specific security issues.
Process for selecting training providers
Evidence from the case studies suggest that most enterprises do not have a formalised process for selecting providers. In the majority of cases, the process involves collecting several quotes (typically three or four) from external providers, before making a final decision. Only a handful of very large companies publish public calls for tender or run structured procurement processes, such as is the case in a large financial services multinational in Ireland. An interviewee in a high-tech manufacturing enterprise in Estonia provides insight into a typical selection process.
The principle is to consider 2–3 providers and to rely on the content of the training programme as well as the price of the training when making the decision.
The criteria for the selection of training providers generally include previous experience, the content of training, cost and the value for money. Some enterprises also take into consideration the physical proximity of the training provider in order to minimise the journey time for employees. For instance, an urban KIS enterprise in Austria tends to choose providers that are not too far away, so that the employees do not have to travel far, especially if they need to rely on public transport.
Some enterprises establish long-term partnerships with one or few training providers to minimise transaction and search costs. For example, a medium Estonian LKIS enterprise has a long-term relationship with one external provider, which also offers new suggestions for different training programmes. In some instances, the training is offered directly by the providers of software or machinery, in line with the findings from the CVTS. For example, in a large high-tech manufacturing Austrian enterprise, external training primarily takes place at machine producers to train staff for using new machines purchased.
Size seems to be an important driver of the level to which procurement processes are formalised. Although medium-sized enterprises are more likely to outsource training, they are less likely to have a formalised process to select training providers. This represents an area where public support or support from social partners can be beneficial. For instance, some French medium enterprises in the sample rely on the Competence Operators (Opérateurs de compétences) to identify competent training providers (see Chapter 5). As pointed out by an Italian enterprise offering wholesale services, social partners may be in a good position to help firms identify training providers covering different opportunities and offering good value for money.
Most providers are local and are often linked to local employers’ organisations or associations. This link with umbrella organisations and their service branches (specialised for instance in training) allows a good coverage of different areas and a good price-quality relationship.
Alternatively, as suggested by the CVTS, social partners or publicly funded agencies might develop their own provision directly. Some medium enterprises in the sample rely heavily on training programmes organised by social partners or their training institutes, such as the WIFI in Austria and sectoral associations in Italy and Estonia, or publicl -funded agencies, e.g. Skillsnet in Ireland (see Chapter 5). These enterprises believe that the offering is tailored to their needs, in terms of content and accessibility, as showcased by a medium LKIS enterprise in Austria.
Given the large inequalities in accessing training between different types of employees, the issue of how access to training is given to different individuals warrants specific attention. Chapter 3 has already outlined which groups participate less frequently in training than others. It has also highlighted that individuals for whom benefits of training are higher and costs of training are lower have more access to training than others. Data from the case studies can further enhance the understanding of i) how decisions about selecting individuals into training are made in practice and ii) what influence this has on who gets access to training.
Existing evidence on selecting individuals
Once enterprises have assessed training needs and made a decision on who will deliver the training, they need to select which individuals should attend which training. The psychology literature has analysed this issues through the lens of person analysis (Tannenbaum and Yukl, 1992[17]; Salas et al., 2012[1]). Person analysis is used to identify who does and does not have the required competences for the job, as well as for whom training may be most effective due to individual characteristics, such as motivation or personality (Salas et al., 2012[1]). It can also help companies understanding of how training should be adapted to suit the needs of individual learners. However, research evidence on how enterprises conduct this analysis and the effect that this analysis has on individual and business outcomes is currently lacking.
New evidence on selecting individuals
This sub-chapter presents new evidence on the approaches enterprises take to select individuals for training, the particular role of line-managers as gatekeepers to training participation and nascent evidence on individual learning budgets in enterprises to empower employees.
Top-down, bottom-up and balanced approaches to giving individuals access to training
Evidence for the case study suggests that enterprises follow one of three approaches when giving individuals access to training. They either i) impose training participation top-down, ii) take a balanced approach incorporating the views of both managers and employees, or iii) take a bottom-up approach, allowing employees to access training on their own accord.
More than one in three enterprises in the sample take a top-down approach to training access. This approach is more frequently applied in low-tech manufacturing and low-knowledge intensive services, but not exclusively so. When taking a top-down approach, workers in (line) management positions identify the training needs of individuals, for example by monitoring them at work to assess if their skills meet the requirements of the job. They then approach them about taking part in training. In enterprises with a top-down approach to training, much of the training offer is obligatory, for example, when it comes to the renewal of health and safety certifications. In this context, the role of (line) management is to remind employees that they are due training to renew any certifications they hold. The example of an Austrian manufacturer of rubber and plastic products illustrates how management assess individual training needs without direct involvement of employees.
The need for training is determined by the team leader. Team leaders are responsible for their teams and know the existing and required skills of individuals. When they get the impression that an employee has to be educated in a certain field, they approach the managing director. The managing director approves the suggestion and both look together with the payroll accountant, where relevant courses are offered. Sometimes the team leader identifies a course themselves.
A balanced approach taking into account management and individual views is taken slightly less frequently, that is in approximately one in four enterprises in the sample. Negotiations between both actors most often takes place in the context of progress interviews, developmental conversations or appraisal interviews between line managers and individuals. In around one in two enterprises included in the sample, these formats are the key method used to identify individual training needs. There is little information about how the assessment of individual training needs is made in the context of these conversations, e.g. if line managers use any specific tools or approaches to assess individual skills needs. The example of an Estonian enterprise offering water transport services highlights how such conversations take place. It also shows that in many enterprises some training can be accessed freely, while access to external training in particular needs to be negotiated.
Several training courses are open to all employees – participation in these programmes does not require the approval from the immediate manager or the training department. Hence, it is important how the information on training reaches employees and how motivated employees are to taking up such training opportunities. In case a training is provided by an external provider and is not organised in-house, the decision on training is done in co-operation between the employee (interest in learning), the manager (the need for training) and the training department (the budgetary opportunities for training).
In some enterprises, the results of negotiations between individuals and (line) management are formalised in individual training plans. These plans set out the identified training needs of an individual, as well as the steps that will be taken to satisfy this training need. The example of a large Austrian retailer illustrates how these plans are used in practice.
Every employee has a training plan, which must be checked once a year. It is a training matrix, which consists of certain training that every employee is obliged to attend. New employees, for example, have to take one or two weeks of online training. This is monitored annually. If an employee changed job for example, they may need a different training and an updated plan. Another reason for updating the training plan may be a bad performance evaluation. A new training plan aims to ensure that the performance in the following year will be better. Employees must sign the development plan, which means that there is a feedback discussion between employee and manager in advance, where the employee has the opportunity to express his or her wishes.
In the sample, an equally large sample relies on bottom-up approaches to accessing training. In approximately one in five of the enterprises interviewed, individuals are expected to identify their training needs largely independently. They may select appropriate training from an existing training catalogue or may even be required to approach management with specific suggestions for what training they want to attend. Online training has facilitated a larger reliance on bottom-up approaches to training. In many companies, online learning opportunities are freely available to employees and can be accessed without consent of their (line) mangers. The bottom-up approach is illustrated by an interviewee in a medium-size manufacturer of vehicles in Austria.
Key role of line managers in moderating training access
Line managers frequently act as gatekeepers to training in both the top-down and balanced models described above. In one out of two enterprises for which data is available, they take the decision if an individual takes part in training. This is in particular the case in large enterprises with four or more levels of hierarchy, i.e. where HR and management departments may be far removed from the training needs on the ground.
When taking training decisions, line managers inform other actors, such as HR, department managers or overall management of the training need of individuals and get formal sign-off for their participation. This strong role of line managers can be problematic where there are communication issues between line managers and HR, e.g. where line managers are not up-to-date with the latest training offer, as highlighted by an interviewee in one French manufacturer of transportation equipment.
In companies where line managers do not take the final decision on training participation, this decision is made by department, division of unit managers (15% of enterprises in the sample), the human resource function (10%) or even senior management (10%). In a small number of enterprises interviewed (<5%), individuals have the final say on their training decisions.
Evidence on empowering employees through individual training budgets
The case studies produce some nascent evidence on how some enterprises try to empower employees through individual training budgets. Both tools remove the role of the line manager as gatekeepers for accessing training.
A small number of enterprises in the sample give individuals an individual training budget, which they have the liberty to spend either on their own accord or in discussion with their line manager. Budgets are explicit tools to encourage employees to take responsibility for their own learning. Enterprises that make use of this approach are exclusively providing knowledge-intensive services or involved in high-tech manufacturing. In one case – a large Estonian manufacturer of machinery – individual training budgets are only available for employees in office jobs.
Individual learning budgets generally range between EUR 1 000 and 2 500 per employee and year. In some cases, different budget envelops exist for different roles. One Irish insurance company, for example, gives regular employees a training budget of EUR 1 200 per year and managers a training budget of EUR 2 400 per year. One enterprise in the sample, an Austrian enterprise offering computer-programming services, gives its employees training credits instead of training budget. Each full-time employee has access to 50 training credits, which can either be converted into time off for learning or into money to pay conference, travel or similar fees related to learning. Training budgets are typically to be spend on training related to the employee’s job, as illustrated by the Austrian example below.
Right now, the marketing department takes a course in photography. We have a list of further training courses in our internal system, where everyone can access and see what opportunities there are for further training. A colleague, a software developer, said that she still has a few credits left this year and would like to take this photography course, and I’m sorry to say that this is simply not the purpose of your further training as a software developer, but simply something for marketing.
In one rare case, a large Austrian retailer, grants employees a EUR 1 000 training budget that can be used without restrictions, even for courses that are not directly related to their job. In addition, mandatory and other job-related training is provided and does not fall under this budget allocation.
Not all employees who have access to individual training budgets make use of the opportunity. The aforementioned Austrian computer-programming business estimates that about 80% of employees use their credits in a given year. The interviewed HR representative said that they lacked an understanding of why some of the employees did not use all their training credits.
Enterprises spend significant amounts of time and resources on training. Yet, existing evidence suggests that they rarely assess if training produces the desired results and where they do, they only use basic methods. Some experts argue that HR departments of enterprises spend most of their resources on the development and implementation of training, and then “hope for the best” (Kirkpatrick and Kirkpatrick, 2010[18]).
Training evaluation could help companies understand the effectiveness of their training programme in achieving individual, group or organisational goals through systematic investigation of data (Aguinis and Kraiger, 2008[19]; Goldstein, 1986[20]). This sub-chapter presents new evidence from the enterprise case studies on i) how training evaluation take places in enterprises; ii) what methods they use for the assessment of individual behaviour and organisational performance; and iii) what obstacles they face in the assessment of training outcomes.
Existing evidence on training evaluation
The academic literature on training evaluation primarily focuses on the theory of training evaluation, notably on different training evaluation models and taxonomies. These models set out at which levels the effects of training can be assessed (e.g. individual, group or organisational levels) and suggest evaluation criteria (e.g. learning, behaviour change or organisational results). There are numerous training evaluation models suggested in the literature, the most prominent and influential of which is more than 60 years old (Langmann and Thomas, 2019[21]; Sitzmann and Weinhardt, 2019[22]; Passmore and Velez, 2015[23]). Kirkpatrick’s model of training evaluation suggests looking at four steps to evaluate training (Kirkpatrick, 1979[24]). The steps become successively harder to measure and require increasingly sophisticated data collection techniques:
Reaction: How participants reacted or responded to the training, for example if they found it engaging or relevant to their jobs. This may be assessed, for example, through a satisfaction survey at the end of the training.
Learning: What participants have learnt from the training, i.e. which knowledge, skills or attitudes they have gained. Ways of assessing learning includes quizzes that test the knowledge gained or practical tests.
Behaviour: If participants put their learning into practice when they are back on the job. This is typically evaluated through a self-assessment or the observations through supervisors at work.
Result: What the impact of the training is on wider organisational goals and objectives.
Despite its popularity, the Kirkpatrick models has attracted much criticism, for example for implying causal links between the different steps, while this is not supported by empirical evidence (Bates, 2004[25]; Alliger and Janak, 1989[26]). There is an abundance of alternative training evaluation models. Some of these expand Kirkpatrick’s four steps to include an analysis of societal impact of training (Kaufman and Keller, 1994[27]) or the evaluation context (Warr, Bird and Rackham, 1970[28]). Other models deviate from Kirkpatrick by taking quantitative approaches, for example by suggesting estimating the return of investment of training (Chmielewski and Phillips, 2002[29]), or by promoting qualitative approaches to understanding of the effect of training by looking at success cases (Brinkerhoff, 2005[30]). Multi-level and omnidirectional models suggest that the levels at which training evaluation takes place and the evaluation criteria used are dependent on purpose and use of the evaluation (Langmann and Thomas, 2019[21]; Sitzmann and Weinhardt, 2019[22]).
It is worth noting that the quantitative approaches promoted in the psychology literature on training evaluation overlap with the approaches observed to analyse the benefits or returns of training observed in the economics literature (see also Chapter 3). The difference between both bodies of research is that the psychology literature focuses more on HR practices on training evaluation, while the economics literature focuses on analysing the impact of training often outside organisational practice and using non-organisational data sources.
What all models for training evaluation have in common is that any empirical evidence on their application is scarce. In practice, it is thought that enterprises undertake training evaluation only hesitantly and unwillingly, while their methods for assessment are mostly basic, such as self-administered ‘training evaluation forms’ filled in by participants (Langmann and Thomas, 2019[21]; Aragon and Valle, 2013[31]).
Data from firm-level surveys provide some high-level insights on the prevalence of training evaluation in enterprises and the methods used. According to CVTS data, a surprisingly high share of companies assesses the outcomes of training. On average across the EU-27, 59% of enterprises with ten or more employees assess the outcomes of at least some of their training activities (Figure 4.8). However, there are large differences between countries and differently sized enterprises, as well as some more limited differences between sectors. While 99% of large enterprises in France report assessing the outcomes of training, only 25% of small enterprises in Poland do.
The CVTS also provides some limited information on the types of methods used for training evaluation (Figure 4.9). On average across the EU-27, basic satisfaction surveys of participants are most frequently used (34%), followed by an assessment of participants behaviour or performance in relation to the training objectives (33%) and certification after a written or practical test (31%). Only 24% of enterprises evaluate the impact of training on the performance of the relevant departments or the enterprise as a whole. This data highlight a small deviation of the Kirkpatrick model, in that more enterprises assess behaviour than learning.
Differently sized enterprises use the various training evaluation methods to a different extent. Large enterprises are around twice as likely to implement satisfaction surveys and certify training as small enterprises, while the evaluation of the effect of training on department or enterprise performance is used to a roughly similar extent across size groups (not displayed in graph). This may be because satisfaction surveys and certification necessitate the set-up of additional systems, which smaller enterprises have limited capacity for. Assessing individual behaviour, department and enterprise performance, by contrast, may make use of existing systems of performance measurement, such as key performance indicators.
New evidence on training evaluation
The qualitative evidence collected in the context of this study facilitates a deeper understanding of i) how training evaluation take places in enterprises; ii) what precise methods they use for the assessment of individual behaviour and organisational performance; and iii) what obstacles they face in the assessment of training outcomes.
Approaches and methods used for training evaluation
The majority of enterprises in the sample assesses the benefits of training, either by collecting individual’s reaction to training (e.g. through participant surveys), assessing their learning (e.g. through testing), analysing their behaviour (e.g. through observation at work) or by evaluating the effect of training on company results (e.g. through monitoring performance data). This aligns with the findings of the firm level surveys. There is some indication that there are sectoral differences, with enterprises providing knowledge-intensive services evaluating training less frequently than other enterprises in the sample. This may be explained by the fact that these enterprises provide training on skills which are difficult to assess, i.e. training on soft skills (see Chapter 2). As observed in the CVTS data, small and medium enterprises in the sample are less likely to evaluate training than companies with more than 250 employees.
New data from the case studies gives some more life to the information on assessment methods collected in the CVTS. It shows that enterprises use a wide range of methods to evaluate training and that the majority of enterprises uses more than one method. Table 4.6 presents the different types of evaluation methods reported by respondents and how frequently they were reported. It also relates the methods used back to the steps of the Kirkpatrick model.
The most common method that can be observed is verbal feedback, used in approximately half of the enterprises in the sample. Its popularity is explained by its simplicity. In most cases, feedback on participant’s experiences of the training and/or if the training should be offered to other individuals in the enterprises is collected ad-hoc by line, department or HR managers. Verbal feedback on the training is also collected in the context of the appraisal process. Participant surveys or other forms of standardised training evaluation forms are slightly less frequent. In rare cases participant surveys link to online learning platforms and feedback is directly provided there. One Italian enterprise in the publishing business highlighted that they were in the process of developing an integrated learning platform that would allow individuals to directly rate teacher performance, training contents and impact on the job.
In line with what is observed in the CVTS, monitoring training participants at work following training participation is a very common method to evaluate training. In the sample, it is most common in low-tech manufacturing enterprises, where supervisors observe training participants and assess if they apply their (new) skills when operating machinery. For example, a supervisor in an Austrian manufacturer of rubber and plastic products mentioned that they evaluated the effect of training by provoking a fault on a machine and observing if the employee was able to correct the fault.
The use of certification to assess the outcomes of training is still relatively common in the sample, while other forms of direct assessments of skills are rare. In the sample, it is typically formal, compulsory or external training that is being certified. An Estonian enterprise specialised in the repair of machinery for example, obtaining a certificate is the precondition for specialists to be able to perform certain tasks (see also Chapter 2). Certification should only be considered a method for the evaluation of training when obtaining it involves an assessment of the skills or knowledge acquired during the training. Documents that simply confirm participation in training are not a method to assess outcomes of training. In practice, however, enterprises seem to use the word certification of training loosely and it is not always clear that an assessment takes place.
Other types of assessment that do not lead to certificates are typically knowledge tests at the end of training. Interestingly, one Italian enterprise in the sample internally re-assesses individuals that have obtained a certification, if the enterprise is not sure that the competences have not been fully acquired. Three enterprises in the sample mention the use of competence databases to evaluate training. They provide limited information on how these databases are used to evaluate the training in practice. They seem to record the skills and competences of employees in a software tool, which facilitates the monitoring of any changes following training participation. No information on how the skills and competences of employees are assessed is provided.
Finally, data from the case studies suggest that enterprises only rarely assess the effect of training on the wider performance of departments or the organisation. Few enterprise use performance data to evaluate training and where they do, they make use of existing business monitoring systems. One Austrian enterprise specialised in providing business support activities, for example, monitors if key performance indicators (e.g. customer satisfaction and sales) differ before and after the training provided. A small number of enterprises in the service sector rely on customer feedback to assess the effects of training.
In summary, the case studies confirm the existing findings from the literature and quantitative surveys that, while many enterprises engage in the evaluation of training, they do so using basic methods and only very rarely assess the effect that training has on the wider performance of the enterprise. There are rare cases of planning training, anticipating its benefits and measuring them, however: One Irish manufacturing company stated to have a formal system of weekly planning, target setting, reporting and review that was linked to performance and the demonstration of skills acquired in training. One Austrian enterprise emphasised that the purpose and desired effect of each training course was defined a-priori. The effect of training was then assessed by comparing key performance indicators before and after training had occurred.
Obstacles to training evaluation
The case studies provide only limited information on what prevents enterprises from evaluating training. Several enterprises interviewed noted that they thought that the skills conveyed in some training are too difficult to measure. One enterprise voiced that they found it easier to assess the learning outcomes for production workers, because they could test if an individual was able to fix broken machinery, while the skills of office workers were much more difficult to assess in general.
Measuring whether benefits of training materialise is a big challenge that has not been mastered yet. It is hardly possible to assess to what extent a conflict management seminar has paid.
A small number of enterprises mentioned lack of time as one of the key obstacles for evaluating training. Even where external providers collect some information on training outcomes, some enterprises do not have the capacity to analyse and make use of the information.
There is no assessment done by the company. For external training, there is a satisfaction survey, but the company has no time to exploit the results.
Finally, some firms see a trade-off between monitoring and motivating employees. Not assessing the outcomes of training is a conscious choice for them and aims to give employees the responsibility for their own learning process.
References
[19] Aguinis, H. and K. Kraiger (2008), “Benefits of Training and Development for Individuals and Teams, Organizations, and Society”, Annual Review of Psychology, Vol. 60, pp. 451-74, https://doi.org/10.1146/annurev.psych.60.110707.163505.
[26] Alliger, G. and E. Janak (1989), “Kirkpatrick’s levels of training criteria: Thirty years later.”, Personnel Psychology, Vol. 42/2, pp. 331-342, https://doi.org/10.1111/j.1744-6570.1989.tb00661.x.
[31] Aragon, I. and R. Valle (2013), “Does training managers pay off?”, The International Journal of Human Resource Management, Vol. 24/8, pp. 1671-1684, https://doi.org/10.1080/09585192.2012.725064.
[12] Arthur, W. et al. (2003), “Effectiveness of Training in Organizations: A Meta-Analysis of Design and Evaluation Features”, Journal of Applied Psychology, Vol. 88/2, pp. 234-45, https://doi.org/10.1037/0021-9010.88.2.234.
[25] Bates, R. (2004), “A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence”, Evaluation and Program Planning, Vol. 27/3, pp. 341-347, https://doi.org/10.1016/j.evalprogplan.2004.04.011.
[30] Brinkerhoff, R. (2005), “The Success Case Method: A Strategic Evaluation Approach to Increasing the Value and Effect of Training”, Advances in Developing Human Resources, Vol. 7/1, pp. 86-101, https://doi.org/10.1177/1523422304272172.
[5] Bryson, J. and B. Zimmermann (2020), “Workers’ participation in continuing training decision‐making”, International Journal of Training and Development, Vol. 24/3, pp. 167-172, https://doi.org/10.1111/ijtd.12191.
[2] Cardon, M. and S. Valentin (2017), “Training and development in small and medium enterprises”, in The Cambridge Handbook of Workplace Training and Employee Development, Cambridge University Press, https://doi.org/10.1017/9781316091067.018.
[16] Chaudhuria, S. and K. Bartlett (2014), “The relationship between training outsourcing and employee commitment to organisation”, Human Resource Development International, Vol. 17/2, pp. 145-163, https://doi.org/10.1080/13678868.2014.886444.
[29] Chmielewski, T. and J. Phillips (2002), “Measuring Return-on-lnvestment in Government: Issues and Procedures”, Public Personnel Management, Vol. 31/2, https://journals.sagepub.com/doi/pdf/10.1177/009102600203100208.
[10] Cotes, J. and S. Ugarte (2021), “A systemic and strategic approach for training needs analysis for the International Bank”, Journal of Business Research, Vol. 127, pp. 464-473, https://doi.org/10.1016/j.jbusres.2019.05.002.
[3] Eurofound and Cedefop (2020), European Company Survey 2019: Workplace practices unlocking employee potential, Publications Office of the European Union, Luxembourg, https://doi.org/10.2806/030551.
[15] Galanaki, E., D. Bourantas and N. Papalexandris (2008), “A decision model for outsourcing training functions: Distinguishing between generic and firm-job-specific training content”, International Journal of Human Resource Management, Vol. 19/12, pp. 2332-2351, https://doi.org/10.1080/09585190802479579.
[20] Goldstein, I. (1986), Training in Organizations: Needs assessment, Development and Evaluation, Brooks/Cole.
[27] Kaufman, R. and J. Keller (1994), “Levels of evaluation: Beyond Kirkpatrick”, Human Resource Development Quarterly, Vol. 5/4, pp. 371-380, https://doi.org/10.1002/hrdq.3920050408.
[24] Kirkpatrick, D. (1979), “Techniques for evaluating training programs”, Training and Development Journal 33, pp. 78-92.
[18] Kirkpatrick, J. and W. Kirkpatrick (2010), Training on Trial: How Workplace Learning Must Reinvent Itself to Remain Relevant, AMACOM, https://www.amazon.com/Training-Trial-Workplace-Learning-Reinvent/dp/0814438148 (accessed on 20 April 2021).
[8] Kraiger, K. et al. (eds.) (2015), Training Needs Analysis at Work, John Wiley & Sons.
[23] Kurt Kraiger, J. (ed.) (2015), Training Evaluation, John Wiley & Sons, https://onlinelibrary.wiley.com/doi/10.1002/9781118736982.ch8.
[21] Langmann, S. and S. Thomas (2019), “Rethinking training evaluation: omnidirectional evaluation, range of audience and intentional change”, The International Journal of Human Resource Management, Vol. 30, https://doi.org/10.1080/09585192.2017.1303782.
[9] Leigh, D. et al. (2000), “Alternate models of needs assessment: Selecting the right one for your organisation”, Human Resource Development Quarterly, Vol. 11/1, pp. 87-93, https://doi.org/10.1002/1532-1096(200021)11:1<87: AID-HRDQ7>3.0.CO;2-A.
[13] Macmahon, J. and E. Murphy (1999), “Managerial effectiveness in small enterprises: Implications for HRD”, Journal of European Industrial Training, Vol. 23/1, pp. 25-35, https://doi.org/10.1108/03090599910253519.
[11] McGehee, W. and P. Thayer (1961), Training in Business and Industry, Wiley.
[6] Mowbray, P., A. Wilkinson and H. Tse (2015), “An Integrative Review of Employee Voice: Identifying a Common Conceptualization and Research Agenda”, International Journal of Management Reviews, Vol. 17/3, pp. 382-400, https://doi.org/10.1111/ijmr.12045.
[14] OECD (2019), Getting Skills Right: Future-Ready Adult Learning Systems, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264311756-en.
[4] OECD (2019), Negotiating our way up: Collective bargaining in a changing world of work, OECD Publishing, Paris, https://doi.org/10.1787/1fd2da34-en.
[1] Salas, E. et al. (2012), “The Science of Training and Development in Organizations: What Matters in Practice”, Psychological Science in the Public Interest, Vol. 13/2, pp. 74-101, https://journals.sagepub.com/doi/pdf/10.1177/1529100612436661 (accessed on 20 April 2021).
[22] Sitzmann, T. and J. Weinhardt (2019), “Approaching evaluation from a multilevel perspective: A comprehensive analysis of the indicators of training effectiveness”, Human Resource Management Review, Vol. 29/2, pp. 253-269, https://doi.org/10.1016/j.hrmr.2017.04.001.
[7] Sparkmann, R. (2018), Strategic Workforce Planning: Developing Optimised Talent Strategies for Future Growth, Kogan Page.
[17] Tannenbaum, S. and G. Yukl (1992), “Training and development in work organisations”, Annual Review of Psychology, Vol. 43/1, pp. 399-441, https://doi.org/10.1146/annurev.ps.43.020192.002151.
[28] Warr, P., M. Bird and N. Rackham (1970), Evaluation of management training: a practical framework, with cases, for evaluating training needs and results, Gover press, London.
Note
← 1. It should be noted that it does not provide information of the extent to which enterprises apply full TNA.