6. Evaluation of labour market training
This chapter measures the effect of labour market training (LMT) on the employment, earnings and occupational mobility of participants. The change induced along the distribution of occupational quality is also explored. To account for the selection of participants into the programme, the counterfactual impact evaluation deploys a propensity score matching methodology. The estimated effects are examined across sub-groups of unemployed (age, gender, education level and urban or rural location) and across different durations of the programme. In the long term, employment of LMT participants rises significantly compared to non-participants, whilst earnings are similar. LMT does not lead, on average, to moves up the occupational ladder. Older individuals and women enjoy the largest gains from participation, and longer labour market trainings have larger effects in the long term for employment and earnings.
This chapter examines Finland’s labour market training (LMT) programmes symmetrically to the analysis of self-motivated training (SMT) provided in Chapter 5 of this report. The sample studied consists of jobseekers with an ongoing unemployment spell at the end of years 2012-14. The results are examined from one to four years after this point. The impact evaluation measures the effects for unemployed people that participate in the programme, on a rich set of labour market outcomes, relative to similar jobseekers who did not participate in LMT but could have benefited from other PES services and programmes.1 In addition to measuring the impact of the programme on employment probability and earned income, the analysis examines the effects of labour market training on occupational mobility. The estimated effects are measured across sub-groups of jobseekers based on their age, gender, education level and urban or rural location. In the main results, labour market training programmes are restricted to those that last at least three months; this chapter explores the extent to which effects vary when different durations of labour market training programmes are considered. It then links the results found to previous literature on LMT. The chapter ends with a conclusion section.
Jobseekers that participate in LMT differ from the rest of the unemployed (Table 6.1, columns 1 and 2). LMT participants are more likely to be men (+4 percentage points), foreign nationals (+4 percentage points) and single (+6 percentage points). They are on average 4.6 years younger than other jobseekers and have more children. Individuals that enrol in LMT are more likely to live in urban municipalities (+6 percentage points) and in blocks of flats (+8 percentage points). LMT participants are less likely to have a level of education below upper-secondary (-3 percentage points) but are also less likely to speak the Finnish language (-6 percentage points). Regarding the fields of study, degrees in health and welfare are underrepresented (-4 percentage points) while information and communication technology degrees are overrepresented (+4 percentage points) among LMT beneficiaries. However, there is not a sizeable difference in terms of professions in the last job held. Finally, the number of days spent in unemployment over the past (two) year(s) is slightly higher for LMT participants. Participants and non-participants in LMT are thus not directly comparable. A control group of jobseekers similar to participants along observable characteristics, built through propensity score matching, is used to account for these differences (Table 6.1, column 3 for the control group and column for the treated group).
Individuals that enrol in LMT are also considerably different from SMT beneficiaries (columns 1 and 4 of Table 6.1). SMT attracts more women. In fact, the share of women participating on it is around 20 percentage points higher than in LMT. SMT participants are on average two years younger, have more children and are more likely to be the spouse of the family (+8 percentage points). On the other hand, LMT participants are more likely to be foreign nationals (+4 percentage points) and less likely to speak the Finnish language (-5 percentage points). Compared to SMT beneficiaries, LMT participants are less likely to hold a general programme degree (-5 percentage points), a degree in arts and humanities (-4 percentage points), or in health and welfare (-5 percentage points) but are more likely to hold a degree in engineering (+14 percentage points). Regarding professions, they are less likely to belong to the service and sales industry (-8 percentage points) and more likely to have experience in elementary occupations (+9 percentage points). On average SMT participants have spent less time in unemployment over the past (two) year(s) than LMT participants (around 20 days less over the past two years).
Since participants in SMT and in LMT are intrinsically different, the result of Chapters 5 and 6 are not directly comparable. In fact, it is impossible to disentangle at what point the differences encountered come from differences in the effectiveness of the programmes or from the fact that different people select into each programme.
The estimation results show that LMT has a significant lock-in effect on employment at the end of the year the programme starts (recall that this corresponds more precisely to 31 December of the year the programme takes place2) (Figure 6.1, Panel A). At this point, treated individuals are around 24 percentage points less likely to be employed than similar jobseekers in the control group. However, this lock-in effect is considerably diminished the following year (less than -4 percentage points) and two years after the start of the programme (three years from the starting point of observation) the impact of LMT on the employment probability becomes positive and significant. In terms of monthly earnings, LMT has a lock-in effect that lasts longer than its lock-in effect on employment (Figure 6.1, Panel B). Treated individuals have lower earnings than their control counterparts until two years after the start of the programme. Four years after the starting point of observation treated individuals catch-up, they have on average the same monthly earnings as jobseekers in the control group.
Note: The analysis presents nearest-neighbour propensity score matching results which matches individuals based on several characteristics: duration in unemployment until the point of observation, history in unemployment over the past two years (spells and days), age, gender, marital status, education level and field, Finnish national, type of municipality, type of building, quarter of registration into unemployment (time trend), as well as previous year employment status, earnings and occupational quality (rank). The confidence intervals are shown at the 5% level of significance and represented by the whiskers delimiting the dotted lines on the charts. Outcomes are observed at the end of each calendar year (31 December). Year zero is the first year in the sample and identifies the pool of people that are unemployed at the end of that year. All training takes place in year one (therefore between points zero and one in the graph). Other years are relative to this.
Source: OECD calculations based on Statistics Finland’s repository: FOLK and TEM datasets.
It is not possible to disentangle to what extent the shorter lock-in effect of LMT as compared to SMT comes from differences in the efficiency of the programmes or from differences in their participants’ characteristics, however, at least part of it comes mechanically from the fact that LMT have shorter duration than SMT. Participants in LMT are kept out from job search for a shorter amount of time and can accept jobs faster than jobseekers participating in SMT.
The median of the estimates on the probability of being employed, found by other studies in the international literature at a similar time horizon, is of 5 percentage points (ranging from -2 to 25 percentage points) (Card, Kluve and Weber, 2018[1]). The effect found for LMT is consistent with these findings. Three years after the start of the programme participants in LMT are around 4 percentage points more likely to be employed (with a 95% confidence interval ranging from around 3 to 5 percentage points). Furthermore, running the same estimation on a smaller sample of unemployed for whom employment and earnings can be observed over one more year, leads to larger effects for both outcomes after four years than after three years from the start of the programme (Annex Figure 6.A.1).
In addition to employment outcomes and earnings, this analysis investigates whether LMT participants are more likely to change occupations and, if by doing so, they move up or down the jobs ladder. As explained in Chapter 4, the quality of occupations is measured by an occupational index that builds on Finnish data on earnings at the occupation level from 2010 to 2018. It measures quality both as a percentile rank and in monetary units (occupational earnings in euros).
The impact of LMT on the probability of changing occupation is positive and significant as of one year after the start of the programme (Figure 6.2, Panel A). After this point the positive impact increases steadily in magnitude and, four years after the starting observation point, LMT participants (that is treated individuals) are 12 percentage points more likely to have a different occupation than the one they had before the start of the programme as compared to non-participants (jobseekers in the control group). Nevertheless, even if LMT participants are more likely to move along the occupational ladder, their upward occupational mobility is not affected. The change in the occupational index before and after labour market training is not significantly different for LMT participants than for their counterparts (Figure 6.2, Panels B and C).
This null average effect on upward occupational mobility does not give information on how LMT affected the shape of the distribution of occupations. This null effect could come from an absence of change along the distribution, but different tails of the distribution could also be disproportionately affected. To explore the distributional effect of LMT, Annex Figure 6.A.2 plots the distribution of the occupational index (in percentiles) of LMT participants and non-participants respectively, at the starting point of observation (before LMT participation) and four years after this point (three years after LMT starts). The change in the distribution of the occupational index of LMT participants does not follow the same pattern than for non-participants. While only minor changes are observed in the distribution of the occupational index between the starting point of observation and four years after this point for the non-participants, the distribution of LMT participants has considerably changed. Notably two bumps appear in the low-middle (around percentiles 20 to 32) and the middle of the distribution (around percentiles 40 to 53) while occupations below the 20th percentile and between the 60th and the 80th are less represented. Therefore, the null average effect on upward occupational mobility hides a shift towards specific occupations in the middle of the distribution for individuals that participated in LMT as compared to similar individuals that did not participate in it.
Note: The analysis presents nearest-neighbour propensity score matching results which matches individuals based on several characteristics: duration in unemployment until the point of observation, history in unemployment over the past two years (spells and days), age, gender, marital status, education level and field, Finnish national, type of municipality, type of building, quarter of registration into unemployment (time trend), as well as previous year employment status, earnings and occupational quality (rank). The confidence intervals are shown at the 5% level of significance and represented by the whiskers delimiting the dotted lines on the charts.
Source: OECD calculations based on Statistics Finland’s repository: FOLK and TEM datasets.
Analogous to the analysis of SMT in Chapter 5, Section 5.4, this section provides separate estimates for the results along several jobseeker’s characteristics: (i) gender, (ii) age, (iii) level of education and (vi) urban vs. rural municipality of residence. Furthermore, this section also provides separate estimates based on the duration of LMT. The results presented in the previous sections have documented the effects of LMT programmes that lasted at least three months (Sections 6.3 and 6.4). The aim of this restriction is to avoid the results to be driven by short-term LMTs present in the data that are unlikely to have real up-skilling effects. This section explores how the results change if this restriction is relaxed in order to include all LMTs observed in the data, but also how the results change with a stronger restriction that keeps only trainings that last more than six months.
6.5.1. Women and older individuals benefit more from labour market training
All subgroups of jobseekers, except for individuals under 49 years of age, exhibit positive and statistically significant effects of LMT on the probability of being employed four years after the starting observation point (Figure 6.3, Panel A). Individuals aged 50 and over are the subgroup with the largest effects on employment probability (around 19 percentage points). This gap on the impact of LMT across age groups holds also for monthly earnings: individuals under 30 present the smallest effect of all groups (EUR -93) while individuals 50 and above are those most positively impacted (EUR +420) (Figure 6.3, Panel B). Regarding upward occupational mobility, despite their positive employment outcomes, older jobseekers seem to be moving down the jobs ladder (-2.7 percentage points) (Figure 6.3, Panel C). Thus, similar to SMT, age seems to be a key driver of heterogeneity in the effectiveness of LMT.
The results on gender also parallel the ones for SMT and add up to the suggestive evidence that training programmes may be more effective for women than men (Card, Kluve and Weber, 2018[1]). Even if men are more likely to participate in LMT than women (Section 6.2), the impact of the programme is higher for women than for men in terms of employment probability (6.2 against 2.5 percentage points) and monthly earnings (EUR +146 against EUR -55). The opposite is true for upward occupational mobility; women seem to be moving down the jobs ladder (-1 percentage point) while there is no statically significant effect for men.
The effects of LMT on the probability of employment, earnings and upward occupational mobility do not differ substantially by the education level of jobseekers and the type of municipality of residence.
Note: The analysis presents nearest-neighbour propensity score matching results which matches individuals based on several characteristics: duration in unemployment until the point of observation, history in unemployment over the past two years (spells and days), age, gender, marital status, education level and field, Finnish national, type of municipality, type of building, quarter of registration into unemployment (time trend), as well as previous year employment status, earnings and occupational quality (rank).
Source: OECD calculations based on Statistics Finland’s repository: FOLK and TEM datasets.
6.5.2. Longer labour market trainings exhibit better results on employment and earnings in the long term
LMT is open to all individuals registered as unemployed. Jobseekers can enrol in the LMT course they select after the TE Office reviews if it suits well their profile (Chapter 4, Section 4.3). LMT aim at upskilling jobseekers and improve their employability; however, LMT is not uniform. A large variety of courses are proposed, with different contents, targeting different professions and having different durations. Investigating which of these features make LMT more effective can inform policy making on what works and give insight on how to redesign trainings to improve their effectiveness. Nevertheless, the data gathered for this study does not contain information on the contents, the type or the target occupation of trainings, it contains only reliable information on the duration of LMT. While the main results presented in this chapter focus on trainings that lasted more than three months, the subsequent analysis explores how the results change by modifying this restriction.
When short LMTs are not excluded, and thus LMTs of all durations are included in the analysis, the estimation results exhibit a lock-in effect smaller in magnitude (Figure 6.4). At the end of the year the programme starts, LMT participants are 16.5 percentage points less likely to be employed and earn EUR 484 less than controls while in the main specification this numbers rise to almost -24 percentage points and EUR -645. The opposite is true when we restrict the analysis to LMTs that last at least six months. When only long duration trainings are considered, the participants are around 30 percentage points less likely to be employed at the end of the year that follows the start of LMT and earn EUR 790 than similar jobseekers who do not participate in this type of training.
One year after the start of LMT, there is no statistically significant difference on the impacts on employment and earnings across the three different samples. However, two years after the start of the training, the more the sample is restricted to longer duration trainings the more positive the results are in terms of employment and earnings. At the last observation point of the study, LMT that last at least six months make their beneficiaries almost 7 percentage points more likely to be employed and earn around EUR 107 more per month. These numbers decrease to 4 percentage points and EUR 24 when trainings that last between three and six months are also included. Finally, when no restriction is made on duration the effects for employment are null and of EUR -102 for earnings.
The effects of LMT on upward occupational mobility do not differ substantially with the duration of the programme.
Altogether, these results suggest that longer LMTs generate stronger lock-in effects but also lead to better outcomes afterwards. Longer trainings may allow more upskilling than shorter ones steering greater improvements on employability.
Note: The analysis presents nearest-neighbour propensity score matching results which matches individuals based on several characteristics: duration in unemployment until the point of observation, history in unemployment over the past two years (spells and days), age, gender, marital status, education level and field, Finnish national, type of municipality, type of building, quarter of registration into unemployment (time trend), as well as previous year employment status, earnings and occupational quality (rank). The confidence intervals are shown at the 5% level of significance and represented by the whiskers delimiting the dotted lines on the charts.
Source: OECD calculations based on Statistics Finland’s repository: FOLK and TEM datasets.
Several previous studies have reviewed LMT and have found beneficial effects that are coherent with the results found in this evaluation. These studies usually use matching as the statistical technique to identify programme impacts. This allows evaluation to be completed post hoc, whilst the programme is in live running and relies on comparing programme participants to non-participants that are similar to them across a range of observable characteristics. The most recent study of its effects found that it raised employment rates by 5 to 10 percentage points and increased annual earnings by around EUR 2 500 (Alasalmi et al., 2022[2]). Positive effects on employment emerged one to two years after participation, as lock-in effects (whereby participation in training means people delay looking for jobs) attenuated. These outputs persist over the longer term and are still present ten years following participation. The study also presents evidence that outcomes are fairly stable over time, suggesting that the benefits to participation have remained consistent over different parts of the economic cycle. This gives some reassurance when evaluating long-term effects, for which one needs to assess training that started some 15 years previously. As impacts are consistent at shorter durations there is more confidence that long-term effects might also persist for training courses that are undertaken more recently.
Alasalmi et al. (2022[2]) also assess outcomes by individual status prior to participation. Jobseekers who were unemployed prior to their job search experience positive employment rate impacts of 8-10 percentage points. Effects are similar for those that are employed prior to their participation. Those individuals made redundant prior to training experience a 5 percentage points increase to employment rates. For those outside of the labour force, employment effects are around 10 percentage points. Matching is used to construct two different comparison groups to participants. One group is of all eligible non-participants, and another is comprised of those that applied but were not successful.3 This is done to compare whether the choice of group of non-participants influences the overall programme estimate. The choice of control group does influence effect sizes. The comparison to all eligible non-participants yields a larger effect size than when comparing to non-participants who had applied for training (the former having roughly a 3 percentage points higher impact). Employment impact effect sizes are also analysed by the type of LMT. Re-training courses have the largest impact (8-10 percentage points), followed by initial vocational training (7-9 percentage points), further training (5-8 percentage points) and then entrepreneurship and other training (4-7 percentage points). These differences are not as pronounced when income is the outcome variable being evaluated.
Aho et al. (2018[3]) examine a cohort of 2010 LMT participants and find an employment rate impact of around 10 percentage points with highest effects for jobseekers over the age of 50. Alasalmi et al. (2019[4]) similarly find positive effects for LMT, commencing around 1-2 years after participation, with no differences across age and gender groups in the longer term. However, they do find that training seems to benefit older (55-64 years old) workers in the shorter term too, a feature that does not occur for younger age groups. They also find larger income effects for these older workers, though note that small sample sizes mean there is a high degree of uncertainty around the estimates. These results are further corroborated when looking at LMT back as far as the early 1990s, with positive impacts detected for jobseekers participating in training, which were similar even when observed at different points in the economic cycle (Hämäläinen, 1999[5]). Using data from a similar period (1989-98), Tuomala (2002[6]) finds LMT raises employment by around 6.5 percentage points on average and the effect is higher for those with higher levels of previous employment (3.9 percentage points for those with basic education compared to 7.9 percentage points for those with secondary and higher education).
Taken together there is consistent evidence across a range of studies including this one, that LMT offers positive impacts across both the probability of employment and, to a lesser but still positive extent, the income from that employment. Variation has been found within different sub-groups suggesting that analysis which considers different types of individuals is beneficial to explaining a rich story of LMT impacts. It also gives pause for thought to policy makers in exploring how to improve the LMT offering. Particularly on whether to encourage participation among sub-groups or how to alter training content to improve outcomes for some groups. The results of this study encompass previous findings and broaden them to analyse the impact of LMT on occupational mobility. LMT increases mobility across occupations but does not lead to highly paid occupations.
LMT exhibits positive long-term effects on employment that are in line with national and international evidence. Expanding the window of observation of this study would plausibly lead to observe higher magnitude effects in terms of employment and earnings over the following years. LMT does not affect upward mobility on average. However, it does affect the shape of the distribution of occupational quality decreasing the frequency of bottom and top occupations in favour of occupations in the middle of the distribution. This raises the question of what the objective of policy makers should be: designing programmes that improve the quality of jobs on average or programmes that reduce inequalities in job quality leading to a more concentrated distribution of occupational quality.
LMT effects vary across subgroups of the populations and benefits more women and older individuals. LMT participants are on average younger and more likely to be male than the rest of the unemployed. Therefore, efforts could be made to encourage these two groups of the population to increase their participation in LMT, as they are more likely to benefit from it. Furthermore, longer LMTs generate larger gains in terms of employment and earnings, pointing to the importance of training that provides perspectives, a structured rhythm and allow the acquisition of more and better skills. To better understand the mechanisms behind the effects found, collecting data on different features of LMT such as the contents, the skills or the occupations targeted by trainings could help complete this analysis. Data on the occupations targeted could be, for instance, linked with the occupational barometer data (see Box 2.1 in Chapter 2) to shed light on whether LMT leads to better results and helps reducing occupational mismatch when it is targeted to occupations identified as being in high demand. Moreover, a cost-benefit analysis could complete this evaluation by studying whether the positive effects of these programmes are sufficient to offset their cost.
References
[3] Aho, S. et al. (2018), Työvoimapalvelujen kohdistuminen ja niihin osallistuvien työllistyminen, Valtioneuvoston selvitys-ja tutkimustoiminnan julkaisusarja 19/2018.
[4] Alasalmi, J. et al. (2019), Työttömyyden laajat kustannukset yhteiskunnalle, Valtioneuvoston selvitysja tutkimustoiminnan julkaisusarja 16/2019.
[2] Alasalmi, J. et al. (2022), Ammatillisen työvoimakoulutuksen toimivuuden ja vaikutusten arviointi (Evaluation of the effectiveness and impact of vocational education training), Prime Minister’s Office Helsinki 2022.
[1] Card, D., J. Kluve and A. Weber (2018), “What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations”, Journal of the European Economic Association, Vol. 16/3, pp. 894-931, https://doi.org/10.1093/JEEA/JVX028.
[5] Hämäläinen, K. (1999), Aktiivinen työvoimapolitiikka ja työllistyminen avoimille työmarkkinoille., ETLA Sarja B 151.
[6] Tuomala, J. (2002), Työvoimakoulutuksen vaikutus työttömien työllistymiseen., VATT-tutkimuksia 85, Valtion taloudellinen tutkimuslaitos.
Note: The sample is restricted to individuals having an ongoing unemployment spell at the end of years 2012-13. The analysis presents nearest-neighbour propensity score matching results which matches individuals based on several characteristics: duration in unemployment until the point of observation, history in unemployment over the past two years (spells and days), age, gender, marital status, education level and field, Finnish national, type of municipality, type of building, quarter of registration into unemployment (time trend), as well as previous year employment status, earnings and occupational quality (rank). The confidence intervals are shown at the 5% level of significance and represented by the whiskers delimiting the dotted lines on the charts. Outcomes are observed at the end of each calendar year (31 December). Year zero is the first year in the sample and identifies the pool of people that are unemployed at the end of that year. All training takes place in year one (therefore between points zero and one in the graph). Other years are relative to this.
Source: OECD calculations based on Statistics Finland’s repository: FOLK and TEM datasets.
Note: The heights of the lines indicate the relative share of individuals in occupations whose index in percentiles are on the horizontal axis. This figure compares treatment and control distributions in years 0 (before) and 4 (after).
The two sample Kolmogorov-Smirnov test for equality of the distribution of the occupational between treated and control groups leads to a p-value of 0.070 before LMT and of 0.000 after LMT.
Source: OECD calculations based on Statistics Finland’s repository: FOLK and TEM datasets.
Notes
← 1. Note that the control group is identified from the universe of jobseekers in the final sample. The sample was not restricted to jobseekers that did not benefit from any other service as these individuals are likely to have very specific characteristics and not be representative of the population of interest, leading to potential selection issues.
← 2. Outcomes are observed at the end of each calendar year (31 December). Year zero is the first year in the sample and identifies the pool of people that are unemployed at the end of that year. All training takes place in year one. Other years are relative to this. For example, year four identifies outcomes four years after first observing an individual as unemployed, but three years after having participated in training.
← 3. Note that data on applications were not available for this study.