3. Programmes evaluated, data used and approach adopted for counterfactual impact evaluation in Greece
This chapter explores the main aspects of two active labour market policies (ALMPs) evaluated in this report: training and wages subsidies offered to unemployed individuals by DYPA, the public employment service of Greece. It provides an overview of the key features of these programmes and the programme participants. It also describes the rich, individual level administrative data that provide the foundation for the empirical analysis and the econometric approach used in the counterfactual impact evaluation of these two measures in the following chapters of this report. In addition to outcomes commonly examined in impact evaluations of ALMPs, such as employment probabilities, this chapter describes additional outcomes examined, most notably career progression. For the latter, the chapter outlines the construction of an occupational index calculated based on the observed wages of individuals by detailed occupational codes.
The preceding chapter noted that Greece’s spending on active labour market policies (ALMPs) is relatively modest: its ALMP spending is less than half of the OECD average and its labour force participation in ALMPs is less than one-third of the OECD average. At the same time, Greece’s registered unemployment rate has remained relatively high in recent years (although the survey-based unemployment rate – which measures active job seekers – has fallen in recent years). Taken together, these factors mean that spending per jobseeker is low and that only a small fraction of jobseekers in Greece is engaged in ALMPs. These factors point to the need to carefully consider the design and targeting of Greece’s ALMPs to maximise their impact.
To what extent are Greece’s ALMPs successfully placing jobseekers into high-quality, sustained employment – and which ones work best for whom? Which aspects of ALMPs are working well and which could be improved? To answer such questions, policy makers often rely on (key) performance indicators – job placement rates, participant satisfaction – or on feedback from staff in public employment services (PES) or jobseekers. Both sources of information can play an important role in assessing the merits of a policy. For example, performance indicators can be useful to understand which ALMPs have the highest probability of employment after participation, to examine the extent to which these have improved over time, or to monitor the performance of particular training providers in real time. Similarly, feedback from PES staff and clients can help to provide a nuanced view of the benefits and drawbacks of a particular programme, as well as concrete suggestions for improvement. At the same time, however, such approaches cannot provide a rigorous answer to the crucial question of the precise impact of a policy – this requires accounting for what would have happened to individuals in the absence of the policy. This is the motivation for conducting counterfactual impact evaluations (CIEs), such as the one outlined in this chapter.
The impact evaluation presented in this chapter focuses on two of Greece’s main ALMPs: training and wage subsidies for the unemployed. The two policies provide, respectively, training lasting several months intended to fill gaps in jobseekers’ skills, and subsidies of generally up to one year to offset part of the employers’ wage costs associated with hiring workers from specific groups. Outcomes are tracked continuously for up to three years from the start of the programme. The empirical analysis is based on rich and comprehensive data, allowing for a wide range of outcomes to be analysed and for a number of different jobseeker characteristics to be taken into account. Several types of data are used in this evaluation: unemployment register data, ALMP participation data, and employment and earnings data.
The chapter begins with a description of the two programmes analysed and the characteristics of the individuals and employers participating in the programmes. It then describes the rich, individual-level administrative data that form the basis of the empirical analysis, as well as the econometric approach used in the CIE of these two measures in subsequent chapters of this review. The final sections describe the labour market outcomes examined in the impact evaluation. In addition to the outcomes commonly examined in impact evaluations of ALMPs, such as employment probabilities, this chapter describes additional outcomes of interest, in particular career progression. For the latter, the chapter describes the construction of an occupational index calculated on the basis of the observed wages of individuals by detailed occupational codes.
Training for the unemployed and wage subsidies together account for 40% of Greece’s ALMP expenditures on ALMPs during the 2017-21 period, excluding the wage subsidy measures introduced during the COVID-19 crisis. Training and wage subsidies were implemented as many distinct programmes, each with their specific features in terms of their content, target groups, duration, and objectives. Nevertheless, they share enough key similarities to allow them to sensibly be grouped as two sets of programmes for the purposes of the evaluation (although some of the different implementations of specific programmes are analysed separately as well).
From the perspective of the impact evaluation, it is worth emphasising that broad sets of the unemployed are eligible for participating in training and wage subsidies. Some programmes do have strict participant eligibility: for example, the information and communication technology (ICT) training programmes require individuals to have tertiary education, and most of the wage subsidies are targeted to specific age groups. However, taken as a whole, the programmes are not limited to specific groups – for the purposes of identifying effects, the eligibility criteria are rather broad. For example, three of the largest wage subsidy programmes in the period analysed are targeted, respectively, to individuals aged 18-29, 30-49 and at least 50 years. From the perspective of choosing an approach for identifying the programme effects, another important consideration is that the Greek public employment service (DYPA) counsellors have considerable discretion in deciding whether to refer an individual to a specific measure in the case of the wage subsidies. This fact informs the choice of the econometric procedure used, with comparisons of similar individuals made based on detailed information on their observed characteristics (for details on the methodology, see Section 3.5.
3.2.1. Three training programmes are analysed in the impact evaluation
The system of training offered to jobseekers in Greece includes a multitude of different channels and has undergone considerable changes in recent years. In addition to having a network of vocational training schools, the Greek PES refers its clients to vocational training courses offered by external training providers. With the onset of the COVID-19 pandemic, the Greek PES partnered with international online training platforms to offer remote training options, a co-operation that continues in a more limited scope (DYPA, 2023[1]). Furthermore, as training constitutes an important component of Greece’s Recovery and Resilience Fund, Greece has recently considerably expanded the scale of training offered, with 150 000 jobseekers to be trained in green and digital skills (DYPA, 2023[2]). These have been implemented with a stronger accountability framework in place for providers, including a performance-based payment scheme.
In contrast to more recent programmes, the training programmes examined in this evaluation were not administered by the Greek PES. Prior to the reforms of the PES enacted in April 2022, the Greek Ministry of Labour and Social Affairs was responsible for the management of the main training programmes for the unemployed. The programmes were organised by providing vouchers with open enrolment, and PES counsellors would provide information on these programmes for interested jobseekers. Three training programmes were selected for the current impact evaluation, one relating to training in high-demand sectors and two programmes providing tertiary education graduates with ICT training. The programmes were selected taking into account several considerations. First, these programmes were identified as being of interest to DYPA: the two ICT training programmes are particularly relevant given broader changes in the labour market, including the increased importance of digital skills, and the other programme is intended to address skills shortages in high-demand sectors, such as tourism. Second, the selected programmes have sufficient sample sizes in terms of participation and time horizons for observing outcomes (the selected programmes have at least one thousand participants during the period January 2017 to July 2021).1 All three training programmes contain both theoretical and practical components. They last approximately five months in total for the high-demand sectors training and seven months for the ICT training (Figure 3.1).
Note: Includes information on individuals entering training from June 2017 to June 2020.
Source: OECD calculations based on data provided by the Greek public employment service (DYPA).
The training programmes analysed have the following key features:
Training for high-demand sectors. This programme provides training in occupations that are in high demand, such as retail salespeople, waiters, and warehouse workers. The first, theoretical component generally lasts one month and contains 120 hours of theoretical training. The second component consists of an internship of 500 hours, which generally last five months. The programme contains an exam, which virtually all programme participants (97%) complete successfully.
ICT training programmes. Two programmes are examined, one aimed at individuals aged 25-29 and another aimed at those aged 30-45. Compared to the training for high-demand sectors, they contain a greater emphasis on theoretical training, with 400 hours of training spread over roughly five months. The practical component consists of an internship of 200 hours which generally lasts seven weeks. In each programme, three training tracks are offered, web and application design, database development, and software application. They are targeted exclusively at individuals with tertiary degrees. The programmes also contain an exam with very high proportions of successful completions (93%).
All three training programmes contain payments to the training providers (training vouchers), as well as training allowances paid to the participants. The total payments made for each individual’s participation amount up to EUR 3 955 and EUR 5 990 for the training for high-demand sectors and ICT training respectively. The precise breakdown of the costs is as follows:
Training for high-demand sectors. The training voucher amounts to EUR 1 155 (EUR 1 355 if the jobseeker is employed by the employer for at least six months), with specific amounts for training (EUR 2 360), counselling (EUR 30) and certification (EUR 150). In addition to the training provided via the voucher, participants receive an allowance of EUR 2 600 for the period of theoretical and on-the-job training. This includes an education subsidy of EUR 600 (EUR 5 per hour for 120 hours) and a traineeship subsidy of EUR 2000 (EUR 4 per hour for 500 hours).
ICT training. The training voucher amounts to EUR 2 990 (EUR 3 190 if the jobseeker is employed by the employer for at least six months), with specific amounts for training (EUR 2 360), counselling (EUR 60), certification (EUR 150) and an internship (EUR 420). In addition to the training provided via the voucher, participants receive an allowance of EUR 2 800 for the period of theoretical and on-the-job training. This includes an education subsidy of EUR 2 000 (EUR 5 per hour for 400 hours) and a traineeship subsidy of EUR 800 (EUR 4 per hour for 200 hours).
During the period analysed, the training for high-demand sectors has considerably higher number of participants than the programmes for ICT training: 19 599 compared to 1 108 and 1 396 for the ICT training programmes targeted at 25-29 and 30-45 year-olds, respectively. For the programmes analysed, participants entered the training for high-demand sectors in the period from June 2017 through December 2018 and the ICT training from January 2020 through June 2020. While these specific training programmes were only in place during this period, other similar types of programmes were implemented during other times.
3.2.2. Several different wage subsidy programmes are analysed
A total of 17 different wage subsidy programmes containing entrants from March 2017 through July 2021 are examined in the impact evaluation. The programmes vary slightly in terms of the specific parameters, such as target groups and payment levels, but they share several key features relating to their generosity (see Table 3.1 for an overview of the largest programmes).2 First, they reimburse employers for a fixed percentage of participant’s wages (mostly ranging from 50-75%), with higher rates generally associated with vulnerable jobseekers based on factors, such as age and unemployment duration but also varying based on macroeconomic conditions. Second, they generally last around 12 months, with the possibility of extensions in some programmes for specific categories of jobseekers (although these are not commonly applied in practice).
Wage subsidy programmes also impose a number of additional conditions designed to minimise strategic behaviour by employers and to minimise deadweight costs, i.e. the hiring of workers through the subsidy programme who would have been hired in the absence of the subsidy. They often restrict eligibility to employers who have not reduced employment (or workers’ hours) in the previous three months, with specific exceptions allowed in cases such as retirements (although separate documentation must be provided to support this). They generally exclude the hiring of workers who have worked for the employer in the previous 12 months or who have other personal ties to the employer (more recent programmes have extended this period to 24 months and also exclude workers with recent experience in the same industry). In addition, four of the 17 examined programmes impose retention requirements on employers, requiring workers to remain in (unsubsidised) employment with the employer for three months after the end of the subsidy. Compliance with the subsidy programmes is strictly monitored, with at least two on-site visits to each wage subsidy participant required to verify compliance.
3.2.3. Important changes were made to the wage subsidy programmes beginning in July 2020
Important innovations were introduced into the wage subsidy programmes beginning in July 2020. Before July 2020, employers could search for suitable candidates themselves through the employer portal and suggest their own candidates to hire using the wage subsidy. PES staff would check the eligibility of candidates and could also suggest candidates. However, as long as a registered jobseeker met the eligibility criteria, the employer was free to choose whom to hire, increasing the likelihood that employers would hire people they would otherwise have hired without the subsidy.3 Programmes implemented after July 2020 modified this aspect of the selection process. After posting a vacancy and indicating that they wanted to hire someone through a specific wage subsidy programme, an employer would receive a short list of potential candidates compiled by a PES counsellor, taking into account the skills and experience required for a particular vacancy (with candidates possibly ranked to assist the employer in the selection process). The employer could then hire someone from this list of candidates through the subsidy.
In addition to changes to the procedure of selecting job candidates for the subsidised positions, changes have been implemented since July 2020 that are intended to expedite and simplify the administrative procedure. The time needed to hire someone via the subsidy has been considerably shortened in practice, as employers can apply to the programme at the same time as they post a vacancy with DYPA. DYPA then conducts an expedited, basic eligibility verification of the employer, who can then hire a jobseeker from the list of candidates suggested by DYPA counsellors. The employer must then submit the necessary paperwork to prove its eligibility for the programme within two months of hiring the jobseeker. Payments, which are made on a quarterly basis, are only made once DYPA has verified that the full list of eligibility criteria has been met. Changes have also been made to reduce the administrative burden on jobseekers and DYPA counsellors. For example, DYPA counsellors used to require all shortlisted candidates to provide documentation to prove their eligibility (e.g. that they had not worked for that employer in the last two years). DYPA now only requires this information from those eventually selected for the subsidy.
The programmes analysed in the impact evaluation in this report had a total of roughly 60 000 participants during the period analysed (Figure 3.2). Of these, roughly 60% of participants entered the older programmes, before the changes to the programmes were implemented in July 2020. The changes made to the new round of programmes implemented from July 2020 were successful in their goal of increasing take-up rates by employers. The increases in participation were quite large: monthly participant inflows increased from less than 1 000 in preceding three-year period to a peak of roughly 5 000 in October 2020.
Source: OECD calculations based on data provided by the Greek public employment service (DYPA). Wage subsidy programmes are those selected for evaluation in this report.
3.2.4. Training for jobseekers is primarily self-initiated, but DYPA counsellors play an active role in wage subsidy referrals
Much of the initiative in deciding whether to participate in an ALMP rests with the individual jobseeker, as the high caseload of DYPA counsellors precludes intensive, proactive interaction with their clients. While recent digitalisation efforts have streamlined and automated certain procedures, consultations with DYPA counsellors indicate that much of their time is still dedicated to administrative tasks (such as processing claims), limiting the amount of time they can dedicate to directly working with clients. This means that in practice, with the exception of mandated meetings to compile individual action plans at the beginning of unemployment spells, many meetings with jobseekers are conducted based on jobseekers’ requests.
Eligibility and need for different ALMPs, including training and wage subsidies, is established when the jobseeker and their DYPA counsellor first discuss and assess the jobseeker’s employment opportunities and needs for support. Since November 2018, counsellors are supported by a digital jobseeker profiling tool that uses rules-based profiling based on a questionnaire to classify jobseekers into five client groups. Those who are classified into the most job-ready category receive an automated individual action plan and are not offered counselling sessions with counsellors; by contrast, those classified as least employable (category five) are not referred to ALMPs in principle.
From the introduction of a voucher system in 2014 until a recent reform in 2022, jobseekers enrolled into training based mostly on their own initiative. PES counsellors referred interested jobseekers to a website with information on available training and eligibility criteria, and the jobseekers could then apply to enrol in the training. The implementation of the training programmes was led by the Ministry of Labour and Social Affairs, which contracted with a number of private training providers to conduct the training.
While PES counsellors have played a more advisory role in helping jobseekers enter training during the period analysed in the impact evaluation, they have played a central role in placing individuals into wage subsidies throughout the 2017-21 period – but with the important changes from the summer of 2020 onwards. As discussed above, since 2020, PES counsellors have played a more proactive role by compiling the candidate shortlists from which employers can select wage subsidy candidates.
This section examines the characteristics of individuals participating in the training and wage subsidy programmes, as well as the attributes of firms who hire the participants (either directly during the wage subsidy period or after the end of their training). The analysis across individuals focusses on differences across gender, age, duration of unemployment, education level and location. Before discussing the take-up rates of different demographic groups, the section begins by presenting some stylised facts on the employability of these different groups. This is used to help establish the degree to which the ALMPs studied are targeted toward groups that are closer or farther from the labour market.
3.3.1. Average job-finding rates confirm that groups such as older jobseekers have a greater distance to the labour market
The patterns of exit from unemployment show that certain types of jobseekers, such as older jobseekers or those with lower levels of education, are less likely to exit unemployment. Figure 3.3 examines the subsequent employment outcomes of individuals who were registered as unemployed during the first part of the period (i.e. March 2017 to July 2020) examined in the impact evaluation (based on monthly unemployment data). For groups such as men under 30 or those registered as unemployed for less than one month, a majority will become employed within two years.4 The probability of exiting unemployment decreases with unemployment duration: 54.8% of the individuals who are newly unemployed will be employed within two years, compared to 15.2% of those registered as unemployed for at least two years. This pattern is broadly consistent with stylised statistics from other countries, although the discrepancy in the exit rates into employment between short- and long-term unemployed in Greece is considerably larger than in most other EU countries (Eurostat, 2023[3]).
Note: Statistics are calculated based monthly statistics of individuals unemployed from March 2017-July 2020 for employment outcomes experienced through July 2022.
Source: OECD estimates based on data from the Greek public employment service (DYPA) and Ergani.
3.3.2. Participation in the ALMPs examined is generally higher amongst certain demographic groups
To provide a sense of the extent to which specific categories of individuals are likely to enter ALMPs, this section contrasts the characteristics of the participants in the selected training and wage subsidy programmes with the characteristics of all individuals who are registered as unemployed with the PES. These comparisons are presented in Figure 3.4, taking the averages of monthly unemployment stocks during the period for which the ALMP entrants are examined (2017 to 2021).
Note: ALMPs stand for the training and wage subsidy programmes evaluated in this report. Shares are calculated within each of the five broad categories in the figure: if a demographic category of ALMP participants were represented in proportion to their share amongst all registered unemployed, the length of the bars would coincide with the red squares. Statistics for stocks of all unemployed are calculated based on averages of monthly statistics during the 2017-21 period. Participant numbers refer to totals during the 2017-21 period for individuals entering either training or wage subsidies.
Source: OECD calculations based on data from the Greek public employment service (DYPA), ERGANI and Diofantos.
Relative to their share of the unemployed, women are slightly more likely to enter training than men, while men are slightly more likely to enter wage subsidies. Women account for 67% of training participants, even though they accounted for 63% of registered unemployed during the 2017-21 period. Encouragingly, gender stereotypes do not appear to be an important factor in the choice of training programmes – for example, in the programme targeted towards individuals aged 30-45, women accounted for a disproportionately large share (74%) of participants in the training for ICT, a field where women are traditionally underrepresented (OECD, 2018[4]).
The age profile of the selected training and wage subsidy participants indicate that older workers are disproportionally less likely to enrol in the programmes studied and that prime-age women are particularly likely to enrol in training. Jobseekers over 50 represent a considerably smaller share of participants in both programmes, with the share of older women entering wage subsidies is particularly small relative to their stock of the unemployed: only 7.6% of wage subsidy participants were women over 50, even though such women comprised 18.1% of jobseekers. On the other hand, prime-age women (those aged 30-50) were disproportionally more likely to enter a training programme, with such women comprising 47.3% of participants and 34.9% of the stock of unemployed. Younger groups of workers are systematically more likely to enrol in wage subsidies, as their share of participants is much higher than their share in the stock of unemployed. These statistics by age and gender suggest that the same profiles of individuals who are statistically more likely to become employed are the ones receiving additional support (for this direct comparison, see Annex Figure 3.A.1, Panel A).
The pattern of higher participation among more employable groups is also present in relation to education, although this reflects the design of the ALMPs studied. Jobseekers with a lower level of education are disproportionately less likely to participate in either of the two ALMPs analysed. This partly reflects the fact that certain programmes were targeted at people with higher educational attainment. For example, two of the ICT training programmes were targeted at jobseekers with tertiary education. Furthermore, several of the wage subsidy programmes in the period analysed are targeted at jobseekers with tertiary education, while only one is targeted at those with lower educational attainment. Nevertheless, it is worth noting that many of the wage subsidy programmes target disadvantaged or particularly vulnerable groups – groups that are likely to have higher proportions of jobseekers with lower educational attainment.
In terms of the location of jobseekers entering the ALMPs examined, individuals from outside the two largest urban areas are slightly more likely to participate. Some of the wage subsidy programmes specifically target individuals living in some of these regions, which could partly explain the higher uptake rates for the wage subsidy programmes. The largest of the training programmes, targeted towards high-demand sectors, was also used to help train individuals in the tourism sector, which experienced large growth also outside the major cities.
Wage subsidy programmes also have higher uptake among the short-term unemployed, while entering training is more common after very long unemployment spells. Specifically, up to unemployment spells of around 30 months, jobseekers are disproportionally more likely to enter wage subsidy programmes and disproportionally less likely to enter training (Figure 3.5). One feature of the ALMPs worth bearing in mind is that for roughly half of the wage subsidy programmes analysed, eligibility is not directly tied to unemployment duration. But the precise programme parameters typically are related to unemployment duration – for example, in many of the programmes implemented prior to July 2020, the obligation to retain workers for several months after the end of the subsidy period is imposed only for those who have been unemployed for less than one year. This may explain the small spike in wage subsidy entrants after 12 months of unemployment.
The pattern of participation in ALMPs by duration of unemployment may also be partly explained by the patterns of interactions between DYPA counsellors. These are systematically conducted early on in the unemployment spells for most jobseekers when they meet to develop individual action plans (the exception being the most readily employable, who do not necessarily meet with counsellors for the individual action plans). This approach leads to more jobseekers entering wage subsidies earlier on in their unemployment spells. By contrast, entry into training – which is initiated by the jobseeker – is more common later on in the unemployment spell. This may be partly attributable to the payments to training participants, which represent a more compelling financial incentive to individuals once they have exhausted their unemployment benefits.
In terms of their unemployment duration, the groups of wage subsidy participants entering into the programmes are also the ones who, on average, are more likely to become employed (Annex Figure 3.A.1, Panel B). The exception to this pattern relates to jobseekers registered for less than three months, who form a relatively small share of all participants.
Note: ALMPs stand for the training and wage subsidy programmes evaluated in this report. Statistics for stocks of all unemployed are calculated based on averages of monthly statistics during the 2017-21 period. Participant numbers refer to totals during the 2017-21 period for individuals entering either training or wage subsidies.
Source: OECD calculations based on data from the Greek public employment service (DYPA) and Diofantos.
3.3.3. Larger firms tend to hire vocational training participants while smaller ones use wage subsidies
While the discussion so far has focused on the characteristics of jobseekers engaging in ALMPs, another interesting aspect concerns the characteristics of the firms who participate in the wage subsidy programmes or hire individuals after they complete their training programmes. Figure 3.6 shows the distribution of firms hiring participants – either during the programmes themselves (in the case of wage subsidies) or after they have completed the training programmes – across size categories and compares them with the distribution of firms hiring unemployed individuals in general. Small firms make disproportionately large use of wage subsidies, both relative to the number of total wage subsidy participants and relative to the number of jobseekers (registered with the PES) that they hire. Firms with ten or less workers accounted for three-quarters of all the wage subsidy participants, even though they accounted for only half of jobseeker hires. By contracts, firms with at least 50 workers accounted for 30% of total jobseeker hires but only 14% of wage subsidy participants. The low take-up rates of larger firms may be partially tied to the relatively onerous conditions tied to the receipt of the wage subsidies (although experiences from other OECD countries indicates that large firms tend to use wage subsidies less). For example, in most programmes examined, employers are required to demonstrate that they have not reduced their employment in the past three months. Such conditions are clearly more binding for larger firms. In addition, EU limits on the absolute amount of government financial supports that can be paid to firms – and that have become more binding with the COVID-19 support measures – are also more binding for larger firms.
In contrast to wage subsidy programmes, larger firms are considerably more likely to hire individuals after they have completed training programmes. This suggests that larger firms are disproportionately affected by shortages of suitably qualified workers. Expanding the scale of training, combined with greater involvement of employers in the design and delivery of training initiatives, could effectively address these challenges, while increasing the employment opportunities of jobseekers in higher-paid and more productive firms. As in other OECD countries, larger firms in Greece have higher output per worker, reflecting their increasing returns to scale through capital-intensive production (OECD, 2021[5]). However, the disparity between small and large firms in Greece is among the largest in the OECD: for example, output per worker in micro firms in business services averaged only 22% of the output of large firms in 2018 (the comparable OECD average is 71%).
Note: ALMPs stand for the training and wage subsidy programmes evaluated in this report. Shares are calculated within each of the five broad categories in the figure: if a demographic category of ALMP participants were represented in proportion to their share amongst all registered unemployed, the length of the bars would coincide with the red squares. Size categories are based on the total number of workers at a given employer registered in ERGANI on 1 January of the calendar year in which an individual became employed at that employer. Firms with zero employment can occur in at least two cases: i) small firms with seasonal employment (e.g. in the tourism sector) who hire during the year but do not have employees on 1 January, and ii) firms which began operations after 1 January of a given calendar year. newly created firms which did not yet have. Statistics for wage subsidy participants refer to the employers receiving the subsidies; statistics for the training participants refer to the first (unsubsidised) employer after completion of the theoretical and practical training. Statistics for stocks of all unemployed are calculated based on averages of monthly statistics during the 2017-21 period. Participant numbers refer to totals during the 2017-21 period for individuals entering either training or wage subsidies.
Source: OECD calculations based on data from the Greek public employment service (DYPA), Diofantos and ERGANI.
In terms of the sectors of economic activity of firms making use of the ALMPs examined, several sectors stand out in terms of their use compared to their hiring of other jobseekers registered with the PES. A disproportionately large share of hires in wholesale and retail trade involves hiring unemployed individuals via wage subsidies – on average accounting for 30.8% of all wage subsidy participants during the 2017-21 period (and twice the share of all jobseekers hired during the same period). The wholesale and retail trade sector also hires a disproportionately high share of training participants. Another sector with particularly large shares of ALMP participants compared to their hiring of other jobseekers is the professional services sector. In absolute terms, manufacturing firms employ large shares of ALMP participants, but these are commensurate with their higher of jobseekers in general. Sectors making extensive use of training may be disproportionately facing labour shortages, likely due to a combination of shortages of workers with adequate skills and possibly more challenging working conditions.
In contrast to other sectors, the tourism sector – hotels and restaurants – stands out in terms of its low shares of ALMP participants, accounting for the hiring of roughly 10% of wage and training participants but 34% of other jobseekers. This may be partly explained by the fact that the seasonal nature of employment in this sector means that a large share of workers cycle into and out of unemployment, leading to high hiring rates to which ALMP participants are compared – in this case, jobseeker hiring rates are not the best basis for gauging the demand for workers. Survey evidence indicates that undeclared work has in the past been most common in the restaurant industry in Greece (ILO, 2016[6]), so the low-take-up could also be attributable to businesses trying to avoid the additional scrutiny from DYPA inspectors that constitute a mandatory monitoring component of wage subsidies. Finally, the current staffing shortages experienced in the tourism sector were arguably less of an issue during the period examined, from 2017-21.
Comprehensively assessing the impact of ALMPs on subsequent labour market outcomes requires rich data with detailed information on jobseekers’ characteristics, their participation in ALMPs and their employment outcomes. The data used to conduct the evaluation in this report were obtained from several sources, as outlined in Table 3.2, and span various time periods from January 2013 to August 2022. Unique individual identifiers – pseudonymised to protect individuals’ privacy – allow the data to be combined from several sources. This enables a rich understanding of individuals’ participation in ALMPs, their background characteristics, including past labour market history, as well as their labour market outcomes and wages.
The resulting database contains detailed information on the 2 578 038 unique individuals who were registered as unemployed at any point during the 2017-21 period. These individuals experienced 5.8 million distinct unemployment spells in total. The final analytical database contains detailed information on the 22 115 entries into training and 61 141 entries in wage subsidies for the programmes analysed and for people who registered as unemployed from March 2017 through July 2021. Individuals are observed to enter into training and/or wage subsidies in 2.8% of unemployment spells during this period. The data span various periods from January 2013 to August 2022. More information on the data used and how they were processed are available in the technical report accompanying this publication (OECD, 2024[7]).
One potential problem often encountered in impact evaluations of ALMPs concerns the question of how to deal with multiple, sequential entries into ALMPs. In the presence of multiple interventions and possible overlap between different ALMPs, identifying the precise effects of one specific ALMP presents an important challenge. In the case of Greece, this is not a major concern: for those who were observed to enter an ALMP, the vast majority (98.2%) entered into only one ALMP during their entire unemployment spell. For the purposes of the evaluation, in the small number of cases of multiple ALMPs in a given unemployment spell, we focus on the first ALMP entered during the spell.
Despite the richness of the data on which this evaluation is based, there are some limitations in terms of data quality. In particular, the Ergani data on earnings and employment contain a significant number of implausible and inconsistent values, especially in earlier periods. For example, individual employment spells were constructed based on information that needs to be submitted separately to mark the beginning and end of employment spells, but this information was apparently not subject to rigorous cross-checks in earlier periods. This required a number of assumptions in the construction of the final analytical database, which are described in detail in the accompanying technical report (OECD, 2024[7]). From the point of view of the analysis, the presence of measurement error means that the estimates may be less precise than they otherwise would have been (the confidence intervals will be wider) and that the point estimates may be biased towards zero – i.e. that any positive or negative effects may be underestimated (Levi, 1973[8]). This also means that the analysis does not take into account information that has been found to be particularly inconsistent in individual-level data, such as hours worked or type of contract. This needs to be borne in mind when interpreting the results, particularly those relating to days worked. For example, if participation in an ALMP increases the likelihood of an individual being employed on a part-time rather than full-time basis, this would bias the estimated results: actual hours worked would be lower than the observed days worked, while hourly earnings would be higher than the observed daily earnings. In practice, this may not be a problem given the relatively low prevalence of part-time work in Greece: at 8.8%, the part-time employment rate in Greece in 2022 was well below the OECD average of 16.1% (OECD, 2023[9]). Nevertheless, for future evaluations, establishing an analytical database which incorporates additional data sources, such as income tax data, could be used to improve the accuracy of the information on employment and earnings.
Another limitation of the employment data is the lack of data on public sector employment. While this limitation should be borne in mind when interpreting the results, two factors suggest that its impact on the net effects is not large. First, none of the ALMPs examined are targeted at promoting self-employment, and wage subsidies are specifically designed to promote dependent employment in the private sector. This suggests that any resulting bias in the estimated effects, if any, would be in the direction of underestimating the effectiveness of the ALMPs studied. Second, stakeholder consultations suggest that public sector hiring was negligible during the period under review due to austerity measures. By contrast, the private sector employment has witnessed considerable growth during the period studied. This suggests that the analysis can capture most of the relevant effects.
Additional questions related to data are discussed in an accompanying technical report (OECD, 2024[7]). This report discusses the data in more detail, identifying how the analysis could be enriched with additional data and discussing ways to make better use of data in the future.
Assessing the impact of an ALMP requires comparing the labour market outcomes of ALMP participants, such as employment or earnings, with the outcomes that would have occurred if they had not participated in the ALMP. As the latter ‘counterfactual’ outcomes cannot be observed, it is necessary to find a way to construct them from the data. A simple way of doing this would be to compare the outcomes of those who participated in training (or other ALMP) with those who did not. However, as discussed in more detail below, in the absence of random assignment in the programme, such groups are unlikely to be comparable, and making such simple comparisons may introduce selection bias that would not provide accurate estimates of the true effect of the programme.
As in other evaluations adopting a similar estimation technique, several sources of selection bias may be relevant in this impact evaluation. Certain types of individuals (e.g. more motivated individuals) may be more likely to participate in training and have better employment outcomes for reasons other than their participation in training. Conversely, certain individuals who face additional barriers to employment – and therefore have worse employment outcomes – may be more likely to be referred to ALMPs by caseworkers. Many of those who do not participate in an ALMP may not be included simply because they quickly find a job (and leave unemployment) without the support of DYPA. This latter group of individuals may actually have better future employment outcomes than ALMP participants: if they leave unemployment quickly, they would have a good chance of keeping that job and are much more likely to be employed in a few years or months than if they had remained unemployed. In addition, DYPA counsellors are less likely to see such individuals as needing the support of an ALMP such as training, as this could mechanically extend their unemployment spell by the duration of a course.
To address such sources of bias, the approach used in this report controls for differences in demographic characteristics (e.g. gender, education, age, etc.), observed skills and barriers to employment between ALMP participants and non-participants. Such an approach is then used to estimate the ‘treatment effect’ by comparing individuals who appear similar in terms of their observable characteristics. The outcomes of participants (the “treatment” or “intervention” group) are compared with the outcomes of a similar group of non-participants (the “control” or “comparison” group).
The econometric approach has several features designed to ensure the comparability of the treatment and control groups and to provide unbiased results:
Only individuals with similar (formal) employment histories are compared with each other. This compares the labour market outcomes of those who enter an ALMP in a given month with those who have not (yet) entered an ALMP and who had similar patterns of formal employment in the preceding three years. The application of this methodology – similar in some respects to the “dynamic selection-on-observables” initially adopted by Sianesi (2004[10]) – is explained in the accompanying technical report (OECD, 2024[7]).
Individuals are also only compared with each other if they have identical values of several additional characteristics. In addition to comparing individuals with similar employment histories, comparison individuals are constructed to have exactly the same characteristics along a number of additional dimensions: calendar month and year of entry into the programme, gender, earnings and, for those who have not been employed in the last three years, age group and broad level of education.
A rich set of additional personal characteristics is used to identify individuals with similar probabilities of entering the ALMP under consideration. Within the precise groups mentioned above, individuals are further matched to similar individuals on the basis of an estimate of their probability of entering the ALMP under consideration. Such an approach – based on a so-called propensity score – is commonly used in the literature to address the difficulty of otherwise accounting for a wide range of additional personal characteristics (Card, Kluve and Weber, 2018[11]). The propensity score is a measure of the probability of participating in the programme under analysis. The following factors are taken into account when calculating the propensity score: (i) each individual’s employment history (duration of employment, earnings, occupation), (ii) unemployment duration, (iii) employability assessment (based on a rule-based profiling score that can be modified by DYPA counsellors), (iv) demographic characteristics (education, gender, nationality), (vi) foreign language skills, and (vi) whether an individual lives in a large urban region. Details on these characteristics are presented in the accompanying technical report (OECD, 2024[7]).
The choice of research design is dictated by the relatively broad eligibility criteria of ALMPs in Greece and the availability of rich administrative data. In the case of Greece, it is not possible to use a research design that would exploit strict eligibility criteria, such as an age threshold. In such a research design, groups of individuals who are ineligible for a programme – for example, in the case of a programme targeted at young people, because they have just crossed an age threshold – could serve as a natural basis for establishing what would have happened to participants if they had not participated (the so-called counterfactual outcomes). Instead, in the case of this evaluation, in the absence of strict eligibility criteria based on a threshold, the research design makes use of the rich administrative data available to match individuals along a number of dimensions, including their previous employment history and the exact calendar month in which an individual enters an ALMP. The approach is explained in detail in the accompanying technical report (OECD, 2024[7]).
Propensity score matching approach is often used in impact evaluations. Of the 95 impact evaluations of ALMPs examined in a recent meta-analysis (European Commission and Ismeri Europa, 2023[12]), 85% used propensity score matching. Canada, for example, regularly and systematically evaluates its ALMPs based on this approach (OECD, 2022[13]). It has also been used in several recent OECD evaluations of ALMPs in Finland, Lithuania and Latvia (OECD, 2023[14]; OECD, 2022[15]; OECD, 2019[16]).
Counterfactual impact evaluations of ALMPs typically examine outcomes related to labour force participation, such as the change in employment probability for ALMP participants compared to similar non-participants. The effect of ALMPs on the probability of employment has been most widely studied, with a meta-analysis by Card, Kluve and Weber (2018[11]) including employment probability estimates from 111 impact evaluations of ALMPs and newer one focusing on ESF-funded programmes identifying 94 impact evaluations (European Commission, 2022[17]). While this outcome is certainly important, as an ultimate goal of ALMPs is to help individuals find employment, the focus on this outcome may also be partly driven by data availability: data on other outcomes are often more difficult to obtain. But focusing on other outcomes – particularly ones relating to job quality – can help provide a more nuanced view of the potential benefits and trade-offs involved in ALMP participation.
In the case of Greece, the rich and comprehensive data available allow the analysis to track a wide range of outcomes in the evaluation of the programmes studied and over a relatively long period. Outcomes are tracked continuously for up to three years from the start of the programme. Outcome values are calculated on a monthly basis and tracked over time relative to a reference month, which is defined as either the month in which an individual enters an ALMP (for the treatment group) or the same calendar month for a control group individual matched to someone in the treatment group. Details on the calculation of these outcomes are provided in the technical report (OECD, 2024[7]).
The following outcomes are examined:
Probability of entering employment. This probability is measured using a binary outcome variable which is equal to 1 if individual is employed at a certain time, and equal to 0 otherwise.
Probability of remained registered as unemployed. This is measured using a binary outcome variable which is equal to 1 if individual is unemployed at a certain time, and equal to 0 otherwise.
Cumulative employment duration. This measures the cumulative duration of all jobs held during the observation time, in calendar days, after the reference month.
Occupational mobility. The analysis maps the occupation of individuals entering employment onto an occupational index, which can be interpreted as a “job ladder”. The construction of the index is detailed in Section 3.7.
Cumulative earnings. This measures total earnings, gross of taxes and contributions, in constant prices, in all jobs held during the observation time.
One important limitation in the use of administrative data sources is that it cannot account for undeclared employment. The Greek authorities have enacted a number of measures in recent years in an attempt to limit informal employment. In 2013, an electronic registry of employment was implemented for reporting of hiring and dismissals (ILO, 2016[6]) and since 2014, employers have been required to declare, in advance of workers beginning their first shifts, their employees into the ERGANI system (European Commission, 2017[18]). Labour inspectors from the authorities tasked with monitoring compliance – the Hellenic Labour Inspectorate (SEPE) as well as inspectors from DYPA and the Financial Police – have access to digital tools and the ERGANI system to verify compliance, and fines for non-compliance are very high. This has led to dramatic declines in the incidence of undeclared and under-declared work: according to the European Labour Authority (2020[19]), by more than 10 percentage points between 2014 and 2018 (from 19.2% to 8.9%). Nevertheless, informal employment arguably still constitutes an important share of employment in Greece. Such types of employment are likely disproportionally common with individuals who lack a history of stable employment and are exiting from unemployment. Accounting for informal employment (including partially undeclared work) is thus important in interpreting the results, particularly those relating to employment outcomes and earnings.
In addition to aggregate effects, results are presented across sub-groups of individuals and as well as by selected programme attributes. The results in Chapters 4 and 5 examine sub-groups of workers based on their gender, age, education level, location and unemployment duration.
The OECD’s work on impact evaluations of ALMPs aims to go beyond the outcomes commonly examined in such evaluations, such as effects on employment or wages. As in other participating countries, the work with Greece aims to address another important issue: the impact of participation in ALMPs on occupational mobility. A large body of empirical evidence has documented the “scarring” effect of job loss, with measurable effects on wages that can persist long after an individual is re-employed (for example, Lachowska, Mas and Woodbury (2020[20])). Empirical evidence also shows that jobseekers exiting unemployment tend to disproportionally enter (or return to) low-skills occupations compared to the employed population (Bisello, Maccarrone and Fernández-Macías, 2020[21]). ALMPs can help to counteract these effects by mitigating or possibly even reversing the typically observed negative effects of job loss on individuals’ career trajectories. Training programmes can provide opportunities to acquire skills or qualifications needed for employment in higher-skilled occupations. Wage subsidies may make employers more willing to hire a particular jobseeker and possibly invest in on-the-job training.
To provide a tractable measure of occupational mobility, the analysis uses an occupational index calculated from observed wages. Following the approach adopted by Laporšek et al. (2021[22]) and used in past ALMP impact evaluations in Lithuania (OECD, 2022[15]), Finland (OECD, 2023[14]) and Spain (OECD, 2021[23]), a wage index is calculated for each detailed occupational code using data on the wages and employment of individuals in Greece during the 2013-22 period.5 This index maps each of the 4 355 distinct occupational codes observed in the data into an index that has an intuitive and practical interpretation: an occupation whose index value is one unit greater than another occupation’s index value has an average real monthly wage that is 1 percentage point higher relative to the average wage. Furthermore, increases and decreases in the index can be interpreted, respectively, as positive and negative changes in an individual’s occupation: climbing up or down the occupational ladder.
The occupational index distribution for Greece shows remarkably small changes in the distribution following unemployment: individuals who become re-employed in aggregate become employed in similar occupations. This finding is broadly true also for individuals entering training programmes (Figure 3.7). Following an unemployment spell, the aggregate distributions of individuals are remarkably similar, with only very slight differences: after training, a slightly larger share of individuals work in occupations with wages of index values ranging from 103 to 117. On average, individuals becoming re-employed after training have an occupational index that is only 0.4 percentage points higher than before they were unemployed.
The lack of aggregate effects of unemployment on the occupational index contrasts considerably with the findings of similar analyses in Lithuania and Finland. In Lithuania, unemployment was found to have a considerable scarring effect, with individuals who become re-employed disproportionally entering lower-paid occupations – although the effect of training was less clear-cut (OECD, 2022[15]). In Finland, ALMPs served to decrease the dispersion in the occupational distribution, serving to decrease earnings inequality (OECD, 2023[14]). The lack of strong observed aggregate effects in Greece may be due to the strong seasonal nature of unemployment fluctuations in Greece: large shares of individuals in the tourism sector cycling in and out of unemployment, often in similar occupations.
Note: The heights of the lines indicate the relative share of individuals in occupations whose average wages are on the horizontal axis, relative to the average real wage observed in the Ergani employment data. The distributions are calculated for all individuals who participated in the training programmes analysed during the 2017-21 period who were observed to be employed before or after entering a training programme. Observations with index values above 131 are excluded from the kernel density chart.
Source: OECD calculations based on data from the Greek public employment service (DYPA), Ergani and Diofantos.
Although a descriptive analysis of the occupational index distributions as shown in Figure 3.7 is instructive for understanding the underlying data, it does not take into account a variety of possible underlying factors that could explain the differences in the distributions. For example, differences in the occupational index distributions before and after unemployment may be subject to composition effects, with a subset of individuals more likely to be re-employed. To take account of such factors, the impact evaluation results in the following chapters consider counterfactual outcomes for participants if they had not participated in the programmes, as described in Sections 3.4 and 3.5.
The chapter has outlined selected key ALMPs in Greece, training and wage subsidy programmes. It has also described the rich administrative data available for the evaluation of these programmes: detailed data from the unemployment register, data on participation in training and wage subsidies, and data on employment outcomes. These data sources form the basis for the impact evaluation results presented in the following chapters. The impact evaluation uses a wide range of observable characteristics of jobseekers, including their previous labour market history, to form similar treatment and control groups. The causal effects of the programmes are then estimated by comparing the observed outcomes of programme participants with the counterfactual outcomes that would have occurred in the absence of the programmes. The richness of the administrative data allows several outcomes to be examined: in addition to the most commonly analysed outcome in such evaluations, the probability of employment, the analysis examines the impact of the programmes on employment duration, earnings and occupational mobility.
References
[21] Bisello, M., V. Maccarrone and E. Fernández-Macías (2020), “Occupational mobility, employment transitions and job quality in Europe: The impact of the Great Recession”, Economic and Industrial Democracy, p. 0143831X2093193, https://doi.org/10.1177/0143831x20931936.
[11] Card, D., J. Kluve and A. Weber (2018), “What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations”, Journal of the European Economic Association, Vol. 16/3, pp. 894–931, https://doi.org/10.1093/jeea/jvx028.
[1] DYPA (2023), Κατάρτιση [Training], https://www.dypa.gov.gr/more-katartisi (accessed on 13 November 2023).
[2] DYPA (2023), Προγράμματα Κατάρτισης για το Ταμείο Ανάκαμψης [Recovery Fund Training Programmes], https://www.dypa.gov.gr/proghrammata-katartisis-ghia-to-tamio-anakampsis (accessed on 13 November 2023).
[17] European Commission (2022), Meta-analysis of the ESF counterfactual impact evaluations: Final report, https://op.europa.eu/en/publication-detail/-/publication/66f78a02-96e1-11ed-b508-01aa75ed71a1/language-en.
[18] European Commission (2017), Member State Factsheets on Undeclared Work, https://ec.europa.eu/social/BlobServlet?docId=18723&langId=en.
[12] European Commission and Ismeri Europa (2023), Meta-analysis of the ESF counterfactual impact evaluations – Final report, Publications Office of the European Union, https://doi.org/10.2767/580759.
[19] European Labour Authority (2020), Training labour inspectors to use the new IT tools, https://www.ela.europa.eu/sites/default/files/2021-09/EL-TrainingInspectorsToUseNewITtools.pdf.
[3] Eurostat (2023), Transition from unemployment to employment by sex, age and duration of unemployment - annual averages of quarterly transitions, estimated probabilities, https://ec.europa.eu/eurostat/databrowser/view/lfsi_long_e01 (accessed on 13 November 2023).
[6] ILO (2016), Diagnostic report on undeclared work in Greece, https://www.ilo.org/wcmsp5/groups/public/---ed_emp/documents/projectdocumentation/wcms_531548.pdf.
[20] Lachowska, M., A. Mas and S. Woodbury (2020), “Sources of Displaced Workers’ Long-Term Earnings Losses”, American Economic Review, Vol. 110/10, pp. 3231-3266, https://doi.org/10.1257/aer.20180652.
[22] Laporšek, S. et al. (2021), “Winners and losers after 25 years of transition: Decreasing wage inequality in Slovenia”, Economic Systems, Vol. 45/2, p. 100856, https://doi.org/10.1016/j.ecosys.2021.100856.
[8] Levi, M. (1973), “Errors in the Variables Bias in the Presence of Correctly Measured Variables”, Econometrica, Vol. 41/5, p. 985, https://doi.org/10.2307/1913819.
[7] OECD (2024), “Technical report: Impact Evaluation of Training and Wage Subsidies for the Unemployed in Greece”, OECD, Paris, http://www.oecd.org/els/emp/Greece_ALMP_Technical_Report.pdf.
[14] OECD (2023), Evaluation of Active Labour Market Policies in Finland, Connecting People with Jobs, OECD Publishing, Paris, https://doi.org/10.1787/115b186e-en.
[9] OECD (2023), Part-time employment rate (indicator), https://doi.org/10.1787/f2ad596c-en (accessed on 3 July 2023).
[13] OECD (2022), Assessing Canada’s System of Impact Evaluation of Active Labour Market Policies, Connecting People with Jobs, OECD Publishing, Paris, https://doi.org/10.1787/27dfbd5f-en.
[15] OECD (2022), Impact Evaluation of Vocational Training and Employment Subsidies for the Unemployed in Lithuania, Connecting People with Jobs, OECD Publishing, Paris, https://doi.org/10.1787/c22d68b3-en.
[23] OECD (2021), Impact evaluation of Send@ - a digital tool for PES counsellors in Spain, https://www.oecd.org/els/emp/dg-reform-spain-digital-tool-for-pes-counsellors.htm.
[5] OECD (2021), OECD Compendium of Productivity Indicators 2021, OECD Publishing, Paris, https://doi.org/10.1787/f25cdb25-en.
[16] OECD (2019), Evaluating Latvia’s Active Labour Market Policies, Connecting People with Jobs, OECD Publishing, Paris, https://doi.org/10.1787/6037200a-en.
[4] OECD (2018), Bridging the Digital Gender Divide: Include, Upskill, Innovate, https://www.oecd.org/digital/bridging-the-digital-gender-divide.pdf.
[10] Sianesi, B. (2004), “An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s”, Review of Economics and Statistics, Vol. 86/1, pp. 133-155.
[24] World Bank (2021), Lessons from active labor market programs in Greece, https://blogs.worldbank.org/jobs/lessons-active-labor-market-programs-greece (accessed on 26 June 2023).
Note: Statistics are calculated based on monthly statistics of all individuals unemployed from March 2017-July 2021 for employment outcomes experienced through July 2022. The sizes of the circles are proportional to the number of wage subsidy entrants in each category.
Source: OECD calculations based on data from the Greek public employment service (DYPA) and ERGANI.
Notes
← 1. In addition, the analysis excludes some programmes also based on some other criteria, excluding for example a specific wage subsidy scheme where individuals cannot remain at the same employer after the end of the subsidy or programmes which had previously been subject to evaluation, such as the pilot study contained in World Bank (2021[24]).
← 2. A detailed table containing an overview of all the programmes analysed is available in the accompanying technical report (OECD, 2024[7]).
← 3. An additional restriction – that employers retain wage subsidy participants after the subsidy period – was commonly applied in the programmes as well.
← 4. The rates of transitions into employment are lower in Greece than in most other EU countries. Using internationally comparable (survey-based) definitions of employment and unemployment, the quarterly transition probability of moving from unemployment into employment for workers aged 25-54 was 11% in Greece in 2022, the third lowest in the EU (Eurostat, 2023[3]). In one of the countries with the highest rates, Denmark, it amounted to 46%.
← 5. The analysis uses 6-digit codes and is calculated from real monthly wages at constant 2015 prices. Further restrictions are made in calculating the index, such as excluding individuals who are not employed full-time, individuals with earnings below the statutory minimum wage, and outliers with extremely high reported wages.