1. Assessment and recommendations
To address long-standing labour market challenges, Greece is undergoing an extensive reform of its Public Employment Service (DYPA), its provision of active labour market policies (ALMPs) and its unemployment benefit system. To realise the full potential of the reform, sustainable and sufficient funding needs to be allocated for ALMPs and the job counsellor caseloads need to be lowered. In addition, DYPA needs to develop a concise strategy for its modernisation and engage its staff in the implementation of this change agenda. DYPA has already introduced many changes in ALMP design, including training and wage subsidies, to increase the effectiveness of ALMP support to jobseekers and employers. The evaluations show that wage subsidies and training (for high-demand sectors and ICT) indeed have a positive impact on the participants’ labour market outcomes. To increase the overall impact of its ALMPs through better targeting, Greece should introduce modern jobseeker profiling tools, rely more on the expertise of DYPA’s job counsellors, and align the targeting with the insights from the current impact evaluation, which highlight the differential effectiveness of ALMPs across jobseeker groups.
1.1.1. The Greek labour market has improved in recent years, yet unemployment remains high
The Greek labour market has improved markedly in recent years and weathered the COVID-19 crisis well. Indeed, during the pandemic annual unemployment declined slightly. As of 2022, the unemployment rate among the working age population (those aged 15-64 years) stood at 12.6%, much improved after reaching 27.7% in 2013 following the global financial crisis. However, despite this marked improvement many labour market challenges remain, with the unemployment rate still among the highest in OECD and EU countries.
Of these labour market challenges, many are of a structural nature. This includes widespread informality, work disincentives arising from the tax and benefit system, challenges with low skill levels and skill mismatch in the labour market, seasonal work, and an ageing and declining population. In addition to high overall unemployment and structural challenges, some groups lag behind others. Notably, this includes youth (with Greece having the highest unemployment rate in the OECD among those aged 15-29, at 24.3% in 2022), older workers, women (an employment gap of 20 percentage points with men in 2022 among the working age population), and those with disabilities (with working age people with disabilities having a 30 percentage point lower employment rate than people without disabilities during 2016-19).
1.1.2. Greece spends relatively little on active labour market policies, with a large share going to public works
Despite high levels of unemployment, Greek spending on active labour market policies (ALMPs) has been low in recent years. About 0.30% of GDP was allocated to ALMPs in 2021, much lower than many other countries and around three-quarters of the OECD average of 0.42% (although new programmes currently being implemented in Greece will increase this percentage, particularly with new training programmes).1 Such low levels of spending, coupled with high demand, have translated into little support for jobseekers. In the past, few people have participated in training and wage subsidies programmes. Counsellors face high caseloads, making it hard to meet jobseekers’ needs and provide tailored support.
In the past, a disproportionately large share of Greece’s ALMP spending has been allocated to direct job creation programmes, i.e. public works. Across the 2017-21 period, Greece spent about 48% of its ALMP budget on direct job creation programmes, far more than the OECD average share of around 11%. Public works schemes are often the only broad category of measures available to the most vulnerable jobless Greeks. Alternative measures such work-related rehabilitation, supported work or mixed support measures are essentially non-existent in Greece, although specific measures for groups like ex-prisoners and people with disabilities are available under other types of ALMPs, such as training and employment incentives. International evidence suggests that public works schemes are less effective at facilitating employment compared to other types of ALMPs, which have demonstrated greater efficacy in supporting job integration, though public works may nevertheless be helpful for very specific groups over long time horizons, facilitating their pathways to employment. Spending on training programmes is especially low in Greece, at only 0.02% of GDP over the 2017-21 period, less than a fifth of the OECD average.
ALMP funding from the European Union (EU) through the European Social Funds plus (ESF+) programme and its predecessors, as well as more recently through the Recovery and Resilience Fund (RRF), is crucial for Greece. While exact figures are hard to come by, consultations with stakeholders suggest EU funding accounted for a large share of Greek spending on ALMPs. Such funding is on a project basis, which by its nature leaves some uncertainty for the future and can leave gaps especially for funding of public employment service administration. Given the high demand for ALMPs relative to the currently low levels of funding, Greece should consider increasing its own national expenditure on ALMPs.
1.1.3. Monitoring and rewarding performance will help the Greek Public Employment Service DYPA to become a modern, agile and effective organisation
To address many of the long-standing labour market challenges in Greece, an extensive reform called “Jobs Again” was launched in April 2022 in the design and implementation of ALMPs, benefit schemes, as well as the Greek Public Employment Service, (DYPA). DYPA is undertaking a major re-branding to communicate a change of era unequivocally to its staff, customers and the society at large. Furthermore, while DYPA’s high-level governance model has not changed, the composition of its tripartite Board of Directors was downsized to further increase the speed and agility of taking operational decisions. Additional changes were made to modernise the organisation, including to the organisational structure. While DYPA currently has some channels for incorporating stakeholder feedback, it will be important for DYPA to establish new ways of needs-based involvement of additional stakeholders and experts in its decision-making process. The involvement of such additional experts should aim to ensure that the decisions will meet the labour market needs, particularly given the change in the composition of its Board of Directors.
DYPA should continue developing its performance management system. DYPA has made good progress in starting to establish the performance management foreseen in the “Jobs Again” act, using data in its Data Warehouse for eight indicators focusing on administrative processes, and developing dashboards with data visualisations that both DYPA management and staff in the local offices (KPA2) can access and subsequently discuss. DYPA needs to further ensure that the performance management system is binding for the staff and increase the feeling of ownership among them by involving and engaging staff in designing the system and disseminating the objective and methodology of the system. It is important for DYPA to find ways to incentivise good performance via monetary (at the team level and office level, if counsellor level is not possible) and non-monetary (recognition) ways to create accountability within the organisation. In addition, the best performance cases should be studied to identify good practices that could be rolled out nationwide.
The overall monitoring framework of ALMPs should cover ALMP provision systematically, and measure different dimensions in ALMP provision to be able to identify challenges operatively. The narrower set of indicators for performance management should focus on the key objectives of DYPA and be targeted to drive performance, taking into account the local labour market situation to be fair across local offices. As DYPA’s new business model will be maturing over time, DYPA will need to gradually move towards a performance management system that will focus on ultimate outcome indicators, i.e. jobseekers finding good jobs and employers finding staff meeting their needs for skills.
1.1.4. After the first successful steps towards a modern digital backbone, DYPA needs to define its future digital journey strategically
DYPA was very quick in developing urgently needed digital solutions during the COVID-19 pandemic, and has continued to modernise its digital backbone at a fast pace. DYPA is further planning to update its main operational IT system away from the current legacy platform, develop new digital tools for jobseekers to access and prove their eligibility to benefits and rights, and adopt a cutting-edge AI tool to match jobseekers and vacancies based on competencies. Additional modernisation is likely needed, such as updating the jobseeker profiling tool and developing a feedback mechanism to inform training programmes (a project with the European Commission and the World Bank and a project with the OECD are currently being undertaken to address the different aspects of this modernisation). Further modernisation needs include automating and digitalising processes that are not yet digital, and streamlining the more recently developed tools to fit well within the renewed digital infrastructure.
DYPA’s digital solutions have already hugely improved due to the newly developed possibilities to exchange data with other registers via the infrastructure of the Central Interoperability Centre (KED), and now has web services to exchange data on employment records, taxes, social security, property, refugee status etc. While additional web services for operational reasons are being implemented, the capacity to systematically link administrative data for evidence generation and share these securely with researchers still needs to be developed. In addition, it would be important to link data from DYPA with data from the National Social Security Fund (EFKA) which, in addition to being helpful for evaluations, is a necessary precondition for implementing potential reforms to the Greece’s unemployment benefits scheme. Finally, data on public sector employment which would be crucial to fully capture the labour market outcomes of ALMP participants after they are no longer participating in ALMPs. A fully-fledged evidence-based approach to ALMP design and implementation would need additional financial and human resources to be able to conduct evaluations both in-house and contract them out, in addition to developing technical capacity to link and share data.
While DYPA needs to stay agile in its progress in digitalisation, an underlying strategy is needed to guide DYPA in this pathway. The strategy should state the objective and principles for digital innovations, helping to prioritise the different IT projects and set a clear framework for digitalisation. A clear strategy would help to ensure that initiatives are well integrated both technologically and in the processes followed.
1.1.5. Improvements in ALMP design have started
DYPA has recently introduced many changes in ALMP design to increase the effectiveness of ALMP support to jobseekers and employers. The “Jobs Again” reform seeks to further improve training measures and decrease skill mismatches in the Greek labour market. For example, the concept of “payment by results” for both the training provider and the participant was implemented in 2023, aiming to decrease drop-out rates, and increase training quality and post-training employment rates. It is crucial to monitor and evaluate this new concept and consult with the providers to ensure a successful implementation.
The “Jobs Again” act also aims to strengthen the regulation of job-search requirements to decrease misuses of the benefit system, nudge jobseekers to increase job search efforts, give counsellors more tools to activate jobseekers and allocate counselling resources to those jobseekers who need these. To implement the job-search requirements successfully, the counsellors need clear guidelines, thorough training and digital solutions supporting the new processes. In addition, the counselling capacity of DYPA needs to be scaled up. While a proper application of job-search requirements would eventually decrease counsellors’ caseloads, current caseloads are too high to implement job-search requirements appropriately and equally to all jobseekers.
Additional changes should be considered in the ALMP package in Greece, such as introducing longer or continuous programmes with wider eligibility criteria but narrowly targeted to the individual needs of the jobseekers, to relieve the administrative burden of programme management. Further reforms on entitlements associated with registering with DYPA beyond unemployment benefits might be relevant to target ALMPs. Public works schemes, which are generally found to be ineffective measures in the evaluation literature (at least in the short to medium term and when labour market outcomes are considered), should be revised by the Ministry of Employment and Social Affairs (YEKA) after analysing which groups benefit from such measures and how to increase the positive effects, and which groups need different support altogether. It would be also important to estimate the impact of such measures on social integration outcomes, if data are available.
1.1.6. DYPA needs to increase its capacity to counsel jobseekers
International comparisons of client numbers per job counsellor between OECD countries are problematic as the responsibilities of Public Employment Services (PES) and the organisation of tasks within PES differ considerably across countries. Nevertheless, the caseloads in Greece are very high even after doubling the number of counsellors in 2022, reaching 1 847 jobseekers per counsellor in the second half of 2022. This is 10 to 20 times that found in modern and well-performing ALMP systems where the caseloads amount to 100-150 jobseekers per counsellor and are even lower concerning jobseekers furthest from the labour market.
In addition to supporting counsellors with appropriate tools so that they can deliver their services as effectively and efficiently as possible and reserve time for other functions, DYPA needs to have more counselling capacity to provide counselling and meaningful support to jobseekers. Research in other countries has found that reducing the number of jobseekers per counsellor not only helps jobseekers find work more effectively, but that the savings from reduced benefit expenditure and increased tax revenues save the government money over the long term. Increasing the number of counsellors may also require co-ordination with other Greek entities, such as the National Organisation for Certification of Qualifications and Vocational Guidance, who also play a role in ensuring that there are sufficient numbers of certified DYPA counsellors in Greece.
1.1.7. DYPA needs a clear and concise strategy for its transformation
DYPA has not yet developed a clear strategy for its transformation, but follows instead an abundance of acts, regulations, circulars and ministerial decisions that state DYPA’s long list of tasks and responsibilities.
DYPA should draw up a concise strategic plan and that sets out its clear long-term objectives, even though such a practice is not (yet) common for Greek public sector organisations. This should be aligned with relevant regulations and DYPA’s other strategic documents such as a digitalisation strategy, a strategy for generating knowledge and annual plans to implement the long-term strategy.
A clear and concise strategy is vital to get the whole DYPA on board with the change agenda. Every staff member needs to easily understand the strategy to be able to contribute to the common goals. Furthermore, DYPA should engage staff in the strategy development process to create joint ownership of the change agenda. Involving and engaging staff in DYPA’s planning and innovation processes would facilitate the implementation of the changes (decrease resistance to change and embrace responsibilities), as well as encourage innovative ideas that might not occur with purely top-down processes.
1.2.1. Training programmes are effective at connecting people with jobs
The counterfactual impact evaluation (CIE) conducted in this report examines two of Greece’s ALMPs, wage subsidies and training for the unemployed. The evaluation of the training programmes focuses on Greece’s three main training programmes in the recent past. One of the training programmes is intended to train individuals to enter “high-demand sectors” (although in practice training participants end up working in essentially all sectors of economic activity, with the wholesale and retail trade sector accounting for roughly one-quarter of participants). The other two training programmes evaluated provide tertiary education graduates with information and communication technology (ICT) training, targeting individuals aged 25-29 and 30-45 years, respectively. All three programmes are considered important by DYPA as they have the potential to support the digital transition and address skills shortages in key sectors, including tourism. They are voucher-based training where jobseekers can select from accredited training providers and contain both theoretical and practical components. They last approximately five months in total for the high-demand sectors training and seven months for the ICT training. During the period analysed in this report, participation in the three training programmes amounted to about 22 000 persons, with 90% of them being in the high-demand sectors training. The high-demand sectors training ran for the period from June 2017 through December 2018 and the ICT trainings from January 2020 through June 2020.
Following a “lock-in” period of about four months (corresponding approximately to the training duration) where participants are typically not actively seeking work, the impact of training on participants’ employment becomes positive and is statistically significant. One year after starting a training programme, the probability of employment is 7 percentage points higher for training participants compared to that for similar non-participants. The effect of training participation on employment reaches 9 percentage points two years after entering training, a quite sizeable effect. When compared to the international literature on the impact of training on employment, training programmes in Greece appear to be particularly effective, situating in the upper quartile of the estimated effects in international studies the short, medium, and long run.
Training programmes are also effective for many different groups of jobseekers, but even more so for younger people and those with higher levels of education. For instance, two years after entering training programmes, men under age 30 are 19 percentage points more likely to be employed than similar jobseekers who do not participate in training (the comparable figure for women is 16 percentage points). The higher effect for jobseekers with higher education levels is, to some extent, driven by ICT training which is available to jobseekers with tertiary education.
Training programmes lead to a lower likelihood of registered unemployment for participants. Twelve months following the start of training, participants are about five percentage points less likely to be registered as unemployed, an effect which mostly persists over the medium term, but falls slightly over the longer term and amounts to two percentage points three years after entering training. Training participation is also associated with a higher probability of unemployment benefit receipt starting 15 months after entering the training programme, with participants up to 10 percentage points more likely to receive benefits in long term. This effect, combined with the positive effect on employment, likely reflects that some training participants transition in and out of employment, and accumulate rights which trigger the payment of unemployment benefits when they become unemployed.
1.2.2. Training programmes boost earnings of participants and ICT training promotes upward occupation mobility
Participation in training leads to more days in employment and higher earnings. Three years after the start of training, participants have experienced 66 more days in employment than similar jobseekers who did not participate in training. The effects of training on cumulative earnings build consistently over time and, in the three-years following the start of training, participants earned a cumulative total of EUR 1 500 (in 2015 prices) more than similar jobseekers who did not participate in training. These effects, as well as the overall effects on employment, registered unemployment and benefit receipt are quite similar for the three programmes.
Beyond the effects of training on employment and earnings, it is important for policy makers to assess if training leads to upward occupational mobility, looking closer at possible differences between the training for high-demand sectors and the ICT programmes. The ICT programmes are indeed found to boost occupational mobility. Participants of the programme for individuals aged 25-29 years receive a fairly strong boost to their earnings – entering occupations that pay on average 6 percentage points more than their control group counterparts – with the programme for individuals aged 30-45 years providing a moderate boost, entering occupations paying 3 percentage points more on average. These results highlight the important role that such training can play in supporting mobility in the context of the digital transition in Greece. In contrast, the training for the high-demand sectors seems to have a small negative effect on occupational mobility, likely reflecting that many participants of such training end up in lower-paid occupations such as sales clerks.
1.2.3. Expand training programmes to reach more people
While training programmes are effective at helping individuals into employment, they reached relatively few people in the period examined. Hence, many unemployed persons who could have benefited from training missed out. Greece has spent much less than other countries on training programmes for unemployed persons. Over the period 2017-20, Greece spent around 0.02% of GDP on training programmes for unemployed people, which is less than one-fifth of the OECD average (0.11% of GDP). The introduction of new training programmes, supported by EU funding and intended to train 700 000 individuals from 2022 to 2025 (including both unemployed and employed persons), is a significant and positive step. Given that this evaluation shows large, positive effects on the programmes studied, the new programmes can be expected to be worthwhile – and have a positive impact – even if they end up being considerably less effective than the ones examined in this evaluation.
The analysis in this report shows that training programmes can be effective for a number of different groups of jobseekers. Nevertheless, it is important to target the training programmes to those that would benefit from these the most, both in the design and implementation of training. These groups include mainly young people (below the age of 30), jobseekers with higher education and those who have been unemployed for less than one year. To provide training based on individual needs, the underlying needs assessment could combine inputs from data (a profiling model and using the results of the current evaluation) and counsellor discretion.
1.3.1. Wage subsidies appear effective in placing individuals in employment, with generally positive effects on other labour market outcomes
The results of the impact evaluation show that wage subsidies have large and statistically significant positive effects on the probability of individuals being in employment. Although the effects are largest immediately after individuals enter the programme and decline thereafter, they remain large throughout the time horizon examined in the evaluation. Even three years after entering the programme, wage subsidy participants are almost twice as likely to be employed as similar individuals in the comparison group. Specifically, 59% of wage subsidy participants are employed three years after entering the programme, compared to 32% of the comparison group – a difference of 27 percentage points. These estimated effects are large also when compared with the results of similar programmes in other countries.
Corroborating the positive employment effects are the findings that programme participants also have higher earnings and days in employment. Over the three-year time horizon studied, wage subsidy participants were employed for 524 days more than individuals who did not participate in the programmes. Likewise, three years after entering the wage subsidy programme, participants earned EUR 13 784 more than non-participants over the three-year period. As with employment, these effects are most pronounced during the first 12 months after entering the programme, but they continue to have positive effects throughout the three-year observation window during which individuals are tracked in the analysis. The effects of programme participation thus outlast the duration of the subsidy, as well as any additional period during which firms are required to continue to employ individuals hired via the subsidies.
Assessing how these increased earnings compare to the direct costs of the programmes is difficult due to a lack of data on the exact costs of participation. Such comparisons would ideally include actual costs of programme participation, accounting for different programme parameters and attrition in programme participation. However, a rough comparison using the parameters of the largest programmes suggest sizable gains even after accounting for the estimated costs, highlighting the programme’s potential cost-effectiveness: for one of the larger programmes examined, roughly half of the additional earnings are not attributable to the wage subsidy payments made to employers.
While the wage subsidy programme significantly enhances employment and earnings for jobseekers, it also has a small adverse impact on their occupational mobility in the short run. Participants in subsidised employment tend to enter occupations that, on average, pay slightly less compared to the occupations of those in the comparison group. This effect is small but statistically significant for the first 21 months of participation in the programme. This suggests that while jobseekers benefit from improved employment prospects and overall higher total earnings through the subsidy programme, they may be doing so in occupations which generally pay less, possibly by lowering their reservation wage (i.e. the wage at which they will accept a job offer). This trade-off indicates a concession in occupational status for gains in employment stability and cumulative earnings during the three-year period tracked in the analysis.
1.3.2. The positive estimated effects of the wage subsidies may be overstated due to specific institutional features in Greece
Although Greece’s wage subsidies are probably effective, the precise magnitude of the effects should be interpreted with caution due to several specific features. While similar caveats are relevant for most impact evaluations of wage subsidies using administrative data, especially those employing non-experimental methods, there are specific reasons why they could be particularly relevant in the case of Greece.
One factor that could lead to the results being overstated relates to deadweight effects. These occur when subsidies unintentionally support hiring that would have otherwise occurred without them. The potential for these effects to be present is arguably exacerbated by certain features in the Greek wage subsidy programmes before July 2020. These include employers being able to propose their own desired candidates and the high administrative burden associated with receiving the subsidies, which made it more probable for those certain about hiring to opt for the subsidies.
A second factor that could contribute to the large estimated effects relates to the mismeasurement of counterfactual outcomes resulting from undeclared work. Despite recent efforts aimed at addressing it, undeclared employment remains a relevant feature of Greece’s labour market, particularly for the sub-population of individuals registered as unemployed with DYPA. Furthermore, individuals engaging in such work have an incentive to register as unemployed to gain access to a range of benefits: even if these are not sizable, they may not be negligible especially given the relatively small cost associated with registering. Many registered as unemployed might be engaged in undeclared work, thereby lowering the measured employment rates of individuals in the comparison group and inflating the estimated impact of the programmes (in this case, of both the wage subsidy and the training programmes).
1.3.3. Target wage subsidies towards long-term unemployed
The targeting of wage subsidies in Greece is a particularly relevant issue given the relatively low level of expenditure on ALMPs in Greece and the relatively limited reach of programmes. Subsidies should be targeted to maximise impact and minimise deadweight effects – employment that would have occurred in the absence of the subsidy.
While wage subsidies in Greece are found to be highly effective in helping different groups of workers into employment, there are some important differences in effectiveness that could be taken into account in terms of targeting. In particular, the long-term unemployed – those who have been unemployed for more than 12 months – experience a slightly larger improvement in their employment probability than those who have been unemployed for less than a year. Although the magnitude of the difference is relatively small, this finding is important, particularly in view of the relatively low take-up of wage subsidies among the long-term unemployed. The long-term unemployed accounted for 50% of the registered unemployed but only 45% of the wage subsidy recipients. This suggests that a shift in the targeting of wage subsidies towards the long-term unemployed could increase the overall positive impact of wage subsidies on employment.
Targeting wage subsidies more toward the long-term unemployed could also result in lower deadweight effects. First, it would provide better support to workers with poorer labour market prospects. In Greece, as in many other countries, the probability of finding a job decreases with the duration of unemployment. Thus, making individuals generally eligible for wage subsidies only after longer spells of unemployment may make it less likely that relatively employable individuals will be hired through subsidies. Second, it may make it more difficult for employers to engage in strategic behaviour whereby an employer first decides to hire someone and then refers them to apply through a wage subsidy programme. While such behaviour has been made more difficult by recent changes to the candidate selection process, further changes to the eligibility criteria for jobseekers would make it even more difficult.
1.3.4. Redesign wage subsidies to further increase their attractiveness to employers while mitigating possible deadweight effects
An effective wage subsidy scheme should strike an appropriate balance between several competing factors. First, it should support jobseekers who have poor labour market prospects, such as due to a lack of demonstrable relevant work experience. The scheme should help place such jobseekers into sustained employment in good jobs. Second, it should be attractive for employers, providing them with an incentive to hire someone they otherwise would not, but without excessively onerous conditions. Finally, the scheme should minimise unintended consequences such as deadweight hires.
Compared to their features prior to July 2020, the various wage subsidies programmes in Greece have undergone important changes in this direction. Employers can no longer propose candidates they would like to hire but are obligated to choose from a list of candidates proposed by the PES. Programmes no longer impose retention requirements at the end of the wage subsidy period, recognising that one of the important functions of subsidised employment is to test the suitability of a job match in practice. The administrative burden of the programmes has been reduced by, for example, only verifying whether an employer meets the subsidy eligibility criteria after a suitable candidate has been identified.
However, a further change could be made to improve uptake amongst employers while still further mitigating possible deadweight effects. The requirement that an employer has not dismissed workers in the three-months prior to applying for the wage subsidy should be reconsidered. While the terms allow for exceptions to this criterion – non-renewed fixed-term contracts constitute one such exception – the requirement nevertheless imposes a significant administrative burden, uncertainty about eligibility, as well as excluding the largest firms in practice. This has contributed to a low take-up rate of wage subsidies amongst larger firms – the very firms that tend to have higher paid jobs. To offset the potential for strategic behaviour, DYPA could consider imposing retrospective conditions on retention rates to establish eligibility: employers that had insufficiently low rates of retention of wage subsidy recipients could be deemed ineligible for subsidies.
1.4.1. Impact evaluations of ALMPs identify what works and for whom
Evidenced-based policy making for ALMPs requires conducting CIEs in addition to regular policy monitoring. Producing basic monitoring statistics, such as on the number of participants, programme completion rates, demographic characteristics of participants, and basic data on employment outcomes for participants, are helpful for providing an immediate view on ALMPs, assessing whether targets (such as targets related to programme uptake) are being met, and can help to spot issues with implementation. To understand whether a programme is effective in connecting people with jobs, it is not enough to observe the percentage of individuals finding a job after their participation. This is because some people who did not participate in the programme would have found a job anyway even if they had not participated. To measure the effect of a programme a CIE is needed, a method which estimates a “counterfactual”, that is what would have happened to participants if they had not participated in the programme. A CIE can also inform whether the programme produces the desired effects for all participants and identify groups for which a programme is most or least effective. It is necessary for deciding whether a programme should be stopped, continued, improved or expanded, allowing for resources to be allocated to where they are most effective at supporting jobseekers. CIEs are also needed to conduct cost-benefit analyses which can tell policy makers if a programme is cost-effective.
The method used in this report – propensity score matching – compares programme participants with similar non-participants, that is “a matched comparison” or “control” group. It is crucial to make this matched comparison group as comparable as possible to programme participants, so that differences in outcomes can be attributed to the programme, rather than to other differences between participants and non-participants.
1.4.2. Greece’s linked administrative data offer excellent opportunities for ALMP evaluations
The impact evaluations carried out in this report were only possible because of the linking of rich administrative data across three registers. Firstly, data on participation in wage subsidies, detailed background characteristics on participants and non-participants and benefit receipt were obtained from the DYPA register. Secondly, data on participation in training programmes were obtained from the Diofantos register, and thirdly, data on employment histories and employment outcomes, as well as more detailed data on subsidised jobs were obtained from the Ergani register. All these data were linked using a unique identifier and were subsequently pseudo-anonymised before sharing for analysis. This is the first time such rich and comprehensive data have been put together for ALMP research purposes in Greece. Similar data with somewhat less detailed information across certain dimensions were also put together for two recent studies conducted by IOBE and the World Bank.
All three evaluations highlight the value that such linked administrative data hold. Yet, such data can be useful beyond impact evaluation with many possible applications in labour and social policy. For instance, such rich data could help to better understand specific client groups, map the delivery of different services, or study wider topics of inequality and the reach and effectiveness of social policies. Therefore, investing in the capacity to link and share data for research purposes could have a significant positive impact on the evaluation of ALMPs, as well as on building evidence-informed policy making in many other employment and social policies.
1.4.3. Greece should invest in capacity to link administrative data for research purposes
Linking, anonymising and sharing the data and metadata for this project required substantial efforts from the Greek authorities. Similar work should be easier and would require less effort in the future if DYPA and YEKA could build on the capacity gained during the evaluation exercise conducted by the OECD. The process could also be optimised to make linking data easier, faster and more accurate.
In a first instance, more could be done to provide comprehensive metadata. The OECD project benefited greatly from the efforts of staff at DYPA, Diofantos, and Ergani through many written, virtual, and in-person exchanges. Some of these learnings have been captured in this report and the associated technical report produced within the project. However, creating additional metadata could speed up researchers’ familiarisation with the data and reduce the need for follow up communication, saving both researchers’ and the Greek authorities’ time and ensuring a more accurate interpretation of the data shared for research.
More could also be done to standardise the data linking process. Currently the processes for extracting and linking data have not been standardised, with new extractions and updates taking a long time and being prone to inconsistencies. For instance, writing a code for extracting and linking the data could make the process more efficient by minimising the time it takes to prepare data for different projects and reducing errors.
In future evaluations of ALMPs, Greece could consider linking data from more registers with the Ergani and DYPA registers. Linking in more data could improve the accuracy of policy evaluations by expanding the number of variables to be controlled for, allow for the examination of a wider set of outcomes and analyse more policies, including social policies. For example, the effects of ALMPs on health or further education could be investigated if data from the relevant agencies were included. While Ergani data does include measures of individuals’ labour earnings, data to cross-check the accuracy of these fields (which are known to have issues in earlier periods) could potentially be obtained from EFKA or from the Independent Authority for Public Revenue (IAPR). Linking these data for research purposes should be tied to a specific policy evaluation and tailored to its needs. Knowledge and capacity acquired through this project by the OECD and the EC could help make the process more efficient.
Building internal capacity in the different organisations involved will be crucial to ensure continuity and leverage on knowledge and experience acquired. Attracting and retaining staff with the right skills profiles will be necessary whether DYPA decides to conduct evaluations of ALMPs internally or contract them to external researchers. In addition, it will be important to ensure regular exchanges between the different institutions sharing data to achieve a common understanding and discuss ways to improve the processes related to extracting, linking and sharing data. In particular, strengthening the existing co-operation between DYPA and the unit of Experts in Employment, Social Insurance, Welfare and Social Affairs (MEKY) of YEKA would be important to ensure co-ordination of activities regarding data exchanges, linking and use for research and strengthen analytical capacity in DYPA.
1.4.4. Greece should embed RCTs in policy making and include elements of evaluation in the design of new policies
Estimating the causal impact of a policy on participants requires knowing what would have happened to ALMP participants had they not participated in the ALMP. One of the most effective ways to do that is through Randomised Controlled Trials (RCTs) that assign individuals to the treatment and the control group in a random manner, so that individuals in the two groups are comparable before treatment starts. Greece should use such RCTs to test the effectiveness of new or re-designed ALMPs, also prior to rolling them out more broadly. If an RCT is well designed and implemented, a simple comparison of average outcomes in the two groups can provide accurate estimates of the impact of a policy on the outcomes of interest. However, RCTs are not without limitations which relate to ethical issues, implementation challenges that can affect the randomisation element, low take-up or high dropout, spill-over effects, unobserved non-compliance and anticipation effects, among many others. In case that RCTs are not possible to implement, Greece could consider alternative approaches, such as implementing policies in a staggered manner across time and space. Another quasi-experimental approach is to compare similar individuals around a threshold, such as unemployment duration, which triggers important changes to eligibility or programme parameters such as the generosity of wage subsidies. In the latter case, sufficient numbers of participants must exist around the threshold to allow for the differences to be meaningfully interpreted.
Continue the transformation of DYPA into a modern, agile, effective and efficient organisation
Draw up a clear and concise strategy to set the long-term objectives, enabling to prioritise activities and investments.
Establish ways to involve additional stakeholders and experts in the decision-making processes of the tripartite Board of Directors of DYPA needs-based to ensure that the decisions do meet the labour market needs.
Continue developing a binding, transparent and streamlined performance management system. Establish fair monetary and non-monetary incentives for good performance. In addition to discussing underperformance, study the best performance cases to identify the best practices that could be rolled out nationwide.
Adopt the results chain framework (inputs-activities-outputs-outcomes) in the overall monitoring framework of ALMPs to identify challenges in ALMP provision systematically and operatively. Establish a narrower set of indicators with assigned target levels for performance management focusing on jobseekers finding good jobs and employers finding staff meeting their needs for skills.
Involve staff in the development of DYPA’s strategy and performance management system, as well as planning and innovation processes more generally to engage and empower staff, generate the feeling of ownership and encourage innovative ideas that might not occur with purely top-down processes for innovation.
Increase, reallocate and streamline resources to build DYPA’s capacity to deliver meaningful support to its clients
Develop a strategic concept for the digitalisation pathway of DYPA, establishing the objective and principles for digital innovations, and helping to prioritise the different IT projects. The digital strategy should address achieving sustainable resources for IT developments, using modern development concepts, ensuring take-up and avoiding staff resistance, managing the different risks associated with digital tools, and monitoring and evaluating digital tools systematically.
Support the counsellors with appropriate tools, such as modern jobseeker and skills profiling tools, so that they are able and motivated to deliver their services as effectively and efficiently as possible.
Continue working on leaner and more automatic administrative processes and modern digital solutions for both counsellors and clients.
Increase counselling capacity and decrease caseloads to provide more effective counselling and meaningful support to jobseekers.
Provide clear guidelines, thorough training and appropriate digital infrastructure for counsellors to implement job-search requirements successfully.
Fine-tune the design and implementation of ALMPs to increase their effectiveness and efficiency
Establish more permanent programmes with wider eligibility criteria but targeted to the individual needs of the jobseekers instead of the many temporary programmes that are split by narrow target groups but can be inefficient to manage and still not reach those jobseekers that need the support the most.
Monitor and evaluate the new model of payments by results for training programmes and establish a continuous dialogue with the providers to ensure that any challenges in this framework are addressed in time.
Consider the evaluation and revision of the public works schemes to target these only to groups that benefit from such measures. Likewise, consider designing (additional or alternative) measures for groups that need different support in integrating into society and labour market.
Reconsider targeting of wage subsidies and training programmes
Target wage subsidies more towards long-term unemployed instead of the newly-unemployed to lower deadweight effects and increase impact.
Conversely, expand training for individuals who have recently become unemployed to maximise their impact.
Target training programmes to young people (below the age of 30) for whom training is more effective.
Combine the results of statistical profiling with the findings of this evaluation exercise as well as counsellors’ discretion to inform targeting of wage subsidies and training programmes.
Abolish the requirement that employers must not have dismissed workers in the three months before applying for wage subsidies. Instead, consider imposing retrospective conditions, temporarily disqualifying employers with low retention rates of wage subsidy recipients from future subsidies.
Expand training programmes and assign a greater role to counsellors in referring unemployed people to ALMPs
Expand training offerings to support more jobseekers into employment and ensure they are available throughout time and space.
Assign a greater role to counsellors in supporting referring jobseekers into ALMPs, including by ensuring clients are aware of these programmes and the benefits they offer. To do this effectively, counsellors would need to be better supported.
Establish a fully-fledged evidence-based approach to ALMP design and implementation
Allocate additional financial and human resources to be able to conduct evaluations both in-house and contract them out, including in co-operation with YEKA and research units in other institutions, meeting both operational and strategic needs for reliable evidence.
Develop the capacity to systematically link administrative data for evidence generation and make these securely available for researchers.
Explore the possibility to use RCTs to overcome challenges in interpreting administrative data results for Greece, by addressing selection bias/unobserved heterogeneity. RCTs could also be used to examine questions such as effects of enforcing conditionality requirements or additional support (e.g. increased intensity of counselling).
Disseminate impact evaluation results widely in a suitable format for different target stakeholders, particularly among DYPA counsellors, to improve buy-in.
Note
← 1. These figures exclude categories 1.2, benefit administration, which is missing for Greece, and 4.2, temporary employment maintenance incentives, which were heavily influenced by the COVID-19 crisis.