Chapter 6. The Slovak Republic’s results, evaluation and learning
Management for development results
While measures are in place to monitor individual projects, the Slovak Republic has some way to go to have a results-based management system. Investing further in this area – particularly defining results and relevant indicators at a strategic level – would provide a more robust basis for strategic decision making, communication, accountability and learning.
A results-based management system is not yet being applied
The Slovak Republic is committed to tracking the impact of its investments and making them more effective. A number of steps have been taken since joining the DAC which will be helpful in establishing a system for managing results. For example:
-
The 2014-2018 mid-term strategy includes a number of goals and objectives for the development co-operation programme overall, and country strategies in programme countries include high-level objectives.
-
The Slovak Agency for International Development Co-operation (SAIDC) has introduced logical frameworks with indicators at a project level.
-
Significant effort is invested in project monitoring and project visits in partner countries to track expected activities and outputs.
However, there is some way to go before policies, structures and systems are focused on results. The broad objectives set out in the mid-term strategy and country strategies are not suitable for identifying the specific, measurable results to which SlovakAid is expected to contribute in a particular country or sector. As a result, the annual ODA report, which is useful in many ways, focuses on disbursements and project descriptions rather than the more global development results to which the Slovak Republic is contributing (MFEA, 2018; MFEA 2017b).
The preparation of the next mid-term strategy provides an opportunity to move to the next step, i.e. monitoring (and managing for) the outcomes brought about by SlovakAid support. Including specific, measurable results in the mid-term strategy –linked to the goals set out in legislation – would allow MFEA as the national co-ordinator to ensure that, as far as possible, all ODA-funded interventions contribute to one or more of these results. Designing a strong theory of change and a logical link from project, to programme and to corporate level will allow the Slovak Republic to use results for strategic direction, accountability and learning, as well as for communicating its objectives and achievements in a more comprehensive manner. In addition, it will allow the Slovak Republic to identify where its mid-term strategy results align with the priorities of its partner countries, thus helping to identify and communicate its specific contribution within chosen sectors. Developing links to the SDGs and targets prioritised by partner countries would be a useful first step (Engberg-Pedersen and Zwart, 2018).
The DAC Results Community has much useful experience to share in this area and may be a useful platform for linking the Slovak Republic with other DAC members who are at a similar stage in setting up their results systems.1
Evaluation system
The Ministry for Foreign and European Affairs (MFEA) is increasingly investing in evaluations and reviews. Further steps are needed to ensure the independence of the evaluation process, including through a more strategic role for the evaluation committee. There is also scope to extend the evaluation strategy and annual plan to development co-operation managed by other parts of government, particularly the Ministry of Finance.
Steps have been taken to strengthen MFEA’s evaluation function, but external scrutiny is not yet adequate
The mandate for evaluating Slovakia’s ODA is assigned to the Development Co-operation and Humanitarian Aid Department (DCHAD) of the Ministry of Foreign and European Affairs (MFEA). Previously, evaluations were undertaken sporadically but an Evaluation and Monitoring Strategy was developed and approved in 2014, which provides basic guidance on standards and procedures (MFEA, 2014; OECD, 2016). A separate budget line for evaluations was also established in 2014. In 2018 the evaluation budget was significantly increased, from EUR 20 000 per year to EUR 150 000, to ensure that a review of existing activities could feed into the new mid-term strategy (MFEA, 2017b). While the emphasis to date has been on evaluating individual projects, the 2018 evaluation plan includes an evaluation of the 2014-2018 mid-term strategy and a thematic evaluation of development education. As resources are limited, it will be important to ensure that priority is given to areas of high risk or high potential for learning.
There is no specific unit responsible for conducting evaluations within the MFEA or in the SAIDC (Figure 6.1). SAIDC staff are involved in the process of deciding what to evaluate, developing terms of reference and engaging external evaluators. The development co-operation division of MFEA is responsible for preparing an annual evaluation plan, developing terms of reference, initiating evaluations and disseminating evaluation results internally and externally, including on the SlovakAid website. It engages external consultants for contracts under EUR 20 000, while the selection procedure for contracts exceeding EUR 20 000 is managed by the Public Procurement Department (PPD) in the MFEA. One official in the division leads on evaluation, but has a number of other operational roles, blurring the distinction between the official’s implementation and evaluation functions, and possibly risking the independence of the evaluation system.
The 2014 Evaluation and Monitoring Strategy introduced an Evaluation Advisory Board – with representatives from SAIDC, Director Generals from MFEA, Slovak diplomatic mission personnel and independent experts (but on an ad hoc basis) – to oversee evaluations, assess evaluation reports and findings, and propose a management response. In practice, management responses to evaluations and follow-up actions are not yet a standard part of the Slovak evaluation cycle.
A more strategic role for the board could strengthen the independence of the Slovak evaluation system. This could include deciding what should be evaluated, both in MFEA and in other parts of government, reviewing and approving terms of reference, tracking implementation of management responses and reporting on the uptake of evaluation findings to the minister and parliament. The more consistent inclusion of external experts and representatives from civil society could increase both learning and accountability, as is the case in France and the Czech Republic.
One joint evaluation has been conducted with the Czech Development Agency. No evaluations have yet been conducted in partnership with host governments.
Training in various aspects of evaluation has been provided for MFEA and SAIDC staff by the UNDP and the Slovak Evaluation Society. However, discussions in Bratislava and a review of evaluations suggest that further professional capacity building in evaluation is required at all levels of development co-operation management and implementation. For example, a number of evaluations that were reviewed by the peer review team would typically be classified as reviews – though they were still useful and instructive.
Source: OECD (2016), Evaluation Systems in Development Co-operation: 2016 Review, https://doi.org/10.1787/9789264262065-en.
Institutional learning
The Slovak Republic is open to learning from its own experience and that of other DAC members, and is committed to sharing its knowledge and information. However, other than a few public events and publications, formal systems are not yet in place to facilitate structured learning. While information flows well within the small teams managing development co-operation in the various ministries and embassies, knowledge exchange is less fluid between ministries.
Structured lesson learning and reflection would strengthen grant appraisal
Within the MFEA, the recording and reporting of activities and lessons learned was made mandatory in 2014 (MFEA, 2013). The compact nature of MFEA and SAIDC means that information flows informally between the two teams, as well as with relevant embassies. However, this is undermined by frequent turnover of key staff, particularly at management level (Chapter 4).
The Slovak Republic has invested in data management systems for its development co-operation. There is an opportunity to expand this valuable data resource further to capture results and lessons. Some DAC members, e.g. Korea, have found it beneficial to extend access to this information to implementing partners.
As mentioned above, management responses to evaluations and follow-up actions are not yet a standard part of the Slovak evaluation cycle. This is a concern given the tendency to roll over project grants for several years. When deciding on a new project, learning could be enhanced by asking the following questions, inter alia, without burdening existing processes:
-
what similar initiatives have been identified?
-
what lessons have been taken into account in the design of this project?
It is clear from documentation and discussions in Bratislava that officials are open to learning and keen to use their own experience and that of others to refine their approaches. Development co-operation fora in partner countries are a good example of where this has been formalised. The first such forum was held in Kenya in 2012 when the first country strategy was being developed. This consultative process is intended as a management tool to aid decision making, and each forum considers the country context, the Slovak Republic’s contribution to development to date and what shape its future contribution might take.
In April 2018, the MFEA together with Slovak Embassies both in Nairobi and Chisinau organised two development fora. In order to provide grounds for a wider debate between all relevant development actors and stakeholders, the events were attended by representatives of the MFEA, SAIDC, Slovak and local NGOs, entrepreneurs, representatives from other donor countries, as well as Slovak and local media. Forum participants reviewed the experience of the Slovak Republic's development activities in the respective programme countries and raised recommendations for future strategic, effective and sustainable development co-operation.
Lesson learning and information flow would benefit from a more structured approach
The Slovak Republic hosts a number of annual public events to raise awareness and inform debate on development co-operation and humanitarian assistance. These include an annual conference and the international documentary film festival “One World”. There is regular dialogue with the NGO platform, and an annual foreign policy review includes a chapter on development co-operation.
There is less evidence, however, of structured reflection and learning on an official level between MFEA and other ministries. For example, as set out in Chapter 4, a number of decisions have been made recently on reforms and funding instruments. Most of these are perceived as MFEA decisions, and even if relevant ministries were consulted, the evidence and rationale for those decisions is not clear to all stakeholders. This undermines ownership of the Slovak development co-operation effort and the potential for drawing on Slovak experience across the system.
The annual ODA report mentioned in Chapter 4 is an invaluable tool for information sharing across government and provides a strong basis for strategic reflection – a useful exercise may be to extract the key lessons from each report for consideration by the Committee for Development Co-operation once that body resumes its functions.
References
Government sources
MFEA (2018), “Annual report of the development cooperation of the Slovak Republic” (provisional text, publication forthcoming), Ministry of Foreign and European Affairs of the Slovak Republic, Bratislava.
MFEA (2017a), “Annual report of the development cooperation of the Slovak Republic” (in Slovak), Ministry of Foreign and European Affairs, Bratislava, http://www.rokovania.sk/Rokovanie .aspx/BodRokovaniaDetail?idMaterial=26455.
MFEA (2017b), “OECD DAC Peer Review 2018: memorandum of the Slovak Republic”, unpublished, Ministry of Foreign and European Affairs of the Slovak Republic, Bratislava.
MFEA (2014), “Evaluation and monitoring strategy of the Slovak Republic”, unpublished, Ministry of Foreign and European Affairs of the Slovak Republic, Bratislava.
MFEA (2013), “Mid-term strategy for development cooperation of the Slovak Republic for 2014-2018”, Ministry of Foreign and European Affairs of the Slovak Republic, Bratislava, http://www.mzv.sk/web/en/foreign_policy/slovak_aid.
Other sources
OECD (2016), Evaluation Systems in Development Co-operation: 2016 Review, OECD Publishing, Paris, https://doi.org/10.1787/9789264262065-en.
Engberg-Pedersen, P. and R. Zwart (2018), "The 2030 Agenda and development co-operation results", OECD Development Policy Papers, No. 9, OECD Publishing, Paris, https://www.oecd-ilibrary.org/development/the-2030-agenda-and-development-co-operation-results_2f391534-en.
Note
← 1. See the results community platform at: http://www.oecd.org/dac/results-development.