copy the linklink copied!1. Legal and institutional framework of the public procurement function in Kazakhstan
This chapter describes and analyses the normative framework of Kazakhstan’s public procurement system, which is currently undergoing a process of centralisation, and the role and relevance of the institutions managing it. It assesses the impact of the public procurement Law (PPL) adopted in 2015 and the impact of the December 2018 amendments to revise the 2015 PPL. The chapter also analyses the potential benefits and risks of more centralised purchasing, as Kazakhstan is to allow the aggregation of purchases from different contracting authorities, and identifies necessary steps on the path to centralisation. Lastly, it introduces the benefits of framework agreements, a tool commonly used by OECD countries in public procurement.
The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law.
According to the Ministry of Finance, public procurement in Kazakhstan accounted for a total of approximately KZT 4.15 trillion in 2018 (EUR 10.2 billion). This was about 7.1% of GDP in 2018, up from 5.4% in 2017. Public procurement spending as a proportion of general government expenditures was 35.6% in 2018, up from 22% in 2017. While the share of public procurement spending in GDP is lower than the OECD average (11.8% of GDP in 2017), its share of general government expenditures is higher than the OECD average (29.1% of in 2017) (OECD, 2019[1]). Therefore, public procurement represents a large market in Kazakhstan. Several public services are conducted by state-owned enterprises that are not accounted for in the above-quoted figures. This further underlines the fact that the government has a substantial economic footprint within Kazakstan.
The framework for public procurement in Kazakhstan has undergone significant and frequent changes in recent years, taking account of the growing importance this function has for effective public service delivery and adapting it to increasingly complex demands of public institutions and Kazakhstan’s citizens. Frequent changes to the law are likely to increase the number of mistakes made by procurement practitioners as they might not be aware of the changes or do not know how to apply the new rules being introduced.
The legal framework for public procurement is based on the Public Procurement Law (PPL) adopted in 2015. Amendments adopted in December 2018 (hereafter: December 2018 amendments) introduced substantial modifications to the PPL in an effort to increase the efficiency and effectiveness of public procurement.
This chapter introduces the legal and regulatory basis for conducting public procurement in Kazakhstan and presents the governance of the public procurement function (first and second sections). A third section summarises overarching strategies in the area of public procurement and the related public procurement reforms adopted in December 2018. A general trend in Kazakhstan’s public procurement system has been to concentrate increasingly on the management of public procurement procedures. Therefore, the final section of the chapter analyses the benefits of centralising public procurement for greater efficiency and effectivess in the context of Kazakstan.
copy the linklink copied!1.1. The normative framework for Public Procurement in Kazakhstan
Public procurement in the Republic of Kazakhstan is regulated by several laws and regulations that will be discussed in this section. In addition, Kazakhstan is party to several international agreements that have a bearing on public procurement as well. In general, the public procurement normative framework consists of the provisions in the following laws and regulations:
-
The “Law on government procurement” dated 4 December 2015, No. 434-V ЗРК, as updated (hereafter: the Public Procurement Law, PPL).
-
The “Rules for the Implementation of Government Procurement”, as approved by the Decree of the Minister of Finance of Kazakhstan No. 648, dated 11 December 2015, as updated (hereafter: the Public Procurement Rules).
-
“The list of goods works and services whose procurement is conducted through Single Organiser”, as approved by Decree of the Minister of Finance of Kazakhstan No. 1127 dated 29 December 2018 (as updated).
-
Other decrees of the Minister of Finance of Kazakhstan related to public procurement.
This public procurement framework is in line with the procurement-related agreements in the Treaty on the Eurasian Economic Union (EAEU, current version adopted on 29 May 2014), as well as with the Civil Code (adopted on 27 December 1994, N° 268-XIII) and the Budget Code of the Republic of Kazakhstan (adopted on 4 December 2008, N° 95-IV), of which the Republic of Armenia, the Republic of Belarus, the Republic of Kazakhstan, the Kyrgyz Republic and the Russian Federation are full right members as of 14 May, 2019. Kazakhstan joined the World Trade organisation (WTO) on 30 November 2015 and became an observer in the WTO Committee on Government Procurement (GPA) in October 2016. As part of its WTO accession protocol, Kazakhstan undertook commitment to initiate accession to the GPA. The GPA is a WTO multilateral agreement covering the procurement of goods, services and capital infrastructure by Governments and other public authorities.
The public procurement legal and regulatory framework applies to government organisations (ministries, government agencies, local administrations, etc.; see definition below) and to enterprises in which the government owns more than 50%. Dedicated rules apply for state-owned enterprises (national holdings), including Joint Stock Company (JSC) Baiterek, JSC KazAgro and JSC Samruk-Kazyna. The Sovereign Wealth Fund Samruk-Kazyna accounts for the vast majority of public procurement spending in Kazakhstan (see chapter 6.) Similarly, the central bank (National Bank of the Republic of Kazakhstan) has its own procurement rules and is out of the scope of the public procurement framework.
The PPL covers public procurement of any goods, works and services, irrespective of the amount of the respective public procurement contract, except for those that are expressly excluded. The list of exclusions is exhaustive and provided in article 1 of the PPL. The law presumes equal treatment of suppliers, including local and foreign ones. However, hurdles to the participation of foreign suppliers exist in practice, for example due to the structure of the e-procurement system. These hurdles are due to the national interest of Kazakhstan, according to stakeholders, and in fact, there appears to be a general push to support local suppliers.
Kazakhstan’s Industry and Expert Center is the main instrument of its efforts to support local content, including through the implementation of industrial development programs. The Institute distributes various grants and subsidies to support business investments to increase productivity in priority sectors, or to support exporters. For instance, resident businesses can claim reimbursement of their expenditures to obtain the certification of their products against International Standards, including quality management certificates.
As a party to the EAEU Treaty, Kazakhstan also follows a “national treatment provision” in the sphere of public procurement for the EAEU member countries. This allows suppliers from EAEU member countries to participate in public procurement tenders on equal terms with domestic suppliers in most cases (Eurasian Economic Union, 2015[2]).
Before a contracting authority proceeds with a selection of a supplier, it shall determine who will be the so-called “organiser” responsible for organising and performing all public procurement processes. A contracting authority itself can undertake the role of the organiser (i.e. purchaser and organiser would be the same entity), or it can appoint one of its departments or a subordinated government organisations as an organiser. State-owned enterprises can appoint an affiliate or a parent company as procurement organiser.
The public procurement framework also contains the notion of single organiser, a concept that is similar to a specialised procurement unit. Contracting authorities have to procure certain goods, works and services through a single organiser, as defined by decree. The Government Procurement Committee (GPC) of the Ministry of Finance is the single organiser for the central government. The “list of goods, works and services whose procurement is conducted through the single organiser” is defined by decree of the Ministry of Finance. At the local level, single organisers are usually Public Procurement Offices or Departments under the Akimats (local administrations).1 See section 1.2.3 for a detailed analysis of single organisers.
Public procurement in Kazakhstan can be performed through one of the following five methods:
-
1. competitive tender (open tender, two-stage tender),2
-
2. auction,
-
3. requests for quotations,
-
4. direct award,
-
5. purchases at commodity exchanges.
The law does not provide for a default method. Organisers and single organiser are free to choose the method that they deem most appropriate, except if a decree of the Minister of Finance mandates a specific procurement method. For all methods, rules and thresholds define the circumstances under which they can be applied. However, many other countries in the OECD or the EU emphasise the expectation that by default, all procurements should be open and competitive procedures.
As a matter of principle, under the PPL, public procurement in Kazakhstan shall be conducted electronically and disclosed through the government e-procurement system (web portal: https://www.goszakup.gov.kz/). There are few exceptions, such as “special orders” (Rus. Особый порядок) procurements involving confidential information and state secrets and purchases at commodity exchanges.
The legislation prohibits procurement of goods, works and services that are absent from annual public procurement plans. Public procurement contracts are standard civil law contracts. The government e-procurement system contains different contract templates of mandatory use by contracting authorities, even though they can complement these templates with new clauses upon agreement with suppliers. Chapter 2 provides for a detailed description of the procurement cycle in Kazakhstan, from procurement planning to the contract execution phase.
The PPL along with the introduction of a new version of the government e-procurement system significantly changed several key aspects of the process and the procedures which contracting authorities need to abide by. It introduced the mandatory use of the government e-procurement system and the electronic signature of public procurement contracts through the national digital signature system (Rus. Электронная цифровая подпись). The PPL provided a legal ground for several new functionalities such as the electronic submission of complaints (available since January 2018) or remote “desk control” of procurement transactions by state auditors. Chapter 4 elaborates on complaints, audits and risk management processes while Chapter 3 provides further details on the functionalities of the government e-procurement system and the transparency it brought to public procurement transactions.
copy the linklink copied!1.2. Governance of the Public Procurement Function
The procurement system in Kazakhstan can be considered decentralized with different government organisations and state-owned enterprises managing specific procurement projects. Over 24 201 distinct contracting authorities are registered on the government e-procurement system. There is no centralised procurement body that is conducting procurement on behalf of contracting authorities, whether that is by aggregation or through framework agreements. Hence, procurement is still the responsibility of each contracting authority.
Beyond the GPC, the key management bodies regarding public procurement matters are the departments of public procurement legislation, and the Internal Audit Committee within the Ministry of Finance, the Centre of Electronic Commerce and the other single organisers at the local level (see Figure 1.1). This section focusses on three important institutions in the public procurement framework in Kazakhstan: 1) the Ministry of Finance, 2) single organisers, and 3) government organisations and state-owned enterprises.
1.2.1. Role of the Ministry of Finance
The Ministry of Finance sets the policy in the field of public procurement. The Ministry of Finance is the body that carries out methodological functions for central government organisations and subnational governments, and acts as the body responsible for the functioning of the public procurement web portal. At the same time, the Internal Audit Committee of the Ministry of Finance acts as a supervisory authority, and the GPC of the Ministry of Finance acts as a single organiser for central government entities and organisations. These Committees are departments of the Ministry of Finance. Any activities related to budget planning, as well as the development of planning rules, are implemented by the Ministry of National Economy.
Leaving aside state auditors (Internal Audit Committee), around 60 people work in the sphere of public procurement in the Ministry of Finance. Most of them are specialists who directly conduct public procurement in the GPC. A separate department, the Department of Public Procurement Law, carries out the ministry’s methodological functions, including developing and drafting amendments to the public procurement legislation and rules and providing clarification on the legal provisions in this field. It employs around 15 people. The entire legislative base is published open to general access on the web portal for public procurement and official websites of the Government and Parliament. In addition, the Ministry of Finance regularly provides briefings, round tables and seminars for potential suppliers, the media and the public.
1.2.2. Defining government organisations and state-owned enterprises
As mentioned above, public procurement can be conducted by all “government organisations” (Rus. Государственные органы) in Kazakhstan. Government organisations at the central level are typically ministries and their subordinated committees.
In accordance with paragraph 2 of Article 1 of the Law "On Administrative Procedures", government organisations are state institutions authorized by the Constitution, laws, other regulatory legal acts for the performance on behalf of the state of functions for:
-
1. Issuance of acts that determine the generally binding rules of conduct;
-
2. Management and regulation of socially significant public relations;
-
3. Monitoring compliance with the generally enforceable rules of conduct established by the state.
On the other hand, a state-owned enterprise (Rus. Государственные предприятия) is a commercial organisation that is endowed with state property. It is a legal entity, with 50% and more of shares (participatory interests) belonging to the state or to another state-owned legal entity. The activities of state-owned enterprises are strictly delineated by the Law on State Property (Articles 133 and 134) (Republic of Kazakhstan, 2011[3])
The main differences between government organisations and state-owned enterprises and other commercial legal entities are the specific powers and type of activity of government organisations. State-owned enterprises have a variety of legal status, and they all conduct procurement within the framework of the PPL, if they do not have the status of “national company”, as defined by the Government of Kazakhstan. Many state-owned enterprises in Kazakhstan have central level government organisations as their sole shareholder and would have the status of state agency or state administration in OECD countries.
1.2.3. Role of single organisers
Kazakhstan’s government has been establishing single organisers within ministries across the government since 2014, including the GPC at the central level. The overall purpose of this institutional change has been to increase efficiency and effectiveness of public procurement through greater centralisation and concentration of procurement. In addition, the government hopes to reduce corruption and increase transparency. Single organisers manage part of the procurement process on behalf of contracting authorities concerning certain purchases. Kazakhstan has registered more than 40 single organisers of public procurement at the central, regional (oblast) and district levels. At the regional level, 14 regional (oblast) Akimats as well as Astana, Almaty and Shymkent cities have created single organisers. Some districts and larger cities established single organisers within their local administration or are encouraged to do so. In general, this figure will grow as the Government is currently setting up “central procurement bodies” (on the basis of existing single organisers) in all districts and cities of the country.
The rationale for establishing single organizers was to professionalize public procurement, to ensure the proper implementation of the public procurement rules and regulations and at the same time ensure greater transparency and efficiency. Indeed, the decentralized nature of Kazakhstan’s procurement framework means that purchases outside of single organisers are often not conducted by public procurement experts, particularly in local public administrations. In small or medium-size government organisations, such as public hospitals, medical doctors or accountants often conduct purchases in addition to their regular work. The government also sees single organisers as a way to prevent corruption in public procurement. The reason is that when a single organiser is involved, tender commissions consist of both professional procurers from the single organiser and representatives of the contracting authority, which provides for “mutual control” and decreases the risk of undue influence on the procurement process.
Since its creation in 2013, the GPC has been functioning as the single organiser of public procurement for central government organisations and agencies, but with a rather limited scope. The employees of the Government Procurement Committee number approximately 40 people and their duties include assisting contracting authorities with the drafting of tender documents and technical specifications, the conduct of public procurement procedures and the selection of suppliers on the government e-procurement system. The service provided by the GPC is free of charge for contracting authorities; GPC operations are entirely funded from the Central State Budget.
Compared to central purchasing bodies in other countries, the GPC has a limited role, focusing mostly on reviewing tender documents. No real aggregation was carried until now. However, the number of purchases it carries out is increasing every year. It carried out approximately 211 procedures in 2015, 2011 procedures in 2016 and 2099 procedures in 2017, i.e. less than 1% of all public procurement procedures during that year. According to the Ministry of Finance, procedures processed through the GPC accounted for 11.6% of overall central-level procurement value in 2017. Purchases with the highest monetary value relate to construction works. On the other hand, motor vehicles are the most frequent item procured through the GPC. As mentioned before, the GPC carries out procurement on behalf of central Ministries and agencies, as well as of local Akimats for goods, works and services strictly defined by a decree of the Minister of Finance (currently Decree N° 1127 from 29 December 2018).
According to this decree, the GPC automatically carries out procurement for three items purchased by any public body, agency and state-owned enterprises covered by the public procurement framework:
-
Motor vehicles
-
Civilian helicopters.
-
Construction works regarding buildings procured by central-level government organisations and agencies.
Otherwise, for instances concerning furniture, software, computers, repair works or engineering services (technical supervision and project management services) or the preparation of construction submittals, any procurement must be processed through the GPC as soon as the planned procurement value exceeds a certain threshold. The decree provides for a different threshold for each one of the 13 items for which there is such a threshold. For instance, the GPC is the single organiser for any purchase of furniture exceeding 5 000 monthly calculated indices (i.e. around EUR 31 000). Regarding computers, notebooks, scanners and printers, the GPC is the single organiser for any purchase exceeding 20 000 Monthly calculated indices (i.e. around EUR 124 000). Using a similar system of value thresholds, the decree defines goods, services and works to be procured through regional and local single organisers by local administrations (Akimats) and government organisations in the regions. Beyond items to be procured through the GPC according to the decree, a contracting authority can define the GPC as procurement organizer for any given procurement process upon prior approval by the GPC itself (Clause 26 of the Rules of the Conduct of Public Procurement).
Thus, for each single request from a contracting authority, single organisers currently set up a distinct tender commission and conduct procurement processes in the government e-procurement system separately. According to the Ministry of Finance, the GPC has been consolidating purchases only regarding budget investment projects (Rus. Бюджетные инвестиционные проекты), which account for a very small share of public procurement processes in Kazakhstan. Budget investment projects are public capital expenditures, for instance on transport infrastructure (roads, railway network) or in special economic zones. The central administrations, i.e. the ministries, still have to carry out the market analysis and develop the technical specifications prior to sending it to the GPC for review. OECD fact-finding missions revealed that the creation of the GPC has sometimes delayed procurement processes for up to two months, while the GPC is being perceived as an additional intermediary with little value added by some contracting authorities.
Partly because of this lack of aggregation, single organisers have had a limited impact on the conduct of public procurement in Kazakhstan. Their role is currently limited to the ex-ante supervision of tender documents (ensuring that technical specifications drafted by contracting authorities comply with public procurement rules and regulations) and to the bid submission and award stages of the procurement cycle, up to the signature of the procurement contract. However, the December 2018 amendments reforming the public procurement framework allow the GPC and other single organisers to consolidate purchases from different contracting authorities.
1.2.4. Strategic vision for public procurement
The strategic vision for public procurement in Kazakhstan is laid out in eight principles in the PPL (article 4):
-
1. Supporting domestic suppliers (Rus. оказания поддержки отечественным производителям товаров).
-
2. Openness and transparency (Rus. открытости и прозрачности процесса государственных закупок).
-
3. Fair competition (Rus. добросовестной конкуренции среди потенциальных поставщиков).
-
4. Equal opportunities to suppliers (Rus. предоставления потенциальным поставщикам равных возможностей для участия в процедуре проведения государственных закупок).
-
5. Spending money efficiently (Rus. оптимального и эффективного расходования денег, используемых для государственных закупок).
-
6. Prevention of corruption (Rus. недопущения коррупционных проявлений).
-
7. Procurement of technologically advanced products (Rus. приобретения инновационных и высокотехнологичных товаров, работ, услуг).
-
8. Responsible participants (Rus. соблюдения прав на объекты интеллектуальной собственности, содержащиеся в закупаемых товарах).
Indeed, the OECD Recommendation for Public Procurement also emphasise most of these principles. Other aspects, such as the dedicated support for domestic suppliers might harbour potential to limiting competition if such a support is structured in a way that it would limit access for foreign suppliers, for example. Figure 1.2 below summarises all principles of the OECD Recommendation.
Through the principles outlined in the PPL, the foundations for modernising the procurement system in Kazakhstan are in place. The challenge for the Government of Kazakhstan, especially the Ministry of Finance, will be to identify ways to ensure that these principles are not only detailed in the law, but being adhered to in practice. This is a long-term project, and to date there are plenty of opportunities in place for authorities to introduce new methods that will improve the spending of money used for public procurement in the most optimal and efficient manner. In the following chapters, these principles will be discussed and analysed in depth, such as provision of equal opportunities to potential suppliers to participate in the public procurement procedure (chapter 2), openness and transparency of the public procurement process (chapter 3) and prevention of corruption (chapter 4).
In Kazakhstan, there is no single document summarising the government’s strategic vision for the public procurement framework over the long term. However, government organisations adopt a new strategic plan every five years, which summarises their core missions and its strategic goals, and details their key performance indicators. Since the Ministry of Finance is in charge of the public procurement framework (central methodological functions), its strategic plan includes public procurement among many other policy areas (Ministry of Finance, 2018[4]). Moreover, Vice-Minister Beketaev gave a speech to the Parliament of Kazakhstan in early 2018 about the public procurement reform (Beketaev, 2018[5]). Based on these two documents, it is possible to define four overarching objectives of the government of the Republic of Kazakhstan regarding the public procurement framework:
-
Increasing competition, by reducing the share of public procurement processes conducted through direct award, i.e. without competitive tendering.
-
Increasing centralisation of public procurement, by expanding the list of goods, works and services purchased through single organisers.
-
Improving the functioning of the government e-procurement system, which should materialise through an increase of the share of satisfied users.
-
Simplifying public procurement processes and enhance their efficiency.
copy the linklink copied!1.3. Revising the Public Procurement Law
On 26 December 2018, the Parliament of Kazakhstan adopted legislative amendments that constitute a substantial reform of the public procurement system (Law from 26 December 2018 № 202-VІ ЗРК, the December 2018 amendments). Accordingly, it is important to consider the ramification of these changes to the public procurement framework. The Ministry of Finance organised public debates with the participation of civil society to discuss the main policy measures of these amendments. Meetings to engage stakeholders have taken place in the cities of Almaty, Astana and Uralsk with the active participation of the business community and local authorities (Beketaev, 2018[5]).
These legislative changes are part of the implementation of measures announced in a Presidential Address to the People of Kazakhstan dated 31st January 2017 (Ministry of Finance, 2018[4]).The need to improve public procurement was one of the main issues mentioned in the address. For that purpose, the government set out a plan to address two challenges:
-
Further centralising public procurement through the introduction of “central procurement bodies”.
-
Reshuffling the procurement function and procedures in the quasi-state sector (i.e., national holdings and other SOEs).
Beyond these two priorities, the December 2018 amendments contain many other revisions to the public procurement framework and of the framework for procurement by national holdings and national companies (“quasi state sector”). This section introduces some of the most important ones.
1.3.1. Centralisation of public procurement
The December 2018 amendments provide for the strengthening and expansion of single organisers, including the GPC of the Ministry of Finance and procurement departments of regional Akimats (local administrations) and those of the cities of Astana, Almaty and Shymkent. At the level of district and other cities, they provide for the creation of procurement units of local district Akimats, if they do not already exist. These reforms provide the basis for a pilot exercise in centralisation that the Ministry of Finance will undertake with the other levels of government in 2020. This exercise is expected to provide further insight into beset avenues for implementing increased centralisation of public procurement in Kazakhstan.
The December 2018 amendments allow single organisers to consolidate procurement of homogeneous goods, works and services from different contracting authorities under their auspices. This is the case even if there are several different places of delivery (or of execution of construction or engineering works). As mentioned earlier, the Ministry of Finance defines the list of homogeneous goods, works and services to be procured by single organisers. The consolidation of purchases from different contracting authorities by single organisers was very limited in Kazakhstan. Therefore, if adequately implemented this reform has the potential to improve value for money and efficiency; it can also be considered a fundamental pillar of the ongoing centralisation of public procurement.
The December 2018 amendments also empower the Ministry of Finance to determine which procurement methods to apply for specific goods, services or works. Previously, contracting authorities chose the procurement method freely within the limits provided for in the public procurement legal framework. There is no default method like in other countries. According to the Ministry of Finance, this additional prerogative will help rationalise public procurement, accompanying the centralisation of purchases while bringing in increased efficiency. In February 2019, the Ministry of Finance defined a list of works and goods (including furniture, construction and installation works, preparation of construction submittals and design documents, software and electronic products.) for which public tender with prequalification is the mandatory procurement method (Ministry of Finance of the Republic of Kazakhstan, 2019[6]).
Last but not least, as part of the centralisation of public procurement, the Ministry of Finance expects to expand the list of goods, services and works to be procured by single organisers at the different territorial levels on the basis of an analysis of past procurements. Consequently, the authority of the territorial and national central procurement entities will be clearly delineated to avoid overlap. (Beketaev, 2018[5]) Furthermore, according to stakeholders, the EAEU is currently in the process of updating its provisions regarding framework agreements. Once accepted, these changes would introduce the possibility to conduct public procurement via framework agreements. This promises progress towards greater value for money.
1.3.2. Procurement of national holdings and national companies (“quasi state sector”)
The issue of regulating procurements of the national holdings and national companies (part of the so-called “quasi-state sector”) and introducing administrative liability for violations of procurement rules and regulations by procurement officers in the quasi-state sector has been raised several times by MPs. To implement instructions from the President, the December 2018 amendments comprised the following measures:
-
Introduced administrative liability of quasi-state (or parastatal) sector employees3 for violations of procurement rules and regulations.
-
Transferred all procurement of the quasi-state sector to an e-procurement electronic format.
-
Approved unified rules of procurement for the national holdings and national companies, excluding the Samruk-Kazyna Fund, whose rules will be approved by a decision of the board of directors with the formal agreement of the Ministry of Finance. This exception is related to the fact that some of the Fund’s companies are preparing for initial public offerings (IPO) to attract investors.
According to the Ministry of Finance, unified procurement rules applicable to national holdings and national companies, along with the new procurement rules of the Samruk-Kazyna Fund, will harmonise to the maximal extent possible procurement regulations and practices of national holdings and companies. Chapter 6 elaborates on the procurement of these national holdings and companies.
1.3.3. Exceptions to competitive tendering
The December 2018 amendments modified the scope of direct awards. There are two types of exceptions to competitive tendering:
-
a. direct award based on a list of exceptions (PPL, Article 39 point 3).
-
b. direct award after failed bidding.
Previously, the PPL included 54 exceptions providing for direct award based on exceptions, i.e. direct purchasing without advertising. After the December 2018 amendments, it now includes 50. Aside from abolishing some, the amendments added a new exception and expanded the scope of at least two exceptions. In terms of procurement value, the most significant exception now abolished concerned procurements based on proposal by the President.
The new exception introduced in 2018 concerns “goods, works and services earmarked for specialised law enforcement units (riot police), Special Forces, antiterrorist units and bomb disposal units”. According to the Ministry of Finance, the rationale for the addition resides in the specificity of the purchased goods for special units; lack of suppliers in the domestic market; the goods for special units are manufactured abroad using special technologies and the purchase of goods for special units takes a long time (export, licensing and customs procedures).
OECD standards on exceptions from competitive tendering call for a more thorough analysis by the Ministry of Finance before expanding the list exceptions allowed in Kazakhstan. Indeed, it appears that the new exception aims at circumventing obstacles created by the e-procurement system regarding the access of foreign suppliers from accessing public procurement opportunities. The exception provides a way to procure items from foreign suppliers through direct award based on exceptions, rather than through public tenders that frequently fail due to a lack of responsive bids from the domestic supply market. Easing the access of foreign suppliers to the government e-procurement system would be a more efficient way to solve the problem while preserving an adequate level of competition and bringing Kazakhstan in line with the OECD Recommendation on access to public procurement opportunities (see Chapter 2).
The December 2018 amendments also exonerate any procurement of works, services and goods as part of projects funded by international organisations (whose membership includes Kazakhstan) from complying with the PPL. However, the exemption extends to investment projects funded entirely or partly by “other international banks”, if they fund at least 50% of the investment project and have a credit rating at least equal or equivalent to A- (as defined by Standard & Poor’s). The exemption also requires the project to be executed by State owned companies or their affiliates, and the project should not involve a sovereign (state) guarantee on behalf of the Republic of Kazakhstan. Under these conditions, procurement regulations from these “foreign banks” would apply to all acquisitions in the framework of the aforementioned investment projects.
1.3.4. Trademarks, brands and company names in technical documentation
The December 2018 amendments allow contracting authorities to include trademarks and brand names of goods that are widely available in the market in technical specifications. However, they establish a maximal threshold for the use of trademarks and brand names: their use is authorised only for requests for quotation with a procurement value under 1 000 monthly calculated indices (KZT 2.5 million or around EUR 6 209). Above 1 000 monthly calculated indices, contracting authorities can use requests for quotation, but without mentioning trademarks and brand names of goods in technical specifications.
The rationale of this measure is to support contracting authorities in purchasing goods of higher quality. Previously, according to the Ministry of Finance, goods requested through requests for quotes without reference to trademarks and brand names were frequently counterfeited goods or items of low quality. They formally complied with technical specifications, but their longevity was very short.
Allowing the inclusion of trademarks or brand names is not a common practice in OECD or EU member countries. For instance, in the EU, technical specifications should in principle not refer to trade marks, patents, types or a specific origin or production with the effect of favouring or eliminating certain products ( (European Union, 2014[7]).
However, in Kazakhstan this possibility is limited to purchases below a threshold that corresponds to a purchase value that would be commonly handled as direct award for small purchases (below a defined value threshold) in most OECD countries. This raises the issue of the value threshold for direct awards being possibly too low in Kazakhstan. Regulations set this threshold at 100 monthly calculated indices for goods (KZT 252 500 or around EUR 620). Most OECD countries allow direct awards of procurement contracts (without advertisement) up to a certain threshold. For instance, in the Netherlands direct award is authorised for contracts below EUR 33 000, provided that certain conditions are respected. In Korea contracts below EUR 15 665 (KRW 20 million) can be awarded directly without competition. In Canada, the unified set of rules for Federal public procurement allows a common exception to competition for contracts with a value of less than EUR 16 623 (CAD 25 000) (OECD, 2019[8]).
Similar challenges relate selection and award criteria: in the current framework, requests for quotations often lead to the acquisition of lower quality goods because price is the predominant or the only award criteria. Therefore, setting up selection and award criteria taking into account quality concerning requests for quotation might be a way to tackle this issue. Chapter 2 elaborates on the issues of selection and award criteria and on direct award below the maximal value threshold.
1.3.5. Bid bonds requirements
The December 2018 amendments strengthen the bid bonds requirements in various ways. Article 25 of the PPL establish bid bond requirements in public tenders and auctions at a rate of 1% of the planned monetary amount allocated for the purchase.
The December 2018 amendments create bid bonds requirements for requests for quotations. The amount of the bid bond is the same as for public tenders and auctions (1%). Moreover, the e-procurement system will automatically exclude bids where the full bond amount has not been paid. Indeed, the bid bond will have to be paid through an electronic wallet on the e-procurement system. Currently, it is up to the tender commission to exclude bids without bid bonds, i.e. this measure would save some time for tender commission members. Both measures will enter into force in January 2020 with 2019 as a transition year.
The December 2018 amendments also restrict the rights of bidders that did not fully pay the bid bonds in various ways: for instance, they will not be entitled to access application documents from other bidders or to correct their bids after receiving remarks from tender commissions (see Chapters 3 and 2 for details on the public tender submission process).
All these measures aim at preventing dubious bidders using fictions companies from influencing the tendering process. Spurious bidders are an important issue in public tendering in Kazakhstan, as detailed in Chapter 3 of this report.
1.3.6. Increasing the threshold for direct award procurement
As in most OECD countries, in Kazakhstan, contracting authorities can purchase goods, works and services below a defined value threshold through direct award (i.e. direct purchasing). This is one of the 50 exceptions to competitive tendering in the PPL (Article 39 Point subparagraph 42 of the PPL). This value threshold is currently 500 monthly calculated indices or KZT 1 262 500 (roughly equivalent to EUR 3 100 or 7.2 average wages in Kazakhstan) for services and works, and 100 monthly calculated indices for goods or KZT 252 500 (roughly equivalent to EUR 620 or 1.4 average wages in Kazakhstan).
In order to simplify procurement processes for low value purchases, the December 2018 amendments increased fivefold the threshold for services and works: it was previously equivalent to the one for goods (100 monthly calculated indices). For procurement by administrations in villages, districts and small towns, headed by Akims (heads of local administrations) and their offices, the amendments also set a much higher maximal threshold for direct award, i.e. 3 000 monthly indices (KZT 7 575 000, or around EUR 18 627 or 43 average monthly wages). However, this higher threshold would be in force temporarily, up until the end of 2020. Chapter 2 elaborates on the issue of direct award procurement, including direct award under the maximal value threshold (section 2.3.4).
copy the linklink copied!1.4. Towards more concentrated and centralised procurement in Kazakhstan
Increasing concentration and centralisation of procurement in Kazakhstan has been a trend in recent years, and continues to be one of the focus areas of the current reform. For 2020, the Ministry of Finance plans on launching a first pilot exercise in centralisation, in collaboration with the different levels of government. Indeed, Kazakhstan could achieve substantial benefits from further centralising public procurement institutionally and at the same time using procurement tools that aggregate demand across institutions or the entire administration. Given the current relevance of these issues, the associated complexities as well as challenges in implementing it, this section takes a closer look at benefits and approaches to concentrating and centralising public procurement in Kazakhstan’s context.
1.4.1. Centralised procurement offers numerous benefits to countries
Kazakhstan embarked on some centralisation efforts (mostly institutionally) recently, by establishing the GPC. The initial purpose was to mitigate against corruption. As it stands, the GPC is not fully functioning as a fully-fledged centralised procurement body (CPB) in the way that CPBs operate in most OECD countries. However, the potential of a unit like GPC reaches beyond anti-corruption functions towards increasing the effectiveness and efficiency of public procurement. This is especially the case if GPC should begin consolidating purchasing requests of public entities. On the positive side, data from the Ministry of Finance show that the average value of a procurement process increased from KZT 539 300 in 2016 (around EUR 1 320) to KZT 980 771 (around EUR 2 414) in 2018, which testifies to a higher centralisation of public procurement.
Several tools have proven useful in centralising and aggregating public procurement, as evidenced by OECD country experience. In order to reap the benefits of the consolidation of purchases, the Ministry of Finance could allow for more centralized procurement of commonly procured products in demand in all types of government organisations, such as food, office supplies, IT equipment and fuel (Figure 1.3). This approach is quite common in other OECD countries and likely to lead to more efficient spending of public funds (SIGMA, 2016[9]).
As demonstrated by Figure 1.4, aggregating demand and awarding framework agreements are among the core functions of CPBs in most OECD Countries.
Centralised procurement bodies are common in OECD countries. Indeed, the vast majority of OECD countries operate have at least one CPB to conduct central purchasing – according to data from the OECD Public Procurement Surveys (OECD, 2016[10]), only two countries in the OECD do not have CPBs. Contracting authorities, suppliers and the CPB owner (usually the central or subnational governments) are key stakeholders in their operations (Box 1.1).
The rationale for establishing and operating a centralised purchasing system (CPS) needs to be examined from various perspectives.
The important stakeholders are:
-
the users of the CPB’s services - the contracting authorities, representing the wide spectrum of procuring entities that either purchase goods or services acquired by the CPB or use the framework agreements operated by the CPB;
-
the private sector market, represented by the suppliers providing the goods or services to the CPB, either directly or under framework agreements;
-
the owners of the CPBs – usually ministries, associations of local authorities, and other public bodies representing taxpayers’ interests, which may recognise that the CPB contributes to reduced public expenditure, increased value-for-money, and the realisation of certain important policy goals, such as those related to environmental or social issues or to SMEs.
The efficiency and attractiveness of the CPB can only be measured in terms of how well it satisfies the needs of these external stakeholders.
The main rationale for establishing a CPB is often described in the following terms:
-
Large procurement volumes generate better prices
-
Transaction costs are reduced
Centralised purchasing systems may also offer advantages that cannot be directly expressed in economic terms, the following arguments in favour of centralised purchasing arrangements should be mentioned:
-
The need for standardisation or increased administrative efficiency within the public administration,
-
Many procuring entities may lack sufficient capacity of their own to prepare and carry out complex tenders in areas requiring specific product or market expertise.
-
Professional, centralised purchasing provides certainty to procuring entities in many key aspects legal, technical, economic and contractual
-
Simplicity in the acquisition of goods and services
-
Governments may use the CPBs as instruments for the execution of policy goals in specific sectors, such as promoting green procurement, innovations and SME participation in public sector tenders.
Source: (OECD, 2011[11]).
The geographical size of the Republic of Kazakhstan and the number of inhabitants allows centralised purchasing bodies to be set up at a regional level. With the introduction of single organisers in the country, the authorities have already moved towards a common practice of setting up central purchasing units within both central and subnational levels of government to serve internal departments with procurement services. In most OECD countries, such central purchasing units manage the award of framework agreements (Rus. рамочные соглашения) (OECD, 2016[10]), which do not exist in Kazakhstan. Section 1.4.2 elaborates on the issue of framework agreements and their potential benefits.
The development of standard technical specifications for popular and homogeneous goods and services is a powerful tool that goes along with the centralisation and standardisation of public procurement. As of today, standard technical specifications are available only regarding furniture. The development of standard technical specifications for other goods and services is one of the keys to aggregated purchases since they will make it easier for central purchasing bodies (former single organisers) to aggregate similar lots from different contracting authorities. The Ministry of Finance plans to develop a catalogue of standard technical specifications in 2019. This is a positive development that will enhance the efficiency of public procurement processes, and support increased centralised purchasing by single organisers. The development of standard technical specifications could focus on homogeneous goods and services in demand in all types of government organisations.
Expanding the remit of the GPC and other central purchasing bodies, particularly through the consolidation of purchases, would create negotiating leverage for the state in the market and increase efficiency and productivity by decreasing the number of public procurement procedures. However, this will change the role of GPC and single organisers and will require them to enhance their procurement competency in relation to market and supplier studies.
Contracting authorities will appreciate the value added of bodies like the GPC and other central purchasing bodies once tangible results become visible. Among those results, a lower administrative burden of contracting authorities and better value for money might be particularly relevant. In turn, this increased awareness of the benefits of centralised procurement could create additional demand and increase the procurement conducted by centralised purchasing bodies as contracting authorities would be increasingly incentivised to conduct procurement through the GPC (and the other central purchasing bodies) even concerning non-mandatory items (i.e. items for which they are entitled to could procurement themselves).
Beyond strengthening centralised purchasing bodies, the Ministry of Finance could consider allowing collaborative purchasing arrangements between the contracting authorities or single organisers of different districts belonging to the same region (oblast), for instance. Such arrangements are common in a number of OECD Countries These arrangements may be established formally and permanently, but the government organisations involved may also have a looser relationship in matters of purchasing.
1.4.2. Introducing framework agreements
As mentioned above, the PPL does not provide contracting authorities with an opportunity to use framework agreements. Introducing this kind of possibility in the law, and supporting the implementation of framework agreements promises great benefits in terms of value for money.
Framework agreements generally involve the advertisement of an opportunity by a contracting authority (or authorities). This authority then enters into a contract or other arrangements with one or more economic operators for the provision of works, supplies or services to different contracting authorities over a fixed period. The rationale behind the framework method of purchasing is to achieve savings – both in terms of the cost of procurement and the time spent on the procurement process. Commonly, the most significant savings in procurement are achieved when framework agreements are combined with centralised procurement and e-procurement (OECD, 2011[12]). Framework agreements have a number of other benefits for stakeholders: Box 1.2 summarises them.
Framework agreements:
-
Deliver value for money: Aggregating different purchasers’ potential needs means individual purchasers can buy goods and services at prices below those normally charged. It also means that individuals can buy goods with special added benefits or more advantageous conditions.
-
Are compliant: Centrally initiated framework arrangements are fully compliant with procurement regulations. Thus, they reduce the potential for fraud or corruption.
-
Are consistent: The use of these arrangements drives a more consistent and professional approach to procurement processes. This consistency and professionalism helps educate officials and suppliers dealing with the central procurement body, as well as improve their approaches to each tender exercise.
-
Are faster: Framework agreements remove the need for contracting authorities to conduct full tender exercises or lengthy supplier evaluations. This saves time, and reduces the costs associated with procurement exercises.
-
Have less risk: Each of the suppliers in a framework agreement has been subject to a rigorous procurement process. This ensures that they offer the type of goods and services required, and that they meet a certain threshold of quality. Framework agreements establish terms and conditions. This fact means that there is no need to re-draft terms or renegotiate when each procurement is undertaken.
-
Leverage scale: The public sector has substantial purchasing power. Framework agreements harness the opportunity to leverage this purchasing power and thereby ensure that value for taxpayer money is achieved.
Source: (OECD, 2018[13]).
Kazakstan does not currentlyhave any framework agreements in place and currently they are not envisaged to be incorporated in the public procurement legislation. As for the national reform in Kazakhstan, the December 2018 amendments mention public tenders with pre-qualification, which the Government plans to use extensively regarding construction works. This amendment creates the status of pre-qualified supplier. A single qualification body will be created to pre-qualify suppliers, which would then be able to apply to tenders with pre-qualification. In February 2019, the Ministry of Finance defined a list of works and goods for which public tender with prequalification is the mandatory procurement method (Ministry of Finance of the Republic of Kazakhstan, 2019[6]).
The pre-qualification could act as a stepping stone towards framework agreements as suppliers who are awarded framework arrangements are sometimes termed 'approved suppliers' or 'preferred suppliers' as they have been subject to a supplier appraisal process and therefore considered to offer value for money from a relatively secure source. The expression 'qualified list' of suppliers is also used when referring to suppliers who have been awarded framework arrangements (OECD, 2011[12]).
However, the instrument is widespread among OECD countries. 97% of responding countries reported using framework agreements in 2012, according to the 2012 OECD Survey on Public Procurement (OECD, 2016[10]). Ireland is one of many European countries that have a long tradition of using framework arrangements. In Ireland, the Office of Government Procurement (OGP) has put in place 122 frameworks agreements. These agreements have delivered a total estimated value of EUR 2.5 billion. 89% of frameworks in Ireland were established with a mix of both quality and cost as award criteria. Of those, the median split was 60/40 in favour of quality. (OECD, 2018[13])
The range of items that are usually covered by framework agreements includes:
-
1. ICT (information and communication technology) products and services (computers, photocopiers, printers, servers, software), generally the largest product area in terms of purchasing volume;
-
2. Telecommunications products (networks, mobile phones, landline phones, telephone exchanges);
-
3. Office furniture;
-
4. Travel services;
-
5. Office equipment and supplies;
-
6. Vehicle and transport services;
-
7. Fuel (for heating and transport) and electricity;
-
8. Food (foodstuffs, meal tickets);
-
9. Organisational and human resources development services (SIGMA, 2016[9]).
In Latin America, framework agreements are becoming popular in countries like Mexico, Peru, Chile and Colombia. Colombia Compra Eficiente, the central purchasing body of Colombia, has deployed 34 framework agreements since 2013, when the first one was set up. Its combination of framework agreements and advanced e-procurement tools offers an example of good practice that is particularly relevant for Kazakhstan (Box 1.3).
The framework agreement was a breakthrough for government agencies in Colombia. It helped public agencies in buying fuel at service stations in Bogota. Prior to the agreement, it took four months to complete an open tender. With the agreement, procurement now takes less than an hour, as officials can place purchase orders via the Colombian State Online Store (Tienda Virtual del Estado Colombiano), an e-procurement platform within Colombia’s SECOP suite of e-procurement systems. In addition, after the implementation of the framework agreement, suppliers offered a discount on the reference price for fuel in a greater number of service stations. Furthermore, the government created a management system for fuel supply to increase control over fuel expenditure. Colombia Compra Eficiente, the central purchasing body of Colombia, currently offers public agencies 34 frameworks agreements, including insurance, vehicles, janitorial services and cloud services, among others. Law mandates that central government agencies procure through framework agreements. Other institutions, including Congress, the judiciary system, oversight institutions and institutions at the subnational level, may use the agreements if they deem them useful. The total value of purchase orders placed is more than EUR 1.25 billion, and there is a growing trend for subnational-level entities to use these contracts.
Source: (OECD, 2018[13]).
Framework agreements in Ireland are aggregated procurement arrangements established through competitive procurement processes. These arrangements are agreements with suppliers or service providers. They set out the terms and conditions under which specific contracts can be made during the period of the agreement. Frameworks agreements are put in place by central purchasing bodies like the OGP. Public sector clients can then modify the amount of goods and services they purchase from suppliers in the framework by means of direct draw down or mini-competitions (OECD, 2018[13]).
The vast majority of OECD countries that use framework agreements distinguish between contracting authorities at central and sub central level. The majority of OECD countries make it mandatory for contracting authorities at the central level to use framework agreements for available categories. Most often, other institutions such as subnational governments are free to join on a voluntary basis. Only Korea and the Slovak Republic make it mandatory for all contracting authorities on all government levels to use framework agreements (Figure 1.5).
Furthermore, the European Commission has stressed the benefits of making the use of existing framework agreements mostly mandatory (PWC & EU Commission, 2017[14]). However, the question of whether framework agreements should be mandatory or not also depends on a country’s capacity to implement different models. The voluntary use of framework agreements reinforces the need to develop attractive value propositions appealing to contracting authorities and suppliers. A mandatory scheme provides for greater certainty and potential additional spill over effects on the entire administration.
The benefits of framework agreements have already been established. However, in introducing them, the Government of Kazakhstan should recognise that framework agreements might not be suitable for all types of purchasing. The most appropriate use of a framework agreement is in a situation where a contracting authority has a repeated need for works, services or supplies, but does not know the exact quantities that are required. In order to assess the suitability of a framework agreement, contracting authorities need to understand the advantages and disadvantages of framework agreements, the different types of framework agreements, how they are set up and how they operate in practice.
The use of framework agreements (or the lack thereof) follows the procurement rules established by the Eurasian Economic Union (EAEU). In fact, the EAEU rules currently do not allow for procurements to be conducted using framework agreements. According to the Ministry of Finance, an amendment to the union’s procurement rules was drafted and is currently pending adoption by the member countries. Once approved, this amendment would establish the EAEU’s members’ ability to introduce framework agreements.
The public procurement system in Kazakhstan has already undergone many revisions in recent years, the last one being the December 2018 amendments.
The overall assessment of the institutional setting and procurement structures in Kazakhstan does suggest that the country is moving in the right direction with opportunities to make the system both more streamlined and efficient while achieving greater value for money. To further support the modernization of the system the Government of Kazakhstan could consider the following proposals:
-
The Ministry of Finance could set a long-term vision and develop a plan based on that vision and including specific goals and actions. The Ministry could then then report progress on them annually.
-
There is a need to expand the remit of the GPC and other central purchasing bodies to allow it to function as a genuine centralised procurement body to realise benefits of central purchasing. This would be achieved by introducing an ambitious plan for the consolidation of purchases from different Contracting authorities, particularly concerning commonly procured products in demand in all government organisations. Such ambitious plan could encompass the preparation of standard technical specifications for these products.
-
To complement its efforts towards greater centralisation of purchases, the Ministry of Finance could consider introducing framework agreements, a tool used to centralise certain common categories of goods and services while providing flexibility to contracting authorities to focus on their own primary and more specific procurement needs.
-
The Ministry of Finance would want to ensure that the introduction of single organisers at all local levels of government do not lead excessive delays in carrying out procurement procedures. The analysis of the impact of these actions could be assessed as part of the centralisation pilot that the Ministry of Finance is undertaking in 2020.
-
The Ministry of Finance could balance the need for reform with the drawbacks of frequent changes to the public procurement legal framework: procurement practitioners and potential bidders require time to familiarise themselves with new procedures and rules. This can result in a loss of efficiency and should be considered before attempting reforms that aim at increasing the efficiency and effectiveness of the public procurement system.
References
[5] Beketaev, V. (2018), Adress to Parliament regarding the draft amendments to the Procurement Law (25 January 2018, in Russian), https://www.zakon.kz/4900564-vystuplenie-vitse-ministra-finansov.html.
[2] Eurasian Economic Union (2015), Treaty of the Eurasian economic Union (Section 22 “On government (municipal) procurement” and appendix 25), as updated, https://docs.eaeunion.org/docs/en-us/0003610/itia_05062014.
[7] European Union (2014), Directive 2014/24/EU of the European Parliament and the Council, https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32014L0024&from=EN (accessed on 13 February 2019).
[4] Ministry of Finance (2018), Strategic Plan of the Ministry of Finance for 2017-2021, http://www.minfin.gov.kz/irj/portal/anonymous?NavigationTarget=ROLES://portal_content/mf/kz.ecc.roles/kz.ecc.anonymous/kz.ecc.anonymous/kz.ecc.anonym_activities/activities/plans.
[6] Ministry of Finance of the Republic of Kazakhstan (2019), Decree N° 154 from 28 February 2019 on the mandatory procurement method for certain goods, works and services (in Russian), http://adilet.zan.kz/rus/docs/V1900018364.
[1] OECD (2019), Government at a Glance 2019, OECD Publishing, Paris, https://doi.org/10.1787/8ccf5c38-en.
[8] OECD (2019), Reforming Public Procurement: Progress in Implementing the 2015 OECD Recommendation, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/1de41738-en.
[13] OECD (2018), Public Procurement in Nuevo León, Mexico: Promoting Efficiency through Centralisation and Professionalisation, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/9789264288225-en.
[10] OECD (2016), 2016 Public Procurement Survey (partially published), https://qdd.oecd.org/subject.aspx?Subject=GOV_PUBPRO_2016.
[11] OECD (2011), “Centralised Purchasing Systems in the European Union”, SIGMA Papers, No. No. 47, OECD Publishing, Paris, https://doi.org/10.1787/5kgkgqv703xw-en.
[12] OECD (2011), Framework Agreements, SIGMA Public Procurement Briefs, https://www.oecd-ilibrary.org/governance/sigma-public-procurement-briefs_23109246 (accessed on 10 October 2018).
[14] PWC & EU Commission (2017), Make better use of framework agreements, http://ec.europa.eu/regional_policy/en/policy/how/improving-investment/public-procurement/e-library (accessed on 9 October 2018).
[3] Republic of Kazakhstan (2011), Law No. 413-IV on State Property dated 1 March 2011, http://adilet.zan.kz/eng/docs/Z1100000413.
[9] SIGMA (2016), Central Purchasing Bodies, http://www.sigmaweb.org/publications/Public-Procurement-Policy-Brief-20-200117.pdf.
Notes
← 1. (Akimat = local public administration, headed by an Akim =Appointed Governor)
← 2. The public procurement framework also provides for tenders with pre-qualification, but no procurement has been conducted through this method yet;
← 3. These employees are employees of SOEs and of the national holding companies that own them, including Samruk-Kazyna.
Metadata, Legal and Rights
https://doi.org/10.1787/c11183ae-en
© OECD 2019
The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.