3. E-procurement to strengthen transparency and develop performance evaluation of public procurement in Kazakhstan

The chapter covers e-procurement and transparency of public procurement in Kazakhstan. While the e-procurement system generally provides adequate transparency regarding individual purchases, the quantity and the quality of aggregated procurement data are insufficient. Kazakhstan would gain from collecting and disclosing more aggregated statistics on its public procurement system and move towards an open data approach. Certain mechanisms providing information to suppliers need to be fine-tuned, such as procurement plans or the reciprocal review of suppliers’ offers. Enhanced availability of aggregated procurement data would allow Kazakhstan to measure and disclose performance indicators in a more structured and systematic manner, allowing for evidence-based assessments of the procurement function of contracting authorities, and of the public procurement system as a whole.

    

E-procurement is the integration of digital technologies to replace and redesign paper-based procedures in public procurement (OECD, 2017[1]). OECD countries have long used e-procurement systems to increase transparency and efficiency in public procurement. Regarding transparency, e-procurement systems and online platforms are essential tools to provide free access to relevant procurement information for all stakeholders, making contracting authorities more accountable to citizens. Concerning efficiency, e-procurement systems allow for automation and standardisation of procedures along the entire procurement cycle, reducing the time needed to perform tasks and the room for human error. E-procurement drives efficiency gains by facilitating market access, thereby increasing competition and decreasing administrative burden and transaction costs (EBRD, 2015[2]). The essential role of e-procurement is recognised by the 2015 OECD “Recommendation of the Council on Public Procurement” (hereafter, the “OECD Recommendation on Public Procurement”) which includes e-procurement as one of its 12 integrated principles (Box ‎3.1).

Box ‎3.1. OECD Recommendation on e-procurement

VII. RECOMMENDS that Adherents improve the public procurement system by harnessing the use of digital technologies to support appropriate e-procurement innovation throughout the procurement cycle.

To this end, Adherents should:

i) Employ recent digital technology developments that allow integrated e-procurement solutions covering the procurement cycle. Information and communication technologies should be used in public procurement to ensure transparency and access to public tenders, increasing competition, simplifying processes for contract award and management, driving cost savings, integrating public procurement, and public finance information.

ii) Pursue state-of-the-art e-procurement tools that are modular, flexible, scalable and secure in order to assure business continuity, privacy and integrity, provide fair treatment and protect sensitive data, while supplying the core capabilities and functions that allow business innovation. E-procurement tools should be simple to use and appropriate to their purpose, and consistent across procurement agencies, to the extent possible; excessively complicated systems could create implementation risks and challenges for new entrants or small and medium-sized enterprises.

Source: (OECD, 2015[3]), OECD Recommendation of the Council on Public Procurement”, www.oecd.org/gov/ethics/OECD-Recommendation-on-Public-Procurement.pdf.

Since 2016, contracting authorities in Kazakhstan have advertised and performed almost all government procurement through the government e-procurement system (called Goszakup, www.goszakup.gov.kz). It is mandatory for all public contracting authorities (i.e. Ministries and Committees, subnational governments, government agencies and some SOEs). JSC “Center for Electronic Commerce” (hereafter: Center for Electronic Commerce) under the Ministry of Finance operates and maintains the government e-procurement system. Up until the end of 2018, the government budget provided funding for the system’s operations and maintenance. However, since January 2019, supplier subscription fees have become the main source of funding for the operations and maintenance of the e-procurement system.

In addition to the government e-procurement system, national companies (i.e. SOEs with a special status) conduct procurement through distinct e-procurement systems. For instance, Samruk-Kazyna and Baiterek holdings have their own e-procurement systems. This chapter focuses on the government e-procurement system while Chapter 6 addresses procurement by national companies and national holdings, such as the Sovereign Wealth Fund JSC Samruk-Kazyna.

This chapter assesses the use of digital technologies in the procurement process in Kazakhstan against the OECD Recommendation and international good practices. This chapter will first provide an overview of the government e-procurement system, with a focus on functionalities and transparency throughout the whole procurement cycle. An adequate and timely degree of transparency is indispensable in order to ensure fair and equitable treatment of suppliers and promote competition. Transparency also relates to procurement laws, regulations and policies, as well as the choice of the procurement method.

During the last decades, a majority of OECD countries focused on building centralised systems for publishing public procurement information online. However, investment in e-procurement systems has gradually shifted away from this original purpose towards developing systems that help increase efficiency and streamline procurement procedures (OECD, 2018[4]). This chapter also assesses the status of e-procurement in Kazakhstan and possible avenues for improvement, based on OECD and international best practices. This includes better coordination through integration of the government e-procurement system with other government IT systems (such as tax, budgeting and public finance management).

E-procurement systems are also key to the evaluation of the effectiveness of public procurement, because they allow for automated generation and aggregation of procurement data at different levels (national, contracting authority, and for every contract). The last section of this chapter therefore covers performance assessments of the public procurement system, for instance through key performance indicators (KPIs, for instance savings) that should be a tool of strategic policymaking.

3.1. Main features of Kazakhstan’s e-procurement system

Kazakhstan has developed its government e-procurement system gradually, expanding its coverage and functionalities to account for frequent changes to the public procurement regulatory framework. The Government defined JSC "Center for Electronic Commerce" as the operator of the government e-procurement system and introduced e-procurement into the country’s legislative framework in 2008. JSC "Center for Electronic Commerce" launched a pilot of the government e-procurement system that hosted requests for quotations (RFQs) in the same year, and added other procurement methods to the system during the following years. In 2014, the center introduced e-contracts to the system.

The current version of the government e-procurement system, launched in 2016, is based on the Public Procurement Law (PPL) and the accompanying government procurement rules. All public contracting authorities and public entities falling under the scope of the PPL (see Chapter 1) must perform procurement through the government e-procurement system. Openness and transparency of the public procurement process are among the eight principles of public procurement in Kazakhstan defined in Article 4 of the PPL. Nevertheless, there are a few exceptions to transparency stipulated in article 1 of the PPL regarding goods, services and works of military and double usage as part of defense procurement or concerning “special orders” (Rus. Особый прядок) procurement involving confidential information and state secrets. These purchases are not published on the e-procurement system.

Kazakhstan’s e-procurement system provides a wide range of functionalities, covering the different phases of the procurement cycle, from publishing procurement plans to the e-submission of bids and some elements of ex post contract management. This makes the state e-procurement system a rather comprehensive e-procurement system when compared to many OECD countries. Figure ‎3.1 illustrates the coverage of e-procurement functionalities by the system, which expanded from tendering to cover other phases of the e-procurement cycle.

Figure ‎3.1. Expansion of e-procurement functionalities

Source: Ministry of Finance of Kazakhstan.

Already as of 2014, all OECD member countries announce procurement opportunities and provide tender documents through their e-procurement systems. In most OECD countries e-procurement systems provide functionalities at the beginning of the procurement cycle - publishing tender notices (97%), electronic submission of bids (80%), and notification of award (97%) (Figure ‎3.2). In contrast, a lower number of OECD member countries provides those towards the end of procurement cycle. Fewer countries provide ordering, electronic submission of invoices (56%) and ex post contract management (57%) through their e-procurement systems. It is worth noting, though, that e-invoicing has become mandatory for all contracting authorities in the EU as of April 2019.1

Figure ‎3.2. Use of e-procurement functionalities in OECD countries

Note: The chart is based on data from 30 OECD countries that answered the 2016 Survey on public procurement (OECD countries except Lithuania, France, United States, Switzerland, Czech Republic and Luxembourg).

Source: (OECD, 2016[5]).

By expanding functionalities offered by its government e-procurement system, Kazakhstan is in line with many OECD countries (Figure ‎3.3).

Figure ‎3.3. Functionalities of Kazakhstan’s e-procurement system

Note: √ provided by the e-procurement system, ≈ partially provided ✘ not provided.

Source: OECD elaboration based on the e-procurement system, available at: https://goszakup.gov.kz/.

In 2017, Kazakhstan made e-invoicing mandatory for all public procurement purchases. Suppliers and contracting authorities must now use the national e-invoicing system provided by Kazakhstan’s state revenue committee, an online platform that is distinct from the e-procurement system. A module in the e-procurement system manages acceptance acts or receipts. This helps accelerating payment of suppliers through the electronic national treasury system. The acceptance act and receipt module contains data in electronic format, as submission of acceptance acts and receipts is electronic since the beginning of 2018.

The home page of the e-procurement system provides access to legislation and norms related to public procurement (i.e. the PPL, government procurement rules and various orders of the Ministry of Finance), as well as detailed instructions for different types of users (suppliers, contracting authorities, single organisers and banks) without registration. In addition, it contains information about updates or modifications of the PPL or public procurement rules, as necessary. Standardised protocols provide for easy access to details regarding suppliers’ compliance with requirements in technical specifications and contract award decisions.

In terms of the documents available to the public regarding each public procurement procedure, Kazakhstan compares well to most OECD countries, with almost all key documents and modules of its e-procurement system available online, and clarification to bidder’s questions available to registered suppliers (Figure ‎3.4 and Figure ‎3.5). Transparency was indeed Kazakhstan’s strongest dimension in the EBRD 2012 Regional self-assessment of public procurement legislation (OECD, 2016[6]).

Figure ‎3.4. Availability of public procurement documents in OECD countries

Note: The chart is based on data from 30 OECD countries that answered the 2016 Survey on public procurement (OECD countries except Lithuania, France, United States, Switzerland, Czech Republic and Luxembourg.)

Source: (OECD[5]).

Kazakhstan’s e-procurement system provides a number of procurement documents to the public, see Figure ‎3.5.

Figure ‎3.5. Availability of public procurement documents in Kazakhstan

Note: √ Available to the general public, ♦ Available to registered suppliers.

Source: OECD elaboration based on the government e-procurement system, available at: https://goszakup.gov.kz/.

In addition to past public procurement transactions, the e-procurement system makes publicly available various “registers”: a supplier blacklist (“list of unreliable suppliers”), a list of public procurement plans published by contracting authorities, and a register of complains.

Kazakhstan’s government e-procurement system contains a contract management module that generates standard contracts for different categories of purchases. After the text of the contract is agreed upon, contracts can be signed though an electronic signature. The e-procurement system provides for the possibility of signing supplementary contracts introducing modifications to public procurement contracts. Suppliers and contracting authorities sign and manage these contracts through the e-procurement system’s contract management module. However, the PPL (Article 45) sets more restrictive rules regarding possible revisions to signed public procurement contracts. Since Kazakhstan’s legislation does not encompass framework agreements or product catalogues available for contracting authorities (see Chapter 2), these features are absent from the e-procurement system.

The e-procurement system is among the 20 most popular websites in Kazakhstan in terms of web frequentation. At the end of 2018, the number of registered users including suppliers was 225 896, while the number of registered contracting authorities (government entities, agencies and different types of SOEs) reached 24 201 (Table ‎3.1). In 2016, because of the launch of the new version of the e-procurement system, suppliers had to register again, hence the low number of registered users and suppliers for that year. Almost 60% of registered suppliers are individual entrepreneurs.

Table ‎3.1. Main indicators of the government e-procurement system (Goszakup)
Year	Registered users (end of year)	Registered suppliers (end of year)	Overall amount of purchases (Billion KZT)
2014	230 180	181 728	1 367
2015	270 023	214 055	2 368
2016	130 093	108 826	1 948
2017	194 185	171 520	2 875
2018	250 099	225 896	4 152
Note: In 2016, Kazakhstan launched a new version of its e-procurement system, requiring all users (suppliers, CAs, etc.) to go through registration again. This is why the number of registered users and suppliers were lower at the end of 2016 than one year before.
Sources: Ministry of Finance of the Republic of Kazakhstan (Strategic plan for 2017-2021); Extracts from https://www.goszakup.gov.kz/, Register of participants in public procurement, retrieved on 23 April 2019.

The e-procurement system is not the largest e-procurement system in Kazakhstan. National holdings (a specific sub-group of SOEs) conduct procurement activity through their own e-procurement platforms, accounting for a larger amount of purchases. This is due to high procurement volumes of national holding Samruk-Kazyna. It purchased around KZT 4.1 trillion in 2016, i.e. more or less two times the volume of public procurement during this year (KZT 1.9 trillion). Other national holdings have much lower procurement volumes, even though they also operate their own e-procurement systems. Chapter 6 provides more details on the procurement of national holdings, including the Sovereign Wealth Fund JSC Samruk-Kazyna.

Amendments to the PPL adopted on 26th December 2018 (hereafter: December 2018 amendments) introduce a mandatory user subscription fee on Kazakhstan’s e-procurement system. The access to most information will remain free and accessible without subscription – including tender notices, lots, contracts, tender specifications, and protocols regarding different procedural steps. However, two key functionalities are accessible to suppliers only after payment of the mandatory user subscription fee:

• Submitting a bid to any public procurement process (including for direct award processes; exceptions apply where national, confidential information is involved).

• Agreeing upon and signing any public procurement contract.

The Center for Electronic Commerce determines and collects the subscription fee, with the approval of the Ministry of Finance. The updated PPL stipulates that “subscription fees for using the e-procurement system must entirely compensate expenditures related to the functioning of the e-procurement system” (Article 17 of the updated PPL). This is a significant change as, up until the end of 2018, the National Budget had been funding the development, operations and maintenance of the e-procurement system. According to discussions with the Center for Electronic Commerce, previously, the Ministry of Finance earmarked EUR 1.5 million (KZT 600 million) annual for maintenance, operations and new developments of the e-procurement system.

The level of fees depends on the procurement volume earmarked for the procurement process in which the supplier participates, or on the value of the contracts signed by the supplier through the e-procurement system. The number of bids and contracts signed are unlimited. Suppliers pay the subscription fee once a year. Table ‎3.2 presents the annual subscription fees established for 2019.

Table ‎3.2. Subscription fees for using the e-procurement system
Maximal procurement value	Annual fee (2019)	Ratio of yearly fee to average monthly wage*
Up to KZT 1 million (EUR 2 460)	KZT 2 525 (EUR 6)	0.01
Up to KZT 10 million (EUR 25 591)	KZT 25 250 (EUR 62)	0.15
Up to KZT 100 million (EUR 245 906)	KZT 50 500 (EUR 124)	0.29
Up to KZT 1 billion (EUR 2 459 057)	KZT 151 500 (EUR 373)	0.86
No limit	KZT 308 050 (EUR 758)	1.75
Note: * Based on the last available average monthly wage (for Q4 2018) from the Committee on Statistics.
Source: (Government e-procurement system, 2019[7]), Kazakhstan’s Committee on Statistics and Central Bank.

Supplier fees are not the most common funding arrangement for public e-procurement systems in OECD countries. Instead, in most countries the government funds the operations of e-procurement systems or financing is provided by contracting authorities (often through a fee for each tender process). However, the subscription fees in Kazakhstan are modest and proportionate to contract value, and consequently the possible detrimental impact on the access of SME bidders to procurement opportunities should be limited. More importantly, subscription fees create new obstacles for foreign bidders in accessing Kazakhstan’s e-procurement system, because suppliers need to pay the fees by bank transfer from an account in a resident Kazakhstan bank registered in Kazakhstan, and payment is possible only in KZT. What is more, tax or business register identification numbers are mandatory on the transfer documents, even though non-resident do not have any tax or business registration. Applying for tax or business register identification is possible for non-residents, but requires cumbersome administrative procedures (including opening a bank account in a resident Kazakhstani bank) (Government e-procurement system, 2019[7]).

These additional obstacles for foreign (non-resident) bidders are in contradiction with the Principle on Access of the OECD Recommendation on Public Procurement. Chapter 2 of this report elaborates on foreign bidders’ access to public procurement opportunities. The Ministry of Finance should revise the payment process for subscription fees to open it to foreign and non-resident suppliers without tax or business register identification and without bank account in a resident Kazakhstani bank.

From 1st January 2020 onwards, each potential supplier will have to set up an electronic wallet (e-wallet) on the e-procurement system. E-wallets will be used for two purposes: depositing and returning bid bonds, and paying the annual subscription fee to the Center for Electronic Commerce as operator of the government e-procurement system. The introduction of E-wallets is a positive development as it simplifies and automates transactions between suppliers and the Center for Electronic Commerce. The Ministry of Finance should pay attention to the potential additional obstacles that the E-wallet might create for foreign suppliers.

3.2. Using digital technologies to increase transparency and integrate Goszakup with other government IT systems

3.2.1. Improving the public availability of high quality procurement data

Transparency has been a cornerstone of the development of Kazakhstan’s e-procurement system. It is based on the principle that almost all details of each purchase are publicly available on the system. This same principle presided for example over the reform of public procurement in Georgia in 2010, and led to the creation of the Georgian electronic Government Procurement (Ge-GP) system, one of the most transparent e-procurement system worldwide (EBRD, 2015[2]). One should note, however, that in Kazakhstan only de facto domestic users can use the e-procurement system fully. Indeed, there are significant barriers for non-residents to obtain the national electronic signature. This electronic signature is indispensable to submit bids and participate in the preliminary discussion of tender documentation. Beyond public procurement, in Kazakhstan the national electronic signature is widely used in interactions between citizens, businesses and the government (for instance, to submit tax declarations).

A well-implemented e-procurement system enables and facilitates timely collection of comprehensive and reliable data. In Kazakhstan, the e-procurement system provides an adequate level of transparency at the level of each public procurement purchase (see previous section). However, the availability of data and information on the e-procurement system does not allow for a meaningful performance assessment of specific contracting authorities or of the public procurement system as a whole. Providing access to more detailed, aggregated data on public procurement, and making this data accessible through user-friendly tools, would require reasonable efforts and funding. This is because, contrarily to several OECD countries, Kazakhstan has a centralised government e-procurement system used by most public entities. This greatly facilitates the centralised collection of procurement data.

The OECD Recommendation on Public Procurement calls for free access, through an online portal, to information on the performance of the public procurement system, including meaningful data for different groups of stakeholders (Box ‎3.2). The availability of aggregated procurement data is important to external parties interested in the performance of the government e-procurement system, such as journalists, representatives of civil society and suppliers, or even public servants outside of the Ministry of Finance. An adequate level of transparency is widely regarded as an effective tool to deter or detect corruption, but the relationship is not automatic: the availability of timely and high quality data is one of the conditions for effective accountability of public procurement officials and suppliers (OECD, 2016[8]).

Box ‎3.2. OECD’s Recommendation of the Council on Public Procurement – principle on transparency

II. RECOMMENDS that Adherents ensure an adequate degree of transparency of the public procurement system in all stages of the procurement cycle. To this end,

Adherents should:

i) promote fair and equitable treatment for potential suppliers, by providing an adequate and timely degree of transparency in each phase of the public procurement cycle, while taking into account the legitimate needs for protection of trade secrets and proprietary information and other privacy concerns, as well as the need to avoid information that can be used by interested suppliers to distort competition in the procurement process. Additionally, suppliers should be required to provide appropriate transparency in subcontracting relationships.

ii) allow free access, through an online portal, for all stakeholders, including potential domestic and foreign suppliers, civil society and the general public, to public procurement information, notably related to the public procurement system (e.g. institutional frameworks, laws and regulations), the specific procurements (e.g. procurement forecasts, calls for tender, award announcements) and the performance of the public procurement system (e.g. benchmarks, monitoring results).

iii) ensure visibility of the flow of public funds, from the beginning of the budgeting process through the public procurement cycle, to i) let stakeholders understand government priorities and spending, and ii) allow policy makers to organise procurement strategically.

Source: OECD (2015[3]) OECD Recommendation of the Council on Public Procurement”,www.oecd.org/gov/ethics/OECD-Recommendation-on-Public-Procurement.pdf.

The Ministry of Finance centralises detailed aggregated data, develops performance indicators for the government e-procurement system, and occasionally publishes such indicators in strategic documents (such as its strategic plan for 2017-2021) or uses them to substantiate speeches and presentations by high-ranking officials. For instance, Vice-Minister Beketaev presented members of parliament (MPs) with such aggregated statistics during a presentation of the (at the time) draft amendments to the PPL in January 2018.

In contrast, the scope of aggregated statistics and real-time reports publicly available on the government e-procurement system is very limited. Each module provides a statistical tool with basic real-time statistics and the corresponding infographics. For instance, the lots (Rus. Лоты) and notices (Rus. Объявления) module provides basic statistics on the breakdown of lots and notices by status (completed purchase, cancelled, under review…). Nevertheless, this piece of data is insufficient for any detailed analysis: Statistical definitions (“meta data”) are lacking and there is no option to query data for a specific time-period or export data in machine-readable format.

The main statistical reporting module (“report on conducted purchases of construction works, services and goods”) provides a minimal degree of transparency regarding past purchases. It automatically generates data tables on the number of purchases, budgeted amounts, purchase amounts, procurement method, the country of residence of suppliers, and conditional savings.2 The module provides for some degree of disaggregation, i.e. it allows querying data for each contracting authority, for different regions of Kazakhstan, etc. Moreover, it is possible to query data for 100 different aggregated categories of goods, works and services corresponding to the public procurement catalogue. Once again, the module lacks clear statistical definitions that would allow users to make sense of each statistical category. More importantly, data from the main statistical reporting module does not correspond to data reported elsewhere (such as in its strategic plan for 2017-2021) by the Ministry of Finance, leading to questions regarding data reliability or the existence of multiple databases.

There is a lack of transparency regarding aggregated statistics on different aspects of public procurement in Kazakhstan. No data is publicly available on the amounts corresponding to each of the 50 exceptions to competitive procedures allowing for direct award based on exceptions. The government e-procurement system does not make available detailed statistics about complaints filed through the government e-procurement system and the result of their examination. Most of the performance indicators that the Ministry of Finance uses to assess the performance of the government e-procurement system, such as the average number of bidders in tenders or the share of competitive procedures resulting in failed competitive bidding, cannot be retrieved or computed from the government e-procurement system. Some statistical reporting tools are available online on the e-procurement system, although they report old data (2012-2016) from the previous version of the government e-procurement system.

Some contracting authorities make available their procurement data through Kazakhstan’s open data portal (https://data.egov.kz/) in different machine-readable formats. For instance, the Ministry of Finance made available its 2017 procurement plan in the open data portal in December 2016. However, these pieces of data are not available on the government e-procurement system and there is no consolidated dataset for all contracting authorities. This means that there is no meaningful way to collect aggregated data about public procurement for Kazakhstan as a whole.

In order to provide better access to aggregated data, the Ministry of Finance could publish an annual statistical report on public procurement in Kazakhstan. Some countries have found this a useful approach, for example the government of Québec. It publishes such a report on the website of its Secrétariat du Conseil du Trésor.3 Such a report would contain detailed aggregated data and feature prominently on the government e-procurement system. It could also encompass a set of KPIs selected by the Ministry of Finance to measure the performance of the public procurement system (see section ‎3.3.1). A download centre on the government e-procurement system could give users access to the report’s data in different machine-readable formats, as well as to all information on the system as whole, including datasets in different machine-readable formats, following the example of the USA Spending platform in the United States (Box ‎3.3).

Box ‎3.3. USA Spending and its associated data lab

USAspending.gov1 is the official source for spending data for the U.S. Federal Government. It contains a wealth of user-friendly, easily accessible information about federal government procurement, and clearly demonstrates the link between federal budget spending and procurement information.

It regroups many sources of information from a variety of government systems, including the system of record for federal procurement data (the Federal Procurement Data System Next Generation (FPDS-NG)), and data from sub-awards records.

Federal agencies submit contract, grant, loan, direct payment, and other award data at least twice a month to be published on USAspending.gov. Federal agencies upload data from their financial systems and link it to the award data quarterly.

USA Spending contains several user-friendly modules:

• Spending profiles of federal agencies help users understand how each agency spends its funding. It entails an award tool enabling the identification of contract awards to different government suppliers for each budget program and type of expenditure.

• An advanced federal award data search tool makes is easy to find any government contract, and includes a spending map of federal contract awards by state.

• A download center allows users to download details providing the exact mapping of the flow of federal funds by type, agency/sub-agency and date range in a variety of machine-readable formats.

The US Treasury is also developing a data lab (https://datalab.usaspending.gov/) accompanying USAspending.gov website and providing innovative infographics on aggregate federal government spending, broken down to the level of each Agency, Department and Sub Agency. A dedicated contracting tool shows the percentage of contracts awarded through competitive procedures for each federal government agency.

← 1. https://www.usaspending.gov/#/

Sources: OECD Elaboration based on USA Spending Website https://www.usaspending.gov/#/ and its associated data lab (https://www.usaspending.gov/#/.

The Ministry could also enhance the main statistics-reporting module to include more details, including the amounts and number of purchases corresponding to each of the 50 exceptions to competitive procedures (Article 39 Point 3 of the PPL) and key performance indicators (average number of bidders, share of failed biddings, etc.) The government e-procurement system could also include a specific statistical reporting tool regarding complaints, detailing the number of complaints upheld or rejected by the Internal Audit Committee. Transparency regarding contract execution could be improved by making acceptance documents available systematically on the government e-procurement system, along with signed contracts.

In order to increase the quality and coverage of the procurement data it makes available, and to mobilise the different stakeholders (Ministry of Finance, Center for electronic Commerce, Ministry of Information and Communication), Kazakhstan is transitioning to the Open Contracting Data Standard (OCDS).4 One of the countries that recently adopted open contracting is Mexico. Mexico’s approach to developing open contracting was characterised by its inclusiveness: in 2017, the government created an alliance for open contracting involving stakeholders from the public and private sectors. Expected benefits include increasing traceability and auditability of tender submissions and make available data for subsequent analysis. (OECD, 2018[4]). This transition aims at providing increased availability of aggregated indicators and statistics regarding the public procurement system as a whole, as well as concerning all phases of the public procurement cycle: planning, tendering, awarding, contracting and implementation. As the government e-procurement system’s coverage extends to the post-award phase of the procurement cycle, a systematic approach such as the OCDS would make sure that the corresponding data (acceptance acts, invoices, effective payment dates, etc.) is published in structured and machine-readable format.

Better transparency of aggregated public procurement data can be instrumental in enhancing the role of civil society and the media in overseeing public procurement. In Kazakhstan, Forbes.kz published a ranking of the largest suppliers of the government – based on their share of public procurement spending.5 Nevertheless, journalists had to use a sample of 1 800 purchases extracted from the government e-procurement system and to hire an external data-analytics company to create the ranking, because of the lack of user-friendly, machine-readable data on the system. More data in such a format would facilitate this kind of analysis, bringing additional transparency to public procurement. Several OECD countries provide detailed statistical reports regarding public procurement. For example, the public procurement system of Chile (Chile Compra) makes available annual and monthly reports providing aggregated data and performance indicators of the public procurement system.6 In the United States, the transparency platform USA Spending and its associated data lab provides user-friendly access to federal government spending and public procurement data from 2017 onwards (Box ‎3.3 above).

3.2.2. Providing adequate and timely information to suppliers

The legislative December 2018 amendments comprise several measures related to transparency and information available to suppliers during the pre-tendering and bid submission phases of the public procurement cycle.

Reciprocal review of offers from other bidders in open tenders

On Kazakhstan’s government e-procurement system, registered suppliers bidding in a tender procedure can access all documents from other bidder’s offers without restriction. As discussed in the next paragraphs, this is at odds with the OECD best practices that emphasize the need to take into account the legitimate needs for protection of trade secrets, proprietary information and commercially sensitive information of suppliers as well as to avoid disclosing information that facilitate collusion or can be used to distort competition in the procurement process. Such open access to competitor’s offers allows bidders to file complaints regarding irregularities or errors in their competitor’s offers. This is possible through the complaint module of the e-procurement system. In practice, field research show that spurious suppliers often behave as “professional complainers” by submitting systematic complaints based on irregularities or missing documents in their competitors’ offers. Chapter 4 provides more details on this issue.

The timing of the access to other bidders’ offers (at the bid opening stage, Figure ‎3.6) was a major flaw in current tender procedures (auctions and tenders). It allowed bidders to review offers from other bidders before submitting their own final bid documents. Suppliers can change the documents in their offers after tender commissions issue preliminary qualification protocols. Protocols detail the reasons for the disqualification of each supplier. Disqualified suppliers have then three days after preliminary qualification to re-submit improved their offer, e.g. by submitting missing documents and certificates or redrafting the documents in their offers (Figure ‎3.6). Given access to all documents in their competitors’ offers while they could still re-submit their own, disqualified suppliers often used to copy elements from their competitors, including technical characteristics of products, in their own final offer.

Figure ‎3.6. Timeline of tender submission processes

Note: This timelines apply to open tenders (Rus. Конкурсы) and auctions (Rus. Аукционы) in Kazakhstan.

Source: Article 25 and 27 of the PPL, Points 86 to 141 of the Government procurement Rules.

The December 2018 amendments removed this loophole by changing the stage when bidders can access and review offers from other bidders. Indeed, the updated PPL (Article 24) grants such an access only at the final qualification stage (i.e. when bidders can no longer re-submit, see Figure ‎3.6), solely to bidders having paid the tender’s performance bond (1% of the planned budget for the contract). Removing this loophole will affect positively the efficiency and the integrity of public tender processes in Kazakhstan.

Suppliers also report that the reciprocal transparency of documents in offers often leads to the disclosure of commercially sensitive or confidential information (such as internal financial documents) to competitors. According to stakeholders, some bidders on the e-procurement system are shell companies, i.e. companies that do not conduct any actual business and therefore are not participating in the tender because of the business opportunity, but rather to access sensitive information in the bidding documents of competitors.

The OECD Recommendation on Public Procurement (Principle on transparency) highlights that transparency must “take into account the legitimate needs for protection of trade secrets and proprietary information and other privacy concerns, as well as the need to avoid information that can be used by interested suppliers to distort competition in the procurement process”(Box ‎3.2). Indeed, sharing commercially sensitive information such as pricing models and profit margins can discourage businesses from participating in public procurement. According to the OECD Competition Committee, disclosing such commercially sensitive information can also facilitate collusion or bid-rigging between bidders (OECD, 2016[9]).

Therefore, to avoid disclosure of commercially sensitive information to competitors, the Ministry of Finance could consider making the most sensitive documents in their offers available only to contracting authorities, tender commissions and auditors. Such sensitive or confidential information would no longer be available to other suppliers through the e-procurement system. There are several options available: the Ministry of Finance could set up a list of such commercially sensitive or confidential documents for which access through the e-procurement system would be restricted. Alternatively, the e-procurement system could include a tool allowing the bidders themselves to mark confidential information contained in the bids. The regulatory framework needs to be revised accordingly to introduce confidentiality provisions and appropriate procedures to manage specific confidentiality clause.

Preliminary discussion of technical specifications in tenders

Kazakhstan’s e-procurement system has a dedicated mechanism to make sure that suppliers have opportunities to provide feedback regarding technical specifications of tenders and auctions. After procurement organisers publish tender notices on the e-procurement system, suppliers have five days to analyse technical specifications and submit comments or request clarifications from contracting authorities (see timeline on Figure ‎3.6). Contracting authorities then have five days to provide clarifications or to amend technical specifications if necessary. Clarifications and amended technical specifications are accessible to all registered suppliers through the e-procurement system. This “preliminary discussion of technical specifications” (Rus. Предварительное обсуждение проекта конкурсной документации) has proven popular with bidders, who submitted 19 000 remarks and requests for clarifications in 2017. Approximately 11 000 of these remarks and requests led to a revision of technical specifications prior to the opening of the submission period. According to the Ministry of Finance, in 2016 contracting authorities revised 53% of all technical specifications for which suppliers submitted remarks regarding excessive or unachievable requirements.

Preliminary discussion of technical specifications has become one of the main avenues for contracting authorities to engage with suppliers, with the Ministry of Finance considering this a useful mechanism to increase the quality of technical specifications of tenders. This mechanism is in line with the OECD Recommendation on Public Procurement. Its Principle on participation highlights that “effective communication should be conducted to provide vendors with a better understanding of the country’s needs, and government buyers with information to develop more realistic and effective tender specifications by better understanding market capabilities” (OECD, 2015[3]).

The December 2018 amendments created a link between the e-complaint mechanism and the preliminary discussion of technical specifications, which takes place before the bid submission period (see timeline on Figure ‎3.6). In order to be entitled to file a complaint regarding any provision of technical specifications after bid opening, suppliers need to have submitted comments on these provisions during the preliminary discussion of technical specifications. Suppliers who did not comment on specific provisions of technical specifications cannot file complaints regarding these provisions after the contract award.

This reform makes the preliminary discussion of technical specifications central in enabling bidders to challenge flawed technical specifications. According to the Committee on Internal State Audit, the most common violations and irregularities in procurement processes are technical specifications that artificially exclude some suppliers from the bidding process. Therefore, it is paramount that suppliers have enough time to identify excessive or biased requirements in technical specifications during the preliminary discussion stage, and submit the corresponding comments. The period of five days after the publication of procurement notices is too short to allow for an adequate analysis of complex technical specifications and the drafting of comments.

Kazakhstan could extend the period for suppliers to submit comments and requests for clarifications to at least 10 days. Such an extension is all the more important after the limitation on complaints after contract award, in order to give more time for a meaningful preliminary discussion of technical specifications. It would also give more time to suppliers to prepare their bids: as mentioned in Chapter 2, the minimal submission period in Kazakhstan is short compared to good practices in several OECD countries.

Introducing prequalification of suppliers

The PPL also includes a separate category on public tender with prequalification, although it is not used in practice. The December 2018 amendments establish that a single qualification body will be in charge preselecting suppliers for public tenders with pre-qualification. According to the Ministry of Finance, this mechanism would be modelled after the prequalification of suppliers currently in force in Samruk-Kazyna, which aims at improving the performance of suppliers. Chapter 6 provides a description of the prequalification of suppliers in Samruk-Kazyna. It entails a questionnaire and a risk based audit depending on the complexity of the procurement. While in Samruk-Kazyna prequalification covers 9 000 items, the Ministry of Finance plans to introduce pre-qualification to the public procurement system regarding around 150 items, mostly related to construction works.

In order to ensure transparency of the pre-qualification mechanism, it would be advisable to disclose all prequalification requirements suppliers have to comply with online with a link and references on the government e-procurement system. Training videos, questions and answers (Q&A) would also be useful to inform suppliers about each step of the pre-qualification mechanism. The time allotted for suppliers to respond should be proportionate to the extent and complexity of the information they have to provide in order to prequalify. In order to ensure an equal treatment of all bidders and to maximise supplier’s participation in public procurement, information about an upcoming opportunity to participate in pre-qualification should be widely available before the pre-qualification actually enters into force.

3.2.3. Pursue integration with other Government IT systems

Integrating public procurement with public finance management, budgeting and service delivery systems can lead to greater effectiveness in public resource utilisation. These benefits come from various channels: improved information transmission, timeliness of information and enhanced data quality, improved accountability through stronger potential for control; and efficient management, since automation and standardisation do reduce redundant and unnecessary tasks and activities. Recognising the aforementioned potential benefits, the OECD Recommendation on Public Procurement underscores the importance of integration of public procurement with public finance management. (OECD, 2015[3])

In line with the OECD Recommendation on Public Procurement, Kazakhstan’s e-procurement system is already integrated with several other government IT systems, though the degree of integration differs. The most significant integration are automated checks on suppliers. The e-procurement system automatically checks suppliers against various registers of companies and/or individuals to make sure that suppliers are solvent, do not have tax arrears and are not on a List of Unreliable Suppliers (“blacklist”)(Chapter 4 of the report). Regarding budget planning, it is not clear whether there is a real integration of budget IT tools with the e-procurement system. State entities prepare procurement plans based on their draft budget in November-December for the following year, and publish them once Parliament approves the final budget at the end of December. One challenge for OECD countries includes integrating e-procurement systems into other government IT platforms. According to an OECD survey, 60% of e-procurement systems are not integrated with other e-Government digital solutions such as budgeting, business and tax registries, social security databases, financial systems for payments or Enterprise Resource Planning systems. (OECD, 2016[5])

However, many OECD countries are now evolving towards more integration of their e-procurement system with a goal of achieving a fully integrated, end-to-end procurement system. Korea’s KONEPS e-procurement system is an example of successful integration of an e-procurement system with many other public and private sector IT Systems (Box ‎3.4). Moreover, Korea measured savings focusing on the amount of public and private funds saved thanks to integration, demonstrating the potential benefits of further integration of government IT Systems with e-procurement systems.

Kazakhstan plans further integration of its e-procurement system with an external, inter-agency IT system.

Box ‎3.4. Greater efficiency and reduced costs through data connections in KONEPS

KONEPS (Korea Online E-procurement System) is interconnected with over 160 diverse electronic systems, both within and outside the government’s reach. This resulted in increased efficiency, reduced duplication and cost savings. A study conducted by Hanyang University in 2009 indicates that these changes led to an estimated USD 8 billion in annual transaction-cost savings. Out of this, around 16% of savings (USD 1.3 billion) accrued to the public sector, thanks to reduced labour and processing time. The rest of the savings accrued to suppliers and account for reduced costs in obtaining certificates, proof documents, and registering or updating accounts in multiple systems.

Businesses are able to participate in biddings after a one-time registration with KONEPS: users save their documents and certificates on the system to be automatically retrieved by KONEPS for future biddings. Otherwise, KONEPS extracts documents and certificates from other government IT Systems through data exchange interfaces. For instance, business registration certificates and tax payment certificates are transmitted through appropriate data exchange interfaces.

Connection with 19 surety companies allows for automated verification of 4 types of sureties, including bid bonds and performance bonds. Interfaces with 12 private sector associations and 9 credit ratings companies allows for the automatic collection of credit and past performance data, which is used to verify qualifications and evaluate bids. Fifteen commercial banks are connected for e-payment through electronic funds transfer, and for processing loans that are available to support government contract holders.

Because of this integration, 477 document forms used in public procurement including bid forms, contract forms, inspection requests and payment requests have all been converted to digital equivalents. Moreover, for construction work tenders, bidders are no longer required to submit certificates on their experience, because data is electronically collected though data interchange with databases of construction industry associations.

Figure ‎3.7. The KONEPS E-procurement process

Source: (OECD, 2016[10]).

An electronic portal providing a single access point to procurement opportunities is under development by the National Chamber of Entrepreneurs (“Atameken”). The access point is expected to be a modular system that aggregates all procurement notices and opportunities from the government e-procurement system, Samruk-Kazyna’s e-procurement system, as well as other national holdings and companies. It is intended to function as an aggregator providing access to procurement opportunities from a single website, with a link to each platform for bid submission. While private-sector tender aggregators already exist, the access point would be free and therefore enhance the access of SMEs to procurement opportunities from the government procurement system and national holdings and companies. In order to maximise its effect, the access point could provide registered suppliers with RSS feeds or email alerts on newly published tender notices in their business area and in their regions.

Kazakhstan could further integrate its e-procurement system with other government IT information systems, and consider setting up data exchange interfaces with private sector IT platforms or data systems. One priority area would be to develop inter-operability and data exchange interfaces between the e-procurement system and the national tax system. This could help identify shell companies that did not actually pay any tax (no genuine economic activity). The Ministry of Finance could consider excluding such companies from the most complex and the largest tenders, for instance as part of the prequalification procedure for construction works. However, the Ministry needs to balance the efficiency benefits of eliminating bidders without previous economic activity (and potential spurious bidders) and the need to grant reasonable access of new entrants and SMEs to public procurement opportunities. Moreover, the Ministry needs to set up adequate safeguards to ensure data privacy (i.e. for instance, restrict procurement officer’s access to tax records data to the minimum necessary so that they can enforce applicable regulations). Such safeguard is necessary to minimise the risk the inappropriate use of the data.

Appropriate data exchange interfaces could also help streamlining other tasks of the e-procurement system, such as assessing the completeness of procurement plans. An automated review of procurement plans against the budget would require data exchange between the e-procurement system and the budget planning systems of the Ministry of Finance and/or of contracting authorities.

The Ministry of Finance could also simplify the calculation of the number of years of experience of suppliers for establishing conditional discounts (see Chapter 2). Procurement officers report that determining the years of experience of a potential supplier is a cumbersome process and that the complexity of the rules allow for different interpretations. Kazakhstan could explore the computation of supplier experience to determine conditional discounts through automated data exchange with relevant business associations. Korea’s public e-procurement system (KONEPS) offers such a functionality through data exchange with business associations from the construction industry (Figure ‎3.7).

3.3. The e-procurement system provides opportunities to strengthen evaluation and performance management

The 2015 OECD Recommendation on Public Procurement encourages countries “to drive performance improvements through evaluation of the effectiveness of the public procurement system” (Box ‎3.5). Such evaluation is highly dependent on the availability of reliable and up-to-date procurement data at all levels: contract management, contracting authority and national level (performance of the public procurement system as a whole). When aggregated and developed into performance indicators, such data can support substantial reforms to increase efficiency and eliminate waste.

Box ‎3.5. OECD Recommendation on evaluation

X. RECOMMENDS that Adherents drive performance improvements through evaluation of the effectiveness of the public procurement system from individual procurements to the system as a whole, at all levels of government where feasible and appropriate. To this end, Adherents should:

i) Assess periodically and consistently the results of the procurement process. Public procurement systems should collect consistent, up-to-date and reliable information and use data on prior procurements, particularly regarding price and overall costs, in structuring new needs assessments, as they provide a valuable source of insight and could guide future procurement decisions.

ii) Develop indicators to measure performance, effectiveness and savings of the public procurement system for benchmarking and to support strategic policy making on public procurement.

Source: OECD (2015[3]) OECD Recommendation of the Council on Public Procurement”,www.oecd.org/gov/ethics/OECD-Recommendation-on-Public-Procurement.pdf.

As mentioned earlier in this chapter (section ‎3.2.1.), in Kazakhstan the collection of aggregated procurement data is insufficient for meaningful performance assessment at the contracting authority and national levels. Nonetheless, the Ministry of Finance periodically collects limited aggregated data and computes some performance indicators on an ad hoc basis. Overall, the Ministry or other government entities do not seem to conduct evidence-based, systematic performance evaluation of public procurement. One exception is the price module, an analytical tool available on the e-procurement system that contracting authorities use to inform their budget planning process.

3.3.1. Monitoring key performance indicators to provide strategic guidance of the e-procurement system

The 2015 OECD Recommendation of the Council on Public Procurement calls on OECD countries to develop indicators to measure the performance, effectiveness and savings of the public procurement system to support strategic policy making on public procurement. In Kazakhstan, the existence of a centralised, mandatory e-procurement system and the wide coverage of electronic procedures create favourable conditions for collecting more consistent, up-to-date and reliable data on procurement processes.

Such data is indispensable to measure performance through the development and deployment of procurement metrics and indicators. Indicators of this sort provide insights into trends over time, for instance regarding market changes that call for new procurement approaches. If combined with better integration of various existing data systems, as recommended above, such indicators allow for the comparison of performance across different contracting authorities or different regions (oblasts), eventually driving improved performance by sharing best practices. It would also provide a mechanism to detect inefficient practices, irregularities and fraud.

Kazakhstan already monitors a few performance indicators on an ad hoc basis

In line with best practices from OECD countries, Kazakhstan already collects and computes a few performance indicators concerning public procurement. However, the Ministry of Finance computes such indicators in a centralised manner on an ad hoc basis, i.e. to contribute to documents such as the Ministry of Finance’s strategic plan or to speeches and presentations by high-level officials. It is unclear whether these performance indicators are used to benchmark the performance of contracting authorities or of the public procurement system as whole, and to what extent performance is measured against predefined targets. Table ‎3.3 details the main performance indicators that the Ministry of Finance puts forward in analytical documents regarding the e-procurement system.

Table ‎3.3. Main performance indicators for public procurement in Kazakhstan
Aspect of public procurement	Performance indicator	Year	Value
Level of competition	Average number of bidders	2016	Tender: 4 Request for quotations/auction: 3
Efficiency	Budget savings (or conditional savings)	2017	KZT 2019 billion
Level of competition	Share of failed competitive procedures	2017	63%
Feedback from suppliers	Share of revised technical specifications following remarks by suppliers (during preliminary discussion)	2017	53%
Risk management	Share of purchases with irregularities (desk review)	2016	20%
Note: Failed competitive bidding means a competitive procedure with no or a single bidder. This entitles contracting authorities to resort to direct award (direct award after failed public tender).
“Share of revised technical specifications following remarks by suppliers” refers to preliminary discussion of technical specifications in tenders. Another associated key indicator could be the share of technical specifications for which suppliers submitted remarks.
Budget savings (or conditional savings) is the difference between the budgeted amount for a specific procurement and the actual amount in the procurement contract. It is measured for each procurement purchase and then aggregated for the public procurement system as a whole.
Sources: Strategic plan of the Ministry of Finance for 2017-2021, External analysis of corruption risks in public procurement (2017), Ministry of Finance.

Beyond the share of failed competitive bidding (Table ‎3.3), the Ministry of Finance could monitor other indicators regarding exceptions to competitive biddings. The percentage of direct award purchases as a share of overall purchases is an important indicator of the coverage of competitive procedures. The same applies for the number of procedures using the direct award method as a share of overall procurement value. A breakdown of direct award procurements into direct award based on exceptions (exceptions to competitive procedures as per article 39 point 3 of the PPL) and direct award after failed public tender brings valuable additional information, as demonstrated in Chapter 2 of this report.

Overall, Kazakhstan would need to define a comprehensive set of key performance indicators (KPIs) regarding each aspect of its public procurement system and closely monitor them on a regular basis. The Ministry of Finance would need to track and report these KPIs in a systematic, structured manner, as official indicators of public procurement performance. Since one of the main issues of public procurement in Kazakhstan is the high share of non-competitive procurement procedures, monitoring performance indicators related to the level of competition and the coverage of exceptions to competitive tendering would be a priority. The Ministry could also develop indicators regarding supplier performance and payment delays thanks to the supplier database suggested in the next section (‎3.3.2).

These indicators could be published annually as part of the annual statistical report on the government e-procurement system suggested in section ‎3.2.1. The same indicators could also be a valuable tool to benchmark the performance of the procurement function of contracting authorities and government agencies both at the central level and at the level of regional and local administrations.

Best practices from OECD countries regarding performance indicators in public procurement

Performance indicators rely on up-to-date and reliable data on prior procurements. These indicators are at an early stage of development in many OECD countries. Many countries focus on indicators measuring efficiency, i.e. the proportionality between, on the one hand, the costs and time requirements for each public procurement procedure and, on the other hand, the value of the contract. (OECD, 2016[5]).

There is no unique methodology to measure savings in public procurement. Turkey measures savings the same way as Kazakhstan, i.e. as the ratio of contract value to estimated cost (or budgeted amount for the purchase). Chile measures savings as the difference between the average value of bids received in each process and the value of the awarded bid. Greece has a business intelligence module that generates market prices that allow for a benchmark of prices resulting from tenders. Along with savings from the tendering process itself, Portugal measures efficiency generated by aggregating purchases (OECD, 2016[5]). This would be a good practice for Kazakhstan, since it plans to introduce for aggregated purchases by single organisers as part of the current reform of the government procurement system.

As part of its e-procurement system SECOP II, Colombia recognised the importance of developing key performance indicators derived from data provided by its e-procurement system. The central procurement body Colombia Compra Eficiente identified and defined eleven key indicators across four areas, and assessed baseline values regarding the overall public procurement system for year 2014. The six most relevant performance indicators for Kazakhstan are presented in Table ‎3.4. Each indicator has a baseline value for the previous year and Colombia Compra Eficiente systematically develops target values for the years to come. Some of these KPIs could be a source of inspiration for the Ministry of Finance, particularly the percentage of contracts with modification of time or value or the average time of the selection process.

Table ‎3.4. Key procurement indicators of Colombia Compra Eficiente
Value for money
Average time of the selection process	Period of time between the beginning of the procurement process and contract signature	Open tender: 37 days Direct contracting: 26 days
Integrity and transparency in competition
Average of new suppliers	Percentage of new suppliers in a public entity (New suppliers/Number of suppliers during previous year)	24.1%
Concentration of the contracts’ value by contractor	Concentration of a public entity’s procurement budget by supplier measured by the Gini coefficient	0.638
Percentage of contracts awarded to plural bidders	Frequency of awarded contracts to plural bidders by a public entity	10%
Percentage of contracts awarded in non-competitive processes	Percentage of public contracting that is done under non-competitive processes	38.5%
Risk management
Percentage of contracts with modifications in time and/or value	Proportion of contracts modified as regards contract value or contract execution time	23%
Sources: Adapted from Box 1.6 in (OECD, 2016[11]).

Colombia’s KPIs are consistent with the efforts of other OECD countries to develop performance indicators for public procurement.

The Methodology for Assessing Procurement Systems (MAPS) is a universal tool that aims to catalyse and accelerate the implementation of modern, efficient, sustainable and more inclusive public procurement systems in all countries. MAPS contains a number of quantitative indicators. Box ‎3.6 presents key indicators in four areas that might be relevant for Kazakhstan, particularly regarding contract management and payment.

Box ‎3.6. Key procurement indicators from MAPS

Suppliers

• Number of registered suppliers as a share of total number of suppliers in the country (in %)

• Share of registered suppliers that are awarded public contracts (in % of total number of registered suppliers)

• Total number and value of contracts awarded to domestic/foreign firms (and in % of total)

Audit and risk management

• Number of training courses conducted to train internal and external auditors in public procurement audits

• Share of auditors trained in public procurement (in % of total number of auditors)

• Share of internal and external audit recommendations implemented within the time frames established in the law (in %)

Competition and Submission of bids

• Value of contracts awarded through competitive methods (most recent fiscal year)

• Average time to procure goods, works and services: number of days between advertisement/solicitation and contract signature (for each procurement method)

Contract management and payment

• Time overruns (in %; and average delay in days)

• Contract amendments (in % of total number of contracts; average increase of contract value in %)

• Quality-control measures and final acceptance is carried out as stipulated in the contract (in %)

• Invoices paid on time (in %).

Source: Methodology for Assessing Procurement Systems, 2018.

Australia’s National Audit office developed guidelines on Key Performance Indicators in Public Procurement, which can also be a source of inspiration for Kazakhstan (Box ‎3.7), particularly concerning indicators focusing on the contract management phase of the procurement cycle (Delivery, customer service, product in Box ‎3.7).

Box ‎3.7. Establishing good key performance indicators for procurement

In order to be useful tools to benchmark performance, good key performance indicators must possess some fundamental qualities.

• Relevant, i.e. linked to key objectives of the organisation (critical outcomes or risks to be avoided), rather than on process.

• Clear, i.e. spelled out as simple as possible to ensure common understanding

• Measurable and objective, i.e. expressed on pre-determined measures and formulas, and based on simple data that can be gathered objectively and in a cost-effective manner.

• Achievable, i.e. realistic and compatible with existing regulations/constraints

• Limited, i.e. as few as required to achieve the objectives while minimising their disadvantages (costs, efforts). To the extent possible, they should rely on information and documentation already available on the e-procurement system.

• Timed, i.e. include specific timeframes for completion.

Procurement key performance indicators can also be established to manage the performance of suppliers. While a wide variety of dimensions of supplier performance can be considered, the following ones may be appropriate:

• Delivery, i.e. whether the supplier delivers on time, delivers the right items and quantities, provides accurate documentation and information, responds to emergency delivery requirements, etc.

• Pricing: competitiveness, price stability, volume or other discounts, etc.

• Customer service: number of product shortages due to the supplier, training provided on equipment and products, warranty services, administrative efficiency (including order acknowledgement and accurate invoice), etc.

• Product: meets specifications (including percentage of rejects/defects), reliability/durability under usage, packaging, quality and availability of documentation and technical manuals, etc.

Finally, key performance indicators should ideally be monitored on the same frequency, ideally quarterly or at least annually.

Source: (Australian National Audit Office, 2011[12]).

3.3.2. Using data on past procurements to inform future budgeting and procurement decisions

Kazakhstan’s e-procurement system and the existence of e-contracts enabled data collection about the prices of purchases. This enabled the development of a price module on the government e-procurement system. The module reports standardised data on average, maximal and minimal prices for goods, works and services procured through the government e-procurement system after 2016. For instance, Table ‎3.5 shows the output of the price module for the most popular car with contracting authorities in 2017, for Kazakhstan as a whole. The price module reveals large differences in purchasing prices for the same category of cars. Part of it may be related to different automotive brands that all fall under the same category in the public procurement nomenclature.

Table ‎3.5. Price module output (KZT) for year 2017
Passenger car (European class: C). Manual transmission, engine volume up to 2000 CC, power steering, air conditioning, airbags
			PLANNED PRICE			PRICE OF ACQUISITION
Nomenclature code	Name	Quantity	Minimal	Average	Maximal	Minimal	Average	Maximal
29.10.22.300.000.0796.02	Passenger car	1 296	KZT 863 101	KZT 4 497 535	KZT 15 178 571	KZT 2 345 536	KZT 4 311 225	KZT 14 687 500
Note: This table is based on the nomenclature codes from Kazakhstan’s unified nomenclature guide of products, services and goods, which contains 30 numbers.
Source: Price module of the government e-procurement system, available at http://portal.goszakup.gov.kz/portal/index.php/ru/pricereport/report1

The module entails a search function, making it possible to consult prices for a single region or for a single contracting authority and to combine these different search criteria. The Ministry of Finance sees the price module as a precious tool to tackle the issue of public procurement purchases with artificially inflated prices, as they could be the result of breaches of integrity or corruption schemes involving suppliers and contracting authorities.

Contracting authorities use the price module for market research and budget planning purposes. The government e-procurement system does not have any other market research or business intelligence module. Contracting authorities complement data from the price module with online research and/or consultations with potential suppliers.

The price module is a good example of systematic and aggregated data collection on Kazakhstan’s government e-procurement system. Kazakhstan could collect more procurement data to set the foundation for evidence-based performance assessments of public procurement. For instance, collecting and structuring data on supplier performance has the potential to improve supplier compliance with contract obligations and reduce the waste of public funds related to inadequate suppliers.

Kazakhstan maintains a register of suppliers on its e-procurement system. This register provides basic information about a given supplier, such as its address or its tax registration number. However, the database contains no information about suppliers’ activities, past experiences or past performance in public procurement contracts. Currently, the only input about past performance of potential suppliers available to contracting authorities and tender commissions is the black list (officially “list of unscrupulous suppliers”). Chapter 4 discusses the debarment process in details. The e-procurement system automatically excludes debarred suppliers from bidding to any public procurement during 24 months.

Kazakhstan could consider setting up a more detailed supplier database that would be available for contracting authorities, single organisers and auditors. Such supplier database would contain data regarding suppliers’ activities, experience (at least regarding public procurement contracts) and past performance, as well as financial and tax information from data exchange with the IT financial system of the Committee on State Revenue (Tax Authorities). The Past Performance Information Retrieval System (PPIRS) and the Federal Awardee Performance and Integrity Information System (FAPIIS) databases accessible to US Procurement officers in the United States could provide examples of how to track and use supplier performance information (Box ‎3.8).

Box ‎3.8. Supplier performance information in the United States

In working to build supplier relationships, the United States focuses on doing business with contractors who place a premium on integrity, performance and quality. To this end, government agencies have been directed to improve the quantity, quality, and utilisation of supplier performance information through the use of two systems.

Supplier past performance information including an identification and description of the relevant contract, ratings across six dimensions (quality, schedule, cost, utilisation of small business, etc.), and a narrative for each rating is contained within the Past Performance Information Retrieval System (PPIRS, www.ppirs.gov). Government agencies are required to report past performance information on this system, which will then be available to other contracting officers within PPIRS, on all contracts and orders above USD 150 000.

This web-based, government-wide application provides timely and pertinent information on a contractor’s past performance to the federal acquisition community for making source selection decisions. PPIRS provides a query capability for authorised users to retrieve report card information detailing a contractor's past performance. Federal regulations require that customers complete report cards annually during the life of the contract. The PPIRS consists of several sub-systems and databases (e.g. Contractor Performance System, Past Performance Data Base, and Construction Contractor Appraisal Support System).

Additional information regarding certain business integrity issues, including contracts terminated for default or cause, information about criminal, civil, or administrative procedures related to a federal contract; and prior findings that a contractor is not responsible, is captured in the Federal Awardee Performance and Integrity Information System (FAPIIS). Agencies are taking steps to improve the value of both systems by providing information that is both more complete and more useful.

Source: (Office of Federal Procurement Policy, 2013[13]).

Such a database could guide the choice of contracting authorities regarding direct award procurement while potentially increasing competition and reducing the risks of integrity breaches. Contracting authorities would have access to a wide pool of registered suppliers with detailed information allowing them to make an informed judgment. Moreover, such a detailed supplier database would simplify the process for determining (computing, in fact) bidders’ past experience.

3.3.3. Making better use of e-procurement to implement Kazakhstan’s procurement strategy

Kazakhstan has an advanced public e-procurement system, which offers functionalities that cover almost the entire procurement cycle, as recommended by the OECD Recommendation on Public Procurement. Nonetheless, benefits from e-procurement are usually a result of better management and co-ordination facilitated by technology, rather than technology per se (Asian Development Bank, 2013[14]). E-procurement can play a significant role in public procurement reform, but it will not necessarily remedy poor procurement practice and it may not solve underlying problems in public procurement operations. Poor practices can easily be perpetuated through e-procurement (OECD, 2011[15]).

In order to make the most of e-procurement, Kazakhstan could develop new procurement functionalities to support the ongoing modernisation and centralisation of its procurement system. In Kazakhstan, e-procurement has the potential to drive the modernisation of public procurement in at least 3 areas:

• Consolidation of purchases: As detailed in Chapter 1, Kazakhstan reformed its public procurement regulations to allow central procurement entities to consolidate purchases of homogeneous goods, works and services from different contracting authorities. The December 2018 amendments entitle the Ministry of Finance to define the exact goods, services and works to be procured through central procurement entities. This would provide for the reduction of the number of public procurement procedures and therefore drive cost-savings on pair with the gradual centralisation of the public procurement system (Chapter 1).

• To define the exact goods, services and works whose procurement will be centralised, the Ministry of Finance could undertake a systematic and automated analysis of procurement plans based its unified nomenclature of goods, services and works ENS-TRU (Russian : ЕНС-ТРУ).

• One-time registration of suppliers: Suppliers could save all their documents and certificates on the system to be automatically retrieved in future biddings, eliminating the need to submit documents and certifications for every new bid submission. Such a system already exists for suppliers on the Korean e-procurement system KONEPS (Box ‎3.4 above). In the EU, a similar one-time registration system entered into force along with the transition to mandatory e-submission of bids in November 2018, as stipulated in EU Directive 2014/24/EU.

• Development of standard technical specifications: As mentioned in Chapter 1, the Ministry of Finance plans to develop a catalogue of standard technical specifications in 2019. Standard technical specifications are one of the keys to aggregated purchases since they will make it easier for central procuring entities to aggregate homogeneous lots from different contracting authorities. In this regard, it is important that they are available to contracting authorities on the e-procurement system.

Until now, Kazakhstan has focused its efforts on transitioning from paper-based procurement to e-procurement and using e-procurement to streamline pre-existing procedures and increase transparency. The ongoing reform of the public procurement system offers an occasion to build on the existing e-procurement system and the procurement data it collects to improve the efficiency and the transparency of the procurement system, and to strengthen the performance evaluation of each purchase, of contracting authorities and of the public procurement system as a whole. The centralisation of e-procurement will be a success only if it is conceived and implemented with due account of the opportunities offered by e-procurement.

Proposals for action

The updated e-procurement system launched in 2016 along with the PPL brought more transparency to public procurement in Kazakhstan. It also made available useful new functionalities, such as the Preliminary discussion of technical specifications in open tenders or the e-complaint module available from the beginning of 2018. The e-procurement system can be a tool for further improvement of public procurement in Kazakhstan.

The Ministry of Finance could address certain drawbacks regarding transparency in open tenders, such as excessive transparency in the reciprocal review of offers from other bidders, or the short bid submission period. More and better consolidated data concerning the public procurement system as a whole could be disclosed on the e-procurement system in a machine-readable format. This would contribute to reducing corruption risks and bring more transparency to the system, for the benefit of interested third parties (journalists, CSOs, etc.) The Ministry could also make a better use of procurement data to monitor performance of suppliers, contracting authorities and conduct evidence-based evaluation of the procurement system as a whole.

Using digital technologies to increase transparency and integrate Goszakup with other government IT systems

• The Ministry of Finance could consider restricting access to the most sensitive information. What kind of information or documents should be confidential could be considered in cooperation with the suppliers’ community. Possible documents and information could include business secrets or financial information that is not publically available. Another option is that the e-procurement system could include a tool allowing the bidders themselves to mark confidential information contained in the bids. Upon review, such information could be made accessible only to contracting authorities, tender commissions and state auditors. The public procurement regulatory framework needs to be revised accordingly to introduce confidentiality provisions and appropriate procedures to manage specific confidentiality clause.

• In undertaking further reforms of the e-procurement system, and considering the pending accession process to the Agreement on Government Procurement (GPA), Kazakhstan could consider implicit technical barriers to access and participation by non-resident bidders. Further improvements could aim at removing existing barriers to facilitate access to the largest number of potential suppliers in order to maximise competition and value for money for Kazakhstan’s citizens. Innovations to the e-procurement system could be vetted from this angle, ensuring that no additional barriers are introduced.

• For complex procurements, Kazakhstan could extend the period for suppliers to submit comments and requests for clarifications in the preliminary discussion of technical specifications to at least ten days (instead of five days now) to ensure potential suppliers understand the tender and can prepare adequate bids.

• In a similar vein, data related to public tenders with prequalification should be published once prequalification is implemented. In addition, to ensure transparency of the pre-qualification mechanism, it would be advisable to disclose all prequalification requirements suppliers have to comply with on a single web-portal, easily accessible from the e-procurement system. Information about upcoming opportunities to participate in pre-qualification could be accessible a few months before pre-qualification actually enters into force.

• The Ministry of Finance could further integrate the e-procurement system with other government IT information systems, and consider setting up data exchange interfaces with private sector IT platforms or data systems. One priority area would be to develop inter-operability between the e-procurement system and the national tax system to detect shell companies without genuine business activity on the e-procurement system.

• The Ministry of Finance could set up an electronic depository on the e-procurement system, providing for one-time registration of suppliers, i.e. all suppliers’ documents and certificates would be stored electronically and retrieved by the system in all future bids by the same supplier.

The e-procurement system provides opportunities to strengthen evaluation and performance management

• The Ministry of Finance could define a comprehensive set of key performance indicators (KPIs) regarding different aspects of the public procurement system. The Ministry could monitor them in a systematic, structured manner. These KPIs would cover several phases of the procurement cycle, including contract management (supplier performance, payments time, modification of contracts after signature). In addition, the Ministry could publish these KPIs annually as part of an annual statistical report on public procurement.

• The Ministry of Finance could collect and monitor indicators regarding the level of competition in public procurement, including the percentage of direct award procurement both as a share of procurement purchases and as a share of overall procurement volumes, the average number of responsive bids per public tender, etc. The Ministry of Finance could set up a detailed supplier database regarding suppliers’ activities, experience (at least regarding public procurement contracts) and past performance, as well as financial and tax information from data exchange with the IT financial system of the Committee on State Revenue (Tax Authorities). This database should be available to tender commissions and procurement officers. Such a registry could also provide an opportunity to consolidate and store information that is supplied by bidders repeatedly.

• The Ministry of Finance could publish an annual statistical report on public procurement using the data gathered through the e-procurement system. The report is recommended to include aggregate statistics of the public procurement system and selected performance indicators. To improve public access to public procurement data, a download option should provide access data from the report in machine-reading format, and cover issues such as the exceptions and complaints.

References

[14] Asian Development Bank (2013), E-Government Procurement Handbook, https://www.adb.org/sites/default/files/institutional-document/34064/files/e-government-procurement-handbook.pdf (accessed on 13 August 2018).

[12] Australian National Audit Office (2011), Development and Implementation of Key Performance Indicators, https://www.anao.gov.au/sites/g/files/net4981/f/201112%20Audit%20Report%20No%205.pdf (accessed on 14 August 2018).

[2] EBRD (2015), Are you ready for eProcurement? Guide to Electronic Procurement Reform, https://www.ppi-ebrd-uncitral.com/index.php/fr/component/content/article/427-ebrd-is-launching-a-guide-to-eprocurement-reform-are-you-ready-for-eprocurement (accessed on 8 August 2018).

[7] Government e-procurement system (2019), Tariffs & payment details, https://www.goszakup.gov.kz/ru/welcome/pay.

[4] OECD (2018), Mexico’s e-Procurement System: Redesigning CompraNet through Stakeholder Engagement, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/9789264287426-en.

[1] OECD (2017), Public Procurement in Peru: Reinforcing Capacity and Co-ordination, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264278905-en.

[5] OECD (2016), 2016 OECD Survey on public procurement, OECD Publishing, Paris, https://qdd.oecd.org/subject.aspx?Subject=GOV_PUBPRO_2016.

[9] OECD (2016), Fighting Bid Rigging in Public Procurement: Report on Implementing the OECD Recommendation 2016, http://www.oecd.org/daf/competition/Fighting-bid-rigging-in-public-procurement-2016-implementation-report.pdf (accessed on 8 August 2018).

[6] OECD (2016), Multi-dimensional Review of Kazakhstan: Volume 1. Initial Assessment, OECD Development Pathways, OECD Publishing, Paris, https://doi.org/10.1787/9789264246768-en.

[8] OECD (2016), Preventing Corruption in Public Procurement, http://www.oecd.org/gov/ethics/Corruption-in-Public-Procurement-Brochure.pdf.

[10] OECD (2016), The Korean Public Procurement Service: Innovating for Effectiveness, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/9789264249431-en.

[11] OECD (2016), Towards Efficient Public Procurement in Colombia: Making the Difference, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/9789264252103-en.

[3] OECD (2015), OECD Recommendation of the Council on Public Procurement, OECD, Paris, http://www.oecd.org/gov/public-procurement/recommendation/ (accessed on 11 September 2017).

[15] OECD (2011), “E-Procurement”, SIGMA Public Procurement Briefs, No. 17, OECD Publishing, Paris, https://doi.org/10.1787/5js4vmnvwp8w-en.

[13] Office of Federal Procurement Policy (2013), Improving the collection and use of information about contractor performance and integrity, https://www.acq.osd.mil/dpap/policy/policyvault/USA001513-13-DPAP.pdf (accessed on 20 July 2018).

Notes

← 1. See EU Directive 2014/55/EU, avalable at https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32014L0055

← 2. Conditional savings are the difference between initially budgeted amount for the purchase and the final purchase amount. The Ministry of Finance uses it as one of the main performance indicators of the public procurement system.

← 3. See Québec’s public procurement statistics, available at (in French) : https://www.tresor.gouv.qc.ca/nouvelles/?tx_ttnews%5Btt_news%5D=470&cHash=ad9e526b8a8537e59e7f18c566f0779a

← 4. See a presentation of the standard on the OCP Website : https://www.open-contracting.org/data-standard/

← 5. See article “Aydin Rakhimbaev leads the ranking of the largest recipients of public procurement orders in Kazakhstan”, Forbes Kazakhstan, available (In Russian) at : https://forbes.kz/leader/gosudarstvennyiy_zakaznik_2_1523606498

← 6. See Chile Compra analysis module : http://www.analiza.cl/web/Modulos/Cubos/CubosOlap.aspx