4. Built for purpose: Towards a more efficient and effective public procurement system

This chapter is centred on the Principle on efficiency, which calls upon Adherents to implement sound technical processes to satisfy customer needs efficiently. The analysis focuses on how Adherents are emphasising value for money through centralisation processes such as collaborative procurement tools like framework agreements. The uptake of other efficiency tools such as e-catalogues and e-auctions is also analysed. This chapter included an assessment of the overall digital transformation undertaken by Adherents, in order to improve procurement procedures through monitoring, forecasting and simplifying procurement procedures.

    

The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law.

The Recommendation calls upon Adherents to “implement sound technical processes to satisfy customer needs efficiently” (Principle on efficiency, paragraph VII). The definition of value for money itself has continued to evolve with an increasing emphasis on LCC considerations.

An emphasis on increasing value for money is often found with the centralisation of purchasing activities, enabling governments to reduce administrative red tape and costs while obtaining better terms and conditions through aggregation of purchases. Central Purchasing Bodies increasingly focus on collaborative procurement tools such as framework agreements, with the goal of boosting efficiency through the strategic aggregation of needs. Similarly, CPBs frequently resort to efficiency tools (e-catalogues, e-auctions) and make them available to a large number of contracting authorities.

In parallel, usage of e-procurement solutions is expanding in OECD countries, facilitating data collection by governments. The digital transformation of procurement goes much beyond having good e-procurement systems/platforms in place. At a strategic level some OECD countries have a “digital by design” perspective, where digital technologies are embedded from the start into the design, development, delivery and monitoring of procurement frameworks and processes.

The digital transformation of procurement can also be understood as the government’s ability to treat data as a core asset that can be used to improve procurement procedures through monitoring, forecasting and simplifying procurement procedures. Better availability of procurement data and information underpins a growing emphasis on measuring and managing the performance of public procurement. Finally, proper integration of the public procurement function into overall public finance management can help with achieving value for money (for instance, by providing contracting authorities with budget flexibility through multi-year budgeting) and providing data to assess the performance of purchasing activities. Moreover, e-procurement systems are increasingly connected to public financial management IT Systems (OECD, 2016[1]; 2018[2]).

4.1. Achieving greater efficiency through public procurement

4.1.1. Market efficiency and procurement processes

Market engagement to develop realistic and effective tender specifications

As per the Recommendation, “Adherents should engage in transparent and regular dialogues with suppliers and business associations to present public procurement objectives and to assure a correct understanding of markets” (Principle on participation, paragraph VI). A good understanding of markets is essential if contracting authorities are to develop more realistic and effective tender specifications and provide vendors with a better understanding of the country’s needs. Engaging suppliers at different stages of the procurement process also helps reduce the information asymmetry between the market and the procuring entity. Indeed, suppliers often have more information than the procuring entity regarding their own costs, prices, market trends, products or services, and their substitutes. Early exchanges with suppliers may also maximise participation in the tender procedure, allowing potential bidders the time to prepare their offers.

Early engagement mechanisms – such as publishing Requests for Information (RFI) and Prior Information Notices (PINs); one-to-one consultations with suppliers; or holding industry/supplier days – can help contracting authorities improve the quality of technical specifications (OECD, 2016[3]). According to the 2018 Survey, 73.5% of respondents hold regular dialogues with suppliers and business associations in a variety of institutional settings. In some countries, for example Belgium, Norway and Hungary, business associations or chambers of commerce participate in institutional committees to discuss the procurement system as a whole. In Ireland, the Office for Government Procurement engages with suppliers through around six Meet the Buyer workshops per year.

Some CPBs conduct formal and informal consultations directly with a panel of suppliers. This is the case in Canada, France, Italy and Korea, for instance. Canada and France hold “industry days” (conventions entreprises-acheteurs in France) to allow buyers to exchange directly with suppliers. Canada issues RFIs prior to tenders while in Greece the CPB establishes dialogue with relevant suppliers selected from the Central E-Registry of Public Procurement depending on the goods and services to be procured. In Latvia, contracting authorities advertise pre-tender market consultation meetings on their website. In New Zealand, there are “Meet the Buyer” events earmarked for certain categories of suppliers, such as SMEs (Box ‎4.1). Many countries publish minutes or summaries of early engagement or roundtable meetings with suppliers.

Box ‎4.1. Procurement “speed dating” in New Zealand

“Meet the Buyer” is an event that brings small and medium-sized enterprises (SMEs) together with large purchasing organisations, allowing both parties an informal platform for engagement. The highlight of the event is a series of 15-minute prearranged meetings where small businesses (sellers) get to meet with the large purchasing organisations (buyers) and find out about their upcoming procurement activity and/or present their product/service offerings.

A typical Meet the Buyer setup will also have areas for a mini-expo and presentations where businesses can network with others, speak with exhibitors, and access topical information and tools.

• Meet the Buyer is advertised through various channels: potential suppliers will get to know the details of participating buyers and their interests, overviews or key projects or essential service needs.

• Sellers send in their expressions of interest to meet with particular buyers via a simple web form.

• With a limited number of meetings available, buyers shortlist businesses they want to meet based on their expressions of interest. The shortlisting is done after the period for expressions of interest closes so that the meetings arranged are beneficial to both parties.

The approach has also been used in New Zealand after a Request for Proposals was released to the market in a major All of Government Banking tender where all the suppliers were invited to sit down with government agencies and ask three questions in 15 minutes. A stopwatch was used to ensure exact timings and equal opportunity for each supplier. The event was conducted strictly to ensure a high level of probity was maintained. The suppliers found the face-to-face time with agencies invaluable and it improved the responses from suppliers.

Source: (OECD, 2018[4]); (Clinton, 2017[5]).

Redefining efficiency: MEAT criteria and value for money

The Recommendation states, “Adherents should implement sound technical processes to satisfy customer needs efficiently”, including through “identifying appropriate award criteria” (Principle on efficiency, paragraph VII). Award criteria must be objective, relevant to the subject matter of the contract, and precisely defined in the tender/solicitation documents.

Following the establishment of the 2014 EU Directive on public procurement, the 2018 Survey results show that a majority of respondents are employing MEAT (Most Economically Advantageous Tender) award criteria rather than price only. The MEAT criteria are based on costs and encompass other aspects using a ‘best price-quality ratio’ (e.g. quality of product, organisation, qualification and experience of the supplier, delivery time and conditions, etc.). Tender/solicitation documents available to bidders typically define award criteria, including how they are combined and the relative weight allocated. Percentage or points systems for evaluation criteria can include environmental and social factors, i.e. secondary policy objectives (OECD, 2016[3]).

The 2018 Survey suggests that CPBs respondents often use MEAT award criteria for certain types of purchases. Around a third of them apply MEAT criteria to all purchases. Approximately 63% of CPBs use MEAT award criteria for most purchases (75% of purchases and more). Accordingly, the share of purchases made using price-only criteria for evaluation varies across countries (OECD, 2018[2]).

Adapting the procurement method depending on value

In the Recommendation it is stated that “Adherents should streamline the public procurement system and its institutional frameworks … Where possible, a more service-oriented public procurement system should then be built around efficient and effective procurement processes and workflows…” (Principle on efficiency, paragraph VII).

EU respondents apply the EU public procurement rules from EU Directives 2014/23; 2014/24; and 2014/25. EU public procurement rules prescribing openly advertised tendering and contract award procedures are mandatory above certain financial thresholds, revised once every two years (Table ‎4.1). Below these financial thresholds, different national rules apply to public contracts from EU respondents.

Table ‎4.1. Thresholds triggering EU-wide procurement rules
Type of public contract	Type of contracting authorities	Value
Supply and services contracts and design contests	Central government bodies	EUR 144 000
Supply and services contracts and design contests	Local or regional government bodies, or authorities operating in the defence sector	EUR˜221 000
Supply and services contracts and design contests	Utilities sector (e.g. operators of gas/electricity distribution networks and operators of public transportation services)	EUR 443 000
Works contracts	Any contracting authority	EUR 5 548 000
Note: Thresholds are valid until 1 January 2020 and exclusive of VAT. The European Commission establishes the corresponding values in national currencies other than euros in a separate communication.
Source: Synthesis by Morrison & Foerster LLP, “New Procurement Threshold Values Apply across Europe from January 1, 2018”, Available at: www.lexology.com/library/detail.aspx?g=be3496bd-af6d-48d6-a0fd-fd3b7fe0779d.

Respondents display a wide variety of institutional arrangements and processes regarding low-value contracts. In order to achieve efficient public procurement processes, all respondents apply some kind of simplified procurement rules and regulations under a certain threshold, though with some elements of competitive tendering. For instance, Greece applies a simplified procedure (brief informal tendering) for contracts with a value between EUR 60 000 and EUR 20 000, while direct award to a single economic operator is possible for low-value contracts up to EUR 20 000. Turkey has a similar approach for contracts below its national threshold. In Israel, under a threshold value of around EUR 12 100 (ILS 50 000), public tender formalities are not mandatory. Instead, an administrative code prescribes a competitive process that is faster and simpler than a public tender.

There are strong variations in value thresholds under which contracting authorities can conduct simplified procurement processes. The threshold varies from EUR 12 100 (ILS 50 000) in Israel to EUR 143 650 (ISK 1 072 094) in Iceland.

Korea has a simplified price-based evaluation method through its e-procurement system KONEPS for “smaller-value contracts” for goods and services between EUR 15 665 (KRW 20 million) and EUR 38 935 (KRW 50 million). In the same fashion, Italy operates the MePA (the Public Administration e-Marketplace) for goods, services and maintenance works under the EU thresholds. The e-marketplace provides operational flexibility, allowing for direct awarding from standardised e-catalogues and for requests for quotation.

Most countries allow direct awards of procurement contracts (without advertisement) up to a certain threshold, which is typically much lower than the EU threshold. For instance, in the Netherlands direct award is authorised for contracts below EUR 33 000 and in France for contracts below EUR 25 000, provided that certain conditions are respected. In Korea contracts below EUR 15 665 (KRW 20 million) can be awarded directly without competition. In Canada, the unified set of rules for federal public procurement allows a common exception to competition for contracts with a value of less than EUR 16 623 (CAD 25 000).

To a degree, some countries such as Canada and Estonia allow contracting authorities to set their own practices for procurement where the value is below a certain threshold. For instance, in Canada most federal departments can enter into non-competitive goods and service contracts with a value up to EUR 16 623 (CAD 25 000). Federal departments have latitude to create practices and processes within that framework, even though they are often required to use centralised procurement tools such as standing offers and supply arrangements. In Estonia, contracting authorities can set up simplified regulations for purchases of goods and services between EUR 30 000 and EUR 60 000, and works from EUR 60 000 to EUR 120 000.

Simplified procurement processes for low-value contracts are very often associated with collaborative procurement tools aimed at boosting efficiency and streamline processes for low-value purchases.

4.1.2. Using collaboration for improved outcomes

The Recommendation states: “Adherents should develop and use tools to improve procurement procedures, reduce duplication and achieve greater value for money, including centralised purchasing, framework agreements, e-catalogues, dynamic purchasing, e-auctions, joint procurements and contracts with options” (Principle on efficiency, paragraph VII). Centralisation of purchasing activities has been a major driver of the efficient performance of public procurement systems.

Centralisation of procurement activities and aggregation of needs are observed across an overwhelming majority of OECD countries. CPBs are increasingly established to reap the benefits of aggregated demands and outputs of procurement activities. The benefits of centralised purchasing activities – such as better prices through economies of scale, lower transition costs, and improved capacity and expertise – are widely acknowledged.

Recently there have been developments in the roles of CPBs in OECD countries that reaffirm their strategic role as an efficiency enabler. Central or co-ordinated purchasing is carried out in several ways, from facilitating purchasing through framework agreements to a more direct service involving the aggregated purchasing and warehousing of products. Box ‎4.2 provides a good example of the potential benefits of procurement centralisation through aggregated purchasing in the case of energy purchases for central ministries and agencies in Portugal.

Box ‎4.2. Procurement centralisation for energy purchases in Portugal

In 2007 Portugal began deploying an e-procurement system that supported the introduction of framework agreements as part of its Sistema Nacional de Compras Públicas (SNCP, national public procurement system). In 2017, Portugal went further and centralised the purchasing of energy (electricity, natural gas and fuel) in the hands of its central purchasing body, eSPap. The aim was to obtain more purchasing leverage and optimise purchasing strategies across central government agencies and ministries.

This additional responsibility allowed eSPap to aggregate the demand for more than 800 contracting authorities. eSPap defined a three-year business plan to gradually strengthen its public procurement team and achieve full collaborative centralisation of energy purchases in 2020. According to data for 2017 and 2018, eSPap achieved savings of 14.3%, mainly through lower energy prices (“transactional savings”).

Source: Entidade de Serviços Partilhados da Administração Pública (eSPap), Portugal.

Since 2014, CPBs in an increasing number of OECD countries have established framework agreements. OECD Survey results suggest that framework agreements are increasingly widespread among respondents: at least five have introduced them into their public procurement system from 2014 to 2016. The share of CPBs managing framework agreements thus reached 93% during that year. (Figure ‎4.1). CPBs in OECD countries increasingly focus on strategic aggregation of demand through development and use of procurement tools, including framework agreements and dynamic purchasing systems, to achieve greater value for money.

Figure ‎4.1. The roles of central purchasing bodies

Note: Based on data from 29 respondents (27 OECD countries plus Peru and Costa Rica) that answered both the 2016 and the 2014 surveys on public procurement.

Sources: (OECD, 2016[1]; 2014[6]).

CPBs also have other common roles, such as co-ordinating training for public officials in charge of public procurement (9 countries out of 29, i.e. 31%) and establishing policies for contracting authorities (8 countries out of 29, i.e. 28%) (OECD, 2017[7]).

In 28% of respondents, CPBs act as the policy-making body, and so are responsible for implementing the policies that govern the system. However, data collected by the OECD in 2016 show that only 56.5% of respondents regularly measure implementation of the CPB objectives and 22% do not measure it at all (OECD, 2017[7]). It is indicated in the responses that measurement focuses on the delivery of savings and occasionally involves user satisfaction surveys to measure the success of framework agreements and other services. In Finland for example, the employees of the CPB, Hansel, have their performance bonuses attached to scores from customer satisfaction surveys (OECD, 2019[8]).

Data from the 2016 Public Procurement Survey show that an overwhelming majority of respondents have at least one CPB to conduct central purchasing. Almost all CPBs manage collaborative procurement instruments such as framework agreements and Dynamic Purchasing Systems (DPS) to drive efficiency and cost effectiveness throughout the public procurement system (see definitions in Box ‎4.3). Notable exceptions are Japan, the Netherlands (where there are no CPBs) and Turkey, where the CPB purchases on behalf of contracting authorities but does not manage framework agreements or collaborative procurement instruments.

Framework agreements are a key element of the more general shift towards strategic procurement experienced by many respondents. However, framework agreements or DPS are not suited to the procurement of all goods and services. Relatively homogeneous goods or services that are the object of recurrent purchases across contracting authorities are ideal candidates for these collaborative procurement instruments.

Box ‎4.3. Defining framework agreements and DPS

Framework agreements generally involve the advertisement of an opportunity by a contracting authority, most frequently the CPB. This authority then enters into a contract or other arrangements with one or more economic operators for the provision of works, supplies or services to different contracting authorities over a fixed period. The purpose of the framework agreement is to establish standardised terms and requirements under which contracts are awarded by contracting authorities. The rationale behind the framework method of purchasing is to achieve saving through reduction of transaction costs and by obtaining better terms and conditions from suppliers. Framework agreements sometimes include mini-competitions among suppliers inside the framework agreement.

A Dynamic Purchasing System (DPS) is an electronic system that can be used for repeat standardised purchases and operates like a live, online Internet-based framework agreement, which economic operators can join at any time. The EU public procurement directive 2014/24 refers to dynamic purchasing, while other Adherent countries (United States for federal procurement, Korea) operate broadly similar schemes called multiple award schedules.

The contracting authority advertises the system using an open procedure. Interested economic operators then submit indicative tenders that set out the terms on which they will supply the requirements. All qualified economic operators who submit compliant indicative tenders are admitted to the system. New economic operators can apply to participate and submit indicative tenders throughout the life of the dynamic purchasing system. Contracting authorities wishing to purchase from the system must invite tenders from all economic operators registered in the system. The purchasing contracting authority then places an order with the successful economic operator.

Source: (OECD, 2011[9]); (OECD, Forthcoming[10]).

The CPB typically implements centralised framework agreements on behalf of contracting authorities. The tripartite relationship (CPB – contracting authorities – suppliers) and the benefits provided by framework agreements, depend on the nature of the links between these different entities.

Multiple award schedules (MAS) contracts in Korea offer examples of good practices regarding framework agreements focusing on homogeneous goods or services (Box ‎4.4). The CPB in Korea manages a uniform process that gives contracting authorities appropriate flexibility and illustrates the potential of using e-catalogues to implement framework agreements to enhance efficiency and competition throughout the public procurement system.

Box ‎4.4. Multiple award schedules contracts in Korea

Korea began implementing MAS contracts in January 2005 for end-user public buyers. The Public Procurement Service (PPS) manages these contracts and issues unit-price contracts annually with qualified suppliers. These products and prices are then listed in the Online Shopping Mall, and each end-user can make purchases directly without the need for the direct involvement of PPS contracting staff or the issuance of a new contract. As of December 2014, 326 409 items were contracted with MAS, which accounts for 88.5% of total goods registered in the Korean Online E-Procurement System (KONEPS), totalling USD 53.9 million.

Goods or services targeted for MAS must meet four general criteria. They must have a commercialised specification; allow for contracting via unit price; be supported by a competitive market; and have sufficient demand among end-users. For goods or services that satisfy these criteria, the PPS prepares an announcement for purchasing, and the tender notice is posted to KONEPS.

Compared with traditional procurement, MAS contracts increased the number of suppliers and competition. In order to participate, most suppliers have to satisfy only minimum requirements for satisfactory past performance in at least three instances and have a credit rating above a certain threshold (contract fulfilment capability test). Once MAS contracts are established, the products are available within the KONEPS Online Shopping Mall. It is then the responsibility of each end-user to compare, search and purchase within their needs.

For simple transactions, this process is a straightforward ordering through automated processes within KONEPS. However, in certain cases there is a requirement to conduct a second-stage competition for price and quality within the MAS contracting framework. For orders subject to second-stage competition, the buying entity is required to determine evaluation criteria and request proposals from five or more suppliers.

MAS contracts contain provisions to ensure that the most favourable price is offered to buyers. Suppliers are only allowed to increase their prices in the case of inflation of more than 3%. They are allowed to lower the price of their products at any time.

Source: (OECD, 2016[11]).

The example of Portugal, where the use of framework agreements is mandatory for contracting authorities at the central level, illustrates the way many CPBs in the EU turn to this instrument to rationalise public procurement, impose some minimal standardisation and enhance overall efficiency (Box ‎4.5).

Box ‎4.5. Framework agreements in Portugal

Introduced in 2007, framework agreements through e-procurement were paramount to connect all users of the Portuguese national public procurement system. Framework agreements in Portugal cover a list of goods and services commonly purchased by contracting authorities, and call-offs are mandatory for suppliers.

Contracting authorities in the central administration must use the existing framework agreements to buy items using aggregation processes conducted at central purchasing units at ministerial level (UMC). Contracting authorities have to use the e-platform contracted by the Entidade de Serviços Partilhados da Administração Pública (eSPap, the country’s CPB and shared service centre under the Ministry of Finance) to run call-offs free of charge, therefore allowing the eSPap to better control and monitor the performance of the system. eSPap or the Minister of Finance must approve any exception to the use of framework agreements through a specific web process, upon request from the contracting authority.

Figure ‎4.2. Framework dynamic agreements in Portugal

Whenever possible, the design of the framework agreements takes into account environmental criteria, either for use in public tender awards, or during the call-off stage. Some framework agreements are included in the National Green Procurement Strategy, which eSPap prepared jointly with the Portuguese Agency for the Environment.

Source: (Magina, 2013[12]).

The majority of respondents make it mandatory for contracting authorities at the central level to use framework agreements. Only Korea, the Slovak Republic and Peru make it mandatory for all contracting authorities on all levels of government to use framework agreements. Therefore, the vast majority of respondents that use framework agreements distinguish between contracting authorities at central and sub-central level: sub-national governments and other public entities are free to join on a voluntary basis (Figure ‎4.3). In Colombia the procurement agency Colombia Compra Eficiente, which is part of the planning sector of the government, manages three procurement platforms that are used by national, regional and local governments. These systems are SECOP I, SECOP II and the Tienda Virtual del Estado Colombiano (TVEC, or online marketplace of the Colombian State). TVEC is a platform that holds 35 framework agreements. These framework agreements cover the most common goods and services bought by the public sector, including but not limited to cloud services, call centres, fuel, cars and school meals. Frameworks tend to run for 2-3 years with up to 10-15 suppliers.

Figure ‎4.3. Mandatory vs. voluntary use of framework agreements established by CPBs

Source: (OECD, 2016[1]).

In 2015, an OECD survey provided useful insights on how CPBs use these kinds of agreements:

• The countries surveyed in 2015 devote significant time and effort to demand analysis before implementing or renewing a framework agreement. Indeed, the benefits of developing a framework agreement depend on the nature of demand and supply, on the existence of product alternatives, and on the level of competitiveness of a specific supply market.

• Demand analysis is carried out through systematic consultations with contracting authorities, and frequently with suppliers through interviews and meetings. CPBs also analyse data on historical spending by contracting authorities and by product category – collected through e-procurement systems – and often develop forecasts of estimated demand by product for the coming years. More than half of respondents to the 2015 Survey undertake cost-benefit analysis or feasibility studies to identify whether or not a framework agreement is the most efficient procurement route.

• When setting up framework agreements, it is crucial to provide stakeholders (i.e. contracting authorities and suppliers) with guidance and a clear understanding of the instruments’ main characteristics. All CPBs responding to the 2015 Survey organise informative events for contracting authorities and suppliers to provide targeted information. Indeed, the commitment of contracting authorities is key to increasing coverage and to the success of framework agreements. The Chilean CPB ChileCompra has a dedicated unit for users. It provides help-desk and training services to both contracting authorities and suppliers (OECD, 2017[13]).

• As regards implementation of framework agreements, 50% of respondents to the 2015 Survey have framework agreement systems that do not impose obligations on suppliers to respond to call-offs. Seventy-five per cent of the 2015 Survey respondents indicated that discounts obtained during mini-competitions are based on the initial prices proposed by suppliers through their first submission. CPBs managing framework agreements accept other modifications to initial submissions, whether it is a price increase or a change in the offering. However, most respondents provide a structured framework regulating these modifications, sometimes where suppliers are allocated a maximum number of changes or requests for changes during the framework agreement (OECD, 2017[13]).

Leveraging e-procurement tools for efficiency

Procurement has evolved thanks to progress made possible by technological advances. What was once a paper-based function has been shifting to e-procurement systems for more than 20 years. At the beginning of this transition, the focus in the majority of respondents was on developing e-procurement systems that cover the middle of the procurement cycle, namely from the call for tender until the award of a contract, allowing contracting authorities to manage the rest of the process. However, investment in e-procurement systems has gradually evolved from this original purpose towards developing systems that help increase efficiency and streamline procedures (OECD, 2018[14]).

This gradual drive towards a more transactional e-procurement system means that increasingly, electronic modules are made available to procurement officers to improve workflows, automate processes and eliminate inefficient silos. For instance, a growing share of e-procurement systems encompass business intelligence modules and supplier registries: 62% of respondents now have supplier registries available in some or all of their e-procurement systems, be it central government e-procurement platforms or those of specific contracting authorities. The 2018 Survey data also suggest that e-auction modules are increasingly popular; such tools are available in some or all e-procurement systems of 65% of respondents (OECD, 2018[14]).

E-catalogues are another widespread efficiency tool when it comes to low-value purchases that usually qualify for simplified procurement processes or direct award (Box ‎4.6). They allow procurement officers to order on line, usually with pre-established prices and conditions determined through a framework agreement. Over half of respondents report that their e-procurement systems provide for e-catalogues, through either a specific module or an in-built functionality.

Box ‎4.6. E-auctions and e-catalogues

E-catalogues list available products and/or services that can be viewed and purchased in an electronic format; they can include information such as illustrations, prices and product/service description. E-catalogues can reduce transaction and administration costs, simplify ordering processes and reduce unauthorised purchasing outside permitted systems. E-catalogues need to be interoperable with other platforms.

Reverse auction/e-auctions are online functions that allow economic operators to submit new, downward-revised prices (and/or sometimes revisions to elements of their tenders, e.g. delivery dates) in real time – and in direct, anonymous competition with other economic operators. Unlike a traditional auction, suppliers compete to sell a good or service by bidding to lower the price they originally proposed in their bid submissions. Reverse auctions are therefore different from public tenders that entail only one price submission. E-auctions can be used only when specifications can be established with sufficient precision and should be excluded for certain service and work contracts.

Sources: (OECD, 2016[3]; 2013[15]).

Around a third of respondents make e-catalogues available to procurement officers through their central government e-procurement system. The MePA in Italy provides an example of an advanced e-catalogue managed by the Italian CPB, Consip (Box ‎4.7). It processed 600 000 transactions in 2017.

Box ‎4.7. The e-marketplace for small-value procurement in Italy: The MePA

The MePA (Public Administration e-Marketplace), launched in 2004, is currently operating with an e-catalogue of over 10 million items. It is one of the leading e-marketplaces currently operating in Europe. Managed by Consip, the MePA provides a paperless environment that awards low-value public contracts for goods, services and maintenance works. It is a digital marketplace in which contracting authorities can purchase goods and services offered by qualified enterprises for a value below the EU threshold.

The MePA in fact encourages economic operators to engage with contracting authorities throughout Italy. The e-marketplace provides operational flexibility, allowing for direct awarding from standardised e-catalogues and for requests for quotation.

Contracting authorities can choose among a wide range of goods and services offered by an increasing number of economic operators. The MePA is open to any kind of enterprise that meets the qualification criteria.

How does it work?

Contracting authorities can access the MePA e-catalogue, a user-friendly shop window showcasing the goods and services available; easily compare the prices, features and delivery conditions offered by different economic operators; and then proceed with the purchase according to the chosen procedure – direct order or request for quotations.

The process is in three steps:

• Step 1 – Consip publishes the MePA tenders.

• Step 2 – Economic operators qualify and publish their e-catalogues and offers that are compliant with the tender indications.

• Step 3 – Contracting authorities issue direct orders or negotiate the prices and supply conditions during the request for quotations.

Major benefits MePA offers contracting authorities

• time saving

• transparency and traceability of the entire procurement process

• greater range of products to buy with the possibility of comparing prices and characteristics offered by economic operators from all over the country

• opportunity to satisfy customised needs by means of requests for quotations that identify specific requirements.

Major benefits MePA offers economic operators

• decreasing commercial costs and optimised sales times

• wider access to the public procurement market and opportunity to propose offers throughout the entire national territory

• potential enhancement of the company, despite the small size of the enterprise

• competitiveness and direct comparison with the reference market

• incentive for the renewal of sales processes.

The MePA: A growing instrument in an expanding market

At the end of 2017, for the first time, the transactions carried out on the MePA – about 600 000 – reached a value of EUR 3.1 billion. That figure can increase significantly given the potential market for public spending and the exponential growth of the MePA. Only five years ago, at the end of 2012, the value of annual purchases was only EUR 360 million a year.

The average growth recorded during the past few years was over 50% per year.

Today the MePA is progressively becoming larger, indeed “universal”. In August 2017 Consip began reorganisation of the offer on the MePA, simplifying its structure and at the same time extending the product categories available to users. This reorganisation makes it much easier to foresee new categories of purchase and to enlarge the number of products and services available, encompassing the entire potential universe of purchases below the threshold.

A great deal can be purchased through the MePA

If the amounts for individual purchases on the MePA are less relevant than the big tenders, that should not lead to the conclusion that this is a market of little importance. In fact, throughout the public administration, low-value supply purchases represent, in terms of number, the majority of the total amount of purchases – over 99% of the approximately 4.5 million contracts performed annually – and in terms of value they represent around 20% of the total value of tenders published every year (amounting to over EUR 100 billion).

Moreover, in the maintenance work market, which is worth about EUR 5 billion each year, half of the expenditure is for procedures under the value of EUR 1 million (thus potentially via MePA).

Source: (OECD, 2018[16]).

4.2. Understanding the value of procurement

4.2.1. Realising the value of public procurement data

The Recommendation calls upon Adherents to “Assess periodically and consistently the results of the procurement process”. In order to do so, “Public procurement systems should collect consistent, up-to-date and reliable information…” (Principle on evaluation, paragraph X). The methods used to collect procurement data reported by respondents vary significantly, depending on the degree to which the procurement system is centralised. In all countries though, data collection and procurement databases are intrinsically linked to e-procurement systems – and the rapid development of those systems means that both the availability and the quality of procurement data are gradually improving.

The EC promotes data availability, in particular that of structured data, which is critical for the application of emerging technologies such as Artificial Intelligence and Machine learning, Chatbots and big data tools – all of which require large data sets.

Open and inclusive governing builds trust between citizens and governments, and promotes a transparent and accountable government. Open government also supports a level playing field for businesses, and this contributes to economic development. Transparency is widely regarded as an effective tool for fighting corruption. Implementation of internal controls and regulatory oversight, supported by transparency and civil society’s active participation in the public decision-making process (OECD, 2016[17]), enables effective accountability. To be effective, transparency and accountability systems must be linked, and disclosing information should take account of the quality of what is disclosed as well as the quantity (OECD, 2018[14]). The Italian National Database on Public Contracts (NDPC) provides a good example of how the collection of high-quality structured data can improve the supervision and regulation functions in the field of public procurement, and is indispensable to pilot the evolution of a national public procurement system (Box ‎4.8).

Box ‎4.8. Transparency and traceability in public procurement in Italy

In Italy the Authority for the Supervision of Public Contracts has implemented a National Database on Public Contracts (NDPC). It aims at collecting and processing data on public procurement in order to provide indications to the supervising departments and to inform regulators on measures that need to be taken to promote transparency, simplification and competition. It collects data on IT and conducts market analyses. In particular, it collects and assesses data on:

• The structural characteristics of the public procurement market and its evolution. Statistics on the number and value of procurement awards are grouped by localisation, procurement entities and awarding procedures; the different typologies of procurement are periodically published.

• The criteria of efficiency and value for money during the procurement process. Modifications to contractual conditions are recorded in the authority’s database, which in turn detects dysfunctions and anomalies of the market.

• Dysfunctions and anomalies of the market through fixed measures. These dysfunctions and anomalies are detected through:

  • the assessment indexes of excessive tendering rebates, with respect to the average rebates

  • the number of bids to be presented in each awarding process

  • the localisation of awarded companies with respect to the localisation of the contracting authority.

The Construction Company Database (Casellario Informatico) and the data on declarations filed by the economic operators on the reliance on the capacities of the other entities are parts of the NDPC.

Through the quality of the data made available by the NDPC the authority improved its activities, notably supervision and regulation activity, in order to provide guidelines on measures that need to be taken into account to promote transparency, simplification and competition in the entire procurement process –particularly, in the pre-bidding and post-bidding phases.

Source: (OECD, 2016[18]).

According to an OECD estimate based on the 2018 Survey responses, more than two-thirds of respondents have developed centralised collection of procurement data, at least from central government bodies and agencies. Most countries have comprehensive procurement databases through their central e-procurement system (OECD, 2018[14]).

In several countries reforms are ongoing to improve monitoring and data collection. For instance, Germany is setting up a legal basis for more reliable federal electronic public procurement statistics, while Sweden is working to improve access to its procurement data as well as data quality. Better procurement data is a first step towards improving performance management of public procurement systems. In Finland, the CPB hired a team of data analysts and provided them with business intelligence tools that can be used to analyse large quantities of data (OECD, 2019[8]).

In reviews on fighting bid rigging in public procurement, the following principles are recommended for procurement databases: 1) they should include tender (bidding) as well as contract data; 2) they should be of good quality; 3) they should be in a usable, flexible and searchable format; 4) they should be accessible by those who would benefit from using them, both within and outside the procuring entity (OECD, 2018[19]).

4.2.2. Measuring efficiency to understand value

The development of e-procurement systems is a driving force to boost the efficiency of procurement processes. Some respondents assess the efficiency or savings related to e-procurement. For instance, Estonia measures the time spent by procurement officers on processes before and after changes to e-procurement procedures, and conducts cost-benefit analyses to inform decisions on further developments in the field of information technology (IT). In Italy the CPB commissioned an analysis from external advisors on the benefits from digitalisation of its procurement processes. Instead of measuring efficiencies from e-procurement as a whole, Poland, Latvia and Morocco focus on measuring savings obtained through their e-bidding and e-reverse auction platforms.

Measuring the performance of public procurement

The Recommendation calls upon Adherents to “develop indicators to measure performance, effectiveness and savings of the public procurement system for benchmarking and to support strategic policy making on public procurement” (Principle on evaluation, paragraph X). More than half of the Adherents surveyed report that they analyse procurement information and data to provide insights informing further reforms of public procurement systems.

Only a minority of respondents have a formal performance management system established, with Key Performance Indicators (KPIs) reflecting outcomes and specific targets for each contracting authority. Only 33% of respondents that answered the 2018 Survey have a performance measurement system focused on predetermined targets, and 30% of respondents have an authority with a mandate to manage the performance measurement framework. This suggests that many countries analyse the data and indicators available about the public procurement system in a non-structured, non-systematic fashion. In 45% of countries, information and available data are not analysed to inform strategic policy making on public procurement (Figure ‎4.4).

Figure ‎4.4. Performance management system lacking in most countries

Note: Data gathered from 33 respondents (30 OECD countries plus Morocco, Costa Rica and Peru).

Source: (OECD, 2018[2]).

CPBs are often leaders in developing and monitoring indicators to track procurement performance. Based on data collected from Adherents in 2016, only 56.5% of OECD countries regularly measure the implementation of CPB objectives. Responses indicate that measurement focuses on the delivery of savings and occasionally involves user satisfaction surveys to measure the success of framework agreements and centralised purchasing (OECD, 2016[1]). Such data can be used to plot the performance of CPBs. For instance, to incentivise good service delivery, employees of Hansel, the CPB in Finland, have their performance bonuses attached to scores from customer satisfaction surveys (OECD, 2019[8])

Sometimes CPBs measure their performance against set targets as part of a general results-oriented budgeting framework. This is the case in Canada, where Public Services and Procurement Canada (PSPC) has 18 performance indicators measuring procurement outcomes as part of its Departmental Results framework. Similarly, the National Agency for Public Procurement (NAPP) in Sweden has recurring result indicators from monitoring as part of its budget process, while the government will be in charge of evaluating implementation of the National Public Procurement Strategy.

Other common roles of CPBs include co-ordinating training and providing advice for public officials in charge of public procurement (in 35% of OECD countries) (OECD, 2016[1]). For instance, Hansel in Finland provides advisory services and training to purchasing authorities from the central government. It developed specific performance indicators in this area, including:

• satisfaction from advisory services and training (survey responses from relevant contracting authorities)

• information on procurement processes that have received support from the CPB (spend level, type of procurement)

• resources used in providing advisory services/training (staffing levels, additional costs for providing such services).

Perhaps thanks to the increasing availability of data, several countries are introducing or expanding their set of performance indicators regarding public procurement, or establishing a performance measurement framework. Such is the case with the Slovak Republic. Mexico already monitors procurement indicators for its federal procurement system, and is currently developing a set of indicators for evaluating the performance of federal contracting authorities.

Understanding private sector inputs that improve public services

The government of Canada is leading the development of a government-wide Vendor Performance Management (VPM) framework that will apply to federal procurements. The aim of the framework is to optimise value for money by providing strong incentives for suppliers to perform better and to hold vendors accountable for poor performance. The VPM would include performance metrics and monitoring mechanisms. Similarly, the federal procurement system of the United States (which accounts for the largest procurement spends in the world) is systematically monitoring and tracking supplier performance, through centralised databases on past performance and practices of suppliers; the information is shared across all federal departments and agencies (Box ‎4.9).

Approaches to the past performance of suppliers in public procurement differ among countries. Indeed some OECD countries such as the United States and Korea use suppliers’ past performance as award criteria. In Korea, the e-procurement system collects past performance data from private sector business organisations through automated data exchange (OECD, 2016[11]). On the other hand, in the European Union, contracting authorities can exclude candidates or tenderers from participation if, among other grounds, their performance in earlier public contracts has shown major deficiencies. Such exclusion is normally temporary, e.g. up to a maximal duration defined in national public procurement regulations (European Union, 2014[20]).

Box ‎4.9. Supplier performance information in the United States

In working to build the right supplier relationships, the United States focuses on doing business with contractors who place a premium on integrity, performance and quality. To this end, government agencies have been directed to improve the quantity, quality, and utilisation of supplier performance information using two systems.

Supplier past performance information, including identification and description of the relevant contract, ratings across six dimensions (quality, schedule, cost, utilisation of small business, etc.) and a narrative for each rating, is contained within the Past Performance Information Retrieval System (PPIRS, www.ppirs.gov). Government agencies are required to report past performance information on this system, which will then be available to other contracting officers within PPIRS on all contracts and orders above USD 150 000.

That web-based, government-wide application provides timely and pertinent information on a contractor’s past performance to the federal acquisition community for making source selection decisions. Federal regulations require that customers complete report cards detailing a contractor's past performance annually during the life of the contract, and the PPIRS provides a query capability for authorised users to retrieve those report cards. The PPIRS consists of several subsystems and databases (e.g. Contractor Performance System, Past Performance Data Base, and Construction Contractor Appraisal Support System).

The Federal Awardee Performance and Integrity Information System (FAPIIS) captures additional information regarding supplier performance and business integrity issues, including contracts terminated for default and information about criminal, civil, or administrative procedures related to a federal contract.

Source: (Office of Federal Procurement Policy, 2013[21]).

Key performance indicators (KPIs)

It has been suggested that the global fiscal gap of USD 3.3 trillion could be addressed by 2021 if public spending was better managed and followed the practices of best performing countries. In the area of health care, it was suggested that by spending existing funds more efficiently, health life expectancy could be increased by 1.4 years (McKinsey & Company, 2017[22]). Public procurement’s impacts are widespread, yet measurement frameworks are unable to demonstrate the benefits or drawbacks of procurement policies. Where multiple government policies target the same or similar objectives, cross-government measurement frameworks can help to maintain a view of impact at the central level (OECD, 2019[8]).

Adherents often develop KPIs to measure performance, effectiveness and savings. KPIs are also a powerful tool to benchmark contracting authorities and to monitor their performance over time. The Checklist for Supporting the Implementation of the Recommendation (OECD, 2016[3]) suggests contracting authorities:

• benchmark, for instance by comparing their own operation with a similar contracting authority

• compare monitoring indicators against predefined performance targets that are relevant, attributable, well defined, timely, reliable, comparable and verifiable.

There is a considerable diversity across countries in the definition and use of KPIs in public procurement. CPBs often use specific KPIs to measure how efficiently they manage framework agreements and whether they deliver for their customers (purchasing organisations).

The 2018 Survey results indicate that the most widespread indicators among Adherents are economic measuring savings and inputs such as costs and time spent on procurement processes (Table ‎4.2). Savings is a commonly used indicator of performance among respondents. Savings can be tailored to measure benefits from framework agreements, from an e-procurement system, or from a procurement process simplification, for instance.

Table ‎4.2. Savings and inputs indicators
Savings and inputs indicators
Price savings	Difference between prices obtained through procurement and a reference price (average price of bids, maximal allocated budget, prices established through market research/budget intelligence tools)
Cost and time of procurement processes	Time taken (and any associated overt costs, not including employee salaries) by government personnel, including non-procurement roles, to undertake procurement activity
E-procurement inputs	Direct costs for purchasing, upgrading or maintaining e-procurement system; personnel costs associated with system management and maintenance
E-procurement time savings	Assessment of time taken for contracting authorities and businesses to conduct tender procedures with and without use of different digital procurement functions
Cost and time reduction resulting from process simplification	Measurement of time taken by government and business personnel to complete tender procedures both before and after efforts to improve or simplify processes (e.g. use of model contracts)
Sources: (OECD, 2019[8]); (OECD, 2016[1]); (OECD, 2018[2]).

Adherents also frequently measure economic indicators regarding supplier participation and competition in tenders, including transparency (share of open tenders, share of tenders advertised on line) and the coverage of competitive processes (Table ‎4.3).

Table ‎4.3. Supplier participation and transparency indicators
Transparency in government contracting	Proportion of government tender documents that are shared openly in a format allowing review and analysis
Business perceptions on cost and time of participating in government tenders	Survey responses, including quantitative results, on time taken (and resources engaged) in responding to government tenders
Share of open tenders/competitive procurement processes	Proportion of government tenders that use open procedures as opposed to restricted or closed tenders
Average (median) number of bids (responsive bids) per open tender	Measures the degree of competition in open tenders or in competitive procurement processes
Supplier concentration	Measures the extent to which a small group of suppliers account for a large share of the overall purchase value from a contracting authority
Coverage of e-procurement or e-tendering systems	Percentage of procurement processes (or of overall procurement value) conducted through e-procurement processes
Sources: (OECD, 2019[8]); (OECD, 2016[1]); (OECD, 2018[2]).

A third category of economic indicators focuses on the post-award phase of the procurement cycle: contracting authorities typically measure payment delays or the time from invoicing to payment, and various dimensions of supplier performance (compliance with contractual delivery time, reliability, quality of maintenance or associated services). In the United States, the federal government developed a dedicated information system to track supplier performance information.

Finally, respondents develop and monitor specific indicators related to secondary policy objectives, i.e. Green Public Procurement (GPP), social issues, SME participation, and innovation (Table ‎4.4). Respondents develop these indicators based on policy priorities. For instance, many respondents record the outcomes of GPP practices and gather data on the achievement of GPP targets (OECD, 2015[23]). In this regard, the use of award criteria based on life cycle costing is a hybrid, as life cycle costing can bring both economic (lower energy bill/maintenance costs) and environmental benefits (lower energy consumption).

Table ‎4.4. Sustainability and social indicators, including SME access to public procurement
Environmental impacts	Use of whole of life costing	Ratio, value and number of contracts awarded following a procedure containing life cycle costing award criteria
	Reduction in energy consumption	Comparison between energy consumption of historical goods and services bought by government and that of new goods and services selected using MEAT or other criteria
	Reduction of CO2 emissions	Comparison of CO2 emissions from historical goods and services bought by government and those of new goods and services selected using emissions as criteria
	Improvement in air/water quality	Comparison between impacts on air/water quality of historical goods and services bought by government and those of new goods and services selected using environmental considerations as criteria
Social impacts	Stakeholder perception and involvement	Survey responses from different segments of society (e.g. businesses, civil society, NGOs) related to public procurement
	Use of social criteria in government contracts	Ratio of public contracts pursuing social objectives (and where possible, aggregation of social outcomes secured through public contracts)
	Skills/jobs creation	Number of jobs/training courses/qualifications generated through public procurement (note: specifically generated through contract clauses)
Innovation and SMEs	SME success	Ratio of SMEs that are successful in government tenders, and number and value of contracts awarded to SMEs
	Innovative procurement	Ratio of goods and services purchased that meet innovation criteria (e.g. purchased through PCP, first introduction into domestic market, etc.)
Source: (OECD, 2019[8]), (OECD, 2016[1]), (OECD, 2018[2]).

The use of environmental performance indicators such as energy consumption, CO₂ emissions and air/water quality in public procurement appear to be a developing area. In Finland, application of a framework for measuring “procurement productivity” has highlighted the positive impacts of public procurement, such as procuring innovation. Further refinement of the measurement indicators will bring tangible benefits to the economy in terms of value that can be quantified in countries such as Finland, where 5% of the tenders are innovative (OECD, 2019[8]).

Assessing the efficiencies of collaborative instruments and centralisation

Framework agreements yield a number of benefits, such as generating savings through the strategic aggregation of needs, reducing red tape and streamlining processes. Therefore, most respondents assess the coverage of framework agreements in specific product categories. The higher the coverage, the greater will be the potential for generating savings from a consolidation of demand.

Respondents also measure savings from framework agreements, and more broadly from centralising procurement through a CPB. The method of calculating savings differs across countries. Evaluating the financial performance of framework agreements can be achieved from at least two different perspectives: exogenous or endogenous.

Exogenous performance is measured against either prices paid outside the framework agreement or historical prices. For instance, in Chile the CPB ChileCompra compares prices obtained within its framework agreements to the average price provided by three suppliers via decentralised procurement (Box ‎4.10). In France, savings from framework agreements are often calculated against historical prices (OECD, 2017[13]). The federal government in Mexico generally estimates savings in framework agreements by comparing reference prices (established through market research) to the prices obtained through the framework contract.

Box ‎4.10. Savings from framework agreements in Chile

The case of Chile illustrates how framework agreements can generate savings from the consolidation of demand. Chile introduced them in 2003 and the CPB, ChileCompra, carried out the implementation, award and management of these agreements. The procurement law mandates the use of framework agreements, which has been further supported by investment in the national e-procurement system (ChileCompra Express) as the vehicle for their use. From 2014 onwards, there has been a consistent upward trend in usage for some product categories, such as data centres and associated services. As a result, Chile achieved substantial savings from both centralisation and the introduction of framework agreements.

Table ‎4.5. ChileCompra total and average savings amounts
	2015	2016	2017
Savings average	11.7%	19.5%	21.2%
Total amounts transacted (USD million)	2 197	2 661	2 999
Total amounts saved (USD million)	257	518	635
Note: “Savings average” is the sum of savings divided by the sum of transaction amounts.
Source: Based on information provided by ChileCompra.

ChileCompra calculates price savings based on the difference between the prices proposed by bidders awarded under framework agreements, and the average price proposed by at least three suppliers outside the procurement instrument. In addition, increasing framework agreement coverage (in terms of categories of goods and services) also generated process savings. Process savings are estimated from the difference between costs borne by contracting authorities related to the issuance of a purchase order from one of ChileCompra’s framework agreements, and the costs generated by the issuance of a public tender or direct award procedure. According to ChileCompra, process savings amounted to USD 18.6 million in 2017, or 0.62% of the overall transaction amount.

Source: (OECD, 2019[8]).

Endogenous performance involves evaluating the financial benefits achieved within framework agreements. Where a mini competition exists, many countries calculate savings using the difference between the initial budgeted price and those offered by suppliers after the mini competition. For example, Greece and Luxembourg use this methodology to estimate savings from centralised purchasing.

A third method to compute savings, particularly for construction works, is to measure the costs following tendering compared to costs estimated at the design phase (reference). The French Direction des achats de l’Etat (DAE) and the Korean Public Procurement Service (PPS) use this methodology to measure savings concerning construction works. The methodology can include a correcting mechanism where there is an erroneous estimate: if actual prices are more than 20% lower than the reference, the DAE computes savings based on the average price of all bids submitted.

Beyond savings, CPBs that manage framework agreements often measure and monitor customer satisfaction by contracting authorities. It is essential to receive feedback from users of framework agreements. Indeed, for a majority of respondents it is mandatory for contracting authorities at the central level to use framework agreements to survey users. For instance, in both Finland and Chile, CPBs survey individual users of framework agreements from contracting authorities (OECD, 2019[8]).

Insufficient evaluation of procurement policies and systems

In order to assess and benchmark the performance of their public procurement system, respondents are encouraged to “undertake specific evaluation at the national level, […] through national institutions such as supreme audit institutions (SAI) or international assessment tools like the MAPS (Methodology for assessing procurement systems)” (OECD, 2016[3]). The MAPS is a universal tool that aims to catalyse and accelerate the implementation of modern, efficient, sustainable and more inclusive public procurement systems in all countries. In addition, the OECD made available a detailed checklist as a self-assessment tool to guide and support public procurement practitioners in reviewing and revising their public procurement framework, according to each of the 12 principles of the Recommendation (OECD, 2016[3]).

According to data from the 2018 Public Procurement Survey, 45% of the countries surveyed carried out an evaluation (as opposed to monitoring) of public procurement reforms or specific public procurement policies in recent years. Some respondents, such as Germany and Morocco, reported not yet having carried out evaluations because of recent changes to the public procurement laws and regulations. Turkey carried out a specific evaluation to measure the impact of its complaints system on public procurement processes. The 2018 Survey suggests that more countries intend to conduct an evaluation of their public procurement system in the future.

Evaluation of procurement systems appears to be carried out in different ways and to differing degrees by respondents, particularly at the national level. One reason is that such evaluations rely on procurement metrics and indicators being in place to provide insights into trends over time. The lack of data on public procurement systems has been an obstacle for Adherents in this regard. However, the increasing availability of procurement data in recent years means that respondents are better equipped to conduct insightful evaluations of their procurement systems than previously. The OECD has been contributing consistent and evidence-based data on procurement systems in Public Procurement Reviews, developing MAPS, and conducting MAPS assessments. MAPS contains a checklist of quantitative indicators, which -- along with its legal and policy dimensions – can help in evaluating and benchmarking procurement systems and identifying areas for improvement (Box ‎4.11).

Box ‎4.11. MAPS quantitative indicators to assess public procurement systems

Suppliers

• Number of registered suppliers as a share of the total number of suppliers in the country (as a percentage).

• Share of registered suppliers that are awarded public contracts (as a percentage of the total number of registered suppliers).

• Total number and value of contracts awarded to domestic/foreign firms (and as a percentage of total).

Audit and risk management

• Number of courses conducted to train internal and external auditors in public procurement audits.

• Share of auditors trained in public procurement (as a percentage of the total number of auditors).

• Share of internal and external audit recommendations implemented within the time frames established in the law (as a percentage).

Competition and submission of bids

• Value of contracts awarded through competitive methods (most recent fiscal year).

• Average time to procure goods, works and services: number of days between advertisement/solicitation and contract signature (for each procurement method).

Contract management and payment

• Time overruns (as a percentage; and average delay in days).

• Contract amendments (as a percentage of the total number of contracts; average increase of contract value as a percentage).

• Quality control measures and final acceptance are carried out as stipulated in the contract (as a percentage).

• Invoices paid on time (as a percentage).

Source: (MAPS, 2018[24])_].

Measuring the productivity of an entire procurement system requires analysis of a number of factors beyond performance metrics and indicators: a number of qualitative factors must also be taken into account. These are labelled “enablers/conditions”, as they represent the many considerations that affect the performance of the procurement system. Surveyed CPBs often target these considerations in order to improve the performance procurement systems (Box ‎4.12).

Box ‎4.12. Assessing national procurement systems

Scholars have developed increasingly sophisticated models to measure the outputs of procurement exercises, such as efficiency/cost, total cost of ownership, on-time deliveries, accuracy, quality, innovation, sustainability, internal customer satisfaction, and professionalism.

However, measuring the performance of an entire procurement system is undoubtedly more complex than measuring the benefit from a single tender or from acquisitions from a single purchasing authority. A number of additional factors must be taken into account, on top of standard performance indicators.

Figure ‎4.5. Structured performance assessment of a national procurement system

Source: (OECD, 2019[8]), (Phillips, 2018[25]).

Evaluations of national procurement systems increasingly encompass secondary policy objectives, such as GPP, support for SMEs and innovation. In 2016, while most countries did have some measure of the impact of their Green procurement strategy, just under half of them conducted evaluations of measures aimed at supporting SME access to public procurement. Only a quarter of respondents had ever assessed whether their policies supported the acquisition of innovative goods and services (OECD, 2016[1]).

4.2.3. Managing procurement dividends

The Recommendation calls upon Adherents to “rationalise public procurement spending by combining procurement processes with public finance management...Budget commitments should be issued in a manner that discourages fragmentation and is conducive to the use of efficient procurement techniques” (Principle on integration, paragraph XIII). This also means that Adherent countries should use the information from public finance management systems to improve procurement management, reduce duplication, and deliver goods and services more efficiently. Integration with public finance management can help ensure proper monitoring of public procurement spending by internal auditors as well as by independent oversight auditors (OECD, 2016[3]).

As supported by the Recommendation (Principle on integration), most respondents conduct public procurement planning in line with budget planning (Figure ‎4.6). The Recommendation encourages multi-year budgeting to optimise the design and planning of the public procurement cycle – as long as it is justified – to enhance efficiency. In Turkey for instance, a specific budget programme must be approved for any purchase for a period exceeding one year, with strict rules regulating the allocation of expenditures (e.g. the appropriation contemplated for the first year shall not be less than 10% of the project cost). In Lithuania, long-term cycle procurement must be planned in line with strategic multi-year planning documents.

Figure ‎4.6. Integration of public procurement with public finance management

Percentage of “Yes”

Note: Data gathered from 32 respondents (29 OECD countries plus Morocco, Peru and Costa Rica).

Source: (OECD, 2018[2]).

More than half of the respondents (53%) reported also reviewing and analysing public procurement as part of public financial management performance (Figure ‎4.6). In many countries, public procurement is part of broader government financial audits and control procedures typically conducted by internal audit services or/and supreme audit institutions. In Slovenia for example, the Budget Supervision Office conducts budgetary inspection and performs pre-accreditation reviews for any expenditure funded by EU funds, whereas the Court of Audit of the Republic of Slovenia acts as the independent supreme audit institution. Both are competent regarding public procurement spending. In Canada, the Treasury Board of Canada Secretariat is responsible for the Management Accountability Framework (MAF), an annual assessment of management practices and performance in most federal departments.

In addition, in some countries the parliament reviews and analyses the public procurement function as part of results-oriented budgeting. This is the case in Canada, where Parliament examines performance of the CPBs through Departmental Results Reports that feature specific performance indicators (Box ‎4.13).

Box ‎4.13. Public procurement as part of public financial management performance: The example of Canada

Public procurement performance is reviewed and analysed in various ways. The two main ways are:

Departmental expenditure plans

Departmental expenditure plans consist of two documents: Departmental Plans (DP) and Departmental Results Reports (DRR). DPs are expenditure plans for each appropriated department and agency (excluding crown corporations). They describe departmental priorities, strategic outcomes, programmes, expected results and associated resource requirements. DRRs are individual department and agency accounts of actual performance against the plans, priorities and expected results set out in their respective past DPs. DPs and DRRs are tabled in Parliament and inform parliamentarians of the results achieved by government organisations. Performance indicators reported to Public Services and Procurement Canada (PSPC) in this regard include:

• overall level of federal departments’ and agencies’ (i.e. contracting authorities) satisfaction

• percentage of PSPC contracted value awarded through competitive processes

• cost of procurement services per CAD 100 of contracts awarded by PSPC annually.

Management Accountability Framework (MAF)

The MAF is a framework for management excellence, accompanied by an annual assessment of management practices and performance in most departments and agencies of the government of Canada. The MAF is a key tool of oversight that is used by the Treasury Board of Canada Secretariat (TBS) to help ensure that federal departments and agencies are well managed and accountable, and that resources are allocated to achieve results. The MAF establishes expectations for sound public sector management practices and performance. One area of assessment is the “Management of Acquired Services and Assets”, which includes procurement.

Source: (OECD, 2018[2]).

References

[5] Clinton, N. (2017), Procurement Week Day 2 - and a Fascinating Story from New Zealand Public Procurement - Spend Matters UK/Europe, http://spendmatters.com/uk/procurement-week-day-2-fascinating-story-new-zealand-public-procurement/ (accessed on  2019).

[20] European Union (2014), Directive 2014/24/EU of the European Parliament and the Council, https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32014L0024&from=EN (accessed on 13 February 2019).

[12] Magina, P. (2013), “The national public procurement system of Portugal”, Presentation to the MENA-OECD Network on public procurement (unpublished).

[24] MAPS (2018), Methodology for Assessing Procurement Systems, http://www.mapsinitiative.org/methodology/MAPS-methodology-for-assessing-procurement-systems.pdf.

[22] McKinsey & Company (2017), Government Productivity Unlocking the $3.5 Trillion Opportunity, https://www.mckinsey.com/industries/public-sector/our-insights/the-opportunity-in-government-productivity (accessed on  2019).

[8] OECD (2019), Productivity in Public Procurement: A Case Study of Finland: Measuring the Efficiency and Effectiveness of Public Procurement, OECD, Paris, http://www.oecd.org/gov/public-procurement/publications/productivity-public-procurement.pdf.

[2] OECD (2018), Data from the 2018 OECD Survey on the Implementation of the 2015 OECD Recommendation on Public Procurement.

[19] OECD (2018), Fighting Bid Rigging in IMSS Procurement: Impact of OECD Recommendations, http://www.oecd.org/daf/competition/IMSS-procurement-impact-OECD-recommendations2018-ENG.pdf.

[14] OECD (2018), Mexico’s e-Procurement System: Redesigning CompraNet through Stakeholder Engagement, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264287426-en.

[4] OECD (2018), Second Public Procurement Review of the Mexican Institute of Social Security (IMSS): Reshaping Strategies for Better Healthcare, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264190191-en.

[16] OECD (2018), SMEs in Public Procurement: Practices and Strategies for Shared Benefits, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264307476-en.

[7] OECD (2017), Government at a Glance 2017, OECD Publishing, Paris, https://dx.doi.org/10.1787/gov_glance-2017-en.

[13] OECD (2017), Public Procurement in Chile: Policy Options for Efficient and Inclusive Framework Agreements, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264275188-en.

[3] OECD (2016), Checklist for Supporting the Implementation of the OECD Recommendation of the Council on Public Procurement, http://www.oecd.org/governance/procurement/toolbox/search/checklist-implementation-oecd-recommendation.pdf (accessed on  2019).

[1] OECD (2016), Data from the 2016 OECD Survey on public procurement (partially published), https://qdd.oecd.org/subject.aspx?Subject=GOV_PUBPRO_2016.

[17] OECD (2016), Putting an End to Corruption, http://www.oecd.org/tax/transparency/.

[11] OECD (2016), The Korean Public Procurement Service: Innovating for Effectiveness, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264249431-en.

[18] OECD (2016), Towards Efficient Public Procurement in Colombia: Making the Difference, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264252103-en.

[23] OECD (2015), Going Green: Best Practices for Sustainable Procurement, http://www.oecd.org/gov/public-procurement/publications/Going_Green_Best_Practices_for_Sustainable_Procurement.pdf.

[6] OECD (2014), Data from the 2014 OECD Survey on public procurement (Unpublished survey).

[15] OECD (2013), Fighting Bid Rigging in Public Procurement in Mexico, https://www.oecd.org/daf/competition/MexicoISSSTEBidRiggingENG.pdf (accessed on 22 November 2018).

[9] OECD (2011), “E-Procurement”, SIGMA Public Procurement Briefs, No. 17, OECD Publishing, Paris, https://dx.doi.org/10.1787/5js4vmnvwp8w-en.

[10] OECD (Forthcoming), Public Procurement in Kazakhstan.

[21] Office of Federal Procurement Policy (2013), Improving the collection and use of information about contractor performance and integrity, https://www.acq.osd.mil/dpap/policy/policyvault/USA001513-13-DPAP.pdf (accessed on 20 July 2018).

[25] Phillips, L. (2018), Improving the Performance of Sub-national Governments through Benchmarking and Performance Reporting,, https://doi.org/10.1787/22265848.

4. Built for purpose: Towards a more efficient and effective public procurement system