8. Report with recommendations on benchmarking the performance of water utilities
Benchmarking – and some level of transparency more generally – can foster consolidation of water utilities in Lithuania. The chapter explores how mere cost comparison could be supplemented by the benchmarking of broader service performance measures and that of business plans and business planning processes. The latter in particular ensures that a range of consolidation options have been explored and duly assessed.
This Chapter is Output 8 from the OECD project aimed at supporting the preparation of a roadmap for the consolidation of the water utility sector in Lithuania. As was noted in the earlier OECD Outputs,1 further consolidation is viewed as a necessary feature of being able to deliver a sustainable and socially acceptable financing strategy for the future provision of water and sanitation services (WSS) in Lithuania.
The focus of this report is the scope to further develop WSS benchmarking arrangements in Lithuania in ways that can be expected to support the securing of efficiency improvements, including through consolidation. In line with this, some options for extending and enhancing the use of benchmarking are identified below in the light of relevant international experience. The approach adopted below is to focus attention on circumstances that apply - and the current and emerging challenges faced - in WSS provision in Lithuania, with international experience drawn upon selectively to illustrate potential options that look to merit particular attention.
Benchmarking can be focussed on different areas of activity, and the issues and options that are relevant to consider can vary between those different areas. This report considers the following types of benchmarking in turn:
The final section of the report provides a series of recommendations drawing on the various assessments that are made throughout.
The use of cost assessment approaches in the determination of allowed charges is central to the economic regulation of utilities. The use of these approaches can be highly relevant to consolidation incentives in Lithuania because they can provide a basis for the economic regulator to constrain the funds that companies are allowed to recover from their customers over time. The application of such constraints has the potential to incentivise companies to find ways of achieving efficiency improvements, including through consolidation approaches, in order to improve the financial circumstances they face, and can expect to face in future years.
The benchmarking of costs is an important tool that is available to regulators, and that can help with the identification of what cost constraints it may be reasonable and appropriate to apply, and of what an efficient level of costs should be regarded as in relation to a given area. Benchmarking can also be used to help promote a degree of comparative competition between utilities with different supply areas over time.
Regulators often apply benchmarking approaches at different levels of aggregation that can include:
Totex benchmarking: i.e. the benchmarking of total opex + capex requirements.
The benchmarking of ‘base’ totex (or ‘Botex’): i.e. the benchmarking of total opex + capex, excluding expenditure on enhancements, such as the achievement of water quality improvements.
Totex, Botex or Opex benchmarking focused on particular business units/activities: e.g. water treatment, treated water distribution, wastewater collection, wastewater treatment.
The benchmarking of the costs associated with more narrowly defined activities: e.g. pipe replacement costs, billing and customer support costs, etc.
There is typically benefit in adopting a pragmatic approach, considering multiple aggregation possibilities, depending on the availability and reliability of relevant data, and the complexities associated with drawing reasonable comparisons.
International experience has highlighted some of the complexities that can be faced when seeking to compare costs between areas when seeking to develop reasonably robust benchmarking models, and some frameworks that can help guide the development of cost modelling.2 However, in practice, those complexities tend to be magnified when a set of smaller companies stand to be considered, as they do in Lithuania. That is, the average costs of larger, regional companies will reflect the fact that a portfolio of supply areas are being served, such that the higher costs that may be associated with serving some localities within a region remain not directly visible as they may be off-set by lower costs in some other areas. The modelling of the costs of larger, regional companies therefore needs only to be concerned with cost drivers where there is a material net difference between the companies under consideration once their overall portfolios of supply areas is taken into account. In a more fragmented context, of the kind faced in Lithuania, a segmented approach – which begins by grouping utilities into broadly comparable sets - looks likely to be much more appropriate than efforts towards broader comparative modelling.
That said, the similar tensions can clearly arise in the identification and specification of the criteria that should define what differentiates between segments: some definitions may tend to favour utilities in some areas – by making them appear relatively lower cost – while disadvantaging others. There tend to be no quick solutions to this kind of complex comparison problem, and the suitability of different segment definitions may vary materially depending on the relevant circumstances. Given these difficulties, there can be considerable benefit from focusing on seeking to establish effective processes through which the scope for different comparative bases for segmentation and assessment can be raised and tested, by – and in consultation with – stakeholders. In line with the comments below on the broader use of performance benchmarking, an important policy-level issue here concerns the extent to which utilities expect their future performance to be assessed through benchmarking in ways that could have material financial implications. Where that is the case, utilities may have significant incentives to seek to engage in processes aimed at developing and refining benchmarking approaches in order to determine how best –rather than whether – cost benchmarking may be applied.
Attention so far has been focused on performance in relation to costs. While this is central to economic regulation, regulators typically also put considerable effort into providing for broader performance assessments, and associated incentives. One reason for this is simply that there are a broader range of measurable aspects of performance that can be expected to have significant relevance for the overall outcomes that are delivered for customers and environment. The monopoly nature of WSS services can mean that unduly limited attention would be given to these factors in the absence of some form of regulatory pressure, and that customers have limited access to information that can help them identify and compare the cost and quality of the services they are required to pay for. Where bill increases are required, this kind of lack of transparency and accountability can underpin significant customer acceptability problems, and make it more difficult to articulate – in credible ways – why bill increases should be viewed as justified, and as delivering demonstrable improvements.
An important additional consideration here concerns the risks of focusing incentive regulation on costs (benchmarked or otherwise) in a relatively narrow way. A standard concern in incentive regulation is that cost pressures may be resolved (deliberately or otherwise) through some form of ‘under-delivery’. That is, one way in which a company may be able to out-perform a price control settlement (or lessen the extent of financial underperformance that might otherwise result), is to simply deliver less. This could manifest itself is through cost savings being made in ways that tend to undermine some aspects of service quality, and the risk of this has tended to be an important factor in the attention regulators in wide range of jurisdictions and sectors have given to the identification of service quality measures that can then be monitored alongside (or as part of the mechanics of) price control arrangements.
The transparency of performance information is a key consideration here, and the approaches that are adopted to providing for transparency – and, more broadly, for stakeholder engagement – provide an important part of the way that regulators typically seek to encourage performance improvements and guard against the deterioration of performance. The following section sets out some of the different ways in which transparency can help generate better outcomes from regulatory processes, before describing a particular example – the approach used by ERSAR in Portugal – that looks well suited as a relevant reference point against which potential developments to the transparency arrangements in Lithuania could be considered.
8.3.1. Recognising the scope of potential transparency benefits
Transparency requirements have been used as an important tool by many regulators internationally, and can help promote improvements in a wide range of ways, including by:
1. Improving company, and company owner, awareness of how performance compares with that of others in terms of those measures that are made available, and of what ‘good’ might look like.3 This may, in and of itself, help to motivate desirable change by ‘shining a light’ on relevant disparities in relation to features of performance that may otherwise be receiving relatively limited attention (given other prevailing company and company owner priorities).
2. Improving customer, and other stakeholder awareness of the comparisons that are made available. This can increase the scope for customers and other stakeholders to challenge companies, and local governments, on their performance in ways that may create desirable pressures for improvement.
3. Increasing the quality and sophistication of performance comparisons that can be made (which can in turn magnify the impact of (1) and (2)). Important underlying issues here typically include improvements to the development of standardised ways in which information must be compiled, and made available. This can have a range of different dimensions, including because:
With more comparative information being made available, companies can face strong incentives to seek to ensure that comparisons are made on a reasonable basis, in a context where observed performance differences for some measures may relate closely to differences in relevant underlying circumstances (such as the density of the population that different companies serve). That is, a context where there may be greater scope for undesirable inferences to be drawn from available comparative information can result in greater effort being put into refining the basis upon which it is viewed as reasonable to make such comparisons, which can provide a more robust basis for subsequent regulatory assessments.
Transparency arrangements typically raise important questions over how potentially complex and extensive information on different aspects of company performance can be communicated in more accessible ways. In line with this, regulators often put considerable effort into the development of standardised and streamlined performance reports that can provide a relatively simple means for customers and other stakeholders to get a high-level view of WSS company performance across some key areas of interest (further comments on how this might be done are included below in the discussion of the Portuguese ERSAR example).
4. Extending the ways and enhancing the effectiveness with which the regulator can seek to use comparative information in its price review determinations, and its associated development of incentive arrangements.
5. Improving customer and other stakeholder awareness and understanding of the trade-offs faced in relation to the sector, and improving the credibility of company and other communications related to those trade-offs (because those communications sit within a broader framework of information provision and challenge). This can provide a basis for better informed and more credible engagement with water customers in ways that can improve the likely acceptability of bill increases where that can be shown to be necessary for the delivery of valued improvements.
It is important to note that the information under discussion here concerns different aspects of the performance of monopoly public service providers. While there is likely to be some relevant performance information that it is appropriate to treat as confidential (for example, for security reasons), experience from other countries clearly shows that substantial levels of performance information can be made available while at the same time taking appropriate account of relevant confidentiality concerns. This is the case even where companies are privately owned (as in England), notwithstanding the potential for this to raise additional types of commercial confidentiality concerns.
Given the public service nature of WSS companies, and the broad range of benefits that can be associated with transparency requirements, there looks to be a strong case for adopting a presumption that the regulator is able to introduce transparency requirements, other than where companies are able to provide compelling reasons as to why that would not be appropriate.
In line with the above comments, there may be significant benefits associated with enhancing the transparency of – and the accessibility of, and prominence given to – WSS company performance information. ERSAR provides a useful example of what that kind of enhancement of transparency might look like, and its performance benchmarking arrangements are summarised below.
8.3.2. ERSAR as a helpful reference point
The approach to WSS quality of service regulation that has been developed and applied by ERSAR, the Water and Waste Services Regulation Authority in Portugal, looks to be particularly well suited as guide for considering ways in which it may be desirable for the Lithuanian service performance arrangements to be developed. It is notable, that the ERSAR quality of service arrangements are applied in a context where there are currently 263 water supply utilities, and 266 wastewater management utilities, all state or municipality owned.4 ERSAR has described the goals of its quality of service regulation as being to:
Protect the interests of users regarding the quality of service provided.
Compare results between entities through benchmarking.5
Consolidate a culture of providing information that is: concise, credible and easy to understand.6
These goals look to fit well with the circumstances faced in Lithuania, and the approach to quality of service regulation that ERSAR applies – which relies on the development of ‘soft’, reputation-based incentive – could provide a valuable complement to current regulatory activity. While it is notable that regulators in some other jurisdictions (including in England and Wales, and Scotland) have applied financial incentives to service performance metrics, the use of such approaches can generate further risks of unwanted effects arising, and the relatively limited regulatory use that has been made of such metrics to date in Lithuania strongly suggests that the consideration of such approaches would be premature at present. In any event, experience strongly suggests reputation-based approaches, focused on the provision of concise, credible and easy to understand comparative information, can have powerful incentive effects.
ERSAR operates an annual process that involves utilities submitting the required data, that data being validated and treated to provide for benchmarking, and utilities then getting a right of reply before the finalised data is then published and publicised (including through an App). The approach focuses on providing information on around 15 Key Performance Indicators for each service (i.e. water and wastewater), with indicators designed to reflect performance in relation to the protection of user interests, service provision sustainability and environmental sustainability. The specific KPIs to be used could, of course, be adapted to the Lithuanian context where appropriate. Also, it is notable that many of the KPIs used by ERSAR appear routinely in service performance assessments that are produced in a number of other jurisdictions, including, for example:
The notable features of the ERSAR approach, therefore, are less to do with the specific indicators that it provides for collection of (because, as above, it is common for similar types of indicators to be collected in other jurisdictions), and more to do with the processes and approach through which that performance information is presented and communicated in clear, concise and accessible ways. For each performance indicator, companies are ranked and compared with their peers through the use of clusters, based on the different regions in which they operate, and the characteristics of the area (e.g. rural vs urban).7 As can be seen in diagrams below, this is used to provide an easy to understand ‘traffic light’ based presentation of comparisons that allow for straightforward identification of those operators that are best performing, above and below average, and so on. The annual reports allow for comparison between expected and actual performance, and for performance levels to be monitored over time, with this assisting with the prioritisation of improvement opportunities.
Note: In line with the comments above, the purpose of including these diagrams is not to emphasise the specific methods through which the underlying measures were calculated (and, in line with this, those methods are referred to in only high-level terms), but rather to illustrate the approach through which comparative performance information (once measured) is then presented and communicated.
In line with the comments above, the purpose of highlighting the ERSAR approach here is not to present the specific methodologies that are used to calculate and compare performance indicators as ones that should be considered for usage in Lithuania. Rather, the ERSAR example is intended to provide a useful reference point when considering how WSS performance data can be communicated in ways that can enhance the scope for comparative, reputational pressures to highlight better and worse performing areas, and in doing so to help make more visible where improvements may be both possible and appropriate. The adoption of this kind of approach could be tailored to reflect the relevant circumstances in Lithuania, with a range of underlying choices being required in relation to matters including:
How KPI information should be clustered and otherwise organised and adjusted when benchmarking results are being presented.
In line with the comments on cost assessment above, the most appropriate way to develop the specific performance benchmarking approaches that are to be applied will depend on a range of detailed and context specific matters that go well beyond the scope of this assessment, and is best viewed as something that would be expected to evolve over time. The ERSAR approach looks to provide a helpful reference point when considering the framework and processes within which benchmarking arrangements could be developed and applied. In particular, it focuses attention on trying to make available, and communicate, clear and easy to understand information on comparative performance. With this treated as an appropriate objective, attention can then be turned to the detailed and ongoing work that is likely to be needed to deliver on that. This is not a question of simply seeking to identify what the ‘right’ set of measures and underlying methodologies (e.g. for clustering municipalities) are as a stand-alone exercise. Rather, it is more a question of seeking to develop processes that can be expected to help provide ways of building and refining more appropriate approaches over time, recognising that this is challenging to do.
The challenges arise because there are different dimensions of performance that could be measured and compared in different ways, and decisions in relation to those dimensions and measurement and comparison techniques may imply materially different outcomes in terms of apparent relative performance. This tends to make the process through which methods are developed important, as that process can potentially help give legitimacy to the overall outcomes that result. A commitment to providing clear, concise and easy to understand performance information – one of the high-level goals that ERSAR identifies – is important here, because it makes it clear to stakeholders, including importantly WSS companies, that performance comparisons are going to be made and presented to the public in relatively simplified formats of the kind illustrated above.
Having made such a commitment, it is then important to consider the processes through which the specific performance measurement and comparison methods will be determined. But the context is then one in which all companies know that this kind of information will be produced in one form or another, and they know that how they are shown as performing is likely to be affected by the specific methodology choices that are made. Company interests, though, will clearly differ in a range of important ways, as relative assessments will show some as performing well and others poorly by comparison. This difference of interests across companies provides a valuable source of information and input, and the tension it can create between companies can be used by the regulator to try to help improve the robustness and reasonableness of the measures being generated. Again, the commitment to producing and publicising the comparative information is key as it can allow attention to be focused on more productive questions concerning how that information should be developed, rather than on the question of whether it should be developed, where company interests may be more aligned, in that there may be a general preference for limiting the extent to which there is broader emphasis put on company performance levels.
A different form of benchmarking that has been used internationally, and may provide a desirable means of seeking to encourage more efficiency-enhancing consolidation activity, relates to business planning processes. The following considers two options of benchmarking incentive that look to merit careful consideration:
The first option focuses on how ambitious a plan is (according to the company itself and to the regulator)
The second assesses the planning process, and in particular the adequacy of options appraisal processes: the regulator wants to ensure the company has explored a range of options (even though the selected one may look conservative).
Note that these options are not mutually exclusive and can be used in parallel.
8.4.1. Incentives to encourage the development of efficiency-enhancing consolidation plans
A number of regulators internationally – including the Essential Services Commission (ESC) in Australia,8 and Ofwat in England and Wales9 - have introduced forms of business plan incentives, typically to try to address concerns that companies may otherwise have an incentive to be unduly conservative in their planning, and to do too little to address the future challenges that are faced. Such approaches can be understood as effectively rewarding early movers for the information they provide in terms of the improvements that their plan presents as achievable. Where the company delivers on that more challenging plan, the outcomes can then form part of future benchmarking efforts that can increase the pressure on other companies to improve, while also providing a practical example of how that improvement may be achievable.
The PREMO framework
The approach developed by the ESC (the economic regulator of the Victoria water sector in Australia) is referred to as the PREMO10 framework, and provides a useful reference point. Under the PREMO framework, instead of applying a uniform weighted average cost of capital (WACC) across companies, the allowed return on equity is varied depending on the level of ‘ambition’ shown in the relevant company’s price submission. The diagram below illustrates how a higher return on equity is when setting allowed charges where the ESC identifies a plan as more efficient, with four different categories having been identified:
Basic: where the submission is identified as reflecting stagnating or declining performance the allowed return on equity would be set at a level commensurate with the benchmark real cost of debt.
Standard: where a slightly higher return on equity allowance is provided for to reflect that the submission is viewed as a good value proposition for customers but that represents a continuation of existing outcomes and cost efficiency targets.
Advanced: a more ambitious submission that will generally commit to improved outcomes in terms of services, prices or both, and would receive a higher equity return allowance.
Leading: where the proposals place the company as a sector leader on key aspects of performance.
Source: Figure 2.1 in: https://www.esc.vic.gov.au/sites/default/files/documents/Water-Pricing-Framework-and-Approach-Final-Paper-Oct-2016.pdf
As can be seen in the diagram, the allowed return on equity is also made dependent on the company’s own assessment of the level of ambition of its plan, with this intended to encourage companies to put forward their ‘best offer’. In particular, it is notable that - under the approach – a company gets a lower allowed return on equity if ESC judges it as having a lower level of ambition than had been presented in the company’s self-assessment. The ESC identifies the red shaded area as indicating where it reserves the discretion to adopt a different approach, such as setting a shorter control and/or requiring resubmission.
Some core features of business plan quality incentives
The PREMO approach provides an interesting example in part because of the clear and explicit way in which it adjusts the return equity allowance depending on assessments of level of ambition. At the same time, it includes levels of complexity (including the use made of company self-assessments) that seem unlikely to be necessary or well suited in the current Lithuanian context. In practice, this kind of business plan quality incentive approach can be viewed as comprising of three core features:
1. Scope for identifying company business plans as falling into more than one quality category.
2. Identification of the criteria that would be used to determine which quality category a business plan should be identified as in.
3. Explicit and credible up-front identification of how companies will be treated differently when identified as falling in one quality category rather than another.
The following considers how each of these features might be applied in an Lithuanian WSS context.
Identifying categories of business plan quality
The PREMO approach involves the regulator having to determine which of four ‘quality’ categories a price submission falls into: basic, standard, advanced, leading. Ofwat (the economic regulator in England and Wales) also used four categories – significant scrutiny, slow-track, fast-track, exceptional – in its review of price controls for the 2020-25 period, although in practice only allocated companies to three of those categories (with no business plans categorised as ‘exceptional’). It is notable that in an earlier development of this kind of approach, the British energy regulator Ofgem applied a simpler categorisation that distinguished only between whether companies should treated as ‘fast-track’ - because their proposals had been identified as of ‘high’ quality and therefore appropriate to implement quickly - or ‘slow-track’ – because their proposals had been identified as of relatively lower quality and as requiring further, more detailed scrutiny. The use of four (rather than two) categories in the PREMO and Ofwat approaches can be viewed as a simple refinement of Ofgem’s approach, that allows a further subcategorization of the ‘high’ (fast-track) and ‘lower’ (slow-track) quality categories. It is not obvious, however, that this further refinement would be particularly helpful if such an approach were to be introduced in Lithuania, at least in a first iteration. In particular, the use of additional categories increases complexity, and the burden that the regulator (in seeking to specify and apply the categories) and regulated companies (in seeking to understand and determine how to respond to the categories) can be expected to face, but may provide little additional benefit over a simpler two category approach.
Given this, the development of a two-category approach, where companies can explicitly expect more favourable treatment if categorised in the ‘high quality’ rather than in the lower quality category, looks likely to provide the most appropriate starting point for developing this kind of approach in Lithuania. That said, there may be a case for explicitly identifying the possibility of using two separate ‘high quality’ categories (with only one lower quality category), if there are some sources of additional benefit – for example, additional support from EU funds – that may be available in some circumstances but not others. Where this is the case, access to the additional support could be made conditional on achieving ‘high quality’ status in the regulator’s business plan assessment, but would then involve some further hurdles having to be overcome.
Criteria for assessing which category a plan should be identified as in
For its review of 2020-25 business plans, Ofwat developed (ahead of company submission of business plans) a relatively extensive assessment framework that highlighted both the ‘test areas’ that were going to be explicitly assessed, and what the characteristics of high quality, ambitious and innovative plans would be likely to be in each of those test areas.11 The test areas included a range of core priority matters such as securing costs efficiently, addressing affordability and vulnerability, and securing long-term resilience, and for each test area, Ofwat identified some questions it would be relevant to consider. For example, in relation to securing long-term resilience, the following questions were identified:
How well has the company used the best available evidence to objectively assess and prioritise the diverse range of risks and consequences of disruptions to its systems and services, and engaged effectively with customers on its assessment of these risks and consequences?
How well has the company objectively assessed the full range of mitigation options and selected the solutions that represent the best value for money over the long term, and have support from customers?
For a plan to be viewed as ‘high quality’, Ofwat identified that (among other things): the company will provide clear evidence that they have objectively considered and assessed the full range of resilience management options. For a plan to be viewed as ambitious and innovative, Ofwat identified that the company would need to present strong evidence that it has used robust, ambitious and innovative approaches to assess and mitigate risks to long-term resilience in the round.
Developing this kind of business plan inventive approach in Lithuania would not require the extent of development – in terms of assessment criteria and questions - that Ofwat undertook. However, some up-front specification of what the key test areas would be, and what sorts of questions would be expected to guide the assessment in those areas, is likely to be helpful both because it can give greater clarity to companies on what they can expect, but also because the regulator plans and delivers the subsequent assessments (by making it clear what practical steps that is likely to involve). When seeking to specify how companies should be assessed, it can be helpful to distinguish between the following:
Hygiene factors: to what extent are there criteria that should be viewed as a necessary condition for any company’s business plan to even be considered as potentially ‘high quality’ (such that the meeting of this criteria can be treated as a form of hygiene factor in the assessment process)? This may include reference to current performance levels and financial health.
Other differentiating factors: given its strategic importance for the Lithuanian WSS sector, it would be expected that consideration of other differentiating factors would be heavily focused on the extent to which companies are bringing forward new consolidation options, and the extent to which they are able to demonstrate, robustly, that those consolidation options can be expected to be efficiency enhancing.
The relevance of different potential forms of consolidation is considered later in this section when some of the potential constraints to consolidation options emerging are considered.
The benefits of a plan being identified as of higher quality
Regulators have typically sought to provide for financial, reputational and procedural incentives to be associated with the identification of a company’s business plan as ‘high quality’ within this kind of assessment framework. In line with this, in Lithuania, the development and submission of credible, efficiency enhancing consolidation plans could be encouraged in a number of different ways, including through:
The use of a higher WACC in the tariff setting methodology than would otherwise have been allowed for (as is explicitly provided for in the PREMO approach).
The explicit provision of some other form of financial reward: for example, access to grant funding or preferential borrowing opportunities.
Greater scope for support with respect to financeability using accelerated depreciation (where that can be shown to be consistent with bill affordability and acceptability issues being sufficiently addressed).
Scope for the price control to be determined for a longer period: in line with the comments earlier, this may be important in providing consolidating parties with an opportunity to share in the benefits of the plans they bring forward (particularly where there may be some time lag associated with the securing of those benefits).
Presentation of the outcomes of the assessment in a way that can be expected to provide material reputational benefits for those associated with successful companies: the regulator can actively seek to highlight and publicise its assessments of ‘high quality’ proposals, and then use the companies actions as a positive case study to promote further change.
Procedural benefits associated with less extensive review requirements, providing overall performance remains sufficiently ‘on-track’.
The importance and implications of credibility
For business plan incentives to encourage companies to put more effort into developing efficiency enhancing consolidation plans than they would otherwise, attention needs to be given to the overall attractiveness of the rewards that are potentially on offer. The scale of financial rewards can be expected to be an important part of this, and to merit careful consideration by reference to the sorts of decision making that companies face (i.e. in terms of the risks they may be taking on if pursuing novel consolidation approaches). Another key factor, however, concerns credibility. Credibility can be enhanced to some extent through the publication of up-front information on available rewards, and the PREMO framework can be viewed as notable in this respect in terms of its clear articulation of the different WACC outcomes that can arise. However, interpretations of the relevance of reward information will be heavily dependent on perceptions of how the arrangements might be applied in practice. This can leave scope for a significant dampening of incentives to arise as a result of concerns over the likelihood of a company actually securing a reward even where it seeks to respond in appropriate ways through the development of consolidation plans. For example, a company may envisage a situation where it has developed a challenging and innovative consolidation proposal only for the regulator to classify it as somehow deficient and not meriting the ‘high quality’ classification, and the associated securing of the identified rewards.
To some extent, this kind of issue is inevitable with the introduction of a novel assessment mechanism, and given uncertainty over what plans might actually be presented to the regulator, it can be very difficult to address this kind of issue up-front, other than through the development and publication of the kind of a assessment criteria discussed above. However, in practice, much can be done to address this matter through the way in which the regulator engages throughout the process. Uncertainty (and the potential dampening of incentives) may be greatest where the regulator adopt a relatively arms-length approach to the process, such that – once its broad intended approach has been outlined – it is viewed as being for companies to interpret and respond to that approach, with the regulator’s next key role in the process coming at the business plan evaluation and categorisation stage. Under this kind of arms-length approach companies may expect to be poorly sighted in terms of how well-aligned their plans are with regulatory expectations, and thus may heavily discount the prospect of actually securing the rewards that have been presented as potentially available. There may be a significant risk that the incentive arrangements have little impact, and indeed there may end up being little basis for the regulator to provide any rewards.
A different approach, though, would be for the regulator to be very clear up-front that it recognises this uncertainty (and the effects it could have), and is committed to seeking to work with companies to help reduce that uncertainty, and then to allow for rewards under the incentive mechanism providing it is presented with credible consolidation plans. Under this kind of more active approach, the business plan assessment process can be explicitly presented as an iterative one, in which the regulator will seek to provide guidance to companies who consider themselves potentially in contention for a reward, in order to try to resolve concerns they may have about misalignment of views. This can provide a situation in which there is limited remaining uncertainty over the final regulatory assessment (i.e. companies should have good sight of how their plans will be assessed) because of commentary the regulator has already provided along the way. In practice, this could be applied by adopting something like the following stages:
1. Clearly identify the hygiene factors (referred to above) that the earning of rewards would be conditional on: the development of these could include some engagement with companies that might be expected to bring forward consolidation plans to ensure the criteria are not unduly restrictive (while at the same time act as an appropriate initial filter).
1. Provide an explicit option for companies to check they have satisfied the hygiene factor requirements: this would not be a requirement, but would be expected to initiate a process of engagement with potentially successful companies.
2. Provide scope for companies to ‘check-in’ periodically on their developing plans, with the aim of identifying potentially material limitations (and where there are significant misalignments of view) at an early stage.
3. Where common issues are identified through the check-in process (for example, with respect to sufficiency of evidence on likely long-term impacts), the regulator could publish a brief update note in order to improve on the broader transparency of the arrangements.
4. Allow for early submission of draft plans on which formal feedback (with ‘points to address’) could be provided.
To some extent, the above can be viewed as similar to a form of procurement process in which – after an initial ‘pre-qualification’ phase – some ongoing negotiation and engagement is often an important part of ensuring that final ‘bids’ are well suited to the buyer’s requirements. Although the context clearly differs here in some key respects, it is notable that there is scope for regulator and company interests to be relatively well-aligned, and the purpose of adopting something like this kind of more active engagement approach is to try to keep the scope for that high-level alignment clearly in mind for all sides, and to reduce the scope of unhelpful surprises to emerge (the prospect of which – in line with the above comments – may be viewed as a significant deterrent to the development of potentially beneficial plans).
8.4.2. Benchmarking of business planning processes: ensuring a range of (consolidation) options have been explored and duly assessed
The above has focused on how incentives based on the overall quality of company business plans might be used to encourage the development of beneficial consolidation proposals, in particular by seeking to reward plans that are identified as ‘high quality’. However, a different form of incentive that also merits consideration (alongside the use of such approaches), involves providing scope for penalties to be applied as means of encouraging the more extensive and effective use of options appraisal methods.
The extent to which investment plans are based upon sufficient consideration of alternative potential options – including options that involve consolidation – is likely to become an increasingly important factor over time and raises questions over the appropriate scope of the regulator’s cost assessment activities. A distinction can be drawn between:
Identifying how desired outcomes are best met (i.e. the choice of approach); and,
Identifying the efficient cost of delivering the approach that has been selected.
A narrow cost assessment exercise may focus only on the second bullet point above. However, there may be substantial scope for efficiency improvements associated with the first bullet, particularly where there may be opportunities to deliver services in more coordinated and consolidated ways that enable greater economies of scale benefits. In line with this, there may be some benefit in seeking to directly target the sufficiency of ‘how’ assessments.
When considering ways of addressing this, it is helpful to distinguish – in principle – between the following two types of assessment that a regulator might undertake:
Providing for detailed, expert reviews of the options appraisals that companies have undertaken.
Assessing whether companies have undertaken appropriately robust options appraisals processes.
Historically, many regulators have tended to adopt the first ‘expert reviewer’ role (or have appointed engineering consultants to undertake it on their behalf). However, this can involve the regulator effectively taking on responsibility for demonstrating why a given assessment by the relevant company should be regarded as not sufficiently well founded, by reference to the regulator’s own assessment of alternatives. Such an approach can be resource intensive as it will often require considerable detailed work in relation to specific matters where the regulator inevitably has relatively limited information and expertise. This can then put significant limitations on what it is realistic to expect the regulatory review process to achieve (given, in particular, relevant information asymmetries).
The second role noted above seeks to address matters in a different way by taking a step back and focusing regulatory attention on the adequacy of the options appraisal processes that companies have undertaken, rather than on the detailed analysis and findings of the specific appraisals they have produced. From this perspective, it is viewed as for companies to demonstrate to the regulator that they have conducted appropriate options appraisal processes, and if companies are unable to do that sufficiently, then that – in and of itself – could be treated by the regulator as a basis for some form of penalty to be applied (through making a downward adjustment to the cost allowance the company).
Ofwat has adopted this latter kind of approach in England and Wales in some of its stand-alone (i.e. not benchmarked) assessments of proposed investments, where it has applied the following approach:12
If a company provided evidence that a lower cost option was available but gave no reasons as to why it was rejected, Ofwat would use the lower cost option when calculating the allowed costs.
Where a company has not provided evidence that its selected option is optimal (i.e where there is insufficient evidence that the potential for using alternative, lower cost, options was explored and assessed), Ofwat would apply a 20% reduction to the company’s proposed cost estimate, intended to protect customers from the risk that a potentially sub-optimal solution was being adopted.
On the face of it, this kind of approach might be viewed as relatively arbitrary, in that it could result in a substantial gap between the amount a company has identified as needed to deliver on a specific project, and the amount the regulator allows to be recovered through charges, without the regulator having explicitly identified that the lower amount should be viewed as sufficient. However, the regulatory approach can be understood as intended to incentivise companies to ensure that high quality options appraisal processes have been undertaken (and that the company can demonstrate this), in a context where the quality of the appraisal processes is viewed as potentially having a substantial impact on costs that may be recovered from customers over many years.13 The approach can therefore be understood as having risk-based foundations, in that companies that seek to proceed with highly material projects without having undertaken an adequate options appraisal process can be viewed as exposing customers to significant risks of funding inefficient investments.
Ofwat’s penalty-based approach is applied in a context where it sits within a broader set of reward and penalty arrangements that private water companies are subject to. The different context in Lithuania may mean that such an approach would be unlikely to be feasible or desirable. However, an alternative to applying this kind of downward adjustment to allowed costs would be to effectively not accept the relevant part of the price control submission, and to send it back to the company to address the limitations in its assessment of options. That is, the regulator could be viewed as introducing a form of assessment ‘gateway’ that companies must successfully pass through in order to secure funding for significant new investment projects. If the company has not shown that relevant options have been appropriately taken into consideration, then that part of the price control application could get stripped out and sent back.
This may provide a more practical and appropriate means of proceeding in Lithuania, with the regulator providing some guidance on what is expected in terms of options appraisal including in terms of the consideration of consolidation options. In this way, evidence of having conducted an adequate options evaluation process could effectively become treated as a condition of a successful application for relatively large-scale requests for the recognition of new investment costs in the tariff formula. The availability of practical evidence of potential consolidation options – including from pilot study activity – can be of considerable importance under such an approach. In particular, such evidence provides a concrete basis upon which a regulator could question the adequacy of a company’s consideration of options, as it can look for evidence that the lessons from the pilot study have been recognised, and that the potential relevance of those lessons have been explored and tested. By making pricing assessments in other company areas dependent, to some extent, on the consideration that has been given to pilot study evidence, the regulator can effectively raise the prominence and importance of that evidence, and increase the scope of the likely impact of pilot study activity.
This Chapter has sought to identify potentially desirable developments to benchmarking arrangements in Lithuania, with a particular focus on the scope for encouraging efficiency improvements through consolidation. It has considered – drawing, where helpful, on examples of international experience – some ways in which the identification and achievement of WSS efficiency gains might be further encouraged through the use of different forms of benchmarking. The key points that have been identified are summarised below through the identification of recommendations and suggestions concerning how the current arrangements could be developed so as to help tackle the key WSS challenges that are likely to be faced.
8.5.1. Recommendation 1: Commit to developing a Service Performance Incentive framework
Commit to develop, make publicly available and publicise a KPI framework that provides concise, credible and easy to understand comparisons between companies, using the Portuguese ERSAR approach as a guide.
The development of this kind of framework has the potential to deliver substantial benefits by providing more robust protection of customer interests, and guiding companies toward the use of more efficient and effective approaches, using reputational incentives. A key starting point would be a commitment to the adoption of such an approach so that industry attention could be focused on how that would be best achieved (rather than whether performance comparisons should be made more prominent).
8.5.2. Recommendations 2: Develop incentives that focus on the benchmarking of company plans, and planning processes
Develop guidance setting out how the economic regulator would enable companies that present credible, efficiency-enhancing consolidation plans to share the benefits they result in, through the treatment of consolidation costs, and commitments concerning how rapidly future efficiency savings will be reflected in allowed prices.
Develop guidance setting out regulatory expectations with respect to companies being able to demonstrate that robust options appraisal processes have been undertaken in the development of capex plans, and how capex applications will be treated where a company is unable to adequately demonstrate that.
These proposals directly target key aspects associated with encouraging the bringing forward of efficiency enhancing consolidation plans: the extent to which companies can expect to benefit from bringing forward such plans (given the scope for ‘ratchet effects’ to otherwise undermine such incentives); and the risk that companies do not adequately explore or consider different ways of addressing outcome requirements when developing their capex plans. Examples from Australia and the UK provide useful reference points.
Notes
← 1. OECD (2021), Reform of the water supply and wastewater treatment sector of Lithuania by consolidation of utilities: Output 3; featured in this report as Chapter 3.
← 2. See, for example: https://www.severntrent.com/content/dam/stw-plc/our-plans/Severn-Trent-water-cost-modelling-framework-Final.pdf.
← 3. The identification of appropriate indicators is discussed further below.
← 4. With direct management, delegation or concession operating models used.
← 5. The selection of specific indicators is discussed below.
← 6. The extracts from ERSAR reports shown below illustrate how this has been done through the use performance comparison charts and a map-based comparison of performance between utilities using a traffic light system.
← 7. Some brief comments on the development of clustering approaches are provided below.
← 8. https://www.esc.vic.gov.au/sites/default/files/documents/Water-Pricing-Framework-and-Approach-Final-Paper-Oct-2016.pdf.
← 9. See, for example: https://www.ofwat.gov.uk/regulated-companies/price-review/2019-price-review/initial-assessment-of-plans/.
← 10. The terms PREMO comes from the different identified elements of the assessment process: Performance; Risk; Engagement; Management; Outcomes.
← 11. A detailed description of Ofwat’s assessment approach is provided in: https://www.ofwat.gov.uk/wp-content/uploads/2017/12/Appendix-13-IAP-FM.pdf.
← 12. See p54-55 of: https://www.ofwat.gov.uk/wp-content/uploads/2019/12/PR19-final-determinations-Securing-cost-efficiency-technical-appendix.pdf.
← 13. It is notable that, in presenting its approach, Ofwat highlighted that options can range considerably in cost, and pointed to companies as having provided evidence that there could be a 35% difference in cost between reinforcing as opposed to replacing a main.