2. Review of the Federal Planning Bureau’s analytical models
The objective of the review of analytical models is to help the FPB increase the quality of its work related to the ex ante assessment of reforms envisaged in the future by the federal and regional governments in areas such as pensions, taxation, the labour market, energy, and investment, particularly those linked to Country Specific Recommendations from the Council of the European Union.
The technical assessment looks at the appropriateness of the Bureau’s economic models across seven academic and practical criteria that an independent fiscal institution should consider when developing tools to deliver its mandate: (1) Theory; (2) Accuracy; (3) Communication; (4) Transparency; (5) Proportionality; (6) Sustainability; and (7) Precedent.
The review team assessed the models of the FPB according to the assessment framework for IFIs developed by the OECD’s Directorate for Public Governance. That framework answers the question: Are the institution’s models comprehensive and appropriate for delivering its mandate?
To answer the question, the review team identified the needs of the Bureau to deliver its mandate and the constraints it faces in fulfilling these. An assessment was then made regarding whether the suite of models that the Bureau has developed meets those needs given its constraints. Finally, each of the Bureau’s current models are reviewed individually and in depth to determine their individual appropriateness according to seven academic and practical considerations summarised in Table 2.1.
Some criteria are complementary, while others conflict. For example, a structural model grounded in economic theory may score highly in its ability to provide an intuitive narrative to stakeholders but may have higher forecast errors than a simple univariate time series model that relies only on its own history. Analysts at IFIs must consider these trade-offs and strike a balance when choosing models. For this reason, it is not possible to offer a total score or pronouncement on whether a model is the best tool for the analysis.
Instead, the assessment criteria is used to form an opinion on whether the chosen tool is appropriate or inappropriate for delivering the Bureau’s mandate in the country’s context — that is, whether any models or analytical decisions are not currently suited to their purpose, fail to advance the Bureau’s mandate, or do not adhere to the OECD Principles. If a model is assessed as appropriate but yet there are recommendations to bring it in-line with best practices, a qualified opinion may be issued (Table 2.2).
This review is not a line-by-line code audit, nor does it undertake out-of-sample validation of alternative specifications. Instead, it seeks to identify any analytical gaps or areas where the Bureau should invest in broadening or deepening models.
External model reviews are an important element of an IFI’s accountability mechanisms and help reassure stakeholders of the quality of the IFI’s work. However, macro-fiscal forecasting is above all a human process that relies on considerable judgment — no two analysts with the same model will produce the same results. A periodic external assessment cannot take the place of an IFI’s other legislated channels of accountability. For the Bureau, this is formal scrutiny by the Federal Parliament and its ongoing dialogue with academics, peer institutions and the public.
IFIs fall across a spectrum of roles and responsibilities. The assessment framework must be adapted for the needs of an IFI’s institutional arrangements — that is, the functions defined by its primary and secondary governing legislation, memorandums with government agencies and the discretionary operating guidelines it sets for itself. The framework must also consider the constraints of the Bureau — its resources and the economic and fiscal data available to it, which will drive model selection.
2.3.1. The Bureau’s modelling needs
The Bureau’s main responsibilities and modelling needs are laid out in an exceptionally complex array of laws. New laws in Belgium that have a requirement for monitoring or technical expertise tend to name the Federal Planning Bureau as the institution responsible for it either directly, or by serving as the technical secretariat for another responsible body. Table 2.3 lists some of the Bureau’s main responsibilities named in legislation. The Bureau also has responsibilities arising from established practices and agreements of various formality with government departments.
2.3.2. Identifying the constraints of the Bureau
As illustrated previously in Figure 1.7 in Chapter 1, the Bureau has one of the largest teams of analytical staff among European IFIs. However, it also has among the most tasks and the most diverse responsibilities among IFIs.
The Bureau is generally able to attract staff with the required expertise and the pool of high-calibre analysts in Belgium is large, including a large pool of Belgian PhD economists and PhD economists from other European countries with the right to work in Belgium. That said, there are some constraints that limit the Bureau’s ability to find the expertise for its modelling needs:
There are language constraints, with the Bureau working primarily in French and Dutch.
Competition for analysts in Brussels is fierce among the many domestic and international institutions, governments and think tanks.
The tendency for the Bureau’s analysts to remain in the same position on the same model for many years has lowered its attractiveness to graduates and early-career staff.
The Bureau has enviable access to data, both through legislation for its participation in the National Statistics Institute and membership on numerous official commissions, councils, and committees, as well as through well-established informal peer-to-peer relationships.
However, the Bureau does face some challenges relating to data access:
2.4.1. Comprehensiveness of the FPB’s model suite
The Bureau’s responsibilities are summarised under five areas and mapped to its current model suite in Table 2.4. For providing the macroeconomic forecast for the budget, the Bureau takes its quarterly MODTRIM macroeconometric model and splines it with its medium-run annual forecasting and policy model HERMES. The Bureau uses monthly monitoring, expert judgment and smoothing to arrive at the most recent quarters for which national accounts data is not yet available and current quarters. Most peer IFIs rely either on the same quarterly macroeconometric model for the short term and medium term or extend it from a nowcasting model for the current and subsequent two quarters. The Bureau’s use of an annual model for outer years of the medium-term forecast is somewhat unique.
For financial and economic policy costing and analysis of the taxes and transfer system, the Bureau relies primarily on HERMES and microsimulation models. The Bureau has quickly caught up to the CPB Netherlands Bureau for Economic Policy Analysis in pursuing a form of election costing that focuses on economic impact assessments — the effects of policies on employment, productivity, growth, inflation, and other macro aggregates. This reflects the Bureau’s history and its staff skillsets.
For sectoral modelling for government ministries, the Bureau has its PLANET model to support the Federal Public Service Mobility and Transport and CRYSTAL SUPER GRID to support its work on energy modelling for various stakeholders such as the climate team within the Ministry of Health. For the Bureau’s work monitoring the Recovery and Resilience Plan, it uses the Belgian-adapted QUEST III R&D model for the long term, the framework most common among EU analysts and promoted by the European Commission, and the HERMES model for the short-medium term. The structural studies team is also working on a new tool for structural reform analysis, the DynEMIte DSGE model.
For long-run fiscal sustainability analysis, the Bureau has developed MALTESE, a tool for projecting social expenditures over the long-run. The Bureau does not regularly publish summary statistics of sustainability such as fiscal gap calculations.1
For its statistical compilation, the Bureau relies on its partners in the statistics framework—particularly those that comprise the National Accounts Institute—to procure the data it needs, which is then processed (mainly) in its Python-LArray platform. Stakeholders are satisfied with the largely mechanical compilation and dissemination of statistics that the Bureau offers as a service provider.
Overall, the Bureau has a broad and diverse range of tools at its disposal to cover an area of vast swath of policy analysis both broadly and in-depth. While there is a risk that in defining an institution’s scope too broadly it loses focus, the Bureau’s stakeholders universally praised the advantages of having all the Bureau’s diverse modelling expertise under one roof. The review team noted some areas of the overall workflow that set the Bureau’s model suite apart from its peers:
The Bureau’s models are generally more sophisticated than those of peers. This partially reflects the age of the Bureau and the experience of its analysts, many of whom have been there since the modern Bureau’s formative years. However, sophistication is not always better. Models are a tool to help think through a problem and tell a story. When the development and deployment of sophisticated models becomes the goal unto itself, it may distract from timely and responsive analysis that may not be as elaborate but could better fulfil the Bureau’s purpose of informing stakeholders when it matters most during the policy process. Although stakeholders are not looking for quick and dirty analysis when they approach the Bureau with a question, there is a balance to be found between providing a timely answer and providing an answer based on sophisticated modelling.
The Bureau has considerably more resources devoted to microsimulation than its peers. This is largely because they can: they have far better access to administration data than peers and Belgium has a wealth of interesting public microdata sets. However, the availability of microsimulation options can be a crutch and come at the expense of analytical options that have a weaker footing in administrative micro-data but may sometimes be more informative.
Potential gaps in the model suite
The review team indicated several areas in Table 2.4 as underserved according to their modelling needs or in comparison to the practices of other peer IFIs with similar mandates. Not all underserved areas can be addressed (for example, if missing data is irresolvable).
Task 1: Macroeconomic forecasting
Nowcasting. Many peer IFIs have adopted nowcasting models that use dynamic factor analysis or principal component analysis to statistically assess high-frequency data (monthly, daily, or continuous) to arrive at the recent past (to fill the lag in national accounts publication), the current quarter, and the next two quarters. For example, the Independent Authority for Fiscal Responsibility in Spain (AIReF) uses its MIPRed dynamic factor model at the monthly frequency to determine the concurrent two quarters.
Task 3: Estimating potential GDP and the business cycle (including long-run potential GDP projections)
Contribution to the business cycle debate. The Bureau’s analysis of the business cycle—particularly its estimates of potential output and the output gap—does not receive the same attention or carry as high a modelling priority as many of its peers. This is partly because the structural budget balance in the context of the EU Stability Programme has not yet become as heated a national debate in Belgium as elsewhere. However, it cannot be taken for granted that it will not be an issue in the future and the Bureau would be well-advised to get ahead of it.
Other institutions have made valuable contributions to the business cycle debate using two approaches that would complement the Bureau’s existing model work: (1) Preparing simple, intuitive visual heat maps that assess specific industries and regions and how they are performing relative to their trend, for example in Finland, Latvia, Estonia and others from the Baltic-Nordic network, along with Ireland; (2) Preparing several alternative projections of actual and potential GDP using different model types or specifications, either as a sense check of the primary forecasting model, or to be averaged for their published outlook in a suite modelling approach, as in the case of Ireland. The Bureau accomplishes some sense checks on the model results; however, it could be more systematic and provide greater discussion surrounding the different results and how they have been reconciled.
Task 5: Costing the financial impact of polices
Ad hoc financial models and satellite structural tax and transfer models. While the Bureau does some financial cost assessments off model on an ad hoc basis, it primarily views policy costing through a macroeconomic lens—that is, working out the many ways that a policy could potentially affect the macroeconomy, such as output, inflation, wages, unemployment, household disposable income, and purchasing power. That perspective is guided by the Bureau’s traditional role in economic research. However, the new costing mandate requires the Bureau to be much more focused on the financial and accounting elements of new policies. In focusing on the macroeconomic implications of policies, they may miss important financial details important in getting the budgetary impact of measures correct and making them useful for decisions makers. These include aspects such as administration costs, accruals and cash considerations for financial statements, take-up or noncompliance considerations, and base erosion and planning or evasion, which should all be incorporated in a cost estimate or presented as supplementary analysis.
As Belgium looks to improve its public finances over the medium term, the provision of rich financial information on policies will be useful for stakeholders to undertake effectiveness evaluations, impact assessments, and ex post audits. Other institutions provide simple back-of-the-envelope arithmetic explanations underlying cost estimates such as the number of taxpayers affected, or the number of benefits recipients and average payment, so that stakeholders understand the moving parts underlying the results and can approximately replicate them.
Behavioural adjustments. The Bureau’s approach to costing means the analysis is presented without adjusting for likely behavioural responses, giving rise to systemic bias in the Bureau’s results. Other institutions invest more in complementing the initial results of microsimulation models with top-down spreadsheet financial models that adjust the results for behavioural assumptions derived from the academic literature or empirical assessments of similar policy changes in their own country’s past or in other jurisdictions.
Tools for evaluating the impact of taxes on income from wealth. The Bureau’s tools for assessing the impact of personal income tax measures commonly exclude any income from capital sources and the surrounding costs and economic implications.
Task 9: Forecasting and analysis of emissions and climate change
Environment and climate modelling. The Bureau’s models for monitoring efforts to reduce emissions and act on climate change have fallen behind its leading-edge approaches to evaluating macroeconomic issues and are undeveloped compared to some peer institutions like the Danish Economic Councils, the CPB Netherlands Bureau for Economic Policy Analysis, and the Parliamentary Budget Officer of Canada. They have begun on a work programme to address these gaps and are developing an environmental CGE model.
Task 10: Modelling transportation and freight demand
Freight transportation modelling. The Bureau has not found a sophisticated solution to model freight transportation, as required to fulfil its transport modelling mandate. This is largely owing to data gaps.
Task 11: Monitoring the Recovery and Resilience Plan
Structural reform assessments. The Bureau’s solutions for assessing the Recovery and Resilience Plan remain under development. This is an issue common across institutions in the European Union and elsewhere, where there are no easy fixes. The Bureau’s work programme for developing the new DynEMIte tool may make progress toward this goal.
2.4.2. The FPB’s analytical workflows
The FPB has a diverse range of modelling responsibilities and uses a wide array of software packages for managing it. It is a mixture of proprietary packages like Stata, SAS, Gams, Matlab and Excel; open source languages (and their usual libraries) such as Python and R; and in-house developed software packages or libraries like IODE, LIAM2 or LArray, which are further described below.
For econometric models, the econometrics platform IODE2 is the backbone of the Bureau’s workflow to co-ordinate its data resources, inputs, and outputs across models, teams, and projects. It is a powerful software package for statistical analysis and model solving.
IODE was developed in-house by the IT Unit of the FPB. It assists analysts by streamlining activities such as (1) Automating data retrieval from databases, (2) importing and exporting series between the office’s open-source and licensed software packages like Python and the LArray library, Stata, R, Excel, (3) documenting databases, (4) writing and estimating equations, (5) facilitation scenario simulations, and (6) generating graphs and tables, among other helpful functions like scripting.
While it has many benefits and is fast and efficient in keyboard navigation and processing, such an in-house software solution is unique among IFIs. It is largely a carry-over from an earlier computing workflow — it was developed as the replacement for the Bureau’s mainframe computer econometric software in the 1980s.
The software is written in C and C++ and requires dedicated specialists to maintain, refine and add functionality. New techniques must be translated into IODE rather than simply being applied as imported libraries from R or Python that outside researchers often publish alongside their results (although such files can be passed back and forth to IODE).
The look and feel of the IODE GUI divides users internally, with some having an affinity and others wanting a more modern solution. Some outside stakeholders also see it as outdated.
Overall, IODE is observed to play a crucial role in the Bureau’s workflows. Nonetheless, the Bureau’s ongoing commitment to IODE should be reviewed with an eye to converting it over the long-term to a more modern software solution with greater penetration in the field of economics, along with the gradual conversion of models specific to it, such as HERMES. Doing so will have several benefits:
The ability to quickly incorporate leading-edge techniques from outside academic working papers and other researchers, that are increasingly published open-source in Python and R.
The ability to leverage the tools coming online to assist code drafting, such as AI “co-pilot” programmes that autocomplete code based on code comments which is improving the productivity of researchers by leaps and bounds.
The ability to participate in, and benefit from, larger modelling communities providing support for choices like Python and R.
LIAM2 is an open-source software package developed in Python to help economists develop microsimulation models. The MIDAS model is developed in LIAM2. The toolbox is made as generic as possible so that it can be used to develop almost any microsimulation model as long as it uses cross-sectional ageing, i.e. all individuals are simulated at the same time for one period, then for the next period, etc. The goal of the software is to let modellers concentrate on what is strictly specific to their model without having to worry about the technical details. It was made available for free to outside researchers to build a community to reduce the development costs of microsimulation modelling.
LArray (which stands for Labelled Array) is an open-source Python library and GUI for analysing multi-dimensional matrices and creating models with them. It is used for many models of the Bureau (demographic projections, MALTESE). The most important feature is to access data via meaningful labels to make models more readable and easier to maintain, but it also helps modelers automate large parts of their workflow from importing data in various formats and cleaning it to generating data reports, charts or even dashboards.
All in all, the Bureau is considered to be further ahead than many of its peers in adopting collaborative open-source software in several areas of its model suite.
Through discussions with the Bureau the review team has identified 12 models in the Bureau’s primary toolset that are currently in use and appropriate to review individually, along with three that are in development for the future.
For in-depth assessments of each model, interviews were held with the relevant modelling team along with a review of work papers, specifications of equations, and in some cases the model code to scrutinise the suitability according to the framework described above. The remainder of this sections provides the results of the individual model assessments.
2.5.1. HERMES
HERMES (Harmonised Econometric Research for Modelling Economic Systems) was the outcome of a 1981 project proposed by the Commission of the European Communities (Directorate-General for Science, Research and Development) to create an econometric model that could simulate alternative assumptions about the world environment and economic and energy policies. It was motivated by the contemporary energy crisis and a realisation that energy distribution, prices, and policy would spread across borders (Donni, Valette and Zagame, 1993[1]).
The Federal Planning Bureau was a key member of the HERMES Club of 12 institutions in 12 countries (Belgium, Denmark, Spain, France, Greece, Ireland, Italy, Luxembourg, Netherlands, Portugal, Germany, and the United Kingdom) that was intended to work closely through co-ordination at the European level, although the goal was never fully realised and the “H” in HERMES is a misnomer (Donni, Valette and Zagame, 1993[1]).
The first version of the model in Belgium was finalised in the late eighties after four years of development, with the first results published on the topic of the impact of economic activity on the environment in 1989 and the use of taxes to reduce CO2 emissions in 1990.
Since then, most other HERMES Club members have stopped supporting development of their domestic models, with a few exceptions such as Ireland. The FPB has carried the torch and Belgium’s model has been regularly updated and significant improvements have been made over the years.
2.5.2. HERMREG
HERMREG is the regional companion of the national model HERMES for estimating regional economic output and its components. A first top-down version (HERMREG 1) was developed to produce regional economic projections in 2006 as a collaborative and jointly-owned owned initiative of the FPB and its three regional counterparts: SV (Statistiek Vlaanderen), IWEPS (Institut Wallon de l’Évaluation, de la Prospective et de la Statistique) and IBSA (Institut Bruxellois de Statistique et d’Analyse). In 2015, the Bureau developed a new bottom-up structure for policy impact assessments by constructing a block of regional equations for sectoral production factors and regionalised demand (private consumption, investments by delivery sector, trade regionalised external demand) relying on interregional input-output tables.
A multi-phase model development programme has since deepened the bottom-up version by improving the household income block, integrating the public finances block of the HERMES model, and improving the link between public finances and public consumption. For 2022 to 2026, the four partners have decided to fund a sixth phase of enhancements extending and improving the short- and medium- term projections of the top-down model, improving impact assessments using the bottom-up model, and enhancing the backend database. The sixth phase will also involve broadening and deepening the regionalising of the energy component of HERMES.
2.5.3. QUEST III R&D
QUEST III R&D is a Dynamic Stochastic General Equilibrium (DSGE) model that the FPB uses to calculate the long-run steady state impact of some structural reforms that affect the productivity of labour and capital. For example, it was used during the 2019 election costing period to assess proposals to invest in research and development, improve market functioning and increase public investment. The QUEST III R&D model was also used to calculate the medium- to long-term impact of Belgium’s draft National Recovery and Resilience Plan (only the investment part of the plan).
The model is the Belgian module of the QUEST III R&D model developed by the European Commission (DG ECFIN), that has been calibrated to Belgium’s national accounts data. Researchers in other EU member countries have similarly received their country-specific module, have updated the calibration and have applied it to a wide range of country-specific policy cases.
The Bureau has begun to look at extending the model to handle more sophisticated modelling of public investments and public support for private-sector investment, among other areas. It is also developing its own in-house DSGE model, which is based on the structure of QUEST III R&D but incorporates multiple industries, intermediate consumption (with input-output linkages between industries) and labour-augmenting semi-endogenous technological growth.
2.5.4. EXPEDITION and TYPECAST
EXPEDITION is a static microsimulation model developed in-house for analysis of policy measures related to personal income taxation, social security, and social assistance. Its key output is the impact of measures on disposable income in nominal terms by different household categories. EXPEDITION was developed over the 18 months leading up to the 2019 election platform costing exercise and is based on the EUROMOD platform, through using administrative data in place of EU-SILC (Statistics on Income and Living Conditions), which is the default data source for EUROMOD.
The model covers six policy areas: (1) pensions; (2) allowances payable by the National Employment Office; (3) compensation for sickness and disability; (4) personal income tax, (5) personal social security contributions and deductions from allowances; and (6) social assistance allowances and family allowances.
EXPEDITION is able to assess the effects of policy changes on the full set of households represented in the administrative microdata. By contrast, the TYPECAST module uses EXPEDITION’s analysis of disposable income to simulate the impact of a measure on a selection of specific standard household types that are useful for illustrating policy effects.
The Bureau has considered freezing development of EXPEDITION to switch to BELMOD, a similar project developed with other partners; however, negotiations surrounding the development and maintenance of BELMOD are ongoing and its future is currently uncertain.
2.5.5. HINT
HINT was developed for the 2019 election costing exercise to calculate the impact of policy measures on consumer prices faced by different household types (incomes and family composition). The results of the model complement the results of EXPEDITION to provide a more complete picture of the welfare effects of policies that alter the prices of specific goods or services. For example, a subsidy for public transportation may raise the real disposable income of households in lower income brackets more than wealthier households.
The model traces the effects of price changes both on the standard CPI consumer price basket faced by different households and on Belgium’s “health index” indicator that excludes alcoholic beverages, tobacco, and motor fuels. The latter is used for the indexation of housing rents and certain salaries, social benefits, and pensions.
2.5.6. MIDAS
MIDAS is a microsimulation and projection model that the Bureau has used since 2009 to study the risk of poverty and inequality among the elderly and the long-term effects of social and economic policies on pension adequacy.
The model starts with a cross-section of the Belgian population from administration data and the national census then guesses the life path of each individual as they choose a level of education, form a family, pursue a career, and save for retirement. It then takes the projections, works out the implications for pensions, and combines the results with simulations for social benefits to form a set of indicators for income inequality and the risk of poverty in each year of the outlook.
It is distinct from EXPEDITION in that it is a longitudinal and dynamic model. Moreover, the core model focuses on pensions and it uses the LIAM2 modeling apparatus. The model is aligned as much as possible with the financial sustainability projections of MALTESE and the composition of households is aligned with the Bureau’s LIPRO lifestyle projections that models the position of individuals within households.
2.5.7. MODTRIM
MODTRIM is the Bureau’s quarterly national accounts forecasting model for the short- to medium-term. It was built in 2003 but has undergone several reviews and major overhauls. It is a structural macroeconometric model that uses behavioural equations to forecast the demand components of expenditure-based GDP. Error-correction models underpin most relationships in aggregate demand, allowing for long-run equilibrium conditions with short-run correction paths to reach them following shocks. Short-run MODTRIM forecasts are combined with medium-term HERMREG modelling to feed into the Bureau’s forecasts for the Economic Budget.
2.5.8. PROMES
PROMES is a microsimulation model developed at the request of the National Institute for Health and Disability Insurance. The model was also used in 2019 for the Bureau’s election platform costing mandate to compute the medium-term budgetary impact of changes to health care policy in detail. It can estimate the budget implications of measures such as a percentage reduction in user fees, an increase in coverage for sickness insurance, an increase in dentistry fees, or other measures targeted at health expenditures.
The model consists of 25 modules corresponding to major expenditure groups, for example consultations and visits, dentistry, or physiotherapy. For each expenditure group, a behavioural model was estimated to explain the use of care according to individual characteristics, living environment, and previous use of care to arrive at projected healthcare volumes and expenditures. It was developed with the assistance of the National Institute for Health and Disability Insurance (NIHDI) and relies on the Permanent Sample (EPS), a longitudinal administrative database on the use of health care with more than 300 000 respondents. The results obtained for the sample are extrapolated to the future population using reweighting factors.
Expenditure projections from PROMES also contribute to the economic outlook in HERMES and the acute and long-term care in MALTESE (for the medium-term part).
2.5.9. MALTESE
The MALTESE model was developed to support the Bureau’s mandate added in 2001 to serve as the secretariate for the Study Committee on Ageing (SCA). Results have appeared in the Committee’s reports since 2002. The FPB also uses the model to represent Belgium at the EU’s Working Group on Ageing Populations and Sustainability (AWG) established in 1999 by the Economic Policy Committee of the Economic and Financial Affairs Council (ECOFIN). The model has also informed several high-profile policy impact studies for pension reforms, such as those in 2015. The model is also used to answer pension questions posed to the Bureau as part of the Knowledge Centre of Pensions (established in 2015).
The model consists of a set of modules for translating demographic projections into budgetary developments for social protection spending, particularly public pensions, over a horizon of 50 years (currently until 2070). Results are published by branch of social protection. Total revenues and indicators of public finances (balances, debt) are also modelled.
2.5.10. PLANET
PLANET is a model developed in-house to make long-term projections of the demand for passenger and freight transport in Belgium and to carry out transport-related policy analysis. The model derives transport demand by mode and period (peak and off-peak hours) from the evolution of demographic, economic and price variables (fuel prices, transport fees, and time costs). It also considers externalities such as pollutions and congestion.
The latest PLANET version used for the 2019 election costing period including teleworking in the commuting module and distinctions between morning and evening peak hours, outward and return trips, and private and company cars.
2.5.11. CRYSTAL SUPER GRID
CRYSTAL SUPER GRID is a “unit commitment” and “economic dispatch” model linking up to thirty-three European countries to assess the impact of different assumptions on prices and distribution within the electricity sector. Unit commitment determines the start-up and shut-down schedule of energy production units. Economic dispatch determines the power output of each energy production unit according to its cost and operational constraints, as well as the limits of the transmission network. The model determines both unit commitment and economic dispatch with optimisation routines that match supply and demand, while enforcing operational constraints (e.g. production limits, ramping constraints), by minimising total system production costs.
The Bureau has used the model since 2015. It is maintained and developed by Artelys, an external commercial software provider that specialises in energy modelling. In addition to the optimisation solvers, Artelys maintains an extensive library of physical and financial assets (thermal power plants, renewable energy sources, power lines, etc.) that is used in the scenario’s construction.
2.5.12. Models under construction
The assessment also included three models that are not yet in full production (LASER and DynEMIte) or are very early in their development stage. While it is not possible to issue an opinion on the appropriateness of the models at this stage, the models were nonetheless partially assessed, with preliminary feedback provided below.
LASER
LASER is a static structural discrete-choice model that estimates the change in an individual’s labour supply in response to a policy that affects household disposable income. The policy’s affect on household disposable income is taken from the EXPEDITION model.
The model has been in an ongoing state of development since 2017 to contribute to the Bureau’s election platform costing mandate. It is currently undergoing a refinement of its parameters and elasticities in response to the lessons learned during the 2019 election and to allow a link with the HERMES model.
DynEMIte
Following the 2019 election platform costing exercise, the Bureau began developing a new in-house DSGE model it has named DynEMIte. The model is largely based on the structure of QUEST III R&D, but incorporates multiple industries, intermediate consumption (and hence, input-output linkages between industries) and labour-augmenting semi-endogenous technological growth. It is currently in the calibration and estimation phase. The team is having problems with the convergence of the model under certain parameter values .
New environmental CGE model (unnamed)
The Bureau is developing a new computable general equilibrium (CGE) model to focus on climate and energy policy. It is in the very early stages of conceptual development and is supported by the Belgium Research Action Through Interdisciplinary Networks (BRAIN-BE). It is intended to be a standard multi-sector recursive dynamic model covering Belgium and its regions with an emphasis on energy inputs and heterogeneous labour demand. It is also to be linked to microsimulation data for distributional analysis.
During interviews with analysts and stakeholders it was clear that the Bureau has many strengths. For example:
The Bureau has renowned expertise internally and productive relationships with external experts in both the academic and practitioner communities.
It has unparalleled data access and relationships with government ministries compared to other OECD fiscal institutions stemming from its engagement in Belgium’s statistical framework and its role in directly supporting government departments, committees, and other stakeholders. More generally, the administrative and survey data available in Belgium facilitates economic and fiscal models would make researchers in other countries envious.
Analysts at the Bureau have leveraged their expertise, data, and relationships to develop a remarkable suite of sophisticated models and have cemented the institution as a leader in the modelling community.
Even institutions at the leading edge of policy analysis can learn from outside views and fresh perspectives. In that spirit, the review team identified several areas where the Bureau could benefit from reviewing its practices and receiving support from its peers in the OECD Working Party of Parliamentary Budget Officials and Independent Fiscal Institutions.
2.6.1. Summary of overall recommendations for model coverage and workflow
1. The institution’s strength — its collection of sophisticated models — may become a weakness. The strict commitment to rigid supermodel frameworks means long lead times to fulfil requests in new areas of analysis. Where other fiscal institutions in some cases prefer to fulfil novel requests in a quick manner, often within 48 hours, the Bureau generally prefers several months or even years of lead time to analysis to the same sophistication it has grown accustomed. In all cases, there should be a reflection on the potential gain in terms of precision and accuracy from additional months of development.
Further, sophisticated modelling can miss simple but valuable analysis. For example, IFIs commonly introduce behavioural adjustments from the academic literature to make simple calculations that may not have the depth of a microsimulation model run on administrative data but may ultimately provide reasonable results.
The Bureau also overestimates the sophistication of stakeholders and their ability to understand the models and commit their purpose to memory. For example, on the election platform website and publications, there are references to model names, which – when seen out of context - may be confusing for non-technical stakeholders.
Finally, with such gold-standard hammers, all analysis tends to become nails—policies that may not be worth an exhaustive assessment of every facet of economic consequences receive full treatment, taking analytical effort away from areas where it may be better prioritised.
Recommendation: As a sense test and communication device for cost estimates, the Bureau should make a habit of accompanying the results of its showpiece models with high-level “back-of-the-envelope” calculations of the financial costs with simple behavioural responses taken from the literature to make their results more transparent and intuitive for outside stakeholders to replicate. It could also provide more information and detail on the financial costs of policies as part of its election platform costings. For example, rather than only showing the ultimate impact on the budget deficits, it could show each offsetting line of higher revenues or expenses, the different affected budget line categories, and any breakdowns of more granular costs and revenues underlying the ultimate budget impact.
2. The Bureau has undertaken many more open-sourced projects than stakeholders and peers may be aware of.
Recommendation: The Bureau should promote its existing open-source tools and models and continue to proactively make its work available to the public. It should advertise a Bureau-branded GitHub repository and participate in or host conferences, code camps in collaboration with universities and think tanks, and do more modelling community outreach. These efforts can pay dividends in forming relationships that result in other people contributing to model refinements that the Bureau can then leverage. The Bureau should continue to transition its models to open-source collaborative working approaches with modern software with widespread use in the economic community.
3. The Bureau once was a leader in the multi-country HERMES Club initiated by EU institutions to jointly address challenges of the 1980s energy and inflation crisis. With the European Union facing similar conditions again, it may be worth revisiting that project. The world has changed, with better modelling software options, better harmonisation of national accounts, and a more connected EU with greater institutional support. Further, large-scale macroeconometric models are regaining favour in the academic and practitioner community.
Recommendation: The Bureau should explore the appetite among institutions in other countries for restarting the HERMES project. If there is sufficient demand, it could look for partners in other domestic institutions or international organisation for funding to explore converting HERMES to open-sourced software hosted on GitHub for a collaborative modelling initiative across European Member States.
4. The Bureau does not devote as much attention to the measurement of the business cycle and cyclical budget as other jurisdictions. This is largely because the business cycle in Belgium is subject to more limited variation, making the issue of measuring the output gap less important. However, Belgium is qualified by the European Commission as a “high risk country” in terms of public debt and the issue may come under the spotlight if Belgium was to be under the Excessive Deficit Procedure.
Recommendation: To get ahead of potential future contention, the Bureau could explore alternative tools to assess the output gap from different perspectives, such as the heat maps used by peers, or the suite of model averaging used by the Irish Fiscal Advisory Council.
5. The Bureau does not currently have statistical forecasting tools as a sense check for its structural models. For example, dynamic factor analysis for nowcasting, or vector autoregression models (VARs) for short-run forecasts up to eight quarters.
Recommendation: The Bureau should explore ready-made or out of the box dynamic factor models and simple reduced-form VARs to serve as a sense check for its structural models and for adding nowcasting capacity to its model suite.
2.6.2. Summary of key recommendations for individual models
HERMES. The Bureau should review the theoretical basis for using futures markets quotations as short-run forecasts for oil prices, exchange rates and interest rates—a practice that is common but has a poor theoretical justification and poor forecasting track record.
EXPEDITION and TYPECAST. The Bureau should continue to develop EXPEDITION and invest in the following areas to improve its usefulness for policy analysis: (1) Work with data providers to refine the co-ordination process, (2) Improve the model’s link to HINT to leverage the greater detail that HINT will produce in the next election, which will take place in the context of a sustained period of significant price volatility, (3) Work with STATBEL to compare and contrast its analysis on disposable income, (4) Review construction of the weights and matching to uprate and reweight tax benefit years to bring them forward to more recent years, (5) Support the main model with satellite models to adjust results for implementation date and policy year, (6) Explore how to present the results of TYPECAST in a more comprehensive and comprehendible way for a broader audience, (7) Explore a link with the LASER labour supply model, (8) Refine the model so it has the capacity to simulate policies in more detail for wealth taxes, personal income taxes, and regional policies.
HINT. The Bureau should explore additional links to environmental taxes, more direct interaction with EXPEDITION, and options to refine income quartiles, which are not immediately comparable to other models which present results as quintiles and deciles. The Bureau should explore satellite models that could impose assumptions on own-price elasticities of demand and cross-price elasticities that could be used to complement or refine the results of HINT.
PLANET. The Bureau should continue to invest in a solution to redesign the freight transport module and adopt an approach more in line with physical flows and less constrained by a theoretical allocation of economic flows. This would allow improved modelling of mode choices and geographical influences of freight transport demand. They should leverage their close connection with the network of Belgium statistical agencies to fill any data gaps to address the gap in freight transport modelling capacity.
LASER. As the Bureau continues to improve LASER, it should study the possibility of (1) expanding the target population with more types of workers, such as unemployed and self-employed, (2) linking administrative data to data from the labour force survey, and (3) using LASER’s elasticities in the new environmental CGE model.
References
[4] Bassilière, D. et al. (2013), A new version of the HERMES model - HERMES III.
[3] Bassilière, D. et al. (2018), Description and use of HERMES: : Document drafted in the framework of preparing for the 2019 costing of electoral programs, https://www.plan.be/uploaded/documents/201901110952260.WP-1-DC2019_HERMES_11843_F.pdf.
[2] Bassilière, D., L. Dobbelaere and F. Vanhorebeek (2018), How the HERMES model works - Description using variants, https://www.plan.be/publications/publication-1822-fr-le_fonctionnement_du_modele_hermes_description_a_l_aide_de_variantes.
[9] Baudewyns, D., A. Dewatripont and P. Michiels (2020), Labor cost reduction measures: what is the effect on employment and public finances in the Brussels Region?, https://www.plan.be/publications/article-2013-fr-les_mesures_de_reduction_du_cout_du_travail_quel_effet_sur_l_emploi_et_les_finances_publiques_en_region_bruxelloise.
[7] Baudewyns, D. and V. Lutgen (2022), How the HERMREG bottom-up model works: A description using variants, https://www.plan.be/uploaded/documents/202202030749560.WP_2202_12562_F.pdf.
[6] Baudewyns, D. and V. Lutgen (2022), The HERMREG bottom-up model: A multiregional model of the Belgian economy, https://www.plan.be/uploaded/documents/202202030739580.WP_2201_12561_F.pdf.
[10] Biatour, B. et al. (2018), Description of the QUEST III R&D model, https://www.plan.be/publications/publication-1848-fr-description_du_modele_quest_iii_r_d.
[12] Biatour, B. et al. (2017), Public Investment in Belgium - Current State and Economic Impact, https://www.plan.be/publications/publication-1650-fr-public_investment_in_belgium_current_state_economic_impact.
[25] Bureau, F. (2022), Economic Budget – Economic forecasts 2022-23 for September 2022, Federal Planning Bureau, https://www.plan.be/publications/publication-2283-fr-budget_economique_previsions_economiques_2022_2023_de_septembre_2022.
[1] Communities, C. (ed.) (1993), HERMES, Elsevier, https://doi.org/10.1016/C2009-0-10171-X.
[32] Daubresse, C. et al. (2018), Description and use of the PLANET model, https://www.plan.be/uploaded/documents/201901111505430.WP-6-DC2019_PLANET_11848_F.pdf.
[31] Daubresse, C. and B. Laine (2020), Telework and transport demand: an evaluation in the PLANET model, https://www.plan.be/publications/publication-2059-fr-teletravail_et_demande_de_transport_une_evaluation_dans_le_modele_planet.
[30] Daubresse, C. and B. Laine (2020), The PLANET Model: Methodological Report, PLANET 4.0, https://www.plan.be/publications/publication-1967-fr-the_planet_model_methodological_report_planet_4_0.
[23] De Ketelbutter, B. et al. (2014), A new version of MODTRIM II, https://www.plan.be/uploaded/documents/201407111401360.WP_1405_10748.pdf.
[16] De Vil, G. et al. (2018), Description and use of the EXPEDITION model, Federal Planning Bureau, https://www.plan.be/publications/publication-1849-fr-description_et_utilisation_du_modele_expedition.
[19] Dekkers, G., R. De smet and K. Van den Bosch (2023), MIDAS 2.0: Revision of a dynamic microsimulation model, Federal Planning Bureau, https://www.plan.be/uploaded/documents/202301240701580.WP_2302_12752.pdf.
[20] Dekkers, G., R. Desmet and G. De Vil (2010), The long-term adequacy of the Belgian public pension system: An analysis based on the MIDAS model, https://www.plan.be/publications/publication-946-fr-the_long_term_adequacy_of_the_belgian_public_pension_system_an_analysis_based_on_the_midas_model.
[34] Devogelaer, D. (2018), Description and use of SUPER CRYSTAL GRID, https://www.plan.be/uploaded/documents/201901111032530.WP-5-DC2019_CrystalSG_11847_F.pdf.
[5] Federal Planning Bureau (2021), Macroeconomic and fiscal effects of the draft National Recovery and Resilience Plan: Report to the Secretary of State for Recovery and Strategic Investments, https://www.plan.be/publications/publication-2106-en-macroeconomic_fiscal_effects_of_the_draft_national_recovery_resilience_plan_report_to_the_secretary_of_state_for_recovery.
[28] Federal Planning Bureau (2006), Tools and methods used at the Federal Planning Bureau, Federal Planning Bureau, https://www.plan.be/uploaded/documents/200610230931210.wp0607_en.pdf.
[26] Geerts, J., K. Van den Bosch and P. Willemé (2018), Description and use of the PROMES model, https://www.plan.be/publications/publication-1850-fr-description_et_utilisation_du_modele_promes.
[33] Gusbin, D. et al. (2010), The PLANET model methodological report: Modelling of short sea shipping and bus-tram-metro, https://www.plan.be/uploaded/documents/201007060844530.wp201016.pdf.
[24] Hertveldt, B. and I. Lebrun (2003), MODTRIM II: a quarterly model for the Belgium economy.
[29] High Council of FInance (2020), Outlook 2019-2070: a sharp increase in social spending until 2040 but a decrease in the risk of poverty of pensioners, https://www.plan.be/press/communique-2019-fr-perspectives_2019_2070_une_augmentation_prononcee_.
[22] High Council of Finance (2022), Annual report of the Study Committee on Ageing, https://www.plan.be/publications/publication-2263-en-study_committee_on_ageing_yearly_report_2022.
[8] Hoorelbeke, D. et al. (2007), HERMREG: A regionalisation model for Belgium, https://ecomod.net/sites/default/files/document-conference/ecomod2007/93.pdf.
[11] Kegels, C. and D. Verwerft (2018), Economic impact of professional services reform in Belgium - A DSGE simulation, https://www.plan.be/publications/publication-1798-fr-economic_impact_of_professional_services_reform_in_belgium_a_dsge_simulation.
[35] Korinek, A. (2015), Thoughts on DSGE Macroeconomics: Matching the Moment, But Missing the Point?, https://www.ineteconomics.org/uploads/downloads/Korinek-DSGE-Macro-Essay.pdf.
[27] Lefèvre, M. and S. Gerkens (2021), Assessing the sustainability of the Belgian health system using projections, https://kce.fgov.be/en/evaluatie-van-de-duurzaamheid-van-het-belgische-gezondheidssysteem-op-basis-van-projecties-1.
[15] Liu, E. and S. Ma (2022), Innovation Networks and R&D Allocation.
[18] Nevejan, H. (2021), Regional Child Benefit Reforms - An Impact Analysis Using the EXPEDITION Microsimulation Model, https://www.plan.be/uploaded/documents/202105180544440.WP_2104_12405.pdf.
[13] Rotemburg, J. (1982), “Monopolistic Price Adjustment and Aggregate Output”, Review of Economic Studies, pp. 517-531.
[14] Smets, F. and R. Wouters (2003), “An estimated dynamic stochastic general equilibrium model of the euro area”, Journal of the European Economic Association, Vol. 1, pp. 1123-1175.
[21] Social Protection Committee (SPC) and the European Commission (DG EMPL) (2021), 2021 Pension adequacy report, https://op.europa.eu/en/publication-detail/-/publication/4ee6cadd-cd83-11eb-ac72-01aa75ed71a1.
[36] Stiglitz, J. (2017), Where Modern Macroeconomics Went Wrong, NBER Working Paper 23795, https://www.nber.org/papers/w23795.
[17] Thuy, Y., G. Van Camp and D. Vandelannoote (2020), COVID-19 Crisis: A simulation of the impact of the loss of wages for temporary unemployment in the case of force majeure and the loss of income in the case of bridging rights, pp. 149-174, https://socialsecurity.belgium.be/sites/default/files/content/docs/nl/publicaties/btsz/2020/btsz-2020-1-covid-19-crisis-simulatie-impact-van-het-loonverlies-en-het-inkomensverlies.pdf.
Notes
← 1. Although the Bureau does compute these statistics and publishes them occasionally. See for instance - https://www.plan.be/uploaded/documents/201902280925280.PP_117_11866_F.pdf
← 2. Initially "Intégrateur d'outils de développement économétrique”