Annex A. Examples of digital transformations in national statistical offices (country examples)
The authors and PARIS21 staff conducted an extensive literature review as well as interviews with representatives of a few select national or territorial statistical offices (NSOs) in October 2021. The small number of interviews, however, does not allow to imply any statistically significant representativeness. The interviewed NSOs have, however, been selected based on PARIS21 and the OECD’s experience and shall provide meaningful insights to understand the key challenges that NSOs from low and middle-income countries and territories face towards a successful digital transformation.
Representatives of these NSOs were interviewed:
ANSD Agence Nationale de Statistique et de la Démographie, Senegal
DANE National Administrative Department of Statistics, Colombia
KNBS Kenya National Bureau of Statistics
NIS National Institute of Statistics of Cambodia
NSO-MN National Statistics Office of Mongolia
PCBS Palestinian Central Bureau of Statistics
NSO-MN is increasingly using digital technologies for data collection. While the tools and methods are not yet fully harmonised and integrated yet, computer-assisted personal interviewing, computer-assisted web interviewing and computer-assisted telephone interviewing are increasingly used for most surveys and censuses. The lack of dedicated resources and staff prevents the necessary harmonisation and integration of tools and approaches across statistical sectors. While some surveys have been successfully digitalised, others lag behind, which means paper must be used alongside digital tools. A few key blockers have been identified:
A comprehensive change management unit or department to support a radical shift in the way surveys are conceptualised is lacking. In some cases, the surveys are built along with the same logic as when they were paper based (old planning templates) and only subsequently made digital, though the entire statistical chain should have been re-thought to accommodate digital approaches.
Inertia is a constraint. In an NSO, as in any company or institution, people are often reluctant to change, especially when change comes (at first) with additional burdens but unclear and/or unknown benefits (leading to reactions such as “it’s not the right time for this” or “try this somewhere else first”).
Some surveys need to be filled out by respondents. While they are given the option to do so either by completing a paper questionnaire or electronically through a smartphone application, many respondents refused the digital option, fearing that the application might secretly collect private information.
Lessons learned
Even when the technological tools exist and are in use — in this case, digital tools for data collection — a comprehensive digital transformation can only succeed if frame conditions that are not purely technical are in place. For instance, a strategic and holistic approach to digital transformation is needed. Doing business as usual and merely replacing the paper questionnaire with tablets and mobile apps might offer short-term successes (e.g. data will be collected more quickly and more efficiently). But other pitfalls will present themselves, including the difficulty to adapt the data collection tool to other surveys or integrating the collected data within the statistical production chain. Other requirements of a comprehensive digital transformation include:
communication and trust building, both internally (to convince staff that the digital transformation will benefit them and their work) and externally (to demonstrate to respondents and the general public that the NSO is collecting data in a transparent way and for virtuous purposes and that privacy is fully respected.
All in all, strong political support and high-level co-ordination, within the NSO but also within the whole statistical system, are of paramount importance, especially to facilitate the resolution of situations that are not well underway.
The Senegalese NSS is decentralised, with some surveys and statistics conducted by the statistical departments of line ministries rather than the ANSD. Since the reform of the NSS in 2004, the ANSD has, therefore, in addition to its traditional role as a producer of statistics, taken on new roles to promote, co-ordinate, harmonise and improve the production of data and statistics in the country as well as across the African continent. These include:
With the establishment in 2021 of the visa statistique, any institution performing a survey must first have the approval of the ANSD, which is to validate the survey’s quality and coherence, among other aspects, and ensure the standardisation of data (e.g. the usage of standard and/or official coding systems and ontologies).
Through the National School of Statistics and Economic Analysis that it hosts, the ANSD leads initial and continuing statistical training programmes aimed at strengthening the capacities of other institutions in terms of statistical production.
The ANSD has established a certification programme for surveyors that, among other things, has provided new professional opportunities to staff (e.g. data entry officers) whose roles are becoming less relevant as a consequence of the ongoing digitalisation.
The ANSD is one of the “centers of reference” for the Collection of Electronic Data initiative, which sets the bar for South-South co-operation in terms of digital transformation. In the frame of this initiative, the ANSD, together with its counterparts from Brazil and Cabo Verde, shares its experience as an African frontrunner in terms of electronic data collection and electronic censuses.
The shift in the ANSD’s main areas of responsibility to include national and regional data stewardship is embodied in several laws and policies, including the following:
Senegal’s third national strategy for the development of statistics) rests on four pillars aimed at increasing, among other things, the user-centricity of statistical production, digital transformation, and regional and international co-operation.
The country’s Digital Strategy 25 (Stratégie Numérique Sénégal) envisions a national digital transformation for all and for all usages within an efficient digital ecosystem. One of its priorities is to put a digital administration at the service of citizens and businesses.
Lessons learned
As the users of statistics are diversifying their needs and requirements, no NSS can limit itself to its traditional roles, tools and processes. Moreover, synergies and harmonisation between all NSS actors are needed to rationalise, improve and speed up the production and dissemination of data and statistics in various forms and formats.
To address those emerging needs and requirements, the ANSD has efficiently taken up new roles and responsibilities, advancing from being purely a producer of statistics to being the data steward and guardian of data best practices and expertise. Two core framework conditions supported this shift:
strong political and institutional support in the form of laws, policies and strategies that explicitly call for new mindsets and approaches toward the digital transformation
user-centricity to address the needs and expectations of both data producers and data users.
By diversifying its activities and providing technical support to other state institutions, the ANSD is directly contributing to a smoother and wider adoption of modern digital technologies and to a better and more harmonised data production within the entire NSS beyond the ANSD itself.
This diversification of the ANSD's activities and competencies is also strengthening its role in terms of co-operation with donors and development partners. Thanks to the clear digital vision and strategies, it is able to take a big-picture approach to evaluate suggestions and ideas from externally funded projects after examining them critically and transparently. Among other benefits, this can help avoid an unwanted multiplication of platforms or a breach of national laws and priorities (e.g. refusing or adapting a project that would host official data on servers outside of Senegal).
User needs and requirements are at the core of the strategy and operations of the NIS, as outlined in the first strategic objective of its National Strategy for the Development of Statistics 2019-2023: to “develop, implement, and maintain a core data set of high quality by all institutions of the NSS under the leadership of the NIS to adequately respond to the statistical requirements of development processes arising from [the National Strategic Development Plan, the Sustainable Development Goal and Association of Southeast Asian Nations] sectoral strategic plans commitments and the needs of other users”.
This core principle, therefore, guides the digital transformation of the NIS, particularly observed in the field of data dissemination. Several portals — among them CamStat (http://camstat.nis.gov.kh/) — are based on the Statistical Information System Collaboration Community’s .Stat Suite and have been deployed to facilitate user interactions with statistical data. Specific training and workshops have been conducted to help line ministries and other partners use these portals (e.g. in the production of customised tables).
To decrease the risks of duplication of platforms and to simplify and rationalise their institutional and technical maintenance, the NSI is in the process of finalising an information and communication technology (ICT) policy and strategic plan that will set a flexible and coherent frame for further digital developments. This plan focuses heavily on data security and data governance.
Lessons learned
User-centricity is a core element of a successful digital transformation. However, since users and partners often have different and even contradictory needs and requirements, a fully frameless user-centricity would increase the risk of engaging in ad hoc, poorly co-ordinated developments (duplicated platforms and tools, incompatible approaches and formats, weak data governance due to an increasing diversification of data storage locations, etc.).
As the example of the Cambodia NSI shows, a clear and binding (albeit flexible) ICT strategic plan can ensure that digital developments always take place within a coherent frame. Besides increasing the security and maintainability of the digital ecosystem, such a plan also allows an NSO to stay in charge of its own data governance, for example, by ensuring that the statistical data along its entire value chain are stored and processed within a data warehouse and not distributed across a wide array of external servers and cloud infrastructures.
By facilitating data integration across statistical sectors and by minimising the extent of external user interfaces, implementing an ICT strategic plan will also help the NSO respond more efficiently to more complex and transversal user requirements and thus increase its level of user-centricity.
The PCBS has achieved a very advanced level of digitalisation for both statistical processes and support and administrative processes:
Most statistics and surveys are collected electronically using tablets aided by geographic information systems.
There is a coherent digital data dissemination ecosystem consisting of:
a statistical catalogue (metadata) - https://pcbs.gov.ps/Stat-Data-Catalogue/default
co-operation within a national team to produce the interoperability portal - http://zinnar.pna.ps/InteroperabilityPortal/.
a central dissemination portal - https://pcbs.gov.ps/site/lang__en/507/default.aspx.
Most administrative services are digitalised in a management information system (human resources, payroll, assets, etc.).
The digital development of the PCBS is strongly anchored in the 2018-22 National Strategy for the Development of Official Statistics, its third such strategy. The following expected outputs (and their corresponding number in the PCBS strategy) are fundamental to its continuous digital development:
1.2.2. Statistical databases for the national statistical system (NSS) are modernised, including Sustainable Development Goals.
1.2.3. Geospatial statistical databases are available to be utilised.
2.3.2. Updated version of computerised forms for administrative indicators.
3.1.2. IT infrastructure (including security for storage and data flow) is available and used in the NSS.
An indirect result of its ongoing digital transformation is that the PCBS is considered a regional leader in terms of open data, which highlights how digitalisation and open data are closely interlinked. To buttress its position as a local and regional frontrunner, the PCBS also invests in data science, mainly by contributing to the design and development of a data science curriculum for local universities.
Lessons learned
Cutting-edge digital technologies and approaches such as data science will require NSOs to acquire and retain specific technical skills in the coming years. As those skills are not widely available in most low- and middle-income countries, NSOs can take a proactive role to help build the next generation(s) of statisticians whose skills will be vastly different from those of the current generation. In addition to skilled developers, researchers and data miners, it will be necessary to train a new generation of managers and sectoral statisticians who better understand the potential of cutting-edge digital technologies as well but also their risks and challenges and who can better plan, design and integrate these in the statistical chain.
The digital transformation of KNBS is very closely intertwined with the support and involvement of donors and development partners, especially in the field of data dissemination. Partners and projects have deployed various platforms used to visualise and disseminate official statistics. As a result, the KNBS website currently references eight data-sharing and dissemination platforms, all with some level of overlap. Not all of the platforms are fully functional and updated. The eight are as follows:
KNBS Data Visualisation is geared to enhance the statistics bureau’s data dissemination capabilities (https://knbs.or.ke/visualizations/).
Kenya OpenData enhances the data dissemination capabilities of the KNBS to increase the openness of national data (https://opendata.go.ke/).
KenInfo http://devinfo.org/keninfo/.
Kenya Data Portal presents Kenya data (https://kenya.opendataforafrica.org/).
Integrated Multisectoral Information System, used to visualise various published datasets (https://statistics.knbs.or.ke/binken/RpWebEngine.exe/Portal).
KeNADA is the Kenya National Data Archive (https://statistics.knbs.or.ke/nada/index.php/home).
FinAccess data includes financial access status (https://sites.google.com/view/finaccess-kenya/financial-inclusion-status).
Socio-Economic Atlas of Kenya features high-resolution data (https://www.kenya-atlas.org/index.html).
The existence of multiple platforms presents challenges and difficulties for KNBS, including:
ensuring a high level of technical and institutional ownership over the platforms
less clarity for the users searching for official data and statistics.
Lessons learned
Externally funded and externally conceptualised projects tend to have a technology-first approach, mostly emphasising the advantages of a specific tool or technology over the tools and technologies already used within the NSO. Often, this approach overrules the other dimensions of a successful digital transformation and leads to inadequate ownership of the newly introduced technologies, duplication of efforts and poor sustainability. Decision makers can avoid such a situation by considering a set of core questions before determining whether an NSO should adopt a suggested new tool or technology:
Are there tools in place that do the same or more or less the same?
Which tool — the old or the new — offers more universality (given that a tool that is widely used by the statistical community will usually be easier and cheaper to use and maintain due to synergies, etc.)?
Are the already existing tools and/or approaches well anchored and maintained in the NSO? Is there already strong ownership? If yes, the (presumed) technological benefit of the newly suggested tool should be weighed against the challenges it entails, and the following points should be considered:
integration and/or replacement (moving data and practices from the old to the new tool and adapting the processes within the data value chain)
capacity development (training staff who are proficient in the old system to be able to use the new one)
change management (careful planning for and implementation of the technological change)
finances (i.e. is the new tool or approach cheaper or more expensive to maintain in terms of license costs, required skills, etc.?).
need to inform internal and external users about using the new tool (reasons for adopting a new tool, expected benefits, release calendar, impact on the users, etc.).
These questions are not meant to deter NSOs from making changes or consider technological changes to be bad. Rather, they are reminders that it is not advisable to deploy a new tool simply because it is believed to be technologically better before analysing, understanding and addressing the implications in other dimensions.
Additionally, a strong and transparent digital governance model can clearly frame and set boundaries for the usage and deployment of technologies:
A clear information technology (IT) and digital strategy can provide a strong and coherent frame to avoid the ad-hoc deployment of new tools, approaches and technologies.
An efficient and transparent co-ordination mechanism can help harmonise all internally and externally funded projects to avoid duplication of effort and increase synergies.
Transparent co-operation is needed among donors, development partners and NSOs to ensure that the long-term vision, requirements and strategies of the NSO take precedence over the shorter-term needs and requirements of specific external projects.
DANE started its digital transformation in the field of data collection in 2003 and now relies heavily on digital tools and approaches, for instance, using tablets for face-to-face surveys and dedicated web applications for business surveys. Given the fast pace of technological change, one of DANE’s highest priorities is to ensure the continuous updating and improvement of its web tools and platforms to stay ahead of unavoidable digital obsolescence, guaranteeing users’ access to the services provided, and increase the efficiency of its digital infrastructure.
To speed up the adoption of new technologies, DANE has established an informal innovation and development lab aimed at evaluating advanced, promising technologies and methods such as NoSQL, Hadoop, Spark and machine learning and to assess the extent to which these technologies can improve the services DANE provides to its users. As an example, the online census carried out in Q1 2018 was built on more advanced, scalable and robust technologies (Mongo) and had a much higher response rate than expected, reaching more than 4 million people in 40 days and lower overall costs of data collection.
Lessons learned
The direct and indirect benefits of a specific digital tool or approach are very difficult to anticipate: The same technology might allow a significant advance in one context but prove to be an unmaintainable burden in another one. It is important to recognise that digital transformation should also be an enabling asset to achieve something as complex as organisational culture and even the mindset of civil servants of the NSO, which reminds us of the need of mainstreaming this progress across the NSO and not just in a specific unit.
Moreover, enabling flexible testing and piloting scenarios for promising technologies helps an NSO identify technologies that have good potential and will reduce the challenges and difficulties once the technology is put into use. Such an approach can build upon the following concepts:
An innovation lab and/or department should be explicitly tasked to actively scout for technology in close collaboration with sectoral departments and with users.
Agile project management (Scrum or similar) can vastly reduce the costs and duration of testing new technologies by ensuring that frequent feedback loops with all stakeholders are timely and wisely taken into account.
Putting the users at the centre of the testing process — considering what the users need and want and how the NSO can benefit them — determining which technologies are more hype than relevant will also ensure that technologies of low value can be discarded early on.