3. Cities’ innovation capacity – the way forward

This chapter begins with a discussion on the importance of assessing innovation initiatives in the public sector. Then it will discuss some of the key gaps that researchers and policy makers need to fill to advance the field and support cities’ innovation capacity. The chapter will then move to present the Athens Road Map on Innovation for Inclusive Growth as an additional tool for cities. Finally, the chapter will propose a checklist of measures cities could take to improve their innovation capacity based on the findings of the OECD/Bloomberg Survey of Innovation Capacity in Cities.

    

The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law.

Assessing innovation outcomes

Why assessing innovation matters

Assessing the outcomes of innovation in cities fosters accountability to citizens and donors, helps determine the effectiveness of the use of public resources, and establishes the contribution of the projects to achieve the city’s socio-economic development goals. Like in the rest of the public sector, assessment is of particular political, economic and social importance. It could also help legitimise the city administration’s engagement in particular innovation projects or areas. Assessing innovation in the local public service is motivated in part by the demand for benchmarking the efficiency and quality of public services and identifying the factors that contribute to desirable innovation outputs and outcomes (OECD/Eurostat, 2018[1]).

However, defining to what extent the success of an innovation policy stems from the city administration’s effort to innovate is a complex matter. One of the key barriers is that the existing evidence is still rather anecdotal and limited to specific sectors. Moreover, while the project that brings about new services, products or processes may be due, in part, to an innovative measure, it may also be influenced by a range of other policy decisions and measures. For example, in 2015 Louisville, KY (United States) launched the AIR Louisville project, which aimed to measure air quality through crowdsourced data. The project is reported to have led to better health outcomes for residents and the entire community through new municipal policies addressing traffic congestion and increasing tree planting. Assessing the outcomes of the programme has led to the development of data-driven policies to improve air quality. Now the city is in discussion with other cities that face similar challenges with air quality and its impact on public health to replicate the project in their own communities.1

Qualitative and quantitative methods that assess innovation performance deserve further research and analysis. The evaluation of innovation outcomes requires an approach that encompasses a variety of factors and recognises the difficulties of establishing and demonstrating causal links. An innovation project may also have unintended results that should also be considered as part of the evaluation process.

How cities assess the impact of innovation projects

Formal innovation goals are one of the basic elements that cities can use to assess innovation outcomes. According to the survey, of the cities that reported having formal innovation goals, a little over half felt that they were doing “well” or “so-so” in achieving their innovation goals (Figure 3.1). For almost one-third of the cities it is too early to tell, which suggests they are in the primary stages of their innovation efforts.

Formal innovation objectives offer a great starting point to assess the effectiveness of innovation in a city. There are also a number of other activities that can assess their innovation outcomes. Overall, 43% of surveyed cities reported using resident engagement and sense of community, as well as quality and accessibility of public services, to determine whether innovation efforts are effective. A further 40% of cities use economic development. In addition, cities need to customise their methodologies for assessing innovation outcomes depending on the nature of their projects. The Oslo Manual: Guidelines for Collecting, Reporting and Using Data on Innovation provides cities with a framework for collecting, interpreting and measuring data on innovation (OECD/Eurostat, 2018[1]). The use of this manual could help cities create custom assessments to better understand the effectiveness of their innovation efforts.

Figure 3.1. Cities achieving their stated innovation goals

Notes: Out of 89 surveyed cities, 49 reported having formal innovation goals. The figure represents responses from these 49 cities to Question 5.2 “How would you say that your city is doing with regards to meeting your stated innovation goals?”.

Source: OECD/Bloomberg Survey on Innovation Capacity in Cities 2018.

Box 3.1. Select examples of city initiatives to assess innovation outcomes

• In Kansas City, KS (United States), the innovation department has been working with the mayor and the county administrator on evaluating the productivity of land-use policies. The work is shifting perceptions about the relative value of different types of development and driving conversations about how to reinvest in traditional neighbourhoods that can drive a higher return on investment and create a virtuous cycle of positive revenue gains.

• Detroit, MI (United States) measures the key outcomes of its innovation efforts by their contribution to the city’s overall strategic goals. The city also has key indicators of progress on priority areas such as blighted buildings demolished, rental residences registered, etc. For instance, the assessment of the “HHF Blight Elimination Program” revealed that each HHF (Hardest Hit Fund) demolition increased the value of homes within 500 feet by 4.2%.

Source: OECD/Bloomberg Survey on Innovation Capacity in Cities 2018. Answers of Kansas and Detroit to Question 5.7 “What specific outcomes are you measuring to determine whether innovation efforts in your city are effective?”. For Detroit, Policy Brief: Detroit Blight Elimination Program Neighborhood Impact, https://static1.squarespace.com/static/55e8c061e4b018cc4b5864bc/t/56151816e4b0f9222d31ae0a/1444222998440/20151006+Demolition+Report%5B1%5D%5B2%5D.pdf.

What problems are encountered to assess innovation

Overall, 16% of cities with formal innovation goals conduct a comprehensive and systematic evaluation of the impacts of their innovation strategy (Figure 3.2). Across surveyed cities, there seem to be poor evaluation systems of innovation initiatives or projects. The large majority of cities only assess some elements of their innovation strategy and consider it too early to tell if they are effective.

Figure 3.2. Systematic assessment of the impact of innovation strategy by cities

Note: Out of 89 surveyed cities, 86 responded to Question 5.4 “Does your municipality undertake a systematic assessment or evaluation of the impact of your innovation strategy?”.

Source: OECD/Bloomberg Survey on Innovation Capacity in Cities 2018.

There are four main points that can be drawn from these results.

1. 1. First, cities may lack the technical capacity and methodological instruments such as indicators to assess their innovation strategies. Cities seem to struggle to monitor the implementation of their innovation strategies and, therefore, lack information and data that assist them in an intermediate and long-term evaluation.

2. 2. Second, when cities do not monitor or assess their innovation strategies comprehensively, they are unable to identify unexpected results and suggest changes to the project or initiative. Conducting an ex ante evaluation to assess the monitoring process and suggest indicators as baselines for ex post evaluations may be needed (European Commission, 2012[2]).

3. 3. Third, cities may lack the financial resources to conduct systematic and comprehensive assessments of their innovation strategies. This process may be too expensive for some cities as assessments can, for instance, require interdisciplinary evaluation teams, which most of the time are composed of external agents such as universities; collecting data and conducting surveys; and dissemination of the results.

4. 4. Fourth, cities must also be prepared for the political implications of an assessment. Positive results may reflect well on the political leadership of a city, but negative results may require engaging in a process of negotiation with other political forces and justification to citizens. It may be difficult for a city government to budget for innovation and specific tools and projects until they are proven. When programmes aligned to strategies work, it is much easier for the city government to use taxpayer money to continue or scale work and for citizens to appreciate government experimentation and support innovation.2

Moreover, according to the survey, factors such as administrative fragmentation (silos), levels of staff turnover, loss of institutional memory or lack of methodologies may limit the assessment of cities’ innovation strategies (Box 3.2).

Figure 3.3. Factors that limit the evaluation of innovation strategies in cities

Box 3.2. Key challenges to measure innovation outcomes in cities

• The lack of methodologies to measure innovation outcomes, in particular early-stage innovation efforts. Generally, city departments report improvements in their operations, but the lack of a methodology makes it hard to determine to what extent any improvement was the product of innovation work. Due to the lack of methodologies, some departments tend to measure activity-based metrics rather than outcomes.

• The lack of systematic evaluations of innovation programmes, which limits the effectiveness of any measure as a barometer to determine success.

• Cities, in many cases, do not have enough relevant and reliable data to measure the impact of innovation outcomes. Cities still largely work in silos and data are not always shared within the local administration.

• Measuring the effectiveness of an innovation initiative is a medium to long-term process; during this time, loss of institutional memory and changes in leadership via staff transitions may set back efforts in measuring innovation outcomes.

• In some cases, cities focus on aspects that cannot be quantified, such as happiness, solidarity and people’s dignity, thus any measure would be largely subjective.

Source: Based on the OECD/Bloomberg Survey on Innovation Capacity in Cities 2018.

Innovation is helping cities improve services, operations and resident outcomes

Cities are innovating in a wide range of areas. These may vary from new ways of engaging with citizens to the redesign of recruitment processes or the use of ICTs for service delivery. According to the survey, cities considered that innovation is helping the most in improving service delivery (emergency services, housing, mobility and social services, etc.). Some cities are exploring innovative ways of providing services or designing new products to meet citizens’ demands.

Box 3.3. Select examples of service delivery innovation in cities

• Rio de Janeiro (Brazil) has established community centres called naves do conhecimento (naves of knowledge) across the city where the local government provides training courses (seminars, workshops) in areas of innovation and technology to the poorest and most vulnerable residents.

• Syracuse, NY (United States) launched an early intervention service that has reduced eviction rates for at-risk residents by identifying tenants who have trouble paying rent within a week of missing a payment, rather than two or three months when they are already going to eviction court.

• Austin, TX (United States) is prototyping technology solutions for people experiencing homelessness to access their identity and personal records. This initiative has raised awareness in the community about the problem of identity access.

• Long Beach, CA (United States) launched a Justice Lab to provide new tools to first responders to divert residents in need out of the criminal justice system and toward resources like treatment and care.

• Baltimore, MD (United States) – the innovation team and the police department have modernised the recruitment and hiring process of police officers through online applications, new exams focused on the profile of a police officer and piloting fitness boot camps.

Source: OECD/Bloomberg Survey on Innovation Capacity in Cities 2018.

Some other cities introduced innovation strategies to improve their organisational and administrative arrangements by redefining rules, procedures and structures. Another group of cities has introduced new ways of collaborating and co-operating with stakeholders to develop new products or provide new services.

Box 3.4. Select examples of organisational innovations in cities

• Montreal (Canada) is working with citizens, city employees and community leaders through the newly created “Urban Innovation Lab” to provide citizens with the tools to identify urban challenges and co-create solutions to build a more open, integrated and efficient city.

• San Francisco, CA (United States) – the Office of Civic Innovation supports city departments to introduce new approaches, resources and technologies to meet the city’s priorities. It works with city departments, community partners and residents to manage the impact of demographic, economic, social and environmental challenges in the city.

• Denver, CO (United States) – the employee-led performance framework called Peak Performance focuses on technologies, processes and organisations to align and structure the workforce around the mayor’s priorities.

• Georgetown, TX (United States) – the Performance Management Program and the Business Improvement Program seek to find effective solutions and inventive methods for service delivery by cultivating a culture of continuous improvement. The Business Improvement Program tears down department silos by promoting departmental review projects.

• Madrid’s (Spain) Strategic Plan for Innovation and Modernisation of Public Management aims to develop a model of management for a more efficient and effective public service delivery through: improvements in the relationship between citizens and government; modernisation of public policies linked to public service delivery; and budget stability by optimising existing resources.

• Tel Aviv’s (Israel) innovation team works in a collaborative way across the different municipal units to find solutions to problems by conducting research, developing ideas in partnership with stakeholders, preparing delivery (prototyping) and delivery. This way of working helps to overcome administrative silos for problem solving.

• Paris’ (France) open innovation strategy is based on the premise that innovative solutions for the city’s challenges will emerge from the collaboration among the city, the private sector, academia and citizens.

• Seoul’s (Korea) city plan is based on the creation of public-private partnerships and social innovation aimed to encourage the participation of citizens in all aspects of policy making and delivery.

Source: OECD/Bloomberg Survey on Innovation Capacity in Cities 2018. For Madrid: Comunidad de Madrid (2017), Plan Estratégico de Innovación y Modernización de la Gestión Pública de la Comunidad de Madrid, www.madrid.org/es/transparencia/informacion-institucional/planes-programas/plan-estrategico-innovacion-y-modernizacion-gestion.

Innovation is assisting cities to meet their strategic goals in service delivery (e.g. transport, water, waste collection), improvements in government operations (e.g. streamlining budget processes and workflows, fostering inter-agency co-operation), and improving residents’ outcomes (e.g. improving health or job outcomes) (Figure 3.4).

Figure 3.4. How innovation is helping cities improve their performance

Notes: The above results do not mean that innovation is exclusively helping to meet their objectives in one area. Cities were given the opportunity to rank their response and these three areas were the top ranked by cities.

Source: OECD/Bloomberg Survey on Innovation Capacity in Cities 2018, answers to Question 5.3 “What is innovation helping your city do better?”.

“Engaging residents and other stakeholders” was the area with the lowest ranking. This is not surprising either as citizen engagement is generally a means to improve resident outcomes, service delivery and internal government operations. The other areas with the most commonly unranked outcomes were “servicing current obligations” (e.g. pensions), and “generating new sources of revenue or resources for innovation”.

Cities that evaluate their innovation work are thus better positioned to scale up innovative projects that improve operations and less likely to engage in practices or projects that offer little return on investment. According to the survey results, cities that consistently evaluate the results of their innovation work have, across the board, greater familiarity with innovation than cities that lack procedural assessments (Figure 3.5). Due to the novelty of innovation work, it is often difficult to gauge its impact without impact assessments. Figure 3.5 also shows that cities that evaluate their innovation outcomes tend to be more experienced with foresight and prospective exercises, an area where cities engage the least.

Figure 3.5. Frequency of use of innovation approaches between cities with vs. without systematic evaluation of the impact of their innovation strategy

Notes: Out of 89 surveyed cities, 88 cities responded to Question 1.6 and 86 cities responded to Question 5.4. The figure represents results obtained by crossing the responses to Question 1.6 “What would you say is the level of use or experience your city has with each of the following innovation activities?” (1 = Never used; 3 = Use sometimes; 5 = Use often) with the responses to Question 5.4 “Does your municipality conduct a systematic assessment or evaluation of the impact of your innovation strategy?”. The vertical axis denotes average points that surveyed cities assign for each innovation activity (on a scale from 1 to 5). “Foresight, prospective exercises, scenario planning” as one of the pre-defined innovation activities for Question 1.6 was not ranked (i.e. left blank) by 61 out of 89 surveyed cities, accounting for some rank points being lower than 1.

Source: OECD/Bloomberg Survey on Innovation Capacity in Cities 2018.

The gaps in the research agenda

Enhancing innovation capacity in cities is a wide domain that would benefit from further research on several critical issues.

• Funding. Understanding how different forms of funding shape the nature of innovation work in cities – and how this may impact public service delivery through the shaping of the ideas that are developed across cities (when there is collaboration) – is a matter for further research. Moreover, exploring different ways in which cities could co-finance innovation projects for mutual benefit, particularly in metropolitan areas, could help cities of all sizes strengthen their capacity. Further research is needed on how funding and specific staff roles contribute to innovation in cities.

• Innovation teams. These teams have proven critical to strengthen cities’ innovation capacity. There are, however, some aspects that still require further research, including: a definition of the competences and skills of innovation officers, the type of training that is needed to facilitate their work, and the managerial flexibility teams should have to promote innovation across the local public administration. A future area of research may be dedicated to study the measures cities are implementing to enhance the work and longevity of innovation teams beyond political cycles.

• Human resource management for innovation. Exploring how cities can promote creativity and associative thinking in the local workforce through a more flexible management of human resources is an area that requires further exploration. More information is needed regarding how mobility programmes can increase the talent available in the local workforce, and how cities manage change in a way that is conducive to innovation. In addition, further research is needed on performance management and the kind of awards and recognition programmes that promote innovation in the local workforce. There is little information on how cities link the management of local employees to the institutional innovation goals and strategies of the city. How cities communicate these the goals and strategies to employees is also an area that requires further research.

• Risk management. A better understanding of how cities manage and reduce uncertainty when venturing into innovative projects and how cities assess the environment and set the pre-conditions for successful implementation of an innovation strategy may help move the research field forward.

• Public procurement for innovation in cities. At the national level, countries have been implementing a number of policy measures and instruments to encourage and foster the strategic use of public procurement to stimulate innovation in the public sector. At present, 80% of OECD countries support procurement for innovation and 50% have set an action plan as part of a broader innovation for procurement strategies (OECD, 2017[3]). Two of the main reasons for using public procurement for innovation are the growing demand for new products and services and the need to improve the performance of existing services and quality of goods at lower prices and higher levels of energy efficiency. However, these studies focus mostly on national level strategies. How cities make the most of procurement to support and unleash the impact of public sector innovation requires empirical research to provide a framework for action tailored to cities’ needs and contexts.

• Property rights. Patents and the ownership of intellectual findings could encourage creative activity. How cities use patents and intellectual property rights to enhance public sector innovation is an area that deserves further empirical research. Patents offer innovators recognition for their creativity and enable them to own the returns of their investments. This could be an incentive for cities to form partnerships with employees in the commercialisation and diffusion of innovative services and products. There is little research on how employees who come up with innovative ideas for improving public service delivery and products benefit from their creations.

• Civic engagement. Opening the public sector to civic engagement and innovation is not automatic. The increase in the implementation of engagement practices has not necessarily led to an increase in the perception of quality, trust, legitimacy and innovation. Two key issues to analyse in future research are the performance of engagement practices and who actually participates and benefits from engagement initiatives.

• Measuring innovation outcomes. How cities measure the impact of their innovation efforts is a matter for further study. There does not seem to be a clear path in how cities measure the effectiveness of their innovation strategy. The survey results do, however, show a growing awareness among cities of the need to measure outcomes. To achieve this, it is necessary to define the most appropriate types of measures to assess innovation. This research should include aspects such as metrics defined from the outset of the programme, monitoring progress, cost effectiveness, data collection for evaluation and indicators to define success.

A checklist for enhancing cities’ innovation capacity

As a result of this survey, we have found that cities can increase their innovation capacity when they excel in the following five factors: 1) innovation strategy; 2) leadership and staffing; 3) data use and capacity; 4) resources and funding; and 5) outcomes (evaluations and results). City governments support innovation in a wide variety of ways that largely depend on the socio-economic context, resource allocation and the sophistication of their administrative culture.

Based on these five factors, Box 3.5 provides a checklist that city leaders may wish to consider as they enhance their city’s innovation capacity. It is intended to be a tool for cities to evaluate their innovation capacity efforts. The checklist highlights key issues that may be considered during the process of promoting innovation as part of the city’s strategies for growth and development, while recognising that the social, economic and political diversity of cities require flexibility in the methods through which they promote innovation. The checklist is not an instrument for comparative purposes, but it can provide useful information for those cities interested in moving closer to international practice in developing their capacity and capability for innovation. The checklist provides key ideas to guide city leaders through the complexities of strengthening their organisational and administrative arrangements for innovation. The checklist is complemented with four actions that governments need to take to enhance their innovation capacity based on previous OECD (2015[4]; 2017[5]) research, which are organisational attributes influencing public sector innovation.

Box 3.5. Considerations for enhancing innovation capacity: A checklist for action

Innovation strategy

• Formulate an innovation strategy that gives direction for the short, medium and long term and align it to existing city priorities. Involve stakeholders from within and outside the local public sector in the development of the innovation strategy.

• Adopt a clear political message communicated by leadership that shows the importance of innovation in meeting the city’s goals.

• Create or engage in networks that allow learning from other organisations’ and cities’ innovation strategies. Develop norms that promote collaboration and embed it into the organisation’s culture.

• Advance new organisational structures and leverage partnerships to enhance approaches and tools, share risks, and harness available information and resources. This includes developing innovative methods to structure teams, break down silos and work in partnership across organisations and sectors (OECD, 2015[4]; 2017[5]).

Leadership and staffing

• Consider the installation of innovation units/teams and their placement within the administration according to the city’s priorities and enable distributed leadership.

• Develop the organisational, regulatory and governance structure that provides the environment and conditions to support a culture of innovation.

• Invest in the capacity and capability of public servants, which includes building the culture, incentives and norms to facilitate new ways of working. The way employees are selected and managed should also be considered, as it has an impact on an organisation’s innovation capacity (OECD, 2015[4]; 2017[5]).

• Ensure middle and senior managers as well as members of the innovation units/teams receive regular training in human-centred design and behavioural nudges. Ensure that products, services and systems address the core needs of those who experience the problem.

• Promote diversity in experiences and skills in the local public workforce through attracting and recruiting people from different backgrounds regardless of gender, race, age, income level, political views, etc.

• Ensure that internal rules and processes are balanced in their capacity to mitigate risk while processing resources and processing information (OECD, 2015[4]; 2017[5]).

• Promote a culture of taking reasonable risks and learning from failure.

Data management capacity

• Ensure the production, free flow, and utilisation of data and knowledge across the public sector to support decision making (OECD, 2015[4]; 2017[5]).

• Create collaborative partnerships with external actors to strengthen data management capability.

Resources and funding

• Set up a specific financing framework for supporting innovation work within the local administration.

• Allow more flexibility in budgeting management to reallocate funds and carry over unused funds.

• Promote innovative ways to pool financial resources across the administration where objectives align. Combining different sources of funding allows cities to build new partnerships internally and optimise resources.

• Explore and establish local revenue-raising initiatives to specifically target innovation efforts (e.g. competitive grants, competitions, land-based finance tools, crowdfunding, etc.).

Outcomes: Evaluations and results

• Conduct an impact evaluation of innovation projects/strategies and ensure their monitoring at all phases of implementation. This includes the impact evaluation of pilots and the creation of feedback groups to revise and adapt products and services.

• Ensure data (quantitative and qualitative) as well as the results of impact assessments and evaluations are used for decision making and improve as need be.

Another relevant instrument for cities to leverage the full potential of innovation is the Athens Road Map on Innovation for Inclusive Growth in Cities (see Annex B). In 2019, the OECD Champion Mayors for Inclusive Growth Initiative created and adopted the Road Map to promote policies and practices to build more prosperous and equitable cities where inhabitants enjoy high levels of well-being. Taking into account the specific national and local contexts, the Road Map proposes a series of actions in social innovation, technological innovation and public sector innovation which intend to upscale cities’ efforts to fight inequalities and foster inclusive growth.

Next steps

The first phase of the project, based on the pilot OECD/Bloomberg Survey of Innovation Capacity in Cities, scanned and identified initial patterns related to cities’ innovation capacity and their strategy, goals and approaches, staffing and structure, funding and resources, data use, and outcomes. Building on the findings of the pilot, the next phase of the study aims to close gaps in understanding about data use and innovation investments in cities and explore whether cities’ efforts and investments to strengthen their innovation capacity and capacity to use data is reflected in their citizens’ outcomes. This phase will:

• Go deeper on data. In connection with Bloomberg Philanthropies’ What Works Cities programme, as the move toward open data governance initiatives, evidence and data-driven decision making grows, it is important to understand the current state of data use in cities, how local governments are managing their data and to what extent their data use capabilities are leading to the well-being of residents. In this new exploration between the OECD and Bloomberg Philanthropies based on cities participating in the What Works Cities programme’s assessment framework, an examination of the data ventures based on the What Works Cities’ 45-point criteria will be used to determine the association between using data and selected well-being dimensions such as perceived quality and trust in local government, and of other well-being outcomes: education, employment, health, public transport access, satisfaction with city administrations or their service provision. This will include an examination of how cities use and produce data to guide decision making and will be a central part of the empirical research.

• Investigate innovation capacity and well-being: Developing an evidence base for innovation in cities by understanding the relationship between particular innovation capacity components and the impacts that result along material, quality of life and subjective well-being indicators related to employment, public resource access and satisfaction with city services.

• Develop policy recommendations. Transforming research into action by making the findings actionable is a key feature of this phase of the project. The findings from this study will result in guiding principles for decision makers unpacking the key “to-dos” to get innovation goals and strategies right, and to make the best possible use of data for decision making and policy improvement.

• Build an innovation website. Innovation is for everyone, this online website will showcase aggregate findings on innovation capacity in cities according to goals and approaches, staffing and structure, data access, and outcomes cities reported in the survey. The website will also include individualised snapshots of participating cities’ innovation capacity, and will eventually reflect the findings and links between innovation capacity, data use and resident well-being indicators. It will serve as a repository of good practices and a tool for policy makers and decision makers in local governments around the world.

References

[2] European Commission (2012), Evaluation of Innovation Activities - Guidance on methods and practices, European Union, Brussels, http://ec.europa.eu/regional_policy/information/evaluations/guidance_en.cfm#1 (accessed on 15 April 2019).

[5] OECD (2017), Fostering Innovation in the Public Sector, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264270879-en.

[3] OECD (2017), Public Procurement for Innovation: Good Practices and Strategies, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264265820-en.

[4] OECD (2015), The Innovation Imperative: Contributing to Productivity, Growth and Well-Being, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264239814-en.

[1] OECD/Eurostat (2018), Oslo Manual 2018: Guidelines for Collecting, Reporting and Using Data on Innovation, 4th Edition, The Measurement of Scientific, Technological and Innovation Activities, OECD Publishing, Paris/Eurostat, Luxembourg, https://dx.doi.org/10.1787/9789264304604-en.

Notes

← 1. For further information see: www.oecd-opsi.org/innovations/air-louisville.

← 2. Answer from Los Angeles to Question 5.7 “Please tell us anything else about innovation outcomes in your city that you think could be relevant for us”.

3. Cities’ innovation capacity – the way forward