Chapter 5. Implications of computer capabilities for policy and research1

This chapter offers an analysis of computer capabilities in the three cognitive skill areas addressed by the Survey for Adult Skills (PIAAC), and the resulting implications for education and labour policy. Drawing upon the analysis of changes in skills and skill use over time outlined in Chapter 2, the chapter assesses the potential for computers to further change the use of those skills at work in the future. The assessment is based upon the judgments of the group of computer scientists set out in Chapters 3 and 4 about the level of current computer capabilities. The Chapter concludes with suggestions for how this project could be used for future research.

The preceding chapters of this report have presented two substantially different analyses. First, a discussion of past changes in literacy skills and skill use is provided in Chapter 2. Second, a discussion of current computer capabilities in literacy and other general cognitive skills is discussed in Chapters 3 and 4. This chapter brings the two analyses together, to consider how computer capabilities in general cognitive skill areas are likely to change the use of those skills in the workplace in the future. This consideration has implications for the development of general cognitive skills. It also raises questions about the ways that skills are assessed for shaping education and labour policy.

Linking current computer capabilities to workforce skill trends

The exploratory assessment of computer capabilities described in Chapters 3 and 4 results in several different aggregate ratings for each of the three skill areas included in the Survey of Adult Skills (PIAAC): literacy, numeracy and problem solving with computers.2 The discussion set out in Chapter 4 compares the different ratings for computers to adults at varying proficiency levels. This involves analysing the difference between expected computer performance and actual adult performance for questions at varying levels of difficulty. In general, human performance decreases more steeply than computer performance as the questions become more difficult. Therefore some approximation is required to choose a human proficiency level that roughly corresponds to expected computer performance.

Table 5.1 summarises the proficiency levels identified in Chapter 4 that correspond to the three different skill areas and three of the aggregate computer ratings. These ratings should be treated as preliminary, resulting from an exploratory process. There was insufficient time to try several proposed ways of resolving the disagreements between the experts about their judgments. However, it is worth taking this set of aggregate ratings at face value and considering their implications for workplace skills.

Table 5.1. Approximate proficiency level of computer capabilities in the Survey of Adult Skills (PIAAC)
Computer rating	Literacy	Numeracy	Problem solving with computers
Current capabilities, average with Maybe as 50%	Level 2	Level 2	Level 2
Current capabilities, 3-expert minimum	Level 3	Level 4	Level 2
Capabilities in 2026	Level 3	Level 3	Level 3
Source: See Annex A Tables A4.5, A4.6 A4.8, A4.13, A4.14, A4.15, A4.17, A4.18, and Annex B Table B4.8.

The broadest aggregate rating in Table 5.1 is the simple average that counts Maybe as 50%. This reflects the judgments of the full set of computer scientists for literacy and numeracy. This group included experts with a number of different specialties and a range of overall optimism about computer capabilities. When this rating has a high value, it means that most of the group was able to suggest current approaches that they believe would allow computers to answer a particular question.

The other two aggregate ratings in the table reflect the judgments of a smaller group of experts. The 3-expert minimum rating generally reflects more “optimistic” experts, since it requires only three of the experts to indicate that a question could be answered by computers. The 2026 rating was made by only three members of the group. These members turned out to be somewhat more “pessimistic” than the group as a whole.

In short, the first aggregate rating provides a relatively conservative judgment, requiring agreement from a broad set of experts that a question could be answered by current computer capabilities. By contrast, the second and third ratings provide two different ways of thinking about the boundaries of what may be possible in the near future. One of these predictions is offered by the more “optimistic” experts, who say that such a level of performance is possible today. The other is offered by the more “pessimistic” experts, who say that such a level of performance is possible ten years from now.

Proficiency levels of computers for literacy

For literacy, the lowest computer rating is at Level 2 and the second and third are both for Level 3, as set out in Table 5.1. For comparison, Figure 5.1 shows the proportion of workers at different levels of literacy proficiency who use literacy on a daily basis, averaged over all OECD countries and economies included in the Survey of Adult Skills. This figure is similar to the PIAAC results in Figure 2.8 from Chapter 2, although the average includes data for ten countries that are not included in the earlier figure because they did not participate in the International Adult Literacy Survey (IALS).3

The fact that the lowest computer score for literacy is at Level 2 suggests that the literacy-related tasks of 29% of the workforce could be affected by current computer capabilities. By contrast, the second and third ratings, which place computers at Level 3, suggest that the literacy-related tasks of 57% of the workforce could be affected. On the other hand, this leads to the conclusion that 43% of the workforce would not be strongly affected by these computer capabilities. This is either because literacy-related materials are not a daily part of their work or because their literacy proficiency is above the level that computers will be able to provide in the near future.

Proficiency levels of computers for numeracy

For numeracy, the first computer rating is for Level 2 and the second and third for Levels 4 and 3 respectively, as set out in Table 5.1. This indicates that a wider range of worker proficiency levels could potentially be affected by computer capabilities. Figure 5.2 shows the proportion of workers at different levels of numeracy proficiency who use numeracy on a daily basis.4 Not surprisingly, there are fewer workers who use numeracy than those who use literacy on a daily basis. Based upon calculations in this exploratory project, numeracy-related tasks of 20% of the workforce could be affected by current computer capabilities of Level 2. This increases to 37% for computer capabilities at Level 3 and to 44% at Level 4, effectively the entire workforce that uses numeracy on a daily basis at work.5

Proficiency levels of computers for problem solving using computers

For problem solving with computers, the first and second computer rating is for Level 2 and the third is for Level 3, as set out in Table 5.1. Figure 5.3 shows the proportion of workers at different levels of proficiency in problem solving with computers who use computers on a daily basis.6 More than three-quarters of workers use computers on a daily basis at work. Therefore, according to data in this project, the computer-related tasks of 69% of the workforce could be affected by current computer capabilities of Level 2.7 This increases to all workers using computers on a daily basis (76%) with computer capabilities of Level 3.

Bringing together the three sets of results on computer performance

Figure 5.4 combines the analyses for the three general cognitive skills to identify the portion of the workforce that will potentially be affected by computer capabilities according to the first and third computer ratings set out in Table 5.1. That is: current capabilities of computers using an average with Maybe as 50%, and capabilities in 2026.

At the lower end of skill use, 25% of the workforce does not use any of the three general cognitive skills on a daily basis at work. Therefore, their regular work tasks will not be substantially affected by the computer capabilities examined in this study.

At the upper end of skill proficiency, there are workers who use one or more of these skills on a daily basis and have proficiency above the projected level of computer capabilities. Because these workers have proficiency in the three skill areas that is above projected computer capabilities for the near future, it is reasonable to expect they will continue to have regular work tasks using these skills that are not substantially affected by computer capabilities in these areas. This proportion is 44% for the projected level of computer capabilities in 2016 using the first rating, and 13% for the projected level of computer capabilities in 2026 using the third rating.

In the middle, there is a large proportion of workers who use one or more of these three cognitive skills on a daily basis, but have proficiencies only at the level of projected computer capabilities. This proportion is 31% for the projected level of computer capabilities in 2016, and 62% for the projected level of computer capabilities in 2026.

Identifying the workers who will be the most affected by computer capabilities related to PIAAC

The workers in the middle of the spectrum are the ones whose work tasks seem most likely to be substantially affected by the projected computer capabilities in these three areas of general cognitive skill. Figure 5.4 suggests that the next two decades are likely to see a reversal in the pattern of skill use change that Chapter 2 describes for the last two decades, at least with respect to the three general cognitive skills measured by PIAAC.

Between the times when IALS and PIAAC were conducted (that is, from the 1990s to the 2010s), there was an increase in the proportion of the workforce using written materials on a daily basis with a low to medium level of proficiency. However, over the next two decades, that increase is likely to reverse, since computers will increasingly be able to substitute for workers in carrying out tasks requiring the three cognitive skills measured by PIAAC at a low to medium level of proficiency. The assessment of computer capabilities in the skills measured by PIAAC suggests that workers with only low to medium proficiency may be less likely to use their skills regularly at work in the coming decades.

There are large differences across countries in the proportion of the workforce that regularly uses the three cognitive skills measured by PIAAC with proficiency levels at or below the capabilities projected for computers. Figure 5.5 shows that the potentially affected workforce ranges from 17% for Japan to 56% for Chile, using the rating for 2016. With the rating for 2026, the potentially affected workforce ranges from 48% for Turkey to 70% for the United States. For some countries, such as Chile, the figure shows that many workers may be affected by computers because relatively few workers have proficiency above the projected level of computer capabilities. For other countries, such as the United States, a high proportion may be affected because more workers regularly use these skills at work.

Implications of computer capabilities for employment, based upon this study

The preceding comparison between computer capability projections and the proficiency and skill use of the workforce raises questions about how such computer capabilities will affect employment. Although there is insufficient information for a full answer, several points can be made about the results.

First, the analysis is only preliminary. Chapters 3 and 4 identify a number of limits that affected the exploratory judgments of computer capabilities that form the basis for the projection of affected workers.

Second, the analysis focuses on technical capability rather than economic application. It is well established that the application of new technologies often takes a decade or more when it occurs, and sometimes it never occurs (Comin and Hobijn, 2010; Griliches, 1957; Mansfield, 1961). To understand which applications are likely to take place and which are not, further analysis would need to be carried out. Such an analysis would need to take into consideration the economic and organisational factors that will affect the application of the projected computer capabilities. Existing research also suggests that the speed of diffusion is likely to vary substantially by country (Comin and Hobijn, 2010) and by firm (OECD, 2015a).

Third, most jobs involve a mix of different types of skill. Tasks can vary considerably in the relative importance of skills required, and how closely they are linked together. This has implications for the technical scope for using computers that have capabilities in some skill areas but not others to automate job tasks. For example, most receptionists, nurses and housekeepers regularly use both language and physical skills, but the role of those skills is different in each job: many receptionist tasks could be automated with language skills alone, whereas many housekeeping tasks could be automated with physical skills alone; many nursing tasks require both language and physical skills. Without knowing computer capabilities in other skill areas and the skill mix required in different jobs, it is hard to know how computer capabilities in the three general cognitive skills alone would affect employment.

Despite difficulties in drawing clear conclusions about employment effects, the level of current computer capabilities laid out here suggests that it is unlikely that demand for workers with low and medium general cognitive skills will increase in the next several decades. Even without significant decreases in the employment of such workers, it would be prudent to expect that the demand for workers with low to medium cognitive skill levels will weaken. It is likely that many workers with general cognitive skills at such levels will still be employed. However, they may be employed primarily because of other skills they have – for example, physical or social skills, or special expertise in some particular content area. This shift has implications for how researchers and policymakers should analyse and understand skill development.

Realistic aspirations for general cognitive skill development in the general population

PIAAC assesses a set of general cognitive skills that are an important focus of development during education and widely used at work. The findings on skill use demonstrate that large proportions of the workforce use these skills every day at work, even many workers with modest levels of proficiency. However, these positive findings about skill use apply to today’s economy at a time when many existing computer capabilities have not yet been broadly applied. The same conclusions will not necessarily hold as these capabilities are diffused on a wider scale in the workplace. At that point, what levels of education and skill should we expect in the general population?

With respect to the skills included in PIAAC, it is unlikely there will be strong demand for human workers except for those who have relatively high proficiency levels. The projections developed in this report suggest that in one or two decades’ time, workers will need to be proficient in literacy and numeracy at Level 4 or 5 to clearly outperform computers in these areas.8 However, on average, only 11% of working age adults in OECD countries has proficiency in literacy and numeracy at these levels (Annex Table A5.6). As a result, most of the workforce may not be able to compete with computers in these skill areas.

One likely response to increasing computer capabilities would be to attempt to increase the level of skills in the workforce so that more people have skills that are greater than computer capabilities. Most countries around the world have worked to increase the education and skills of their populations and this strategy could have a number of beneficial effects. However, the available data on adult skills in OECD countries over the past two decades does not show a general increase in the proportion of workers at higher proficiency levels as a result of past education improvements. Indeed, the analysis in Chapter 2 suggests instead that there has been a modest decrease.

Of course, future efforts to improve adult education and skills could be more successful. By looking across countries, it is possible to identify those that are more successful in achieving high proportions of adults with proficiencies in literacy and numeracy at Levels 4 and 5. These examples indicate what improvements may be possible in other countries. Figure 5.6 shows the proportion of adults at the higher proficiency levels for all 34 countries and economies that have participated in PIAAC, including both OECD and non-OECD countries. The figure shows a wide range of results across the countries and suggests that many countries could substantially improve. However, the maximum – 23% for literacy and 19% for numeracy, both for Japan – is distinctly limited. The average performance of the best country suggests that only a quarter of the population could be better than projected computer capabilities in literacy and numeracy.

The proficiency of the full population presents a more pessimistic picture of full skill potential, since many older people received less, and less effective, education than people who are educated today. In addition, the skills of older people may have weakened over time if they have not been used regularly. In general, PIAAC finds that skill levels are highest for the cohort of adults that has most recently fully completed formal education and declines for older cohorts. Figure 5.7 shows the results for the highest-performing cohort of adults, those aged 25-34. The OECD average is significantly higher for this group than for the full population, by 5 percentage points for literacy and 4 percentage points for numeracy (Annex Tables A5.6 and A5.7). However, in the highest-achieving country still only about a third of these younger adults reach the higher proficiency levels in literacy and numeracy – 37% for literacy and 32% for numeracy, both for Finland.

Variation across countries and across age cohorts suggests that many more workers could achieve proficiencies in literacy and numeracy at Levels 4 and 5. However, there is no indication in the performance of the highest-performing cohort in the highest-performing countries that a majority of the population could reach the higher levels of proficiency. Furthermore, even if increasing average proficiencies to the levels of the highest-performing cohorts and countries is possible, it would certainly take decades for other countries to achieve these results. During that time, computer capabilities in these skill areas will continue to improve.

With respect to general cognitive skills, higher levels of proficiency in literacy and numeracy are likely to be important for some part of the workforce over the next several decades as computer capabilities for the lower levels of literacy and numeracy are applied. However, it does not appear that these skills can be the key to employability for the majority of the workforce over this period. Given the levels of proficiency demonstrated in the past, it is simply not plausible that most workers over the next couple decades will be able to achieve higher levels of literacy and numeracy than available computer capabilities.

Going beyond the existing understanding of adult and computer skills

Ultimately, if we want to understand what skills workers are likely to need over the next several decades, we need to know much more about the other kinds of skills that workers use beyond the general cognitive skills assessed by PIAAC. We also need to understand the levels of proficiency that computers are developing with respect to these skills.

Research on job analysis in industrial and organisational psychology has resulted in several different approaches for understanding and categorising work-related skills and tasks (Fleishman, Quaintance and Broedling, 1984; National Research Council, 2010). These taxonomies provide a way of systematically considering the range of skills used at work and the way these different skills are brought together in different kinds of tasks. Some of these skills, like literacy and numeracy, are developed during formal education, whereas others, like physical dexterity or social perception, are primarily developed outside of formal education. It is necessary to understand how all these skills come together at work to be able to understand how worker activities will change as new computer capabilities develop, and how the education system should evolve in response.

Existing education assessments understandably focus on the skills that are developed during formal education. The OECD’s Programme for International Student Assessment (PISA) assessment of 15-year olds is an example of this type of test. Although PIAAC provides information about adults rather than students, it still focuses on skills primarily developed during education. Recent efforts have been made to understand the importance of social and emotional skills, making the case that they are affected by education. It has been argued that such skills should therefore be included in the set of education outcomes that are assessed as a part of education research and policy (OECD, 2015b). In addition, PISA continually explores relevant new content domains, such as problem solving and financial literacy. However, there are many key work skills that are not considered in these testing programmes because they are not developed primarily during formal education.

Outside of formal education, there is a rich tradition of assessment of work-related skills used for occupational licensing and worker selection and training (e.g., Fleishman and Reilly, 1995; National Research Council, 1991, 2001, 2015). This work provides a set of tools that could be used to describe more precisely what skills workers need in different situations and how they relate to computer capabilities. The approach taken in this exploratory project provides a way to use such assessments to connect information about the skill proficiency of workers to the judgments of computer scientists about the growing capabilities of computers. To understand how computers will likely change the full range of skills used in the economy, this work should be extended across the full range of work skills.

At a time when computers are developing capabilities across a wide range of skill areas, policymakers need to have a much more systematic picture of work skills than is provided by tests of education-related skills alone. Because different skills are used together to perform work tasks, information about education-related skills alone cannot provide information even about those education-related skills themselves. The interdependence between different skills is clearly demonstrated in the computer scientist review of the PIAAC literacy and numeracy questions. One of the greatest challenges when comparing human and computer capabilities in these two skill areas came from the need for skills related to vision or common sense to answer many of the questions.

This skill interdependence in the context of the PIAAC test questions is merely an example of the interdependence that occurs throughout the workplace. Ultimately, we need information about the full set of skills to understand which skills workers will need in the future and how they are likely to interact with the capabilities that computers will increasingly be able to provide.

References

Comin, D., and B. Hobijn (2010), “An Exploration of Technology Diffusion”, American Economic Review, Vol. 100/5, American Economic Association, pp. 2 031-59.

Fleishman, E.A., M.K. Quaintance, and L.A. Broedling (1984), Taxonomies of Human Performance: The Description of Human Tasks, Academic Press, Orlando, Florida.

Fleishman, E.A., and M.E. Reilly (1995), Handbook of Human Abilities: Definitions, Measurements, and Job Task Requirements, Manpower Research Institute.

Griliches, Z., (1957), “Hybrid Corn: An Exploration in the Economics of Technological Change”, Econometrica, Vol. 25/4, the Econometric Society, New York, pp. 501-522.

Mansfield, E. (1961), “Technical change and the rate of imitation”, Econometrica, Vol. 29/4, the Econometric Society, New York, pp. 741-66.

National Research Council (2015), Measuring Human Capabilities: An Agenda for Basic Research on the Assessment of Individual and Group Performance Potential for Military Accession. Committee on Measuring Human Capabilities: Performance Potential of Individuals and Collectives, The National Academies Press, Washington, DC.

National Research Council (2010), A Database for a Changing Economy: Review of the Occupational Information Network (O*NET). Panel to Review the Occupational Information Network (O*NET), N.T. Tippins and M.L. Hilton, eds. The National Academies Press, Washington, DC.

National Research Council (2001), Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality. Committee on Assessment and Teacher Quality, K.J. Mitchell, D.Z. Robinson, B.S. Plake, and K.T. Knowles, eds. The National Academies Press, Washington, DC.

National Research Council (1991), Performance Assessment for the Workplace: Volume 1. Committee on the Performance of Military Personnel, A.K. Wigdor and B.F. Green, Jr., eds. The National Academies Press, Washington, DC.

OECD (2016a), Skills Matter: Further Results from the Survey of Adult Skills, OECD Skill Studies, OECD Publishing, Paris, https://doi.org/10.1787/9789264258051-en

OECD (2016b), Survey of Adult Skills (PIAAC) (Database 2012, 2015), www.oecd.org/site/piaac/publicdataandanalysis.htm.

OECD (2015a), The Future of Productivity, OECD Publishing, Paris, https://doi.org/10.1787/9789264248533-en.

OECD (2015b) Skills for Social Progress: The Power of Social and Emotional Skills, OECD Skills Studies, OECD Publishing. Paris. https://doi.org/10.1787/9789264226159-en.

Notes

← 1. The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law.

← 2. The formal name used for the problem solving skill area in PIAAC is “problem solving in technology-rich environments.”

← 3. The additional countries are Austria, Estonia, France, Greece, Israel, Japan, Korea, the Slovak Republic, Spain, and Turkey.

← 4. To identify daily numeracy use, the analysis aggregates skill use questions related to reading “bills, invoices, bank statements or other financial statements,” reading “diagrams, maps or schematics”, calculating “prices, costs or budgets” and using or calculating “fractions, decimals or percentages”.

← 5. Level 5 represents only 1% of the population in the OECD average (OECD, 2016a), so the question about whether or not computer capabilities would be able to reach performance in numeracy comparable to Level 5 would not have a substantial effect on the portion of the workforce affected.

← 6. To identify daily computer use, the analysis aggregates skill use questions related to using “email,” using “the internet in order to better understands issues related to your work,” conducting “transactions on the internet, for example buying or selling products or services, or banking,” using “spreadsheet software” and using “a word processor.”

← 7. The category of workers who use computers on a daily basis but have no proficiency data includes those who failed the initial screening test related to basic computer operation or who opted out of the computer test. The calculation assumes that essentially none of these workers would be at the highest level of proficiency on the assessment of problem solving with computers if they had attempted it.

← 8. For the third skill area of problem solving with computers, the projected capabilities of computers are already close to the top of the scale.