Chapter 2. Changes in skills and skill use in the OECD Survey of Adult Skills (PIAAC)

This chapter offers an overview of changes in worker skills and skill use over time, based on findings from the OECD’s Survey of Adult Skills (PIAAC). Since PIAAC has been carried out only once, its results are compared with those from an earlier study, the International Adult Literacy Survey (IALS). The comparison shows a decrease in the proportion of the workforce with high literacy proficiency, combined with a broad increase in the use of literacy skills at work. This picture of the change in skills over the past two decades differs from findings from economics that measure skills using wages, rather than direct assessments of skill levels.

The Survey of Adult Skills, derived from the Programme for the International Assessment of Adult Competencies (PIAAC), measures a set of general cognitive skills that are developed during education and widely used at work, as well as in personal life. The survey tests skills in literacy, numeracy and problem solving with computers.1 It also collects information about the ways that adults use skills, as well as various demographic characteristics. This chapter analyses the results of PIAAC to determine what it tells us about changes in skills and skill use over time.

The objective of the chapter is to provide a rough measure of changes in skill demand that can be compared to the economic research described in Chapter 1. In general, this existing research has analysed shifts in worker skills using data related to worker pay, worker education and occupational activities. By contrast, PIAAC makes it possible to study changes in skill demand with direct measures of worker skill. Although there are limitations to these data, they provide a different perspective on changes in skill demand that can be linked to computer capabilities to consider possible future changes.

After a brief overview of PIAAC, this chapter starts by considering two limitations that need to be taken into account to use the survey to measure change in skill demand over time. It then looks at basic results for skill proficiency and use over the past two decades, producing a picture of changes in skill demand that differs from the “polarisation” of skill demand set out in existing economic literature.

Overview of the Survey of Adult Skills (PIAAC)

The Survey of Adult Skills (PIAAC) is an international survey measuring the key cognitive and workplace skills needed for individuals to participate in society and for economies to prosper. It assesses the proficiency of adults aged 16-65 in literacy, numeracy and problem solving with computers. In addition to assessing these three competencies, PIAAC also collects information about each respondent’s background and context, including participation in activities that use the three competencies.

The survey is administered by trained interviewers, usually in the respondent’s home. It starts with a background questionnaire, typically taking 30-45 minutes to complete. Each respondent then takes the competency assessment in one or two of the three domains, usually taking about 50 minutes. For further information about the design of the assessment see OECD (2016c).

Two rounds of data collection have been completed so far. In the first round, data were collected in 2011-12 in 24 countries and economies. In 21 countries, the entire national population was covered. These countries included Australia, Austria, Canada, Cyprus,2 the Czech Republic, Denmark, Estonia, Finland, France, Germany, Ireland, Italy, Japan, Korea, the Netherlands, Norway, Poland, the Slovak Republic, Spain, Sweden and the United States. In three other countries only part of the population was covered: in Belgium, data were collected in Flanders; in the United Kingdom, data were collected in England and Northern Ireland; in the Russian Federation, data do not cover the Moscow municipal area. In the second round, data were collected in 2014-15 from an additional nine countries. In eight countries, the entire national population was covered; these countries included Chile, Greece, Israel, Lithuania, New Zealand, Singapore, Slovenia and Turkey. In Indonesia, data were collected in the Jakarta municipal area only. The total sample includes about 216 000 adults, with national samples ranging from about 4 000 to a maximum of nearly 27 300.

During the process of scoring the assessment, a difficulty score is assigned to each task, based on the proportion of respondents who complete it successfully. These scores are represented on a 500-point scale for each of the three domains. Respondents are placed on the same 500-point scale, using the information about the number and difficulty of the questions they answer correctly. At each point on the scale, an individual with a proficiency score of that particular value has a 67% chance of successfully completing test items located at that point. This individual will also be able to complete more difficult items with a lower probability of success and easier items with a greater chance of success. To help interpret the results, the reporting scales for each domain are divided into a small number of proficiency levels.

Analyses of the PIAAC scores reveal the close relationship between the three cognitive skills in question and labour force outcomes, including wages and employment (OECD, 2016b; Hanushek, Schwerdt, Wiederhold, and Woessmann, 2015). PIAAC scores have also been used to study mismatch between the skills required by a job and the proficiency of workers (e.g., OECD 2016b).

Using PIAAC to measure changes in skill demand

To provide a measure of changes in skill demand over time using PIAAC, there are two challenges that must be solved related to the limitations of the survey data. First, data on worker skill proficiency needs to be transformed into data on the skills that are actually used at work. Second, data from a one-time survey needs to be transformed into longitudinal data. The nature and structure of PIAAC provides ways to address both of these challenges.

Data about skill use

One problem with using data on worker proficiency is that the measures indicate skills that are potentially supplied to the workplace, but may not actually be used. Extensive literature on skill mismatch indicates that many workers either have skills that may not be used in their job, or lack skills that may be important in their job (Quintini, 2011). A direct measure of skill avoids the problem of assuming that workers in a particular job may have a skill that they do not actually possess. However, it still leaves the problem that the measured skills may not actually be used. PIAAC provides a solution to this challenge, since it offers information on skill use at work. This chapter combines the measures of skill proficiency and skill use to provide a rough measure of skills that are used.

Comparing PIAAC to previous studies on skills used at work

Although PIAAC is so far a one-off survey, it was designed to be comparable in some respects to two prior surveys of adult literacy: the International Adult Literacy Survey (IALS) carried out in 1994-1998 and the Adult Literacy and Life Skills Survey (ALL) carried out in 2003-2007 (OECD, 2016c). In order to address changes over time, the analysis in this chapter combines the results for PIAAC for countries surveyed during 2011-2012 and 2014-2015 with the results for IALS. There are 19 countries or economies that participated in PIAAC that also participated in IALS, with results 13-18 years apart, depending on the country.3

Because of changes between the different surveys, the ability to compare the results of IALS and PIAAC is limited. The literacy domain in PIAAC incorporates material that was assessed in two separate domains of prose and document literacy in IALS (OECD, 2016c). However, the scoring of the literacy data for PIAAC included a re-analysis of the data from IALS to create scores for a comparable joint literacy domain for the earlier survey (OECD, 2013). Over half of the literacy items used in PIAAC had also been used in IALS, and these linking items provided the basis for constructing comparable scales for the two surveys. It is not possible to compare the other two skill areas assessed by PIAAC. Because the numeracy domain is substantially different than the quantitative literacy domain included in IALS, it is not possible to construct a comparable scale for the earlier survey. For the third domain of problem solving with computers there was no analogous assessment in IALS.

PIAAC and IALS also both ask questions about the use of skills in their respective background questionnaires. There was substantial change in the specific questions and the structure of the possible responses between the two surveys. However, there is sufficient overlap in the design of the two questionnaires that it is possible to identify a small set of questions and response categories that can be compared.

Results for literacy proficiency in PIAAC and the International Adult Literacy Survey (IALS)

As noted above, the results for the adult skill surveys are reported on 500-point scales, which are used to describe both the difficulty of individual test questions and the proficiency of individual adults who took the survey. For ease in understanding, the continuous scales are often described using six proficiency levels, from Below Level 1 to Level 5. At Level 1 and below, the literacy questions use short texts of a few sentences with basic vocabulary and ask about information that can be clearly identified in the text from the words used in the question. At the higher levels, the texts are longer and the questions may require interpreting or synthesising, as well as avoiding misleading information that may superficially appear to provide the answer. Although the questions at the higher levels are more difficult, the topics are still limited to subjects that are familiar to most adults in developed countries, and the material is not technical. For more information on the construction of the literacy test and the content of the questions see OECD (2016c).

Figure 2.1 shows the literacy proficiency results by level, averaged across 19 OECD countries and economies included in both IALS and PIAAC. Because of relatively small numbers of adults at the top and bottom of the scale, those who are Level 1 and Below Level 1 are combined in a single category, as are those at Levels 4 and 5. For both surveys, over two-thirds of the adults are in Levels 2 and 3. In the 13-18 years between the two surveys, the primary change is an increased proportion of adults at Level 2 by 4 percentage points, and a decreased proportion in the bottom and top categories by 2 percentage points each.

Of course, many people in the full population are not in the labour force. For comparison, Figure 2.2 shows the literacy proficiency figures for the workforce only. Compared to the full population, the literacy of the workforce is shifted towards higher proficiency levels, with fewer people at Level 2 and below, and more people at Levels 3-5. However, the change between the two surveys is similar, showing the same increase in the proportion of adults at Level 2 and a slightly larger decrease in the proportion of adults at Levels 4 and 5.

The reasons behind the change in literacy proficiency shown between IALS and PIAAC results are not well understood. It would be reasonable to expect that the change would be related to shifts in the composition of the population over this period. Countries have generally increased their levels of education over past decades. This would be expected to lead to increasing levels of literacy proficiency over time (OECD, 2016b). At the same time, countries have also experienced increased ageing and immigration, both of which are generally associated with lower levels of skill. However, attempts to use these different trends to explain the change in literacy proficiency between IALS and PIAAC with shifts in the composition of the population have not been successful (Paccagnella, 2016). It is also not the case that the trend is specific to a few countries. The increase in the proportion of workers at Level 2 is broadly consistent across countries, with only Chile showing a statistically significant decrease instead of the increase shown on average across countries (Annex Table A2.2). The decrease in the proportion of workers at Levels 4 and 5 is somewhat less consistent, but still only three countries (Australia, Poland and Slovenia) show a statistically significant increase in contrast to the decrease shown on average across countries surveyed.

Basic results on skill use for PIAAC and IALS

Both PIAAC and IALS include a number of questions related to the use of skills at work. In each case, several of these questions concern the use of written material. The wording is quite similar between the two surveys for five questions related to the use of directions, letters, articles, manuals, and diagrams (OECD, 2013).4 The response categories for these questions are not exactly the same for the two surveys, but the responses in both cases can be aggregated to identify either daily or weekly skill use.5

This information on skill use frequency makes it possible to identify the proportion of workers who use their literacy skills as a regular part of their job. Of course, this frequency information alone does not indicate whether the amount of time using literacy is large or small, since even daily use could be for the entire day or only a few minutes. In addition, it does not indicate how important the literacy activity is to the job being performed, since it might be central to the task or involve only some secondary activity, such as time-keeping. Further, it does not indicate how difficult the literacy task is. However, the frequency information does provide an indicator of the proportion of workers whose jobs involve some regular use of literacy skills.

Figure 2.3 illustrates the portion of the workforce using each of the skills on a daily basis, averaged across 18 OECD countries and economies included in both IALS and PIAAC.6 The figure indicates that the daily use of both directions and letters increased substantially during the 13-18 years between the two surveys, while the other three types of materials show modest decreases. Figure 2.4 shows the change with respect to weekly use for the same set of skills, with a similar pattern except that the increase in the use of letters is smaller and the other changes are larger.

There is no information available to indicate how adults interpreted the descriptions of these different types of written material. Some of the changes in indicated use between the two surveys may be due to the small differences in wording. However, considering common sense meanings for the different types of materials indicated, the patterns in Figures 2.2 and 2.3 suggest some change towards shorter and less complex written materials between the two surveys: directions are often shorter than manuals, and diagrams are often somewhat complex.

For the purposes of knowing whether literacy skills are being used at all, and without having more detail about how adults understood the different descriptions of written material, it is useful to aggregate the results across the different types of written materials. Relatively comparable literacy proficiency could be used on each of the different types.

In order to aggregate the different literacy skill use questions, Figure 2.5 sets out the proportion of workers who use at least one of the five types of written materials at work on a daily or weekly basis. The figure illustrates a substantial increase of 13 percentage points in the proportion of workers using written materials on a daily basis, along with a more modest increase of four percentage points for use on a weekly basis. Although there are differences across countries, all countries show a statistically significant increase between the two surveys in the proportion of workers using these written materials on a daily basis. The increase ranges from four percentage points for Italy to 26 percentage points for Ireland (Annex Table A2.5). The changes in weekly use are more mixed across countries, with three countries (Denmark, Germany and Italy) showing decreases in weekly use, ranging from three to nine percentage points. All other countries show increases, ranging from one percentage point for Finland to 19 percentage points for Poland.7

Utilised literacy proficiency – combining results on literacy proficiency and use

Overall, workers with higher skill levels are more likely to use their literacy skills. In addition, these rates of use have increased between the two surveys for workers at all skill levels.

Figure 2.6 shows the proportion of workers at each literacy proficiency level who use their skills on a daily basis, with the increases ranging from 18 percentage points for workers with proficiency at Level 1 and below, to 11 percentage points for workers at Levels 4 and 5.

Figure 2.7 shows the same proportions for workers who use their skills on a weekly basis. Although all proficiency levels again show increased utilisation rates, those increases are small except for the lowest proficiency levels. For the workers at the higher proficiency levels, there is little room for further increases. This is because most of these workers already use their literacy skills on a regular basis.

There are relatively few significant differences in the changes in rates of use across countries (Annex Tables A2.6 and A2.7). Germany and Italy both show decreases in the rates of weekly use for workers at all skill levels, with statistically significant differences from the country averages. Chile, Denmark and the United States show decreases in weekly use at one or two proficiency levels, most of which are significantly different from the average increase across countries. In addition, Ireland and Norway both show significantly faster increases in the rates of use for several proficiency levels for both daily and weekly use.

By combining the separate results on literacy proficiency and literacy skill use, it is possible to see the change in the distribution of literacy proficiency in the workforce between IALS and PIAAC for workers who regularly use their skills.

Figure 2.8 shows the distribution of workers with respect to literacy proficiency and daily skill use, averaged across 18 OECD countries and economies included in both surveys. This figure shows that the largest increase in workers using literacy on a daily basis between the two surveys was for workers at Level 2 proficiency, along with smaller increases for those at Level 1 and below, and at Level 3. There is a small decrease in the proportion of workers who both use literacy daily and are at Level 4-5 proficiency.

Figure 2.9 shows the distribution of workers with respect to literacy proficiency and weekly rather than daily use. The largest increase in workers using literacy on a weekly basis is again for workers at Level 2 proficiency. The figure also shows a modest decrease in the proportion of workers who both use literacy weekly and are at Level 4-5 proficiency.

The increase in the proportion of workers using literacy skills who are ranked at Level 2 results from two factors: an increase in the proportion of workers at that level of literacy proficiency, and the increase in the overall use of literacy skill. By contrast, the increased proportion of workers using literacy who are at Level 1 and below and at Level 3 reflects only increased rates of use, since the proportion of workers at these skill levels was relatively stable between the two surveys. The decline in the proportion of workers using literacy who are at Level 4-5 results from the decline in the proportion of workers at those levels. This decline is only partly counterbalanced by increased rates of use.

The increase in the proportion of workers using literacy who are at Level 2 is generally consistent geographically, with all countries showing an increase with respect to daily use (Annex Table A2.8). Only three countries (Germany, Italy, and Slovenia) demonstrate a decrease with respect to weekly use, none of which is statistically significant (Annex Table A2.9).

However, the modest average decrease in the proportion of workers using literacy who are at Level 4-5 reflects clear differences in the patterns across countries. Contrasting with the average decrease across countries, Australia, Finland, the Netherlands, Poland and Slovenia show increases that are statistically significant for either daily or weekly use. In these countries, the increase in the proportion of workers using literacy who are at Level 4-5 averages five percentage points for daily use and four percentage points for weekly use. In comparison, the overall average decreases are one and two percentage points respectively for all included OECD countries and economies.

Understanding the different approaches to measuring skill use

The analysis of changes in skill and skill use presented in this chapter is rather different from the usual description of these changes in the economic research. As noted in Chapter 1, the pattern often found for changes in workforce skill during the 1990s and 2000s is one of polarisation, with increasing employment for workers with higher and lower skills and decreasing employment for workers with mid-level skills (e.g., Autor, 2015; Goos, Manning, and Salomons, 2014).

By contrast, this new analysis reveals the distribution of literacy proficiency in the workforce has increased in the middle of the skill distribution and decreased at the upper level. Although workers of all skill levels are more likely to use their literacy skills regularly at work, in most cases this involves an increase in regular users who have low to middle levels of literacy skill. In tandem, there is a decrease both for workers who do not use literacy regularly, and for those who use literacy regularly and have high levels of proficiency.

It is possible to reconcile these seemingly contradictory findings by understanding that they reflect two very different kinds of measures. Economic research has used differences in wages across occupations as its primary measure of skill differences, along with analyses of education or task content by occupation. These three types of data provide information about work skills that is both broad and indirect: it potentially reflects a wide range of skills important in the workplace, but also other labour market factors related to wages, education and tasks that may not be directly related to skill, such as policies and norms related to wage-setting and educational requirements.

By contrast, PIAAC and IALS provide a direct but narrow measure of the change in skill proficiency and use. The surveys provide direct measures related to literacy proficiency and the use of written materials, but they do not provide measures of many other important skills. Nor do they include questions that measure the amount of time, importance or complexity of certain tasks. It is quite possible that skills other than general literacy or other aspects of skills use are driving the polarisation findings in the economic research.

In all cases, it is important to understand the measures that are being used in any particular study, and to be careful in using them to draw conclusions that go beyond what they describe. Historically, economic research has had access to very little data directly related to worker skills. As a result, conclusions related to “skill” that are actually based on measures of educational attainment, wages or occupational clusters, may not reflect understandings that are meaningful to the education community. With respect to skill itself, it is important to consider the very different picture offered by PIAAC of the changes that have occurred over the past two decades – a picture that suggests an increase in the prevalence and use of mid-level skills in the workforce.

References

Autor, D.H. (2015), “Why Are There Still So Many Jobs? The History and Future of Workplace Automation”, Journal of Economic Perspectives, Vol. 29/3, American Economic Association, pp. 3-30.

Goos, M., A. Manning, and A. Salomons (2014), “Explaining Job Polarization: Routine-Biased Technological Change and Offshoring”, American Economic Review, Vol. 104/8, American Economic Association, pp. 2 509 – 2 526.

Hanushek, E.A. et al. (2015), “Returns to Skills Around the World: Evidence from PIAAC”, European Economic Review, Vol. 73, American Economic Association, pp.103-130.

OECD (2016a), “Skills use at work: Why does it matter and what influences it?”, in OECD Employment Outlook 2016, OECD Publishing, Paris. https://doi.org/10.1787/empl_outlook-2016-6-en.

OECD (2016b), Skills Matter: Further Results from the Survey of Adult Skills, OECD Skill Studies, OECD Publishing, Paris, https://doi.org/10.1787/9789264258051-en.

OECD (2016c), The Survey of Adult Skills: Reader’s Companion, Second Edition, OECD Skills Studies, OECD Publishing, Paris, https://doi.org/10.1787/9789264258075-en.

OECD (2016d), Survey of Adult Skills (PIAAC) (Database 2012, 2015), www.oecd.org/site/piaac/publicdataandanalysis.htm.

OECD (2013), Technical Report of the Survey of Adult Skills (PIAAC), OECD, http://www.oecd.org/skills/piaac/_Technical%20Report_17OCT13.pdf, accessed 19 September 2017.

Paccagnella, M. (2016), “Literacy and Numeracy Proficiency in IALS, ALL and PIAAC”, OECD Education Working Papers, No. 142, OECD Publishing, Paris, https://doi.org/10.1787/5jlpq7qglx5g-en.

Quintini, G. (2011), “Over-qualified or under-skilled: A review of existing literature”, OECD Social, Employment and Migration Working Papers, No. 121, OECD Publishing, Paris, https://doi.org/10.1787/5kg58j9d7b6d-en.

Statistics Canada, International Adult Literacy Survey (1994-1998), www5.statcan.gc.ca/olc-cel/olc.action?ObjId=89M0014X&ObjType=2&lang=en&limit=0.

Notes

← 1. The formal name used for the problem solving skill area in PIAAC is “problem solving in technology-rich environments.”

← 2. Note regarding Cyprus: Note by Turkey: The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”.

Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus.

← 3. The countries or economies participating in both surveys include Australia, Canada, Chile, the Czech Republic, Denmark, England (UK), Finland, Flanders (Belgium), Germany, Ireland, Italy, the Netherlands, New Zealand, Northern Ireland (UK), Norway, Poland, Slovenia, Sweden and the United States. ALL is not used because only 7 of the 19 countries participated in ALL and the shorter time interval provides less opportunity to observe change.

← 4. For directions, IALS asks about “directions or instructions for medicines, recipes, or other products,” whereas PIAAC asks about “directions or instructions.” For letters, IALS asks about “letters or memos,” whereas PIAAC asks about “letters, memos or e-mails.” For articles, IALS asks about “reports, articles, magazines or journals,” whereas PIAAC asks about “articles in newspapers, magazines or newsletters.” For manuals, IALS asks about “manuals or reference books, including catalogues,” whereas PIAAC asks about “manuals or reference materials.” For diagrams, IALS asks about “diagrams or schematics,” whereas PIAAC asks about “diagrams, maps or schematics.”

← 5. For IALS, the response categories are “every day, a few times a week, once a week, less than once a week, rarely or never” (OECD, 2013). For PIAAC, the response categories are “never, less than once a month, less than once a week but at least once a month, at least once a week but not every day, every day.”

← 6. IALS data on daily skill use are not available for Australia.

← 7. OECD (2016a) conducts a similar analysis with respect to weekly skill use with the IALS and PIAAC data. The report finds similar results as Figure 2.4 when use is considered separately by type of material. To aggregate across types of material, the report averages the frequency ratings across the different measures. This produces a finding of no change over time, in contrast to the increase found here by using the maximum frequency across the different measures. The report does not analyse daily skill use.