Annex B. Methodology note on the Survey of Career Guidance for Adults
This report uses data collected in the OECD 2020/2021 Survey of Career Guidance for Adults (SCGA). The SCGA was conducted to better understand adults’ experience with career guidance services and to improve international data on its use, coverage and inclusiveness. For Canada, the data collection and methodology differed from that of other countries that participated in the survey in certain respects.
Data collection for the SCGA was carried out in two initial phases. The first phase of data collection took place between mid-June and early-July 2020 in six countries: Chile, France, Germany, Italy, New Zealand and the United States. The second phase of data collection took place in November 2020 in Argentina, Brazil and Mexico. Fieldwork was conducted by Cint1 and the sample was restricted to adults aged 25-64, to target those who had left initial education.
The survey was prepared in five languages (English, French, German, Italian and Spanish) and disseminated via an online survey to a panel of registered individuals. A stratified sample methodology imposed quotas to have a representative sample of each country’s population in terms of age, gender and region. The age and gender quotas were based on UN World Population Prospects statistics,2 while the regional quotas were based on Cint’s data. Education quotas were added in the second phase of data collection (for more details, see OECD (2021[1])).
After data collection, two quality checks were applied. First, if a respondent completed the survey in two minutes or less, the respondent was excluded. This is based on the assumption that the survey takes more than two minutes to complete with appropriate consideration. Second, if a respondent did not answer the final question of the survey, they were also excluded. This was to ensure that only respondents who completed the full survey were captured in the final dataset.
The data collection and methodology for Canada differed from those of the other countries in several respects.
First, for the Canadian survey, some questions and answer options were phrased differently, and additional questions and answer options were added. This was done at the request of the Canadian partners in order to tailor the survey better to the Canadian context. For instance, the Canadian survey asked adults, “In the past 5 years, have you used a career service?”3 In the other countries in the survey, adults were asked instead, “In the past 5 years, have you spoken to a career guidance advisor?” These differences in question wording likely had an impact on the cross-country comparability of the data on the use of career guidance.
Also, the Canadian survey was carried out by a different survey provider, who applied a slightly different methodology than the one used by Cint. Data sampling was carried out by Ipsos Limited Partnership and the data collection process was overseen by the Canadian Labour Market Information Council (LMIC). Data were collected online between 24 February 2021 and 25 March 2021 through both English and French surveys. As with the other countries, the sample was stratified by provinces4, age, gender and education using the latest Census data (2016). The sample was restricted to adults aged 25-64.
Ipsos used a two-step implementation method to collect data for the Canadian survey. First, the initial survey design targeted a sample of career service users (2 000 respondents) separately from a sample of non-users (1 000 respondents). Then, to be able to compare the two groups, weights were introduced in the sample and were estimated from a representative online omnibus survey designed to measure the incidence of career guidance use among the general population of adult Canadians (18+). Out of 2 001 total observations in the omnibus survey, 405 individuals reported having used a career service, for an estimated incidence rate of 20%.
The above-mentioned quality checks were not carried out for the Canadian survey.
To improve cross-country comparison, this report features weighted data. The weighting served to mitigate differences related to the fact that education quotas were not included in the first phase of data collection but were included for both the second phase and for Canada. For those countries from the first phase of data collection, for which education quotas were not imposed, education weights were constructed using OECD data (2020[2]).
Figure A B.1 compares the composition of the country-level samples with the composition of the actual population in each country in terms of gender, age and education. Thanks to quotas, the sample shares are very close to those of the actual population on gender in all countries. The sample is younger than the actual population in all countries, owing to the fact that older adults are less likely to complete online surveys than younger adults. The sample is over-educated relative to the actual population in some countries (Argentina, Brazil, Chile, and Mexico) and under-educated relative to the actual population in other countries (Canada, France, Germany, Italy, New Zealand and the United States).
Note: The denominator is adults aged 25 to 64.
Source: The sample composition by age, gender and education is drawn from the OECD 2020/2021 Survey of Career Guidance for Adults (SCGA). The population distribution of age and gender were extracted from the UN 2019 Revision of World Population Prospects (https://population.un.org/wpp/). For population estimates by education group, OECD (2020[2]), OECD Education at a Glance 2020: OECD Indicators, https://dx.doi.org/10.1787/69096873-en, was used.
Another factor that affects cross-country comparability of the data is timing. In particular, the online survey was conducted at different stages of the COVID-19 crisis over 2020 and 2021. This might have implications on the response rates and composition of respondents, given that the time a person would have to complete an online survey may have changed in the context of confinement, teleworking or layoffs. The use of quotas mitigates this effect. The differences in the timing of the survey may also have implications on the reported use of career services due to measures adopted to tackle the employment effects of the pandemic. Table A B.1 shows the final sample sizes for each country in the SCGA.
References
[1] OECD (2021), “Methodology note on the Survey of Career Guidance for Adults”, in Career Guidance for Adults in a Changing World of Work, OECD Publishing, Paris, https://dx.doi.org/10.1787/6b6c6c86-en.
[2] OECD (2020), Education at a Glance 2020: OECD Indicators, OECD Publishing, Paris, https://dx.doi.org/10.1787/69096873-en.
Notes
← 1. Cint is a digital insights gathering platform (www.cint.com). The Cint platform and products comply with standards and certifications set out by various market research associations including ESOMAR, MRS, ARF, MRIA, AMA, AMSRO and Insights Association and ISO 20 252 quality standards.
← 2. https://population.un.org/wpp/.
← 3. The survey defined career services as follows: “Career services are offered across Canada through public employment services, community based agencies, educational institutions, workplaces and private providers. Career services offer a range of supports, including:
Helping you to access information to help you make your career-related decisions (e.g., wage information, job prospects, cost of living, etc.)
Helping you to understand your career options and choose your career direction
Helping you to get the skills/credentials/training you need to pursue your career goals
Helping you to keep your job or grow within your position/organisation
Addressing personal/life issues that may be preventing you from moving forward with your goals”
← 4. Alberta, British Columbia, Manitoba, New Brunswick, Newfoundland and Labrador, Nova Scotia, Ontario, Prince Edward Island, Quebec, and Saskatchewan. The territories were not part of the stratification because of extremely low sample sizes.