5. Understanding and testing social-emotional skills
This chapter reviews the background and conceptualisation of social-emotional skills and how these are related to consequential outcomes in education and the labour market. It discusses taxonomies on how to structure and operationalise these skills, converging on an overarching model representing five broad groups. In addition, it explores how these skills are connected to various types of work performance. Multiple common methods to assess social-emotional skills are reviewed. These comprise Likert-based and ipsative self- and observer reports, more objective tests and situational judgement tests, as well as behavioural residue indicators.
In recent years, the assessment and learning of social-emotional skills in education and policy making have garnered more attention (Kankaraš, 2017[1]; Kankaraš and Suarez-Alvarez, 2019[2]). Language or math achievement have long been traditional indicators of scholastic performance. Today, in addition to those skills, social-emotional skills are considered as both means and end products of education processes.
Learning and training social-emotional skills became an explicit part of educational curricula for several reasons. Social-emotional skills are assumed to (in)directly affect consequential outcomes in the short and long term. These include outcomes such as employability and work performance, health and longevity but also happiness, interpersonal relatedness and civic citizenship (Heckman, Stixrud and Urzua, 2006[3]; John and De Fruyt, 2015[4]; Holbein, 2017[5]; Taylor et al., 2017[6]). In addition, there is also evidence that social-emotional skills facilitate learning processes at school and contribute to academic achievement (Durlak et al., 2011[7]; Sklad et al., 2012[8]; Taylor et al., 2017[6]; Corcoran et al., 2018[9]).
This chapter reviews the background and conceptualisation of social-emotional skills and how these are related to consequential outcomes in education and the labour market. It discusses taxonomies on how to structure and operationalise these skills, converging on an overarching model representing five broad groups. In addition, it explores how these skills are connected to various types of work performance. Multiple common methods to assess social-emotional skills are reviewed. These comprise Likert-based and ipsative self- and observer reports, more objective tests and situational judgement tests, as well as behavioural residue indicators.
There are probably as many definitions of social-emotional skills as there are models structuring these skills. Literature identified social-emotional skills as three types of individual capacities (John and De Fruyt, 2015[4]; Scheerens, van der Werf and de Boer, 2020[10]; John and De Fruyt, 2015[4]). First, they manifest in consistent patterns of thoughts, feelings and behaviours. Second, they can be developed through formal and informal learning experiences. Third, they influence important socio-economic outcomes throughout the individual’s life. Such a definition is comprehensive enough to accommodate a wide range of skills, while highlighting their malleability and consequential effects for individuals and society.
The term social-emotional skills is to be preferred over “non-cognitive skills” because many of the skills subsumed under the previous definition also require specific cognitive abilities (De Fruyt, Wille and John, 2015[11]). Collaborative problem solving, for example, entails both cognitive and non-cognitive abilities.
Likewise, the term social-emotional skills is desired over “soft skills”. It is unclear why skills that are sometimes hard to learn are called “soft”. A better term often used in this debate is “transferable skills”. This underscores the idea that these skills are transportable from one context to another, and hence contribute to the individual’s adaptation across different life challenges.
In the labour market, organisations usually describe jobs and job vacancies in terms of “competences”. Hoekstra and Van Sluijs (2003[12]) define a competence as: “the ability to perform a particular type of task effectively or respond appropriately to a particular type of problem.” They conceptualise competences as the result of the interaction between an “expertise” in interaction with a “behavioural repertoire”.
In this interaction between expertise and behavioural repertoire, the expertise component is highly (but not exclusively) determined by cognitive ability factors. Conversely, an individual’s behavioural repertoire is chiefly determined by personality traits. The observed competence levels, however, are not merely a product of someone’s expertise and behavioural repertoire: attention and emotion fluctuations may interfere and impact on the finally manifested competence level.
Understanding socio-emotional skills also requires distinguishing between “how one is behaving typically” (also called a trait approach) relative to “how well one can behave” (or solve a problem). The latter is often represented as a maximal performance kind of construct such as mental abilities.
The distinction between “how well one can behave’” and “how one behaves typically” can be made conceptually. However, it is much harder to distinguish at an operational level. For example, the social-emotional skills measure known as SENNA (Primi et al., 2016[13]) has both self-efficacy (“how well one can behave”) and identity (“how one behaves typically”) items to assess social-emotional skills.
In sum, the terms social-emotional skills and transferrable skills are to be preferred over “soft” or “non-cognitive” skills. These concepts are synonymous with the term “competences” that is used more frequently on the labour market. Personality traits and cognitive abilities are to be considered as key building blocks of social-emotional skills, competences and transferrable skills.
There is a broad amalgam of different social-emotional skill taxonomies. Some advocate only a few, while others propose 100 or more skills.
Domains of social-emotional skills
Elias et al. (1997[14]) described six major domains of social and emotional learning. These include recognising and managing emotions, setting and achieving positive goals, appreciating the perspectives of others, establishing and maintaining positive relationships, making responsible decisions and handling interpersonal situations constructively.
Durlak et al. (2011[7]) advocated that social-emotional learning programmes should foster five broad competence sets: self-awareness, self-management, social awareness, relationship skills and responsible decision making.
From a different angle, Saarni (1999[15]; 2011[16]) focused on what a child needs to learn to become an emotionally and socially competent adult. She distinguished eight affect-oriented behavioural, cognitive and regulatory skills that are presumed prerequisites for emotional competence:
skills in discerning and understanding the emotions of others
skill in using the common vocabulary of emotion and expression
capacity for empathic and sympathetic involvement in others’ emotional experiences
skill in realising that inner emotional states need not correspond to outer expression
capacity for adaptive coping with aversive or distressing emotions by using self-regulatory strategies that ameliorate the intensity or temporal duration of such emotional states
awareness that relationships are defined by emotional genuineness of expressive display and reciprocity
capacity for emotional self-efficacy (i.e. individuals can accept their own emotional experience and view themselves as generally feeling the way they want to feel).
Trilling and Fadel (2009[17]) distinguished among a set of more than 160 different skill terms. These include terms as abnegation and altruism, engagement and enthusiasm, innovation and inquisitiveness, self-discipline and self-control, stability and tranquillity. Conceptually, these authors grouped these terms into the broader themes of what they call the 4Cs: creativity/innovation, critical thinking, communication and collaboration.
Grit: the jingle-jangle fallacy
Although there might be obvious reasons why these frameworks look different, this mixture of models and diversity of vocabularies hampered an integrative and in-depth debate among various stakeholders. The field further suffers from the jingle-jangle fallacy. In this fallacy, similarly named constructs across frameworks refer to different skills (jingle), whereas nearly identical skills are labelled differently (jangle).
The construct of “grit” provides an example of the fallacy. Grit, referring to perseverance and passion for achieving long-term goals, received considerable attention in education over the past decade (Duckworth et al., 2007[18]). Recent behaviour-genetic (Rimfeld et al., 2016[19]) and meta-analytic work (Crede, Tynan and Harms, 2016[20]) showed that grit is similar to the personality trait of “conscientiousness”.
Grit has a well-documented history as an important trait for learning within the personality field (Poropat, 2009[21]; Poropat, 2014[22]; Poropat, 2014[23]). Dumfart and Neubauer (2016[24]) showed that no other factors than conscientiousness and intelligence predicted languages and grade point average in eighth graders in Austria. In line with Rimfeld et al. (2016[19]), these authors demonstrated that grit showed no incremental validity beyond conscientiousness.
Restructuring the field of socio-emotional skills
The field of socio-emotional skills needs the kind of restructuring that occurred in the personality field. The challenge to bring order in the amalgam of (overlapping) social-emotional skill terms closely resembles the efforts of personality psychologists to structure the hundreds of personality descriptive terms available.
Such terms were finally summarised in the Big Five personality dimensions (John, 1990[25]) as Neuroticism, Extraversion, Openness to experience, Agreeableness and Conscientiousness. Today, personality psychologists agree these five dimensions form the largest common denominator to describe personality differences observable in various age and cultural groups (De Fruyt and Van Leeuwen, 2014[26]; McCrae and Terracciano, 2005[27]).
Advances on the Big Five framework
The availability of the Big Five empirical framework helped solve the discussion on differentially labelling rather similar constructs and examine the overlap among presumably distinct constructs. This breakthrough considerably advanced the personality field leading to improved knowledge on how traits are best assessed and how personality traits develop across life.
Primi et al. (2016[28]) advanced the discussion, demonstrating that items and scales of frequently used measures to evaluate social-emotional skills learning could be easily mapped within the Big Five scheme. A joint factor analysis of seven measures showed that all their items could be easily structured under the umbrella of the five major dimensions of personality. The measures analysed were the Nowicki-Strickland Locus of Control Scale (Nowicki and Strickland, 1973[29]), Rosenberg Self-Esteem Scale (Rosenberg, 1979[30]), Strengths and Difficulties Questionnaire (Goodman, 1997[31]), Big Five Inventory (John and Kentle, 1991[32]), Self-Efficacy Questionnaire for Children (Muris, 2001[33]), Core Self-evaluations (Judge et al., 2003[34]) and the Grit Scale (Duckworth and Quinn, 2009[35]).
Following analysis of the socio-emotional skills literature, John and De Fruyt (2015[4]) grouped social-emotional skills into five broad domains: collaboration, engaging with others, emotion regulation, task performance and open-mindedness. These domains parallel the psycho-social systems described by John and Srivastava (1999[36]). They are also fully in line with the empirical analyses by Primi et al. (2016[28]) of the content covered in instruments frequently used to assess social-emotional skills.
In the John and De Fruyt (2015[4]) framework, the engaging with others and collaboration fields describe social-emotional skills related to vertical (hierarchical) and horizontal (attaching/bonding) interactions among persons. Conversely, the emotion regulation field groups those skills that help individuals to deal with anxieties and uncertainties, recover from setbacks and control impulses. Both the interpersonal and the emotion regulation fields have been intensively studied in psychology. Finally, open-mindedness and task performance skills are crucial for adaptation and learning, grouping social-emotional skills related to exploration and exploitation, respectively, of the world around us.
These broad domains group more specific skills that are described in Figure 5.1. For example, a domain like collaboration encompasses skills like compassion, respect, trust and harmonious relationship building. Conversely, open-mindedness includes curiosity, creativity, aesthetic sensitivity, appreciating diversity, self-awareness, and autonomy and independence. The domains of the social-emotional skill framework map directly onto the Big Five framework as used by personality psychologists, i.e. collaboration maps to agreeableness, engaging with others maps to extraversion, emotion regulation maps to neuroticism, task performance maps to conscientiousness, and open-mindedness maps to openness to experience.
The Big Five framework has been successfully applied, along these lines, to structure competence models within the human resources field (De Fruyt et al., 2006[37]) or to classify the numerous social-emotional skills listed in the 21st century educational literature (John and De Fruyt, 2015[4]). The model is further helpful to structure needs and requirements of different jobs on the labour market, and connects well with the job descriptions included in Occupational Network (O*NET).
Combining the Big Five framework with O*NET
The O*NET database has turned out to be indispensable for practitioners and researchers to connect the worlds of education and the labour market. O*NET contains a detailed and updated set of job descriptions and requirements. This provides an overview of social-emotional skill demand at the level of the labour market.
The O*NET skills and abilities’ sections, together with the work styles and values’ descriptions for job titles, easily translate into the social-emotional skills subsumed under the Big Five framework. Work styles, for example, are a more “neutral term” in O*NET to refer to personality traits. Conversely, skills and abilities are manifestations of either personality traits or specific cognitive abilities or the interaction between them (Hoekstra and Van Sluijs, 2003[12]).
Relying on O*NET, McCloy et al. (2017[38]) recently identified those characteristics that are critical for effective performance in a broad range of occupations in 79% of O*NET occupations. This resembles 81% of the total US workforce in 2012 and 82% of the projected workforce in 2022. The top 20 skills of their analysis exclusively contained skills from each of the five domains of the social-emotional skills model proposed by John and De Fruyt (2015[4]), supplemented with problem solving. This type of knowledge makes it possible to compute an index of “work readiness” for each individual student relying on his/her scores on this identified skill set.
The proposed model claims to be fairly comprehensive, accommodating most social-emotional skills listed in the literature. However, the framework is probably less suited to classify those skills with a stronger cognitive component. Such cognitive skills would include critical thinking, metacognition, complex problem solving, or surface/deep-level information processing, for example. The more social-cognitive of the skills certainly have associations with the key dimensions of the proposed social-emotional skill model. However, they are also linked to models of cognitive ability (Bartram, 2005[39]).
Traits and cognitive abilities form the cornerstones of social-emotional skills. However, constructs of other individual differences are important to understand, assess and predict an individual’s behaviour and performance at work. Contemporary psychology on individual differences converges on a well-studied set of four groups of constructs: personality traits, cognitive abilities, interests and values.
There are well-researched models for each of these constructs. For personality traits, there are the Big Five/HEXACO models (Goldberg, 1993[40]; Ashton, Lee and Goldberg, 2004[41]). For cognitive abilities, there is the Cattell-Horn-Carroll (CHC) model (McGrew, 2009[42]). For (vocational) interests, there is Holland’s RIASEC model (Holland, 1997[43]). Finally, for values, there is the Schwartz value model (Schwartz, 1994[44]).
Vocational interests
Vocational interests can be defined as relatively stable individual differences that affect (choice and performance) behaviour through individuals’ preferences for particular work activities and environments (Wille and De Fruyt, 2019[45]). Holland’s (1997[43]) vocational interest model describes people’s interests and preferences in terms of their resemblance with six core interest domains, i.e. realistic, investigative, artistic, social, enterprising and conventional, that are structured in a hexagonal space.
This resemblance is often expressed in a RIASEC letter code, e.g. SEAICR, with the first letter resembling the person’s prime interest field, the second letter her/his second interest theme, etc. The model can be further used to describe and structure environments (e.g. educational majors or jobs), enabling the computation of a fit index between a person’s interests and his/her (new) environment. O*NET provides RIASEC letter codes for a broad range of jobs.
Cognitive abilities
Several models have been proposed over the past 100 years to structure cognitive abilities. CHC (McGrew, 2009[42]) is one of the most comprehensive for research and diagnostic specificity. To represent the positive manifold (Spearman, 1927[46]) among various intelligence measures, the model proposes a hierarchy of cognitive abilities. A broad general factor is on top, with more specific types of intelligence structured underneath (www.themindhub.com).
Personal values
Personal values can be defined as: “broad beliefs concerning desirable, trans-situational goals that serve as guiding principles in the individual’s life” (Vecchione et al., 2020[47]). The Schwartz (1994[44]) model on values provides a cross-cultural and comprehensive account of people’s values, distinguishing among ten core values that people may identify with to various extents.
The values are structured in a circumplex model, in clockwise order from the top: universalism, benevolence, conformity, tradition, security, power, achievement, hedonism, stimulation and self-direction. The quadrants are labelled clockwise from the top as self-transcending, conservation, self-enhancement and openness to change. The model is well-supported and used in large-scale cross-cultural research (e.g. European Social Surveys).
Interplay between the four core groups
The pairwise relationships between these four groups of core constructs have been extensively described, often meta-analytically. They represent distinct, though somewhat related, constructs. Personality and cognitive abilities, for example, correlate poorly. However, interest dimensions are to some extent associated with personality traits (Barrick, Mount and Gupta, 2003[48]) and cognitive abilities (Passler, Beinicke and Hell, 2015[49]). Meanwhile, personality traits are further associated with values (Parks-Leduc, Feldman and Bardi, 2015[50]).
Such background information is key to understanding the complex interplay of ingredients of specific social-emotional skills. For example, a social-emotional skill like “collaboration” requires specific interpersonal personality traits at work and verbal abilities. However, it will also be determined and embedded in a context related to the interests (e.g. the field one is working in) and the values that the individual embraces.
Against this background of (small to moderate) interrelationships, a good conceptual framing and understanding is needed of how these constructs hang together and affect work(-related) behaviour. From this perspective, “personality traits” and “abilities” are arguably key ingredients to explain how people will perform (in their jobs). Conversely, interests and values are predictors of the field (what and where) in which people will use and apply their social-emotional skills. For example, two persons can be alike in terms of their personality traits and abilities. Yet depending on their field of work (different interests and values; e.g. an artist vs. a cardiovascular surgeon), their salaries and options on the labour market may be different.
Social-emotional skills are considered pivotal today to understand and predict work performance. In other words, they relate to attainment of desired outcomes but also avoidance of unwanted outcomes. Work performance is understood in terms of five main types of performance: task, contextual, adaptive, learning and counterproductive. Task performance (quantity and quality) basically refers to the tasks explicitly listed in a job description; contextual performance refers to how well a person gets along with others and contributes to the team; adaptive performance refers to how well a person deals with change and insecurity; learning performance refers to what and how people learn, what they do to strengthen employability; and counterproductive performance/derailment is an undesirable type of performance.
Social-emotional skills and performance
Social-emotional skills are differentially related to these different types of performances as described below.
Task performance (Renn and Fedor, 2001[51]) is overall predicted by cognitive ability (Salgado et al., 2003[52]) and conscientiousness (Salgado, 1997[53]; Barrick, Mount and Judge, 2001[54]). However, other personality traits may be predictive as well depending on the nature of the job. In sales, for example, interpersonal traits are also important.
Contextual performance (Van Scotter and Motowidlo, 1996[55]) is more related to the interpersonal factors of extraversion (explaining the frequency of social interaction) and agreeableness (describing the quality of interpersonal behaviour).
Adaptive performance (Pulakos et al., 2020[56]) relates to flexibility and openness, and how easily change triggers anxiety in the individual. This is related, in turn, to the dimensions of openness to experience and emotional stability (De Fruyt and Rolland, 2013[57]), respectively.
For an understanding of learning performance, the combination of openness to experience and conscientiousness is critical. The so-called learning circumplex (De Fruyt et al., 2008[58]) is supplemented with cognitive ability (Salgado et al., 2003[52]).
Predicting and understanding performance
Various types of counterproductive work behaviour/derailment are predicted by low agreeableness and conscientiousness (Salgado, 2002[59]), honesty/humility (Pletzer et al., 2019[60]) eventually combined with low emotional stability (De Fruyt et al., 2009[61]), especially for the prediction of derailment. A direct translation of these types of work performance into the skill language of the OECD framework can be found in Table 5.1.
Predicting and understanding different types of work performance involves choosing the right social-emotional skill constructs, as well as analysing the nature of the kind of performance that one is interested in. In some cases, performance can be defined in terms of the best possible solution for a problem, referring to the maximal potential of the individual’s skills to accomplish a goal. Other types of performances refer to more daily and habitual expressions. These might include being friendly and respectful, or how one generally deals with stressors and everyday hassles.
From a select-out perspective, one wants to avoid derailment and counterproductive work behaviour in organisations. The prevalence of some of these behaviours might be infrequent, but a single malevolent act (e.g. a pilot making a mistake) can have a devastating impact. Consequently, in addition to selecting the appropriate social-emotional skill constructs, one also has to adequately model the relationships between such constructs and various performance indicators in structural relationship models and equations.
Most contemporary psychology and econometric models are primarily focused on understanding (work) performance differences between individuals. This was mainly driven by a focus in selection psychology to select those individuals among job applicants that will best perform in a job. Work performance in this context was mainly understood as a kind of static process where an individual provides a consistent level of performance.
Today, job performance is also considered a dynamic construct, with people demonstrating “performance fluctuation” around a mean.
Debusscher, Hofmans and De Fruyt (2016[62]) examine state neuroticism and task performance using an experience sampling design. They demonstrate that 60.9% of the variance of momentary task performance and 66.7% of the variance of state emotional stability was within-person. Hence, how well people (maximally) master a particular skill may differ from how people behave typically with variation across the day.
Debusscher, Hofmans and De Fruyt (2016[62]) convincingly demonstrate that people do not always perform to their maximal mastery level but rather show fluctuations. Others observe that roughly half of the variance in job performance is within the person (Debusscher, Hofmans and De Fruyt, 2014[63]); (Sosnowska et al., 2020[64]). However, it requires specific assessment tools and methodology to capture such variance (Lang et al., 2019[65]; Lievens et al., 2018[66]). AI has great potential to complement and eventually replace ambulatory assessments with ecologically valid information that can be electronically assessed and estimated and updated by algorithms.
This section reviews how social-emotional skills are assessed. It identifies several caveats in terminology, reflects on rating scales, and proposes situational judgement tests as an alternative assessment method. It explores more objective assessments to supplement self-report and rating-scale approaches. Finally, it examines personal living spaces and digital footprints as a source of information on personality traits, cognitive abilities, interests and values.
Correct use of terminology
Tests vs. assessments
The term “test” should be avoided (or correctly used) when it comes to the assessment of social-emotional skills. A “test” refers to examining someone’s maximal knowledge or capacity to solve a problem or perform. In a test, one typically assesses a person’s specific knowledge, cognitive abilities, memory, attention span or physical fitness. Often, tests include a set of items or exercises with an increasing level of difficulty (e.g. a reasoning task) or effort (e.g. physical tasks), with cognitive tasks usually having a right or wrong answer.
An exam for obtaining a driver’s licence, for example, first tests the knowledge of traffic rules, with “pass or fail” as the result. In addition, an examiner wants to know how this person performs typically in day-to-day traffic situations. Thus, the (broader) term “assessment” is preferred to “test”, which refers to the examination of these more typical modes of behaving and acting. In assessments, some of the behaviours can be considered “right or wrong”. However, they may also point to a continuum of behaviours that are more subtle and gradual in nature (e.g. “anticipatory driving” or “respecting others”). The broader term “assessment” better reflects this complexity.
Constructs vs. assessment methods
Constructs such as social-emotional skills should not be equated with the methods used to assess them. Constructs can be assessed using various methods that may either supplement or complement each other. A person’s collaborative skills, for example, can be described via self- or peer reports. However, they can also be assessed in a group exercise rated by independent assessors or via a situational judgement test.
Prediction in psychological measurement further relies on the principles of aggregation and triangulation. Aggregation measures across multiple indicators of a construct, while triangulation measures across different methods to assess this construct. Social-emotional skills, for example, are assessed using multiple items covering the bandwidth of a construct.
One also tries to explain incremental validity by picking up variance in a construct using alternative assessment methods. Peer ratings, for example, can complement self-ratings on assertiveness. Meanwhile, scores on a situational judgement test can supplement self- and peer reports. Schmidt, Oh and Shafer (2016[67]) have conducted a meta-analytic summary of the validity of methods commonly used to predict work and training performance.
Rating scales
Social-emotional skills are most frequently assessed using self-reports on an item set covering the bandwidth of a set of skills. Homogeneous groups of items are presented with Likert scales and anchors referring to either descriptive labels or frequency indicators. A descriptive label might be a five-point scale ranging from “not characteristic at all”, “barely characteristic”, “more or less characteristic”, “characteristic” to “very characteristic”. Frequency indicators might include “not at all”, “once a month”, “a few times a month”, “once a week”, etc. An example of such Likert-based assessment for skills enclosed in the OECD model can be found in Kankaraš, Feron and Renbarger (2019[68]) (Table 3.5).
In accordance with the aggregation principle, multiple items referring to various nuances of emotion regulation are administered to assess a person’s standing on this skill. Often, the item set includes both positively and negatively keyed items (e.g. “I can control my emotions well” versus “My emotions overwhelm me completely”). This allows to correct for acquiescence bias (a response tendency to say “yes” to items) (Primi et al., 2020[69]).
The psychometric characteristics of the scale and demonstration of measurement equivalence are both critically important. The former measures reliability, consistency and structural characteristics, while the latter allows comparisons across groups (e.g. across gender, age, culture). Such Likert-based assessments are sometimes preceded by an anchoring-vignette to be in a position to correct raw scale scores for group-reference bias (Primi et al., 2016[28]; Primi et al., 2016[13]).
“Observer reports” of an individual’s social-emotional skills form an alternative and complementary approach to self-ratings. The addition of this type of information adds predictive power to understand various outcomes (Connelly and Ones, 2010[70]). Similar psychometric requirements account for observer reports as for self-ratings. Preferably, the observer is well acquainted with the target individual. For example, the observer could be a parent or teacher rating the social-emotional skills of a child, partners rating each other or a supervisor rating a collaborator or vice versa. However, zero-acquaintance reports also have some small validity. Observer or peer reports are most useful to assess overtly observable behaviours (e.g. assertiveness). They are less suited to assess more internalising forms of behaviour such as emotion regulation skills.
Ipsative assessments of social-emotional skills have been introduced as an alternative for Likert scales. This is because the latter may be subject to different rater biases, including self-presentation bias (i.e. how people tend to portray a more socially desirable picture of themselves). In an ipsative assessment, the person must select from different sets of three to four items. The items in a set, which refer to different social-emotional skills, are illustrated in the triplet of items that follows:
I can calm down myself easily.
I can easily keep my promises.
The three items are indicators of emotion regulation, concentration and sense of responsibility, respectively. The subject indicates the item that is most like her/him and least like her/him. Although ipsative assessment creates interdependencies among the assessed constructs, more recent psychometric techniques have been developed to deal with this problem (Brown and Maydeu-Olivares, 2013[71]). As a result, ipsative item administration can be used to assess differences between individuals. More research, however, is needed on how to design optimal groups of items, including both positive and negative indicators of a construct so the model can be identified properly.
Situational judgement tests
Situational judgement tests (SJTs) are an alternative method to assess personality or social-emotional skills. In an SJT, persons are presented items describing a short situation, the so-called stem, together with a set of possible reactions. The stem can be a narrative description, but more recent approaches also use videos, animations or avatars to present the situation. Assessees are required to select from the set of responses the one that bests represents how they will behave in that situation, or are requested to rank those responses.
The responses are indicators of specific social-emotional skills. Subject matter experts a priori align responses to constructs, or their allocation is determined in a separate study and derived empirically. Assessees are presented several of such SJT items and their position on constructs is computed across their choices or rankings of responses across the different SJTs.
An example of an SJT stem assessing an element of leadership behaviour could be as follows: “You (as the team leader) have scheduled an online meeting with members of your team to discuss an adjustment in the work plan. You see in the chat function of the communication tool that one team member comments disrespectfully about someone from a different department (not participating in the meeting). What do you do?” The assessee can select from:
b) You react during the meeting, and tell him we do not communicate within the organisation with this tone.
c) You wait until the meeting is over and approach him individually on line to tell him that this is not the tone we use to communicate within the organisation.
d) You deny the comment in the chat because you consider it irrelevant for solving the work-planning problem.
SJTs have become popular in assessment and selection psychology due to their face validity and favourable reactions from assessees (because the situations have a realistic and attractive appeal on candidates applying for jobs). There is meta-analytic evidence supporting SJTs’ predictive validity, beyond self-ratings on personality inventories (McDaniel et al., 2007[72]).
However, SJTs have large developmental costs. They must often be developed to reflect the kind of (work) situations and behaviours that one wants to understand and predict. In a recent review, Lievens (2017[73]) suggested that the SJT format provides interesting possibilities for assessing both between and within individual differences. These possibilities emerge because the various situational descriptions in an SJT item set provide opportunities to study trait or skill variability (flexibility) across different situations.
The use and especially the design of SJTs, however, was recently questioned. Several studies demonstrated the situation descriptions were unnecessary or only contributed to increased predictive validity for specific criteria (Krumm et al., 2015[74]; Schäpers et al., 2020[75]). For example, Schäpers et al. (2020[75]) demonstrated that SJT validity to predict in-role performance and organisational citizenship behaviour was not substantively affected when removing the stem. Conversely, they found it did have an impact for predicting interpersonal adaptability and efficacy for teamwork. They further showed the impact of dropping the stems on applicant reactions was negligible. Overall, these studies question the heart of the SJT approach. They suggest that SJTs are less situation-dependent than previously thought.
More objective assessments
Duckworth and Yeager (2015[76]) convincingly argued to supplement self-report and rating-scale approaches with more objective forms of assessing socio-emotional skills. They said rating scales may be less effective, especially for summative educational assessment, when stakes of evaluation are high. Both positive and negative self-presentation styles may distort self-descriptions when financial consequences are attached to the outcome of the evaluation. Schools may be rewarded for good but also for poor performance (getting extra funds to remediate). Therefore, intentional distortion of self-descriptions is possible in both directions.
Other threats of the validity of self-reports are teaching-to-the-test bias or increasing awareness of a skill that was the target of an intervention. For example, after acquiring a better understanding of empathy through an intervention, students may adjust (towards the lower end) their self-described skill mastery level. In other words, they may describe themselves as lower on empathy skills after an intervention to improve empathy. Taken at face value, this would lead to the paradoxical conclusion that empathy skill-level decreased due to the intervention.
Several studies have attempted to develop more objective measures to assess specific social-emotional skills. Santacreu, Rubio and Hernandez (2006[77]), for example, described three computerised tests that examine risk tendency: Roulette, Betting Dice and Crossing the Street. In the Roulette test, people have to bet and can win or lose for obtaining a prize. The Betting Dice test is similar to the Roulette test. In the third test, a figure on the computer screen has to cross a street as quickly as possible without causing accidents with satisfactory levels of reliability and validity. Falk et al. (2015[78]) assessed altruism using a donation task where people could contribute a certain amount of money to a good cause.
The Balloon Analogue Risk Task (BART) is an interesting objective measure to assess risk taking (a specific form of lack of impulse control) (Lejuez et al., 2002[79]) or derivative tasks. In BART, participants are shown 16 balloons on a computer screen, one after another. They are asked to pump the balloon that eventually will burst, and will get a dollar for each (extra) pump.
Balloon burst points, however, are randomised. If a balloon pops before they cash their earnings, they lose everything collected with pumping that single balloon. When a balloon explodes, they are presented a fresh balloon and continue until they finish with all 16. The key dependent variable is the amount of money collected at the end of the experiment, which gives them extra chances for gaining a substantive and attractive prize.
Alternatives to BART are so-called lottery tasks, where people have to imagine they won $100 000 with a lottery. The “bank” approaches them for a financial investment with the following strategy: double the investment in two years with a similar chance that half of the money will be lost. Participants are requested to indicate the amount of money they will invest under this condition: a) nothing (i.e. decline the offer), b) $20 000, c) $40 000, d) $60 000, e) $80 000 or f) the full amount of $100 000.
The BART and lottery tasks are interesting examples because they strongly correlate with self-report measures of risk taking (Dohmen et al., 2011[80]). Grover (2018[81]) reported correlations of .73 and .62 between a self-report measure of seven items (with a Cronbach’s alpha of .93) with the number of adjusted pumps and the number of explosions in the BART task, respectively. Financial risk taking in the lottery task had a correlation of .79 with the self-report measure.
The previous examples made clear that more objective forms of assessment usually require more advanced ways of test administration. They also eventually require extra time beyond the easily administered paper-and-pencil self-descriptions. In addition, the number of constructs that can be evaluated more objectively is probably time-bound. Social-emotional skill inventories such as SENNA 2.0 (Primi et al., 2016[13]) assess 17 different skills using 153 items and can be easily administered in 25 to 30 minutes. Performance-based assessment of the same 17 constructs will probably require considerable extra time but also specific infrastructure.
The available objective measures cover only a small part of the proposed social-emotional skill framework. The domains of emotion regulation, engaging with others and openness are poorly covered by objective assessment tools. In addition to availability, the construct validity of objective measures needs additional research. More objective measures typically correlate poorly with social-emotional skill self-reports, and it is unclear why this is the case (Ortner and Proyer, 2015[82]). In addition, objective measures of social-emotional skills are frequently task-based with considerable cognitive and motivational load. This puts into question their validity as pure indicators of social-emotional skills.
Behavioural residue and digital footprint
Personal living spaces and digital footprints are a final source of information on people’s standing on social-emotional skill building blocks such as personality traits, cognitive abilities, interests or values. An office or bedroom, for example, could represent personal living space. A digital footprint could be represented by social media activity (Facebook profiles, Twitter activity, Internet searches, cell phone use and tracking information).
Gosling et al. (2002[83]) demonstrated that people’s offices and bedrooms contained so-called behavioural residue. Observers can make consensual and reliable ratings of people’s personality based on bits and pieces of information in these personal spaces.
More behavioural residue can be found in people’s use of online social networking sites and their digital footprint. Back et al. (2010[84]) found that Facebook profiles reflect instances of their owner’s personality and not self-idealisation. Ratings of the Facebook pages showed correlation coefficients mounting to .39 and .41 with self- and peer-rated personality for extraversion and openness to experience, respectively, when ratings of the Facebook page were averaged across raters. Correlations were .25 and .24 when single observer ratings were considered. Facebook ratings were not correlated with ideal self-ratings, when corrected for actual personality ratings.
Another study by Park et al. (2015[85]) analysed Facebook status messages and developed algorithms associating language use in Facebook posts with self-descriptions of personality. A subsequent validation study showed that Facebook language-based assessments were correlated on average .38 with self-ratings of personality. Similar accuracy results were obtained from analysing individuals’ smartphone behaviours, looking at communication and social behaviours, music consumption, app usage, mobility, overall phone activity and, finally, day- and night-time activity (Stachl et al., 2020[86]).
Specific combinations of these passive variables that could be read from an individual’s smartphone were useful to predict openness to experience, conscientiousness and extraversion self-ratings, except for Agreeableness. Conversely, only carefreeness and self-consciousness from the emotional stability domain were significantly related with smartphone behaviour indices.
Source: Kankaraš and Suarez-Alvarez, (2019[68]).
References
[41] Ashton, M., K. Lee and L. Goldberg (2004), “A hierarchical analysis of 1,710 English personality-descriptive adjectives”, Journal of Personality and Social Psychology, Vol. 87/5, pp. 707-721, https://doi.org/10.1037/0022-3514.87.5.707.
[84] Back, M. et al. (2010), “Facebook profiles reflect actual personality, not self-idealization”, Psychological Science, Vol. 21/3, pp. 372-374, https://doi.org/10.1177/0956797609360756.
[48] Barrick, M., M. Mount and R. Gupta (2003), “Meta-analysis of the relationship between the five-factor model of personality and Holland’s occupational types”, Personnel Psychology, Vol. 56/1, pp. 45-74, https://doi.org/10.1111/j.1744-6570.2003.tb00143.x.
[54] Barrick, M., M. Mount and T. Judge (2001), “Personality and performance at the beginning of the new millennium: What do we know and where do we go next?”, International Journal of Selection and Assessment, Vol. 9/1-2, pp. 9-30, https://doi.org/10.1111/1468-2389.00160.
[39] Bartram, D. (2005), “The great eight competencies: A criterion-centric approach to validation”, Journal of Applied Psychology, Vol. 90/6, pp. 1185-1203, https://doi.org/ttps://doi.org/10.1037/0021-9010.90.6.1185.
[71] Brown, A. and A. Maydeu-Olivares (2013), “How IRT can solve problems of ipsative data in forced-choice questionnaires”, Psychological Methods, Vol. 18/1, pp. 36-52, https://doi.org/10.1037/a0030641.
[70] Connelly, B. and D. Ones (2010), “An other perspective on personality: Meta-analytic integration of observers’ accuracy and predictive validity”, Psychological Bulletin, Vol. 136/6, pp. 1092-1122, https://doi.org/10.1037/a0021212.
[9] Corcoran, R. et al. (2018), “Effective universal school-based social and emotional learning programs for improving academic achievement: A systematic review and meta-analysis of 50 years of research”, Educational Research Review, Vol. 25, https://doi.org/10.1016/j.edurev.2017.12.001.
[20] Crede, M., M. Tynan and P. Harms (2016), “Much ado about grit: A meta-analytic synthesis of the grit literature”, Journal of Personality and Social Psychology, Vol. 113/3, pp. 492-511, https://doi.org/10.1037/pspp0000102.
[37] De Fruyt, F. et al. (2006), “Police interview competencies: assessment and associated traits”, European Journal of Personality, Vol. 20/7, pp. 567-584, https://doi.org/10.1002/per.594.
[61] De Fruyt, F. et al. (2009), “Assessing personality at risk in personnel selection and development”, European Journal of Personality, Vol. 23/1, pp. 51-69, https://doi.org/10.1002/per.703.
[57] De Fruyt, F. and J. Rolland (eds.) (2013), Personality for Professionals Inventory: PfPI Handleiding, Pearson, Amsterdam.
[26] De Fruyt, F. and K. Van Leeuwen (2014), “Advancements in the field of personality development”, Journal of Adolescence, Vol. 37/5, pp. 763-769, https://doi.org/10.1016/j.adolescence.2014.04.009.
[58] De Fruyt, F. et al. (2008), “Sex differences in school performance as a function of conscientiousness, imagination and the mediating role of problem behaviour”, European Journal of Personality, Vol. 22/3, pp. 167-184, https://doi.org/10.1002/per.675.
[11] De Fruyt, F., B. Wille and O. John (2015), “Employability in the 21st century: Complex (interactive) problem solving and other essential skills”, Industrial and Organizational Psychology-Perspectives on Science and Practice, Vol. 8/2, pp. 276-U189, https://doi.org/10.1017/iop.2015.33.
[62] Debusscher, J., J. Hofmans and F. De Fruyt (2016), “From state neuroticism to momentary task performance: A person x situation approach”, European Journal of Work and Organizational Psychology, Vol. 25/1, pp. 89-104, https://doi.org/10.1080/1359432X.2014.983085.
[63] Debusscher, J., J. Hofmans and F. De Fruyt (2014), “The curvilinear relationship between state neuroticism and momentary task performance”, Plos One, Vol. 9/9, https://doi.org/10.1371/journal.pone.0106989.
[80] Dohmen, T. et al. (2011), “Individual risk attitudes: Measurement, determinants, and behavioural consequences”, Journal of the European Economic Association, Vol. 93/3, pp. 522-550, https://doi.org/10.1111/j.1542-4774.2011.01015.x.
[18] Duckworth, A. et al. (2007), “Grit: Perseverance and passion for long-term goals”, Journal of Personality and Social Psychology, Vol. 92/6, pp. 1087-1101, https://doi.org/10.1037/0022-3514.92.6.1087.
[35] Duckworth, A. and P. Quinn (2009), “Development and validation of the short grit scale (Grit-S)”, Journal of Personality Assessment, Vol. 91/2, pp. 166-174, https://doi.org/10.1080/00223890802634290.
[76] Duckworth, A. and D. Yeager (2015), “Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes”, Educational Researcher, Vol. 44/4, pp. 237-251, https://doi.org/10.3102/0013189X15584327.
[24] Dumfart, B. and A. Neubauer (2016), “Conscientiousness Is the most powerful noncognitive predictor of school achievement in adolescents”, Journal of Individual Differences, Vol. 37/1, pp. 8-15, https://doi.org/10.1027/1614-0001/a000182.
[7] Durlak, J. et al. (2011), “The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions”, Child Development, Vol. 82/1, pp. 405-432, https://doi.org/10.1111/j.1467-8624.2010.01564.x.
[14] Elias, M. et al. (eds.) (1997), Promoting Social and Emotional Learning: Guidelines for Educators, Association for Supervision and Curriculum Development, Alexandria, VA.
[78] Falk, A. et al. (2015), The Nature and Predictive Power of Preferences: Global Evidence, https://EconPapers.repec.org/RePEc:iza:izadps:dp9504.
[40] Goldberg, L. (1993), “The structure of phenotypic personality-traits”, American Psychologist, Vol. 48/1, pp. 26-34, https://doi.org/10.1037//0003-066x.48.1.26.
[31] Goodman, R. (1997), “The strengths and difficulties questionnaire: A research note”, Journal of Child Psychology and Psychiatry, Vol. 38/5, pp. 581-586, https://doi.org/10.1111/j.1469-7610.1997.tb01545.x.
[83] Gosling, S. et al. (2002), “A room with a cue: Personality judgments based on offices and bedrooms”, Journal of Personality and Social Psychology, Vol. 82/3, pp. 379-398, https://doi.org/10.1037/0022-3514.82.3.379.
[81] Grover, H. (2018), The Upside to the Dark Side: An Empirical Investigation into the Moderators and Mediators of the Dark Triad and Work Related Outcomes, University College London, https://discovery.ucl.ac.uk/id/eprint/10063615/.
[3] Heckman, J., J. Stixrud and S. Urzua (2006), “The effects of cognitive and noncognitive abilities on labor market outcomes and social behaviour”, Journal of Labor Economics, Vol. 24/3, pp. 411-482, https://doi.org/10.3386/w12006.
[12] Hoekstra, H. and E. Van Sluijs (eds.) (2003), Managing Competencies: Implementing Human Resource Management, Royal Van Gorcum, Nijmegen.
[5] Holbein, J. (2017), “Childhood skill development and adult political participation”, American Political Science Review, Vol. 111/3, pp. 572-583, https://doi.org/10.1017/S0003055417000119.
[43] Holland, J. (ed.) (1997), Making Vocational Choices: A Theory of Vocational Personalities and Work Environments (3rd. ed.)., Psychological Assessment Resources, Odessa, FL.
[25] John, O. (1990), “The “Big Five” factor taxonomy: Dimensions of personality in the natural language and in questionnaires.”, in Handbook of personality: Theory and research., The Guilford Press, New York, NY, US.
[4] John, O. and F. De Fruyt (2015), Framework for the Longitudinal Study of Social and Emotional Skills in Cities, OECD Publishing, Paris.
[32] John, O. and R. Kentle (1991), The “Big Five” Inventory – Versions 4a and 54, University of California, Berkeley, Institute of Personality and Social Research, Berkeley, CA.
[36] John, O. and S. Srivastava (1999), The Big Five Trait taxonomy: History, measurement, and theoretical perspectives..
[34] Judge, T. et al. (2003), “The core self-evaluations scale: Development of a measure”, Personnel Psychology, Vol. 56/2, pp. 303-331, https://doi.org/10.1111/j.1744-6570.2003.tb00152.x.
[1] Kankaraš, M. (2017), “Personality matters: Relevance and assessment of personality characteristics”, OECD Education Working Papers, No. 157, OECD Publishing, Paris, https://doi.org/10.1787/8a294376-en.
[68] Kankaraš, M., E. Feron and R. Renbarger (2019), “Assessing students’ social and emotional skills through triangulation of assessment methods”, OECD Education Working Papers, No. 208, OECD Publishing, Paris, https://doi.org/10.1787/717ad7f2-en.
[2] Kankaraš, M. and J. Suarez-Alvarez (2019), “Assessment Framework of the OECD Study on Social and Emotional Skills”, OECD Education Working Papers, No. 2017, OECD Publishing, Paris, https://doi.org/10.1787/5007adef-en.
[74] Krumm, S. et al. (2015), “How “situational” is judgment in situational judgment tests?”, Journal of Applied Psychology, Vol. 100/2, pp. 399-416, https://doi.org/10.1037/a0037674.
[65] Lang, J. et al. (2019), “Assessing meaningful within-person variability in Likert-scale rated personality descriptions: An IRT tree approach”, Psychological Assessment, Vol. 31/4, pp. 474-487, https://doi.org/10.1037/pas0000600.
[79] Lejuez, C. et al. (2002), “Evaluation of a behavioral measure of risk taking: The Balloon Analogue Risk Task (BART)”, Journal of Experimental Psychology-Applied, Vol. 8/2, pp. 75-84, https://doi.org/10.1037//1076-898x.8.2.75.
[73] Lievens, F. (2017), “Assessing personality-situation interplay in personnel selection: Toward more integration into personality research”, European Journal of Personality, Vol. 31/5, pp. 424-440, https://doi.org/10.1002/per.2111.
[66] Lievens, F. et al. (2018), “The predictive power of people’s intraindividual variability across situations: Implementing whole trait theory in assessment”, Journal of Applied Psychology, Vol. 103/7, pp. 753-771, https://doi.org/10.1037/apl0000280.
[38] McCloy, R. et al. (2017), “Identifying universally critical characteristics of O*NET occupations. A prelude to assess workforce readiness”, presented at the SIOP Leading Edge Consortium, Atlanta.
[27] McCrae, R. and A. Terracciano (2005), “Universal features of personality traits from the observer’s perspective: Data from 50 cultures”, Journal of Personality and Social Psychology, Vol. 88/3, pp. 547-561, https://doi.org/10.1037/0022-3514.88.3.547.
[72] McDaniel, M. et al. (2007), “Situational judgment tests, response instructions, and validity: A meta-analysis”, Personnel Psychology, Vol. 60/1, pp. 63-91, https://doi.org/10.1111/j.1744-6570.2007.00065.x.
[42] McGrew, K. (2009), “CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research”, Intelligence, Vol. 37/1, pp. 1-10, https://doi.org/10.1016/j.intell.2008.08.004.
[33] Muris, P. (2001), “A brief questionnaire for measuring self-efficacy in youths”, Journal of Psychopathology and Behavioural Assessment, Vol. 23, pp. 145-149, https://doi.org/10.1023/A:1010961119608.
[29] Nowicki, S. and B. Strickland (1973), “A locus of control scale for children”, Journal of Consulting and Clinical Psychology, Vol. 40, pp. 148-154, https://doi.org/10.1037/h0033978.
[82] Ortner, T. and R. Proyer (2015), “Objective personality test”, in Ortner, T. and F. Vijve (eds.), Behaviour-based Assessment in Psychology: Going beyond Self-report in the Personality, Affective, Motivation, and Social Domains, Hogref.
[85] Park, G. et al. (2015), “Automatic personality assessment through social media language”, Journal of Personality and Social Psychology, Vol. 108/6, p. 934, https://doi.org/10.1037/pspp0000020.
[50] Parks-Leduc, L., G. Feldman and A. Bardi (2015), “Personality traits and personal values: A meta-analysis”, Personality and Social Psychology Review, Vol. 19/1, pp. 3-29, https://doi.org/10.1177/1088868314538548.
[49] Passler, K., A. Beinicke and B. Hell (2015), “Interests and intelligence: A meta-analysis”, Intelligence, Vol. 50/May-June, pp. 30-51, https://doi.org/10.1016/j.intell.2015.02.001.
[60] Pletzer, J. et al. (2019), “A meta-analysis of the relations between personality and workplace deviance: Big Five versus HEXACO”, Journal of Vocational Behaviour, Vol. 112, pp. 369-383, https://doi.org/10.1016/j.jvb.2019.04.004.
[22] Poropat, A. (2014), “A meta-analysis of adult-rated child personality and academic performance in primary education”, British Journal of Educational Psychology, Vol. 84/2, pp. 239-252, https://doi.org/10.1111/bjep.12019.
[23] Poropat, A. (2014), “Other-rated personality and academic performance: Evidence and implications”, Learning and Individual Differences, Vol. 34, pp. 24-32, https://doi.org/10.1016/j.lindif.2014.05.013.
[21] Poropat, A. (2009), “A meta-analysis of the five-factor model of personality and academic performance”, Psychological Bulletin, Vol. 135/2, pp. 322-338, https://doi.org/10.1037/a0014996.
[69] Primi, R. et al. (2020), “True or false? Keying direction and acquiescence influence the validity of socio-emotional skills items in predicting high school achievement”, International Journal of Testing, Vol. 20/2, pp. 97-121, https://doi.org/10.1080/15305058.2019.1673398.
[13] Primi, R. et al. (2016), “Development of an inventory assessing social and emotional skills in Brazilian youth”, European Journal of Psychological Assessment, Vol. 32/1, pp. 5-16, https://doi.org/10.1027/1015-5759/a000343.
[28] Primi, R. et al. (2016), “Anchoring vignettes: Can they make adolescent self-reports of social-emotional skills more reliable, discriminant, and criterion-valid?”, European Journal of Psychological Assessment, Vol. 32/1, https://doi.org/10.1027/1015-5759/a000336.
[56] Pulakos, E. et al. (2020), “Adaptability in the workplace: Development of a taxonomy of adaptive performance”, Journal of Applied Psychology, Vol. 85/4, pp. 612-624, https://doi.org/10.1037/0021-9010.85.4.612.
[51] Renn, R. and D. Fedor (2001), “Development and field test of a feedback seeking, self-efficacy, and goal setting model of work performance”, Journal of Management, Vol. 27/5, pp. 563-583, https://doi.org/0.1177/014920630102700504.
[19] Rimfeld, K. et al. (2016), “True grit and genetics: Predicting academic achievement from personality”, Journal of Personality and Social Psychology, Vol. 111/November, pp. 780-789, https://doi.org/10.1037/pspp0000089.
[30] Rosenberg, M. (1979), Conceiving the Self, Basic Books, New York.
[16] Saarni, C. (2011), “Emotional development in childhood”, in Tremblay, R. et al. (eds.), Encyclopedia on Early Childhood Development: Emotions, Centre of Excellence for Early Childhood Development, Montreal, QC.
[15] Saarni, C. (1999), The Development of Emotional Competence, The Guilford Press, New York.
[59] Salgado, J. (2002), “The big five personality dimensions and counterproductive behaviours”, International Journal of Selection and Assessment, Vol. 101/2, pp. 117-125, https://doi.org/10.1111/1468-2389.00198.
[53] Salgado, J. (1997), “The five factor model of personality and job performance in the European Community”, Journal of Applied Psychology, Vol. 82/1, pp. 30-43, https://doi.org/10.1037/0021-9010.82.1.30.
[52] Salgado, J. et al. (2003), “International validity generalization of GMA and cognitive abilities: A European community meta-analysis”, Personnel Psychology, Vol. 56/3, pp. 573-605, https://doi.org//doi.org/10.1111/j.1744-6570.2003.tb00751.x.
[77] Santacreu, J., V. Rubio and J. Hernandez (2006), “The objective assessment of personality: Cattell’s T-data revisted and more”, Psychology Science, Vol. 48/1, pp. 53-68, https://psycnet.apa.org/record/2006-07428-004.
[75] Schäpers, P. et al. (2020), “The role of situations in situational judgment tests: Effects on construct saturation, predictive validity, and applicant perceptions”, Journal of Applied Psychology, Vol. 105/8, pp. 800-818, https://doi.org/10.1037/apl0000457.
[10] Scheerens, J., G. van der Werf and H. de Boer (2020), Soft Skills in Education. Putting the Evidence in Perspective, Springer International Publishing.
[67] Schmidt, F., I. Oh and J. Shaffer (2016), The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years of Research Findings, Department of Management and Organizations, University of Iowa.
[44] Schwartz, S. (1994), “Beyond individualism/collectivism: New cultural dimensions of values”, in Kim, U. et al. (eds.), Individualism and Collectivism: Theory, Method, and Applications, Sage, Thousand Oaks, CA.
[8] Sklad, M. et al. (2012), “Effectiveness of school-based universal social, emotional, and behavioural programs: Do they enhance students’ development in the area of skill, behaviour, and adjustment?”, Psychology in the Schools, Vol. 49/9, pp. 892-909, https://doi.org/10.1002/pits.21641.
[64] Sosnowska, J. et al. (2020), “New directions in the conceptualization and assessment of personality – A dynamic systems approach”, European Journal of Personality, Vol. 34/6, pp. 988-998, https://doi.org/10.1002/per.2233.
[46] Spearman, C. (1927), The Abilities of Man, MacMillan, Basingstoke, UK.
[86] Stachl, C. et al. (2020), “Predicting personality from patterns of behaviour collected with smartphones”, Proceedings of the National Academy of Sciences of the United States of America, Vol. 117/30, pp. 17680-17687, https://doi.org/10.1073/pnas.1920484117.
[6] Taylor, R. et al. (2017), “Promoting positive youth development through school-based social and emotional learning interventions: A meta-analysis of follow-up effects”, Child Development, Vol. 88/4, pp. 1156-1171, https://doi.org/10.1111/cdev.12864.
[17] Trilling, B. and C. Fadel (eds.) (2009), 21st Century Skills: Learning for Life in our Times, Wiley.
[55] Van Scotter, J. and S. Motowidlo (1996), “Interpersonal facilitation and job dedication as separate facets of contextual performance”, Journal of Applied Psychology, Vol. 81/5, pp. 525-531, https://doi.org/10.1037/0021-9010.81.5.525.
[47] Vecchione, M. et al. (2020), “Stability and change of basic personal values in early adolescence: A 2-year longitudinal study”, Journal of Personality, Vol. 88/3, pp. 447-463, https://doi.org/10.1111/jopy.12502.
[45] Wille, B. and F. De Fruyt (2019), “Development of vocational interests in adulthood”, in Nye., C. and J. Rounds (eds.), Vocational Interests in the Workplace: Rethinking Behaviour at Work, Routledge, New York.