Chapter 5. Modelling teachers’ professional competence as a multi-dimensional construct
This chapter presents a framework for comparative studies that explains student achievement by teacher competence. Teacher competence is modelled as a multi-dimensional construct that includes cognitive and affective-motivational resources necessary to master classroom demands. Lesson planning, motivating students, classroom management and diagnosing student achievement can be regarded as crucial demands teachers have to master in many countries. Processes mediating the transformation of teacher’s cognitive and affective-motivational resources into classroom performance in terms of teachers’ situational skills are included in the framework as well. Perception, interpretation and decision-making are highlighted in this respect. Competence profiles describe the patterns how all teacher resources (cognition, affect, motivation, situational skills) play together. In this chapter, we pay attention particularly to teachers’ general pedagogical knowledge and skills which are facets all teachers need to deal successfully with classroom demands. Empirical results from the international “Teacher Education and Development Study in Mathematics (TEDS-M)” are presented, which examined teacher knowledge in 16 countries at the end of their programmes. In addition, results on mediating situated skills such as teachers’ perception or decision-making are presented.
Introduction
Student achievement in school subjects like mathematics or reading depends on a range of preconditions such as students’ demographic background or their knowledge gained prior to schooling. The education system cannot do much about these characteristics but to accept and work with them. In contrast, teacher quality is an important factor that has a proven effect on student achievement and is at the same time prone to change by educational policy. Far-reaching consensus exists that – if student achievement shall be improved – teacher quality is one of the factors to be addressed.
Strangely enough, not much evidence exists about the level of teacher quality from an international comparative perspective. Even less cross-country evidence exists about how teacher quality can be influenced, in particular improved, and how precisely teacher quality is related to student achievement. Such evidence is a necessary precondition if reforms shall take place. Baumert, Blum and Neubrand (2002) critiqued therefore work that is international yet ignores teacher quality. They described this as a conceptual “gap” in OECD’s Programme for International Student Assessment (PISA): “No account is taken of teachers’ professional background, declarative knowledge, belief systems or motivation, or indeed of their procedural knowledge and professional action” (p. 8). The present chapter outlines how to frame an international study that intends to overcome current shortcomings of large-scale assessments such as PISA.
The complex role of teacher characteristics
Evidence suggests that the instructional quality in a classroom and student achievement depend on different types of teacher characteristics, in particular on their knowledge, skills, beliefs, values, motivation and metacognition (Baumert et al., 2010; Schoenfeld, 2010). Even if all other classroom characteristics such as the composition of students in terms of their prior knowledge and demographic backgrounds, school resources and climate or neighbourhood characteristics are held constant, it is not enough just to refer to, for example, teachers’ content knowledge or their job motivation to explain instructional quality and student achievement. Such direct single relations do rarely exist; a broad range of cognitive and affective-motivational teacher characteristics has to be examined to make valid conclusions.
The reason for this complexity, which represents a challenge for the examination of teacher characteristics, is twofold. On the one hand, different types of instructional quality and student achievement are related to different types of teacher characteristics. If we want to explain student achievement – for example – in mathematics, not only teachers’ mathematical content knowledge (MCK) but also their mathematics pedagogical content knowledge (MPCK) is important (Baumert et al., 2010). If we want to explain instructional quality in terms of classroom management, teachers’ general pedagogical knowledge (GPK) is important (König et al., 2014). On the other hand, it is not sufficient for explaining instructional quality and student achievement to look at teacher knowledge only even if its broad range of content knowledge, pedagogical content knowledge and general pedagogical knowledge is covered in a study. These dispositional teacher characteristics, to a large extent acquired during formal teacher training, have to be applied in the practical context of classrooms. Given the multidimensionality and speed of teacher-student interaction (Sabers, Cushinger and Berliner, 1991), teachers’ situational skills are relevant. Perception of classroom situations, interpretation and decision-making are crucial here (Blömeke, Gustafsson and Shavelson, 2015).
Developing a framework for a comparative study that intends to explain student achievement by teacher characteristics means therefore to model teachers’ professional competence as a multi-dimensional construct that includes different types of resources necessary to master different job-related tasks (Weinert, 2001) on the one hand, and to be aware of mediating processes in terms of situational skills and instructional quality on the other hand (Baumert et al., 2010; Blömeke, Gustafsson and Shavelson, 2015). Furthermore, in a comparative study, the context characteristics play an important role and need to be considered because they vary strongly across countries. Otherwise, the link between teacher quality and student achievement cannot be established. Figure 5.1 displays the basic idea of this complexity, restricted on the teacher side to their cognitive dispositions and skills to facilitate the reading. This does not mean that teachers’ affective-motivational characteristics can be neglected.
Note: MCK = mathematics content knowledge, MPCK = mathematics pedagogical content knowledge, GPK = general pedagogical knowledge, PERC = perception and interpretation skills, DM = decision-making skills.
Source: Kaiser, G. et al., (2016), “Professional competencies of (prospective) mathematics teachers: Cognitive versus situated approaches”, Educational Studies in Mathematics, Vol. 92.
In this chapter, the idea of modelling teacher competence as a multi-dimensional construct is elaborated on in more detail. We will pay particular attention to teachers’ GPK which is a facet all teachers need to successfully deal with generic classroom demands, no matter whether they teach mathematics, mother tongue or history. In a next step, results from the international “Teacher Education and Development Study: Learning to Teach Mathematics (TEDS-M)” are presented, which examined for the first time teachers’ MCK, MPCK and GPK with representative samples of future teachers at the end of their teacher education programme in 16 countries. Finally, an approach is presented on how to assess mediating situated skills such as teachers’ perception or decision-making in a standardised way.
Facets of teachers’ professional competence
Teachers’ cognitive and affective-motivational characteristics can be subdivided into content knowledge (CK), pedagogical content knowledge (PCK) and GPK (Shulman, 1987) as well as into beliefs about the nature of the content taught and the nature of teaching and learning (Richardson, 1996), job motivation and personality traits. “Competence profiles” describe the patterns of how these resources play together (Blömeke et al., 2013). The level and quality of teachers’ professional competence is influenced by their preconditions before entering teacher education and their opportunities to learn during it (Blömeke, Suhl and Kaiser, 2011; Blömeke et al., 2012). Teacher education programmes in this sense reflect a country’s vision of what teachers are supposed to know and be able to do in class and how teacher education and professional development should be organised to provide the knowledge and skills necessary for successful accomplishment of teachers’ professional tasks. Whereas moderate differences exist across countries on how to define teachers’ CK, more controversies are seen with respect to PCK (Döhrmann, Kaiser and Blömeke, 2014) and GPK, as well as with affective-motivational facets of teacher competencies, how they are to be defined or whether they are to be included at all. Affective-motivational facets such as orientations and goals, and meta-cognitive facets like self-regulation are in some studies supposed to be decisive facets of teachers’ competencies, whereas in others they do not get recognised at all.
Despite such differences, many recent national large-scale assessments roughly followed the framework displayed in Figure 5.2. Some examples are the “Cognitive Activation in the Classroom (COACTIV)” study (Baumert et al., 2010), the different national TEDS studies that followed TEDS-M (Blömeke et al., 2013, 2014) and the funding initiative “Modelling and measuring competencies in higher education (KoKoHs)” in Germany (Blömeke and Troitschanskaia, 2013). A similar framework has been used in the context of the “Learning Mathematics for Teaching (LMT)” studies which originated in the US (Delaney et al. 2008; Hill, Ball and Schilling, 2008), but have in the meantime been applied in many other countries.
Source: Döhrmann, M., G. Kaiser and S. Blömeke (2012), “The conceptualization of mathematics competencies in the international teacher education study TEDS-M. ZDM”, – The International Journal on Mathematics Education, Vol.44/3, pp. 325–340.
The only comparative studies that examined teacher knowledge on a large scale across countries are the “Mathematics Teaching in the 21st Century (MT21)” study (Schmidt, Blömeke and Tatto, 2011) carried out in 2006 and the already mentioned TEDS-M study carried out in 2008 under the supervision of the International Association for the Evaluation of Educational Achievement (IEA) (Tatto et al., 2012). Currently, TEDS-M represents the largest comparative dataset that provides empirical evidence on the level, structure and predictors of teacher competencies at the end of teacher education. As an IEA study it was required to follow common high-stakes methodological quality criteria so that its results would be highly reliable. Random samples of future teachers were drawn; test monitoring and control of response rates took place, as well as weights and sophisticated statistical procedures were applied.
The development of TEDS-M, focusing on outcomes of primary and lower-secondary teacher education, reflects the growing effort to study teacher quality internationally:
The impetus for TEDS-M . . . was recognition that teaching mathematics in primary and secondary schools has become more challenging worldwide as knowledge demands change and large numbers of teachers reach retirement age. It has also become increasingly clear that effectively responding to demands for teacher preparation reform will remain difficult while there is lack of consensus on what such reform should encompass and while the range of alternatives continues to be poorly understood let alone based on evidence of what works. In the absence of empirical data, efforts to reform and improve educational provision in this highly contested arena continue to be undermined by tradition and implicit assumptions. (Tatto et al., 2012: 17)
While TEDS-M provided empirical data that informs policy and practice related to recruiting and preparing a new generation of teachers, data that allows to look not only within but also across countries with respect to the quality of practicing teachers and its link to student achievement is still missing. In this sense, TEDS-M illustrates how to frame such studies so that it is worthwhile to look at its instruments and results (see below), but it cannot cover the research gap regarding the urgent question how teacher quality and student achievement are linked to each other.
General pedagogical knowledge of teachers as a core competence facet
Two tasks of teachers can be regarded as crucial in all countries: instruction and classroom management (König and Blömeke, 2012). Generic theories and methods of instruction and learning as well as of classroom management can therefore be defined as essential parts of teachers’ GPK. According to Shulman (1987: 8), GPK involves “broad principles and strategies of classroom management and organization that appear to transcend subject matter” as well as knowledge about learners and learning, assessment and educational contexts and purposes. Similarly, and extending this definition, Grossman and Richert (1988: 54) stated that GPK “includes knowledge of theories of learning and general principles of instruction, an understanding of the various philosophies of education, general knowledge about learners, and knowledge of the principles and techniques of classroom management.” Future teachers need to draw on this range of knowledge and weave it into coherent understandings and skills if they are to become competent to deal with what McDonald (1992) called the “wild triangle” that connects learners, content and teachers in the classroom.
In order to define what these broad teacher tasks include, a reference to the state of instructional research is helpful because it provides evidence-based models of teaching and learning (Carroll 1963; Bloom 1976). In particular, the QAIT model by Slavin (1994) describes crucial teacher tasks in more detail. The QAIT model is one of effective instruction that focuses on four elements. “Quality of Instruction” (Q) refers to activities of teaching that facilitate the learning of students, for instance presenting information in an organised way or announcing transitions to new topics. “Appropriate Levels of Instruction” (A) is an element that refers to dealing with a heterogeneous class. For teachers, it is challenging to adapt instruction to students’ diverse needs. Adaptivity includes, for instance, different methods of within-class ability grouping. “Incentives” (I) covers teaching activities meant to enhance the motivation of students to pay attention, to study, and to perform the tasks assigned. For a teacher, this may mean, for instance, to relate the content of a lesson to students’ experiences. “Time” (T), finally, refers to the quantitative aspect of instruction and learning, for example, strategies of classroom management enabling students to spend a high amount of time on tasks. According to Slavin (1994), all elements have to be linked to each other and instruction is only effective if all of them are applied. The four elements of the QAIT model correspond to elements of other models of effective teaching (Good and Brophy, 2007) so that they can be regarded as basic dimensions of teacher quality.
Pedagogical content knowledge and content knowledge of teachers
PCK is a teacher’s subject-specific knowledge for teaching. Shulman (1987: 9) characterises it as an “amalgam of content and pedagogy that is uniquely the province of teachers, their own special form of professional understanding.” A teacher has to know about typical preconditions of his or her students and how to present a topic to them in the best possible way. Teachers should ask questions of varying complexity, identify common misconceptions, provide feedback and react with appropriate intervention strategies. Lesson planning knowledge is essential before instruction in the classroom can begin because the content must be selected appropriately, simplified and connected to teaching strategies. Curricular knowledge is also part of PCK and includes knowledge about teaching materials and curricula. Teachers have to consider issues such as the consequences for future lessons if a key topic in the curriculum were removed or taught in a different context.
The state of research is most elaborated with respect to mathematics teachers, which means that we are talking about MPCK. This form of knowledge includes mathematics-related curricular knowledge, knowledge about how to present fundamental mathematical concepts to students, and knowledge about typical learning difficulties of students in the field of mathematics. These sub-dimensions were used in TEDS-M as well as in other studies such as LMT.
CK includes not only basic factual knowledge of a subject but also the conceptual knowledge of structuring and organising principles of the corresponding academic discipline (Shulman 1987): why a specific approach is important and where it is placed in the universe of approaches to this discipline. Ball et al. (2008) distinguished between common CK that also other professions would have, specialised CK of teachers, and horizon CK. Both CK and PCK of teachers deal with subject-specific knowledge but from different perspectives. Studies by Schilling, Blunk and Hill (2007) and Krauss et al. (2008) demonstrate with respect to mathematics that while it is possible to distinguish between MCK and MPCK, the two are highly correlated.
The particular focus on mathematics teachers’ knowledge is related to warnings that their proficiency level may not be strong enough, given the marginalised role mathematics had been playing in teacher education in many Western countries. Mathematics educators (e.g. Schoenfeld 1994; Kilpatrick et al. 2001) and mathematicians (e.g. Cuoco 2001; Wu 1999) have repeatedly pointed to the risks of weak training in mathematics: teachers’ limited understanding of what mathematics is, a fragmented conception with vertical and horizontal disconnects, less than enjoyable teaching routines and an inability to implement modern mathematical ideas in school. However, systematic evidence supporting these claims is still missing.
TEDS-M instruments and major results
General pedagogical knowledge of future teachers at the end of their training
The TEDS-M test of general pedagogical knowledge, applied in three countries as a national option in addition to the core study (Germany, Chinese Taipei and the United States), was based on the above-mentioned theoretical framework. Four tasks of teachers were addressed: planning and structuring lessons, motivating students and classroom management, diagnosing student achievement and adaptivity (see Table 5.1 for more details).
A conceptual framework of cognitive processes describing the demands on future teachers when they respond to test items was part of the study as well so that the item development was based on a two-dimensional matrix of teacher tasks and cognitive processes (see Table 5.2). Following Anderson’s and Krathwohl’s elaborate and well-known model (2001), three cognitive processes were distinguished that summarised the original six processes: recalling, understanding/analysing and creating/generating. Future teachers had to retrieve information from long-term memory to respond to a test item. They had to understand or analyse a concept, a specific term or a phenomenon outlined by a specific test item. And they were asked to create or generate strategies on how they would solve a typical classroom situation problem which included evaluating the situation.
About 80 multiple-choice (MC) and constructed-response (CR) (also called open-response items [OR]) were developed as a national option in TEDS-M to assess the GPK of future primary and lower-secondary teachers from Germany, Chinese Taipei and the United States. Test validity could be confirmed in all three countries (König and Blömeke, 2012). Figure 5.3 presents an MC and an OR item example with a genuine response from the United States. The first item measured knowledge about motivating students. Future teachers had to recall basic terminology of achievement motivation (“intrinsic motivation” and “extrinsic motivation”) and they were asked to analyse five statements against the background of this distinction. Statement C represented an example of “intrinsic motivation” whereas A, B, D and E were examples for “extrinsic motivation.” In the second item example, future teachers were asked to support another future teacher and evaluate his or her lesson. This is a typical challenge during a peer-led teacher education practicum, but practicing teachers are also regularly required to analyse and reflect on their own as well as their colleagues’ lessons. The item measured knowledge of “structuring” lessons. The predominant cognitive process was to generate fruitful questions. For the open-response items, coding rubrics were developed and reviewed by experts on teacher education in Germany, Chinese Taipei and the US to prevent culturally-biased response coding and scoring.
Source: König et al. (2011), “General pedagogical knowledge of future middle school teachers: on the complex ecology of teacher education in the United States, Germany, and Taiwan”, Journal of Teacher Education, Vol. 62/2, pp. 188-201.
This GPK test was administered as a national TEDS-M option in Germany, Chinese Taipei and the United States after the TEDS-M MCK and MPCK tests had been applied. The data revealed that – on the primary school level – future teachers from the US were significantly outperformed by future teachers from Germany. The difference of nearly 1.5 standard deviations was very large. It meant that there was almost no overlap between US teachers on the one side and German teachers on the other side. Most of the worst achieving teachers from Germany still did better than most of the best achieving teachers from the US. Similar results were reported from the TEDS-M survey of future secondary teachers (for more details see König et al., 2011).
Pedagogical content knowledge and content knowledge at the end of teacher education
To measure future primary and lower-secondary teachers’ MCK and MPCK, two 60-minute paper-and-pencil tests were developed and applied in 2008 to approximately 13 000 future primary and 9 000 future lower-secondary mathematics teachers during standardised and monitored test sessions in 16 countries. An overview of participating countries is provided in Table 5.4. The items were intended to depict knowledge demands of classroom performance as closely as possible (National Council of Teachers of Mathematics, 2000). The primary assessment consisted of 106 items (74 MCK and 32 MPCK items); the lower-secondary assessments consisted of 103 items (76 MCK and 27 MPCK items). The items were assigned to booklets following a balanced incomplete block design to capture the desired breadth and depth of teacher knowledge.
The mathematics items included the content areas of number (as that part of arithmetic most relevant for teachers), algebra and geometry, with each set of items having roughly equal weight, as well as a small number of items about data (as that part of probability and statistics most relevant for teachers). The mathematics pedagogy items included aspects of curricular and planning knowledge and knowledge about how to teach mathematics. These two sets of items were given approximately equal weight. The items relating to curricular and planning knowledge covered areas such as establishing learning goals, knowing different assessment formats or linking teaching methods and instructional designs, and identifying different approaches for solving mathematical problems. The items relating to knowledge about how to teach mathematics covered, for example, diagnosing typical student responses, including misconceptions, explaining or presenting mathematical concepts or procedures, and providing appropriate feedback.
The majority of items were complex multiple-choice items. Some were partial-credit items. In addition and comparable to the GPK test, both the MCK and the MPCK tests covered three cognitive dimensions: knowing (recalling and remembering), applying (representing and implementing) and reasoning (analysing and justifying). Another feature that led the development of the items was their expected level of difficulty (novice, intermediate and expert). The items were developed benefitting from the experiences and items of the MT21 study mentioned at the beginning of this chapter (Schmidt, Blömeke and Tatto, 2011), as well as the “Knowing Mathematics for Teaching Algebra” study (Ferrini-Mundy et al., 2005) and the LMT study already mentioned (Hill, Loewenberg Ball and Schilling, 2008).
Source: Brese, F. and M.T. Tatto (eds.) (2012), User Guide for the TEDS-M International Database. Amsterdam, Netherlands, IEA.
The descriptive TEDS-M results revealed significant country differences in teacher education outcomes in terms of MCK and MPCK. Future primary teachers from Chinese Taipei achieved the most favourable MCK result of all of the countries participating (Table 5.5; Blömeke, Suhl and Kaiser, 2011). The difference from the international mean (of 500 test points) was large – more than one standard deviation, which is a highly relevant difference according to Cohen (1988). The achievement of primary teachers from the US was slightly above the international mean and roughly on the same level as the achievement of teachers in Germany and Norway. Their difference from the international mean was significant but of low practical relevance. These groups of teachers also reached significantly lower performance levels than Swiss and Thai teachers. If we take into account the United Nations Human Development Index to indicate the social, economic and educational developmental state of a country, the high performance of teachers from Russia and Thailand was striking.
Regarding MPCK, the achievement of future primary teachers from the US was roughly on the same level as the achievement of teachers in Norway, which was significantly above the international mean (see also Table 5.5). In this case, the difference from the international mean was of practical relevance. Teachers from Singapore and Chinese Taipei outperformed the US teachers. Whereas Singapore was behind Chinese Taipei in the case of MCK, these countries were on the same level in the case of MPCK. Regarding MPCK, Norway and the US were only half of a standard deviation behind the two East Asian countries, whereas this difference reached one standard deviation regarding MCK.
Senk et al. (2012) pointed out large structural variations across countries in how teachers were trained to teach mathematics. The authors grouped teacher education programmes therefore into four groups. Primary teachers trained as mathematics specialists tended to have higher MCK and MPCK than those trained as generalists. However, within each group of teacher education programmes, differences of about one to two standard deviations in MCK and MPCK occurred between the highest and the lowest achieving countries. The authors inferred from these results that the relative performance within countries might vary greatly, especially if more than one teacher education programme exists.
When it comes to future lower-secondary teachers, those from Chinese Taipei achieved the best MCK results at the end of their training (see Table 5.6). The difference from the international mean (of 500 test points) was very large – more than 1.5 standard deviations. The results of German lower-secondary mathematics teachers were notably above the international mean. They were, however, still a long way behind those of future teachers in Chinese Taipei. German teachers also performed more poorly than teachers from Poland, Russia, Singapore and Switzerland. Taking again into account the Human Development Index, the performance of lower-secondary mathematics teachers from Poland and Russia was remarkable. The results of future teachers from the United States were around the international mean.
With regard to MPCK, future lower-secondary teachers from the US again only performed around the international mean (see also Table 5.6). In contrast, the German teachers’ results were well above the international mean. Even though Chinese Taipei was still a long way ahead, the gap between Germany and Russia was smaller than in MCK and the difference between Germany, Singapore and Switzerland was not significant. These results reveal how important it is to distinguish between MCK and MPCK when looking at teacher knowledge. Whereas Malaysian teachers scored only slightly below the international mean in MCK, they had lower scores when it came to MPCK.
Overall, it is surprising that the ranking of countries in TEDS-M was very similar to the ranking of countries in the Trends in International Mathematics and Science Study (TIMSS) (Mullis et al., 2008), which allows the preliminary tentative conclusion that a cyclic relationship may exist – with the option to improve student achievement by increasing mathematics teachers’ professional knowledge.
In addition to these country-level analyses, there is again much to be learnt by distinguishing between different types of teacher education programmes. This approach must, however, be used with caution. The samples, which are already relatively small in the case of teachers compared to large-scale assessments of student achievement, are even smaller when teacher education programme types are examined. The estimates are thus less precise. With this caution in mind, however, we may hypothesise that, based on the TEDS-M data, teachers in concurrent programmes do just as well as teachers in consecutive programmes. Another hypothesis refers to the relevance of opportunities to learn mathematics, not only for achievement in MCK but also in MPCK. German lower-secondary teachers who were educated to teach on the upper-secondary level in addition to the lower-secondary level and thus had extensive education in mathematics, showed very strong MPCK, for example. Their MPCK was, on average, the same as that of Russian teachers and significantly higher than that of teachers from Singapore. German mathematics teachers who were qualified to teach on the lower-secondary level only did less well.
Processes mediating the transformation of teacher knowledge into classroom performance
A highly reliable assessment of important teacher characteristics has been accomplished through the studies described above. The analytical approach of TEDS-M allows precise diagnostics of strengths and weaknesses in teacher knowledge. Thus, the GPK, MPCK and MCK tests provide a feasible option of knowledge assessment. However, we have to point out that a research gap exists with respect to the transformation of these cognitive resources into classroom performance. Thus, an assessment of situation-specific teacher skills and a measure of instructional quality are necessary if one wants to link teacher knowledge to student achievement.
A theoretical framework on the transformation of cognitive resources into classroom performance was provided by Blömeke, Gustafsson and Shavelson (2015). It is displayed in Figure 5.6. Including a measure of situation-specific skills increases the validity of teacher knowledge assessments with respect to classroom performance. Classroom observations could be one such measure, but it is a very costly one. In addition, it could be unreliable given the unstandardised nature of classroom performance. Standardised video-based assessments provide an alternative approach which can serve as a proxy. A tool specifically developed to address the mediating skills pointed out by Blömeke, Gustafsson and Shavelson (2015) was developed in the context of a German follow-up study to TEDS-M called TEDS-FU (Blömeke et al., 2014; König et al., 2014). It assesses generic and domain-specific perception, interpretation, and decision-making skills of mathematics primary and lower-secondary teachers. Different job requirements are presented via video-vignettes, for example, diagnosing student achievement, explaining mathematics to students, classroom management or dealing with heterogeneity. The clips are designed around critical incidents (see Figure 5.7).
Source: Blömeke, Gustafsson and Shavelson (2015), “Beyond dichotomies: Competence viewed as a continuum”, Zeitschrift für Psychologie, Vol. 223, pp. 3–13.
Expertise research provided the theoretical framework for the development of this video-based assessment. Eighteen open-response items and 16 or 22 rating scales respectively were developed to assess primary or lower-secondary teachers’ skills related to classroom management and content-specific instruction. However, it is a challenge to improve the validity of knowledge assessments by using video-cued testing (Kane, 1992) because new sampling problems arise. The classroom situations selected have to be representative with respect to their frequency and centrality for teaching and learning. Furthermore, new evaluation problems arise because it is less straightforward in complex classroom situations to decide about correct–incorrect or effective–not effective teaching approaches than in distinct MC/CR items. Finally, new generalisability problems arise across the situations presented and towards the real world given the low number and variability of critical incidents that can be presented. Still, it is a worthwhile enterprise to improve the validity of competence assessments with respect to instructional quality.
Improving the validity of competence assessments with respect to student achievement is the next step. As pointed out, student achievement is domain-specific, and thus the value of an assessment that includes GPK only and neglects content and pedagogical content knowledge of teachers is limited. However, instrument development is particularly challenging in the case of MPCK. Depaepe, Verschaffel and Kelchtermans (2013) identified commonalities and differences in frameworks, study designs and instruments in a review of 60 publications about MPCK assessments. They identified as one commonality of several studies that MPCK is regarded as a bridge between CK and GPK and that it is a specific knowledge type of teachers. Döhrmann, Kaiser and Blömeke (2014) identified differences in the definition of MPCK between European (and partly Asian) countries and English-speaking countries though. This difference is also stressed by Hsieh, Lin and Wang (2014). The TEDS-M tests assess a common core of MCK and MPCK in a set of 16 counties and have shown that MCK and MPCK can be reliably and validly assessed across the different countries.
Conclusions
Teacher competence has to be modelled in a multidimensional way including different cognitive and affective-motivational facets to provide a valid picture that can be linked to instructional quality and student achievement. Also, the transformation of knowledge into performance has to be addressed by modelling teacher competence as a continuum that includes situational perception, interpretation and decision-making skills as mediating processes. Frameworks of how to think about this complexity are available from prior research.
The assessment of crucial cognitive resources of teachers – general pedagogical knowledge, pedagogical content knowledge and content knowledge – can also build on prior research. Reliable and valid paper-and-pencil tests are available from TEDS-M. This study has already revealed important country differences in outcomes of teacher education which should be of strong concern for policy makers in those countries lagging behind. Building on existing comparative studies and instruments provides, as an additional benefit, the chance to connect new results to these existing ones. Similarly, although not discussed in much detail in this paper, a survey of affective-motivational characteristics can build on existing experience.
The assessment of situation-specific skills is a bigger challenge. Direct observation of behaviour through trained raters is one option, but it is costly. Video-based assessments provide a standardised alternative. Prior research is available that suggests how to proceed. However, whether such approaches work across different cultures is an open question which would need a lot of research on measurement invariance. If researchers succeed in developing such an assessment, a huge research gap could be closed with respect to the relationship among teacher knowledge, classroom performance and student achievement. We would finally have evidence about where reforms are necessary and how it is possible to improve the latter.
References
Anderson, L.W. and D.R. Krathwohl (eds.) (2001), A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Longman, New York.
Ball, D.L., M.H Thames and G. Phelps, (2008), “Content knowledge for teaching: What makes it special?”, Journal of Teacher Education, Vol. 59, pp. 389-407.
Baumert, J. et al. (2010), “Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress”, American Educational Research Journal, Vol. 47, pp. 133-180.
Baumert, J., W. Blum and M. Neubrand (2002), “Drawing the lessons from PISA 2000 – Long-term research implications: Gaining a better understanding of the relationship between system inputs and learning outcomes by assessing instructional and learning processes as mediating factors”, Symposium on Assessing Policy Lessons from PISA, 18-20 November 2002, Berlin, www.mpib-berlin.mpg.de/Pisa/DrawingTheLessons.pdf.
Blömeke, S. and O. Zlatkin-Troitschanskaia (eds.) (2013), “The German funding initiative “Modeling and Measuring Competencies in Higher Education: 23 research projects on engineering, economics and social sciences, education and generic skills of higher education students”, (KoKoHs Working Papers, 3), Berlin and Mainz: Humboldt University and Johannes Gutenberg University.
Blömeke, S., U. Suhl and M. Döhrmann (2012), “Zusammenfügen was zusammengehört. Kompetenzprofile am Ende der Lehrerausbildung im internationalen Vergleich”, [Joining together what belongs together. An international comparison of competence profiles at the end of teacher training], Zeitschrift für Pädagogik, Vol. 58, pp. 422-440.
Blömeke, S. et al. (2014), “Entwicklung von Lehrpersonen in den ersten Berufsjahren: Längsschnittliche Vorhersage von Unterrichtswahrnehmung und Lehrerreaktionen durch Ausbildungsergebnisse”, [Teacher development during the initial years in the profession: Longitudinal prediction of lesson perception and teacher reactions by means of training results], Zeitschrift für Erziehungswissenschaft, Vol. 17, pp. 509-542.
Blömeke, S. et al. (eds.) (2013), Professionelle Kompetenzen im Studienverlauf: Weitere Ergebnisse zur Deutsch-, Englisch- und Mathematiklehrerausbildung aus TEDS-LT, Waxmann, Münster.
Blömeke, S. et al. (2012), “Family background, entry selectivity and opportunities to learn: What matters in primary teacher education? An international comparison of fifteen countries”, Teaching and Teacher Education, Vol. 28, pp. 44-55.
Blömeke, S., J.E. Gustafsson and R. Shavelson (2015), “Beyond dichotomies: Competence viewed as a continuum”, Zeitschrift für Psychologie, Vol. 223, pp. 3–13.
Blömeke, S., U. Suhl and G. Kaiser (2011), “Teacher education effectiveness: Quality and equity of future primary teachers’ mathematics and mathematics pedagogical content knowledge”, Journal of Teacher Education, Vol. 62, pp. 154-171.
Bloom, B.S. (1976), Human Characteristics and School Learning, McGraw-Hill, New York.
Carroll, J.B. (1963), “A model of school learning”, Teachers College Record, Vol. 64, pp. 723–733.
Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences (2nd ed.), Lawrence Erlbaum, Hillsdale, NJ.
Cuoco, A. (2001), “Mathematics for teaching”, Notices of the AMS, Vol. 48/2, pp. 168–174.
Delaney, S. et al. (2008), “Mathematical knowledge for teaching: Adapting U.S. measures for use in Ireland”, Journal of Mathematics Teacher Education, Vol. 11/3, pp. 171–197.
Depaepe, F., L. Verschaffel and G. Kelchtermans (2013), “Pedagogical content knowledge: A systematic review of the way in which the concept has pervaded mathematics educational research”, Teaching and Teacher Education, Vol. 34, pp. 12-25.
Ferrini-Mundy, J. et al. (2005), A Conceptual Framework For Knowledge For Teaching School Algebra, Authors, East Lansing, MI.
Good, T.L. and J.E. Brophy (2007), Looking In Classrooms, Allyn and Bacon, Boston.
Grossman, P.L. and A.E Richert (1988), “Unacknowledged knowledge growth: A re-examination of the effects of teacher education”, Teaching and Teacher Education, Vol. 4/1, pp. 53–62.
Hill, H.C., D.L. Ball and S.G. Schilling (2008), “Unpacking pedagogical content knowledge: Conceptualising and measuring teachers’ topic-specific knowledge of students”, Journal for Research in Mathematics Education, Vol. 39/4, pp. 372–400.
Kaiser, G. et al., (2016), “Professional competencies of (prospective) mathematics teachers: Cognitive versus situated approaches”, Educational Studies in Mathematics, Vol. 92.
Kilpatrick, J., J. Swafford and B. Findell (eds.) (2001), Adding It Up: Helping Children Learn Mathematics, National Academy Press, Washington, DC.
König, J. and S. Blömeke (2012), “Future teachers’ general pedagogical knowledge from a comparative perspective: Does school experience matter?”, ZDM Mathematics Education, Vol. 44, pp. 341–354.
König, J. et al. (2014), “Is teachers’ general pedagogical knowledge a premise for noticing and interpreting classroom situations? A video-based assessment approach”, Teaching and Teacher Education, Vol. 38, pp. 76-88.
König, J. et al. (2011), “General pedagogical knowledge of future middle school teachers: on the complex ecology of teacher education in the United States, Germany, and Taiwan”, Journal of Teacher Education, Vol. 62/2, pp. 188-201.
Krauss, S. et al. (2008), “Pedagogical content knowledge and content knowledge of secondary mathematics teachers”, Journal of Educational Psychology, Vol. 100, pp. 716–725.
McDonald, J.P. (1992), Teaching: Making Sense Of An Uncertain Craft, Teachers College Press, New York.
Mullis, I.V.S., M.O. Martin and P. Foy (2008), TIMSS 2007 International Mathematics Report: Findings From IEA’s Trends In International Mathematics And Science Study At The Fourth And Eighth Grades, TIMSS and PIRLS International Study Center, Boston College, Chestnut Hill, MA.
National Council of Teachers of Mathematics (NCTM) (2000), Principles And Standards For School Mathematics, NCTM, Reston.
Richardson, V. (1996), “The role of attitudes and beliefs in learning to teach”, in J. Sikula, T. Buttery and E. Guyton (eds.), Handbook Of Research On Teacher Education, 2nd ed., Macmillan, New York, pp. 102–119.
Schilling, S.G., M. Blunk and H.C. Hill (2007), “Test validation and the MKT measures: generalizations and conclusions”, Measurement: Interdisciplinary Research and Perspectives, Vol. 5/2–3, pp. 118–127.
Schmidt, W.H., S. Blömeke and M.T. Tatto (2011), Teacher Education Matters. A Study Of The Mathematics Teacher Preparation From Six Countries, Teacher College Press, New York.
Schoenfeld, A.H. (1994), “Reflections on doing and teaching mathematics”, Mathematical Thinking And Problem Solving. The Study Of Mathematical Thinking And Learning, Lawrence Erlbaum, Hillsdale, NJ, pp. 53–75.
Schoenfeld, A.H. (2010), How We Think: A Theory Of Goal-Oriented Decision Making And Its Educational Application, Routledge, New York.
Senk, S.L. et al. (2012), “Knowledge of future primary teachers for teaching mathematics: an international comparative study”, ZDM Mathematics Education, Vol. 44, pp. 307–324.
Shulman, L.S. (1987), “Knowledge and teaching: foundations of the new reform”, Harvard Educational Research, Vol. 57, pp. 1–22.
Slavin, R.E. (1994), “Quality, appropriateness, incentive, and time: A model of instructional effectiveness”, International Journal of Educational Research, Vol. 21, pp. 141–157.
Tatto, M.T. et al. (2012), Policy, Practice, And Readiness To Teach Primary And Secondary Mathematics In 17 Countries: Findings From The IEA Teacher Education And Development Study In Mathematics (TEDS-M), IEA, Amsterdam.
Weinert, F.E. (2001), “Concepts of competence: A conceptual clarification”, in D.S. Rychen and L.H. Salgnik (eds.), Defining And Selecting Key Competencies, Hogrefe, Göttingen, pp. 45–66.
Wu, H.H. (1999), On The Education Of Mathematics Majors: Contemporary Issues In Mathematics Education, Vol. 36, MSRI, Berkeley, CA.