Thinking Skills and Creativity 10 (2013) 233–249

Contents lists available at ScienceDirect

Thinking Skills and Creativity

journal homepage: http://www.elsevier.com/locate/tsc

Challenges in wide scale implementation efforts to foster

higher order thinking (HOT) in science education across a
whole school system
Anat Zohar ∗
School of Education, The Hebrew University, 91905 Jerusalem, Israel

a r t i c l e i n f o a b s t r a c t

Article history: This study explores the challenges involved in scaling up projects and in implementing
Received 22 January 2013 policies across the whole school system in the area of teaching higher order thinking (HOT)
Received in revised form 12 June 2013
in Israeli science classrooms. Eight semi-structured individual interviews were conducted
Accepted 15 June 2013
with science education experts who hold leading positions pertaining to learning and
Available online 4 July 2013
instruction on the state level of the following school subjects: elementary and junior- high
school science and technology; high-school physics; high school chemistry; and high school
Keywords:
Higher order thinking biology. Some of the challenges that the interviews revealed are common to many types
Thinking strategies of educational change processes. The interviews also revealed several challenges which
Large scale implementation are more specific to the educational endeavor of teaching HOT according to the infusion
From policy to practice approach across large numbers of classrooms: challenges involved in weaving HOT into
Teachers’ knowledge multiple, varied, specific science contents; challenges involved in planning a reasonable
and coherent developmental sequence of thinking goals; the fact that content goals tend
to have priority over thinking goals and thus to disperse of the latter in policy documents
and in implementation processes; and finally, the considerable challenges (pedagogical and
organizational) involved in developing educators’ sound and deep professional knowledge
in the area of teaching HOT and metacognition on a large, nation-wide scale. The data shows
that wide-scale implementation of thinking in Israeli science classrooms often develops as
an evolutionary rather than as a revolutionary process. The implications for designing large
scale implementation programs aimed at fostering students’ reasoning are discussed.
© 2013 Elsevier Ltd. All rights reserved.

1. Introduction

1.1. Issues involved in teaching HOT on a large scale

So teaching for thinking and understanding [across the whole school system]. . . We have not yet entirely deciphered
the code of how to do it [L2]
As the introductory citation indicates, the challenge of teaching thinking on a large, national scale is a huge one. There
is nothing new in acknowledging that a large gap often exists between educational policy and the way it is implemented.
This gap is especially large in the context of policies that address changes in the core of education, i.e., changes in learning
and instruction, such as the change involved in teaching thinking. Unfortunately, it is very difficult to change the core of

∗ Corresponding author. Tel.: +972 2 5345730; fax: +972 2 5345730.

E-mail address: Anat.zohar1@mail.huji.ac.il

1871-1871/$ – see front matter © 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.tsc.2013.06.002
234 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249

education on a large, system-wide scale. Large scale efforts to improve teaching and learning focus more on structural and
administrative characteristics of reform than they do on fundamental changes in the instructional core. Innovations that
require significant changes in the core of educational practice are usually not only limited in their effects to a small scale,
but also do not usually last very long.
Innovations addressing the teaching of thinking are definitely at the very core of learning and instruction. Without
delving into the challenges involved in defining higher order thinking (e.g., Resnick, 1987; Schraw, McCrudden, Lehman, &
Hoffman, 2011), I refer here to the latter concept in its widest sense encompassing issues such as thinking skills/strategies,
critical thinking, argumentation, use of evidence, scientific reasoning, scientific literacy, inquiry, problem-based learning and
problem solving. During the past 30 years there have been a substantial, and rapidly growing number of empirical studies
supporting models and theories that address teaching thinking in science classrooms. Consequently, educators are currently
familiar with many good models that work quite well for teaching science by emphasizing students’ higher order thinking
rather than merely memorization of facts.
Most of the models for teaching thinking in science education classrooms were studied within small scale projects. In
addition, there have been some pioneering attempts to scale up such projects to scores of teachers and classrooms (e.g., Adey
& Shayer, 1993; Adey & Shayer, 1994; Blumenfeld, fishman, Krajcik, Marx, & Soloway, 2000; Osborne, Erduran, & Simon,
2004; White & Frederiksen, 1998; Zohar, 2004). However, at the turn of the 21st century, successful teaching of thinking
on the level of small or even large-scale projects is no longer sufficient. Policy documents from all over the world highlight
the need to teach 21st century skills. HOT is an important component of any list of 21st century skills (Partnership for 21st
Century Skills, retrieved July, PCS, 2011; Pellegrino & Hilton, 2012). Resnick (2010) argues that scaling up the “thinking
curriculum” in a way that will foster proficiency for ALL students is currently a major educational challenge:
Today we are aiming for something new in the world: An elite standard for everyone. . . That is what the term 21st-
century skills really means. The skills are not new (some students have been successfully learning them in some schools
from the beginning of civilization). But the aspiration to successfully teach knowledge-grounded reasoning compe-
tencies to everyone is still just that—an aspiration. . .. But the transformation of the institution of schooling that will
be needed to come close to making the aspirational goal a real achievement is huge (p. 184)
The goal of this paper is to examine the challenges involved in scaling up instruction of higher order thinking. The meaning
of scaling up in this context is to take ideas and practices educators are familiar with on the level of projects and to implement
them on a national level, i.e., across the state’s whole school system. The paper examines these challenges by studying the
views of leaders who had been involved in various large scale efforts to implement HOT in science instruction. Naturally,
some of the pertinent challenges are common to gaps between policy and practice in general, or to scaling-up innovative,
reform pedagogies in other areas (e.g., Blumenfeld et al., 2000; Dede, Honan, & Peters, 2005; Elmore, 2004; Fullan, 2007;
Levin, 2008; Levin & Fullan, 2008; Lee & Krajcik, 2012). Yet, because of the unique features of teaching higher order thinking,
some of these challenges are unique to efforts aiming at fostering students’ thinking across hundreds or even thousands of
classrooms.
Since many of the challenges that will be described in the findings section pertain to the development of teachers’
knowledge, the next section will discuss relevant prior studies addressing teachers’ knowledge in the context of teaching
HOT. This will be followed by a section that will describe the educational context within which the present study took place.

1.2. Teachers’ knowledge in the context of teaching HOT

Instruction of HOT requires much more than adopting a new curriculum because it requires a deep change in teaching
practices. Like the teaching of other issues that pertain to current educational reforms, it stretches and challenges teachers’
capabilities. In order to be able to respond to the unexpected events characterizing “thinking rich classrooms”, teachers
must be able to teach in an intelligent, flexible and resourceful way that cannot be scripted into a fixed set of technical
instructional routines and skills (Carpenter et al., 2004; Loef-Frank, Carpenter, Fennema, Ansel, & Behrend, 1998). In order
to teach thinking successfully teachers need to replace the traditional view of teaching as transmission of information and
learning as passive absorption with more active, constructivist views of learning and an intricate set of specific beliefs and
knowledge about teaching. Let us take a closer look at this knowledge.

1.2.1. Subject matter knowledge and pedagogical content knowledge (PCK) in the context of teaching HOT
As many studies show, familiarity with whatever it is that one is supposed to teach is a necessary condition for instruction.
Another necessary condition for sound instruction is familiarity with appropriate teaching methods. There is a large body
of literature that, following Lee Shulman’s work, addressed various components of teachers’ knowledge and distinguished
(among other things) between subject matter knowledge, general pedagogical knowledge and pedagogical content knowl-
edge (PCK). However, since the classic discourse in this area usually applies to teaching concepts rather than to teaching
thinking, the meaning of these components of teachers’ knowledge is not straight forward when we try to apply it to the
context of teaching HOT. It therefore, requires further clarification.
The term used in the literature for whatever it is that one is supposed to teach is subject-matter knowledge (e.g., Cocharn &
Jones, 1998; Shulman, 1986, 1987; Wilson, Shulman, & Richert, 1987). But because of the unique nature of thinking strategies
this concept is confusing when the focus of our attention is on teaching thinking rather than on teaching facts and concepts.
 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249 235

Although according to Shulman subject matter knowledge includes substantive knowledge (the explanatory structures
or paradigms of the field) and syntactic knowledge (the methods and processes by which new knowledge in the field is
generated), content knowledge (the knowledge of specific facts and concepts) is an essential component. When we focus
on teaching thinking the traditional meaning of content knowledge is not at the core of our educational agenda. Therefore,
in order to avoid confusion and to delineate the unique nature of teaching thinking, I prefer in this context to substitute
the term “subject-matter knowledge” with the term “knowledge of elements of thinking.” This includes: (a) knowledge of
individual thinking strategies such as making comparisons, formulating justified arguments or drawing valid conclusions;
(b) knowledge of genres of thinking such as argumentation, inquiry learning, problem solving, critical thinking, scientific
thinking or creative thinking (Schraw et al., 2011; MOE, 2009;) (c) knowledge of metacognition (for elaboration, see below);
and (d) knowledge of a variety of additional issues which are important for a successful “thinking classroom” such as thinking
dispositions or habits of mind, and an appropriate “culture of thinking” (Perkins, Jay, & Tishman, 1993; Swartz, Costa, Beyer,
Reagan, & Kallick, 2008). It is important to note that several previous studies show that in-service and pre-service teachers’
initial knowledge of thinking strategies are often not sound enough for purposes of instruction (e.g., Bransky, Hadass, &
Lubezky, 1992; Brownell, Jadallah, & Brownell, 1993; Jungwirth, 1987, 1990, 1994; Paul, Elder, & Bartel, 1997; Zembal-Saul,
Munford, Crawford, Friedrichsen, & Land, 2002; Zohar, 2004).
A second component of teachers’ knowledge which is significant for the present paper is pedagogical content knowledge
(PCK). PCK is a blend of pedagogical knowledge and subject-matter knowledge that is specific to each teaching topic (e.g.,
Adams & Krockover, 1997; Cocharn & Jones, 1998; Gess-Newsome & Lederman, 1999; Kennedy, 1990; Loughran, Gunstone,
Berry, Milroy, & Mulhall, 2000; Shulman, 1986, 1987; Van Driel, Verloop, & de Voss, 1998; Wilson et al., 1987; Zeidler, 2002). In
the context of teaching higher order thinking, the classic conceptual distinction made in the literature between pedagogical
content knowledge and general pedagogical knowledge is fuzzy and unclear. Part of the difficulty in aligning teachers’
knowledge in the context of teaching thinking with the prevalent concepts used in the literature is related to the debate
among scholars regarding the question of whether thinking strategies are general or content specific. Teaching thinking
according to the infusion approach i.e., integrating the teaching of thinking with the teaching of specific contents (Ennis, 1989;
Swartz et al., 2008; Abrami et al., 2008) assumes that thinking skills have some elements that are general and other elements
that are content specific (Perkins & Salomon, 1989). This notion presents an innate difficulty in referring to the pedagogical
knowledge teachers have in this field as either pedagogical content knowledge (that tends to be embedded in specific
subject-matters), or as general pedagogical knowledge (that tends to be independent of specific subject-matters). It seems
that because of the special nature of the type of knowledge under consideration the existing constructs are problematic. I had
therefore, suggested addressing teachers’ pedagogical knowledge in relation to instruction of higher order thinking by using
a special term: pedagogical knowledge in the context of teaching higher order thinking (Zohar, 2004, 2008). This term fits well
with the term “knowledge of elements of thinking” explained earlier, and highlights the fact that pedagogical knowledge in
this field has some unique characteristics. At the same time this term does not imply a commitment to treat this knowledge
as either content-specific or general.

1.2.2. Teachers’ knowledge in the context of metacogniton

A third component of relevant teachers’ knowledge pertains to metacognition. One of the most widely used definitions of
metacognition was proposed by Flavell and his colleagues (Flavell, 1979; Flavell, Miller and Miller, 2002), distinguishing
between two major components of metacognition: metacognitive knowledge and metacognitive monitoring and self-
regulation. The latter component is also named in current literature as metacognitive skills (e.g., Veenman, Van Hout-Wolters,
& Afflerbach, 2006). Metacognitive knowledge includes three sub-categories: knowledge about persons, tasks, and strate-
gies. In the metacognitive skills branch of metacognition, Flavell et al. (2002) elaborate on monitoring and self-regulation,
and also describe planning and evaluating. These sub-categories are also used in other prominent frameworks. For example,
Schraw and his colleagues describe the regulation component of metacognition as comprising of processes of planning,
monitoring, and evaluation (Schraw, 1998; Schraw & Moshman, 1995). More recently, Whitebread and his colleagues have
proposed a framework that adopts Flavell’s definitions of metacognitive knowledge and defines regulatory skills of planning,
monitoring, evaluating, and control (Whitebread et al., 2009). Many studies show that using metacognition in the classroom
may improve learning in general (see Veenman, 2011 for review), and learning of problem-solving, inquiry and higher order
thinking in particular (e.g., Chen & Klahr, 1999; Lin and Lehman, 1999; Ross, 1988; Schoenfeld, 1992; Toth, Klahr, & Chen,
2000; White & Frederiksen, 1998, 2000; Zohar & David, 2008; Zohar & Peled, 2008).
Yet, despite its strong effect on learning, recent studies highlight the fact that teachers find enacting a pedagogy for
metacognition difficult. It is not trivial for them to take up research-based ideas in this field and to translate them into
practical recommendations (Leat & Lin, 2007). However, despite the prominent role of metacognition in student success,
only limited research has been conducted to explore teachers’ and pre-service teachers’ metacognitive knowledge, their
pedagogical knowledge, and their ability to make progress in these types of knowledge following PD (Abd-El-Khalick &
Akerson, 2009; Kramarski and Michalsky, 2009; Wilson & Bai, 2009). More specifically, a review of journal articles about
metacognition revealed that between 2000–2012 only three studies were conducted to explore science teachers’ learning
of issues pertaining to the pedagogy of teaching metacognition in science classrooms (Zohar & Barzilai, submitted).
One of the components of metacognition which is particularly significant for teaching and learning thinking strategies is
meta-strategic knowledge, or MSK for short. MSK consists of general knowledge about thinking strategies, i.e., what is the
strategy and when, why and how it should be used. In order to apply MSK successfully in the classroom, teachers need to
236 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249

have sound MSK, as well as a variety of specific elements of pedagogical knowledge consisting of relevant teaching strategies
such as: modeling the use of a thinking strategy in a variety of specific contents; providing opportunities for students to
articulate the thinking strategies they apply during thinking; to introduce the “language of thinking” into the classroom; to
design and to teach careful and thoughtful learning activities in which thinking goals are made explicit; and to engage in
long term and systematic planning of thinking activities across several sections of the science curriculum. Research findings
show that science teachers’ initial knowledge regarding MSK is lacking and unsatisfactory for the purpose of instruction and
that teachers are not aware of the pertinent pedagogical knowledge before they study about it in PD courses designed to
address that knowledge (Zohar, 1999, 2006, 2008).

1.2.3. Teachers’ knowledge pertaining to pedagogies of knowledge construction

A fourth component of relevant teachers’ knowledge pertains to the transition from pedagogies of knowledge transmis-
sion to pedagogies of knowledge construction. Elements of pedagogical knowledge regarding instruction of higher order
thinking seem to be tightly related to teachers’ underlying theories of instruction because teaching thinking cannot be carried
out using knowledge transmission pedagogies (see Zohar, 2004, 2008 for more details). Therefore, professional development
(PD) towards a “thinking curriculum” cannot ignore teachers’ intuitive theories about the nature of learning and instruction,
knowledge construction, transmission of information and the changing role of the teacher in the classroom. Consequently,
courses designed to prepare teachers for instruction of higher order thinking cannot simply focus exclusively on teaching
elements that are related directly to instruction of higher order thinking and must address the considerable challenges
involved in making the transition from pedagogies of knowledge transmission to pedagogies of knowledge construction.
This adds a layer of complexity to any attempt to prepare teachers for teaching HOT.

1.3. Educational context: relevant facets of the Israeli educational school system and a brief history of teaching HOT in
science classrooms in Israel

In order to understand the context of the processes discussed in this paper, some relevant background information about
the Israeli educational school system and about pertinent efforts to teach thinking over the years is called for.
The Israeli educational system is centralized. With approximately 2 million students (k–12th grade) and over 4000 schools
there is basically one mandatory curriculum prescribed by the Ministry of Education that covers a large percentage of what
is taught in most schools. At the end of high school students take the matriculation exams that consist of exams in seven
mandatory core subjects: Language (Hebrew/Arabic), English (as 2nd language), Mathematics, History, Bible, Literature
and Civics. Additional subjects are mandatory in elementary and junior high schools (Science and technology, Geography,
2nd foreign language, etc.). In addition, many other subjects are electives in high school (e.g., Biology, Physics, Chemistry,
Communication, Arts, Computers, etc.). Each subject has a steering committee which consists of academics, Ministry of
Education officials and teachers. The steering committee is usually chaired by a distinguished professor from the pertinent
academic discipline who is interested in contributing to educational practice in his or her field. For each subject there is
also a National Subject’s Superintendent (NSS) who is responsible (in collaboration with the steering committee) for policy
making (i.e., for defining the goals of the curriculum) and for the practical sides of instruction, including teachers’ PD and
assessment in that particular subject. NSSs work with a team of instructors who help to coordinate and to lead the above
activities in each subject. Instructors also provide teachers with pedagogical support through PD courses, visits in classrooms
and schools and by meetings with small groups of teachers to discuss professional matters. As we shall see, instructors have
a prominent role in the implementation processes described in what follows.
Teaching thinking is not a new goal in science education in Israel. In fact, it has quite a long history. Although in the
first half of the 20th century students had sometimes studied science by inquiry and through observations that took place
in field excursions, science education during that period had been, for the most part, descriptive and fact-based. The first
comprehensive science education reform in this area took place in the late 1960’s, highlighting inquiry and higher order
thinking. It centered on a policy decision to change biology education into inquiry teaching by adopting the inquiry-oriented
“Yellow” version of BSCS (Biological Science Curriculum Study) (1963), and adapting it to the context of the Israeli school
system. This reform which had been extremely innovative at the time, assumed that the way to create a long-lasting,
sustainable educational change must be holistic (Tamir, 2006). It was assumed that the route to success must pass through
working simultaneously in three main areas: curriculum materials, professional development, and assessment. The new
assessment methods that had been developed as part of that change process in Israel were innovative and unique in the
1970s’. The biology team under the leadership of Pinchas Tamir understood that a lack of consistency between the inquiry-
goals of the new curriculum and the assessment will put the reform effort at risk, because teachers will continue to “teach
for the test” in terms of content, rather than follow the inquiry-oriented learning environment of the new curriculum. The
matriculation exam was therefore, transformed in a radical way to reflect the goals of the new inquiry-based curriculum. In
effect, Tamir and his team changed the traditional matriculation examination into an assessment that by today’s terminology
can be defined as “alternative assessment.” The “new” matriculation examination consisted of multiple, varied means of
assessment designed to assess science knowledge as well as inquiry skills: a written test, a school-based research project
accompanied by an oral exam, a school-based laboratory test, and a field test. It is remarkable to note that many of the
main facets of the inquiry-based biology curriculum and assessment are still practiced in the biology national curriculum
 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249 237

even today, more than 40 years after the reform had been initiated. Consequently, biology teaching in Israel had been more
thinking oriented than physics or chemistry even in the early 2000s (Zohar & Schwartzer, 2005).
The next noteworthy reform policy is the “Tomorrow 98” reform. Following the nomination of a public committee,
a comprehensive report was written in 1992 about math, science and technology education. The report made a list of
recommendations aimed at the improvement of education in these areas with an eye to preparing students for life in the
21st century (Ministry of Education and Culture, 1992). Among other goals, the report stated that the development of
students’ thinking in science is an important educational goal. The report was followed by a generous budget devoted to
the improvement of k–12 science education. Among the report’s practical consequences were the following: (a) Two new
curriculum documents (for grades 1–6 and for grades 7–9) which included explicit HOT goals to be taught in science lessons.
In those documents lists of thinking skills are delineated in a special section of the curriculum that is separate from the
sections delineating science topics and concepts (b) Several separate projects that created learning materials designed to
integrate the teaching of science topics with the teaching of thinking strategies and to promote project-based learning.
Another important facet of these projects were PD courses aimed at developing teachers’ ability to use the new learning
materials in a proficient way (Eylon & Bagno, 1997; Spector-Levy, Eylon, & Schertz, 2008; Zohar, 2004). Changes aimed at
fostering HOT and inquiry learning were also introduced to high school chemistry (See Avargil et al., this volume).
Finally, in January 2007 the Israeli Ministry of Education (MOE) adopted a new general (i.e., not limited to science edu-
cation) national educational policy: “Pedagogical Horizon (PH)—Education for Thinking (Office of Pedagogical Affairs, MOE,
2007; Zohar, 2008; Gallagher, Hipkins, & Zohar, 2012). The novelty of the PH policy was is in addressing the teaching of
thinking as an explicit, major and universal educational goal; and in planning practical means for wide-scale implemen-
tation throughout the school system. The emphasis of the Pedagogical Horizon policy was on pedagogy rather than on
content: on ‘how to’ rather than on ‘what to’ teach. The policy adopted an infusion approach to teaching higher order think-
ing (HOT): thinking was integrated into school curricula rather than taught as an independent subject. Therefore, the policy
advocated thinking within conceptually rich domains of knowledge. An ideal lesson according to the PH policy consisted of
both content goals and thinking goals each of which were addressed in an explicit way. The lesson was rich in cognitively
challenging questions and tasks that made intense usage of thinking strategies such as argumentation, problem solving,
asking questions, comparing and contrasting, making decisions, controlling variables, drawing conclusions and identifying
assumptions. All these thinking strategies were being used within the lesson’s subject matter and were thus embedded in
rich conceptual contents. The classroom learning environment fostered discourse that was rich in the ‘language of think-
ing’. Inquiry learning was encouraged. Finally, lessons also fostered metacognitive thinking, including intensive engagement
with MSK. This means that meta-level elements of thinking strategies were taught in an explicit way. Principles pertaining
to the “what”, “when” and “why” of thinking strategies were explicitly addressed in class discussions, in active individual
and group-work and in learning materials (textbooks and computerized learning materials). Many studies show that such
explicit engagement with general meta-level features of thinking have a large positive effect on students’ level of thinking
(e.g., Abrami et al., 2008; Dwyer, Hogan, & Stewart, 2012; Zohar & David, 2008; Zohar & Peled, 2008).
Following the model of the BSCS program that had taken place four decades earlier the PH was implemented through
working simultaneously on the development of three areas: curricula and learning materials, PD and assessment.

1.4. Research question

In sum, this brief review shows that several system-wide substantial efforts to promote HOT had taken place in science
education in Israel over the years. These efforts may provide a fruitful context to study system-wide change processes in the
area of teaching HOT. Numerous informal conversations conducted over the years with some of the people who were deeply
involved in leading such implementation processes indicated that they had gained rich knowledge and insights through
their experiences. A set of interviews was thus designed with the overall goal of uncovering that knowledge and insights in
order to turn them into explicit common knowledge. The interviews covered too many topics to be addressed in a single
paper (see below for more details). The present paper centers on one of these topics—the challenges involved in scaling
up instruction of HOT across the whole school system. Consequently, the present paper explores the following research
question: What do science education leaders view as the main challenges in scaling up instruction of higher order thinking
across the whole school system?

2. Methodology

This study is based on a set of eight semi-structured individual interviews conducted with science education leaders
in Israel. These leaders were chosen because of their prominent roles and first-hand experiences in leading activities and
processes on the national level in the following school science subjects: elementary general science, junior high school
general science, high school biology, high school physics, and high school chemistry. Four interviewes were conducted
with National Subject Supervisors (NSS’s), i.e., with people who have had leading positions in directing and supervising
the pertinent school science subjects within the Ministry of Education. Four additional interviews were conducted with
university science education professors who, in addition to their academic roles, had been active and highly influential in
national level science education committees in the Ministry and in university-based science education projects. They served
238 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249

as chairs or members of steering committees who directed the various school science subjects, as directors of large-scale
science projects, and/or as directors of teachers’ national PD centers.
One of the university science education professors was involved intensively in leading educational activities in two
different school science subjects and was therefore, interviewed twice—a separate interview took place for each of the school
subjects s/he was involved in. Consequently there were a total of eight interviews but only seven different interviewees.
All the potential candidates that were asked to participate in this study consented to do so (100% consent). The researcher
had prior professional acquaintance with all seven interviewees. In order to encourage free expression of ideas the inter-
viewer promised anonymity. Because the interviewees are public figures who are well known in the educational arena
in Israel all personal details and potentially identifying information are omitted from the study. Therefore, additional
information about the interviewees cannot be provided here in order to preserve the promised anonymity.
Interviews asked about the present state of teaching for thinking and understanding in the relevant school science
subjects, about past and current change processes related to wide-scale implementation of thinking and about challenges
and assessments related to these processes (for the full interview protocol please see Appendix A). Interviews were between
one and two hours long. Interviews were semi-structured: Following a general presentation of the goal of the study and the
definition of higher order thinking in the interview, the researcher presented the same set of 10 questions to all interviewees.
However, following their diverse areas of expertise and personal experiences, each interviewer was allowed to elaborate on
different sections of the interview protocol. Different follow-up questions were thus presented to each of the interviewees.
The interviews were transcribed and analyzed using content analysis, i.e., “a process of qualitative data reduction and sense-
making effort that takes a volume of qualitative material and attempts to identify core consistencies and meanings” (Patton,
2002, p. 453). The interview transcripts were read several times and several short ideas emerging from the text were written
for each section of the interview, using a word-processor table. Sections consisting of ideas that had a similar meaning were
assembled together to create patterns.
Each pattern was then read and re-read to create codes and sub-codes. Sections consisting of the same codes and sub-
codes were assembled together to create groups of citations. Finally, each of the patterns chosen for the analysis presented
in the present paper (for elaboration on the choice process see below) were analyzed by trying to interpret the meaning
of the citations grouped under the same sub-code and code. The analysis thus focused on trying to make sense of citations
grouped in each pertinent pattern using a narrative approach.

3. Results

As mentioned earlier, due to the interviewees’ high expertise, the interviews were extremely rich in ideas and insights
and the text contained too many significant patterns to be included in one paper. In order to reduce the number of patterns
for the present analysis, it was decided to limit the topic of the present paper to the challenges involved in large-scale imple-
mentation of teaching HOT. Within this wide topic, two major groups of patterns were identified: (a) patterns addressing
challenges in large-scale educational change in general (i.e., challenges that are visible in many areas of educational change,
see Section 3.1); and, (b) patterns addressing challenges pertaining to pedagogical issues that are more specific to large-scale
implementation of HOT (see Section 3.2). In what follows the first group of patterns is described in short and the second
group is described in detail.

3.1. Challenges which are visible in many areas of educational change

The first group of patterns addressed issues involved in large-scale educational changes in general (i.e., changes that
are not specific to the implementation of HOT). Although the term HOT appeared in the text describing these patterns, this
term could have theoretically been replaced with terms describing other educational goals without any loss of meaning.
The ideas expressed in these patterns can be found in numerous previous studies describing educational change processes
in varied contexts. These patterns were judged as less central to the present study and are thus summarized and described
briefly in Table 1.
The data presented in Table 1 confirms that successful large scale implementation of teaching HOT encounters the same
general system-wide challenges as many other large scale educational change processes in numerous areas that appear in
the literature concerning educational change (e.g., Elmore, 2004; Fullan, 2007).

3.2. Challenges that are unique to large scale implementation of HOT

The second group of patterns addressed pedagogical issues which are more specific to the challenges involved in large
scale implementation of HOT (see Table 2). These patterns were judged as central to the present study and therefore,
they are described in detail, using a narrative approach. Since the borderline between the two groups of patterns is not
always clear-cut, my rule of thumb was that boarder -line patterns were judged according to whether or not I estimated
that the challenges described in the patterns contributed in a significant way to the emerging discussion about large scale
implementation of HOT.
In the case of patterns reflecting assessment however, an exception had been made to this rule of thumb. Although issues
pertaining to assessment are central to large scale implementation of HOT they were not described here in detail (but instead
 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249 239

Table 1
A summary of category A patterns.

1 Vision and leadership are needed to advance HOT goals. The driving force for leading implementation processes should be a long-term
vision rather than efforts to respond to the many frequent, ad hoc problems emerging from the field, injuring the developing of the vision.
2 Public support of chief administrators in the process is crucial. They need to state publicly that the educational goal of developing
students’ HOT is a prominent one and to post it high on their list of educational agendas.
3 A large gap exists between statements and policy documents on the one hand and what actually takes place in practice on the other hand.
4 Other policies and changes matter. Seemingly unrelated policy decisions, organizational structures and various changes from the past
may have a strong impact on the implementation of HOT. For example, processes that took place in junior high schools, resulted in
improved physics science content knowledge. This seemingly unrelated process affected the feasibility of teaching thinking in high school
physics.
5 Bureaucracy and budget constraints limit the implementation process. It is most crucial in covering extra teaching time (because more
hours are needed to implement the change successfully), in supporting teachers’ PD (through release time or other financial incentives for
devoting the time), and in providing instructors to support teachers’ classroom practice.
6 When the implementation encounters difficulties, when it becomes wide-scale, or simply as time goes by, the message and requirements
of the implementation may need to be simplified and compromised. There is a danger that the essence of the change will be diluted. The
implementation may then attend to surface, “technical” elements of the educational enterprise rather than to its deep content.
7 Sustainability requires “boosting up” of the main messages after a certain number of years to preserve the original stamina and
coherence of ideas.
8 The implementation requires organizational structures, among which are the following: an “implementation fan” of growing expertise
among instructors and leading teachers, teachers’ centers, incentives for teachers to participate in PD, appropriate technical infrastructure
(such as labs and lab technicians to support inquiry learning).
9 Frequent political and organizational changes result in frequent policy changes that damage the implementation process.
10 Academic expertise is an invaluable resource in planning, developing and implementing the change process. Relationships between
academics, practitioners and policy makers are important.
11 Seemingly disconnected processes that take place in the school system may bring up unexpected opportunities that can be hitched to
advance activities that support instruction of HOT. For example: budgets allocated to raising test scores were intended for promoting
science content knowledge. However, large chunks of that budget were used by various agents to develop learning materials and PD that
support HOT skills.
12 Assessment is a key factor. The Ministry of Education’s requirements for improved quantitative outcomes for students and teachers is an
inhibiting factor. Standardized tests (national and international) inhibit and contradict the development of HOT. The goal of improving
international test scores may bring in useful budgets and activities, but test preparation may interfere with teaching HOT. National
assessments (e.g. the matriculation exam) need to be changed in order to reflect thinking goals.

were described briefly in Table 1). Since space restrictions made it necessary to leave something out, I chose not elaborate
on assessment in this particular paper because: (a) addressing this significant topic required an elaborate theoretical and
data-based analysis that was beyond the scope of this paper; (b) Since assessment was the focus of several of the other
papers in this collection of papers I assumed that the relative contribution of an additional detailed discussion of assessment
would be smaller than that of other topics which were not as prominent in the other papers.
The second group of patterns are described in detail in the following sections and summarized briefly in Table 2.

3.2.1. The challenge of teaching thinking in varied, specific science contents

Integrating the teaching of content and thinking goals to create a coherent curriculum is a considerable challenge even
in small scale projects. However, the teaching of HOT strategies in varied, yet specific science contents across the whole
science curriculum is a much larger challenge. The underlying theoretical framework is that instruction needs to integrate
these two dimensions (i.e., content and thinking) according to the infusion approach to teaching HOT (e.g., Ennis, 1989;
Swartz et al., 2008). Accordingly, teaching thinking should be woven into the teaching of conceptually rich science content.
Moreover, the theoretical approach is that thinking goals should be conceived of as explicit, distinct educational goals
that should be discussed explicitly in the classroom while engaging with authentic, rich science topics. L1 formulated this
theoretical approach by saying: “We are talking about direct teaching of thinking, about teaching according to the infusion
approach—content knowledge together with procedural knowledge”. By the time this study took place (the interviews took
place during the summer of 2012), this approach is no longer an innovation in Israeli science classrooms, because, as L2
states: “Explicit reference to the need for infusing thinking skills has been on the agenda. . .since 1996”. Yet, implementing
this approach across the whole school system encounters difficulties on many levels:

Table 2
A summary of category B patterns.

1 The challenge of teaching thinking in multiple, varied, specific science contents.

2 Planning a reasonable and coherent developmental sequence.
3 The priority of content goals over thinking goals in policy documents.
4 The challenge of human capital: the intricate nature of large scale PD in the context of teaching HOT.
5 Conflict between scope and complexity of knowledge goals for teachers and duration of PD workshops.
6 The challenge of securing a highly-proficient, system-wide, stable infrastructure of instructors.
7 Challenges pertaining to metacognitive knowledge.
240 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249

This whole business of anchoring the thinking in the science topics was not entirely clear, nor was it emphasized. . .
There were a lot of discussions around this issue but there was no clear policy and no clear indication of how to actually
carry it out. . . . Eh. . . In the context of the new syllabus [i.e., the syllabus written in 2009–2010] the notion was that
thinking skills should be combined [with the science topics] in a systematic way. But... at least at the beginning it
was not clear where [i.e., in what science topics] to combine what [thinking skills] and how to perform it in a spiral
way. . . This whole issue of science content and thinking skills, and of teaching them in an integrated way, this is the
discourse you hear on the policy level. You would like to construct it in a way that the thinking skills will support
content knowledge and the content knowledge will support the thinking skills. . .. But nobody knows exactly how to
do it... [L2).
I think that in the [biology] curriculum content goals appear separately. Actually, I think it’s the same in the junior
high school science curriculum. Yes, there was an effort to connect the thinking skills into the blueprint of what needs
to be taught [in terms of content], but there is no indication of how to integrate them (L4)
So these needs [i.e., the needs to combine the teaching of content and thinking] came up, but people did not really
know how to handle them (L6]
These excerpts confirm that although the underlying theoretical framework was clearly that instruction needs to integrate
the content and thinking dimensions, in effect, over the years content goals and HOT goals were detailed in curriculum
documents in two separate lists. There was a list of content goals and a separate list of thinking goals, with no indication
for how to combine them. Moreover, it seems that this lack of indication was not incidental. Rather, it reflected gaps in
pedagogical knowledge even among academic experts. It seems that this type of experts’ pedagogical knowledge has been
developed over the years so that they knew more about it in recent years compared to several years ago. Yet, it seems that
there is still a long way to go because even at the time the interviews took place experts did not feel they had a clear and
systematic method for how to integrate the teaching of thinking and of science content:
The fact that we now have a better methodology [for integrating the teaching of thinking with science content] than
we used to, does not mean that we now have a good methodology. I don’t think it [i.e., a good methodology] really
exists. It’s something that people are currently developing, and will continue to develop in the future [L6]
In conclusion, it seems that the challenge of integrating the teaching of thinking in multiple, varied science content had
not yet been resolved even in the minds of the experts who were leading the implementation processes.

3.2.2. Planning a reasonable and coherent developmental sequence

Matching the thinking goals of a small-scale project to students’ age and developmental stage is a challenging yet man-
ageable task. This may be done based on a combination of research-based recommendations and educators’ experience
and on their intuition regarding the type of learning which is appropriate for students of various ages and cultural back-
grounds. Planning a whole curriculum encompassing varied ages and school subjects, however, is a much more complex
endeavor. In considering the “when” dimension of teaching thinking, Alexander et al. (2011) briefly summarize the vast
developmental psychology literature indicating that age can have significant effects on mental processing. Yet, Alexander
and her colleagues emphasize that the level of thinking competence depends upon a variety of additional reasons, such as
the amount of scaffolding provided to students during instruction, the learners’ prior experiences with HOT, their level of
subject-matter expertise and more. One particular post-piagetian model that deserves mentioning in this context is Fischer’s
Dynamic Skill Theory which postulates that a simple age-related stage theory is too simple to explain the vast variability
observed in human psychological traits. Instead of viewing stages of cognitive development fixed like the steps of a ladder,
this theory assumes a more dynamic metaphor for development—that of a constructive web. Unlike the steps in a ladder,
the strands in a web are not fixed in a determined order but are the joint product of the web-builder’s constructive activity
and the supportive context in which it is built. Therefore, the support (e.g., types of learning experiences and amount of
engagement) that the educational system may provide to develop children’s thinking abilities in younger ages may have
detrimental effect on the types of thinking tasks that would be appropriate for them in later ages (Fischer & Bidell, 2006).
Under these complex conditions it is therefore, not surprising that the issue of creating a coherent developmental trajectory
of thinking goals across the curriculum stood out as a huge challenge for the experts who participated in the interviews.
One major challenge is that it is not easy to determine the levels of difficulty of science thinking tasks. This challenge
came up both in elementary and high school science. L1 expressed this challenge when describing the work she did with
elementary teachers on asking/posing questions:
We have this exercise in which we invite teachers to ask questions on a [given] text. Sometimes the questions are
shallow and low-level, but sometimes we are beginning to receive complex, high-level questions which are really
challenging and demanding. So how can we create a reasonable progression? [L1]
Similarly, L5 expressed the challenge involved in defining the meaning of a new high school physics curriculum that will be
more demanding in terms of its’ levels of thinking: “When you declare a new program with a higher level of thinking, you need
to define what does a higher level of thinking mean. . ..”. L5 emphasizes that over the years there was no explicit theory for
“why a certain task is simpler or more complex, or what does a student need to do in terms of her thinking in order to provide
 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249 241

an answer [to a given task]”. This is because in physics, the complexity of determining the thinking level of a task is affected
by the interaction of several confounding factors such as the depth and quantity of mathematical understanding required for
the task, the difficulty involved in analyzing the visual information entailed in the task, and the degree to which non-explicit
assumptions are involved. In biology, L3 expressed a similar concern regarding the level of inquiry projects in biology. She
stated that it is perfectly OK with her if her high school students choose to investigate the effects of salt concentration on
seed germination for their inquiry projects. But such a research project could also apply to elementary school, and there
needs to be a clear difference between the requirements for these two age groups. For instance, in high school, she would
like students to get into the micro, cellular level rather than stay on the general, macro level. However, L3 reports that she
finds that defining those requirements is not a simple task. Consequently, it is not easy to explain it to teachers in a clear way.
Another, related, challenge is involved in creating a systematic sequence of thinking strategies that will be taught across
ages and subjects. L2 describes how in the 1996 junior high school syllabus thinking goals were addressed explicitly in the
first time as a list of thinking skills:
People always talked about the need to do it, but this was the first time that the syllabus had a page with a list of
thinking skills, for better or for worse . . . [15 years later] I still don’t think we have anything much better than that. I
remember we said that, at least in the syllabus, we should address thinking skills in an orderly manner, rather than
only state that they should be included [L2].
When the junior high school science syllabus had been updated six years later, the issue of how to address thinking skills
in a systematic way was brought up once again: “But. . . em. . . at least at the beginning, it was not clear where to address
which thinking strategies and how to do it in a spiral way. . .” [L2]. By the end of the first decade of the 21st century, this
challenge has not yet been resolved. By now, states L2, it has been widely accepted by the experts who plan the curriculum
that the teaching of thinking skills must be integrated into the science content. However, when the experts attempted to
actually execute this idea in the 2009 updated syllabus, they once again encountered a huge challenge:
You must think [simultaneously] about several content areas as well as about quite a large repertoire of thinking
strategies you need to teach during those three years [grades 7-9]. . .. The coordination and the spiral, and repeating
the same thinking strategy in several different contents [which is significant for students’ ability to achieve transfer],
and the various levels of thinking, and the adaptation to a variety of learners- this is a huge challenge even for experts.
We certainly can’t expect teachers do be able to do it on their own [L2].
Interestingly, at the time of the interviews (which took place 16 years after the 1996 syllabus first included an explicit
reference to thinking skills), the development of a draft of a detailed sequence of science inquiry thinking skills for grades
1–9 has been in progress. L1 and L8 both described this draft document stating that it is being constructed in a spiral way,
detailing for each grade level 3–4 thinking strategies which needs to be taught in an explicit, focused way. In addition, the
document attempts to map the curriculum and prevalent textbooks, suggesting content areas and activities in which it is
appropriate to engage in specific thinking strategies. However, since at the time of the interviews this document was still
only a draft, a deeper discussion of this effort was precluded.

3.2.3. The priority of content goals over thinking goals in policy documents and in their implementation
The fact that on the pedagogical level it was not entirely clear even to academic experts what are the best methodologies
for integrating HOT goals into the teaching of science contents, was perhaps one of the factors contributing to the ambigu-
ous messages sent by policy makers. In the earlier years, it seemed that the policy documents themselves suffered from
inconsistency in this area:
The process of trying to write a syllabus for junior high school took many years, and it [i.e., the inclusion of thinking
skills] went back and forth. Thinking skills were put in, but then they were taken out again.. . .(L2)
However, even after the message in the policy documents had become more coherent so that curriculum documents
emphasized HOT skills as explicit and clear goals, policy makers as well as teachers still devoted much more resources to
teaching content goals then to teaching thinking goals. When describing the process by which the schools were instructed by
the NSS office how to teach the prescribed curriculum, L8 (who is a practitioner working in the Ministry) said the following:
We took out the most important science topics from the curriculum of each school grade. We counted the number
of hours allocated to each topic in the curriculum and practically divided the school year in terms of what should
be taught at each part of the year. Now as you can see here [pointing to a document written for teachers and found
on-line] - it’s all content. There is nothing here about skills. It’s true that the Introduction to the curriculum says
that “skills will be studied in an integrated way . . .” [reading from the introduction]. But there is no policy document
showing that [the NSS office] supported the teaching of thinking in any structured or explicit way [L8].
L8 then described how following low achievements in the TIMSS and PISA international tests the Minister of Education
and the General Director decided to allocate more resources to improve science achievements. She stated that the goal of
that program was defined as improving “science knowledge and skills”. Despite this definition, said L8, the field work that
followed the policy statement centered almost exclusively on improving students’ knowledge:
242 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249

We focused more on content, on what to teach, which topics, principles, concepts, phenomena and scientific
processes. . . [and] much less on the skills, even though the whole policy move was oriented towards raising achieve-
ments, i.e., improving knowledge and skills. During the first two years we were working almost exclusively on the
content . . .. After two years we said, OK, the teachers are already teaching the required science topics, we now have
to start working on the science skills [L8]
This report was corroborated by an academic who had been extensively involved in developing learning kits for teachers
(developed as part of implementing that same policy):
The [recommended] time lines [for teaching] were arranged around contents. Now you need to teach energy, now
this and now that. . .. Developing the teaching and learning kits followed a similar pattern. The time-line for their
development was so tight that it was not possible to treat the skills in an appropriate way. I mean, more work was
done on the conceptual dimension. The treatment of the skill dimension was much less serious. . . from now on, in
the next several years, there will be an opportunity to do this because the learning materials will begin to integrate
thinking skills in a more serious way [L2].
These excerpts show that a similar picture was portrayed by an academic and a practitioner: although policy documents
called for the integration of HOT skills and science contents, work on implementing this policy across the school system
centered initially on the science content, shying away from the HOT skills. This indicates that the content goals were perceived
as more important and tangible. Only after two years of extensive work on implementing the new science contents, attention
turned to integrating the thinking into the science contents. During those first several years, a substantial budget was
allocated to raising science scores. This budget was, however, temporary. By the time this paper is being written, the special
budget had already been reduced in order to attend to newer educational policies in other areas. The big question is whether
by the time the system would be ready to treat the thinking skills seriously, enough of this special budget will still be available
for the substantial processes required for the implementation of HOT across the school system.

3.2.4. The challenge of human capital: The intricate nature of large scale professional development (PD) in the context of
teaching HOT
Like many recent scholars worldwide (e.g., Barber & Mourshed, 2007), all the interviewees in the present study viewed
teachers as a crucial element in successful implementation of HOT in science classrooms. They brought numerous examples
of teachers’ participation in committees that engaged in policy making, in planning curricula and in planning the imple-
mentation process. These examples affirm that representation of teachers in various national committees responsible for
designing these processes has become a common practice. Yet, similar to the findings of the previous studies discussed in
the literature review, the interviewees see the knowledge teachers need in order to become proficient in teaching HOT as
extremely challenging and complex. The development of that knowledge is in essence a pedagogical process which is not
easy to achieve even on a small scale project. When this pedagogical challenge encounters the constraints inherent to large
scale implementations, it becomes even more demanding. The interview data point to two main organizational constraints
characteristic of large scale implementation that affected the feasibility of the deep and long-term learning teachers need:
(a) the amount of time required for deep learning of the relevant complex knowledge relative to the small number of hours
allocated to that learning in teachers’ workshops (see Section 3.2.5); This problem was corroborated by shortage of resources
for school-based support for teachers; and, (b) a shortage in a stable group of highly trained and professional instructors
who could lead the work with teachers (see Section 3.2.6)

3.2.5. Conflict between scope and complexity of knowledge goals for teachers and duration of PD workshops
The enormity of the pedagogical challenge involved in developing the needed teachers’ knowledge conflicted with
the organizational structure that was available for teachers’ PD: the administrative infra-structure for PD simply did not
allow enough time to address this complex pedagogical challenge in a satisfactory manner. This idea was expressed in the
interviews in many different ways. L1 who was intensively engaged in leading teachers’ PD workshops elaborated on this
issue:
Getting to know the [thinking] strategies is not enough. You must also learn to understand their significance, to know
how to adapt them to your own classroom’s needs, to experiment with them. . . to be able to lead reflection processes
with students and with other teachers, to evaluate students’ work, to be able to give them feedback. All these elements
which are entailed in teaching HOT are not part of the culture we are used to. . . for example, what feedback will a
teacher write to her student when she is grading a test? There is a lot of work here, you know. . . How do you work
with teachers on acquiring all these tools? It’s really really not that simple. . . and we don’t have the time to do it
. . ..[L1]
In elementary school, an agreement with the teacher union dictates that teachers’ participation in PD would be limited to
30 h per year. Some of these 30 h must be devoted to content knowledge. The remaining hours are viewed as insufficient for
meaningful learning of the complex knowledge teachers need in order to make the transition to a thinking rich classroom:
Throughout their professional lives they are used to. . . teaching declarative knowledge. The curriculum always
addresses declarative knowledge. When they write lessons plans, it’s always about declarative knowledge. What
 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249 243

do I have to teach tomorrow? Take a look at teachers’ lesson plans and you will see: The respiratory system, or this
or that system. . . Their head is tuned in this direction. Suddenly they are hearing from us: “No. Not only declarative
knowledge. From now on you also need to address procedural knowledge”. This is a paradigm shift. . .. If a science
teacher attends [a workshop] for thirty hours of PD. . . thirty hours means only seven or eight meetings, and part of
the time is devoted to science content. You can’t expect a meaningful learning process in thirty hours. . . What also
seems to me extremely important in assimilating a culture of thinking is to work on habits of mind and on thinking
dispositions so that they would become part of the classrooms culture. I believe it is extremely important to do so
rather than to just be satisfied only with constructing distinct thinking strategies [L1].
L1 then continues to describe the complex demands of inquiry learning. The knowledge teachers need to be able to lead
inquiry learning is not only complex in itself compared to the duration of the workshops, but it becomes even more difficult
to address it appropriately in the face of competing and rapidly changing policies:
It’s also important to work on the development of complex thinking processes [such as inquiry] and to give them tools
to engage in them. . ..eh. . . but we don’t have enough time to do all that- You can’t do all that in thirty hours. And
there are always new policies. This year the NSS declared that inquiry learning is a priority. . ..[quiet for a while]. . ..But
along with inquiry they had also decided to promote the subject of health. You can’t do it all at once. Ehh.. in many
of the courses this year we worked with teachers on how to carry out a complete inquiry process from beginning to
end. I mean, starting with encountering a phenomenon, asking questions. . . this is a very difficult process. We try to
make them aware of the complete sequence of inquiry teaching. But this is not enough. We are missing the ability to
support them once they are back in their classrooms. They come to the workshop and they study but I don’t know
exactly what they absorbed. The fact that they nodded and told me how exciting this is. . . I don’t know. . .I am afraid
there is a large gap here . . .. [L1]

3.2.6. The challenge of securing a highly-proficient, system-wide, stable infrastructure of instructors

The second significant challenge in this section pertains to the problem of securing a highly-proficient, system-wide, stable
infrastructure of instructors who can assist in PD workshops and provide school-based pedagogical support for teachers.
The roles of these professionals include teaching in teachers’ PD workshops; participation in teams that develop model
learning materials and assessments; and visiting classrooms to provide school based feedback and support for teachers. This
level of work obviously requires knowledge that needs to be even more proficient than the knowledge teachers need for
classroom instruction. It therefore, takes quite a long time to prepare the high-quality practitioners needed to fulfil these
roles. In all four areas (general science, physics, biology and chemistry) the interviewees reported that they see instructors
as an extremely important link in the implementation process and that considerable resources were invested in their PD:
Yes. A second strategy which I really believe in pertains to instructors. Those who went through the PD courses . . .
(L5)
Implementation, in all areas, but also specifically in the area of teaching HOT, takes place on several levels. The first
level is that of the NSS and her instructors... They form a support group. A group that brings up new ideas and. . .tries
them out in experimental pilot projects. This is the group of people with whom I do much of my work. . . I have 22
instructors. . .. Plans are made together with the instructors who form the “implementation fan”. Then they meet with
groups of teachers in the various districts and disseminate the decisions and plans that we made. Usually we first try
it in pilot projects and then disseminate it to all teachers. [L3]
Leading teachers and instructors are usually chosen from teachers who are much above average in terms of their
pedagogical abilities, i.e., they are known as “star” teachers. Yet, ample resources are often devoted to their long term
PD:
In order to actually implement this policy document [i.e., about teaching HOT], we first needed to train instructors.
We had a complete training system . . .. We trained a whole group of instructors in fostering HOT [L1]
‘Some groups of instructors met regularly for 6-7 years, other groups met only for 3-4 years. . ..[L2].
Following such long term PD workshops, instructors were indeed capable to support teachers in developing the ability
to teach for thinking and understanding:
Wherever we had projects that supported teachers in their field work we found really interesting things. . . For instance,
[in a Ph.d study conducted under the supervision of L2], we followed the development of teachers with whom we had
worked for several years. We found a huge development. Then we had quite a few courses for leading teachers where
we also saw a substantial development. It’s not true that you cannot help teachers make progress in this area. But we
came to the conclusion that it’s very difficult to do that on a large scale . . .[L2]
In sum, the basic rationale for building the infra-structure of instructors was to support the ability of scaling up. The
assumption was that this group of professionals would be able to work with relatively large numbers of teachers as leaders
of PD workshops and as mentors of school-based instruction. Experience as well as a formal assessment study indicated
244 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249

that it was possible to help this group of excellent teachers make a huge development in their pedagogical knowledge in
the context of teaching thinking. Why then, the pessimistic conclusion regarding the scalability of teachers’ development
in this area? The answer came up in many of the interviews when the interviewees described examples of how various
organizational obstacles blocked the flow of knowledge from instructors to teachers: First, it took several years to prepare
high-quality practitioners who had enough knowledge and experience to guide teachers in the complex PD processes that
needed to take place. Then, there were several organizational reasons that made this group of experts unstable with large
turn-over, including the following: (a) Frequent change of policies required frequent transitions to other areas of teacher
support. Therefore, often after making the long-term investment in the development of instructor’s knowledge in the context
of teaching HOT they were then assigned to work with teachers on other areas (such as ICT or content goals) and the
investment in the development of their knowledge in the context of teaching HOT was lost; (b) Inherent difficulties involved
in working with teacher combined with low salaries for instructors (relative to the time they needed to invest in carrying
out their role), caused a rapid turn-over of instructors. Their less than optimal working conditions encouraged many of these
excellent people to leave their position after a relative short period, thereby contributing to a disrupted flow of knowledge
between policy documents and classrooms.

3.2.7. Challenges pertaining to metacognitive knowledge

Another important yet complex knowledge element that was found to be difficult to address during the PD workshops
in a satisfactory manner is metacognitive knowledge. As explained earlier, metacognition is “thinking about thinking”. This
means that in order to be able to engage in metacognitive teaching about HOT, teachers must first be familiar with the
pertinent cognitive processes i.e., with the strategies involved in the thinking that needs to take place in their classrooms. In
addition, teachers must also acquire the special pedagogical knowledge that pertains to metacognitive teaching. L1 described
how addressing the various components of metacognition during the PD workshops presented additional challenges:
Teachers must first experience metacognition in their own learning, they need to experience learning processes
that include metacognition. Then we need to help them construct two types of meta-level knowledge: First, the
meta- strategic knowledge of the thinking strategies which they are not familiar with. Second, the meta-strategic
knowledge necessary for teaching. . . the pedagogy of how to teach metacognition . . .. If a teacher is not familiar with
the “thinking map” [i.e., with the meta-strategic knowledge of a thinking strategy] it will be very difficult for her to
construct a metacognitive teaching strategy because these two things go together. . .. [quiet. . .]. And we don’t have
enough time to do that. . .. We have only 30 hours and we can’t fit this in in a meaningful way . . .[L1]
Over the years L1 had substantial experience managing numerous elementary science teachers’ workshops. Based on
that experience she described how she had repeatedly witnessed teachers’ gaps in knowledge of thinking strategies. She
also noted that the workshops’ limited number of hours hindered teachers’ ability to acquire the necessary pedagogical
knowledge for teaching metacognition. L1 had been aware of these limitations. Yet, she sadly stated that the PD workshops
she was leading could not adequately address these knowledge gaps. The problem from her point of view, i.e., from the
point of view of a highly qualified professional who planned the curriculum for teachers’ PD, was that the complexity
of the metacognitive knowledge teachers needed, the fact that it must be built on a prior solid familiarity with thinking
strategies, and the fact that the duration of the PD was limited, contributed to the difficulty of addressing metacognition
in an appropriate way. She then continued to describe how in some of the courses, they did try to address metacognitive
teaching. Yet, she was not happy with the result. She viewed metacognitive teaching as so challenging that she was skeptical
as to the effect that a limited duration of engaging with the relevant knowledge could have on teachers’ practice. The fact that
there was no budget for classroom supervision and mentoring mades her even more skeptical as to whether the workshop
indeed made it possible for teachers to be able to actually apply metacognition in their classrooms.
A similar gap was also reported in the case of the ability of high school chemistry teachers to apply metacognitive teaching
in their classrooms:
Metacognition appears in our instructional unit . . . as “Time to Think”.., i.e., the unit includes a section which shows
students how to think, a kind of reflection. . . But not all teachers. . . can work with this section. I just gave a lecture in
a workshop for chemistry teachers and I was astounded to discover that many teachers, even though it appears in the
textbook, do not really understand it. It’s not only that they do not teach the metacognition in the relatively simple
way it appears in the instructional unit, but they don’t even know how to use it for monitoring their own thinking
and for monitoring their students’ responses. . ..even though it appears in the textbook. Well, we know that the fact
that it is written in the textbook does not yet mean that it is present in the classroom [L7].

4. Summary and discussion

This study illuminates the challenges involved in large scale implementation of HOT in science education as they are
viewed by a group of leaders who were engaged in implementation processes in an intensive way. However, since this
paper focuses on challenges, it may give a misrepresentative image. At the outset of the discussion section it is important
to emphasize which conclusions should NOT be drawn from this study. Focusing on the challenges does not mean that the
processes we are studying were not effective or that large scale implementation efforts in this area are doomed to failure.
 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249 245

The analysis presented here is NOT an evaluation study of the various attempts to teach HOT in science education in Israel.
Although a comprehensive evaluation study of these efforts had not yet taken place, studies and reports examining sections of
these efforts show considerable effects and progress (see for example Avargil et al., this volume, Office of Pedagogical Affairs,
MOE, 2009; Spector-Levy et al., 2008; Gallagher et al., 2012). Despite these positive effects, however, the implementation
processes under consideration are far from being completed. Although considerable developments in the desired direction
have been taking place, it is clear that if the aim is for thinking-rich instruction to become a routine in all science lessons,
there is still a long way to go. The gradual nature of these processes therefore, should allow us to recognize progress but still
be concerned by the challenges hindering further improvements.
Indeed, the data indicate that the route to thinking rich instruction in all classrooms is neither short nor smooth. The
challenges described in this study stress the fact that introducing HOT to science classrooms does not entail “all or nothing”
processes because implementation processes of new pedagogies are often evolutionary rather than revolutionary (Dede,
2006; Cohen, 1988). Cohen (1988) explains that the answers provided by educators regarding the feasibility of sweeping
pedagogical changes are not satisfying. He argues that the question regarding the likelihood of pedagogical change and its
nature needs to be asked from a broad historical perspective, rather than from a limited view of one isolated link in the chain
of time. If we consider the magnitude of the pedagogical change required to make the transformation to a thinking-rich
curriculum, we must consider the possibility that we are only at the beginning of a long journey, and that learning from
implementation in the field will be slow. In Cohen’s opinion, it is likely that those who strive to promote instruction that he
calls “adventurous,” are in effect trying to bring about the beginning of a great, slow change in the perception of knowledge,
learning and instruction. This future change, however, is still in its infancy and needs many more years to materialize. Cohen
argues that the early stages of such huge system-wide changes are characterized by examining alternatives, inventing new
patterns of which only few will survive, and developing ideologies and strategies for change. Thus, complex pedagogical
changes that involve deep changes in teachers’ knowledge and beliefs regarding learning and instruction, are more correctly
viewed as long-term, slow, evolutionary processes, rather than revolutionary processes. According to this view, it is important
to study the challenges involved in such long-term change processes because understanding them may be informative for
improving future system-wide implementation efforts.
Part of the challenges encountered by the leaders who were interviewed for this study are common to many types of
educational change processes (see Table 1). In addition, this study revealed several meaningful challenges that are specific to
the educational effort of applying HOT according to the infusion approach across large numbers of classrooms (see Table 2):
challenges involved in weaving HOT into multiple, varied, specific science contents; challenges involved in planning a
reasonable and coherent developmental sequence of thinking goals; the fact that content goals tend to have priority over
thinking goals and therefore, to disperse of the latter in policy documents and in implementation processes; the challenges
involved in supporting teachers’ ability to teach metacognition; and finally, the huge challenges involved in developing
educators’ sound and deep professional knowledge in the area of teaching HOT. Although challenges related to PD are by no
means unique to the area of teaching HOT, the specific nature of these challenges in this area is deeply rooted in the unique
nature of teaching thinking. These challenges are aggravated by lack of adequate organizational support.
Several reasons make metacognition an exceptionally difficult topic to implement in system-wide PD workshops which
almost always suffer from a shortage of hours: the fact that metacognitive knowledge is itself quite complex, that chrono-
logically speaking it can only be addressed after participants had acquired a certain degree of knowledge concerning the
cognitive level of using thinking strategies, and the fact that metacognitive teaching requires particular teaching strategies.
Previous studies show that when a small-scale project addresses metacognitive teaching in a focused way, it can indeed be
effective (e.g. Zohar, 2006; Veenman, 2011). But when moving to a system-wide scale, the challenges involved in metacogni-
tive teaching (relative to the many other important goals that teachers’ workshops need to address during a limited period)
make it an especially difficult issue to address in an appropriate way.
More generally, the data indicate that the interviewees viewed the teaching of HOT according to the infusion approach
as requiring complex teachers’ knowledge. They believed it requires a high level of content knowledge, sound knowledge
of thinking strategies, knowledge of complex thinking processes such as problem-based and inquiry learning, knowledge
about the culture of thinking, knowledge about thinking dispositions, and acquaintance with a variety of pedagogical tools
that are specific to the teaching of HOT.
Due to the magnitude of the shift required in teachers’ knowledge while they make the transition to teaching thinking,
one of the interviewees stated that it merits the label of a “paradigm shift”. Addressing this paradigm shift in an appropriate
way requires long-term learning with highly trained instructors. Organizational constrains which are typical to large scale
implementation made it difficult to support the required long-term teachers’ learning and therefore, hindered the devel-
opment of teachers’ knowledge. These organizational constrains were most apparent in the number of hours devoted to
teachers’ learning and in the limited support for instructors which hindered the feasibility of creating a stable infrastructure
of highly professional instructors.
These findings have several practical and research-oriented implications. The challenges brought up in the interviews
point to the directions that future implementation efforts and future research should center on (in addition to the activities
already conducted in the implementation processes described earlier). First, several of the findings show gaps in experts’
knowledge, indicating that even the experts do not yet have a coherent systematic model and practical plan for weaving HOT
into the topics of the science curriculum over the span of the school years. Consequently, the leaders of the implementation
need to invest time as well as research and development funds in order to: (a) conceptualize theory-driven models for
246 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249

weaving HOT systematically into multiple, varied, specific science contents and, (b) for planning a reasonable and coherent
developmental sequence for teaching thinking goals across different ages and student populations. Subsequently, various
such models may be tried out and evaluated empirically so that decision making as to the optimal model of implementation
in various educational contexts and for different student population would be evidence-based. At the moment very little
research has been conducted around these issues.
Second, leaders and teachers have to learn how to view thinking goals in a way that would render them as important
and tangible as content goals.
Third, the findings have important implications concerning metacognition. Although ample research conducted in labo-
ratory or small-scale studies in schools show that metacognition has tremendous effects in improving students’ reasoning,
hardly any research has been conducted on the effect of metacognitive teaching across the whole school system. The difficul-
ties shown here concerning teachers’ knowledge and PD in the area of metacognition point to a dangerous potential hazard
in scaling up the teaching of metacognition. More research is needed in order to understand what elements of metacognitive
teaching may be more amenable for relatively short-term PD and thus less sensitive to the “dilution” (i.e., loss of focus and
meaning) which may take place in large scale implementation. Such research is crucial for a conceptualization of just what
components of metacognition can we hope to teach in a sound way across a large number of classrooms and just how to
conduct wide scale PD in the area of metacognition.
Finally, and perhaps most importantly, the findings have important implications for PD in the context of teaching HOT.
The mismatch between the scope of the required PD according to pedagogical considerations and the administrative and
organizational support for its execution seems to be a major impediment in the ability to apply policies in this area in a
successful way. Recommendations for overcoming this impediment center on increasing the weight of PD as a determining
factor in any implementation process in this area. The details of the recommendations depend upon the political level
supporting the relevant educational policy. If the educational policy advocating thinking-rich instruction is supported by
a stable (in terms of longevity), high-level political entity that sees education for thinking as a primary goal and has the
power to assign sufficient organizational and financial resources to the implementation process, then a systematic plan of
implementation can be based upon the pedagogical needs for a high quality and lengthy PD. In this case, pedagogy may drive
the planning of administrative and organizational structures and of an appropriate budget that can support the complex
pedagogical needs. Examples of such comprehensive and long term support for PD may be seen in the successful large scale
efforts of pedagogical reforms that took place in locations such as Finland or Ontario (Darling-Hammond, 2010; Fullan, 2007).
In addition, under such favorable conditions, rational and systematic models for taking innovations to scale are in place (Lee
& Krajcik, 2012)
In many cases however, conditions are less favorable and high level political support in policies advocating instruction of
HOT does not exist. In the case of the Israeli efforts to foster policies advocating thinking rich instruction in science education,
the thinking curriculum had been embraced by the highest levels within the Ministry of Education, but it had never been a
primary goal of a high level and stable (in terms of longevity) political entity. I suspect that efforts to implement the thinking
curricula in other countries often operate under similar, less than optimal, conditions. In such cases it is unlikely that the
optimal organizational and administrative conditions needed for profound PD will be met. Consequently, a gap between the
desired and actual teacher knowledge in the context of teaching thinking will be created and at least some of the pedagogical
goals of the thinking curriculum would have to be compromised.
Since this gap in teacher knowledge is likely to affect the implementation process of teaching HOT in a fundamental way,
it is important to recognize it and take it into consideration when planning large scale efforts in this area. Some possible
directions for addressing this problem include the following: (a) Along with the recommendations of numerous previous
reports (e.g., Darling-Hammond, 2010; Elmore, 2004; Fullan, 2007), this study calls to give PD priority over other goals;
In particular, the findings point to the need to improve the structural and organizational infra- structure necessary for
supporting deep and long term teacher learning; (b) An implementation plan cannot be deeper than the depth of the PD
it can support. When a realistic assessment of the organizational infra-structure for PD processes indicates that PD would
necessarily be limited, the thinking goals of the large scale implementation should be re-considered and adjusted. In such
cases, it may be worthwhile to consider more modest thinking goals for the whole enterprise which may be achievable in
a relatively sound way rather than to stick to more ambitious goals which may be in danger of superficial implementation.
For example, an evaluation study of a policy that advocated inquiry learning in elementary schools in Israel in the 1990s
revealed that inquiry learning had become quite common in elementary schools so that most schools exhibited some form
and scope of inquiry learning. Yet, that study also revealed that in most schools inquiry learning was limited to shallow,
surface elements of inquiry. The emphasis was on the product, i.e., on a written “paper” rather than on the process of inquiry
learning. In many cases the product was the result of “cut and paste” processes that students were involved in, often with
addition of nice graphics, rather than the result of the deep thinking processes that are fundamental to any deep inquiry
process (e.g., asking fruitful questions, planning solutions, integrating information from multiple sources, processing data,
drawing valid conclusions, considering evidence, formulating arguments, etc.). According to the study, the main reasons
for the shallowness of students’ inquiry learning were teachers’ limited conceptions and understanding of inquiry and
limited pedagogical knowledge of how to lead deep inquiry processes in the classroom (Gordon, Levin, Keinan, & Bar-On,
2003). When approximately 10 years later a new thinking curriculum was considered for elementary schools (as part of
the “Pedagogical Horizons” policy, see Gallagher et al., 2012), the findings of that study were taken into account. Although
theoretical considerations gave top priority to the goal of implementing inquiry learning, a practical assessment of the
 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249 247

available resources and support led to the conclusion that the deep and long term PD that was judged to be crucial for sound
implementation of inquiry learning among elementary school children will not be feasible. This practical consideration led
to the formulation of a much more modest thinking curriculum that centered on three thinking strategies (posing questions,
making comparisons and argumentation) rather than on a full inquiry process.
In terms of research, the gaps that often exist between the pedagogical needs of PD in this area and the available resources
for such PD open the gate to a new set of questions: if only a limited number of hours of PD devoted to HOT are available
each year, what are the most important things to work on? What elements of teachers’ knowledge can we hope to teach
meaningfully with limited resources for PD? Which elements of teacher knowledge that may seem significant based on
findings from small scale projects, are the most resistant to the “diluting” phenomena inherent to scaling up and therefore,
should be recommended for wide implementations? Future studies that will address such questions may enable evidence-
based decision making when planning large scale PD processes with less than optimal conditions.
In summary, these ideas suggest an alternative strategic direction for taking teaching thinking to scale. This paper empha-
sizes the magnitude of the challenges involved in scaling up the thinking curriculum. Obviously, whenever there is a sound
and strong support for large scale implementation in this area, ambitious and revolutionary goals can and should be aspired
for. However, whenever there is only a limited amount of support in a policy aspiring to take HOT to scale, the magnitude
of the challenges involved suggest that it may make more sense to lower expectations by aspiring for more modest goals
rather than expecting to replicate the effects of small-scale successful projects. This approach suggests that the huge goal of
implementing teaching thinking across a whole system may need to be implemented in small steps, according to an evo-
lutionary rather than a revolutionary model. It means that achieving our far-reaching, ultimate goal will take time (Cohen,
1988) and will involve a succession of interim sub-goals. This “small-steps”, evolutionary model for scaling up the thinking
curriculum opens the door for many new empirical and theoretical questions that may be investigated in future studies.
Finally, a note regarding the main limitation of this study is in place. Since the study was conducted in one specific
educational context with only seven interviewees, generalizations based on its findings are limited. In particular, it should
be remembered that the study was conducted in an educational system which is quite centralized. The findings therefore, may
be less relevant to decentralized educational systems where wide-scale implementation processes of the thinking curriculum
may take on considerably different routes. Future studies may thus also ask what are the similarities and differences between
wide scale implementation of HOT in centralized and less centralized educational systems.

Appendix A. Interview protocol

1. Can you portray an updated picture of your subject in terms of teaching for thinking and understanding? What exists
on the level of policy (ask for reference to relevant policy/curricular documents)? What actually takes place on the level
of schools and classrooms practice?
2. What implementation means were operated in the school system in order to turn the policy into practice? [if relevant. . .
add:] What strategies do you employ in order to implement the policy?
3. Do you think that in your subject the current state of teaching thinking is the desired state? If not what do you think is
missing?
4. Please try to observe the last 10–15 years retrospectively: How did we reach the current state of teaching thinking?
Please tell me about important events over the years that influenced teaching thinking in . . . [name of relevant school
science subject.]
5. [If did not mention the Pedagogical Horizon policy intuitively): Did the Pedagogical Horizon policy affect teaching
thinking in [name of relevant school science subject]? [If yes. . .] What was the effect? [If no. . .] Why do you think there
was not effect?
6. What do you see as the main obstacles/barriers in implementing teaching thinking across the various school subjects
and content domains?
7. What are you predictions regarding the future developments of teaching for thinking and understanding in [name of
relevant school science subject]?
8. Are there any evaluation studies of implementing teaching thinking in [name of relevant school science subject]? If so,
can you please tell me about them? Can you please send me the full study/reference?
9. Did academics (i.e., university or college professors) contribute to the implementation process? If so, what was the
contribution?
10. What are the challenges you see in integrating higher order thinking in science content in an authentic way? Please
bring example(s).

References

Abd-El-Khalick, F., & Akerson, V. (2009). The influence of metacognitive training on pre-service elementary teachers’ conceptions of nature of science.
International Journal of Science Education, 3(16), 2161–2184.
Abrami, P. C., Bernard, R. M., Borokhovski, A. W., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills
and dispositions: a stage 1 meta-analysis. Review of Educational Research, 78, 1102–1134.
Adams, P. E., & Krockover, G. H. (1997). Beginning science teacher cognition and its origins in the preservice secondary science teacher program. Journal of
Research in Science Teaching, 34, 633–665.
248 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249

Adey, P. S., & Shayer, M. (1993). An exploration of long-term far-transfer effects following an extended intervention program in the high school science
curriculum. Cognition and Instruction, 11(1), 1–29.
Adey, P. S., & Shayer, M. J. (1994). Really raising standards. London: Routledge.
Alexander, P. A., Dinsmore, D. L., Fox, E., Grossnickle, E. M., Loughlin, S. M., Maggioni, L., Parkinson, M. M., & Winters, F. (2011). Higher order thinking and
knowledge. In G. Schraw, & D. R. Robinson (Eds.), Assessment of higher order thinking skills (pp. 47–88). Charlotte, NC: Information Age Publisher.
Barber, M., & Mourshed, M. (2007). How the world’s best-performing school systems came out on top. London: Mckinsey and Company.
BSCS (Biological Science Curriculum Study). (1963). Biological science: inquiry into life. New York: Harcot Brace Publishers.
Bransky, J., Hadass, R., & Lubezky, A. (1992). Reasoning fallacies in preservice elementary school teachers. Research in Science & Technological Education,
10(1), 83–92.
Blumenfeld, P., Fishman, B. J., Krajcik, J., Marx, R. W., & Soloway, E. (2000). Creating usable innovations in systemic reform: scaling up technology-embedded
project-based science in urban schools. Educational Psychologist, 35, 149–164.
Brownell, G., Jadallah, E., & Brownell, N. (1993). Formal reasoning ability in preservice elementary education students: matched to the technology education
task at hand? Journal of Research on Computing in Education, 25(4), 439–446.
Carpenter, T.P., Lynn-Blanton, M., Cobb, P., Loef-Frank, M., Kaput, J., McClain, K. (2004). Scaling up innovative practices in mathematics and science. Research
report. NCISLA (National Center for Improving Learning and Achievement in Mathematics and Science), Wisconsin Center for Education Research,
School of Education, University of Wisconsin—Madison.
Chen, Z., & Klahr, D. (1999). All other things being equal: children’s acquisition of the control of variables strategy. Child Development, 70, 1098–1120.
Cocharn, K. F., & Jones, A. L. (1998). The subject matter knowledge of preservice science teachers. In B. Fraser, & K. Tobin (Eds.), International handbook of
science education (pp. 707–718). Dordrecht, The Netherlands: Kluwer.
Cohen, D. K. (1988). Teaching practice: plus que ca change. In P. Jackson (Ed.), Contributing to educational change: perspectives on research and practice (pp.
27–84). Berkeley, CA: McCutchan.
Darling-Hammond, L. (2010). The flat world and education: how America’s commitment to equity will determine our future. New York and London: Teachers’
College Press.
Dede, C., Honan, J. P., & Peters, L. C. (2005). Lessons learned from technology-based educational innovation. San Francisco: Jossey-Bass.
Dede, C. (2006). Evolving innovations beyond ideal settings to challenging contexts of practice. In K. Sawyer (Ed.), The Cambridge handbook of the learning
sciences (pp. 551–566). New York: Cambridge University Press.
Dwyer, C. P., Hogan, M. J., & Stewart, I. (2012). An evaluation of argument mapping as a method of enhancing critical thinking performance in e-learning
environments. Metacognition Learning, 7, 219–244.
Elmore, R. C. (2004). School reform from the inside out: policy, practice, and performance. Cambridge, MA: Harvard Educational Press.
Ennis, R. H. (1989). Critical thinking and subject specificity: clarification and needed research. Educational Researcher, 18(3), 4–10.
Eylon B., & Bagno E. (1997) Professional development of physics teachers through long-term in-service programs: the Israeli experience. In: American
Institute of Physics (AIP) conference proceedings (vol. 399, pp. 299–326).
Fischer, K. W., & Bidell, T. R. (2006). Dynamic development of action and thought. In R. M. Lerner (Ed.), Handbook of child psychology (6th ed., pp. 313–399).
Hoboken, NJ: John Willey & Sons, ch. 7.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: a new area of cognitive—development inquiry. American Psychologist, 34, 906–911.
Flavell, J. H., Miller, P. H., & Miller, S. A. (2002). Cognitive development (4th ed.). Upper Saddle River, NJ: Prentice Hall.
Fullan, M. (2007). The new meaning of educational change (4th ed.). NY and London: Teachers College Press.
Gallagher, C., Hipkins, R., & Zohar, A. (2012). Positioning thinking within national curriculum and assessment systems: perspectives from Israel New Zealand
and Northern Ireland. Thinking Skills and Creativity, 7, 134–143.
Gess-Newsome, J., & Lederman, N. G. (Eds.). (1999). Examining pedagogical content knowledge. Dordecht, Netherlands: Kluwer.
Gordon, D., Levin, M. R., Keinan, A., & Bar-On, N. (2003). Inquiry learning in elementary schools—a research report. Beer Sheva: Ben Gurion University and Kay
Teachers College.
Jungwirth, E. (1987). Avoidance of logical fallacies: a neglected aspect of science education and science-teacher education. Research in Science and Techno-
logical Education, 5, 43–58.
Jungwirth, E. (1990). Science teachers’ sponatanous, latent or non-attentace to the validity of conclusions in reported situations. Research in Science and
Technological Education, 8, 103–115.
Jungwirth, E. (1994). Science-teachers as uncritical consumers of invalid conclusions: incompetence or just poor performance? Paper presented at the 1994 NARST
annual conference. Anaheim, CA.
Kramarski, B., & Michalsky, T. (2009). Preparing preservice teachers for self-regulated learning in the context of technological pedagogical content knowl-
edge. Learning and Instruction, 20, 434–447.
Kennedy, M. (1990). Trends and issues in: Teachers’ subject matter knowledge. ERIC clearinghouse on Teacher Education, Washington DC (ERIC ED 322 100).
Leat, D., & Lin, M. (2007). Developing pedagogy of metacognition and transfer: some signposts for the generation and use of knowledge and the creation of
research partnerships. British Educational Research Journal, 29(3), 383–414.
Lee, O., & Krajcik, J. (2012). Large-scale interventions in science education for diverse student groups in varied educational settings. Journal of Research in
Science Teaching, 49(3), 271–280.
Levin, B. (2008). How to change 5000 schools: a practical and positive approach for leading change at every level. Cambridge, MA: Harvard Education Press.
Levin, B., & Fullan, M. (2008). Learning about system renewal. Educational Management Administration and Leadership, 36, 289–303.
Lin, X., & Lehman, J. D. (1999). Supporting learning of variable control in a computer-based biology environment: effects of prompting college students
reflect on their own thinking. Journal of Research in Science Teaching, 36(7), 837–858.
Loef-Frank, M., Carpenter, T., Fennema, E., Ansel, E., & Behrend, J. (1998). Understanding teachers ‘self-sustaning’ generative change in the context of
professional development. Teaching and Teacher Education, 14(1), 67–80.
Loughran, J., Gunstone, R., Berry, A., Milroy, P., Mulhall. (2000). Documenting science teachers’ pedagogical content knowledge through PaP-eRs. Paper presented
at the annual meeting of the American Educational Research Association (AERA). New Orleans, April.
Ministry of Education and Culture. (1992). Report of the Supreme Committee for Science and Technology Education “Tomorrow 98”. Jerusalem: Ministry of
Education and Culture.
Office of Pedagogical Affairs, Israel Ministry of Education (MOE). (2007). “Pedagogical Horizon” for learning. (in Hebrew). Retrieved from:
http://cms.education.gov.il/EducationCMS/Units/Mazkirut Pedagogit/OfekPedagogi/HinuchLechashiva/Hinoch Le Chashiva.htm (in Hebrew).
Office of Pedagogical Affairs, Israel Ministry of Education (MOE). (2009). Teaching thinking report (“Pedagogical Horizon”): 2006–2009(in Hebrew). Israel
Ministry of Education. Jerusalem.
Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argument in school science. Journal of Research in Science Teaching, 41, 994–1020.
Partnership for 21st Century Skills. (2011). Framework for 21st century learning, http://www.p21.org/overview/skills-framework, retrieved July 2012.
Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). London and New Delhi: Sage Publications.
Paul, R.W., Elder, L., Bartel, T. (1997). Teachers of teachers: examining preparation for critical thinking. Paper presented at the annual meeting of the American
Educational Research Association. Chicago: March 14–18.
Pellegrino, J. W., & Hilton, M. L. (Eds.). (2012). Education for life and work: developing transferable knowledge and skills in the 21st century. Committee on defining
deeper learning and 21st century skills. Washington DC: Center for Education; Division on Behavioral and Social Sciences and Education; National Research
Council, National Academic Press.
Perkins, D. N., & Salomon, G. (1989). Are cognitive skills context-bound? Educational Researcher, 18, 16–25.
Perkins, D. N., Jay, E., & Tishman, S. (1993). Beyond abilities: a dispositional theory of thinking. Merrill-Palmer Quarterly, 39(1), 1–21.
 A. Zohar / Thinking Skills and Creativity 10 (2013) 233–249 249

Resnick, L. (1987). Education and learning to think. Washington D.C: National Academy Press.
Resnick, L. (2010). Nested learning systems for the thinking curriculum. Educational Researcher, 39, 183–197.
Ross, J. A. (1988). Controlling variables: a meta-analysis of studies. Review of Educational Research, 58(4), 405–437.
Schoenfeld, A. (1992). Learning to think mathematically. In D. A. Grouws (Ed.), Handbook of Research in Mathematics Teaching and Learning. New York:
Macmillan.
Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26, 113–125.
Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational Psychology Review, 7(4), 351–371.
Schraw, G., McCrudden, M. T., Lehman, S., & Hoffman, B. (2011). An overview of thinking skills. In G. Schraw, & D. R. Robinson (Eds.), Assessment of higher
order thinking skills (pp. 1–18). Charlotte, NC: Information Age Publisher.
Shulman, L. S. (1986). Those who understand: knowledge growth in teaching. Educational Researcher, 15, 4–14.
Shulman, L. S. (1987). Knowledge and teaching: foundation of the new reform. Harvard Educational Review, 57(1), 1–22.
Spector-Levy, O., Eylon, B., & Schertz, Z. (2008). Teaching communication skills in science: tracing teacher change. Teaching and Teacher Education, 24,
462–477.
Swartz, R. J., Costa, A. L., Beyer, B. K., Reagan, R., & Kallick, B. (2008). Thinking-based learning. New York and London: Teachers College Press.
Tamir, P. (2006). Inquiry in science education and its reflection in the Israeli biology teaching. In A. Zohar (Ed.), Learning by inquiry: an ongoing challenge
(pp. 15–56). Jerusalem: Magnes (in Hebrew)
Toth, E. E., Klahr, D., & Chen, Z. (2000). Bridging research and practice: A cognitively based classroom intervention for teaching experimentation skills to
elementary school children. Cognition and instruction, 18, 423–459.
Van Driel, J. H., Verloop, N., & de Voss, W. (1998). Developing science teachers’ pedagogical content knowledge. Journal of Research in Science Teaching, 35,
673–695.
Veenman, M. V. J., Van Hout-Wolters, B., & Afflerbach, P. (2006). Metacognition and learning: conceptual and methodological considerations. Metacognition
and Learning, 1, 3–14.
Veenman, M. V. J. (2011). Learning to self-monitor and self-regulate. In R. E. Mayer, & P. A. Alexander (Eds.), Handbook of research on learning and instruction
(pp. 197–218). New York and London: Routledge, ch. 10.
White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling and metacognition: making science accessible to all students. Cognition and Instruction, 16(1),
3–118.
White, B. Y., & Frederiksen, J. R. (2000). Metacognitive facilitation: An approach to making scientific inquiry accessible to all. In J. L. Minstrell, & E. H. Van-Zee
(Eds.), Inquiry into Inquiry learning and Teaching in Science (pp. 331–370). Washington D.C: American Association for the Advancement of Science.
Whitebread, D., Coltman, P., Pasternak, D., Sangster, C., Grau, V., Bingham, S., & Demetriou, D. (2009). The development of two observational tools for
assessing metacognition and self-regulated learning in young children. Metacognition and Learning, 4(1), 63–85.
Wilson, N. S., & Bai, H. (2009). The relationships and impact of teachers’ metacognitive knowledge and pedagogical understandings of metacognition.
Metacognition & Learning, 5(3), p269–288.
Wilson, S., Shulman, L., & Richert, A. (1987). 150 Different Ways of Knowing: Representations of Knowledge in Teaching. In J. Calderhead (Ed.), Exploring
Teacher Thinking (pp. 104–124). London: Cassell.
Zeidler, D. (2002). Dancing with maggots and saints: visions for subject-matter knowledge, pedagogical knowledge and pedagogical content knowledge in
science teacher education reform. Journal of Science Teacher Education, 13, 27–42.
Zembal-Saul, C., Munford, D., Crawford, B., Friedrichsen, P., & Land, S. (2002). Scaffolding preservice science teachers’ evidence-based arguments during an
investigation of natural selection. Research in Science Education, 32, 437–463.
Zohar, A. (1999). Teachers’ metacognitive knowledge and instruction of higher order thinking. Teaching and Teachers’ Education, 15, 413–429.
Zohar, A. (2004). Higher order thinking in science classrooms: Students’ learning and teacher’ professional development. The Netherlands: Kluwer Academic
Press.
Zohar, A. (2006). The nature and development of teachers’ meta-strategic knowledge in the context of teaching higher order thinking. The Journal of the
Learning Sciences, 15, 331–377.
Zohar, A. (2008). Science teacher education and professional development in argumentation. In S. Erduran, Jiménez-Aleixandre, & María Pilar (Eds.),
Argumentation in science education: Perspectives from classroom-based research (pp. 245–268). Springer. Chapter 12.
Zohar, A., & Schwartzer, N. (2005). Assessing teachers’ pedagogical knowledge regarding issues pertaining to instruction of higher order thinking. Interna-
tional Journal of Science Education, 27, 1595–1620.
Zohar, A., & Peled, B. (2008). The effects of explicit teaching of metastrategic knowledge on low- and high-achieving students. Learning and Instruction, 18,
337–353.
Zohar, A., & Ben David, A. (2008). Explicit teaching of meta-strategic knowledge in authentic classroom situations. Metacognition and Learning, 3, 59–82.