Available online at www.sciencedirect.

com

ScienceDirect
Procedia - Social and Behavioral Sciences 116 (2014) 3760 3764

5th World Conference on Educational Sciences - WCES 2013

Developing instruments to measure thinking skills and problem

solving skills among Malaysian primary school pupils
Tang Keow Ngang a *, Subadrah Nair b, Bouphan Prachak c
a

School of Educational Studies, Universiti Sains Malaysia, Penang and 11800, Malaysia
Department of Public Health Administration, Faculty of Public Health, Khon Kaen University, Khon Kaen and 40002, Thailand

Abstract
In the new era of information technology, thinking and problem solving skills are crucial for students to master world class
knowledge and contribute towards the building of a modern and progressive society. Thinking skills are given importance in the
Malaysian national school curriculum at the primary and secondary school levels. Thinking and problem solving skills are
incorporated in the learning outcomes to enable learners to solve simple problems, make decisions and express themselves
creatively. Studies related to developing instruments to measure thinking skills and problem solving skills among primary school
pupils are limited. This paper discusses the instrument developed by the researchers to measure pupils' thinking skills and
problem solving skills using de Bono's CoRT thinking tools (PMI, CAF, RULES, C&S, AGI, Planning, FIP, APC, Decision and
OPV). The validity and reliability of the instruments as well as the scale and criteria to rate pupils' responses are also discussed in
this paper.
2013
2013 The
The Authors
Authors.Published
Published by
by Elsevier Ltd. Open
accessreserved
under CC BY-NC-ND license.

All rights
Selection and/or
peer-review
of Academic
World
and Research Center.
Selection
and peer
review under
underresponsibility
the responsibility
of Prof.
Dr.Education
Servet Bayram
Keywords: Problem solving skills, thinking skills, primary school pupils, instrument;

1. Introduction
Thinking is the ability to attack a problem or task and solve it intelligently or rationally, thus provide a reasonable
explanation for the solution. Good thinkers have strong meta-cognitive skills. All human achievements and progress
are simply the products of thought. The evolution of culture, art, literature, science and technology are the results of
thinking. Thought and action are inseparable, all our deliberate actions starts from our deliberate thinking
(http://www.psychology4all.com/Thinking.htm.). The purpose of thinking, paradoxically, is to arrive at a state
where thinking is no more necessary at all. In other words, thinking starts with a problem and ends in a solution.
Thus, thinking is a tool for adapting ourselves to the physical and social environment in which we are in. Edward de
Bono says that thinking can be improved just like any skill because thinking according to him is a skill.
The benefits of developing thinking ability are manifold. By developing one's thinking skills one can make
achievements; can become successful; can shine in social life; can attain emotional, social and economic maturity
and so on. It has been found by Edward de Bono that when school students were taught to think creatively, their illtemper and aggressive tendencies reduced significantly. In this era of information technology, thinking and problem

* Corresponding Author: Tang Keow Ngang. Tel.: +6-017-5396315

E-mail address: tang@usm.my

1877-0428 2013 The Authors. Published by Elsevier Ltd. Open access under CC BY-NC-ND license.

Selection and/or peer-review under responsibility of Academic World Education and Research Center.
doi:10.1016/j.sbspro.2014.01.837

Tang Keow Ngang et al. / Procedia - Social and Behavioral Sciences 116 (2014) 3760 3764

3761

solving skills are very important for students to master world class knowledge and nation building. In addition, these
skills are also essential for the young generation in solving problems and facing challenges. It is undeniable that
schools play a major role in training students to develop their thinking skills (Mohd Dahlan, 1994).
Helping students become effective thinkers is increasingly recognized as a primary goal of education. Rapid
expansion of knowledge points to the importance of curriculum and instruction that empower students to locate and
process knowledge rather than simply memorize facts (Rajendran, 2002). The real question is whether Malaysian
education system as a whole has been receptive and make necessary changes to prepare students who are capable of
performing the various tasks such as interpret, analyze or manipulate information, critically think about information,
ideas and opinion, draw conclusion, inferences or generalizations, make predictions, propose solutions, create solve
life-like problems, judge ideas, express opinions and make choices and decisions.
2. Problem statement
The National Education Philosophy in Malaysia emphasizes ongoing individual development and balancing
intellectual, spiritual, emotional and physical traits. In line with Vision 2020, the educational system is not only
supposed to produce a generation that is competitive but also equipped to use advanced thinking skills and problem
solving skills to face any challenges and issues. The research on developing instruments to measure thinking skills
and problem solving skills (TSPSS) among primary school pupils is important and very limited. De Bono (1991)
promotes CoRT thinking tools to enhance pupils' thinking skills but he did not develop a test instrument to measure
TSPSS. Creative thinking test like TTCT (Torrance Test for Creative Thinking) is to measure general abilities of
creativity and their activities which are fixed. Hence activities in TTCT are appropriate for predicting creative
expression across all domains, especially in real-life problem solving task (Lee, 2004). Therefore, there is an
emergent need to develop instruments in this area.
3. Objectives of the study
The following are the objectives of the study:
To develop a test instrument and questionnaires to measure thinking skills and problem solving skills
among level two primary school pupils (Standard IV, V & VI) in Peninsula Malaysia using de Bono's
CoRT thinking tools (PMI, CAF, RULES, C&S, AGO, Planning, FIP, APC, Decision and OPV).
To obtain validity and reliability of the instruments (test instrument and questionnaires)
To obtain inter-rater reliability for the scale and the criteria for the test instrument.
4. De Bono's CoRT Thinking Tools
De Bono (1991), in his discussion has elaborated the methods of teaching thinking skills to students. According
to him, thinking skills can be taught using the following approaches:
'Osmosis' method - teachers imparting acquired thinking skills to students. According to de Bono, this
method is difficult to achieve.
Intellectual thinking - posing questions to students.
Logic regulations - this approach undermines students' perceptions.
Critical thinking - part of CoRT approach.
Simulation
Having discussion
Using CoRT thinking tools (Cognitive Research Trust)
De Bono's thinking tools in CoRT1 Breath is to broaden pupils' perceptions in various situations. There are ten
thinking tools to enhance pupils' thinking, mainly Plus, Minus and Interesting (PMI), Consider All Factors (CAF),

3762

Tang Keow Ngang et al. / Procedia - Social and Behavioral Sciences 116 (2014) 3760 3764

Rules, Consequences and Sequence (C&S), Aims Goals and Objectives (AGO), Planning, First Important Priority
(FIP), Alternatives Possibilities and Choices (APC), Decision and Other People's Views (OPV).
5. Literature review
According to Beyer (1991), thinking is a complex mental activity which requires certain skills and strategies.
According to him, there are several factors which make a person think and decide which problem need solving,
understanding, judging, retrieving accuracy, decision making and concept formation. He also emphasized that
thinking skills involved three important components, namely mental operation, certain knowledge and attitude.
However, according to Tillery (2006) problem solving skill is an effort to find solutions for difficult situations. One
needs advanced thinking skills in problem solving. Thinking skills help a person to choose the ideal strategy to solve
the problem that is being faced.
According to Ruggiero (1984) thinking is a mental activity that helps a person in problem solving, decision
making, understanding, seeking answers and finding meanings. Thinking skills is a personal trait that should be
given importance in our educational system. Clarke (1990) stressed that thinking skills should be infused in the
education system in teaching and learning of every subject. Beyer (1991), Clarke (1990), Swartz and Parks (1994)
and Meyers (1986) also suggested that thinking skills should not be taught separately but be incorporated in the
teaching and learning process in subjects taught in schools.
Problem solving skill is related to various types of intelligence as presented by Gardner (1983). According to
Campbell, Campbell and Dickinson (1996), various types of intelligence as presented by Gardner can be simplied
into three types of skills, namely problem solving skills in real situations, skills to generate ideas to solve new
prroblems and the ability to do or offer useful service in certain cultures. According to Ruscio and Amabile (1999),
creativity is exhibited when an individual solves problem in a way that is novel and appropriate. Guilford (1967)
stressed that creative intellectual ability is not enough to create novel ideas. He emphasizes on presenting ambigious
problems as the tasks in problem solving. Problem solving skills need to be given attention in our education system
thus producing intellectual citizens and enabling our country to reach the status of developed nation in the world.
Wu Wu-Tien (2004) stressed that emphasis on intellectual development among students is needed in order to get
them used to the existing realities, enabling them to prepare themselves for life in future societies and achieving
success.
6. Methodology
The researchers developed the instrument to measure thinking skills and problem solving skills among pupils
based on de Bono's CoRT thinking tools. Besides, the researchers also developed questionnaires for pupils, teachers
and parents to gauge pupils' thinking skills and problem solving skills. The test instrument and questionnaires were
given to six primary school teachers (two teachers for each Standard IV, V and VI) and they were required to rate
the test instrument and questionnaires according to the content validity, language and clarity. The teachers were
asked to give their response on the Likert's scale, on a range of one to ten, ranged from very low to very high. In
addition, researchers also gave the test instrument and questionnaires to two experts who are the lecturers in the
areas of thinking skills and psychology. This two experts were required to validate the content validity of the test
instrument and questionnaires in similar rating forms given to the teachers. A pilot study was carried out to obtain
the reliability of the test instrument and the questionnaires over a period of two weeks in each from urban and rural
primary schools. The samples of the pilot study comprised of 227 primary pupils (78, 74 and 75 Standard IV, V, VI
respectively), a total of 97 teachers and 124 parents.
There are two types instruments used, namely test instrument which consists of 12 items used to measure pupils'
thinking skills and problem solving skills and 23 items of five point Likert scale questionnaires used for pupils,
teachers and parents. The test instrument followed by questionnaire distributed to the pupils by researchers. The
parent questionnaire distributed through the pupils whereas teacher questionnaire distributed through their respective

Tang Keow Ngang et al. / Procedia - Social and Behavioral Sciences 116 (2014) 3760 3764

3763

headmasters. All test instruments marked by three primary school teachers who were trained by researchers based
on the criteria adapted from Torrance (1969) and Guilford (1967).
7. Findings
7.1. Validity of developed test instrument and questionnaires
Table 1 explains the score rated by the experts and teachers for the test instrument. Therefore, the finding confirm
the content validity, language and clarity is suitable for the study.
Table 1. Rating for test instrument
Groups

Content
Validity
9.5
10.0
9.0
9.0

Expert
Teacher VI
Teacher V
Teacher IV

Language & Clarity

9.5
9.5
8.5
7.5

7.2. Reliability of developed questionnaires

Table 2 shows clarifies the alpha value for questionnaires (pupils, teachers and parents) are high. In conclusion,
all the questionnaires obtained high reliability.
Table 2. Alpha value for questionnaire
Groups
Standard VI pupils
Standard V pupils
Standard IV pupils
Standard VI teachers
Standard V teachers
Standard IV teachers
Standard VI parents
Standard V parents
Standard IV parents

N
75
74
78
30
34
33
42
30
52

Alpha value
0.88
0.89
0.76
0.93
0.92
0.94
0.93
0.91
0.94

7.3. Inter-rater reliability among the markers

The three teachers who marked the test instruments (markers A, B & C) marked 40 scripts each. Finding revealed
that there is high positive correlation between marker A&B, A&C and B&C for originality, appropriateness and
fluency. Furthermore, Table 3 also explains that there is a high positive correlation between marker A&B, A&C and
B&C for marks awarded for novelty and appropriateness for problem solving tasks. Therefore the findings indicated
that there is high inter-rater reliability between the markers as such the scale and criteria developed by the
researchers is reliable to rate the responses of pupils in thinking skills and problem solving skills.
Table 3. Inter-rater reliability for originality, appropriateness and fluency
Content
Originality
Appropriateness
Fluency

Marker
A&B
A&C
B&C
A&B
A&C
B&C
A&B

r
0.988
0.993
0.982
0.962
0.925
0.931
0.739

p
0.000**
0.000**
0.000**
0.008**
0.000**
0.014**
0.000**

3764

Tang Keow Ngang et al. / Procedia - Social and Behavioral Sciences 116 (2014) 3760 3764

Novelty
(problem solving skills)
Appropriateness
(problem solving skills)

A&C
B&C
A&B
A&C
B&C
A&B
A&C
B&C

0.733
0.973
0.990
0.990
0.990
0.980
0.990
0.980

0.000**
0.006**
0.000**
0.000**
0.000**
0,000**
0.000**
0.000**

8. Discussion
The study produces an alternative instrument using de Bono's CoRT thinking tools in measuring levels of
thinking skills and problem solving skills among primary school pupils. The test instrument was validated by
experts and teachers for content validity, language and clarity. The findings also show that the scale and the criteria
used to measure the test instrument are highly reliable. The questionnaires for pupils, teachers and parents developed
by the researchers to gauge pupils thinking skills and problem solving skills have high internal reliability. Besides,
the instruments are also designed to suit the pupils' exposure and experience to the local culture.
Acknowledgements
This project was made possible with funding from Research Creativity and Management Office, Universit Sains
Malaysia, Malaysia.
References
Beyer, B.K. (1991). Teaching thinking skills: A hand book for secondary school teachers. Boston: Allyn and Bacon.
Campbell, L., Campbell, B. & Dickinson, D. (1996). Teaching and learning through multiple intelligence. London: Allyn and Bacon.
Clarke, J.H. (1990). Patterns of thinking. Boston: Allyn and Bacon.
De Bono, E. (1991). The Direct Teaching of Thinking in Education and the CoRT Method, dalam Maclure, S. (1991). Learning to Think Thinking
to Learn (pp. 3-14). Oxford: Pergamom Press.
Gardner, H. (1983). Frames of mind: Theory of multiple intelligences. New York: Basic Book Inc.
Guilford, J.P. (1967). The Nature of Human Intelligence. New York, NY: McGraw-Hill.
Lee, Y.J. (2004). Effects of Divergent Thinking / Instruction on Torrance Test of Creative Thinking and Creative Performance. Doctoral
Dissertation, University of Tennessee, Knoxville. Avaliable from ProQuest Dissetation and Theses Database. (UM13141813).
Meyers, C. (1986). Teaching students to think critically (pp. 1-10). San Franscisco: Jossey-Bass Publishers.
Mohd Dahlan, M. R. (1994). Infusing critical and creative thinking skills in teaching and learning Malay language KBSM. Paper presented in
International Conference - Innovation in Education - Effective Teaching and Learning, (17-19 October 1994), Universiti Sains Malaysia,
Penang, Malaysia.
Rajendran, N.S. (2002). Restructuring teacher education programs to teach higher-order thinking skills. Paper presented at the Universiti
Pendidikan Sultan Idris International Education Conference 2002, Kuala Lumpur, Malaysia.
Ruggiero, V.R. (1984). The art of thinking: A guide to critical and creative thought. New York: Harper & Row Publishers.
Ruscio, A., & Amabile, T. (1999). Effects of instructional style on problem-solving creativity. Creativity Research Journal, 12(4), 251-266.
Swartz, R.J. & Parks, S. (1994). Infusing critical and creative thinking into content instruction. Massachusetts: Critical Thinking Press &
Software.
Tillery, B.W. (2006). Problem solving techniques. Retrieved from http://www.dushkin.com/online/study/problemsolving,mhtml.
Torrance, P. (1969). Curiosity of Gifted and Performance on Timed and Untimed Tests of Creativity. ERIC Digest. Retrieved from ERIC
database (EdJ13667).
Wu Wu-Tien (2004). Multiple intelligence, educational reform, and a successful career. in Teacher College Record, 106(1), Jan 2004, 181-192.
Columbia: Columbia Teachers College University.