Studies in Higher Education

ISSN: 0307-5079 (Print) 1470-174X (Online) Journal homepage: http://www.tandfonline.com/loi/cshe20

Understanding learning difficulties—A predictive

research model

A.H. Johnstone & H. El-Banna

To cite this article: A.H. Johnstone & H. El-Banna (1989) Understanding learning
difficulties—A predictive research model, Studies in Higher Education, 14:2, 159-168, DOI:
10.1080/03075078912331377486

To link to this article: https://doi.org/10.1080/03075078912331377486

Published online: 05 Aug 2006.

Submit your article to this journal

Article views: 211

View related articles

Citing articles: 26 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=cshe20
Studies in Higher Education Volume 14, No. 2, 1989 159

Understanding Learning Difficulties

a predictive research model
A. H. JOHNSTONE & H. EL-BANNA
Science Education Research Unit, University of Glasgow

ABSXRACT This paper arises from investigations of the topics in science which students
perceive to be difficult. An information processing model is described which relates student
performance to the amount of information to be processed in a learning or problem-solving
situation. When the student working memory capacity is exceeded, there is a sharp drop in
performance, but some students (~10% ) continue to operate efficiently with problems which
exceed their capacity; they are probably employing chunking devices that enable them to reduce
the problem demand to less than their limit of capacity. Empirical measurements give support
to the model and suggest hypotheses for future research.

In the 1970s (Duncan & Johnstone, 1974) a detailed analysis was done of the content of
parts of school chemistry about which pupils and teachers had complained as being too
difficult. Part of the approach to the problem consisted of setting test questions on sub-
sections to try to isolate the trouble spot or spots. The results were surprising and, at the
time, inexplicable. A cluster of questions was found in which pupils had a high success rate
( > 0 . 6 ) and another cluster appeared in which there was a poor success rate ( < 0 . 3 ) but
there was a gap in between these clusters. There were no questions which were generating a
middling success rate between 0.3 + and 0.6 (Fig. 1). A broadly normal distribution might
have been expected, but in fact it did not emerge. Other workers prodding into other areas of
the syllabus which pupils and teachers perceived to be difficult came up with similar
performance distributions--a 'hole in the middle'.

10-

~6 5
6
Z

I I
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0,7 0.8 0.9 1.0

Facility

FIG. 1. Number of questions in each band of facility (facility is the proportion of the sample getting a question
correct).
160 A. H. Johnstone & H. El-Banna

It was not until 1980 that a tentative working hypothesis was put forward in terms of
information processing theory (Johnstone & Kellett, 1980).
In our most recent study we re-examined this earlier work and added to it about 100
questions from the Scottish Examination Board based upon these difficult topics (Johnstone
& E1-Banna, 1986). This time, however, we approached the problem in a different way. Each
question was examined by a jury for its demand (Z). By demand we mean the maximum
number of thought steps and processes which had to be activated by the least able, but
ultimately successful candidate in the light of what had been taught. The procedure was to
choose at random 25 scripts where the students had scored full marks in a given question
and to examine their routes. These were largely numerical questions which lent themselves
to a stepwise analysis in terms of what had been recalled, transformed, deduced and
concluded. Each question was treated in the same way with another random sample of 25
successful students. Jury members then compared the results of their analyses for each
question and agreement was obtained (if need be by majority) for the number of 'steps' in
the longest route for each question (the Appendix shows a typical analysis).
This was a purely operational method for assessing demand because the maximum
demand on working space is likely to take place before any sequencing is achieved. In fact
our method may be an underestimate of demand. It is certainly no stronger than an estimate,
but it does enable us to arrive at a relative indication of demand. This question analysis was
carried out by researchers who had no hand in setting or marking the questions.
The facility values were then plotted against 'demand', in the expectation of obtaining a
negative correlation. Such a correlation was obtained, but not in the form expected (Fig. 2)
(this figure has been simplified by choosing every fourth question from the sample for
plotting).
We obtained an S-shaped curve indicating a high and a low plateau with a rapid drop
between them. Here again was the 'hole in the middle' which we had seen long before but
could not interpret. The shape of the curve was reminiscent of curves describing phenomena
in which something is taken to a limit beyond which there is a rapid change. The vertical
part of the curve fell on the demand axis at between 5 and 6, setting us off to reread Miller's
work of the 1950s (Miller, 1956). Perhaps the curve was showing an overload in processing
(working memory) capacity and that the areas of difficulty were those where this breakdown
was likely to occur either because of the nature of the science or because of the way in which
it was being taught and learned or some blend of these.
Since these results were obtained from a composite group of some 20,000 candidates, it
was clear that, if the 'capacity' of individuals varied (as shown by Miller), some of the
curve's peculiarities could be explained. The upper plateau must consist of questions which
were within the working memory capacity of all, but carelessness, forgetfulness or lack of
interest could depress the achievement. The points on the steep part of the curve could
indicate questions which were now proving too much for some, but were still within the
compass of the rest. The lower plateau represented the questions which had now gone
beyond 90% of the sample. The remaining successful 10% had something to teach us.
To test some of these propositions our attention turned to a sample of some 350 tertiary
students in their first year of a science degree who were reading chemistry. We employed
two tests, not of short-term memory, but of the capacity to hold pieces of information and
operate upon them. This was an attempt to quantify the shared holding/thinking space which
an individual might possess (X). The definition of 'piece of information' is a very woolly one
especially if a subiect is operating in a familiar situation. He can bring into play all sorts of
chunking devices by which many pieces of information can be handled as if they were one. A
10-digit telephone number would be 10 pieces of information for one person but many fewer
 Understanding Learning Difficulties 161

1.0
\
o\ • •
0.9 ~ • \ • •

\.
0,8
\
\
0.7
\
\
\
\
0.6

\
> \
ii 0.5 \
\
\
0.4 \
• \

0.3

0.2

0.1
\
\
\
0 I I I i I I I I J [ L \1 l
1 2 3 4 5 6 7 8 9 10 11 12 13

No. of thought steps

FIG. 2. A plot of facility value in objectivechemistryquestions versus the number of thought steps needed to solve
the questions.

for another who recognised the system and used it. 0413398855 is broken in the directory as
041-339-8855. 041 is the 'chunk' for Glasgow; 339 is the district and 8855 is the subscriber's
number.
T o avoid such chunking, tests had to be used which were in areas unfamiliar to the
students. T h e tests were 'digit span backwards' (in which a sequence of numbers had to be
memorised and then turned over and given back in reverse order) and the Figural
Intersection Test (Pascual-Leone, 1974). The agreement between the tests is shown in
Table I). For the purposes of the set of experiments described below we confined ourselves
to the subset with identical scores, but in subsequent work we have taken in those in the +_ 1
category.

TABLEI. Agreementbetween tests

Performance on
two tests No.

Identical score 271 (74.5%)

Difference -+1 40
Difference > -+1 23
Others 30
Total 364
162 A. H. Johnstone & H. El-Banna

The sample was distributed, by 'capacity', as shown in Table II. These students were then
followed for a year and their performance on three university examinations was recorded.
These examinations were set, and marked independently of the researchers. The 'demand' of
each question was determined by a jury in consultation with the setters.

TABLE II. Distribution

according to capacity

Capacity (X) No.

5 98
6 110
7 63
Total 27t

The model on which our hypotheses were raised was as follows. A necessary (but not
sufficien 0 condition for a student to be successful in a question is that the demand of the
question should not exceed the working memory capacity of the student. If the capacity is
exceeded, the student's performance should fall unless he has some strategy which enables him
to structure the question and to bring it within his capacity.
An ideatised set of curves would look like Fig. 3. The actual experimental results are
shown for one examination in Fig. 4. I f we take 50% as a marker, it can be seen that the
capacity 5 students drop from a facility of 0.6 to less than 0.4 as the demand rises from 5 to
6. The capacity 6 students fall from 0.6 to 0.2 as the demand rises from 6 to 7 and the
capacity 7 students fall from 0.7 to 0.3 on moving demand from 7 to 8. There is a clear
similarity between Figs. 3 and 4.

1.0

X=5 " i X=6

~
'~"-'- X= 7

>
u_ 0.5

0 I
1
I
2
I,
3 4
I It
5 6
'

8
i
f~
9
I

FIG. 3. Expected performance in chemistry of students of differing X space in questions of different Z demand.

In another of the examinations there were no questions of demand 6. We therefore

predicted that the students of capacity 5 and 6 would perform similarly since there was no
 Understanding Learning Difficulties 163

1.0

0.9

0.8
o
0.7

0.6
k
>ii 0.5

0,4
\k~\x \
0.3 \\\ Ok.
×\\\\ \. \
0,2

\\.. X "\
0.1

__.% I I J I I I I I
3 4 5 6 7 8 9 10
Z
FIG. 4. C o m p a r i s o n of the average FV f o r all g r o u p s of different X space. (C)-.-C)) X=7; x--- x) X=6;
(O--O) X=5.

question of demand 6 which would allow the capacity 6 students t o show their superiority
over the capacity 5 group. The results for this exam are shown in Fig. 5.
In Figs. 4 and 5 a 'nesting' effect is seen in that the 7 curve is above the 6 curve and it
is above the 5 curve at all points. Even when a question is within the capacity of all the
students, the capacity 7 students on average outperform those of lower capacity. There is
also a very heartening feature in that when their capacity is exceeded, about 10% of students
can continue to function successfully. In other words, strategies can be learnt which enable a
learner to overcome capacity limitations. One might describe a person who is successful in
any discipline as one who has developed a good set of strategies in that subject and who has
learned (consciously or otherwise) to operate well outside his capacity. This also raises the
question about how to improve the performance of the other 90%. Can strategies be taught?
This is the subject of another of our studies not reported in this paper.
If what we have reported so far has any substance, we should be able to detect it in
gross phenomena such as all-over pass rates. If a question paper has questions of different
demands, only those questions with a demand less than 6 will be accessible to those of
capacity 5. Those of higher capacities will have access to more of the paper. This should be
reflected in the results (Table III). Those students who have done well in the term exams
are exempt from the finals. Those who are not exempt in the finals have two opportunities to
sit the final exams (June and September). The relationship between capacity and success is
very clear.
164 A. H. Johnstone & H. El-Banna

1.0-

0.9

0.8

0.7

\
>
0.6 \ \
LL
\ \'~
0.5
~\\ ~

0.4
\ \
~\\ ~

0.,~ ~\\ \~
\ ",,
~\\\ "\.
0.2

0.1

O ~ I I I I I I I
3 4 5 6 7 8 9
Z

FIG. 5. Comparison of the results from university students of different X space on questions of different Z demand
(FV). ( O - - - O ) X=7; ( x - - - x ) X=6; ( O - - O ) X=5.

TABLEIII. Results of the sample according to capacity (%)

Capacity Exempt Pass June Pass Sept. Fail

5 11.2 41.8 17.4 29.6

6 30.9 52.7 7.3 9.1
7 63.5 31.7 4.8 0.0

Since the second and succeeding years are made up of those who are successful in the
first year, the composition of each successive year must become richer in capacity 7 and
poorer in capacity 5 students. This is, in fact, borne out experimentally (Table IV).

TABLE IV. Results for subsequent years (%)

Capacity 1st year 2nd year 3rd year

5 36 27 18
6 41 39 40
7 23 35 42
 Understanding Learning Difficulties 165

Discussion
All that has been reported in this paper has been subjected to a series of replication studies
with very similar results. Composite graphs generated from studies on nine chemistry
examinations are shown in Figs. 6, 7 and 8. Each point through which the graphs are drawn
is the average facility score for all questions of the same demand. The range of the facility
scores is shown by the range bars through each point.

1.0

0.9

0,8

0.7

0,6

>
LL 0.5

0,4

0.3

0.2

0.1

-0
o -~, I I I I I I J I
3 4 5 6 7 8 9 10
Z

FIG. 6. Overall results from X = 5 university students (two years of testing).

The length of these range bars indicates to some extent the uncertainty in the estimate
of the demand of each question. Where no range bars occur, there is either only one question
of that demand or the questions of the same demand had the same facility. The general S-
shaped patterns are maintained.
It would be unwise to suggest that all problems of learning and testing can be explained
by this simple model, but it does suggest a mechanism for some of the problems which do
exist and a mechanism by which they might be overcome. Our colleagues are now exploring
the insights which the model gives for teaching and learning of all kinds. So far very
promising results are emerging in the areas of testing, laboratory work, computer displays,
worksheets, textbooks and university tutorials. The teacher can control the amount of useful
information (the signal) which the learner has to process and can also limit, or even exclude,
the extraneous, distracting information (the noise) in a learning situation. During the early
learning period the 'signal' to 'noise' ratio can be carefully controlled and then the 'noise' can
be increased to give students experience of distinguishing one from the other. Students can
166 A. H. Johnstone & H. El-Banna

0.9

0.8

0.7

0.6

>
u. 0.5

0.4

0,3

0.2

0,1

J~ I t I L I 1 ] I
3 4 5 6 7 8 9 t0
z

FIG. 7. Overall results from X = 6 university students (two years of testing).

0.9,

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

o ---"v I I .,[ ~ I I I I
3 4 5 6 7 8 9 10
Z

FIG. 8. Overall results from X = 7 university students (two years of testing).

 Understanding Learning Difficulties 167

be taught chunking devices by which they can chop problems down to size to fit their
capacity.
Often teachers do not 'hear' the 'noise' because they have long since got used to it, but
for first-time learners, the problem is acute. It may be that the strategies devised by the 10%
of students who perform outside their capacity can be transmitted to their peers. Our adult
strategies may be inappropriate to a novice.
This model has predictive power and should enable us to raise and test important
hypotheses in the future in many subject areas.

Correspondence: Dr A. H. Johnstone, Reader in Chemical Education, Department of

Chemistry, University of Glasgow, Glasgow G12 8QQ, United Kingdom.

REFERENCES
DUNCAN,I.M. & JOHNSTONE,A.H. (1974) Chemistry Check-up (London, Heinemann).
JOHNSTONE,A.H. & EL-BANNA,H. (1986) Capacities, demands and processes--a predictive model for science
education, Education in Chemistry, 23, pp. 80-84.
JOHNSTONE, A.H. & KELLETT, N.C. (1980) Learning difficulties in school science--towards a working hypothesis,
European Journal of Science Education, 2, pp. 175-181.
MILLER, G.A. (1956) The magical number seven, plus or minus two: some limits on our capacity for processing
information, PsychologicalReview, 63, pp. 81-97.
PASCtJAL-LEONE,J. (1974) Manualfor FiguralIntersection Test (York University,Canada).

Appendix 1
Demand can be thought of as being generated by
(i) the information in the question which has to be processed;
(ii) the information which has to be recalled to add to the given information; and
(iii) the processes which have to be activated to deal with the information; processes such
as deduction, transformation and calculation.

Example
'What volume of molar hydrochloric acid would be exactly neutralised by ten grams of
chalk?'
Demand:

1. chalk---calcium carbonate (recall)

2. calcium carbonate =-CaCO 3 (recall or deduce)
3. Formula weight of CaCO3= 100 g (calculate)
4. When it reacts with hydrochloric acid, what are the products? (recall)
5. Write a balanced equation (transformation)
6. Recognise that 1 mole C a C O 3 ~ 2 moles HC1 (deduce)
7. -= 2 litres of molar HC1 (recall)
8. 10 g CaCO3 ~ 1/10 mole ~ 1/5 mole HC1 (deduce)
9. ~ 1/5 litre molar HC1 (recall)
= 200 ml molar HC1
168 A. H. Johnstone & H. El-Banna

This was the longest successful route taken by the students and so was given a demand
number of 9.
Experienced teachers were able to do this in four steps by a series of chunking devices.
1. 10 g chalk~ 1/10 mole CaCO 3 (recall)
2. Chalk versus HC1 is a 1:2 ratio reaction (recaI1)
3. 1/10 CaCO3~ 1/5 mole HC1 (deduce)
4. ------200ml molar HC1 (deduce)
The demand must be related to the stage at which the respondents are in their learning. In
this case the demand was taken to be 9 for O level students and 4 for experienced teachers.