GIORGIO T.

BAGNI

SOME COGNITIVE DIFFICULTIES RELATED TO THE

REPRESENTATIONS OF TWO MAJOR CONCEPTS
OF SET THEORY

ABSTRACT. The main focus of this paper is on the study of students conceptual understanding of two major concepts of Set Theory the concepts of inclusion and belonging.
To do so, we analyze two experimental classroom episodes. Our analysis rests on the theoretical idea that, from an ontogenetic viewpoint, the cognitive activity of representation of
mathematical objects draws its meaning from different semiotic systems framed by their
own cultural context. Our results suggest that the successful accomplishment of knowledge
attainment seems to be linked to the students ability to suitably distinguish and coordinate
the meanings and symbols of the various semiotic systems (e.g. verbal, diagrammatic and
symbolic) that encompass their mathematical experience.
KEY WORDS: belonging, cognition, cultural context, EulerVenn diagrams, inclusion,
semiotic representations, set

1. INTRODUCTION
Mathematical objects do not exist as real, concrete objects. Their perception as well as their construction is always mediated by signs. This fact,
pointed out by Vygotsky in a more general context,1 leads us to consider the
so-called cognitive paradox of mathematical thought, formulated by Duval,
who underlined that although mathematical learning is conceptual, any activity involving mathematical objects takes place only through a semiotic
register of representations (Duval, 2000, p. 61; see also Otte, 2001, p. 33).2
Duval went on to suggest that the functioning of the mind is indivisibly
linked to the existence of several representation registers. It follows that,
in general, the importance of semiotic activity in knowledge acquisition is
fundamental and the study of the representations of mathematical objects
is a crucial issue for mathematics education.
It is in this context that we shall investigate, through two experimental
episodes, some of the central difficulties that students have in theoretically
distinguishing between two major concepts of Set Theory: the concepts
of inclusion and belonging. These concepts cannot be considered outdated
from a curricular point of view: EulerVenn diagrams and classifications
are important in contemporary curricula. Among other things, they underlie
Educational Studies in Mathematics (2006) 62: 259280
DOI: 10.1007/s10649-006-8545-3

C


Springer 2006

260

G. T. BAGNI

the hierarchy of geometrical objects (squares, rectangles, parallelograms,

etc.), the hierarchy of the system of numbers (natural numbers, rational
numbers, real, complex numbers, etc.), and they play an important role in
the construction of mathematical proofs.
From a certain point of view, it may seem that the concepts of set, inclusion and belonging are easy concepts to learn. However, although it is
true that they bear an intuitive meaning, their corresponding mathematical
meanings and the way to symbolize them entail a complex and precise conceptualization. In fact, these concepts attained a mathematical formulation
only in the nineteenth century, when Georg Cantor (18451918) intro
duced the concept of set through some synonyms (Uber
unendliche lineare
Punktmannigfaltigkeiten, 18791884). By set, Cantor meant a collection
of definite and separate objects which can be considered by the mind and
to which one can decide whether or not a given element belongs (Kline,
1972, p. 995; Boyer, 1985, p. 615). Frege also made reference to the concept of set through verbal descriptions in 1902, in his letter to Russell (van
Heijenort, 1967, pp. 126128). One of the main features of the concept of
set according to Cantor is its substantiality (Casari, 1964, p. 21), i.e. the
possibility of belonging to other collections (this feature will be important
in order to introduce the data that we are going to examine) and its independence from our linguistic and theoretical choices. These verbal descriptions cannot be considered as definitions of set: such a definition does not
exist, at least if we make reference to axiomatic theories (e.g. the Zermelo
Fraenkel approach). As for the visual representation of sets, Euler diagrams
were introduced in 1772 by Leonhard Euler (17071783) and resumed in
1881 by John Venn (18341923) in order to show relationships between
sets.
Since as mentioned previously there are no concepts without signs,
teachers and pupils must represent the sets. The introduction of the set
notion in classroom practice is often based upon verbal description. But,
generally speaking, these verbal descriptions are quickly accompanied by
other signs so that diverse semiotic representations are proposed for the set
notion. One of them is the EulerVenn diagrams. In this paper, we shall examine some features of the representation of sets, especially of EulerVenn
diagrams, taking into account the data of two classroom experiments. Our
goal is to provide some insight into the cognitive difficulties that students
encounter as they attempt to cope with the symbolization and the conceptual aspects (Steinbring, 2002) related to the aforementioned fundamental
notions of Set Theory. In Sections 2 and 3 we present some reflections on
the representation of mathematical objects, in order to describe the theoretical framework that will allow us to analyze the data that we shall present
in Sections 5 and 6.

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

2. REPRESENTATIONS: SOME

261

THEORETICAL REMARKS

The study of the semiotic processes underlying the learning of mathematics

is a crucial point in current research in mathematics education. According to
Tall and Vinner (1981) a students concept image for a concept is the total
cognitive structure that is associated with the concept, which includes all the
mental pictures and associated properties and processes (Tall and Vinner,
1981, p. 152; Vinner and Dreyfus, 1989). This perspective is relevant to our
research: if we consider the concept image of set, we realise that students
do not always make reference to the correct associations for approaching
a given task. In fact, a part of the concept image is compartmentalized,
i.e. separated from other parts, so the students do not connect this idea to
other aspects of the concept image. A very important aspect of the concept
image is the concept definition, the statement a student makes when asked
to define the concept: the concept definition, too, may contradict other
aspects of the concept image.
It is necessary to take into account the connections between spatial and
temporal experience (e.g. body movements), and the activity of symbolization, whose importance has been underlined by several works of research,
(Radford, 2002b, 2003a). These considerations will be relevant to our understanding of the students interpretation of EulerVenn diagrams.
The use of (either traditional or new) systems of representation (Kaput,
1993) implies their legitimation, with two linked aspects a social aspect
which selects and sanctions certain kinds of signs, endowing them with a
right or good way of doing and representing, and an epistemological
one:
By a political aspect (from the Greek polis) I mean here an aspect related to social
actions conveying a distinction between what is good and what is not. (. . .) By
an epistemological aspect I mean the manner in which the object of knowledge
becomes known. But, as Kant noticed, how an object becomes known depends
on the manner in which the object is sensed or presented to us, how it affects us
(Radford, 2002a, pp. 236237).

This quotation is very close to the focus of our paper: the representation
registers employed are clearly important. They do not only have a pragmatic
purpose (to do the job), but also bear a social and historical dimension that
influences the way in which the signs cognitively affect us. For example,
the influence of the French Bourbaki group in the educational sphere in
the 1970s, privileged some kinds of signs and purposely discarded others
(visual ones, for instance), favoring a symbolic approach to mathematics
which in turn favored analytical thinking to the detriment of visual thinking.
As we can see, in dealing with the problem of knowledge representation,

262

G. T. BAGNI

it is not possible to dissociate the political from the epistemological (Radford, 2002a, p. 237).3
What the previous remarks suggest is that, from an educative viewpoint,
the cognitive dimension has to be related to the aforementioned social and
epistemological dimensions of semiotic activity. In this line of thought, it is
interesting to discuss the processes by which an object becomes a tool for
the students. We can consider the fundamental distinction between artefact
and tool (Rabardel, 1995), i.e. the artefact associated to a personal or social
schema of action (Artigue, 1998; Radford, 2005). If we make reference to
a symbolic object as artefact (for instance, EulerVenn diagrams), in order
to be able to consider it as a tool, we need a constructive mediated activity
on the part of the subject. Since this activity depends on various conceptual
and social elements, it is impossible to consider a formal representation
in a given register in an absolute sense: it must be framed into a wider
social and cultural context. Symbolizing and conceptualizing are always
culturally framed activities.

3. SETS

AND THEIR REPRESENTATIONS

With regards to the basic concepts of Set Theory, several semiotic registers
are used in mathematical activity:
verbal: the words set, element, belonging, empty set, subset,
inclusion, union, intersection etc. and their definitions (when such
definitions do exist);
symbolic: capital and small letters, different kinds of brackets, symbols
, ,, , etc.
visual: EulerVenn diagrams, etc.
We have underlined that in this work we shall pay attention to visual registers, mainly to EulerVenn diagrams. Certainly, when a pupil
draws a curved line in order to consider a collection of objects inside,
he or she carries out an important and meaningful action: when classes
and sets are visualized in this way, they are conceptualized metaphorically as containers (for instance, according to the metaphor of the container: Lakoff and Nun ez, 2000, p. 45).4 The representation by means
of the container schema relates to our basic, common ways of interacting
with the world. But, of course, EulerVenn diagrams cannot be identified
with the concept of sets (Freudenthal, 1983): the passage from the initial consideration of some elements as mutually connected to the more
general set notion is not simple. Students mathematical conceptualizations are embedded in lengthy processes of sign and meaning production

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

263

(Radford, 2002b, 2003a). The following remarks will be relevant to our

research:
Diagrams evoke the belonging of an element, represented by a point, to a set.
Nevertheless, how can we represent the situation if the considered element is a
set? (. . .) Representing an element with an internal region is misleading because
it causes confusion between belonging and the relation of subset (Ferro, 1993, p.
1086; the translation is ours).

So, educationally-speaking, it is important to consider the difference

between
belonging: x I
inclusion: {x} I
i.e. belonging to the power set (the set of subsets): {x} (I )
This difference, generally rather clear when dealing with verbal or symbolic expressions, must also be clear with regard to visual representations.
However, from an advanced point of view, a b and a b cannot be
considered mutually interchangeable situations: for instance, if we make
reference to natural numbers according to von Neumann, 0 corresponds to
, 1 to {}, 2 to {, {}}, 3 to {, {}, {, {}}} and so on; for a < b
we have either a b or a b. Furthermore, since it is not always true
that x(x a x a), sets that verify the previous condition are called
transitive sets.
The crucial point to be emphasized is the following: the key concepts
of Set Theory, such as the concepts of belonging and inclusion, have an
intuitive meaning that is formalized later, when they appear defined through
precise verbal expressions (which is indeed one of the characteristics of
scientific concepts, according to Vygotsky, 1962).
The passage from the intuitive to the formalized meaning creates a tension between the figural and the conceptual aspects that the students need
to resolve in order to secure a suitable concept attainment. Fischbein (1993)
stated that the integration of conceptual and figural properties in unitary
mental structures, with the predominance of the conceptual constraints over
the figural ones, is not a natural process and it should constitute an ongoing
concern for the teacher. According to Fischbein and Baltsan (1999), pupils
often generalize the way they were initially taught before they develop a
critical attitude: as a matter of fact, mental behaviors tend to act beyond formal control because they frequently shape the experience in a meaningful
way. In the work in question, Fischbein and Baltsan proposed an interesting
approach to the construction of mathematical schemata: pupils are initially
presented with activities to bring about the construction of schemata that
will later be replaced by alternative ones. Nevertheless, sometimes this

264

G. T. BAGNI

replacement process does not lead to the full extinction of the old scheme:
so an initial scheme can become a tacit model shaping the students reasoning despite their subsequent instruction in alternative ones (Linchevski
and Livneh, 2002).
While the fundamental expressions of Set Theory are rooted in the linguistic structure of subjectpredicate, EulerVenn diagrams initially display the predicative structure as points in a closed-plane figure: the predicative structure hence becomes geometrized. In terms of this distinction,
our claim is the following: EulerVenn diagrams are powerful tools for
endowing students mathematical activity with meaning; however these
tools should not be seen as means to replace the meaning of the predicative structure. This point is central for understanding the main discussion of
our research: Radford (2002b), following Freges distinction between sense
and reference, suggests that the various semiotic systems of the technology
of semiotic activity (e.g. the visual EulerVenn diagrams, the verbal and
the symbolic systems) have their own cultural way to bestow meaning on
objects of discourse. Rather than a problem of translatability or conversion,
knowledge attainment becomes possible as a solid articulation of the semiotic systems of the technology of semiotic activity, even if prominence is
sometimes put on one system, and sometimes on another (Radford, 2003c,
p. 136; see also Duval, 1993).
We shall use the aforementioned theoretical reflections in order to explain the experimental data. In particular we shall consider some difficulties
that students have in distinguishing between the concepts of inclusion and
belonging and we shall investigate them in order to point out whether or not
and how the use of a representation (namely EulerVenn diagrams) links
different conceptual aspects.

4. METHODOLOGY
We are going to present the empirical data collected in two experiments
regarding different school-levels: the first (experiment A) dealt with the
introduction of sets to 11-year-old pupils (middle school in Italy) and will
provide us with some pointers for an in-depth study of the second (experiment B), dealing with a 15-year-old student (the initial letters of students
names have been changed for deontological reasons).
The researcher was not the mathematics teacher of the students in question; however he was present in the classroom with the mathematics teacher
and the students. Both experiments took place during a regular lesson in
the classroom; all the students were present (16 in experiment A; 25 in
experiment B).

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

265

Audio-recorded material and transcriptions allowed us to point out some

salient short passages that were analyzed using techniques of qualitative
research (concerning the methodology, we make reference to Arzarello and
Bartolini Bussi, 1998; Boero and Szendrei, 1998). In particular, we examined three extracts; the translations are ours (in order to present original
Italian wording, a transcription of the considered interactions is reported
in a note at the end of this paper).5
Experiment A dealt with a short extract (1 min and 30 s, 14 utterances6 ).
Two 6th Grade pupils (1112-year-old) were required by the teacher to
represent a set and they used an EulerVenn diagram on the blackboard.
As we shall see, this experiment dealt with an interpersonal channel (Ryve,
2004): one of the pupils proposed two answers and the other pupil made
some comments; finally the teacher proposed a third answer.
We shall present two excerpts related to experiment B: in the first one
(1 min and 10 s), a 9th Grade class student (1516-year-old) was required
to solve (on the blackboard) an exercise proposed by the researcher. In the
second excerpt (1 min and 30 s, eight utterances), the same student was
asked by the researcher to reconsider her previous answer.
The students in experiment A knew the basic notions of set, belonging, subset and they were used to represent sets by EulerVenn diagrams;
the educational background related to experiment B will be examined in
Section 6.1.
5. EXPERIMENT A: SETS

AND ELEMENTS IN THE MIDDLE SCHOOL

The teacher writes on the blackboard: Represent the set of the sides of a
triangle by a picture. Two pupils, S. and G., are standing near the blackboard; as we shall see, S. tries twice to give an answer, while G. makes
some comments.
5.1. First representation (proposed by S.)
S. (draws the following picture): Here it is.

Teacher: Your answer is not wrong and your picture is fine. However, when
I look at your picture I could think of a set with only one element.

266

G. T. BAGNI

S.: Why? There are three sides.

Teacher: Yes, but they constitute a triangle: so what comes to mind is a
triangle, the whole geometric figure, and not the three sides.
G. (intervening): When I look at that picture I also think of a triangle and
not the three sides!
5.2. Second representation (proposed by S.)
S. (to the other pupil): And then what should I do? Should I break my
triangle up? (she erases the triangle and draws three segments).

G.: No, this is not a triangle; our exercise is about a triangle.

Teacher (to S.): Just a moment, your previous answer can be considered
right, if we interpret the picture correctly. Can you think of any other
representation?
S.: Using this kind of picture?
Teacher: Yes, with EulerVenn diagrams.
S. (after a while): No.
5.3. Third representation (proposed by the teacher)
Teacher: Now let me try to suggest something. Do you remember that in
Geometry we often use some letters in order to give names to the points
and to the sides of figures? We can do that here. (Draws the following
figures). What do you think?

G.: In my opinion this is the best picture because I can see there are three
of them.
S. (uncertain): But inside the set there is no information regarding my
points. I cannot guess they are the sides of a triangle. I have to look at
the picture outside. . .

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

267

Let us pause now. It is worth noting that the representations are characterised by a different resemblance to the geometric situation under consideration. In particular
the first representation, the elements of the considered set have been
drawn so to speak in full, i.e. as sides of a particular triangle along
with the triangle itself;
the second representation (introduced by S.s reactive utterance: And
then what should I do? Should I break my triangle up?), the elements
are segments. Although they have been drawn in a position that is not
exactly the position on the sides of the triangle, they still retain a close
position to their alleged original one;
finally, the third representation, the elements are represented by isolated
points. However, S believes that their interpretation as sides of a triangle
needs a companion picture: the position of the segments conveys an
iconic information that is lost when segments are represented by points.
In fact, EulerVenn diagrams are not merely containers where objects
are thrown: the predicative meaning of the basic object-predicate structure
mistakenly becomes thought of as the element a is in the set A, suggested
by the geometric representation.
Moreover, it is worth highlighting the teachers remark which holds
that the different visual representations must be interpreted (your previous
answer can be considered right, if we interpret the picture correctly). We
shall consider these issues concerning the different representation registers
in the second experiment.
6. EXPERIMENT B: THE

CASE STUDY OF

K.

6.1. Educational data

K. (15-years-old) is a student in the first year of Ginnasio-Liceo Classico
(Treviso, Italy) who achieves good results.
Here are some quotations from the textbook used by K. (Dodero et al.,
1999; the translation is ours):
Elements, sets, belonging: We shall use the word set in order to indicate
a group, a collection of elements: these can be objects, persons, symbols,
numbers, pictures, etc. We shall consider these elements clearly defined.
(. . .) Generally sets are indicated by capital letters, elements are indicated
by small letters. The expression a A means: a belongs to A (Dodero
et al., 1999, p. 11);
EulerVenn diagrams: We shall use the following representation: an
elliptic line indicates a region of the plane and the elements of the set

268

G. T. BAGNI

are represented inside by some points (it is possible to write the name of
each element near the point) (Dodero et al., 1999, p. 12); The following
example, provided in the textbook, refers to set C of the letters of the
Italian word studente:

Subsets, inclusion: Given (two) sets A and B, we say that B is a subset

of A if every element of B belongs to A too. In symbols we write A B
and we say B is contained in A or is equal to A or B is included in A or
is equal to A (Dodero et al., 1999, p. 13); the example given, as noticed
in the textbook, can be drawn as shown here.

Power set (set of subsets): Given a set A, we call power set of A set
(A) whose elements all subsets of A. (. . .) In general, if A contains n
elements, (A) has 2n elements (Dodero et al., 1999, p. 14).
Two remarks are relevant: first of all, it is important to underline the
ambiguous use of the term contain (in Italian: contiene): in the introduction
to the concept of subset, the expressions is included and is contained
are explicitly compared; nevertheless, concerning the power set, we find
the following statement: if A contains n elements, (A) has 2n elements
so the term contain is now employed in the sense of belonging. Moreover,
it is worth noting the slightly different use of EulerVenn diagrams in the
reported examples: in the first case, elements are clearly marked by some
points and by their names; in the second case, single elements are not
indicated: all the points of the considered part of the plane can be regarded
as elements of the set.

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

269

K. examined some examples of objects and numbers which are frequently represented by EulerVenn diagrams. The notion of subset was
illustrated by making reference to some examples related to geometric figures (for instance, the set of squares is a subset of the set of rectangles and
this is a subset of the set of parallelograms and so on).
6.2. An exercise
The researcher proposed the following exercise to K.:
Let I be the set of the points of the plane.
Let R be the set of the points of a given line in the plane.
Let S be the set of the points of a given line in the plane that is perpendicular
to R.
Let A be the set having elements R and S.
Does A belong to the power set of I?
The student wrote the previous information on the blackboard. Then K.,
standing near the blackboard, was asked to solve it. First of all, K. drew
two perpendicular lines on the blackboard and called them R and S.

After that K. drew an elliptic line around the figure and called the set
capital A. Her gesture was clear; she appeared to be resolute.

K.: This is set A (reading the exercise once again), now I have to see if A
belongs to the power set of I.
K. (after a few seconds): The power set of I contains all the subsets of I.
The figures (referring to S and R) are made up of points [in Italian: sono

270

G. T. BAGNI

fatte di punti], so all the figures of the plane are elements of the power
set of I.
K. (after a few seconds): A contains these two lines (she indicates the lines)
so it is a figure of the plane.
K. (after a few seconds): So A is an element of the power set of I.
Concerning K.s behaviour, she appeared to be calm and resolute;
the tone of her voice was sure. The classroom background, too, was
calm.
Let us now consider K.s argument. From the verbal point of view, it
would be summarised as follows:

The power set of I contains all the subsets of I.

All the figures of the plane are elements of the power set of I.
A contains the lines and so it is a figure of the plane.
So A is an element of the power set of I.

The use of the term contain is ambiguous and hence misleading: in the
first sentence, K. makes reference to belonging; but in the third sentence
she says A contains the lines and, consequently, it is a figure of the
plane. However, the statement: A contains the lines and so it is a figure of
the plane deserves a careful interpretation. The previous analysis would
suggest that the students idea can be rendered as R A and S A;
this would imply that the student made a point-by-point analysis (e.g. that
she said that every point in R is a point in A, etc.), but such an interpretation
in the students statement is not evident. More properly, it seems that K.
is saying that R and S are in A and since A contains lines (or figures), A is
itself a figure.
This situation bears out the previously mentioned comment concerning the dangerous use of terms whose meaning has not been cleared
up enough (moreover, as far as the term contain referring to belonging
and to inclusion is concerned, it is possible to use some non-ambiguous
synonyms).
We will now summarize the steps of Ks solution in terms of representation registers:
the exercise is expressed in a verbal register;
immediately K. expresses the situation in a visual register and traces the
two lines;
in this register, K. joins the two lines by tracing an elliptic line around
them;
then, she makes reference to the picture and indicates the set A by the
words the two lines (in Italian: le due rette).

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

271

So the (perhaps too hurried) use of a visual register seems to prevent the
correct consideration of the crucial point: K. did not realize that A is the set
with elements R and S. K. made reference to the whole figure A = R S.
It is important to underline that K. has expressed the set A in a visual
register, but she used this register (i.e. she interpreted EulerVenn diagrams)
in an improper way.

Indeed, K. considered R, S as subsets of A, instead of considering them as

elements of it. The EulerVenn diagram was hence incorrectly employed.
On the contrary, this different situation, where the EulerVenn diagram
is correctly employed, would suggest the (correct) belonging of R and S to
set A.
It is worth noting that the representation considered by K. is not correctly
used because the geometric representation of the two lines is not relevant
to the concept of set: perhaps the mutually perpendicular lines (a couple of
perpendicular lines) leads the student to consider a single figure, so such a
relationship between the elements of the set would induce an inappropriate
interpretation.
In the case under consideration, would it have been better for K. to
avoid the use of a visual register? We do not think so. A remark that we

272

G. T. BAGNI

are going to return to in the final discussion should be highlighted at this

point: the statements made above do not imply that using visual registers is
didactically inappropriate. The visual registers certainly help the students
to endow the problem about sets under consideration with meaning. In
our opinion, it would be too simple to state that the improper use of a
visual register caused a mistake: in fact, as we shall see, the reason for the
mistake could have been, more properly, the concept image in the students
mind.
Let us try to outline a path towards reaching a correct conclusion. The
History of Logic could suggest an interesting reference to the Theory of
Types: in our case, type 0 objects are the points; type 1 objects are the sets
R, S; type 2 objects are the set A and the sets of sets, (I ). It is impossible
that an object belong to an object of the same type, so the proposed exercise
can be easily solved. Nevertheless, the Theory of Types seems unlikely to
be a good choice from the educational point of view.
Apart from the correct use of EulerVenn diagrams, previously described, another possibility is summarized in the following picture, where
instead of using the visual register, the reference to the original exercise
(verbally expressed) is maintained.

Although we do not state that verbal and symbolic registers should

be treated similarly (students different difficulties can be pointed out
when addressing these two representations), in order to propose a solution dealing with those registers, we shall make reference to the following
argument:
according to the definition: {R; S} (I ) means {R; S} I
so: R I and S I
in our case: that is not true
so we can conclude:
the answer to the question: Does A = {R; S} belong to (I ) ? is: No.

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

273

Let us note that this argument is based upon a definition and clearly
the notions of concept image and concept definition are relevant to this
situation. As we shall see, the role of examples and definitions in K.s justifications, explanations and arguments is important. Let us turn to another
brief excerpt.
Researcher: Lets turn back to our definitions. (He erases the picture and
writes): {R; S} (I ) means {R; S} I and this means R I and
S I . Now consider R I and S I : are they true or not?
K.: I is the plane and R, S are the lines (she is going to draw them once
again).
Researcher: Please, do not draw them now. You said that I is the plane:
could you be more precise?
K. (after a few seconds): Well, I is the set of the points of the plane. It is
not a single thing, it is a set, it is indicated by a capital letter.
Researcher: A set of. . . what?
K. (in a low voice): Points. A set of points of the plane.
Researcher (indicates R I and S I): So, are R and S elements of I?
K. (after a few seconds): No, R and S are sets, they are not elements. They
too are indicated by capital letters.
K. appeared to be rather calm, but the tone of her voice was not completely sure. The classroom background was calm.
However K.s argument is not clear and the student herself is uncertain
in putting it forward.7 Probably the definition suggested by the Researcher
does not coincide with the students concept definition. As a matter of fact,
it looks like K.s argument is just based upon a dichotomy between sets
and elements, and this dichotomy is a part of K.s concept image; so it
does not consider the crucial point: R and S are not points of the plane so
they do not belong to I. In particular, K. underlines that R, S (and I) are
marked by capital letters.
This traditional use of capital letters (for sets) and lower-case letters
(for elements) can suggest a subdivision of mathematical objects into separated classes, sets and elements: this distinction could be educationally
useful, but when we must consider a set which belongs to another set
(for instance when we are dealing with the power set of a set) this situation can cause conflicts with the traditions of symbolic notation. So it
can cause the dangerous misconception that a set cannot belong to another set. However, it should be noticed that the presence of long chains of
mathematical objects like a b c . . . is rare in mathematics textbooks
(apart, of course, from Set Theory handbooks; sometimes different characters are used: x A B...). So the traditional use of lower-case letters for

274

G. T. BAGNI

elements and of capital letters for sets can be accepted, if adequately cleared
up.8
Let us conclude the examined case study with a remark. Although the
researcher has tried to clear up the situation, K. does not apply the formal
definitions (Tall and Vinner, 1981). The researcher explicitly suggested the
use of a different approach, but the student tried to use a visual register for
a second time. In fact, visual registers are concrete and suitable, particularly with reference to the example considered (this evokes sectorialization:
Schoenfeld, 1986): the subject deals with geometrical objects and so with
a sector usually expressed by visual registers (concerning the influence of
visual representations, see Tsamir and Tirosh, 1994, 1999; Tsamir, 1999;
Tsamir and Dreyfus, 2002). Symbolic registers are more general (although
some symbols are sometimes used with implicit meanings, e.g. the difference between x and x0 , or the use of n for a natural number and of p for a
prime etc.) and they require good abilities in the attainment of abstraction.

7. D ISCUSSION
We can now reanalyze the main results exposed in terms of our theoretical
framework. The experimental excerpts show that the students difficulties
relate to their failure to grasp the fact that the concept of inclusion is based in
a synthetic view of sets: following Cantor, sets are comprised of individual
objects and to claim that one set is included in another one requires an
analysis of all the individual objects forming the first set. As previously
noted (Section 3), the fundamental expressions of Set Theory are rooted
in the linguistic structure of subject-predicate, but EulerVenn diagrams
display such predicative structure as points in a closed plane figure, and the
visual nature of the predicate in the semiotic system of Venn-Euler diagrams
may hide this essential point that is put forward by the linguistic formulation
(Radford, 2002b). If this point is not grasped by the students, rather than an
aid, EulerVenn diagrams risk becoming misunderstood. In fact they make
reference to a mental behavior that has shaped the students experience in
a meaningful way (Fischbein and Baltsan, 1999), but the initial scheme has
become a tacit model that influences the students reasoning.
As to the theoretical reflections on representations mentioned in Sections 2 and 3, we can state that the students were affected by signs in the
sense that those signs offered them new paths of conceptual development
(Radford, 2002a): in fact visual signs (EulerVenn diagrams) allowed the
students to achieve a certain understanding where a certain kind of predictability was put forward. Of course, sign-letters employed as a means to
refer to segments affected the students in a different form, allowing them to

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

275

apprehend, even if in a partial manner only, the concepts of belonging and

inclusion in a synthetic way. Through each one of these registers of semiotic
representation, the students built partial aspects of a conceptual image of
the object, which remained nonetheless in separate compartments, at least
to a certain extent.
It is important to note that the coherent articulation of the different
partial meanings induced by the representation registers is not reduced
to an activity of translation or conversion (as we noticed in Section 3),
but rather it should be consistently worked in the educational process, in
order to achieve a flexible transit among them. The coherent articulation
of meanings must be considered by the teacher and made clear to the
students when the use of a visual register (or of a register of another kind) is
proposed.
Systems of representation embody the students spatial and temporal
mathematical experience in a variety of ways, which is made clear here by
the analysis of the different ways to predicate about sets and subsets, one
geometric and one symbolic-linguistic. The use of EulerVenn diagrams
by the students reminds us that it is important to take into account the connections between: (1) the actions by which students indicate and join some
elements together using an elliptic line, and (2) the formation of the concept of set. This connection between actions and concepts is underpinned
by a production of gestures and the use of linguistic deictic terms (such
as questo, i.e. this used by K.; see endnote 5) out of which the students seek to secure a refined mathematical understanding (see Steinbring,
2002, pp. 910, where the author considers the distinction between deictic
function and generative action function introduced by Radford, 2000,
p. 247).
As far as visual registers are concerned, a representation is not always
an exact representation: EulerVenn diagrams clearly work in the visual
sphere, but at the same time, EulerVenn diagrams possess several aspects
that, to a certain extent, belong to the arbitrary symbol: thus, when we
indicate an element we do not use its accurate reproduction. If we unconsciously make reference to any exactness of these diagrams we implicitly
also include any improper relationship existing between the elements of
the set.9
The use of a representation register links many conceptual aspects, and
this can be referred to every kind of register: verbal registers, too, make
reference to words and to the meanings of those words, meanings that link
to other registers (Bagni, forthcoming; concerning verbal descriptions and
graphic representations see Mevarech and Kramarsky, 1997). As a matter
of fact, many of the data previously presented can be explained by looking
at language usage: belongs to and included in are sometimes used in the

276

G. T. BAGNI

language interchangeably, and this leads us to consider the impact of daily

context on mathematical reasoning (Bourguignon, 2001, pp. 175176).
To summarize the previous remarks, let us emphasize that, from an educational point of view, a representation system cannot be considered in
absolute terms: in fact, its legitimation is dependent on a cultural context,
and it is impossible to dissociate the political and the epistemological aspects (as noticed in Radford, 2002a). The case study previously examined
highlights the importance of this dependence if EulerVenn diagrams are
to be considered as a real tool for students (Rabardel, 1995).
Since the use of mathematical signs is acquired by means of social
participation, and not (. . .) according to strict rules (Steinbring, 2002, p.
10), we suggest an adequate negotiation of meanings between teacher and
students (of course this does not mean that if notation and terms are introduced and used correctly, no misconception will occur). Bearing in mind
that there is no a unique or single representation register for a mathematical
object, the problem is then to learn to use them and to articulate their corresponding meanings. As we mentioned earlier, different semiotic systems
affect in different manners the way one thinks about objects of knowledge, and each one of them allows us to deal with the general in different
manners. In this context, our awareness that mathematical knowledge can
only be attained through representations, leads us to insist upon the fact
that the study of students semiotic processes is one of the major issues in
mathematics education.
ACKNOWLEDGMENTS
The author would like to thank the anonymous reviewers of the Journal
for the very valuable help provided for improving this paper, and specially
thanks one of the reviewers for the important editorial and language work
on the paper.
NOTES
1. Throughout his work Vygotsky insisted that there is no concept without signs (see e.g.
Vygotsky, 1962).
2. Duvals use of the term register emphasizes the particular operational character of
signs. This idea of register is different from Hallidays. For the latter, a register is defined
as a linguistic variety based on use (Duval, 1995; Halliday, 1985; Leckie-Tarry, 1995)
linking the text, the linguistic and the social systems (Ferrari, 2004).
3. For a revision of ideas by Vygotsky concerning the relationship between representation
and cognition, taking into account symbolic aspects as constitutive elements of culture
and cognition see Radfords Cultural Semiotic Systems of Signification, in Radford,
2003b.

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

277

4. M. Johnson underlines the importance of the image-schematic level, which gives general form to our understanding in terms of structures such as container (. . .) and allows
us to make sense of the relation among diverse experiences: Johnson, 1987, p. 208.
5. Let us report the original wording (in Italian) of the presented interactions:
S.: Ecco.
T.: Non e` sbagliato, hai fatto un bel disegno. Per`o guardando potrebbe essere un
insieme con un elemento solo.
S.: Perche uno? Ho fatto tre lati.
T.: S`, ma fanno parte del triangolo: e` un triangolo che ti viene in mente, tutta la
figura, non i tre lati.
G.: Eh, anchio ci vedo il triangolo e no i tre lati!
S.: Gi`a, e cosa devo fare? Devo romperlo?
G.: No, cos` non e` un triangolo, lesercizio diceva triangolo.
T.: Momento, la tua risposta andava bene se si interpreta bene la figura. Provi a
pensare a unaltra rappresentazione?
S.: Ancora con quei disegni l`?
T.: S`, coi diagrammi di Eulero-Venn.
S.: Mm, no.
T.: Senti, cerco di darti unidea. Ti ricordi che quando facciamo geometria usiamo
le lettere per dare i nomi ai punti e ai lati? Proviamo anche qui. Eh, ti va?
G.: Secondo me e` questo disegno che va bene perche si vede che ce ne hai tre.
S.: S` per`o dentro non c`e mica scritto che quelli l` sono i lati, cio`e come faccio a
saperlo, devo guardare fuori . . .
K.: Questo e` linsieme A. Adesso guardiamo se appartiene allinsieme delle parti di I.
K.: Linsieme delle parti di I contiene tutti i sottoinsiemi. Le figure sono fatte di
punti e quindi tutte le figure sono elementi dellinsieme delle parti di I.
K.: A contiene le due rette . . . e allora e` una figura del piano.
K.: S`, s`, A e` un elemento dellinsieme delle parti.
R.: Torniamo un po alle definizioni. Adesso pensa: e` vero o non e` vero che R
appartiene a I e S appartiene a I?
K.: I e` il piano e R e S sono delle rette.
R.: No per favore, non disegnare per adesso. Hai detto che I e` il piano, ecco, puoi
dirlo meglio, insomma puoi essere pi`u precisa?
K.: Beh, I sarebbe linsieme di punti, dei punti del piano. S`, cio`e non e` una cosa da
sola, e` un insieme, si scrive con la maiuscola.
R.: Ma un insieme di . . . che cosa?
K.: Mm, punti. Punti del piano.
R.: Allora R e S, sono elementi di I?
K.: Ah no, gi`a, e` vero, R e S sarebbero insiemi, mica elementi. Anche loro si scrivono
con la maiuscola.
6. Following P. Linell, an utterance is a stretch of continuous talk by one person,
regardless of length and structure: Linell, 1998, p. 160.
7. More than ten years ago, Balacheff (1991) pointed out that students aim is to
produce a solution more than theoretical knowledge; see also Arzarello et al.,
2002.
8. Another well-known situation connected to the use of capital and lower-case letters
refers to the belonging of points to lines: in this case elements are represented by
capital letters, sets by lower-case letters.

278

G. T. BAGNI

9. A similar erroneous situation would take place if we listed the elements according to
any given order: for instance, an ordered couple cannot be represented by EulerVenn
diagrams simply by writing the first element to the left of the second.

REFERENCES
Artigue, M.: 1998, Levolution des problematiques en didactique de lanalyse, Recherches
en Didactique des Mathematiques 18(2), 231262.
Arzarello, F. and Bartolini Bussi, M.: 1998, Italian trends of research in mathematics
education: A national case study in the international perspective, in J. Kilpatrick and
A. Sierpinska (eds.), Mathematics Education as a Research Domain: The Search for
Identity, Vol. 2, Kluwer, Dordrecht, pp. 243262.
Arzarello, F., Bartolini Bussi, M.G. and Robutti, O.: 2002, Time(s) in didactics of mathematics: A methodological challenge, in L.D. English, M. Bartolini Bussi, G.A. Jones,
R.A. Lesh and D. Tirosh (eds.), Handbook of International Research in Mathematics
Education, Erlbaum, Mahwah, NJ, pp. 525552.
Bagni, G.T.: in press, Historical roots of limit notion. Development of its representative
registers and cognitive development, Canadian Journal of Science, Mathematics and
Technology Education.
Balacheff, N.: 1991, The benefits and limits of social interactions: The case of mathematical proof, in A.J. Bishop, A.J. Bishop, S. Mellin-Olsen and J. van Dormolen (eds.),
Mathematical Knowledge: Its Growth Through Teaching, Kluwer, Dordrecht, pp. 175
192.
Boero, P. and Szendrei, J.: 1998, Research and results in mathematics education: Some
contradictory aspects, in J. Kilpatrick and A. Sierpinska (eds.), Mathematics Education
as a Research Domain: The Search for Identity, Vol. 1, Kluwer, Dordrecht, pp. 197212.
Bourguignon, J.-P.: 2001, A basis for a new relationship between mathematics and society,
in B. Engquist and W. Schmid (eds.), Mathematics Unlimited-2001 and Beyond, Vol. I,
Springer-Verlag, Berlin, pp. 171188.
Boyer, C.B.: 1985, A History of Mathematics, Princeton University Press, Princeton, NJ
(1st ed., Wiley, New York, 1968).
Casari, E.: 1964, Questioni di filosofia della matematica, Feltrinelli, Milano.
Dodero, N., Baroncini, P. and Manfredi, R.: 1999, Lineamenti di matematica 1, Ghisetti e
Corvi, Milano.
Duval, R.: 1993, Registres de representations semiotiques et fonctionnement cognitif de la
pensee, Annales de Didactique et de Sciences Cognitives IREM Strasbourg, 5, 3765.
Duval, R.: 1995, Semiosis et pensee humaine, Lang, Paris.
Duval, R.: 2000, Basic issues for research in Mathematics Education, in T. Nakahara and
M. Koyama (eds.), Proceedings of the 24th PME, Vol. 1, Nishiki Print, Hiroshima, pp.
5569.
Ferrari, P.L.: 2004, Mathematical language and advanced mathematics learning, in M.
Johnsen Hines and A. Berit Fuglestad (eds.), Proceedings of the 28th PME, Vol. 2,
Bergen, Norway, pp. 383390.
Ferro, R.: 1993, La teoria degli insiemi p. II., Linsegnamento della matematica e delle
scienze integrate 16(11/12), 10771099.
Fischbein, E.: 1993, The theory of figural concepts, Educational Studies in Mathematics
24, 139162.

COGNITIVE DIFFICULTIES RELATED TO CONCEPTS OF SET THEORY

279

Fischbein, E. and Baltsan, M.: 1999, The mathematical concept of set and the collection
model, Educational Studies in Mathematics 37, 122.
Freudenthal, H.: 1983, Didactical Phenomenology of Mathematical Structures, Reidel,
Dordrecht.
Halliday M.A.K.: 1985, An Introduction to Functional Grammar, Arnold, London.
Johnson, M.: 1987, The Body in the Mind. The Bodily Basis of Meaning, Imagination, and
Reason, University of Chicago Press, Chicago.
Kaput, J.J.: 1993, The representational roles of technology in connecting mathematics
with authentic experience, in R. Bieler, R.W. Scholz, R. Strasser and B. Winkelman
(eds.), Didactics of Mathematics as a Scientific Discipline, Kluwer, Dordrecht, pp. 379
397.
Kline, M.: 1972, Mathematical Thought from Ancient to Modern Times, Oxford University
Press, New York.
Lakoff, G. and Nun ez, R.: 2000, Where Mathematics Come From? How the Embodied Mind
Brings Mathematics into Being, Basic Books, New York.
Leckie-Tarry, H.: 1995, Language and Context A Functional Linguistic Theory of Register,
Pinter, London.
Linchevski, L. and Livneh, D.: 2002, The competition between numbers and structure: Why
expressions with identical algebraic structures trigger different interpretations, Focus on
Learning Problems in Mathematics 24(2), 3820.
Linell, P.: 1998, Approaching dialogue: Talk and interactions in dialogical perspective,
John Benjamins, Philadelphia.
Mevarech, Z.R. and Kramarsky, B.: 1997, From verbal description to graphic representations: Stability and change in students alternative conceptions, Educational Studies in
Mathematics 32, 229263.
Otte, M.: 2001, Mathematical epistemology from a semiotic point of view, Paper presented
to the Discussion Group on Semiotics at the 25th PME.
Rabardel, P.: 1995, Les hommes et les technologies: Approche cognitive des instruments
contemporains, Colin, Paris.
Radford, L.: 2000, Signs and meanings in the students emergent algebraic thinking,
Educational Studies in Mathematics 42, 237268.
Radford, L.: 2002a, The object of representations: Between wisdom and certainty, in F.
Hitt (ed.), Representations and Mathematics Visualization, Cinvestav-IPN, Mexico, pp.
219240.
Radford, L.: 2002b, The seen, the spoken and the written. A semiotic approach to the
problem of objectification of mathematical knowledge, For the Learning of Mathematics
22(2), 1423.
Radford, L.: 2003a, Gestures, speech and the sprouting of signs, Mathematical Thinking
and Learning 5(1), 3770.
Radford, L.: 2003b, On culture and mind. A post-Vygotskian semiotic perspective, with an
example from Greek mathematical thought, in M. Anderson, A. Saenz-Ludlow, S. Zellweger and V.V. Cifarelli (eds.), Educational Perspectives on Mathematics as Semiosis:
From Thinking to Interpreting to Knowing, Legas, Ottawa, pp. 4979.
Radford, L.: 2003c, On the epistemological limits of language: Mathematical knowledge
and social practice during the Renaissance, Educational Studies in Mathematics 52,
123150.
Radford, L.: 2005, The semiotics of the schema. Kant, Piaget, and the Calculator, in
M.H.G. Hoffmann, J. Lenhard and F. Seeger (eds.), Activity and Sign. Grounding Mathematics Education, Springer, New York, pp. 137152.

280

G. T. BAGNI

Ryve, A.: 2004, Can collaborative concept mapping create mathematical productive discourses?, Educational Studies in Mathematics 26, 157177.
Schoenfeld, A.H.: 1986, On having and using geometric knowledge, in J. Hiebert (ed.),
Conceptual and Procedural Knowledge: The Case of Mathematics, Erlbaum, Hillsdale,
NJ, pp. 225263.
Steinbring, H.: 2002, What makes a sign a mathematical sign? An epistemological perspective on mathematical interaction, Paper presented to the Discussion Group on Semiotics
at the 26th PME.
Tall, D. and Vinner, S.: 1981, Concept image and concept definition in mathematics with
special reference to limits and continuity, Educational Studies in Mathematics 12, 151
169.
Tsamir, P. and Tirosh, D.: 1994, Comparing infinite sets: Intuition and representation,
Proceedings of the XVIII PME. Lisboa, 2, 345352.
Tsamir, P.: 1999, The transition from comparison of finite to the comparison of infinite
sets: Teaching prospective teachers, Educational Studies in Mathematics 38, 209234.
Tsamir, P. and Tirosh, D.: 1999, Consistency and representations: The case of actual infinity, Journal for Research in Mathematics Education 30, 213219.
Tsamir, P. and Dreyfus, T.: 2002, Comparing infinite sets a process of abstraction. The
case of Ben, Journal of Mathematical Behavior 221(1), 123.
van Heijenort, J.: 1967, From Frege to Godel, Harvard University Press, Cambridge, UK.
Vinner, S. and Dreyfus, T.: 1989, Images and definitions for the concept of a function,
Journal for Research in Mathematics Education 20, 356366.
Vygotsky, L.S.: 1962, Thought and Language. MIT Press, Cambridge, MA (Mylenie i re,
Gosudarstvennoe Socialno-Ekonomieskoe Izdatelstvo, Moskva-Leningrad 1934).

Department of Mathematics and Computer Science

University of Udine, Via delle Scienze 206
I-33100 Udine, Italy
E-mail: bagni@dimi.uniud.it