Escolar Documentos
Profissional Documentos
Cultura Documentos
Chapter 9
Purposes of evaluation
Weir and Roberts (1994) distinguish between two major purposes for lan
guage program evaluation, program accountability, and program develop
ment. Accountability refers to the extent to which those involved in a pro
gram are answerable for the quality of their work. Accountability-oriented
evaluation usually examines the effects of a program or project at signifi
cant end points of an educational cycle and is usually conducted for the ben
efit of an external audience or decision maker. Development-oriented eval
uation, by contrast, is designed to improve the quality of a program as it is
being implemented. It may involve staff who are involved in the program
as well as others who are not and may have a teacher-development focus
(Weir and Roberts 1994, 5). The different purposes for evaluation are re
ferred to as formative, illuminative, and summativeevaluation.
Formative evaluation
Evaluation may be carried out as part of the process of program develop
ment in- order to find out what is working well, and what is not, and what
problems need to be addressed. This type of evaluation is generally known
as formative evaluation. It focuses on ongoing development and improve
ment of the program. Typical questions that relate to formative evaluation
are:
Approaches to evaluation
289
Illuminative evaluation
Another type of evaluation can be described as illuminative evaluation. This
refers to evaluation that seeks to find out how different aspects of the pro
gram work or are being implemented. It seeks to provide a deeper under
standing of the processes of teaching and learning that occur in the program,
without necessarily seeking to change the course in any way as a result.
Questions that might be asked within this framework are:
How do students carry out group-work tasks? Do all students participate
equally in them?
What type of error-correction strategies do 'teachers use?
290
Chapter 9
Approaches
to evaluation
291
learn
Summative evaluation
A third approach to evaluation is the type of evaluation with which most
teachers and program administrators are familiar and which seeks to make
292
Chapter 9
Approaches
293
9
to evaluation
293
Chapter
used to measure achievement. Such tests might be unit rests given at the end
of each unit of teaching materials, class tests or quizzes devised by teach
ers and administered at various stages throughout the course, or as formal
exit tests designed to measure the extent to which objectives have been
achieved. Weir (1995) points out that achievement tests can have an impor
tant washbackeffect on teaching and learning. They can help in the making
of decisions about needed changes to a program, such as which objectives
'need more attention or revision. Brindley (1989) reports, however, that in
programs he studied in Australia, teachers preferred to rely on informal
methods of ongoing assessment rather than formal exit tests. About the use
of informal methods, he comments:
This does not seem to be sufficientlyexplicit to meet the expectations and
requirements of either administrators or learners for more formal
informationon learners' achievement of a course or a unit .... The informal
methods of ongoing assessment provided by teachers do not provide the kind of
explicit information on achievement required by learners and administrators.
(Brindley 1989, 43)
Weir (1995) argues for the need for better measures of surnmative evalua
tion and for the development of progress-sensitive performance tests for use during courses.
Approaches
294
9
to evaluation
294
Chapter
Approaches
295
9
to evaluation
295
Chapter
Students
What did I learn?
How well did I do compare to others?
How well will I rate this course?
How will this help me in the future?
Do I need another course?
Teachers
How well did I teach?
What did my students learn?
Were my students satisfied with the course?
How useful were the materials and course work?
How effective was the course organization?
Curriculum developers
Is the design of the course and materials appropriate?
What aspects of the course need replacing or revising?
Do teachers and students respond favorably to the course?
Do teachers need additional support with the course?
Administrators
. Was the time frame of the course appropriate?
Were the management and monitoring of the course successful in identifying and rectifying problems?
Were clients' expectations met?
Were testing and assessment procedures adequate?
Were resources made use of?
Sponsors
Was the cost of the course justified?
Did the course deliver what was promised?
Was the course well managed?
Is the reporting of the course adequate?
Shaw and Dowsett (1986, 66) suggest that three audiences are identifi
able for all summative evaluation of language courses:
other teachers in the program, for course design and planning purposes
(the main audience)
Approaches
296
9
to evaluation
296
Chapter
Participantsin
process
the
evaluation
Approaches
297
9
to evaluation
297
Chapter
Written comments: anything that has been written about the course by
external assessors, teachers, learners, managers.
Approaches
298
9
to evaluation
298
Chapter