Escolar Documentos
Profissional Documentos
Cultura Documentos
Linguistics: Assessment
Carol A. Chapelle
English Language Learning and Technology (2003)
John Benjamins Publishing Company
Amsterdam
Notes by Sajit M Mathews
With the introduction of technology into testing, the speed of development
seems to be increasing, sometimes leading to tunnel vision (Brown and
Duguid, 2000)
Computer assisted language assessment should help researchers to identify
and explore theoretical issues in assessment.
Technology can give unique perspective on two central theoretical issues:
construct definition and validation.
But efficiency should not be the sole goal of research if this has to be
accomplished.
CALT- Computer Assisted Language Testing
CALT seems to be moving fast to the goal of faster and more efficient life as
imagined by technologists.
By 2000, CALT was trying to develop computer adaptive tests (aim: more
efficiency than pen and paper tests)
Brown: CALT uses a delivery algorithm to select items based on prior
experience and performance of the learner. It terminates test based on
consistency in answering. It reduces testing time and maximises efficiency by
scoring based on consistently correct answers.
Alderson (1991) suggested a number of ways in which technologies can
expand the capabilities of language tests by, for example, measuring time and
navigation patterns during test taking, storing and presenting information in a
variety of ways, and analysing learners language.
In the past 20 years (of 2003), efficiency was the major theme discussed.
Bachman (2000) contains a section on language testing: the new task formats
and modes of presentation that multi-media, computer-based test
administration makes possible raise all of the familiar validity issues, and may
require us to redene the very constructs we believe we are assessing.
principles for making inferences based on observed performance
Related questions: how inferences are justied, how test scores are
computed, and how they are used, as well as questions about the best ways
of dening language constructs
theoretical issues in language assessment grow out of practical needs for
developing tests and justifying their use, but are not limited only to those
questions of immediate relevance for particular tests
language assessment theory is theory for practice
Thus, an appropriate starting point is noting practical questions that come up
in computer-assisted tests: how the construct that a test measures is dened,
and the process of validation
Construct denition
Decision about degree of learner choice during test-taking, and scoring
constructed responses depend on the test developers precise definition of
what the test is intended to measure. This is fundamental for test design and
validation.
Test task design: based on what inference is to be made on learners
performance on them
So language abilities that underlie test performance is discussed
Validation research: a form of hypothesis-testing which examines hypotheses
concerning the relationship between test scores and inferences associated
with those scores
centrality of construct denition in language testing
Major questions researchers ask: what language constructs consist of and
perspectives for dening them
A technology-mediated English test is a particularly well-tting straightjacket
for theory because of the range and detail of elements in computer-assisted
test design
Example: whether a reading test should have options to go back and read the
passage while answering questions. The act/competence of going back and
forth will affect the overall test performance.
In pen and paper test, the factor that can be controlled is time. But in
computer-assisted test, there are other factors, and can be affected by the
design decisions made by the designer.
So construct definition is very important.
Decisions about design of procedures for computer-assisted scoring of
constructed response items are also problematic.
Scoring need to take into consideration only a certain parameter that is
measured by the computer- not all. This selection is left to the designer- how
to decide?!!
It is computers that have given us the detailed data of the process of test-
taking. This enables to select the construct elements and evaluate the
performance based (only) on them.
Ex: speaking test: manual evaluation usually gives overall performance a
rating. But computer system can pick out specific elements mentioned in the
construct definition, and score what was being tested.
Validation
Are inferences and use of computer-delivered language tests valid?
The signicance of test method in inuencing examinees performance is well-
documented in the research on second language testing
In this research, it is assumed that any influence of test method on test score
is undesirable.
If there is a difference between scores of pen and paper, and computer-
delivered test, the difference is taken as an evidence that the computer-
delivered test is not valid.
The possibility that language ability for engaging in technology mediated
communication should be expected to be different than what is required for
performance in other registers