Brandon McLeod

TSL 633 Testing and Assessment in ESL/EFL

Dr. Negueruela-Azarola
30 June 2015
Position Statement
Research-based testing and assessment methodology is integral to educational process. If
a program has effective, authentic, reliable, and valid testing procedures, there are benefits on an
instructional level and an institutional level. Since assessment is something which has widereaching effects, it is important that those involved, including the teachers, students, and
administrators, all work together to ensure that proper assessment practices are being used.
However, it seems to be the case that while the research in testing and assessment
methodology has yielded fantastic insights into effective assessment methods that are authentic,
reliable, and valid, testing and assessment procedures remain slow (and often reluctant) to
reform. In many institutions, the testing procedures are seemingly haphazard. Testing practices
change frequently between levels (and even between teachers of the same level or between tests
in the same class). In this kind of a situation it is rather difficult to ensure that the assessments
that are being used are beneficial to anyone.
Moreover, assessments are rightly regarded as important indicators of student
achievement and proficiency. Despite this importance, assessments that are regularly used in
many programs are woefully inadequate for testing proficiency (achievement tests are typically
given greater attention and, thus, are often more reflective than proficiency tests). It is important
to realize as well that, though achievement tests may be more common, they are based primarily
on content validity and are often not indicative of student proficiency (which would be assessed

McLeod, Brandon 2
through an exam with construct validity). That being said, content validity is of significant
importance and should not be disregarded in favor of construct validity. They work hand-in-hand
along with consequent validity, curriculum validity, test-retest reliability, and inter/intra-rater
reliability (see Savignon and OMalley). For assessments to be truly reflective of both
proficiency and achievement, they must be valid and reliable according to those metrics. When it
comes to implementing assessment reform, however, there are a few issues that profoundly
impact the process. The first is washback. The second is that many instructors and administrators
are unwilling to change. The third is that research-based assessment methodologies are either not
well known or are inadequately understood.
Washback is something that has been plaguing the testing and assessment field for a long
time. When instructors start altering the content of the course to match the assessment outcomes,
they almost invariably end up teaching to the test. This disrupts curriculum outcomes as well as
student proficiency. There is, of course, positive washbackwhich would be encouraging the
students to study or practice morebut it is typically in the form of teachers (and even
administrators) sacrificing various aspects of instruction to try and improve test scores. An
additional issue that compounds the problem of washback is that, frequently, the tests are highstakes, but may not have construct or content validity. It is not uncommon to hear students
complain about having to write essays for the TOEFL-iBT on a subject that they are totally
unfamiliar with. While the test is intended to assess the students writing abilities, it ends up
testing their content knowledge instead. Additionally, these types of high-stakes tests are usually
standardized multiple choice exams. Multiple choice exams run the risk of low test-retest
reliability. If the students are able to guess correct answers, they could achieve scores much

McLeod, Brandon 3
higher than they should get. Washback is a serious problem, but before it can be addressed, the
tests themselves must be reformed.
When it comes to the process of evaluating old tests and creating new authentic, reliable,
and valid assessments, one of the biggest obstacles is the teachers themselves. As pointed out in
Green and Andrades (2010) article, teachers are often quite attached to the tests they have
createdeven if they are evaluated to be invalid or unreliable. It is not just sentimental
attachment either. The test reform process is long and rigorous. It requires a significant
commitment from all the teachers, administrators, and students. The university in Green and
Andrades study spent nearly four years revising their assessments. The process included a
significant amount of research into their current practices, multiple training sessions, evaluation,
collaboration, testing, and reevaluation. Moreover, since many teachers now are parttime/adjunct, there is the added difficult of compensation. Many departments do not have the
funding for compensating these instructors for their extra time, but, if the reform process is to go
smoothly and be successful, it requires the input of all of the instructors. If the instructors are
involved in the process of creating the new assessments, they will be invested in the final
products.
The final issue that complicates the process is lack of training/professional development.
It is not uncommon for teachers to use terms like authentic, formative, alternative, etc. (see
Omaggio). However, these terms are frequently misused. In many programs, the assessments,
even if they are being called authentic, or resemble authentic assessments, they are variations of
old invalid or unreliable tests, usually lacking in construct validity and test-retest reliability. To
combat this issue, there need to be concerted efforts by teachers and administrators to attend and
to offer professional development. If the teachers are unaware that the testing methodologies

McLeod, Brandon 4
being used are inadequate, then it is unrealistic to expect them to change. Before assessment
reform can happen, awareness must be raised.
In the end, testing and assessment has made great strides in recent decades.
Unfortunately, due to the issues or washback, resistance to reform, and lack of
education/understanding there is still significant progress that needs to be made. For this to
happen, it is up to everyone involved in the assessment process to work together to consistently
examine the testing procedures that are currently in place, evaluate them according to strict
assessment rubrics, and collaborate in the creation of new materials that are beneficial for
students, teachers, and administrators alike.

References
Beatty, K. (2013). Leap advanced listening and speaking. Saint-Laurent, QC: Pearson.
Green, B. and Andrade, M. (2010). Guiding principles for language assessment reform: A model
for collaboration. Journal of English for Academic Purposes, 9, 322-334.
http://dx.doi.org/10.1016/j.jeap.2010.06.003
Omaggio. (n.d.). Chapter 9. Classroom Testing.
OMalley. (n.d.). Chapter 2. Designing Authentic Assessment. Portfolio Assessment
OMalley. (n.d.). Chapter 3. Portfolio Assessment
Savignon. (n.d.). Chapter 6. Testing.