Introducing a teachersupportive evaluation system

George Murdoch
This article begins by examining the key principles that underpin a supportive approach to teacher-evaluation. These principles highlight the need to ensure that evaluation plays a part in establishing an institutional concern for teacher development and teacher support. A number of elements are recommended for creating such a system. These elements are described in terms of their functioning in the teacher-performance review system which has been developed for the General Requirements Unit English Program at the United Arab Emirates University. In the final part of the paper, the results of a survey of teachers views on this system are introduced. These suggest that teachers are generally finding the system supportive and effective. It is hoped, therefore, that the description of the system will be of considerable interest to teachers, managers, and administrators in other institutions who would like to introduce more motivating, and more progressive performance review procedures.

Introduction

On quality ELT programs around the globe, a lot of time and resources are regularly spent on such vital activities as revising curricula and teaching materials, ensuring that the evaluation of students progress keeps pace with course developments; providing opportunities for professional development, and introducing new technology. In contrast, teacherevaluation matters are often perceived to be of secondary importance, and as a result, tend to be poorly developed in many institutions. In fact, in many teaching contexts performance review has hardly been systematized. Where it exists at all, it is based largely on irregular observations conducted by an over-burdened director of studies or senior teacher who does not have sufficient time to prepare for, and follow up on, classroom visits. As a result, these observations of classroom teaching performance are often carried out without any strong developmental focus. Such ad-hoc evaluation practices, which are all too often the norm, can only produce universal teacher anxiety, a lack of belief in the validity of observation, and a subtle undermining of other institutional initiatives to support teachers efforts to deliver courses effectively. As Goldhammer (1969) Goldhammer et al. (1980) and Acheson and Gall (1980) emphasize, meaningful discussions of classroom events can only occur when developmental objectives and datacollection procedures have been established. A progressive teacher-evaluation system is, therefore, vitally important to the success of any language teaching program. Many authors have examined particular aspects of teacher-evaluation/development. For

54

ELT Journal Volume 54/l January 2000 Oxford University Press 2000
articles welcome

example, Rea-Dickins and Germaine (1992) discuss extensively the use of self-evaluation questionnaires to encourage a participant-centred approach to structuring observations, while Johnson (1996) advocates the use of portfolios as a teacher-driven assessment tool. However, there seems to be a pressing need for accounts of the fully-developed systems, consisting of multiple procedures and involving the use of many instruments, which have been devised for programs in different institutional contexts. This article is designed to make a contribution to this area by describing one such system which was developed for a major tertiary English language program at the United Arab Emirates University. Key features A progressive teacher-performance review system needs to be founded on five key principles or aims. These will inform both the selection of teacher-evaluation procedures and the ways in which they are implemented. 1. To encourage reflective practice A modern system of evaluation needs to do more than simply ensure that teachers are employing sound teaching methods and covering the required course syllabus. It should, as Bartlett (1990) Schn (1991),Wallace (1991) and others have suggested, encourage teachers to become reflective practitioners. A crucial feature of such reflective practice according to Schn (1991: 40-l) is problem setting: Problem setting is a process in which, interactively, we name the things to which we will attend and frame the context in which we will attend to them. Such a framework for professional action encourages teachers to focus on aspects of their classroom teaching, and to try out new ideas. For example, a teacher might experiment with different types of pre-reading reading activities, or monitor the attention and time given to individual students. The teacher will devise ways of evaluating the success of these initiatives and reflect on their value for students (perhaps by asking a colleague to peer observe and give feedback on a session). The process has clear benefits in professional terms, encouraging teachers to take greater account of student needs.
2. To empower and motivate teachers

An effective system will give teachers an active role in initiating and contributing to the instruments and procedures that are used to evaluate their performance. This means, of course, that the elements of the system must be designed in such a way as to allow teachers the opportunity to become actively involved. Such empowerment can, for example, be achieved by allowing teachers to set their own objectives in relation to those areas of a programs activities that they are most directly involved with-teaching, tutorial services, materials development, testing, etc. Later on they can review their own achievements, and provide feedback on outcomes during performance-appraisal interviews (see McGregor 1992: 287). Another means of achieving this aim is to
Introducing a teacher-supportive evaluation system
articles

55
welcome

involve teachers in the design of instruments, such as questionnaires that they can use to collect data about their students attitudes, preferred learning styles, and reactions to classroom teaching methods. In this way, evaluation mechanisms move beyond the simple monitoring of classroom teaching to develop a full understanding of a teachers wider program work and personal development.
3. To assess all aspects of a teachers professional activity

A common failing of many teacher-performance reviews is that they make judgements about teachers based on unrepresentative samples usually isolated observations - of a teachers work. This situation is unsatisfactory because an observer needs to develop an appreciation of the teachers frame of reference before examining class teaching, and because teachers working on effective programs will be involved in other activities, such as helping to develop independent learning centres, or organizing an in-service training program. A fair system needs to evaluate the breadth of a teachers program-related activities, as well as any other steps that they may be taking to develop professionally, such as updating computer skills, or attending international conferences. This means that data on teachers must be collected from a variety of sources and perspectives. Such sources could include the teachers own action plan, students views on the teachers performance, and reports by colleagues and managers.
4. To take account of students views

If a program is truly committed to supporting students, the evaluation system needs to reflect a student-centred philosophy. One obvious manifestation of a focus on students will be the importance attached to collecting students views about their teacher and the classroom environment. This also makes practical sense, since they are the ones who spend the most time interacting with a teacher. They can therefore comment usefully on such performance points as: the ability of a teacher to establish a rapport with the class; the clarity of explanations and instructions; the variety of learning experiences provided; the use of different groupings, etc. Another concern will be to ensure that teachers regularly collect information from students on relevant topics. These might include their lifestyles, attitudes to learning English, study habits, views on teaching methods used, grouping arrangements, skills emphasis, interest level of texts, reactions to types of tasks being used, etc.
5. To promote collaboration

It is vital for a progressive system to be built on collaborative relations. The relations between a supervisor, senior teacher, or director of studies with the teachers whom he/she evaluates must be built on dialogue (see Gitlin and Smyth 1989: 5). A key role of a supervisor is to aid the teacher to identify developmentally-useful teaching-related foci. This can only occur, however, if the supervisor actively makes efforts to understand the teachers frame of reference on classroom events. In order to tune in on the teachers perspective, a non-dogmatic approach to teaching issues is essential. To be aware of their program involvements, the supervisor
56 George Murdoch
articles welcome

also needs to engage in a continuing dialogue with teachers. Dialogue is therefore vital to successful implementation of all the elements of a teacher-appraisal system, as will be made clear in the following section. Elements
of the system

It is important that the ideas and values discussed above are made explicit when orienting teachers to the elements of a supportive evaluation system. This point is just as relevant to the context of a private language school with a small teaching staff as it is to a large university language program like the one at the United Arab Emirates University which employs around 100 language instructors. For performance review purposes, instructors on the UAEU English Program are organized into teams of twelve, with one supervisor having responsibility for each team. The evaluation system consists of a series of instruments and processes which are implemented sequentially during each 16-week teaching semester. The elements, which are discussed below, are: 1. 2. 3. 4. 5. Teachers action plan Teacher-generated questionnaire Observation and observation conferences Teacher assessment by student questionnaire Supervisors report

1. Teachers action plan The cornerstone of the system is action planning. This process encourages teachers to map out aims, objectives, and planned actions in relation to the overall program and their own professional goals. The plan is expected to cover the key areas relevant to an instructors work on our program: classroom teaching; student support; action research; course development projects; other areas of program involvement, e.g. work in the resource and tutorial centre; more personally-driven interests, e.g. reading journals or improving computer skills. The very process of completing the plan form is seen as professionally valuable because it involves taking stock of duties, interests, short-term objectives and more long-term goals. The content of the plan, worked out in consultation with a team manager, represents a motivating, teacherdriven basis for performance appraisal. A key feature of the action-planning process is that it is dynamic and ongoing. Initial objectives and planned actions may well be modified or revised in the light of new program developments, and the assumption of new responsibilities or changes in the classroom teaching situation. From a developmental/evaluative perspective, it provides an ideal site for a supervisor/director of studies to engage in a significant dialogue with a teacher which will encourage professionalism and support the teachers classroom efforts. Feedback on progress will ensure that the teacher reflects on achievements, and that the supervisor is able to track progress and development from the teachers perspective. In conjunction with the plan and the feedback on it, teachers are encouraged to maintain a professional development folder or portfolio
Introducing a teacher-supportive evaluation system
articles

57
welcome

containing evidence of professional activity. Typical portfolio products might be sample supplementary materials, lesson plans and samples of students work, as well as journal articles and other evidence of professional activity. This folder, which is primarily maintained for the teachers benefit, is also a highly useful means of demonstrating a teachers level of professionalism.
2. Teacher-generated questionnaire on teaching methods

All teachers on the program are asked to gather feedback from their students on their teaching methods early in their courses, and then to adjust their teaching strategies in the light of their findings. To collect data, teachers produce a short questionnaire focused on such key topics as: clarity of the teachers explanations and speech, reactions to different types of activities or grouping arrangements, etc. A concise instrument will ensure that the exercise is not regarded as over-burdensome by teachers, and that the amount of data generated can be easily analysed and reviewed with the supervisor.
3. Observations and observation conferences

Teachers very often feel threatened by the observation experiences-to such an extent that they may have a stultifying effect on their classroom performance and attitudes. The reasons for such negative teacher reactions are not hard to locate. First of all, traditional observation and feedback sessions carried out in close relation to an observers checklist tend to result in a demotivating list of a teachers weaknesses. To make matters worse, they often fail to take into account the teachers perspective and ideas about teaching, which naturally intensifies teacher distrust of the procedure . A progressive system, therefore, needs to find ways of overcoming negative teacher attitudes to observation procedures. One way to make observations more teacher-supportive, even potentially empowering, is to maximize teacher input at all stages of the observation process. An excellent way for a supervisor to signal a concern to tune into the teachers perspective on classroom events is to ask the teacher to suggest a possible focus for the lesson. The pre-observation conference can then be exploited to open a discussion about what data the supervisor should collect for later collaborative analysis. In other words, the teacher and supervisor agree on a role for the supervisor in collecting data and information which will interest the teacher, relate to the teachers view of teaching the class and help him or her developmentally. Examples of such teacher-mediated roles for supervisors have included monitoring the amount of teacher talk, keeping record of the timing/pacing of the parts of a lesson, and monitoring student reactions to different activities and phases of the lesson. As Goldhammer (1969) and Goldhammer et al. (1980) Acheson and Gall (1980), and more recently Wajnryb (1997) have pointed out, the process of providing feedback to teachers needs to be very carefully managed if it is to help them to develop. They all stress the advisability of basing dialogue on concrete events rather than on a supervisors
58 George Murdoch
articles welcome

subjective impressions. If data collected is collaboratively reviewed, it will greatly increase the likelihood of a positive outcome-in terms of a useful dialogue about strategies, and the identification of future foci for lesson preparation/observation, or for action research. Wisely, all these writers caution against a supervisor identifying more than a few weaknesses during post-observation sessions, as excessive criticism is unlikely to help a teacher to develop. They also usefully remind us of the need to reinforce with positive comments those aspects of performance which are successful. Another way of defusing the climate of tension that surrounds the notion of observations is to allow veteran teachers (on our program, teachers who have been teaching for more than one year), to opt for an unseen observation. This variation has all the same elements as a normal observation. The difference lies in the fact that it is the teacher himself or herself who reviews the lesson and presents a report to the supervisor. Even though this procedure involves more written work for the teacher concerned, it is nevertheless proving a very popular option for many of our experienced faculty. The fact that they will not be directly observed by their line manager means that they can be more adventurous in the classroom, since there is not the normal risk of loss of face if the lesson proves to be less than totally successful. Also, as they are more in command of events, they are able to play a more vigorous role in presenting their lesson plan, selecting a teaching focus, and presenting/ evaluating data collected during the post-observation conference. As a result, the dialogue with a supervisor is more valid from the teachers perspective, and any comments by the supervisor may well have more developmental force.
4. Teacher assessment by student questionnaire

In the last quarter of the 16-week semester, a questionnaire is administered to at least one of a teachers classes. The purpose of this questionnaire is to get feedback directly from students on aspects of the teachers performance which they can usefully comment on. The questionnaire covers such basic areas as: a teachers speed of speech; the clarity of his or her explanations; the effective use of groups; the ability to establish a rapport with students; the teachers ability to give attention to all the students, and to make learning interesting, etc. The students respond on a five-point Likert scale which equates with verbal descriptors ranging from agree strongly (5) to disagree strongly (1). They are also given the opportunity to write any other comments that they have about their teacher. The results are analysed by the supervisor to provide average ratings for the different items. The supervisor also examines comments for any points which it may be necessary to share with a teacher. The supervisor monitors results from classes over different courses to see if there is a pattern of low ratings which could usefully be brought to the attention of a teacher. If a negative response pattern emerges related to a particular topic item, such as the ability of the teacher to make
Introducing a teacher-supportive evaluation system
articles

59
welcome

classes interesting, then this could become a mutually agreed area for future professional development.
5. Summary report by the supervisor

A key feature of this system, as discussed above, is that information about a teacher is collected from a variety of sources. This ensures that the teacher appraisal process is seen to be fair, and not based on the views of one individuals classroom observations of a teacher, with all the potential for tension which that could involve. Clearly, it would be just as teacher-threatening for the evaluation to be based on the results from one instrument, such as a teacher assessment by students questionnaire. The different methods of data collection using different sources-project leaders, students, supervisor-ensure that an unbiased picture of a teachers abilities emerges. Very often the data collected during one procedure will provide confirmation of strengths or weaknesses which have been identified via another. For example, during an observation the supervisor may notice that a teacher seems to have established an excellent rapport with students. This finding may be confirmed by positive student questionnaire ratings. Further confirmation may be derived from reports by colleagues in charge of special program areas, such as student support services, about the teachers good work with weak students in tutorial sessions.
Feedback on the system

It is somewhat ironic that one aspect of the teacher-evaluation process which has been rather neglected in the literature is the actual gathering of feedback on teachers views on performance evaluation procedures. In order to evaluate the system developed for the UAE University General Requirement Units English Program, we undertook a study to find out the views of those whom it most directly affects-teachers. A questionnaire (see Appendix 1) was distributed to teachers during our 1998 Spring Semester. The questionnaire was completed by 30 teachers on the Program. The questionnaire covered all the major instruments and processes which comprise our performance review systems. I would like to highlight some significant findings from the survey (see Appendix 2) which will be of interest to those working on other contexts. The second item in the questionnaire deals with action planning: Item 2: The action planning process is a useful way of
encouraging professional growth.

It is interesting that 60 per cent of the teachers actually agreed or agreed strongly with this statement, given the novel demands which this process makes on them in terms of reflecting on and actually documenting their planned teaching and other program involvements, as well as their personal professional development goals. These results seem to indicate that many teachers recognize the value and benefits of action planning. One might anticipate that many teachers would find the requirement to collaborate closely with a supervisor in preparing and reporting on their plans to be an unwelcome imposition. However, judging by the results
60 George Murdoch
articles welcome

for item 3, this is not the case, since 25 of the 30 teacher respondents (83 per cent), agreed/agreed strongly with this item: Item 3: I have found
comments by my supervisor helpful in preparin/evising my action plan.

Further evidence of a positive reaction to the role to a supervisor is provided by the results for two further items. Item 4 in the questionnaire deals with the role of the supervisor in conducting observations: Item 4:
It is helpful to have observations and follow-up discussions conducted by the team supervisor.

Twenty-three of the teachers surveyed (77 per cent) reacted positively to this statement. In Item 10 the focus is on an ongoing dialogue with a supervisor in relation to all the procedures of the system: Item 10: I find
ongoing dialogue with a supervisor about action plans, observations and questionnaires to be a positive feature of the current teacher evaluation/ development system.

Twenty teachers (72 per cent) supported

this statement.

The informal teacher-generated questionnaire was another element on which we were keen to get feedback. This is an innovative aspect of our program, and it was important to see if teachers felt it was valid. Item 7 makes the focus clear: Item 7: It is a good idea to ask teachers to design a
brief, informal questionnaire during the first part of the semester to get some feedback from students on their teaching methods.

Seventy per cent (21 from 30 teachers) concurred with this statement - an unexpectedly strong endorsement of the initiative. Clearly, teachers feel that it is appropriate and interesting to gather feedback from students via a modest questionnaire on their teaching and class management. From an assessment perspective, teachers may also see the value in getting early feedback on how their students are reacting to the course they are delivering, so they can make necessary adjustments before the student assessment of teacher instrument is administered towards the end of the semester. In the light of the above response from teachers, we were somewhat surprised to discover that this student assessment of the teacher questionnaire is in fact the least popular aspect of our system according to the survey. Item 9 targeted this element: Item 9: The formal
assessment of teachers by students towards the end of the semester provides a useful means of gathering summative, standardized feedback about students perceptions of classroom performance.

Only 20 per cent of teachers supported this statement; while 36 per cent were opposed to it and 44 per cent were uncertain. Teachers obviously find this procedure, and the results, to be potentially threatening. It is interesting that this is the one element of the system over which the teacher has least control! The results from Item 12 reveal the most popular elements of the system. These confirm the unpopularity of the student evaluation of teacher (ranked fifth). Action planning was fourth, while interacting
Introducing a teacher-supportive evaluation system 61

articles

welcome

with supervisors was ranked third. The most popular elements were the teacher-initiated questionnaires on classroom teaching (second) and observations (first). It is particularly pleasing that observations are viewed so favourably. This can be attributed mainly to the efforts made, as explained above, to give observations a strong developmental purpose.
Conclusion

The data we have collected suggests that the teacher-evaluation system described in this article is proving generally successful. However, the system was evolved for a particular context at the UAE University. It is not claimed that the model can necessarily be transplanted without modification to other institutional contexts. Nevertheless, it is hoped that the approach presented will be a useful source of ideas for others interested in developing more progressive mechanisms to evaluate their own teachers performance. Our experience indicates that the key to any successful system will be the blending of multiple developmentallyrelevant measures, implemented in a climate of collaborative relations between teachers and those responsible for performance evaluation. A teacher-evaluation system of this type, which is clearly designed to help teachers develop professionally, will certainly have a positive impact both on their morale and on their teaching standards.
Received April 1999 Schn, D. A. 1991. The Reflective Practitioner: How Professionals Think in Action. Aldershot: Vroom, H. V. and E. L. Deci Management and Motivation.

Acheson, K. A. and D. Meredith Gall. 1980. Techniques in the Clinical Supervision of Teachers. New York: Longman. Bartlett, L. 1990. Teacher development through

References

Ashgate.

(eds.). 1992. (2nd edn.).

Goldhammer, R. 1969. Clinical Supervision: Special Methods for the Supervision of Teachers.

reflective teaching in J. C. Richards and D. Nunan (eds.). Gitlin, A. and J. Smyth. 1989. Teacher-evaluation: Educative Alternatives. London: Falmer Press.

Wajnryb, R. 1997 A framework for feedback in

London: Penguin.

A. C. McLean (ed.).
Wallace, M. J. 1991. Training Foreign Language Teachers: A Reflective Approach. Cambridge:

New York: Holt, Rinehart and Winston. Goldhammer, R., R. Anderson, and R. Krajewski.

Cambridge University Press.

The author

1980. Clinical Supervision: Special Methods for the Supervision of Teachers. New York: Holt,

Rinehart and Winston. Johnson, K. 1996. Portfolio assesssment in second language teacher education. TESOL International, Winter 96: 11-14. McGregor, D. 1992. An uneasy look at performance appraisal in V. H. Vroom and E. L. Deci (eds.). McLean, A. C. (ed.). 1997. SIG Selections 97Special Interests in ELT. IATEFL. Rea-Dickins, P. and K. Germaine. 1992. Evaluation. Oxford: Oxford University Press. Richards, J. C. and D. Nonan (eds.). 1990. Second Language Teacher Education. Cambridge: Cambridge University Press.

Murdoch is an English Program team supervisor at the United Arab Emirates Universitys General Requirements Unit in Al Ain. He has recently been involved with professional development and curriculum planning work. He was formerly an ODA/British Council ELT Adviser on the Teacher Education Project in Sri Lanka His career has also taken him to Morocco, Iran, France, Kuwait, Oman, and back to the UK. He has published articles and given presentations on his areas of special interest-teacher-evaluation, curriculum planning, teacher cognition studies, reading and literature in ELT. Email:<time@emirates.net.ae>

George

62

George Murdoch
articles welcome

Appendix

Faculty Performance Review Questionnaire This questionnaire is designed to collect information regarding teachers views on our current faculty performance review system. The information you provide will help efforts to further improve our system. Read each statement below and underline the appropriate response to each question. 1 Please indicate whether in September a) a new faculty member b) a veteran faculty member 97 you were:

2 The action planning process is a useful way of encouraging continuous professional growth. Agree Strongly I Agree I Not Sure / Disagree I Disagree Strongly 3 I have found comments by my supervisor helpful in preparing/revising action plan. Agree Strongly / Agree / Not Sure / Disagree / Disagree Strongly my

4 It is helpful to have observations and follow-up discussions conducted by the team supervisor. Agree Strongly / Agree / Not Sure / Disagree / Disagree Strongly 5 The current number of required observations is reasonable. Agree Strongly / Agree / Not Sure / Disagree / Disagree Strongly 6 I am pleased there is an option for teachers to have an unseen observation after the first year on our program. Agree Strongly / Agree / Not Sure / Disagree I Disagree Strongly 7 It is a good idea to ask teachers to design a brief, informal questionnaire during the first part of the semester to get some feedback from their students on their teaching methods. Agree Strongly I Agree / Not Sure I Disagree I Disagree Strongly 8 It is more helpful to have teachers discuss the results of the informal questionnaire with the supervisor rather than to submit the questionnaire and a written report without such discussion. Agree Strongly / Agree I Not Sure / Disagree / Disagree Strongly 9 The formal assessment of teachers by students towards the end of the semester provides a useful means of gathering summative, standardised feedback about students perceptions of a teachers classsroom performance. Agree Strongly /Agree / Not Sure / Disagree / Disagree Strongly 10 I find ongoing dialogue with a supervisor about action plans, observations and questionnaire results to be a positive feature of the current teacher evaluation/development system. Agree Strongly / Agree / Not Sure / Disagree / Disagree Strongly 11 The current system is better than the one in which teacher evaluation is based mostly on classroom observations and committee work reports. Agree Strongly / Agree / Not Sure/ Disagree / Disagree Strongly 12 Tick the elements of the current system listed below that you feel are proving effective (tick as many items as you want): 0 action planning maintaining a professional development portfolio 0 informal feedback on class teaching (teacher-initiated) 0

Introducing a teacher-supportive evaluation system

articles

63
welcome

0 formal assessment of teachers by students at the end of the semester

0 0 seen/unseen observations dialogues with supervisor related to the above instruments/processes.

Use the space below to write any suggestions you have for changing or modifying our current faculty evaluation system to make it even more effective (attach additional pages if necessary).
THANK YOU FOR COMPLETING THE QUESTIONNAIRE!

Appendix

30 teachers completed the questionnaire

question number 2 3 4 5 6 7 8 9 10 11 agree strongly 10% 23% 20% 20% 40% 33% 37% 3% 22% 32% agree 50% 60% 57% 63% 50% 37% 30% 17% 50% 29%

(Q l results)
not sure 14% 7% 13% 10% 10% 17% 10% 44% 14% 32% disagree 13% 7% 7% 7% 0% 13% 13% 23% 11% 4% disagree strongly 13% 3% 3% 0% 0% 0% 10% 13% 3% 3%

Q12 (selecting elements of system than proved to be effective) Results in rank order of most popular elements: 1 2 3 4 5 seen/unseen observations informal feedback on class teaching (teacher-initiated) dialogues with supervisors related to instruments/processes maintaining a professional development folder formal assessment by teachers of students.

64

George Murdoch

articles

welcome