Action research in education: a review and a case report from operative dentistry

Hak-Kong Yip, BDS, PhD, MEd Faculty of Dentistry The University of Hong Kong Hong Kong Rita Y.P. Chan, BEd, PhD, MA Centre for the Advancement of University Teaching The University of Hong Kong Hong Kong Abstract Action research has long been used in social and educational research and is increasingly being applied to the education of would-be professionals in university departments. This paper provides an overview of the historical development of action research, a range of approaches it covers and common characteristics of these approaches. To illustrate the key elements of action research, an example is drawn from operative dentistry. The steps taken by a lecturer to improve a pre-clinical course in operative dentistry are described and the outcomes of the action are reported and evaluated. The findings positively support the suitability of action research for dental education and the kinds of problems encountered by teachers using its methodology. From their experience within the project, the writers would support the claim that the ability of the investigator to reflect critically on the area researched is an integral component of action research. Introduction Higher education institutions in Hong Kong are working hard at coming to terms with an improvement drive powered by quality assurance measures linked to funding formulae. Concerns about quality stem predominantly from the rapid expansion of higher education in recent years and from shifts in professional and occupational needs. These forces have triggered a critical examination of the effectiveness of established tertiary education programmes and have prompted universities to rethink and improve instructional strategies in the light of new possibilities and challenges. The University of Hong Kong, for example, is pushing for curricula that emphasise self-directed learning and transferable skills, and for replacing didactic topic-centred teaching with interactive problem-centred learning. By awarding teaching development grants, the government has affirmed to universities the expectation that academic staff will investigate and where necessary improve their teaching. This paper explores the use of action research as a strategy for meeting this objective. Approaches to teaching development in higher education Training courses, workshops, professional development seminars and orientation programmes feature widely in higher education as means of helping academic staff improve their teaching. Academics attend such programmes for a variety of reasons, including

personal interest, instructional need, contract requirements, certification and simply as a type of inoculation precaution. One thing is certain about these programmes: the institutions providing them are not always certain about their impact on participants teaching. More recently, a distinct emphasis has been placed on strategies focusing on the teaching of academic staff, for example through classroom observation, peer consultation and projects. The latter, projects, seems to be increasingly favoured. This probably has something to do with the inquisitive and creative disposition of academia. Academic staff are encouraged to try out new ideas, create new materials, experiment with new methods and investigate critical aspects of their teaching with a view to improving its effectiveness. Exciting as these alternatives may seem, they do not necessarily lead to more effective teaching or learning unless they are intelligently conceived, skillfully delivered, carefully monitored and critically evaluated. Initiatives need to be organised within a framework that addresses these quality issues. Action learning and action research provide one such framework and they are now widely employed in the UK and Australia in teaching development in higher education. Action learning Action learning, put simply, is learning from ones actions. In the teaching development literature, it is usually defined as learning from working with concrete problems in the immediate situation with the support of colleagues (Zuber-Skerritt, 1993; McGill and Beaty, 1992). The process is often described to consist of the processes of taking action towards a phenomenon in a situation, observing and reflecting on the consequence of that action, and planning of a new action based on the new understanding generated about the strategies applied and the situation. When the new action plan is implemented, another learning cycle begins. Throughout the action learning process, the action taker is expected to interact with those colleagues associated with the undertaking. In Hong Kong action learning is slowly attracting the attention of academic staff through the establishment of the Action Learning Project based at the Hong Kong Polytechnic University. The project is substantially funded by the University Grants Committee (UGC) to support action learning projects across the universities. Fifty projects were completed in the first round of the mother project and another fifty or so are ongoing. Evaluation of the first round is on the whole positive. Most participants found the experience worthwhile and gained useful insights into the effectiveness of their teaching (Kember et al., 1997). Some of the teaching development projects supported by the UGC have been of the action learning type. It is worth mentioning that academic staff do not always embark on action learning fully aware of what they are supposed to do. The Action Learning Project takes a developmental approach right from the start. It helps academic staff to formulate project proposals and throughout the period of the project provides specialist advice over the monitoring of projects. It even assists in the preparation of the final report. Very often, academic staff find that action learning is substantially different from the kinds of learning usually experienced in their particular discipline. And they often fail to appreciate the methodological rigour of action learning beneath its apparent simplicity. Action research Action research is an established research method in education and the social sciences. It has been applied to the study of processes in schools, social groups and organisations. The term action research was first coined by the social psychologist Kurt Lewin in the United States

half a century ago. He used the term in connection with research that simultaneously aimed to advance theory and social change through democratic decision making and the active involvement of practitioners in the research process. Subsequent work has followed this tradition by involving practitioners in diagnosing problems, designing and implementing actions. The rationale is simple: those people who experience them best understand the problems. Under Lewins influence, many educationists adopted the action research approach and Lewin himself worked on action research programmes with teachers. However, in spite of the interest attracted in various fields of social inquiry, its methodology, effectiveness and practicality were criticised. Interest declined in the late 1950s only to re-emerge in the 1970s in a different guise, largely under the influence of Stenhouse (1975). The negative reaction to action research is understandable in that it is very different from the natural science model favoured by educational researchers and based on the research paradigm widely used in the physical sciences. It also differs from the interpretative research model which seeks to discover and interpret the perspectives of participants in the educational process. Action research critically departs from these research models in that it sets out to promote change in specific situations rather than to discover truth and derive general laws. The involvement of the researchers in the process of inquiry also prevents them from playing the role of neutral observer, isolated from the phenomena under investigation. During its re-emergence in the 1970s, action research took on a different kind of rationale than its original form. Although Lewins early work emphasised fieldwork, it avoided relinquishing the scientific rigour of traditional research in the social sciences. Under the influence of curriculum theorists such as Stenhouse, Schwab, Elliott and Skilbeck, emphasis shifted to the idea of practical deliberation, focusing on human interpretation, negotiation and detailed descriptive accounts in place of measurement and statistical analysis. With this trend came the assumption that the enquiry processes must develop naturally rather than be constrained by preconceived ideas. Hence the expectation that there may be a continuing number of cycles of enquiry. The third development in action research revolves around the work of Kemmis and his colleagues at Deakin University in Australia. These writers recognise the goal of action research as primarily political. Action research frees the individual from traditional ways of thinking that impede effective development and communication. It empowers practitioners to fight for more rational, just and democratic forms of education (McKernan, 1991, p.27). Current understandings of and reactions to action research can be traced back to the developments described above. Cohen and Manion (1989) have provided a succinct description of the approach: Action research is situational ! it is concerned with diagnosing a problem in a specific context and attempting to solve it in that context; it is usually (though not inevitably) collaborative ! teams of researchers and practitioners work together on a project; it is participatory ! team members themselves take part directly or indirectly in implementing the research and it is selfevaluative ! modifications are continuously evaluated within the ongoing situation, the ultimate objective being to improve practice in some way or other (p.217).

Turning to implementation, action research usually proceeds in a spiral of learning cycles consisting of four basic activities as in action learning: planning, acting, observing and reflecting. Usually, there is an initial reflection stage where problems are analysed and tentative ideas about possible solutions are preliminarily identified. These ideas are then translated into action plans, the implementation of which is closely monitored through various data collection procedures. Finally, the effectiveness of the action and its outcomes are evaluated and further improvements are planned. Action research versus action learning Some advocates of action learning and action research seek to distinguish between the two. The emphasis of action learning is on the reflective process whereas that of action research also includes its outcome. Action research is usually seen to include action learning but is generally regarded as needing to be more systematic and rigorous. Another distinction concerns the beneficiaries: action learning focuses on the learning of participants whereas action research targets public learning (Zuber-Skerrit, 1993; Bunning, 1994). Action research often leads to publications but this is not necessarily the case with action learning. Nevertheless, it should be noted that action learning is now also widely disseminated in books, journals and newsletters. For the practitioner, the choice of one approach over the other will depend very much on the objective and the resources available. For example, if the aim is to seek specific knowledge about specific problems in a specific situation, practitioners may need a more methodical approach and the co-operation of people around them. The above account offers a tidier view of action research than what often occurs in reality. Planning is seldom perfect; action invariably reveals the need for further planning; backtracking takes place; and the entire cycle often has to be repeated. Action research in the hands of the inexperienced is likely to be less systematic and rigorous. Conversely, action learning that has gone through several cycles may yield a level of rigour and sophistication that could only have been generated by action research. Academics usually feel more comfortable starting with an exploratory, small-scale undertaking, refining strategies as they go along and moving onto more formal and intensive enquiries as necessary and feasible. Reliability and validity of action research Greenwood (1984) and others have reviewed reliability and validity procedures in action research studies. According to Elliot and Ebbutt (1983), test-retest consistency is not the overriding consideration of researchers carrying out qualitative case studies. The actions taken during the research often preclude the researcher from reproducing exactly the steps taken with the same sample in order to administer commonly applied reliability checks. Elliot and Ebbutt argue that the responsibility for showing external validity lies with those who wish to use the studys findings. Here, the researcher will seek to reproduce a naturalistic regeneration of what happened in the initial project. With the case study approach usually entailed in action research, the emphasis is on collecting multiple interpretations. Although control and experimental groups are not involved, comparisons are made within the case study by analysing participants behaviour, attitudes and knowledge change. Willmott (1975) considers that, ideally, a balance should be struck between as precise a form of measurement as possible on the one hand, and a qualitative exploration and description on the other. Greenwood (1984) argues that face validity is the most important safeguard for ensuring that the findings of action research fit reality, and that reliability can be checked by inspecting the level of agreement in the interpretations of researchers and participants. Although some of the

findings may be generalisable to similar settings, the fact that findings cannot be generalised in their entirety is not a problem since unique situations are being studied. Action research in dental education: an example from operative dentistry Although action research has taken place in clinical medicine and medical education (e.g. Sandow, 1979; Medio et al., 1984; Eizenberg, 1995), its application in dental education is not as widely known. The first author, a member of a university dental department, was aware of a lack of interest in such educational enquiry among his colleagues. This disinterest is partly related to the nature of the work of the department. Academics teaching professional education courses such as dentistry typically have to undertake teaching and research alongside professional service. In the faculty where the first author works, teaching accounts for only 50% of a lecturers workload. Under the pressure of research and professional service, staff can afford little time to be totally involved with teaching matters. Additionally, with only a few full-time members of staff in the department, some of the teaching has to be delegated to part-time staff, with the result that interaction among teaching staff is reduced to a minimum. Thus, although given the benefit of a small department, communication and collaboration is not as easy to achieve as one might expect. Four years ago, the first author was in just such a situation when he felt the need to do something about his departments teaching. He resolved to make a start on his own and to try to involve colleagues as he went along. Through the work undertaken, colleagues were encouraged to consider departmental curriculum development. His experiences are reported below. They give a flavour of the realities faced by academics trying to apply action research in their teaching. Evaluating and redeveloping a pre-clinical course in operative dentistry Informal feedback from students during 1994-95 on a pre-clinical operative technique course (a phantom head laboratory practice course) aroused the first authors concern about the effectiveness of the dental programme. The author, an experienced lecturer in operative dentistry, decided to take steps to investigate and improve the course. A questionnaire was developed with open and closed questions to evaluate the courses effectiveness: six questions on course content and eight on teaching effectiveness. These were then transformed into statements and students asked to indicate their agreement or disagreement with each statement. There was an open comment section to allow students to comment on aspects not raised in the questionnaire. All the second-year dental students were invited to attend a halfhour evaluation session. Twenty-two of the 48 students attended the session and completed the questionnaire. Students responses were collated and each item was scored by allocating a weighting of 5 to 1 from Strongly agree to Strongly disagree. The results are summarised below.

Table 1: Summary of responses to the 1994-95 questionnaire Course Content

I understood the subject matter well The course was poorly coordinated I found the course challenging Course materials were well prepared I learned something valuable Recommended readings contributed to my understanding of the course SA 0% 5% 5% 0% 43% 9% A 32% 27% 81% 64% 43% 23% N 50% 50% 10% 32% 14% 55% D 18% 18% 5% 5% 0% 9% SD 0% 0% 0% 0% 0% 5% Count 22 22 21 22 21 22 Mean score 3.14 3.18 3.68 3.59 4.09 3.23 S.D. 0.69 1.01 0.56 0.56 0.70 0.90

Teaching Effectiveness
Teachers were effective communicators Teachers were enthusiastic about teaching the course Teaching style held my interest Teachers gave clear, lucid explanations Lecture/seminar style made note-taking difficult Teachers stimulated my interest in the subject Teachers used OHP (and/ or blackboard) well Teachers were well organised

SA 0% 5% 0% 0% 14% 5% 0% 0%

A 36% 41 % 23 % 23 % 38 % 27 % 23 % 41%

N 50 % 41 % 64 % 64 % 29 % 64 % 68 % 45 %

D 9% 14 % 9% 9% 19 % 0% 0% 14 %

SD 5% 0% 5% 5% 0% 5% 9% 0%

Count 22 22 22 22 21 22 22 22

Mean score 3.18 3.36 3.05 3.05 3.00 3.23 3.05 3.27

S.D. 0.78 0.77 0.71 0.71 0.96 0.75 0.77 0.78

SA: Strongly agree

A: Agree

N: Neutral

D: Disagree

SD: Strongly disagree

As can be seen in Table 1, in relation to course content, with the exception of course coordination, more students felt positive about the course than did not. The majority of the students found the course challenging and reported having learnt something valuable, even though the subject matter might have been rather difficult for some students. On the whole, students did not seem to have strong views about the course, as can be seen from the mean scores for each item. However, 50% or more of the students took a neutral position towards half of the items. Turning to teaching effectiveness, with the exception of one item, there were again more positive than negative responses. A substantial proportion of the students remained neutral about the statements, to a slightly larger degree than was the case with course content statements. This might be a reflection of a measure of caution on the part of students asked to comment on the performance of their teacher. It might also be a reflection of the limitations of the five-point Likert scale for collecting respondents precise opinions. Other than drawing attention to the quality of course co-ordination and an aspect of teaching style that had to do with note-taking, responses to the closed questions gave little indication

about how the course could be improved. A number of concrete suggestions about the course were nevertheless recoverable in the open comment section: The theoretical and the practical cases were not matched, so the skills were difficult to master and more co-ordination in the subject matter was required. More time needed to be spent on practical exercises rather than on directly treating reallife patients, since a lack of experience made students feel nervous. More practical and clinical sessions were requested as they provide more practice. More emphasis should be placed on (a) practical situations and (b) patient management. At a staff meeting called by the first author (who was co-ordinating all second year courses) to discuss feedback on the years work, the students responses were revealed to colleagues. Particular attention was drawn to the students requests for more practical work and for better integration of theory and practice. This succeeded in drawing the attention of staff to the practical sessions and a number of suggestions were made. For instance, it was suggested that the materials used for practical teaching be revamped; that video demonstrations be used in an attempt to standardise teaching; that work be commenced with the Dental Materials Science Team to develop a new teaching curriculum; and that the relevance for practice of the issues covered in lectures be emphasised more strongly to students. However, these suggestions went only so far in addressing all the issues raised by the students. These recommendations and others were subsequently presented at the departmental staff meeting and a decision was made to revise the whole second-year curriculum. The head invited staff to submit proposals. The first author considered the findings of his evaluation and the comments of colleagues at the staff meeting and revised the curriculum with a personal input of teaching materials. While facilities and resources were limited for technology-based teaching, attempts were made to collaborate with the departmental head to develop audio-visual aids for the teaching of contemporary operative techniques which were not widely presented in standard texts. The experiences of colleagues who had previously had teaching appointments in places other than Hong Kong were also solicited. A revised course syllabus and timetable were submitted to the department for discussion and approval. Course materials were subsequently distributed centrally through the department. In setting up the new course, the lecturer visited other departments (a) to find out what was being taught and (b) to tell them about the new course so that overlap of topics could be brought to a minimum. Such communication was also necessary to prevent clashes of sessions and to assess the availability of dental auxiliary staff. The process of collecting students perceptions about teaching and responding by producing a revised course was not pain-free. However, the data from the course evaluation were extremely useful for initiating and maintaining a dialogue between students and staff. The findings also gave the first author impetus for learning about curriculum development and for not keeping the business in-house. Consultations with staff from the Centre for the Advancement for University Teaching helped to broaden the discussion, and alerted the first author to the fact that academics from a range of subject departments were also interested in improving teaching and professional learning. Course evaluation 1995-96 Subsequent to its implementation the questionnaire designed in 1994-95 was found to be inadequate in a number of areas. Following consultations with staff from Sheffield University with expertise in the design of questionnaires, some questions were reformulated and others

added so as to obtain a more comprehensive understanding of students reactions to teaching and learning. The neutral answer in the five-point scale was also taken out, thereby converting it to a four-point continuum. This time a selection of students and teachers were also interviewed so that further qualitative feedback could be obtained about the revised course. The total of 54 students were targeted by the second survey and only 6 did not respond. The number of respondents was thus 48 (88.9%), a much higher response rate than in the previous years study. The results are summarised in Table 2. Table 2: Summary of responses to the 1995-96 questionnaire
Statements 1. I understood the subject matter 2. The course was poorly co-ordinated 3. I found the course challenging 4. Course materials were well prepared 5. I learned something valuable 6. Recommended readings contributed to my understanding of the course 7. Reading materials were useful 8. Teaching materials were useful 9. Teachers were effective communicators 10. Teachers were enthusiastic about teaching 11. Teaching style held my interest 12. Teachers gave clear, lucid explanations 13. Teachers explained concepts well 14. Lecture/seminar style made notetaking difficult 15. Teachers stimulated my interest in the subject 16. Teachers encouraged me to participate in class 17. Teaching improved ability to think 18. Teachers used English well 19. Teachers used OHP well 20. Teachers were friendly 21. Teachers were approachable 22. Teachers were well organised 23. Teachers were confident 24. Teachers were self assured 25. Teachers were fair in grading 26. Teachers were available outside class SA 6.3% 0.0% 12.5% 6.3% 22.9% 4.2% 6.3% 12.5% 8.3% 6.3% 4.2% 4.2% 6.3% 6.3% 6.3% 6.3% 4.2% 6.3% 0.0% 6.3% 4.2% 2.1% 6.3% 2.1% 0.0% 0.0% A 89.6% 18.6% 79.6% 83.3% 70.8% 64.6% 75.0% 81.3% 79.2% 79.2% 58.3% 66.7% 70.8% 43.8% 70.8% 43.8% 68.8% 93.8% 64.6% 70.8% 70.8% 77.1% 72.9% 70.8% 68.8% 52.1% D 2.1% 79.2% 6.3% 10.4% 2.1% 27.1% 14.6% 6.3% 12.5% 12.5% 27.1% 27.1% 20.8% 47.9% 22.9% 47.9% 25.0% 0.0% 33.3% 22.9% 25.0% 18.6% 18.6% 18.6% 25.0% 35.4% SD 0.0% 2.1% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 4.2% 0.0% 0.0% 2.1% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 2.1% 2.1% 2.1% Count 47 48 47 48 46 46 46 48 48 47 45 47 47 48 48 47 48 48 47 48 48 47 47 45 46 43 Mean Score 4.01 2.35 3.36 3.85 4.20 3.48 3.76 4.00 3.83 3.81 3.33 3.49 3.64 3.35 3.60 3.56 3.53 4.06 3.32 3.60 3.54 3.64 3.60 3.60 3.41 3.14 S.D. 0.39 0.08 0.62 0.63 0.54 0.95 0.91 0.95 0.75 0.73 1.07 0.94 0.90 1.12 0.91 1.09 0.95 0.24 0.95 0.92 0.92 0.81 0.86 0.89 0.96 1.04

As can be seen in Table 2 (items 1-8), respondents found the course to be well co-ordinated (81.3%), it helped their understanding (95.9%), was challenging (92.1%) and course materials were well prepared (89.6%). These elements enabled them to learn something valuable (93.7%). The recommended reading contributed to their understanding of the course (68.8%), reading materials were useful (81.3%) and teaching materials (including slides,
8

transparency demonstration and use of audio-visual equipment) were also helpful (93.8%). The remaining eighteen questions were set in an attempt to evaluate the effectiveness of the teaching of members of staff, including part-time general dental practitioners and full-time teachers. The respondents considered teaching staff to be effective communicators (87.5%), enthusiastic about teaching (85.5%), their teaching style held their interest (62.5%), teachers gave clear, lucid explanations (70.9%), explained concepts well (77.1%), stimulated interest in the subject (77.1%), were friendly (77.1%), well-organised (79.2%), confident (79.2%), self-assured (72.9%) and fair in their grading (68.8%). They also improved the respondents ability to think (73.0%). Half of the respondents indicated that the teaching styles adopted made note taking difficult. This was disappointing since the first author had provided comprehensive lecture and seminar notes, featuring details of the topics covered together with a reading list for every topic. Half of the students considered that staff encouraged them to participate, ask questions and express ideas. About half of the class (52.1%) thought teaching staff were available outside class. Responses given in the open comment section include a list of problems students perceived of the course: inconvenient lecture schedules; different tutors adopting different teaching styles and using different standards in grading; inconsistency in the handling of some topics between tutors; and unfriendly attitude of some teaching staff. There are also requests for a more clinically-oriented course, with more real teeth for practice; more teacher supervision during clinical sessions; more clinical and fewer operative technique sessions; and better preparation of students for dealing with real patient problems. Students interview 1995-96 The 54 second-year dental students groups had been randomly divided into 6 mixed gender groups to allow small group teaching. After the questionnaire had been administered, a student was randomly chosen from each group and an interview conducted about the course. Out of the six interviewees, three agreed to the interviews being tape-recorded. No independent observer was present in the interviews. Immediately prior to the interviews, interviewees were shown the student questionnaire again. The views expressed in the interviews were very similar to those given in responses to the questionnaire for most items. However, the fuller replies given by the interviewees when leading questions were asked indicated a degree of ambiguity in some of the items on the questionnaire. Five of the six interviewees gave positive and favourable comments on the course, despite the general feeling that the course was difficult. One interviewee made a number of criticisms and these are considered further in the next section. The complaint of a lack of consistency in grading perhaps stems from the large number of part-time teachers involved in the teaching. Despite being issued with a grading system, different criteria for grading may have been used. Teachers interviews 1995-96 These were undertaken in an effort to investigate the reactions of fellow university teachers to the research strategies taken and to the outcomes that emerged. Three teachers involved in teaching the course were interviewed, questions asked being based on the statements in the questionnaire. The teachers were all part-time and experienced in teaching various years of the undergraduate dental course. The teachers did not want the interview to be recorded, so transcripts were prepared from notes made during each interview. No independent observer

was present in the interviews. The overall comments favoured the new course arrangements. The attitudes of the students were considered to be satisfactory, but one teacher was critical of the students treatment planning, case presentation and clinical performance. In connection with teaching, suggestions were made about increasing the number of teaching staff to permit closer supervision of the students, particularly repeat and academically weak students. Teaching Staff Meeting 1995-96 Just prior to the annual departmental staff meeting a second year course meeting was held. The recommendations emanating from the second year teaching staff meeting were as follows: to make the second year curriculum less congested; to use video demonstration as a vehicle for self-instruction, in pinned and adhesive amalgam; to update and standardise teaching slides used in the teaching of minimal cavity preparation; to record lecture attendance despite the fact that university regulations do not stipulate compulsory attendance; to implement an interim practical quiz during the term (contrary to student comments about the course); to match the time schedule of teaching to that of the Dental Material Science Course to minimise teaching overlap; to further standardise all dental materials used in the department and to use newer versions of dental materials; to develop the curriculum in order to encourage students thinking and to improve the quality of students academic learning by using a problem-based approach in teaching. A comparison of the results of the 1994-95 and 1995-96 course evaluations A comparison of the mean scores of respondents to statements on the 1994-95 and 1995-96 questionnaires was made to obtain some idea of the impact of the course revision. Only the items that appeared in both evaluation events were compared. Statistical analysis was applied using Students t-test for independent samples (program Instat2TM, GraphPad Software, San Diego, CA, USA). The results are presented in Table 3. As can be seen in Table 3, there was a universal improvement in the mean scores for the 1995-96 evaluation compared with those for 1994-95. Statistically significant differences were recorded in the areas of the students perception of their understanding of the subject matter and the co-ordination of the course. Although the differences did not reach statistical significance, the trend was for the course to be seen as less challenging, better prepared and rewarding in terms of knowledge growth. Turning to statistically significant differences in students perceptions of teaching, staff were thought to have improved their ability to communicate, were more enthusiastic about their work and were better organised. Although the differences did not reach statistical significance, the trend was for staff to be better able to hold the interest of students and explain matters more clearly. Staff were able to stimulate interest in the subject and seen to use visual aids well. However, the students thought that the changed teaching style made note-taking harder.

10

Table 3: Comparison of the mean scores of respondents to statements on the course evaluations for 1994-95 and 1995-96
Course Content 1994-95 (mean and S.D.) 3.14 (0.69) 3.18 (1.01) 3.68 (0.56) 3.59 (0.56) 4.09 (0.70) 3.23 (0.90) 1995-96 (mean and S.D.) 4.01 (0.39) 2.35 (0.80) 3.36 (0.62) 3.85 (0.63) 4.20 (0.54) 3.48 (0.95) p value of t

I understood the subject matter Course co-ordination was very poor I found the course was challenging Course content was well prepared I learned something valuable Recommended readings contributed to my understanding of the course Teaching Effectiveness

<0.0001 <0.0005 n.s. n.s. n.s. n.s.

1994-95 (mean and S.D.) 3.18 (0.78) 3.36 (0.77) 3.05 (0.71) 3.05 (0.71) 3.00 (0.96) 3.23 (0.75) 3.05 (0.77) 3.27 (0.78)

1995-96 (mean and S.D.) 3.83 (0.75) 3.81 (0.73) 3.33 (1.07) 3.49 (0.94) 3.35 (1.12) 3.60 (0.91) 3.32 (0.95) 3.64 (0.81)

p value of t

Teachers were effective communicators Teachers were enthusiastic about teaching the course Teaching style held my interest Teachers gave clear, lucid explanations Lecture/ seminar style made note-taking difficult Teachers stimulated my interest in the subject Teachers used OHP (and/ or blackboard) well Teachers were well organised

<0.001 <0.05 n.s. n.s. n.s. n.s. n.s. <0.05

(n.s. = not statistically significant) It has to be admitted that there are weaknesses in the research. The differences in the scores may merely reflect the spontaneous attitudes of different student cohorts. They may also reflect an awareness among the second group of students that extra effort was being placed into changing their views of the course. Nevertheless, the outcomes were not deliberately engineered and the writers feel confident that they do reflect genuine changes.

11

Concluding discussion Dental lecturers use a variety of criteria for gauging the effectiveness of their teaching and for evaluating courses. Chief among these is the quality and professionalism of the newly qualified dentists graduating from the course. This reflects the quality and professionalism of the teachers and the course undertaken by students. The impact of the teaching can be enhanced and students taught better if staff are more aware of current advances in teaching, educational issues, students learning and curriculum development. Such insights are more likely if education experts are used as sounding boards for ideas. When teachers are sensitive to the impact of their teaching on students, they not only focus on themselves as teachers but on themselves as learners. The action research reported in this paper, summarised in Fig.1, drives home the lesson that teachers in higher education need to consider both the professional content of their teaching as well as how this is presented to students. Just when further improvements were scheduled for existing courses, the decision was made in the faculty to move to problem-based learning. The existing dental curriculum is to be transformed in stages and it has been concluded that all teaching staff will benefit from involvement in curriculum development. At the request of members of the department, to facilitate the process special staff development workshops have been held by CAUT. The ideas derived from the project have been generally incorporated in the new curriculum. This has provided the motivation for another spiral of action learning on a slightly larger scale which is now in progress. It may be concluded from this small-scale study that action research is a useful strategy for improving teaching and learning and for promoting teachers interest in professional learning. Action research is a constructive approach to dealing with social issues. By definition it involves interventions aimed at improvement. One intervention lays the foundation for successive interventions formulated on and enhanced by lessons learnt from the first. The success of action research depends on the researchers ability to interpret the effects and outcomes of these interventions, expected or otherwise. The critical reflection of participants is an integral component of the research and is itself an object for investigation and improvement. Faculties of Dentistry are used to the rigours of scientific research and to testing outcomes using tried and tested statistical procedures. The first author was educated in this vein but his experiences with action research have led him to look anew at research methods, consider innovative approaches to course evaluation used in other department in the university and to look at their applicability for dental education. His conclusions are very positive.

Acknowledgements The authors wish to thank Dr. Stephen Knutton and Dr. Nikki Hedge, Division of Education in the University of Sheffield for advice on the design of the research instrument. REFERENCES Bunning, C. (1994). Action Research: an Emerging Paradigm. Brisbane: The Tertiary Education Institute, The University of Queensland. Cohen, L. and Manion, L. (1989). Research Methods in Education (3rd ed.). London:

12

Routledge. Eizenberg, N. (1995). Research into our own teaching. In C. McNaught and K. Beattie (Eds.) Research into Higher Education: Dilemmas, Directions and Diversions. Melbourne: Herdsa Victoria, 29-37. Elliott, J., and Ebbutt, D. (1983). Action Research into Teaching for Understanding: a Guide to the TIQL Project. London: Schools Council Publications. Greenwood, J. (1984). Nursing research: a position paper. Journal of Advacnced Nursing, 9, 77-82. Kember, D. , Lam, B.H., Yan, L. Yum, J. and Liu, S. (Eds.) (1997). Case Studies of Improving Teaching and Learning from the Action Learning Project. Hong Kong: Action Learning Project. Kemmis, S. (1989). Action research in retrospect and prospect. In The Action Research Reader, Geelong: Deakin University. McGill, I. and Beaty, L. (1992). Action Learning: a Practical Guide. London: Kogan Page. McKernman, J. (1991). Curriculum Action Research. London: Kogan Page. Medio, F.J., Reinhard, J.D., and Maxwell, J.A. (1984). Improving teaching rounds: action research in medical education. Proceedings of the Annual Conference on Research in Medical Education, 23, 283-288. Sandow, S. (1979). Action research and evaluation: can research and practice be successfully combined? Child Care and Health Development, 5, 211-223. Stenhouse, L. (1975). An Introduction to Curriculum Research and Development. London: Heinemann. Willmott, P. (1975). Use of action in developing urban policy. Report of a Colloquium held by the Department of Environment, Department of Environment, UK. Zuber-Skerrit, O. (1993).Research and Development in Management and Higher Education. Brisbane: The Tertiary Education Institute, The University of Queensland. Zuber-Skeritt, O. (1994). Professional Development in Higher Education. London: Kogan Page.

13

4. Reflect The results of the evaluation were discussed in course and departmental meetings and a number of recommendations for course revision were made. Students responses also reflected the need for a more rigorous evaluation procedure.

1. Reflect The early concerns of the projectleader were that the course concentrated too much on psychomotor skills, and failed, largely, to encourage students to develop theoretical skills or problem-solving abilities in the professional context.

3. Act/Observe Students were asked to complete a course evaluation questionnaire at the end of the 1994-95 year. The results were analysed to identify strengths and weaknesses of the course and areas for improvement.

2. Plan It was decided that certain amendments to the course would be made and student feedback would be sought on a regular basis. However, as a first step, students opinion of the current course would be collected.

7. Observe At the end of the course a revised evaluation questionnaire was administered. Students and staff were also interviewed about the course.

5. Plan Based on the recommendations made at the departmental meeting, the course was revised and the evaluation procedure was modified to include interviews with teachers and students.

6. Act The revised curriculum was implemented in 1995-96 to a new cohort of students.

8. Reflect/plan/act ... Feedback from students and teachers was discussed at staff meetings. Further possibilities for improving the course were identified. At this stage, an opportunity arose for the introduction of more radical changes as a result of a faculty-wide curricular revision centred on problem-based learning. The cycle continues.

Fig. 1 Action research spiral of a pre-clinical course in operative dentistry (courtesy of Dr. David Kember for the spiral design) 14