International Dental Education

Assessing the Competency of University of Malaya Dental Graduates: Employers and Graduates Perceptions
Ishak A. Razak, B.D.S., Ph.D.; Raja Latifah RJ, B.D.S., Ph.D.; Nasruddin Jaafar, B.D.S., Ph.D.; Abu Hassan MI, B.D.S., Ph.D.; Norintan Ab Murat, B.D.S., M.Sc.
Abstract: A survey was conducted to assess competencies of dental graduates of the Faculty of Dentistry, University of Malaya, as perceived by the graduates and their employers, based on the five-year undergraduate curriculum introduced in 1995. All senior dental officers in the Ministry of Health (MOH), representing employers, and all 164 dental graduates of the years 2000, 2001, and 2002 were sent a self-administered questionnaire covering eight areas of competency. The respondents had to rate these areas on a scale of 1 (very poor) to 4 (very good). The responses for each area were then dichotomized into poor (1 and 2) and good (3 and 4). If less than 60 percent of the respondents rated an area as good, then it was categorized as needing attention; 6069 percent as satisfactory; and 70 percent and above as excellent. One hundred and six graduates (64.6 percent) and twentynine employers (96.7 percent) responded; of the graduates, 73.6 percent were working in the MOH and 22.6 percent in private practice. About 57.1 percent of employers reported that at least five graduates have worked under them. Graduates (85.7 percent) and employers (83.3 percent) agreed that graduates have excellent skills in communication. Although all graduates perceived their competency to be excellent in the four areas (treatment planning; community-based skills; management, administrative skills, and personal management; and professional development skills), employers felt that these are the areas that are of concern and needed attention. In conclusion, whilst generally the graduates level of competency in almost all areas is acceptable or good, there are areas of concern that need to be addressed to further improve the five-year curriculum at the University of Malaya. Dr. Razak is Professor and Dean; Dr. Latifah is Associate Professor; Dr. Jaafar is Professor and Vice Deanall at the Faculty of Dentistry, University of Malaya; Dr. Hassan is Professor and Dean, Faculty of Dentistry, University Technology MARA, Malaysia; and Dr. Norintan is Lecturer, Department of Community Dentistry, University of Malaya. Direct correspondence and requests for reprints to Dr. Norintan Ab Murat, Department of Community Dentistry, Faculty of Dentistry, University of Malaya, 50603, Kuala Lumpur, Malaysia; 603-79674805 phone; 603-79674532 fax; norintan@um.edu.my. Key words: competency, perception of students, perception of employers Submitted for publication 9/26/07; accepted 11/26/07

dvances in oral health science and technology, changes in the concepts of the provision of oral health care, and the markedly improved possibilities for the prevention of oral diseases present exciting prospects for dental education providers. To remain viable, the dental curriculum should not remain static, but must develop in the light of both the present and anticipated needs of the community. Such curricula should focus on the development of scientific and practical skills and on value orientations appropriate to community needs. In 1995, the undergraduate dental curriculum of the University of Malaya (UM) Faculty of Dentistry was revamped from a four- to a five-year program of study. The first class of students had graduated from this program in the year 2000. An important yardstick in measuring the success of any dental education

program is whether graduates are able to meet the demands of dental practice and the expectations of their employers. Competency is a commonly used measure, referring directly to the behavior expected of independent practitioners who are beginning their professional careers. They are expected to be able to demonstrate sufficient skills and values and to do so in an integrated manner in response to the full range of circumstances encountered in general professional practice.1 Competency-based education, introduced approximately ten years ago, has become the preferred method of delivering dental education and assessing outcomes from the performance of graduates in many parts of the world. While the competency of graduates is an object of internal assessment, it is in the clinical environment that the real evaluation is determined. The intent of this study is to compare the perceptions of recent graduates with

364

Journal of Dental Education Volume 72, Number 3

their public sector employers perceptions regarding the competencies they demonstrate.

Materials and Methods

A self-administered questionnaire was formulated covering eight major competency domains expected from dental graduates. The numbers of items within each competency domain are listed in Table 1. For each item, respondents were asked to rate the performance of UM graduates on a scale of 1 (very poor) to 4 (very good). This questionnaire was face-validated by five lecturers in the university. Ten final-year dental students pretested the questionnaire for the recent graduates, and three dental officers in government services pretested the questionnaire meant for the employers. Twenty-nine employers (96.7 percent) and The survey was conducted in 2003 and was 106 graduates (64.6 percent) responded. Sixteen limited to graduates from classes who had underemployers, or 55.2 percent, had at least five targeted gone the new five-year program (graduates of 2000, UM graduates working directly under them. Table 2001, and 2002) and employers from the Ministry 2 presents a direct comparison of the perceptions of Health. In 2001, the Government of Malaysia of employers and graduates of the competency of introduced supplementary Provisions for National UM dental graduates across the multiple domains of Purposes under the Dental Act 1971. This legislation interest. Both the graduates (85.7 percent) and their enabled the director general of health, Malaysia, to employers (83.3 percent) agreed that the graduates serve notice to those who obtained registration under had excellent skills in communication. Both groups Section 12 to serve as dental officers in the public (graduates 88.3 percent; employers 74.7 percent) also service for a continuous total period of not less than agreed that the graduates are excellent in gathering three years. Thus, almost all students who graduinformation at chair-side. The main areas of comated in 2000 and who obtained registration under petency where 100 percent of graduates perceived this section were still completing their mandatory themselves as being excellent are in treatment planthree-year term in the government sector at the time ning skills, community-based skills, management of the survey. It is because of these circumstances and administrative skills, personal management, and that only the perceptions of public sector employers professional development skills. However, employwere included in this study. ers had a different opinion in these cases, as only Thirty senior dental officers (representing 20 percent felt that the graduates had good commuemployers) were asked to rate the competency level nity-based skills and only 50 percent considered the of their employees on a scale of 1 (very poor) to 4 graduates to be competent in treatment planning. (very good). The employers were asked to respond to the questionTable 1. Competency domains and the numbers of items assessed within each naire only if they had graduates domain in question who had worked or Domain Skills Number of Items were working directly under their supervision. They were reminded Domain 1 Skills involved in gathering information at chair-side 17 that the level of competency was Domain 2 Diagnostic skills 12 Domain 3 Skills in treatment planning 4 for the purpose of general dental Domain 4 Skills in treatment and prevention 24 practice. The same questionnaire Domain 5 Community-based skills 5 was administered to the graduDomain 6 Management and administrative skills 14 ates themselves as a means of Domain 7 Communication skills 6 assessing their own perceptions Domain 8 Personal management and professional development 7 of their competencies.

The responses collected for each item were dichotomized into poor (1 and 2) and good (3 and 4). The proportion of respondents who scored each item as good or poor was then noted. A domain score was obtained by averaging the proportion of respondents who scored good by the total number of items in a domain. The mean proportions of respondents was further categorized as needing attention (less than 60 percent of respondents scored good for the domain), satisfactory (60-69 percent), and excellent (70 percent and above). Data entered were processed and analyzed by the authors using SPSS version 12.0.

Results

March 2008 Journal of Dental Education

365

Table 2. A comparison of the perceptions of employers and graduates about the competencies of UM dental graduates across the main domains
Eight Domains of Competency Mean Proportion of Respondents Who Scored Good for Each Domain* Dental Graduates Senior Dental Officers 1. Information gathering at chair-side 2. Diagnostic skills 3. Treatment planning 4. Treatment and prevention 5. Community-based skills 6. Management and administration 7. Communication skills 8. Personal management and professional development *The mean proportion was interpreted as follows: <60%=needing attention 6069%=satisfactory >70%=excellent 88.3% 75.0% 100.0% 83.3% 100.0% 100.0% 85.7% 100.0% 74.7% 66.9% 50.0% 62.5% 20.0% 57.1% 83.3% 57.1%

All items in each of the eight domains were analyzed to provide an overview of the perception of the graduates competency level. Table 3 and Table 4 display the five items that were rated as being associated with the highest and lowest levels of competency by the graduates and their employers, respectively. Information gathering at chair-side and diagnostic skill were two items rated highly by both groups. Apart from the item managing patients confidentiality, employers and graduates chose the same items in the top five. However, differences arise in their ranking. Five competency items shared fifth position as rated by the employers. All graduates (100 percent) perceive that they are competent in examining teeth, while none (0 percent) of the employers feel that the graduates are competent in obtaining and evaluating diagnostic tests. Both groups perceive that graduates are least competent in performing CPR and conducting echo-training.

Discussion
Numerous studies have looked into either the perceptions of recent graduates of their own competencies or employers perceptions of the competency of dental graduates. However, very few have made comparisons between the two. The purpose of this study was to make these comparisons. The results showed that the students have a better perception of their competencies than do their employers. While the employers assigned the students far more moderate scores than the students assigned themselves, there

were indeed some areas of agreement. These were in the domain of communication (gathering information at chair-side), diagnostic skills, and skills in treatment and prevention. When the data were analyzed as each item in the main domain, there were similarities in the perceptions of both groups in terms of the items in which graduates are most competent. Both graduates and employers perceived that they are very competent in examining teeth, obtaining and identifying patients chief complaint, detecting early caries, and examining oral hygiene status. Almost similar opinions were expressed by graduates globally. Canadian students2 felt most competent in obtaining dental history, identifying patients chief complaint, and determining diagnosis. Students in Trinidad and Tobago3 felt that they are most competent in taking an adequate medical history, recognizing and treating dental caries, conducting an oral examination, and giving health education. Graduates in Australia4 felt they were well prepared to obtain and interpret medical, social, and dental histories, treat single crowns, and perform deep scaling and root planing. Employers who were mainly program directors in the United States5 scored their graduates relatively high on items pertaining to history taking, basic restorations, and uncomplicated endodontics. The skills that involve gathering information at chair-side and diagnosing, treating, and preventing oral diseases have been termed in the literature as the common bread and butter items in dentistry.3,4,6 They are the basics in clinical practice, performed almost everyday by graduates, and it is reassuring to note that both graduates and employers feel that
Journal of Dental Education Volume 72, Number 3

366

Table 3. Perceptions of dental graduates about their own competencies: the five highest scores and the five lowest scores
Competency Item Top Five Examination of teeth Obtain pertinent information about patients complaint Ability to identify patients chief complaint Detection of early caries Examination for oral hygiene status Bottom Five Effectively conduct echo-training Ability to perform cardiac pulmonary resuscitation (CPR) Ability to use clinical and epidemiological data Management of minor tooth movement Prevention and management of drug interactions *The proportion was interpreted as follows: <60%=needing attention 6069%=satisfactory >70%=excellent Domain 1 1 2 2 1 Proportion of Respondents Who Categorized Items as Satisfactory or Excellent* 100.0% 99.0% 98.1% 96.2% 96.1%

6 4 2 4 4

32.0% 36.0% 42.0% 46.0% 47.0%

Table 4. Perceptions of employers about dental graduates competencies: the five highest scores and the five lowest scores
Competency Item Domain Proportion of Respondents Who Categorized Items as Satisfactory or Excellent* 92.3% 90.7% 89.3% 88.9% 85.2%

Top Five Maintain patients confidentiality 6 Examination of periodontal tissues 1 Examination of teeth 1 Detection of early caries 2 Examination for oral hygiene status, obtaining 1 & 2 pertinent dental history, detection of periodontal disease, ability to identify patients chief complaint, scheduling patients appointments and recall visits Bottom Five Determine the need for and, when possible, obtain 2 and evaluate diagnostic tests such as radiographs and diagnostic casts Deliver convincing evidence of the role of prevention 5 at individual, group, or community level Ability to perform cardiac pulmonary resuscitation (CPR) 4 Effectively conduct echo-training 6 Management of staff 6 *The mean proportion was interpreted as follows: <60%=needing attention 6069%=satisfactory >70%=excellent

0 26.3% 42.0% 30.0% 32.0%

graduates of University of Malaya were competent in their conduct of these procedures. The most striking contrast in the views of the two groups in the main domain was the fact that while
March 2008 Journal of Dental Education

100 percent of graduates felt that they were good in personal management and professional development and are excellent in management, administrative, and community-based skills, the employers felt oth367

erwise. The perception of these graduates contrasts with the findings of Greenwood et al.,4 in which less than 35 percent of graduates in Australia and Canada said they felt they were competent in the areas of personnel and financial management. Other studies2,3,7 also suggested that these are the areas where graduates felt least competent. Shugars et al.8 found that dentists perceived community-based skills to be of little or no importance and are not part of day-to-day dental practice. Those areas where the employers felt in need of attention are similar with those identified by Glassman et al.5 Program directors of Australian and Canadian dental graduates surveyed in this latter study perceived that the graduates competency is poor in performing a population needs assessment and initiating community access and prevention programs. The poor ratings by the employers observed in our study are not unexpected if the fact that the experience levels of graduates being surveyed ranged from several months to three years. It would be unreasonable to expect the new graduates to understand matters of staff management and financial planning. They may not have acquired sufficient skills to implement district-level community programs. As new graduates, they are under the direct supervision of senior dental officers whom they rely on for guidance. From the employers perspective, such excessive reliance translates to a need to better train students in this domain. Hence, these will need to be addressed in continuing efforts to improve the curriculum of the dental faculty. Recent modifications to the UM Faculty of Dentistrys curriculum appear to have impacted the perceptions of the graduates in a number of ways. The faculty have reduced the number of didactic teaching hours, giving more focus to self-directed and problem-based learning. Studies9-11 have shown that problem-based learning sessions can improve students communication skills and reasoning abilities. The faculty have also integrated practical communication classes into the program of study. It is likely that this change had a role in the high scores that graduates assigned themselves in the domain of communication skills. Apart from introducing self-directed learning in the new five-year curriculum, the faculty have also introduced a behavioral science course for students in their second, third, and fourth years of study. Through this course, students are exposed to the various social, cultural, and economic conditions that can have an impact on oral health care. Further, students must take on a community project posting

in which they are required to implement a health promotion program and meet the multiple challenges of working in the community. It is thought that the graduates may have rated their competency highly in the areas of management, administrative, and community-based skills because of this exposure. Other possible reasons why the graduates might have felt competent is the fact that the faculty have constantly encouraged students to be competent in information and communications (ICT) technology, to participate in professional organizations, and to keep abreast of the latest technological innovations and the various issues surrounding oral health and health care delivery. Their experiences in these competencies are put to a rigorous test because all students have to organize and present their findings in an Annual Dental Students Conference held in their final year. These activities are compulsory for all students and are incorporated into their training curriculum.

Conclusion
This study found that, on the whole, students have a better perception of their competencies than their employers do. The dental school at the University of Malaya should address the areas of concerns perceived by the employers as it is their needs and the needs of the community that the school aims to fulfil in training dentists to be competent practitioners. Revisions to the curriculum are in the pipeline and should be implemented in due course. It would be interesting if future studies were conducted to look at patients perceptions of UM graduates competencies as compared to other local universities graduates within the three-year period of graduation.

REFERENCES
1. Chambers DW, Gerrow JD. Manual for developing and formatting competency statements. J Dent Educ 1994;58:3616. 2. Rafeek RN, Marchan SM, Naidu RS. Perceived competency at graduation among dental alumni of the University of West Indies. J Dent Educ 2004;68:818. 3. Greenwood LF, Lewis DW, Burgess RC. How competent do our graduates feel. J Dent Educ 1998;62:30713. 4. Greenwood LF, Townsend GC, Wetherell JD, Mullins GA. Self-perceived competency at graduation: a comparison of dental graduates from the Adelaide PBL curriculum and Toronto traditional curriculum. Eur J Dent Educ 1999;3:1538. 5. Glassman P, Redding S, Filler S, Chambers DW. Program directors opinions on the competency of postdoc-

368

Journal of Dental Education Volume 72, Number 3

toral general dentistry program graduates. J Dent Educ 1996;60:74754. 6. Holmes DC, Diaz-Arnold AM, Williams VD. Alumni self-perception of competence at time of dental school graduation. J Dent Educ 1997;61:46572. 7. Gerbert B, Bardner V , Maguire B, Martinoff J, Wycoff S, Crawford W. Recent graduates evaluation of their dental school education. J Dent Educ 1987;51(12): 697700. 8. Shugars DA, Bader JD, ONeil EH. Attitudes of dentists toward emerging competencies for health care practitioners. J Dent Educ 1992;56:6404.

9. Khoo HE, Chhem RK, Gwee MC, Balasubramaniam P. Introduction of problem-based learning in a traditional curriculum in Singapore: students and tutors perspectives. Ann Acad Med Singapore 2001;30(4):3714. 10. Seneviratne RD, Samarasekera DD, Karunathilake IM, Ponnamperuma GG. Students perception of problembased learning in the medical curriculum of the Faculty of Medicine, University of Colombo. Ann Acad Med Singapore 2001;30(4):37981. 11. Khoo HE. Implementation of problem-based learning in Asian medical schools and students perceptions of their experience. Med Educ 2003;37:4019.

March 2008 Journal of Dental Education

369