Community Dent Oral Epidemiol 2009; 37: 399404 All rights reserved

2009 John Wiley & Sons A/S

Reproducibility and accuracy of the ICDAS-II for occlusal caries detection

Diniz MB, Rodrigues JA, Hug I, Cordeiro RCL, Lussi A. Reproducibility and accuracy of the ICDAS-II for occlusal caries detection. Community Dent Oral Epidemiol 2009; 37: 399404. 2009 John Wiley & Sons A/S Abstract Objectives: The aim of this in vitro study was to assess the inter- and intra-examiner reproducibility and the accuracy of the International Caries Detection and Assessment System-II (ICDAS-II) in detecting occlusal caries. Methods: One hundred and sixty-three molars were independently assessed twice by two experienced dentists using the 0- to 6-graded ICDAS-II. The teeth were histologically prepared and classied using two different histological systems [Ekstrand et al. (1997) Caries Research vol. 31, pp. 224231; Lussi et al. (1999) Caries Research vol. 33, pp. 261266] and assessed for caries extension. Sensitivity, specicity, accuracy and area under the ROC curve (Az) were obtained at D2 and D3 thresholds. Unweighted kappa coefcient was used to assess inter- and intra-examiner reproducibility. Results: For the Ekstrand et al. histological classication the sensitivity was 0.99 and 1.00, specicity 1.00 and 0.69 and accuracy 0.99 and 0.76 at D2 and D3, respectively. For the Lussi et al. histological classication the sensitivity was 0.91 and 0.75, specicity 0.47 and 0.62 and accuracy 0.86 and 0.68 at D2 and D3, respectively. The Az varied from 0.54 to 0.73. The inter- and intra-examiner kappa values were 0.51 and 0.58, respectively. Conclusions: ICDAS-II presented good reproducibility and accuracy in detecting occlusal caries, especially caries lesions in the outer half of the enamel.

Michele Baf Diniz1,2, Jonas Almeida ssia Rodrigues1,2, Isabel Hug2, Rita de Ca Loiola Cordeiro1 and Adrian Lussi2
1 Department of Pediatric Dentistry, School of o Paulo State Dentistry of Araraquara, Sa University (UNESP), Araraquara, SP, Brazil, 2 Department of Preventive, Restorative and Pediatric Dentistry, School of Dental Medicine, University of Bern, Bern, Switzerland

Key words: dental caries; early diagnosis; professional training; teeth Michele Baf Diniz, Faculdade de Odontologia de Araraquara, Departamento nica Infantil, Rua Humaita , 1680, de Cl CEP: 14801-903 Araraquara, SP, Brazil Tel: +55 16 3301 6331 Fax: +55 16 3301 6329 e-mail: mibdiniz@hotmail.com Submitted 17 August 2008; accepted 20 May 2009

The detection of caries is a challenge in dentistry, especially regarding the occlusal surfaces. Incipient occlusal caries have proved difcult to detect because of the widespread use of uorides and its supercial remineralization potential, which allows the development of dentinal caries under a macroscopically intact surface (1). For this reason, early detection is important to establish adequate preventive measures and avoid premature tooth treatment by restorations. Dentists have several options at their disposal for the clinical detection of dental caries on occlusal surfaces, such as visual or visualtactile examination, radiographic examination, uorescence-based methods, and electrical conductance measurements (2). Visual inspection and radiographic examination have been commonly used in clinical practice
doi: 10.1111/j.1600-0528.2009.00487.x

in the past, but these techniques are capable of detecting occlusal caries lesions only at an advanced stage (3). Besides, a systematic review has shown that the use of the laser uorescence devices can lead to false-positive results related to its low specicity (4). In 2001, after the analysis of a systematic review presented in a conference in the USA, it was concluded that the reliability and reproducibility of currently available caries detection diagnostic systems, including visual and visualtactile criteria, were not strong (5). Besides, in 2002, a document from the International Consensus Workshop on Caries Clinical Trials held in Scotland agreed with this statement (6). Based on these ndings, a new visual criterion has been introduced for caries detection.

399

Diniz et al.

The International Caries Detection and Assessment System (ICDAS) was developed in 2002 by an international group of researchers (cariologists and epidemiologists) based on a systematic review of clinical caries detection systems to provide clinicians, epidemiologists, and researchers with an evidence-based system for caries detection. This method would allow a standardization of data collection and would also enable better comparability among studies (2, 79). In 2003, the ICDAS-I was devised based on the principle that the visual examination should be carried out on clean, plaque-free teeth, with carefully drying of the lesion surface to identify early lesions. According to this system, the replacement of the traditional explorers and sharp probes with a ball-ended periodontal probe would avoid traumatic and iatrogenic defects on incipient lesions. Later, in 2005, this criterion was modied and the ICDAS-II was created at the ICDAS workshop in Baltimore. The improvement included an exchange of codes to insure that the system would reect increased severity (7, 8). There are few studies in the literature concerning the visual ICDAS-II for occlusal caries detection. ICDAS-II has good reproducibility and accuracy for the detection of occlusal caries lesions at different stages of the disease (2, 7, 8, 10, 11). However, these studies were performed by examiners who received a special training before using the ICDAS-II. The aim of this in vitro study was to assess the inter- and intra-examiner reproducibility and the accuracy of the ICDAS-II in detecting occlusal caries when experienced but not ICDAS specialist were involved.

Triengen, Switzerland) and for 10 s with a waterpowder jet cleaner (PROPHYex II, KaVo, Biberach, Germany) with sodium hydrogen carbonate powder. The teeth were rinsed with the three-in-one syringe for 10 s to remove any possible powder remainders in the ssure (12). During the measurements, the teeth were stored under relative humidity of 100%. Photographs of the occlusal surfaces were taken (magnication of 6.25) and one spot on each tooth was selected in the ssure surface (test site). All assessments were carried out twice by two experienced dentists (A and B), with previous experience in others caries detection methods, observing a 1-week interval between the measurements. Both dentists were introduced to the ICDAS-II by the supervisor. The details of each code were discussed until a consensus was reached.

ICDAS-II examination
The visual examination was performed following the administration of ICDAS-II (2), with direct visualization of the teeth, guided by black and white photographs printed in draft quality paper after a full circle was inserted to cover the entire test site and coded as shown in Table 1. The teeth were examined in the same room with the aid of a light reector and a three-in-one air syringe. First they

Table 1. ICDAS-II criteria (2) ICDAS-II code 0 1 Clinical criteria description Sound tooth surface: no evidence of caries after prolonged air drying (5 s) First visual change in enamel: opacity or discoloration (white or brown) is visible at the entrance to the pit or ssure after prolonged air drying, which is not or hardly seen on a wet surface Distinct visual change in enamel: opacity or discoloration distinctly visible at the entrance to the pit and ssure when wet, lesion must still be visible when dry Localized enamel breakdown due to caries with no visible dentin or underlying shadow: opacity or discoloration wider than the natural ssure fossa when wet and after prolonged air drying Underlying dark shadow from dentin localized enamel breakdown Distinct cavity with visible dentin: visual evidence of demineralization and dentin exposed Extensive distinct cavity with visible dentin and more than half of the surface involved

Materials and methods

Sample selection
One hundred sixty-three permanent human molars, without sealants or restorations, were selected from a pool of extracted teeth, which were stored frozen at )20C until use. Prior to extraction, consent was obtained and the patients were informed about the use of their teeth for research purposes. All teeth had been extracted by dental practitioners in Switzerland (no water uoridation, 250 ppm F- in table salt). The teeth were defrosted for 3 h and the calculus and debris were removed using a scaler (Cavitron; Dentsply Professional, York, PA, USA). The teeth were then cleaned for 15 s with water and toothbrush (Trisa ultra super-sensitive; Trisa,
2

4 5 6

400

ICDAS-II for occlusal caries detection

were analyzed moist and then dried, according to Ismail et al. (8) and Jablonski-Moneni et al. (2). The codes ranged from examination of the rst visible carious change in enamel to extensive cavitation.

Validation
After visual examination, the teeth were ground longitudinally on a Knuth-Rotor polishing machine (Struers S/A, Ballerup, Denmark) using silicon carbide paper (60 lm of grain size) cooled under tap water. Progression of the grinding process (papers of grain size 30, 18, 8 and 5 lm) was constantly checked under the microscope (magnication 6.25) until the periphery of the site was reached. The teeth surfaces were then colored with saturated rhodamine B (Fluka, Buchs, Switzerland). For histological assessments, the rhodamine B penetration either into the enamel or both the enamel and the dentin tissues (magnication 10) was analyzed. Two different histological classication systems Ekstrand et al. (13) and Lussi et al. (1) were used to record caries extension at each tested site (Table 2).

Mariakerke, Belgium) at D2 and D3 thresholds for both the histological classications, using the mode value among all examinations. The use of a gold standard is a prerequisite in assessing the ROC curve (16). This analysis involves a plot of pairs of sensitivity (true positive rate) and 1 ) specicity (false-positive rate) for a given cut-off value of a diagnostic test (17).

Results
From the 163 occlusal test sites analyzed in this study (one site in each tooth), the histological examination revealed that seven of them were caries-free, 12 had caries extending up to halfway through the enamel, 67 had caries extending in the inner half of enamel, 47 had caries in dentin and 30 had deep dentin caries. Table 3 shows the cross tables with both Ekstrand et al. (13) and Lussi et al. (1) histological scores for the ICDAS-II.
Table 3. Cross-tabulation for ICDAS-II with the two histological classication system ICDAS-II 0 1 2 3 2 4 51 12 7 76 2 4 27 29 14 76 4 5 6 Total 7 11 107 21 17 163 7 12 67 47 30 163

Statistical analysis
Inter- and intra-examiner reproducibility was assessed by calculating unweighted kappa coefcient (14). Kappa values above 0.75 denoted excellent agreement, while values between 0.40 and 0.75 indicated good agreement (15). For each examiner, the relationship between the ICDAS-II criteria and both the histological classication systems were determined using the Spearmans correlation coefcient. A correlation coefcient of 0.70 or above indicates a strong relationship between two variables (2). Sensitivity, specicity, accuracy, area under the ROC curve (Az) and likelihood ratios (LR+) were calculated (MedCalc for Windows, version 9.3.0.0,

Ekstrand et al. (13) 0 4 1 1 3 1 2 6 6 3 1 4 Total 14 8 Lussi et al. (1) 0 4 1 1 3 1 2 5 4 3 2 2 4 Total 14 8

3 42 3 2 50 4 31 12 3 50

2 3 5

2 3 5 10

5 5

2 8 10

Table 2. Criteria used in the histological examination Score 0 1 2 3 4 Criteria proposed by Ekstrand et al. (13) No enamel demineralization Demineralization limited to the outer on e-half of the enamel thickness Demineralization between inner one-half of the enamel and outer one-third of the dentin Demineralization in the middle third of the dentin Demineralization in the inner third of the dentin Criteria proposed by Lussi et al. (1) Caries free Caries extending up to halfway through the enamel Caries extending in the inner half of the enamel Caries in the outer half of the dentin Caries in the inner half of the dentin

401

Diniz et al.

The agreement assessed by calculating the unweighted kappa was 0.51 for inter-examiner reproducibility. For intra-examiner reproducibility, the values were 0.58 and 0.59 for examiner A and B, respectively. The Spearmans correlation was 0.49 for examiner A and 0.42 for examiner B considering the Ekstrand et al. (13) histological classication, and 0.53 for examiner A and 0.46 for examiner B, considering the Lussi et al. (1) histological classication. Table 4 presents the inter- and intra-examiner reproducibility and Spearmans correlation values. Specicity, sensitivity, accuracy, area under the ROC curve (Az) and likelihood ratios (LR+) are shown in Table 5, where the accuracy of the tested method can be observed. The best value of sensitivity (1.00) was observed at threshold D3 of Ekstrand et al. (13) histological classication. For specicity, the ICDAS-II compared with the Lussi et al. (1) histological classication showed values of 0.47 and 0.62 and with the Ekstrand et al. (13) histological classication showed values of 1.00 and 0.69 for D2 and D3, respectively.

Discussion
Occlusal surfaces are the most caries-affected sites in children and adolescents. This is related to the
Table 4. Inter- and intra-examiner reproducibilities and Spearmans correlation for ICDAS-II examinations Spearmans correlation

Examiner

Intra-examiner unweighted Ekstrand Lussi kappa et al. (13) et al. (1) 0.49 0.42 0.53 0.46

A 0.58 B 0.59 Inter-examiner 0.51 unweighted kappa

Table 5. Specicity, sensitivity, accuracy, area under the ROC curve (Az) and LR+ of ICDAS-II and each corresponding histological system at D2 and D3 thresholds D2 Ekstrand et al. (13) Specicity Sensitivity Accuracy Az LR+ 1.00 0.99 0.99 0.63 1.50 Lussi et al. (1) 0.47 0.91 0.86 0.73 1.96 D3 Ekstrand et al. (13) 0.69 1.00 0.76 0.54 1.08 Lussi et al. (1) 0.62 0.75 0.68 0.73 1.73

D2: codes 01 = sound; codes 26 = decayed. D3: codes 02 = sound; codes 36 = decayed.

complex invaginated anatomy of the pits and ssures and to the difculty in plaque removal, which makes caries detection more difcult. For this reason, the importance of early occlusal caries detection has increased now (1820). There are some methods aimed at aiding clinicians in detecting caries lesions and in deciding the most appropriate treatment. This study assessed the visual examination and stressed the importance of early caries detection. In this way, the advantage of the ICDAS-II is its ability to detect the rst changes in dental surfaces because of caries development. To date, few studies are available researching the reproducibility of ICDAS-II. Other scoring systems have been used previously, such as the ve-point scoring system proposed by Ekstrand et al. (13) and the four-point system proposed by Souza-Zaroni et al. (21), who found good to excellent reproducibility. The kappa values for ICDAS-II found in this study were lower than those found by Ekstrand et al. (7) in an in vitro study, for both intra- and inter-examiner reproducibility. In another in vitro study also similar lower kappa values for ICDAS-II reproducibility were observed when no training was given to the examiners (11). However, Jablonski-Momeni et al. (2), after a training session in an in vitro study, found unweighted kappa values ranging from 0.32 to 0.61 for inter-examiner reproducibility and from 0.54 to 0.65 for intra-examiner reproducibility, which are similar to the values found in our study. Ismail et al. (8) showed higher weighted kappa values for intra-examiner (varying from 0.59 to 0.82) and inter-examiner reproducibility (varying from 0.63 to 0.75). However, this was an in vivo study performed in Detroit, Michigan. The differences among the present investigation and those studies could be explained by the subjective aspects involved in visual examination, such as knowledge and clinical experience of the examiners (22). Besides, it is important to stress that some investigations were performed in vivo, and others in vitro, and these could be an explanation of the different reproducibility values found among the studies. The methodological difference between our study and the others is due to the examiners training. According to Ismail et al. (8), the ICDAS-II presents good to excellent reproducibility, even when used by examiners who have no previous experience in epidemiological dental examination. This is in contrast to this study, as shown by the lower kappa values for intra- and interreproducibility. The examiners of this study were

402

ICDAS-II for occlusal caries detection

experienced dentists involved in caries research, concerning different methods of early occlusal caries detection. The examiners had previously participated in other dental caries studies, using different methods and they received a short introduction to the ICDAS-II. Besides, in this study, the relationship between the ICDAS-II and the both histological classication systems was not strong (Spearmans correlation coefcients varied from 0.42 to 0.53). Jablonski-Momeni et al. (2) also found fair correlation, showing values ranging from 0.48 to 0.72 using Downer histology and from 0.43 to 0.68 for Ekstrand histology. In previous studies comparing visual examination and histological classication systems, the relationship was stronger (13, 23). The method used in this study to validate caries criteria was based on the quantitative correlation between the clinical assessment of tooth surfaces with histological presence and the extent of demineralization in enamel and dentin (3, 13). The percentage of enamel caries correctly detected by the ICDAS-II conrmed its good ability to rst detect visual changes in enamel. This good performance has also been shown by recent studies published assessing the ICDAS-II for oclusal caries detection (7, 8, 11). Regarding the Lussi et al. (1) histological classication, the area under the ROC curve (Az) (0.73) showed good performance of the ICDAS-II in detecting occlusal caries lesions. Other studies had shown that the ICDAS-II produced areas under the ROC curves of 0.70 (2) and 0.75 (11), which agrees with our results. The advantages of ROC curve are: (a) it includes several cut-off points; (b) it shows the relationship between the sensitivity and specicity; and (c) it is not affect by the prevalence of disease (17). In this study, at D2 threshold, the sensitivity and specicity of the ICDAS-II for Lussi et al. (1) histological classication were 0.91 and 0.47, respectively (Table 5). This means that 6% of sound sites were incorrectly scored as carious, according to the distribution of caries (Table 2). This situation must be carefully interpreted in clinical practice in view of the fact that the amount of sound sites was short in our study. However, for Ekstrand et al. (13) histological classication sensitivity and specicity were higher (0.99 and 1.00, respectively). Different values were found in a previous study (2), where the optimal sensitivity (0.69) and specicity (0.82) achieved by the ICDAS-II was observed at cut-off 12. At D3 threshold, sensitivity was 0.75 and

specicity was 0.62 for Lussi et al. (1) histological classication. These results are very similar with those observed by Rodrigues et al. (11), who found sensitivity of 0.73 and specicity of 0.65 when no training was given to the examiners. Nevertheless, considering Ekstrand et al. (13) histological classication, sensitivity was 1.00 and specicity was 0.69. The value of specicity is different from the value (0.89) found in a recent study concerning the ICDAS-II (2). This could be due to the different examiners experience using this new visual criterion. In their study, the examiners were trained and had experience, which differ from our study, where the examiners were not trained, but had clinical experience using a previous visual criterion for occlusal caries detection (13). It is important to point out that the values of sensitivity and specicity obtained in this study for D2 and D3 thresholds using the cut-off points proposed by the ICDAS-II matched up to the optimal values obtained by the highest sum of sensitivity and specicity. In addition, the ICDASII presented the highest value of LR+ at the D2 threshold, which shows how much the odds of the disease increase when a test is positive. The ICDAS-II also showed highest value of accuracy at the D2 threshold, conrming its ability to detect enamel changes. At D3 threshold, the lower accuracy value presented was similar to that presented by Rodrigues et al. (11). The main difference between both histological classication systems used in the present study is the caries severity of score 2. For Ekstrand classication, this score combines deep enamel caries and caries restricted to outer one-third of dentin. At D2 threshold, which considers ICDAS-II codes 26 as decayed, the highest values of sensitivity and specicity were observed for this classication. This could suggest that when an initial lesion is observed in a wet surface, one-third of dentin can be already decayed. Some meta-analysis reviews have shown that visual examination is poor for caries detection, presenting high specicity and low sensitivity (5). However, recent studies have shown that a meticulous examination of a clean and dry surface can improve caries detection, especially when the examiners are trained with this new method (2, 7, 8). This new visual examination criterion seems to be promising in the case of having to very accurately examine and closely describe the characteristics of the tooth surface. Furthermore, the

403

Diniz et al.

rst enamel changes can be better identied when the tooth surface is dry and has been cleaned before, emphasizing the importance of an accurate examination to improve the diagnosis of caries lesions. An advantage of this visual criterion is the possibility to record detailed characteristics of the teeth and monitor the changes in their surfaces. It can be concluded that this in vitro evaluation of the ICDAS-II in detecting occlusal caries lesions presented good inter- and intra-examiner reproducibility and validity even after non-trained examinations. Besides, this system is useful, easy to use, and clearly dened by scores for clinical visual caries detection. Nevertheless, more research will be needed to validate this system for caries activity as well as for caries detection in smooth surfaces, in vitro and in vivo.

References
1. Lussi A, Imwinkelried S, Pitts NB, Longbotton C, Reich E. Performance and reproducibility of a laser uorescence system for detection of occlusal caries in vitro. Caries Res 1999;33:2616. 2. Jablonski-Momeni A, Stachniss V, Ricketts DN, Heinzel-Gutenbrunner M, Pieper K. Reproducibility and accuracy of the ICDAS-II for detection of occlusal caries in vitro. Caries Res 2008;42:7987. 3. Ricketts DN, Ekstrand KR, Kidd EA, Larse T. Relating visual and radiographic ranked scoring system for occlusal caries detection to histological and microbiological evidence. Oper Dent 2002;27:2317. 4. Bader JD, Shugars DA. A systematic review of the performance of a laser uorescence device for detecting caries. J Am Dent Assoc 2004;135:141426. 5. Bader JD, Shugars DA, Bonito AJ. Systematic review of selected dental caries diagnostic and management methods. J Dent Educ 2001;65:9608. 6. Pitts NB, Stamm JW. Consensus Workshop on Caries Clinical Trials (ICW-CCT) nal consensus statements: Agreeing where the evidence leads. J Dent Res 2004;83:C1258. 7. Ekstrand KR, Martignon S, Ricketts DNJ, Qvist V. Detection and activity assessment of primary coronal caries lesions: a methodologic study. Oper Dent 2007;32:22535. 8. Ismail AI, Sohn W, Tellez M, Amaya A, Sen A, Hasson H et al. The International Caries Detection and Assessment System (ICDAS): an integrated system for measuring dental caries. Community Dent Oral Epidemiol 2007;35:1708.

9. Ismail AI, Sohn W, Tellez M, Willem JM, Betz J, Lepkowski J. Risk indicators for dental caries using the International Caries Detection and Assessment System (ICDAS). Community Dent Oral Epidemiol 2008;36:5568. 10. Sohn W, Ismail A, Amaya A, Lepkowski J. Determinants of dental care visits among low-income African-American children. J Am Dent Assoc 2007; 138:30918. 11. Rodrigues JA, Hug I, Diniz MB, Lussi A. Performance of uorescence methods, radiographic examination and ICDAS II on occlusal surfaces in vitro. Caries Res 2008;42:297304. 12. Lussi A, Reich E. The inuence of toothpastes and prophylaxis pastes on uorescence measurements for caries detection. Eur J Oral Sci 2005;113:1414. 13. Ekstrand KR, Ricketts DN, Kidd EA. Reproducibility and accuracy of three methods for assessment of demineralization depth of the occlusal surface: an in vitro examination. Caries Res 1997;31:22431. 14. Lin LIK. A concordance correlation coefcient to evaluate reproducibility. Biometrics 1989;45:25568. 15. Fleiss IL. Statistical methods for rates and proportions, 2nd edn. New York: Wiley, 1981:21225. 16. Htoon HK, Peng LL, Huak CY. Assessment criteria for compliance with oral hygiene: application of ROC analysis. Oral Health Prev Dent 2007;5:838. 17. Obuchowski NA. Receiver operating characteristic curves and their use in radiology. Radiology 2003;229:38. 18. Sheehy EC, Brailford SR, Kidd EAM, Beighton D, Zoitopoulos L. Comparison between visual examination and a laser uorescence system for in vivo diagnosis of occlusal caries. Caries Res 2001;35:4216. 19. Diniz MB, Rodrigues JA, Paula AB, Cordeiro RCL. In vivo evaluation of laser uorescence performance using different cut-off limits for occlusal caries detection. Laser Med Sci 2009;24:295300. 20. Rodrigues JA, Diniz MB, Josgrilberg EB, Cordeiro RCL. In vitro comparison of laser uorescence performance with visual examination for detection of occlusal caries in permanent and primary molars. Lasers Med Sci 2008; doi: 10.1007/s10103-008-0552-4. 21. Souza-Zaroni WC, Ciccone JC, Souza-Gabriel AE, Ramos RP, Corona SAM, Palma-Dibb RG. Validity and reproducibility of different combinations of methods for occlusal caries detection: an in vitro comparison. Caries Res 2006;40:194201. 22. Fung L, Smales R, Ngo H, Moun G. Diagnostic comparison of three groups of examiners using visual and laser uorescence methods to detect occlusal caries in vitro. Aust Dent J 2004;49:6771. 23. Ekstrand KR, Kuzmina I, Bjorndal L, Thylstrup A. Relationship between external and histologic features of progessive stages of caries in the occlusal fossa. Caries Res 1995;29:24350.

404