Escolar Documentos
Profissional Documentos
Cultura Documentos
Calculations based on equations presented in Fleiss (Statistical Methods for Rates and Proportions, 1981, pp. 229-232)
(Copyright © 2004, 2009 Jason King, Wallace Judd. All rights reserved. Available at http://www.ccitonline.org/jking/homepage/kappa1.xls)
Directions: Enter rating data in the Rater1, Rater2, Rater3, Rater4, and Rater5 worksheets, as needed.
Edit the shaded numbers below to indicate the number of raters, items and categories.
Enter # of raters (m): 5 =N_Raters 0.885 = Generalized Kappa for Rater 1, … Rater n data
Enter # of items (n): 100 =N_Items
# of categories (k): 6
The table below summarizes the data entered in the corresponding spreadsheets. Highlighting indicates areas of disagreement.
******************
OVERALL
num = nm2 - sum_x2 190
den = nm(m-1)sum_pq 1657.6
gen kappa = 0.885
(p(e)) 0.171
(sum p3) 0.030
a
This approximate standard error formula is based on Fleiss (Psychological Bulletin, 1971, Vol. 76, 378-382)
b
This approximate standard error formula is based on Fleiss, Nee & Landis (Psychological Bulletin, 1979, Vol. 86, 974-977)
Rater 1 Classifications
Item Cat
1 3
2 5
3 2
4 4
5 6
6 5
7 5
8 1
9 2
10 5
11 3
12 4
13 5
14 3
15 6
16 4
17 3
18 3
19 2
20 6
21 3
22 2
23 3
24 6
25 4
26 3
27 5
28 3
29 3
30 2
31 3
32 4
33 4
34 4
35 3
36 1
37 6
38 4
39 6
40 1
41 2
42 5
43 3
44 5
45 4
46 4
47 1
48 6
49 6
50 2
51 6
52 2
53 3
54 6
55 4
56 4
57 3
58 2
59 2
60 3
61 2
62 1
63 3
64 1
65 6
66 6
67 1
68 5
69 2
70 4
71 5
72 3
73 2
74 6
75 6
76 5
77 1
78 1
79 1
80 3
81 6
82 4
83 2
84 5
85 5
86 2
87 5
88 4
89 2
90 2
91 2
92 1
93 1
94 5
95 3
96 2
97 2
98 6
99 6
100 3
Rater 2 Classifications
Item Cat
1 3
2 4
3 2
4 4
5 6
6 5
7 5
8 1
9 3
10 5
11 3
12 4
13 5
14 2
15 6
16 4
17 3
18 3
19 5
20 6
21 4
22 2
23 3
24 6
25 4
26 3
27 5
28 3
29 3
30 2
31 2
32 4
33 4
34 4
35 3
36 1
37 6
38 1
39 6
40 1
41 2
42 5
43 3
44 5
45 4
46 4
47 1
48 6
49 5
50 2
51 6
52 2
53 3
54 6
55 4
56 4
57 3
58 2
59 2
60 4
61 2
62 1
63 3
64 1
65 6
66 6
67 1
68 5
69 2
70 4
71 5
72 3
73 6
74 6
75 6
76 5
77 1
78 1
79 1
80 3
81 6
82 4
83 2
84 2
85 5
86 2
87 5
88 4
89 2
90 2
91 2
92 1
93 1
94 5
95 3
96 2
97 2
98 6
99 6
100 4
Rater 3 Classifications
Item Cat
1 3
2 5
3 2
4 3
5 6
6 5
7 5
8 2
9 2
10 5
11 3
12 4
13 5
14 4
15 6
16 4
17 3
18 5
19 2
20 6
21 3
22 2
23 4
24 6
25 4
26 3
27 5
28 3
29 2
30 2
31 3
32 4
33 4
34 4
35 3
36 1
37 6
38 4
39 6
40 4
41 2
42 5
43 3
44 5
45 4
46 4
47 1
48 6
49 6
50 2
51 6
52 2
53 2
54 6
55 4
56 4
57 3
58 2
59 2
60 3
61 2
62 4
63 3
64 1
65 6
66 6
67 1
68 5
69 2
70 4
71 5
72 1
73 2
74 6
75 6
76 5
77 1
78 1
79 1
80 3
81 6
82 4
83 2
84 5
85 5
86 2
87 5
88 4
89 5
90 2
91 2
92 1
93 1
94 5
95 3
96 3
97 2
98 6
99 6
100 3
Rater 4 Classifications
Item Cat
1 3
2 5
3 2
4 4
5 6
6 5
7 5
8 1
9 2
10 5
11 3
12 4
13 5
14 3
15 6
16 4
17 3
18 3
19 2
20 6
21 3
22 2
23 3
24 6
25 4
26 3
27 5
28 3
29 3
30 2
31 3
32 4
33 4
34 4
35 3
36 1
37 6
38 4
39 6
40 1
41 2
42 5
43 3
44 5
45 4
46 4
47 1
48 6
49 6
50 2
51 6
52 2
53 3
54 6
55 4
56 4
57 3
58 2
59 2
60 3
61 2
62 1
63 3
64 1
65 6
66 6
67 1
68 5
69 2
70 4
71 5
72 3
73 2
74 6
75 6
76 5
77 1
78 1
79 1
80 3
81 6
82 4
83 2
84 5
85 5
86 2
87 5
88 4
89 2
90 2
91 2
92 1
93 1
94 5
95 3
96 2
97 2
98 6
99 6
100 3
Rater 5 Classifications
Item Cat
1 3
2 5
3 2
4 4
5 6
6 5
7 5
8 1
9 2
10 5
11 3
12 4
13 5
14 3
15 6
16 4
17 3
18 3
19 2
20 6
21 3
22 2
23 3
24 6
25 4
26 3
27 5
28 3
29 3
30 2
31 3
32 4
33 4
34 4
35 3
36 1
37 6
38 4
39 6
40 1
41 2
42 5
43 3
44 5
45 4
46 4
47 1
48 6
49 6
50 2
51 6
52 2
53 3
54 6
55 4
56 4
57 3
58 2
59 2
60 3
61 2
62 1
63 3
64 1
65 6
66 6
67 1
68 5
69 2
70 4
71 5
72 3
73 2
74 6
75 6
76 5
77 1
78 1
79 1
80 3
81 6
82 4
83 2
84 5
85 5
86 2
87 5
88 4
89 2
90 2
91 2
92 1
93 1
94 5
95 3
96 2
97 2
98 6
99 6
100 3