Escolar Documentos
Profissional Documentos
Cultura Documentos
IPGKTHO
Item Analysis
Item Distractor
Item Difficulty
Discrimination Analysis
Item analysis is a process which examines
student responses to individual test items
(questions) in order to assess the quality of
those items and of the test as a whole.
Purpose of Item Analysis
To select the best available items for the final form of
the test.
Item 1 8 3 1 0
Item 2 2 8 2 0
Item 3 4 8 0 0
Item 4 1 3 8 0
Item 5 5 0 0 7
Distractor Analysis
For Item 1, the discrimination index for each distractor can be
calculated using the discrimination index formula.
From Table 2, we know that all the students in the upper group
answered this item correctly and only one student from the lower
group did so. If we assume that the three remaining students
from the lower group all selected distractor B, then the
discrimination index for item 1, distractor B will be:
This negative value indicates that more students from the lower
group selected the distractor compared to students from the
upper group. This result is to be expected of a distractor and a
value of -1 to 0 is preferred.
Why Do Item Analysis?
Encourage teachers to undertake an item analysis
as often as practical
Allowing for accumulated data to be used to make
item analysis more reliable
Providing for a wider choice of item format and
objectives
Facilitating the revision of items
Facilitating the physical construction and
reproduction of the test
Accumulating a large pool of items as to allow for
some items to be shared with the students for
study purposes.
Benefits of Item Analysis
1. It provides useful information for class
discussion of the test.
2. It provides data which helps students
improve their learning.
3. It provides insights and skills that lead to the
preparation of better tests in the future.
Limitations of Item Analysis
It cannot be used for essay items.
Teachers must be cautious about what
damage may be due to the table of
specifications when items not meeting the
criteria are deleted from the test. These
items are to be rewritten or replaced.
Outline
1. Introduction
2. Where, when, how the test is administered,
number of students involved and which Year
and class
3. Test blueprint
4. Test format
A 80-100
B 60-79
C 40-59
D 20-39
E 0-19
1
2
3
4
5
discard
0.26 – 0.75 Right difficulty retain
discard
Table 4: Analysis of Item Discrimination
Student Total Correct Response in Each Item
Score
Item 1 Item 2 Item 3 Item 4 Item 5
1 1 0 1 0 1
2 0 1 1 0 0
3 1 1 1 1 1
4 0 0 0 0 0
discriminating
Item 1
Item 2
Item 3
Item 4
Item 5
TOTAL
Interpreting Distractor Analysis
The distractors are important component of an item,
as they show a relationship between the total test
score and the distractor chosen by the student.
Distractor analysis is a tool to inform whether the
item was well structured or failed to perform its
purpose.
The quality of the distractor influences students
performance on a test item. Ideally, low-scoring
students who had not mastered the subject, should
choose the distractor more often, whereas high
scorers should discard them more frequently while
choosing the correct option.
Interpreting Distractor Analysis
Any distractor that has been selected by 5% of the
students is considered to be non-functioning
distractor.
Reviewing the options can reveal potential errors of
judgment and inadequate performance of
distractors. These poor distractors can be revised,
replaced or removed.
Interpreting Distractor Analysis
Any distractor that has been selected by 5% of the
students is considered to be non-functioning
distractor.
Reviewing the options can reveal potential errors of
judgment and inadequate performance of
distractors. These poor distractors can be revised,
replaced or removed.
Internal Consistency Reliability
The reliability of a test refers to the extent to which the
test is likely to produce consistent scores.
The measure of reliability used is Cronbach's Alpha.
This is the general form of the more commonly reported
KR-20 and can be applied to tests composed of items
with different numbers of points given for different
response alternatives.
When coefficient alpha is applied to tests in which each
item has only one correct answer and all correct answers
are worth the same number of points, the resulting
coefficient is identical to KR-20.
High reliability indicates that the items are all measuring
the same thing, or general construct.
The higher the value, the more reliable the overall test
score.
Internal Consistency Reliability
We can estimate the proportion of true score variance that is
captured by the items by comparing the sum of item variances
with the variance of the sum scale. Specifically, we can compute:
= (k/(k-1)) * [1- (s2i)/s2sum]
This is the formula for the most common index of reliability,
namely, Cronbach's coefficient alpha (α). In this formula, the
si**2's denote the variances for the k individual items; ssum**2
denotes the variance for the sum of all items.
If there is no true score but only error in the items (which is
esoteric and unique, and, therefore, uncorrelated across
subjects), then the variance of the sum will be the same as the
sum of variances of the individual items. Therefore, coefficient
alpha will be equal to zero.
If all items are perfectly reliable and measure the same thing
(true score), then coefficient alpha is equal to 1. (Specifically, 1-
(si**2)/ssum**2 will become equal to (k-1)/k; if we multiply this
by k/(k-1) we obtain 1.)
Internal Consistency Reliability
Cronbach’s Internal Consistency
Alpha
(Reliability)
α≥0.90 Excellent
0.80≤α≤0.90 Very good
0.70≤α≤0.80 Good (There are probably a few items which could
be improved)
0.60≤α≤0.70 Acceptable (There are probably some items which
could be improved)
0.50≤α≤0.60 Poor (Suggests need for revision of a test)