Escolar Documentos
Profissional Documentos
Cultura Documentos
The hopkins verbal learning test: Development of a new memory test with
six equivalent forms
Jason Brandta
a
The Johns Hopkins University School of Medicine,
To cite this Article Brandt, Jason(1991) 'The hopkins verbal learning test: Development of a new memory test with six
0920- 1637191/0502-0125$3.OO
0 Swets & Zeillinger
CLINICAL ISSUES
ABSTRACT
A new test of verbal learning and memory, the Hopkins Verbal Learning Test, was
developed. The test consists of three trials of free-recall of a 12-item, semantically
categorized list, followed by yes/no recognition. Six parallel forms yielded equivalent
results in normals. The performance of patients with Alzheimers disease and chronic
amnesia is described. The test is likely to be useful in patients too impaired for more
comprehensive memory assessments and where repeated testing is necessary.
126
JASON BRANDT
complexity and/or lack of parallel forms. The WMS-R, for example, requires 45
to 60 min for administration and, at present, is available in only one form. The
California Verbal Learning Test (Delis, Kramer, Kaplan, & Ober, 1986) is gaining
popularity as a relatively comprehensive verbal memory test, and an alternate
form has been developed (Delis et al., 1991), but its length and complexity often
make it unwieldy for use with demented or otherwise difficult-to-test patients.
Both the Randt Memory Test (Randt & Brown, 1979; Randt, Brown, & Osborne,
1980) and the Selective Reminding Test (Buschke, 1973; Buschke & Fuld, 1974)
have parallel forms available (Franzen, Tishelman, Smith, Sharp, & Friedman,
1989; Hannay & Levin, 1985; Ruff, Light, & Quayhagen, 1988), but they too are
impractical in situations where brevity is essential.
This paper introduces a new, very brief test of verbal memory, the Hopkins
Verbal Learning Test. The development of six equivalent forms of the instrument
is described, and preliminary standardization and validation data are presented.
METHODS
Description of the Test
Each form of the Hopkins Verbal Learning Test (HVLT) consists of a 12-item word list,
composed of four words from each of three semantic categories (see Appendix). The
subject is instructed to listen carefully as the examiner reads the word list and attempt to
memorize the words. The word list is then read to the subject at the approximate rate of
one word every 2 s. The patients free recall of the list is recorded. The same procedure is
repeated for two more trials. After the third learning trial, the patient is read 24 words and
is asked to say yes after each word that appeared on the recall list (12 targets) and no
after each word that did not (12 distractors). Half of the distractors are drawn from the
same semantic categories as the targets (related distractors) and half are drawn from other
categories (unrelated distractors).
Construction of the Test
For each list, three semantic categories were chosen from among the 56 word categories
studied by Battig and Montague (1969). Four words which represent very common responses
to each category name were chosen for the list, with the exception of the two most
common responses (i.e., the best category exemplars). The two most commonly given
responses to each category name were chosen as semantically-related distractors for the
recognition test. For example, the words emerald, sapphire, opal, and pearl were chosen
for Form 1. They are drawn from the category precious stones. The two most frequent
responses to that category name, according to the Battig and Montague (1969) norms, are
diamond and ruby. Diamond and ruby do not appear on the to-be-remembered list, but
they do appear as semantically related distractors on the recognition portion of Form 1.
The six recall lists were very closely matched for mean frequency of occurrence of the
words as responses to the category names in the Battig and Montague (1969) normative
study (F5,67
= 0.05, N.S.). The construction of the lists was further constrained by the
requirement that they be composed of words with relatively low frequencies of occurrence
in printed text (mean word frequency of each list 5 50 occurrences per million; Francis &
KuCera, 1982). Mean word frequencies of the lists did not differ (F5,67
= 2.01, N.S.).
127
RESULTS
Study 1 :
For initial determination of the equivalence of the six forms, the test was administered to 129 normal individuals who were serving as control subjects in a
number of research studies. These subjects ranged in age from 19 to 77, and all
had at least a high-school education. They were thoroughly screened by interview
for the absence of histories of neurological or psychiatric disorders, including
alcohol or drug abuse, head injury, or learning disability. None was taking any
medicalion known to affect cognition. Each subject was administered one form
of the HVLT, selected at random. Most of the subjects (102 out of 129) were also
administered the Mini-Mental State Exam (Folstein, Folstein, & McHugh, 1975)
as a screen for gross cognitive impairment, and all scored at least 25 out of 30.
Table 1 describes the subject sample. Analyses of variance revealed that the
groups of subjects did not differ in mean age (F,,,,, = 1.35, N.S.) or score on the
Mini-Mental State Exam (F5,96= 1.95, N.S.). Similarly, there was no significant
difference among the groups in sex distribution (x2 = 3.87, d.f. = 5 , N.S.) or level
of education (Kruskal-Wallis H = 9.16, d.f. = 5 , N.S.).
Table 2 shows the group means for performance on the recall and recognition
portions of the HVLT. A form-by-trial (6 x 3) repeated-measures analysis of
variance was performed on the recall performance. While the trial effect was
highly significant (F2,246= 300.00, p < .OOl), neither the form effect (F,,,,3 = 1.21,
N.S.) nor the form-by-trial interaction (F,,,,,, = 0.84, N.S.) was significant. The
Form2
Form3
Form4
Form5
Form6
17
22
23
20
26
21
Ma1es:Females
10:7
9:13
10:13
8:12
16:lO
11:lO
Age, yrs.
43.18
(14.46)
41.09
(14.28)
43.43
(17.19)
46.00
(17.01)
37.38
(13.61)
36.29
(1 3.26)
Mini-Mental
State Exam
28.81
(1.56)
29.64
(0.70)
29.30
(0.86)
29.41
(0.94)
29.33
(0.84)
28.7 1
(1.14)
Education
Level'
3.29
(0.92)
2.50
(0.96)
2.74
(1.21)
2.40
(1.14)
3.00
(1.17)
2.86
(0.96)
'
128
JASON BRANDT
variances of the total recall scores (summed over trials) were homogeneous
(Cochrans C = 0.29, N.S.). This equivalence of means and variances suggests a
virtual equivalence among the test forms.
Recognition performance was near perfect on all the forms. The number of
true-positives (hits) was either 11 or 12 in every case, and did not differ
significantly among the groups (F5,123= 2.13, N.S.). The incidence of false-positives was extremely low, but an analysis of variance revealed that the forms
differed on mean number of semantically related false-positives (F5,123
= 4.93, p
< .OOl). A Scheffk test revealed that Form 3, with zero related false-positives,
differed significantly from Forms 1 and 4. Semantically unrelated false positives
were virtually nonexistent, and their number did not differ among the forms
(F5,123= 0.77, N.S.). A recognition discrimination score that corrects (crudely)
Table 2. Hopkins Verbal Learning Test performance for 129 normal subjects. Means
(+
m.
Form1
Form2
Form3
Form4
Form5
Form6
Trial I
7.17
(2.18)
8.09
(1.48)
7.26
(1.29)
7.25
(1.59)
7.85
(2.07)
8.10
(1.64)
Trial 2
9.17
(2.07)
9.86
(1.75)
9.70
(1.18)
9.75
(1.83)
9.96
(1.51)
9.90
(1.55)
Trial 3
9.88
(1.90)
11.09
(1.15)
10.43
(1.12)
10.55
(1.43)
10.54
(1.55)
10.71
(1.10)
26.24
(5.54)
29.05
(3.42)
27.39
(3.17)
27.55
(4.41)
28.35
(4.55)
28.71
(3.61)
113 8
(0.33)
11.68
(0.65)
11.95
(0.21)
11.80
(0.41)
12.00
(0.00)
113 6
(0.35)
0.59
(0.71)
0.41
(0.73)
0.00
(0.00)
0.60
(0.60)
0.23
(0.43)
0.10
(0.30)
0.00
(0.00)
0.00
(0.00)
0.04
(0.21)
0.00
(0.00)
0.00
(0.00)
(0.22)
11.29
(0.92)
11.27
(1.16)
11.91
(0.29)
11.20
(0.70)
11.77
(0.43)
11.71
(0.56)
Recall
TOTAL
Recognition
True-Positives
False-Positives
Related
Unrelated
DISCRlMIN-
ATION
0.05
129
the number of true-positive recognitions for guessing can be computed by [(TruePositives + True-Negatives) - 121. This simplifies to [True-Positives - FalsePositives] (Snodgrass & Corwin, 1988). While the six forms differed in their
= 3.89, p c .005), a Scheff6 test rediscrimination scores in this sample (F5,123
vealed that no two forms were significantly different at the 5% confidence level.
In addition, the range of discrimination scores was so narrow (1 1.20 to 11.91) as
to have little practical significance.
In this sample of 129 normal subjects, there was no relationship between age
and accuracy of recall or recognition. The Pearson product-moment correlation
between age and number of words correctly recalled over the three learning trials
was -0.05 (see Figure 1). The correlation between discrimination score and age
was +0.08, but again the restricted range of discrimination scores should be
noted.
Study 2 :
The next study used a within-subjects design in order to control for possible
differences between groups due to sampling error. A group of 17 normal individuals between the ages of 35 and 55 was recruited from among hospital and
university employees. These subjects were selected using the same criteria as in
Study 1. Their mean age was 44.47 years (SD = 6.47), and they averaged 14.18
years of education (range = 10-19 years; SD = 2.56). Each subject was adminis-
.........
. . . .. . .
.
.
... . . .
.
........
. ... ......
.
.
.
.
.
.
.
.
.
.
.
. .. .. .. . .. .
.. .
.
0
0 .
g
;
w
0.
Hrn.
0
0
0 . 0
0 .
0-
I-
25--
15--
10
0..
0 .
20--
lo!
30--
"
H .
J 35J
15
20
25
30
35
40
45
50
55
60
65
70
75
80
AGE
Fig. 1. Total recall scores (summed over three trials) on the Hopkins Verbal Learning
Test as a function of age for 129 normal subjects.
130
JASON BRANDT
tered the Mini-Mental State Exam. Their scores ranged from 27 to 30 (mean =
28.82; SD = 1.07), indicating the absence of any gross cognitive impairment.
Each subject came to the laboratory on 6 days within a 2-week period. On
each day, one form of the HVLT was administered, along with several unrelated
tests also being standardized. The order in which the HVLT forms were administered was haphazardized for each subject. Subjects were paid $20 upon completion
of the six brief sessions.
An analysis of variance, with test form and trial as within-subjects repeatedmeasures factors, revealed the six forms to be very similar (see Figure 2). There
was a highly significant effect for learning trial (F2,,, = 240.46, p < .001), but no
form effect (Fs,80= 1.15, N.S.) and no trial-by-form interaction (F1,,,,,= 0.82, N.S.).
The intraclass correlation coefficient (ICC)can be used to estimate the degree
of correspondence among several raters or, as in this case, forms of a test (Bartko
& Carpenter, 1976; Shrout & Fleiss, 1979). Using Shrout and Fleisss formula
3.1, the ICC among the forms for total recall was +0.36. This is a highly significant
correlation (F,,,8S= 4.31, p c .OOl).
Analysis of recognition scores revealed that subjects made slightly fewer
=2.40,<
~ .05) (see
true-positive recognitions on Form 6 than on the others (FS,*,
= 2.23, N.S.) did not differ
Figure 3). Semantically related false-positives (F5,80
among the forms, and no subject made any semantically-unrelated false-positives. Discrimination scores for the six forms were equivalent (Fs,80= 1.91, N.S.).
0-0
12-
v--0
11 --
A.
List 1
List 2
-0
0
10-
e,
I
Y
e,
9--
E
3
z
8-
List 4
List 5
List 6
7 --
2
Trials
Fig. 2. Learning curves for the six forms of the Hopkins Verbal Learning Test in normal
subjects. Means (+SO).
131
Yes/No Recognition
T
6T
12
v)
5/
v)
@ Q
Semantically
Unrelated
.-> lo
@
Semantically
Related
7
6
Downloaded At: 14:12 24 November 2010
List
Fig. 3. Performance of normal subjects on recognition portion of the Hopkins Verbal
Learning Test. Left panel shows number of true-positive recognitions (hits) for
each list. Right panel shows number of semantically related and semantically
unrelated false positive errors for each list. Means (+ SD).
Lists
List
132
JASON BRANDT
Table 3. Characteristics of amnesic, Alzheimers disease, and older normal subjects administered the Hopkins Verbal Learning Test. Means (+ SD). The Wechsler Adult
Intelligence Scale - Revised and the Wechsler Memory Scale - Revised were
administered only to the amnesic patients.
Amnesic
Alzheimer
Normal
45
18
Age, yrs.
77.00
(2.45)
73.64
(8.78)
69.44
(3.70)
Education Level
3.00
(1.73)
2.84
(1.22)
2.39
(0.92)
Mini-Mental
State Exam
27.33
(1.25)
18.50
(6.45)
29.50
(0.73)
WAIS-R
Full-Scale I.Q.
109.33
(16.21)
-_
__
AttentionlConcentration
Index
1 1 0.00
(15.29)
-_
__
80.00
(3.74)
-_
_-
74.00
(1 3.64)
__
_-
DISCUSSION
These initial studies suggest that the six forms of the Hopkins Verbal Learning
Test are nearly identical and highly intercorrelated in normal subjects. While
Study 1 found fewer semantically related false-positives on Form 3 than Forms 1
133
127
11 -lo--
- 9-B
5
2
lx
a-765-
;
L
43 -2 -r
1
0,
2
Trials
Fig. 4. Learning curves on the Hopkins Verbal Learning Test for normal subjects age 65
and older ( n = 18). patients with probable Alzheimers disease ( n = 45). and patients with chronic global amnesia ( n = 3). Means (+ SD).
Yes/No Recognition
11
Semantically
Related
Semantically
Unrelated
31 i
4-I:
1
Normal
Aiz
Groups
Amn
o : : :
Norm Ak Amn
Groups
i
.
: e !
Norm Ak k n n
- +
Groups
Fig. 5. Performance of older normals, Alzheimers disease patients, and amnesic patients
on recognition portion of the Hopkins Verbal Learning Test. Left panel shows
number of true-positive recognitions (hits). Right panel shows number of semantically related and semantically unrelated false positive errors. Means (+ SD).
134
JASON BRANDT
and 4, the incidence of these errors in normals is so low and the variability
among forms so small as to be of no practical significance. Similarly, the very
slightly lower number of correct recognitions on Form 6 seen in Study 2 is not
likely to compromise the clinical utility of the test. The intraclass correlation
among forms reached an acceptable level.
There are several advantages of the HVLT over many existing memory tests.
First, it requires no more than 10 min to administer. Second, it is well-tolerated
by even moderately to severely demented patients, while not having a ceiling
effect (in recall) in neurologically normal subjects. Third, the existence of six
comparable forms makes the HVLT particularly useful in research where patients
are assessed at frequent intervals.
The HVLT was recently employed in a study of the effects of intravenous
physostigmine in Alzheimers disease (Tune et al., in press). Patients received an
intravenous infusion of one of three doses of drug, or placebo, on four consecutive days. Memory and other cognitive functions had to be administered rapidly,
during the 20 min of maximum drug effect. The HVLT proved an ideal test for
this purpose.
The lack of correlation of HVLT performance with age in the normal sample
may be due to at least two factors. First, only seven individuals in Study 1 were
age 70 or older. Recent studies have found significant verbal learning impairments
only in this older age range (Geffen, Moar, OHanlon, Clark, & Geffen, 1990;
Trahan, Larrabee, Quintana, Goethe, & Willinghamet, 1989). Second, the 129
normal subjects were volunteers in a number of neuropsychological and psychiatric research studies, not a random sample of the population. The older people
who opted to participate in these studies may have been particularly healthy and
well-functioning individuals (as is suggested by their high level of education).
While the primary data derived from the HVLT are a few, simple parameters
of verbal learning, many more qualitative features can also be examined. The
extent to which subjects appreciate the semantic structure of the list can be
assessed by computing a category clustering score, similar to that on the California
Verbal Learning Test (Delis et al., 1986). Number of perseverations and intrusion
errors, or percentage of total errors that are of these varieties, can also be computed
from the recall trials. Short- and long-delay recall and recognition trials can, of
course, be added. However, this would add significantly to administration time
and defeat the purpose of developing a simple, brief test.
Clearly, additional studies of the HVLT would be needed before it could be
recommended for widespread clinical use. More extensive age- and educationnorming is needed. It would also be important to establish the HVLTs construct
and concurrent validity, and its sensitivity and specificity to the forms of memory
disorder seen in various diseases. Finally, determination of the tests predictive
value in detecting memory impairment in various settings would be necessary
before it could be used for screening.
Without a doubt, the Hopkins Verbal Learning Test does not provide a comprehensive memory assessment, nor can it substitute for a thorough clinical
135
examination. It certainly can not compete with the California Verbal Learning
Test, for example, for assessing the memory processes that contribute to adequate or inadequate memory performance. However, there are times and situations in every neuropsychologists practice when the choice is between a very
brief assessment of verbal learning capacity or no assessment at all. For these
situations, or as part of a larger memory battery, the HVLT may become a
valuable instrument.
REFERENCES
Bartko, J.J., & Carpenter, W.T. (1976). On the methods and theory of reliability. Journal
of Nervous and Mental Disease, 163,307-311.
Battig, W.F., & Montague, W.E. (1969). Category norms for verbal items in 56 categories:
A replication and extension of the Connecticut category norms. Journal of Experimental Psychology, Monograph, 80, 1-46.
Buschke, H. (1 973). Selective reminding for the analysis of memory and learning. Journal
of Verbal Learning and Verbal Behavior, 12,543-550.
Buschke, H., & Fuld, P.A. (1974). Evaluating storage, retention, and retrieval in disordered
memory and learning. Neurology, 24, 1019-1025.
Cullum, C.M., Butters, N., TrtSster, A.I., & Salmon, D.P. (1990). Normal aging and
forgetting rates on the Wechsler Memory Scale - Revised. Archives of Clinical
Neuropsychology, 5.23-30.
Delis, D.C., Kramer, J.H., Kaplan, E., & Ober, B.A. (1986). California Verbal Learning
Test. New York: The Psychological Corporation.
Delis, D.C., McKee, R., Massman, P., Rramer, J.H. Kaplan, E., & Gettman, D. (1991).
Alternate form of the California Verbal Learning Test: Development and reliability.
The Clinical Neuropsychologist, 5 , 154-162.
Erickson, R.C., & Scott, M.L. (1977). Clinical memory testing: A review. Psychological
Bulletin, 84, 1130-1149.
Folstein, M.F., Folstein, S.E., & McHugh, P.R. (1975). Mini-Mental State: A practical
method for grading the cognitive state of patients for the clinician. Journal of
Psychiatric Research, 12, 189-198.
Francis, W.N., & KuFera, H. (1982). Frequency analysis of English usage: Lexicon and
grammar. Boston: Houghton Mifflin.
Franzen, M.D., Tishelman, A,, Smith, S., Sharp, B., & Friedman, A. (1989). Preliminary
data concerning the test-retest and parallel forms reliability of the Randt Memory
Test. The Clinical Neuropsychologist, 3, 25-28.
Geffen, G., Moar, K.J., OHanlon, A.P., Clark, C.R., & Geffen, L.B. ( 1 990). Performance
measures of 16- to 86-year-old males and females on the Auditory Verbal Learning
Test. The Clinical Neuropsychologist, 4.45-63.
Hannay, H.J., & Levin, H.S. (1985). Selective Reminding Test: An examination of the
equivalence of four forms. Journal of Clinical and Experimental Neuropsychology ,
7, 251-263.
Lezak, M.D. (1983). Neuropsychological assessment (2nd ed.) New York: Oxford University Press.
Loring, D. W., & Papanicolaou, A.C. (1 987). Memory assessment in neuropsychology:
Theoretical considerations and practical utility. Journal of Clinical and Experimental Neuropsychology, 9, 340-358.
McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., & Stadlan, M. (1984).
Clinical diagnosis of Alzheimers disease. Neurology, 34, 939-944.
136
JASON BRANDT
Prigatano, G.P. (1978). Wechsler Memory Scale: A selective review of the literature.
Journal of Clinical Psychology, 34, 816-832.
Randt, C.T., & Brown, E.R. (1979). The Randt Memory Test. Bayport. NY: Life Science
Associates.
Randt, C.T., Brown, E.R., & Osborne, D.P. (1980). A memory test for longitudinal
measurement of mild to moderate deficits. Clinical Neuropsychology, 2,184-194.
Ruff, R.M., Light, R.H., & Quayhagen, M. (1988). Selective Reminding Tests: A normative
study of verbal learning in adults. Journal of Clinical and Experimental
Neuropsychology, 11.539-550.
Russell, E.W. (1981). The pathology and clinical examination of memory. In S.B. Filskov
& T.J. Boll (Eds.).Handbookof clinical neuropsychology (pp. 287-340). New York:
John Wiley & Sons.
Shout, P.E., & Fleiss, J.L. (1979). Intraclass correlation: Uses in assessing rater reliability.
Psychological Bulletin, 86.420-428.
Snodgrass, J.G., & Corwin, J. (1988). Pragmatics of measuring recognition memory:
Applications to dementia and amnesia. Journal of Experimental Psychology:
General, 117,34-50.
Trahan, D.E., Larrabee, G.L., Quintana, J.W., Goethe, K.E., & Willingham, A.C. (1989).
Developmental and clinical validation of an expanded paired associate test with
delayed recall. The Clinical Neuropsychologist, 3 , 169-183.
TrSster, A.I., Jacobs,D., Butters, N., Cullum, C.M., & Salmon, D.P. (1989). Differentiating
Alzheimers disease from Huntingtons disease with the Wechsler Memory ScaleRevised. Clinics in Geriatric Medicine, 5,611-632.
Tune, L., Brandt, J., Frost, J.J., Harris, G., Mayberg, H., Steele, C., Sapp, J., Folstein,
M.F., Wagner, H., & Pearlson, G. (in press). Physostigmine in Alzheimers disease: Effects on cognition, cerebral glucose metabolism analyzed by PET and
cerebral blood flow analyzed by SPECT. Acta Psychiatrica Scandinavica.
Wechsler. D. (1945). A standardized memory scale for clinical use. Journal of Psychology, 19, 87-95.
Wechsler, D. (1987). Wechsler Memory Scale-Revised. New York: Psychological Corporation.
137
APPENDIX
LION
EMERALD
HORSE
TENT
Downloaded At: 14:12 24 November 2010
SAPPHIRE
HOTEL
CAVE
OPAL
TIGER
PEARL
cow
HUT
# Correct
Part B: Recoanition
HORSE
ruby*
CAVE
balloon
coffee
LION
house*
OPAL
TIGER
boat
scarf
PEARL
HUT
EMERALD
SAPPHIRE
dog*
apartment* penny
TENT
mountain
cat*
HOTEL
COW
# True Positives:
diamond*
-I12
# False-Positive Errors,
Related: -16
Unrelated:
(# False-Positives) =
-I6
138
JASON BRANDT
FORK
RUM
PAN
PISTOL
SWORD
SPATULA
BOURBON
VODKA
POT
BOMB
RIFLE
WINE
# Correct
Part 8: Recosnlflon
spoon*
PISTOL
doll
whiskey*
FORK
POT
harmonica
can opener*
SWORD
pencil
gun*
VODKA
knife*
RUM
trout
BOMB
PAN
gold
WINE
lemon
SPATULA
BOURBON beer*
# True Positives:
-112
# False-Positive Errors,
Related: -16
Unrelated: ___I6
(# False-Positives) =
RIFLE
139
SUGAR
TRUMPET
VIOLIN
COAL
GARLIC
KEROSINE
VANILLA
WOOD
CLARINET
FLUTE
CINNAMON
GASOLINE
# Correct
Part 6: Recoanition
pepper*
GARLIC
WOOD
drum*
oil*
SUGAR
ball
salt*
priest
chair
COAL
CLARINET
TRUMPET
basement
CINNAMON
FLUTE
electricity*
moon
GASOLINE
sand
piano*
VIOLIN
KEROSINE VANILLA
# True Positives: -112
# False-Positive Errors,
Related:
-I6
Unrelated: __I6
(# False-Positives) =
140
JASON BRANDT
CANARY
SHOES
EAGLE
BLOUSE
NAILS
CROW
BLUEBIRD
SCREWDRIVER
PANTS
CHISEL
SKIRT
WRENCH
# Correct
Part 8: Recoqnltlon
BLUEBIRD shirt*
CHISEL
EAGLE
chocolate
robin*
chapel
SCREWDRIVER
CROW
sparrow*
WRENCH
PANTS
NAILS
socks*
child
SHOES
hair
hammer*
CANARY
apple
SKIRT
saw*
silver
BLOUSE
# True Positives:
-I12
# False-Positive Errors,
Related: -16
Unrelated:
(# False-Positives) =
-I6
141
TEACHER
BASKETBALL
LETTUCE
DENTIST
TENNIS
BEAN
ENGINEER
POTATO
PROFESSOR
GOLF
CORN
SOCCER
# Correct
Part B: Recosnitlon
TENNIS
football*
PROFESSOR
spinach*
lawyer*
submarine
GOLF
DENTIST
LEl-rUCE
spider
water
BEAN
CORN
baseball*
TEACHER
snake
SOCCER
POTATO
tulip
BASKETBALL doctor*
carrot*
ENGINEER glove
# True Positives: __ I1 2
# False-Positive Errors,
Related: -16
Unrelated:
(# False-Positives) =
-I6
142
JASON BRANDT
SHARK
WALL
HERRING
RAIN
FLOOR
HAIL
CATFISH
ROOF
SALMON
STORM
CEILING
SNOW
# Correct
Part 8: Recoan/f/on
HAIL
bass*
SNOW
bank
FLOOR
mustard
window*
CEILING
canyon
RAIN
ladder
STORM
HERRING
SALMON
tornado*
trout*
melon
ROOF
SHARK
hurricane*
elbow
CATFISH
WALL
door*
# True Positives: /
# False-Positive Errors,
Related: /
Unrelated: /