Top Lang Disorders

Vol. 28, No. 2, pp. 115130

Copyright c
2008 Wolters Kluwer Health | Lippincott Williams & Wilkins
Emerging Procedures in
Narrative Assessment
The Index of Narrative Complexity
Douglas B. Petersen, MA, MEd; Sandra Laing Gillam, PhD;
Ronald B. Gillam, PhD
This article summarizes norm-referenced and criterion-referenced measures of narration and intro-
duces a new criterion-referenced narrative scoring system called the Index of Narrative Complex-
ity (INC). The INC was designed to be used as a progress-monitoring tool. A preliminary study of
the use of this scoring system is reported that yielded evidence suggesting that the INC can be
scored consistently, yields similar scores across five elicitation formats, is sensitive to change after
intervention, and correlates highly with the Test of Narrative Language (Gillam & Pearson, 2004).
These results support the use of the INC as a clinical tool for evaluating the outcomes of narrative
intervention. Key words: narrative, language assessment, progress monitoring, storytelling,
psychometrics
N
ARRATIVES are stories about real or
imagined events that are constructed
by weaving together sentences about situ-
ational contexts, characters, actions, moti-
vations, emotions, and outcomes (Gillam &
Pearson, 2004). Narration is an important ve-
hicle for academic, social, linguistic, and cul-
tural learning. Children use narratives to relate
events, establish and maintain friendships,
and express their thoughts and feelings about
important topics (McCabe & Bliss, 2003).
From the Department of Communicative Disorders
and Deaf Education, Utah State University, Logan.
The authors thank Chad Bingham for helping design
the intervention and for providing therapy. Casey Laing
illustrated the wordless picture books and Olivia Pe-
tersen designed the icons that were used in interven-
tion. We also thank Laura Hendricks, Kirsten Hox-
ley, Alyssa Donaldson, Capri Dana, Angie Miles, Amy
Montero, and Katherine Christensen, who administered
tests, transcribed the stories, and scored them. We spe-
cially thank the children who participated in the study
and their parents. The last author has a financial inter-
est in the Test of Narrative Language.
Corresponding author: Ronald B. Gillam, PhD, Depart-
ment of Communicative Disorders and Deaf Education,
Utah State University, 1000 Old Main Hill, Logan, UT
84321 (e-mail: ron.gillam@usu.edu).
Children with language disorders often
have problems understanding and produc-
ing narratives. Studies have shown that their
narratives are less complete, complex, and
well organized than narratives told by same-
age, typically developing children (Bishop &
Adams, 1992; Boudreau & Chapman, 2000;
Chapman et al., 1997; Fey, Catts, Proctor-
Williams, Tomblin, & Zhang, 2004; Gillam &
Johnston, 1992; Greenhalgh & Strong, 2001;
Newman &McGregor, 2006; Scott &Windsor,
2000). Difficulties with narrative comprehen-
sion and production can negatively impact
childrens academic, social, linguistic, and cul-
tural development (Mehta, Foorman, Branum-
Martin, & Taylor, 2005; Nation, Clarke, &
Marshall 2004; Snyder & Downey, 1991).
Given the developmental importance of narra-
tive comprehension and production, numer-
ous authors have suggested that clinicians
evaluate spoken narratives as a routine part of
language assessment (Boting, 2002; Fey, Catts,
Proctor-Williams, Tomblin, & Zhang, 2004;
Mu

noz, Gillam, Pe

na, & Gulley-Faehnle, 2003;

Pe

na et al., 2006).
Two kinds of tests, norm-referenced and
criterion-referenced, have been used to as-
sess narrative performance in children. Using
115
116 TOPICS IN LANGUAGE DISORDERS/APRILJUNE 2008
norm-referenced tests, a clinician can quickly
and confidently determine whether children
are producing age-appropriate narratives and,
if not, how far their narrative skills are from
average. At present, there are more than a
dozen norm-referenced instruments available
to evaluate childrens speech sound produc-
tion skills, and many more for use in evalu-
ating childrens receptive and expressive lan-
guage skills. The situation is different when
it comes to standardized tests of childrens
narration. Currently, there are few norm-
referenced instruments that focus on chil-
drens narrative skills.
Norm-referenced tests provide scores
that reflect how an individual compares
to the standardization sample. A standard
score indicates how far an individuals score
falls from mean in terms of the standard
deviation of the normative distribution
(Anastasi & Urbina, 1997). A commonly used
standard score is a deviation quotient with a
mean of 100 and a standard deviation of 15.
Usually, deviation quotients of 85 or lower (1
or more standard deviations below the mean)
are interpreted as being outside the normal
range. Another commonly used standard
score is a percentile. Percentile scores repre-
sent the number of same-age children in the
normative sample who scored below a given
raw score (Anastasi & Urbina, 1997). A child
who scores at the 20th percentile earned a
raw score that was higher than the raw scores
for 20% of the children in the normative
sample in his or her age group. Examiners are
usually concerned about children who score
at or below the 10th percentile.
Criterion-referenced tests (also known as
domain-referenced tests) are not designed for
comparing one child to another. Rather, they
are useful for describing the knowledge and
skill that a child has with respect to a particu-
lar ability (Bond, 1996). Examiners administer
criterion-referenced tests if they want to de-
termine whether or not a child can perform
a given task (Anastasi & Urbina, 1997). For
example, to produce a simple episodic narra-
tive, a child would need to talk about a char-
acter who performs an action in response to
some event or problemand explain the conse-
quence of the action in relation to the initiat-
ing event. Checklists, rating scales, or rubrics
often are used as tools for analyzing the results
of criterion-referenced tests.
Scoring of criterion-referenced measures
may be accomplished in a number of ways.
It is possible to calculate the percentage of
appearance of a particular structure (e.g.,
30% of the utterances in a story contain two
or more clauses). Some criterion-referenced
measures also set a standard for mastery or
proficiency. For example, a clinician might
decide that at least one initiating event,
one attempt, and one consequence must be
present for a child to have mastereda simple
episodic narrative. Other criterion-referenced
measures simply describe the childs perfor-
mance (e.g., a childs story is an action se-
quence). Later in this article, we summarize
criterion-referenced scoring systems that as-
sign points for various attributes of childrens
stories. For now, it is sufficient to say that
criterion-referenced measures represent the
extent to which a child performs a task irre-
spective of how other children perform on
the same task (Linn & Gronlund, 2000).
Both norm-referenced and criterion-
referenced measures are useful in the
assessment of narrative comprehension and
production. In this article, we summarize
norm-referenced and criterion-referenced
measures of narrative performance, outline
strengths and weaknesses associated with
each method, and offer suggestions about
how these measures may be useful in the
assessment and intervention of narrative
production problems in children. Finally,
we present a preliminary investigation of a
new criterion-referenced scoring procedure
designed to be a valid and reliable measure of
progress during narrative intervention.
NORM-REFERENCED ASSESSMENTS
OF NARRATION
Two standardized norm-referenced instru-
ments that were specifically designed for the
assessment of narratives are The Renfrew Bus
Emerging Procedures in Narrative Assessment 117
Story (Cowley & Glasgow, 1997) and the Test
of Narrative Language (Gillam & Pearson,
2004).
The Renfrew Bus Story
The Renfrew Bus Story (Cowley &
Glasgow, 1997; hereafter referred to as The
Bus Story) is based on a British screening test
of narrative recall (Renfrew, 1969). It assesses
narration in children between the ages of
3 years 6 months and 6 years 11 months.
Children look at a series of 12-sequenced
pictures while they listen to the examiner
read a scripted story about a bus that runs
away from its driver. Then, children retell
the story while looking at the sequenced
pictures. Examiners transcribe the retold
stories from audiotape before scoring them
according to four quantitative categories: in-
formation (number of details in the retelling
that match the story); sentence length (mean
number of words for the five longest sen-
tences); complexity (number of sentences
containing subordinate or relative clauses);
and independence (amount of examiner
prompting). Administration time for The Bus
Story is about 30 min; however, this test
demands considerably more examiner time
for transcription, coding, and scoring (Bain,
2001). The Bus Story was normed on 418
children in the United States who attended
public, private, and parochial schools.
The British and American versions of The
Bus Story have been studied by a num-
ber of independent investigators. Bishop and
Edmundson (1987) and Stothard, Snowling,
Bishop, Chipcase, and Kaplan (1998) found
that the British version of The Bus Story was
useful for predicting language impairment
outcome. In these studies, 4-year olds with
language impairment who earned higher in-
formation scores on The Bus Story were more
likely to have good outcomes at age 5 years
6 months and at age 15 years. Pankratz, Plante,
Vance, and Insalaco (2007) examined the diag-
nostic and predictive validity for an American
version of The Bus Story. They concluded that
the test was useful for predicting language
outcome after 3 years, but it was not useful
for diagnosing language impairment. Specifi-
cally, the American version of The Bus Story
tended to overidentify children with normal
language development as having language im-
pairments.
The Test of Narrative Language
The Test of Narrative Language (TNL;
Gillam & Pearson, 2004) is another standard-
ized, norm-referenced measure of narrative
ability. The TNL was designed to measure nar-
rative comprehension and production in chil-
dren aged between 5 and 12 years of age.
The TNL elicits information in three narra-
tive formats: (1) a familiar fast-food restau-
rant script with no picture cues, (2) common
personal events presented with sequenced
picture cues, and (3) fictional events pre-
sented with single picture cues. In each for-
mat, children listen to a story, answer ques-
tions about it, and then create a new story.
Scoring is accomplished by listening to audio-
tape recordings of the responses made by chil-
dren during each task. Examiners check for
story content, macrostructural elements, and
microstructural elements. No transcription is
necessary; the clinician indicates the nature
of the response observed for each item di-
rectly on the test protocol. The raw scores
obtained on the TNL yield a Narrative Com-
prehension standard score, an Oral Narration
standard score, and a Narrative Language In-
dex that reflects a composite score for com-
prehension and production. The TNL takes
about 25 min to administer and another 40
min to score.
The TNL was normed on 1,059 children
stratified by geographic area, gender, race,
ethnicity, family income, exceptionality sta-
tus, and age; however, there were fewer 5-
year-olds than in the other age groupings and
upper family income groups are slightly over-
represented (Baxter, 2006). Interscorer relia-
bility is above .90 and testretest reliability
is above .80 for all subtests, suggesting min-
imal test error. The content validity of the
TNL scoring was demonstrated by high corre-
lations between TNL scores and language sam-
ple analyses. In a randomized clinical trial of
118 TOPICS IN LANGUAGE DISORDERS/APRILJUNE 2008
216 children with language impairments, the
TNL has been shown to be useful for identify-
ing language impairments in children (Gillam,
et al., 2008). Spaulding, Plante, and Farinella
(2006) found that the TNL was one of only five
tests with acceptable identification accuracy
(sensitivity = 0.92; specificity = 0.87).
Once a child has been identified as having
a language disorder that involves difficulties
with narrative language, an intervention pro-
gram is typically implemented and progress
toward specific goals is measured (Deno,
2003; Fuchs, Deno, & Mirkin, 1984). Al-
though norm-referenced tests are very useful
for diagnosing and predicting impairment in
narrative performance, they are not designed
for the purpose of monitoring progress
(McCauley & Swisher, 1984; Zhang &
Tomblin, 2003). This is because norm-
referenced tests are designed to be stable
over time. Thus, they may not be sensitive
to incremental changes in a childs per-
formance on a certain skill. In addition,
norm-referenced tests do not always contain
items that measure the specific learner out-
comes being taught. The next section focuses
on criterion-referenced assessments, which
can serve as a guide for establishing specific
treatment objectives and for monitoring
progress toward these objectives over time.
CRITERION-REFERENCED ASSESSMENTS
One commonly known criterion-refer-
enced procedure for analyzing language
content, form, and use is language sample
analysis (Balason & Dollaghan, 2002; Bennett-
Kastor, 1988; Brown, 1973; Evans & Craig,
1992). Language sample analysis is especially
useful for assessing the presence or absence
of specific linguistic structures and how these
structures interact in discourse. Consistent
methods can be used to standardize how
language samples are collected, transcribed,
and scored to permit comparison at several
points over time.
Narrative language sample analysis has
been conducted at both the macrostruc-
tural and microstructural levels. Clinicians
may be interested in macrostructural ele-
ments such as story grammar (e.g., char-
acter, setting, and initiating events) or
microstructural elements (e.g., pronoun ref-
erence and cohesive adequacy). Other mea-
sures of overall productivity (e.g., length) and
artfulness (elaboration) have been examined
as well (Hughes, McGillvray, & Schmidek,
1997; Justice et al., 2006; Ukrainetz, 2006;
Ukrainetz, Justice, Kaderavek, Eisenberg, &
Gillam, 2005).
One of the most commonly used methods
to elicit a narrative has been to ask children
to tell a story that is based on the wordless
picture book, Frog, Where Are You? (Mayer,
1969). This wordless picture book consists
of a sequence of illustrations that depict a
story about a boys attempt to find a frog that
escaped from his room. Berman and Slobin
(1994) found this procedure useful for obtain-
ing stories from children who spoke differ-
ent languages because it gave the investiga-
tors some control over the plot structure and
the general content of the stories the children
told. The use of narrative language sample
analysis procedures can provide descriptive
information about mean length of utterance,
lexical diversity (number of different words
in sample), number of story propositions and
episodes, and types of complex and/or gram-
matically incorrect utterances that appear in
the story.
The Narrative Structure Score
One example of a computer-assisted
approach to language sample analysis is
Systematic Analysis of Language Transcripts
(SALT; Miller & Chapman, 2004). SALT con-
tains a narrative coding scheme called the
Narrative Structure Score (NSS). The NSS
is based on Stein and Glenn (1978, 1982)
and includes instructions on how to code
categories of story grammar and cohesion (in-
troduction, character development, mental
states, referencing, conflict/resolution, cohe-
sion, and conclusion). Each category is scored
Emerging Procedures in Narrative Assessment 119
using a 0- to 5-point Likert scale, with anchors
(and intervals) ranging from 5 (proficient),
3 (emerging or inconsistent), 1 (immature
or minimal), or 0 (poor). Scores of 2 and 4
are used to describe performance that falls
somewhere between the other levels.
One concern about the NSS scoring system
is that the descriptions of the criteria for the
scores are quite brief. For example, a profi-
cient score (5 points) under the referencing
category calls for necessary antecedents to
pronouns (with) references (that) are clear
throughout the story. The emerging score
(3 points) calls for inconsistent use of refer-
ents/antecedents. It is unclear what percent-
age of ambiguous reference is necessary to
warrant a score of 2, 3, or 4. Similar problems
are encountered throughout the NSS coding
scheme, particularly with the mental states
and cohesion coding. The scores for each
story characteristic can be calculated individ-
ually or combined for a total score. A section
in the SALT manual is designed to assist new
coders in assigning scores to transcripts, but
no reliability data are reported for how accu-
rately narratives are coded and scored by in-
dependent raters.
The authors (Miller & Chapman, 2004) re-
port three databases for use in comparing
childrens narratives. The Wisconsin database
consists of narrative and conversational sam-
ples from 394 children between the ages of
3 and 13 years. The narrative elicitation con-
text included asking children to tell about fa-
miliar stories, movies, or episodes from tele-
vision. The Texas English Language Learn-
ers database consists of narratives from 800
Spanish-speaking bilingual children aged be-
tween 5 years 5 months and 9 years. The nar-
rative elicitation context required children to
tell the story Frog, Where Are You? (Mayer,
1969). The San Diego database consists of
250 typically developing children aged from
4 years 4 months to 9 years 11 months.
Children participated in conversational and
story-retelling tasks. Story retelling prompts
included the use of the story Frog, Where
Are You? (Mayer, 1969), Pookins Gets Her
Way (Lester, 1987), and A Porcupine Named
Fluffy (Lester, 1986).
The Strong Narrative Assessment
Procedure
Another criterion-referenced assessment
protocol is The Strong Narrative Assessment
Procedure (SNAP; Strong, 1998). This mea-
sure yields standardized samples of text-level
discourse in children aged between 7 and 10
years. The SNAP contains four audiotaped sto-
ries that correspond to the wordless picture
books Frog, Where Are You? (Mayer, 1969), A
Boy, a Dog and a Frog (Mayer, 1967), Frog
Goes to Dinner (Mayer, 1974), and One Frog
Too Many (Mayer & Mayer, 1975). The story,
Frog Goes to Dinner (Mayer, 1974), is used to
train the procedure. Examiners can adminis-
ter any one of the other stories for the analy-
sis. Children listen to one of the tape-recorded
narratives while looking at the pictures in the
corresponding wordless picture book. Ana

ve
listener situation is created by either having
the children listen to the story with head-
phones while the examiner is in the room or
by having the examiner leave the room when
the story is playing. Children then retell the
narrative (without using the book as a refer-
ence) and respond to a set of inferential and
factual questions about the story.
Examiners are instructed to select any of
the three books to administer and score be-
cause they are reported to elicit stories of simi-
lar length, syntactic complexity, cohesive den-
sity, and story grammar complexity. However,
in an investigation of the equivalency of the
SNAP stories, John, Lui, and Tannock (2003)
found that the story, A Boy, a Dog and a Frog
was easier to retell, resulting in more accurate
story grammar components and higher infer-
ential comprehension scores than the other
two stories. Therefore, clinicians should ad-
minister either, Frog, Where Are You? or One
Frog Too Many if the SNAP is to be used as a
progress-monitoring instrument.
The narrative retelling is recorded, tran-
scribed, and analyzed for the use of 26
narrative macrostructure and microstructure
120 TOPICS IN LANGUAGE DISORDERS/APRILJUNE 2008
Table 1. Summary of the strengths and weaknesses of standardized norm-referenced and
criterion-referenced tests of narration
Test Main Strengths Main Weaknesses
The Renfrew Bus Story
(Cowley & Glasgow,
1997)
Easy and fast to administer
Reliable scores for information
and sentence length
Normed on preschool-aged
children
Lengthy scoring process
Weak psychometric qualities for the
complexity dimension
Not useful for children more than
7 years of age
Potential bias against nonmainstream
children
The Test of Narrative
Language (Gillam &
Pearson, 2004)
Useful for children aged 6 years
to 11 years 11 months
Stratified normative sample
Short scoring time
Good reliability and validity
evidence
Good sensitivity and specificity
Smaller normative sample for
5-year-olds
Reliability calculations based on a
portion of the normative sample
The Strong Narrative
Assessment Procedure
(Strong, 1998)
Utilizes well-illustrated stories
Assesses many aspects of
narration
Acceptable interrater reliability
Includes three equivalent
stories
Lengthy scoring process
No validity evidence
Criterion-based primarily from a
small sample, which is racially and
geographically limited
No data to support sensitivity and
specificity
No method for indexing overall
narrative complexity
elements that are organized under the fol-
lowing broad categories: length, syntax, co-
hesion, and story grammar. The SNAP offers
instructions for the calculation of both the fre-
quency and percentage of narrative features,
thereby controlling for narrative length as a
function of narrative element frequency. In an
independent investigation of the SNAP proce-
dures, John et al. (2003) reported interrater
agreements of 81% for segmentation and 87%
for coding.
The SNAP manual suggests interpretive
guidelines for comparing childrens perfor-
mance to that of children who participated
in field-testing. Because of the small sample
size and limited information on psychomet-
ric properties, norm-referenced comparisons
should be made with great caution. As with
other assessment techniques that rely on tran-
scription, the SNAP is time intensive. Further-
more, the SNAP does not necessarily focus on
the critical attributes of narratives, nor does
it provide clinicians with a composite score.
However, a comprehensive examination of
narrative samples using the procedures out-
lined in the SNAP may be useful for describing
narrative samples and for monitoring ongoing
changes in performance.
The criterion-referenced measures that we
have outlined thus far could be helpful as pe-
riodic assessments for monitoring change dur-
ing intervention. However, both the NSS and
the SNAP are time-intensive procedures, and
their psychometric properties for monitoring
change are yet to be established. Given the
concerns with the NSS and the SNAP, there
appears to be a need for a valid and reliable
criterion-referenced narrative scoring proce-
dure that evaluates critical attributes of nar-
ratives and provides a composite score that
Emerging Procedures in Narrative Assessment 121
reflects the overall quality of a narrative. In
addition, a measure is needed that is useful
for reliably evaluating progress over time and
that correlates well with a valid and reliable
norm-referenced narrative assessment. The re-
mainder of the paper describes a newly devel-
oped criterion-referenced scoring procedure
for evaluating narratives and presents prelim-
inary data regarding its reliability and validity.
The Index of Narrative Complexity
The Index of Narrative Complexity (INC)
integrates existing research and concep-
tual/analytic approaches related to the as-
sessment of macrostructural and microstruc-
tural aspects of oral narratives (Gillam &
Pearson, 2004; Hughes et al., 1997; Justice
et al., 2006; Miller & Chapman, 2004; Strong,
1998; Ukrainetz, 2006; Ukrainetz et al., 2005).
Foundational to the INC are the high point
analysis of Labov (1972), the story grammar
analysis presented by Stein and Glen (1978,
1982), and refinements of Peterson and
McCabe (1983).
The INC scoring system (see Table 2) in-
cludes categories for rating the complexity of
characters, setting, initiating events, internal
responses, plans, action/attempts, complica-
tions, consequences, narrator evaluations, for-
mulaic markers, temporal markers, and causal
adverbial clauses. The categories included in
the INC are differentially weighted in favor
of episodic complexity and narrative cohe-
sion. The weighted scores were designed to
reflect the relative importance of each ele-
ment in contributing to the narrative style
of mainstream academic culture (Guthrie &
Hall, 1983). Table 3 illustrates how this story-
coding procedure was used in documenting
steady increases in the complexity of one
childs narrative production skills over the
course of intervention.
When clinicians want to know whether a
child is making progress during intervention,
they need a scoring system that is sensitive
to incremental changes in narrative produc-
tion skills. The INC incorporates an incremen-
tal scoring system so that higher scores re-
flect the appearance of more complex narra-
tive structures. For example, children may be-
gin telling stories using ambiguous referents
such as heor she,then move to more spe-
cific terms including a boyor the girl,and
finally transition to the assignment of names
to their characters such as Juan or Sally.
These features are captured with the INC.
Preliminary reliability and
validity study
A preliminary investigation was conducted
to evaluate the reliability and validity of the
INC for monitoring progress in narrative lan-
guage intervention. Twelve children between
the ages of 6 years 4 months and 9 years
1 month (mean = 8 years 2 months) par-
ticipated in a narrative intervention study.
All the children had been diagnosed as lan-
guage impaired and were receiving language
intervention in the schools. To determine
the reliability of the INC scoring system, we
assessed the children at 1-month intervals
(pretest 1 in early June and pretest 2 in early
July) before we administered treatment. Dur-
ing the treatment phase (described subse-
quently), the children attended narrative lan-
guage intervention sessions four times each
week for a month. The participants were
tested again immediately after the treatment
ended (posttest). The TNL was administered
at the same time to evaluate criterion validity.
The INC scoring system was applied to five
stories that were produced at each assessment
point (pretest 1, pretest 2, posttest). Children
told the Late for School story and Aliens
story from the TNL (described above) and
three stories that were produced in response
to wordless picture books: Bird and his ring
(Miller, 2000), One Frog Too Many (Mayer &
Mayer, 1975), and A Boy, a Dog and a Frog
(Mayer, 1967).
A team of undergraduate students who
were not involved in the narrative interven-
tion collected the oral narratives. The Late
for School and Aliens stories were admin-
istered according to standardized TNL proce-
dures outlined in the manual. For all three
wordless picture books, examiners asked chil-
dren to look through all the pictures in the
122 TOPICS IN LANGUAGE DISORDERS/APRILJUNE 2008
T
a
b
l
e
2
.
I
n
d
e
x
o
f
n
a
r
r
a
t
i
v
e
c
o
m
p
l
e
x
i
t
y
s
t
o
r
y
c
o
d
i
n
g
f
o
r
m
N
a
r
r
a
t
i
v
e
E
l
e
m
e
n
t
0
P
o
i
n
t
s
1
P
o
i
n
t
2
p
o
i
n
t
s
3
p
o
i
n
t
s
C
h
a
r
a
c
t
e
r
A
c
h
a
r
a
c
t
e
r
i
s
a
n
y
r
e
f
e
r
e
n
c
e
t
o
t
h
e
s
u
b
j
e
c
t
o
f
a
c
l
a
u
s
e
i
n
a
n
a
r
r
a
t
i
v
e
.
N
o
m
a
i
n
c
h
a
r
a
c
t
e
r
i
s
i
n
c
l
u
d
e
d
,
o
r
o
n
l
y
a
m
b
i
g
u
o
u
s
p
r
o
n
o
u
n
s
a
r
e
u
s
e
d
.
E
x
a
m
p
l
e
s
a
.
T
h
e
y
w
e
r
e
w
a
l
k
i
n
g
.
b
.
H
e
w
a
s
w
a
l
k
i
n
g
.
I
n
c
l
u
d
e
s
a
t
l
e
a
s
t
o
n
e
m
a
i
n
c
h
a
r
a
c
t
e
r
w
i
t
h
n
o
n
s
p
e
c
i
f
i
c
l
a
b
e
l
s
o
n
l
y
.
N
o
t
e
:
O
n
l
y
c
o
d
e
e
a
c
h
c
h
a
r
a
c
t
e
r
o
n
e
t
i
m
e
.
E
x
a
m
p
l
e
s
a
.
O
n
c
e
t
h
e
r
e
w
a
s
a
b
o
y
.
b
.
T
h
e
b
o
y
w
a
s
w
a
l
k
i
n
g
.
I
n
c
l
u
d
e
s
o
n
e
m
a
i
n
c
h
a
r
a
c
t
e
r
w
i
t
h
a
s
p
e
c
i
f
i
c
n
a
m
e
f
o
r
t
h
e
c
h
a
r
a
c
t
e
r
.
E
x
a
m
p
l
e
O
n
c
e
t
h
e
r
e
w
a
s
a
b
o
y
n
a
m
e
d
C
h
a
r
l
e
s
.
I
n
c
l
u
d
e
s
m
o
r
e
t
h
a
n
o
n
e
m
a
i
n
c
h
a
r
a
c
t
e
r
w
i
t
h
s
p
e
c
i
f
i
c
n
a
m
e
s
.
E
x
a
m
p
l
e
O
n
c
e
t
h
e
r
e
w
a
s
a
b
o
y
n
a
m
e
d
C
h
a
r
l
e
s
a
n
d
a
g
i
r
l
n
a
m
e
d
M
a
r
y
.
S
e
t
t
i
n
g
A
s
e
t
t
i
n
g
i
s
a
n
y
r
e
f
e
r
e
n
c
e
t
o
a
p
l
a
c
e
o
r
t
i
m
e
i
n
a
n
a
r
r
a
t
i
v
e
.
N
o
r
e
f
e
r
e
n
c
e
t
o
a
s
p
e
c
i
f
i
c
o
r
g
e
n
e
r
a
l
p
l
a
c
e
.
E
x
a
m
p
l
e
T
h
e
b
o
y
a
n
d
t
h
e
g
i
r
l
w
e
r
e
w
a
l
k
i
n
g
.
I
n
c
l
u
d
e
s
r
e
f
e
r
e
n
c
e
t
o
a
g
e
n
e
r
a
l
p
l
a
c
e
o
r
t
i
m
e
.
E
x
a
m
p
l
e
s
a
.
T
h
e
b
o
y
a
n
d
t
h
e
g
i
r
l
w
e
r
e
o
u
t
s
i
d
e
.
b
.
I
t
w
a
s
d
a
y
t
i
m
e
.
c
.
O
n
e
d
a
y
,
t
h
e
y
w
e
n
t
t
o
t
h
e
p
a
r
k
.
O
n
e
o
r
m
o
r
e
r
e
f
e
r
e
n
c
e
s
t
o
s
p
e
c
i
f
i
c
p
l
a
c
e
s
o
r
t
i
m
e
s
.
E
x
a
m
p
l
e
s
a
.
O
n
c
e
t
h
e
r
e
w
a
s
a
b
o
y
a
n
d
a
g
i
r
l
w
a
l
k
i
n
g
i
n
C
e
n
t
r
a
l
P
a
r
k
.
b
.
T
h
e
y
w
e
r
e
w
a
l
k
i
n
g
a
t
n
i
g
h
t
.
I
n
i
t
i
a
t
i
n
g
E
v
e
n
t
A
n
i
n
i
t
i
a
t
i
n
g
e
v
e
n
t
i
s
a
n
y
r
e
f
e
r
e
n
c
e
t
o
a
n
e
v
e
n
t
o
r
p
r
o
b
l
e
m
t
h
a
t
e
l
i
c
i
t
s
a
r
e
s
p
o
n
s
e
f
r
o
m
t
h
e
c
h
a
r
a
c
t
e
r
(
s
)
i
n
a
n
a
r
r
a
t
i
v
e
.
A
n
e
v
e
n
t
o
r
p
r
o
b
l
e
m
l
i
k
e
l
y
t
o
e
l
i
c
i
t
a
r
e
s
p
o
n
s
e
f
r
o
m
t
h
e
c
h
a
r
a
c
t
e
r
i
s
n
o
t
s
t
a
t
e
d
.
E
x
a
m
p
l
e
T
h
e
g
i
r
l
l
o
o
k
e
d
a
t
t
h
e
b
o
y
.
T
h
e
b
o
y
a
n
d
g
i
r
l
w
e
r
e
w
a
l
k
i
n
g
i
n
t
h
e
p
a
r
k
.
I
n
c
l
u
d
e
s
a
t
l
e
a
s
t
o
n
e
s
t
a
t
e
d
e
v
e
n
t
o
r
p
r
o
b
l
e
m
t
h
a
t
i
s
l
i
k
e
l
y
t
o
e
l
i
c
i
t
a
r
e
s
p
o
n
s
e
f
r
o
m
t
h
e
c
h
a
r
a
c
t
e
r
,
b
u
t
t
h
e
r
e
i
s
n
o
r
e
s
p
o
n
s
e
d
i
r
e
c
t
l
y
r
e
l
a
t
e
d
t
o
t
h
a
t
e
v
e
n
t
.
E
x
a
m
p
l
e
T
h
e
g
i
r
l
w
a
s
w
a
l
k
i
n
g
i
n
a
p
a
r
k
a
n
d
s
a
w
a
s
p
a
c
e
s
h
i
p
l
a
n
d
(
e
v
e
n
t
/
p
r
o
b
l
e
m
)
a
n
d
s
h
e
s
a
w
s
o
m
e
a
l
i
e
n
s
,
a
n
d
s
h
e
s
a
w
a
d
o
g
,
a
n
d
a
t
a
b
l
e
a
n
d
.
.
.
.
I
n
c
l
u
d
e
s
a
t
l
e
a
s
t
o
n
e
s
t
a
t
e
d
e
v
e
n
t
o
r
p
r
o
b
l
e
m
t
h
a
t
e
l
i
c
i
t
s
a
r
e
s
p
o
n
s
e
f
r
o
m
t
h
e
c
h
a
r
a
c
t
e
r
(
s
)
.
E
x
a
m
p
l
e
T
h
e
g
i
r
l
w
a
s
w
a
l
k
i
n
g
i
n
a
p
a
r
k
a
n
d
s
a
w
a
s
p
a
c
e
s
h
i
p
l
a
n
d
a
n
d
s
h
e
s
a
w
s
o
m
e
a
l
i
e
n
s
(
I
E
)
.
T
h
e
g
i
r
l
s
t
a
r
t
e
d
t
o
r
u
n
a
w
a
y
(
a
c
t
i
o
n
)
.
T
w
o
o
r
m
o
r
e
d
i
s
t
i
n
c
t
s
t
a
t
e
d
e
v
e
n
t
s
o
r
p
r
o
b
l
e
m
s
t
h
a
t
e
l
i
c
i
t
a
r
e
s
p
o
n
s
e
f
r
o
m
t
h
e
c
h
a
r
a
c
t
e
r
(
s
)
.
E
x
a
m
p
l
e
s
T
h
e
g
i
r
l
w
a
s
w
a
l
k
i
n
g
i
n
a
p
a
r
k
a
n
d
s
a
w
a
s
p
a
c
e
s
h
i
p
l
a
n
d
a
n
d
s
h
e
s
a
w
s
o
m
e
a
l
i
e
n
s
(
I
E
-
1
)
.
T
h
e
g
i
r
l
s
t
a
r
t
e
d
t
o
r
u
n
a
w
a
y
(
a
c
t
i
o
n
)
.
B
u
t
w
h
i
l
e
s
h
e
w
a
s
r
u
n
n
i
n
g
,
h
e
r
s
h
o
e
g
o
t
s
t
u
c
k
i
n
a
h
o
l
e
(
I
E
-
2
)
.
S
h
e
q
u
i
c
k
l
y
k
n
e
l
t
d
o
w
n
a
n
d
t
o
o
k
o
f
f
h
e
r
s
h
o
e
t
o
g
e
t
u
n
s
t
u
c
k
(
a
c
t
i
o
n
)
.
(
c
o
n
t
i
n
u
e
s
)
Emerging Procedures in Narrative Assessment 123
T
a
b
l
e
2
.
I
n
d
e
x
o
f
n
a
r
r
a
t
i
v
e
c
o
m
p
l
e
x
i
t
y
s
t
o
r
y
c
o
d
i
n
g
f
o
r
m
(
C
o
n
t
i
n
u
e
d
)
N
a
r
r
a
t
i
v
e
E
l
e
m
e
n
t
0
P
o
i
n
t
s
1
P
o
i
n
t
2
p
o
i
n
t
s
3
p
o
i
n
t
s
I
n
t
e
r
n
a
l
R
e
s
p
o
n
s
e
A
n
i
n
t
e
r
n
a
l
r
e
s
p
o
n
s
e
i
s
a
n
y
r
e
f
e
r
e
n
c
e
t
o
i
n
f
o
r
m
a
t
i
o
n
a
b
o
u
t
a
c
h
a
r
a
c
t
e
r

s
p
s
y
c
h
o
l
o
g
i
c
a
l
s
t
a
t
e
i
n
c
l
u
d
i
n
g
e
m
o
t
i
o
n
s
,
d
e
s
i
r
e
s
,
f
e
e
l
i
n
g
s
,
o
r
t
h
o
u
g
h
t
s
.
N
o
o
v
e
r
t
s
t
a
t
e
m
e
n
t
a
b
o
u
t
a
c
h
a
r
a
c
t
e
r

s
p
s
y
c
h
o
l
o
g
i
c
a
l
s
t
a
t
e
.
O
n
e
o
v
e
r
t
s
t
a
t
e
m
e
n
t
a
b
o
u
t
a
c
h
a
r
a
c
t
e
r

s
p
s
y
c
h
o
l
o
g
i
c
a
l
s
t
a
t
e
n
o
t
c
a
u
s
a
l
l
y
r
e
l
a
t
e
d
t
o
a
n
e
v
e
n
t
o
r
p
r
o
b
l
e
m
.
E
x
a
m
p
l
e
T
h
e
d
o
g
w
a
s
s
a
d
,
t
h
e
g
i
r
l
w
a
s
h
a
p
p
y
.
O
n
e
o
r
m
o
r
e
o
v
e
r
t
s
t
a
t
e
m
e
n
t
s
a
b
o
u
t
a
c
h
a
r
a
c
t
e
r

s
p
s
y
c
h
o
l
o
g
i
c
a
l
s
t
a
t
e
c
a
u
s
a
l
l
y
r
e
l
a
t
e
d
t
o
a
n
e
v
e
n
t
o
r
p
r
o
b
l
e
m
.
E
x
a
m
p
l
e
T
h
e
a
l
i
e
n
s

l
a
n
d
e
d
.
S
a
r
a
s
a
w
t
h
e
s
h
i
p
a
n
d
w
a
s
t
e
r
r
i
f
i
e
d
.
P
l
a
n
A
p
l
a
n
i
s
a
n
y
c
o
g
n
i
t
i
v
e
v
e
r
b
r
e
f
e
r
e
n
c
e
t
h
a
t
i
s
i
n
t
e
n
d
e
d
t
o
a
c
t
o
n
o
r
s
o
l
v
e
a
n
i
n
i
t
i
a
t
i
n
g
e
v
e
n
t
.
I
t
m
u
s
t
i
n
c
l
u
d
e
a

c
o
g
n
i
t
i
v
e
v
e
r
b

t
h
a
t
i
n
d
i
c
a
t
e
s
a
p
l
a
n
.
N
o
t
e
:
T
h
e
p
l
a
n
a
n
d
t
h
e
a
c
t
i
o
n
/
a
t
t
e
m
p
t
c
a
n
s
h
a
r
e
t
h
e
s
a
m
e
c
l
a
u
s
e
(
s
e
e
2
P
o
i
n
t
s
e
x
a
m
p
l
e
b
)
N
o
o
v
e
r
t
s
t
a
t
e
m
e
n
t
i
s
p
r
o
v
i
d
e
d
a
b
o
u
t
t
h
e
c
h
a
r
a
c
t
e
r

s
p
l
a
n
t
o
a
c
t
o
n
o
r
s
o
l
v
e
t
h
e
e
v
e
n
t
o
r
p
r
o
b
l
e
m
.
E
x
a
m
p
l
e
T
h
e
g
i
r
l
w
a
s
v
e
r
y
e
x
c
i
t
e
d
a
n
d
s
h
e
r
a
n
o
u
t
t
o
m
e
e
t
t
h
e
a
l
i
e
n
s
.
O
n
e
o
v
e
r
t
s
t
a
t
e
m
e
n
t
a
b
o
u
t
h
o
w
t
h
e
c
h
a
r
a
c
t
e
r
m
i
g
h
t
s
o
l
v
e
t
h
e
c
o
m
p
l
i
c
a
t
i
o
n
o
r
p
r
o
b
l
e
m
.
E
x
a
m
p
l
e
T
h
e
g
i
r
l
t
h
o
u
g
h
t
t
h
a
t
i
t
w
o
u
l
d
b
e
n
e
a
t
t
o
g
o
a
n
d
m
e
e
t
t
h
e
a
l
i
e
n
s
.
T
w
o
o
v
e
r
t
s
t
a
t
e
m
e
n
t
s
a
b
o
u
t
h
o
w
t
h
e
c
h
a
r
a
c
t
e
r
m
i
g
h
t
a
c
t
o
n
o
r
s
o
l
v
e
t
h
e
e
v
e
n
t
(
s
)
o
r
p
r
o
b
l
e
m
(
s
)
.
E
x
a
m
p
l
e
s
a
.
T
h
e
g
i
r
l
w
a
s
v
e
r
y
e
x
c
i
t
e
d
a
n
d
s
h
e
t
o
l
d
t
h
e
b
o
y
t
h
a
t
s
h
e
w
a
n
t
e
d
t
o
g
o
m
e
e
t
t
h
e
a
l
i
e
n
s
.
b
.
T
h
e
b
o
y
w
a
s
v
e
r
y
s
c
a
r
e
d
s
o
h
e
d
e
c
i
d
e
d
t
o
s
n
e
a
k
a
w
a
y
q
u
i
e
t
l
y
.
T
h
r
e
e
o
r
m
o
r
e
o
v
e
r
t
s
t
a
t
e
m
e
n
t
s
a
b
o
u
t
h
o
w
t
h
e
c
h
a
r
a
c
t
e
r
m
i
g
h
t
a
c
t
o
n
o
r
s
o
l
v
e
t
h
e
e
v
e
n
t
(
s
)
o
r
p
r
o
b
l
e
m
(
s
)
.
A
c
t
i
o
n
/
A
t
t
e
m
p
t
A
c
t
i
o
n
s
a
r
e
t
a
k
e
n
b
y
t
h
e
m
a
i
n
c
h
a
r
a
c
t
e
r
s
b
u
t
a
r
e
n
o
t
d
i
r
e
c
t
l
y
r
e
l
a
t
e
d
t
o
t
h
e
I
E
.
A
t
t
e
m
p
t
s
a
r
e
t
a
k
e
n
b
y
t
h
e
m
a
i
n
c
h
a
r
a
c
t
e
r
(
s
)
t
h
a
t
a
r
e
d
i
r
e
c
t
l
y
r
e
l
a
t
e
d
t
o
t
h
e
I
E
.
N
o
a
c
t
i
o
n
s
a
r
e
t
a
k
e
n
b
y
t
h
e
m
a
i
n
c
h
a
r
a
c
t
e
r
(
s
)
,
E
x
a
m
p
l
e
T
h
e
r
e
i
s
a
g
i
r
l
.
T
h
e
r
e
i
s
a
b
o
y
.
I
t
i
s
s
u
n
n
y
.
A
c
t
i
o
n
s
b
y
m
a
i
n
c
h
a
r
a
c
t
e
r
a
r
e
n
o
t
d
i
r
e
c
t
l
y
r
e
l
a
t
e
d
t
o
t
h
e
I
E
.
E
x
a
m
p
l
e
s
a
.
T
h
e
b
o
y
a
n
d
t
h
e
g
i
r
l
w
e
r
e
w
a
l
k
i
n
g
i
n
a
p
a
r
k
.
b
.
T
h
e
y
s
a
w
a
b
o
y
a
l
i
e
n
w
a
v
i
n
g
.
A
t
t
e
m
p
t
s
b
y
m
a
i
n
c
h
a
r
a
c
t
e
r
a
r
e
d
i
r
e
c
t
l
y
r
e
l
a
t
e
d
t
o
t
h
e
I
E
.
E
x
a
m
p
l
e
T
h
e
g
i
r
l
t
h
o
u
g
h
t
t
h
a
t
i
t
w
o
u
l
d
b
e
n
e
a
t
t
o
g
o
a
n
d
m
e
e
t
t
h
e
a
l
i
e
n
s
s
o
s
h
e
g
o
t
a
w
a
y
f
r
o
m
t
h
e
b
o
y
a
n
d
w
a
l
k
e
d
o
u
t
o
n
t
h
e
g
r
a
s
s
.
(
c
o
n
t
i
n
u
e
s
)
124 TOPICS IN LANGUAGE DISORDERS/APRILJUNE 2008
T
a
b
l
e
2
.
I
n
d
e
x
o
f
n
a
r
r
a
t
i
v
e
c
o
m
p
l
e
x
i
t
y
s
t
o
r
y
c
o
d
i
n
g
f
o
r
m
(
C
o
n
t
i
n
u
e
d
)
N
a
r
r
a
t
i
v
e
E
l
e
m
e
n
t
0
P
o
i
n
t
s
1
P
o
i
n
t
2
p
o
i
n
t
s
3
p
o
i
n
t
s
C
o
m
p
l
i
c
a
t
i
o
n
A
c
o
m
p
l
i
c
a
t
i
o
n
i
s
a
n
e
v
e
n
t
t
h
a
t
p
r
o
h
i
b
i
t
s
t
h
e
e
x
e
c
u
t
i
o
n
o
f
a
p
l
a
n
o
r
a
c
t
i
o
n
t
a
k
e
n
i
n
r
e
s
p
o
n
s
e
t
o
a
n
i
n
i
t
i
a
t
i
n
g
e
v
e
n
t
.
N
o
t
e
:
A
c
o
m
p
l
i
c
a
t
i
o
n
c
a
n
a
l
s
o
b
e
a
s
e
c
o
n
d
i
n
i
t
i
a
t
i
n
g
e
v
e
n
t
.
I
n
t
h
i
s
c
a
s
e
c
o
d
e
b
o
t
h
a
c
o
m
p
l
i
c
a
t
i
o
n
a
n
d
i
n
i
t
i
a
t
i
n
g
e
v
e
n
t
.
N
o
c
o
m
p
l
i
c
a
t
i
o
n
s
.
O
n
e
c
o
m
p
l
i
c
a
t
i
o
n
t
h
a
t
p
r
o
h
i
b
i
t
s
a
p
l
a
n
o
r
a
c
t
i
o
n
f
r
o
m
b
e
i
n
g
a
c
c
o
m
p
l
i
s
h
e
d
.
E
x
a
m
p
l
e
T
h
e
s
p
a
c
e
s
h
i
p
l
a
n
d
e
d
.
T
h
e
g
i
r
l
d
e
c
i
d
e
d
t
o
g
e
t
a
w
a
y
f
r
o
m
t
h
e
a
l
i
e
n
s
a
n
d
s
t
a
r
t
e
d
r
u
n
n
i
n
g
f
r
o
m
t
h
e
s
p
a
c
e
s
h
i
p
.
W
h
i
l
e
s
h
e
w
a
s
r
u
n
n
i
n
g
,
h
e
r
s
h
o
e
g
o
t
s
t
u
c
k
i
n
a
h
o
l
e
.
S
h
e
c
o
u
l
d
n
o
t
g
e
t
a
w
a
y
f
r
o
m
t
h
e
a
l
i
e
n
s
.
T
w
o
d
i
s
t
i
n
c
t
c
o
m
p
l
i
c
a
t
i
o
n
s
t
h
a
t
p
r
o
h
i
b
i
t
p
l
a
n
s
o
r
a
c
t
i
o
n
s
f
r
o
m
b
e
i
n
g
a
c
c
o
m
p
l
i
s
h
e
d
.
E
x
a
m
p
l
e
T
h
e
g
i
r
l
w
a
s
w
a
l
k
i
n
g
i
n
a
p
a
r
k
a
n
d
s
a
w
a
s
p
a
c
e
s
h
i
p
l
a
n
d
a
n
d
s
h
e
s
a
w
s
o
m
e
a
l
i
e
n
s
(
I
E
-
1
)
.
T
h
e
g
i
r
l
s
t
a
r
t
e
d
t
o
r
u
n
a
w
a
y
(
a
c
t
i
o
n
-
1
)
.
B
u
t
w
h
i
l
e
s
h
e
w
a
s
r
u
n
n
i
n
g
,
h
e
r
s
h
o
e
g
o
t
s
t
u
c
k
i
n
a
h
o
l
e
(
c
o
m
p
l
i
c
a
t
i
o
n
-
1
/
I
E
-
2
)
.
S
h
e
q
u
i
c
k
l
y
k
n
e
l
t
d
o
w
n
a
n
d
t
o
o
k
o
f
f
h
e
r
s
h
o
e
t
o
g
e
t
u
n
s
t
u
c
k
(
a
c
t
i
o
n
-
2
)
b
u
t
s
h
e
w
a
s
s
h
a
k
i
n
g
t
o
o
m
u
c
h
t
o
g
e
t
h
e
r
s
h
o
e
o
f
f
(
c
o
m
p
l
i
c
a
t
i
o
n
-
2
)
.
C
o
n
s
e
q
u
e
n
c
e
A
c
o
n
s
e
q
u
e
n
c
e
r
e
s
o
l
v
e
s
t
h
e
p
r
o
b
l
e
m
o
r
d
o
e
s
n
o
t
r
e
s
o
l
v
e
t
h
e
p
r
o
b
l
e
m
.
I
t
m
u
s
t
b
e
r
e
l
a
t
e
d
t
o
t
h
e
I
E
a
n
d
b
e
e
x
p
l
i
c
i
t
l
y
s
t
a
t
e
d
.
N
o
t
e
:
A
c
o
n
s
e
q
u
e
n
c
e
f
o
r
o
n
e
e
p
i
s
o
d
e
c
a
n
o
f
t
e
n
b
e
t
h
e
I
E
f
o
r
a
n
o
t
h
e
r
.
N
o
c
o
n
s
e
q
u
e
n
c
e
t
o
t
h
e
a
c
t
i
o
n
/
a
t
t
e
m
p
t
i
s
e
x
p
l
i
c
i
t
l
y
s
t
a
t
e
d
.
E
x
a
m
p
l
e
s
a
.
S
h
e
g
o
t
a
w
a
y
f
r
o
m
t
h
e
b
o
y
a
n
d
w
a
l
k
e
d
o
u
t
o
n
t
o
t
h
e
g
r
a
s
s
.
b
.
T
h
e
a
l
i
e
n
g
i
r
l
h
a
d
a
d
r
e
s
s
o
n
.
O
n
e
c
o
n
s
e
q
u
e
n
c
e
E
x
a
m
p
l
e
T
h
e
s
p
a
c
e
s
h
i
p
l
a
n
d
e
d
.
T
h
e
g
i
r
l
w
e
n
t
o
u
t
t
o
s
e
e
t
h
e
m
.
T
h
e
a
l
i
e
n
s
w
e
r
e
s
c
a
r
e
d
o
f
h
e
r
.
T
h
e
y
r
a
n
b
a
c
k
t
o
t
h
e
s
h
i
p
a
n
d
f
l
e
w
o
f
f
.
T
w
o
c
o
n
s
e
q
u
e
n
c
e
s
E
x
a
m
p
l
e
s
a
.
T
h
e
y
t
o
l
d
t
h
e
i
r
p
a
r
e
n
t
s
t
h
e
s
p
a
c
e
s
h
i
p
w
a
s
i
n
t
h
e
p
a
r
k
.

B
u
t
t
h
e
i
r
p
a
r
e
n
t
s
d
i
d
n

t
b
e
l
i
e
v
e
t
h
e
m
.

W
h
e
n
t
h
e
y
t
o
o
k
t
h
e
i
r
p
a
r
e
n
t
s
t
o
t
h
e
p
a
r
k

t
h
e
s
p
a
c
e
s
h
i
p
w
a
s
g
o
n
e
.

b
.
T
h
e
b
o
y
w
a
n
t
e
d
a
f
r
o
g
.
H
e
w
e
n
t
t
o
t
h
e
w
o
o
d
s
t
o
f
i
n
d
o
n
e
.
H
e
c
o
u
l
d
n

t
f
i
n
d
a
f
r
o
g
.
H
e
d
e
c
i
d
e
d

h
e
r
e
a
l
l
y
w
a
n
t
e
d
a
d
o
g
.

T
h
r
e
e
o
r
m
o
r
e
c
o
n
s
e
q
u
e
n
c
e
s
(
c
o
n
t
i
n
u
e
s
)
Emerging Procedures in Narrative Assessment 125
T
a
b
l
e
2
.
I
n
d
e
x
o
f
n
a
r
r
a
t
i
v
e
c
o
m
p
l
e
x
i
t
y
s
t
o
r
y
c
o
d
i
n
g
f
o
r
m
(
C
o
n
t
i
n
u
e
d
)
N
a
r
r
a
t
i
v
e
E
l
e
m
e
n
t
0
P
o
i
n
t
s
1
P
o
i
n
t
2
p
o
i
n
t
s
3
p
o
i
n
t
s
F
o
r
m
u
l
a
i
c
m
a
r
k
e
r
s
A
f
o
r
m
u
l
a
i
c
m
a
r
k
e
r
i
s
a
n
y
s
t
a
n
d
a
r
d
u
t
t
e
r
a
n
c
e
u
s
e
d
t
o
m
a
r
k
t
h
e
b
e
g
i
n
n
i
n
g
o
r
e
n
d
i
n
g
o
f
a
n
a
r
r
a
t
i
v
e
.
F
o
r
e
x
a
m
p
l
e
,
T
h
e
e
n
d
,
o
n
c
e
,
o
n
c
e
u
p
o
n
a
t
i
m
e
,
t
h
e
y
l
i
v
e
d
h
a
p
p
i
l
y
e
v
e
r
a
f
t
e
r
.
N
o
f
o
r
m
u
l
a
i
c
m
a
r
k
e
r
s
O
n
e
f
o
r
m
u
l
a
i
c
m
a
r
k
e
r
E
x
a
m
p
l
e
O
n
c
e
u
p
o
n
a
t
i
m
e
T
w
o
o
r
m
o
r
e
f
o
r
m
u
l
a
i
c
m
a
r
k
e
r
s
E
x
a
m
p
l
e
O
n
c
e
u
p
o
n
a
t
i
m
e
.
.
.
T
h
e
e
n
d
.
T
e
m
p
o
r
a
l
m
a
r
k
e
r
s
F
o
r
e
x
a
m
p
l
e
,
w
h
e
n
,
n
e
x
t
,
t
h
e
n
,
i
m
m
e
d
i
a
t
e
l
y
,
i
n
s
t
a
n
t
l
y
,
a
f
t
e
r
,
a
g
a
i
n
,
a
l
r
e
a
d
y
,
a
l
w
a
y
s
,
b
e
f
o
r
e
,
l
a
t
e
l
y
,
n
o
w
,
o
n
c
e
,
p
r
e
s
e
n
t
l
y
,
r
a
r
e
l
y
,
t
o
d
a
y
,
w
e
e
k
l
y
,
w
h
i
l
e
N
o
t
e
m
p
o
r
a
l
m
a
r
k
e
r
s
O
n
e
t
e
m
p
o
r
a
l
m
a
r
k
e
r
E
x
a
m
p
l
e
s
a
.
T
h
e
g
i
r
l
w
a
l
k
e
d
o
v
e
r
t
o
t
h
e
a
l
i
e
n
s
.

T
h
e
n

t
h
e
y
a
l
l
a
t
e
s
o
m
e
l
u
n
c
h
.
b
.

A
f
t
e
r

t
h
e
a
l
i
e
n
s
l
a
n
d
e
d
,
t
h
e
g
i
r
l
s
c
r
e
a
m
e
d
.
T
w
o
o
r
m
o
r
e
t
e
m
p
o
r
a
l
m
a
r
k
e
r
s
E
x
a
m
p
l
e

W
h
e
n

t
h
e
g
i
r
l
s
a
w
t
h
e
a
l
i
e
n
s
,
s
h
e
r
a
n
o
u
t
t
o
m
e
e
t
t
h
e
m
.
S
h
e

a
l
r
e
a
d
y

k
n
e
w
t
h
e
y
w
o
u
l
d
b
e
n
i
c
e
.
C
a
u
s
a
l
a
d
v
e
r
b
i
a
l
c
l
a
u
s
e
s
F
o
r
e
x
a
m
p
l
e
,
b
e
c
a
u
s
e
,
s
i
n
c
e
,
s
o
t
h
a
t
,
t
h
e
r
e
f
o
r
e
,
a
s
a
r
e
s
u
l
t
,
c
o
n
s
e
q
u
e
n
t
l
y
,
t
h
u
s
,
h
e
n
c
e
.
N
o
c
a
u
s
a
l
a
d
v
e
r
b
i
a
l
c
l
a
u
s
e
s
.
O
n
e
c
a
u
s
a
l
a
d
v
e
r
b
i
a
l
c
l
a
u
s
e
E
x
a
m
p
l
e
T
h
e
a
l
i
e
n
s
w
e
r
e
n
o
t
n
i
c
e
t
o
t
h
e
g
i
r
l
b
e
c
a
u
s
e
t
h
e
y
w
e
r
e
s
c
a
r
e
d
.
T
w
o
o
r
m
o
r
e
c
a
u
s
a
l
a
d
v
e
r
b
i
a
l
c
l
a
u
s
e
s
E
x
a
m
p
l
e
T
h
e
a
l
i
e
n
s
w
e
r
e
n
o
t
n
i
c
e
t
o
t
h
e
g
i
r
l
b
e
c
a
u
s
e
t
h
e
y
w
e
r
e
s
c
a
r
e
d
.
S
i
n
c
e
t
h
e
y
w
e
r
e
m
e
a
n
,
s
h
e
r
a
n
a
w
a
y
.
K
n
o
w
l
e
d
g
e
o
f
d
i
a
l
o
g
u
e
K
n
o
w
l
e
d
g
e
o
f
d
i
a
l
o
g
u
e
i
s
r
e
g
i
s
t
e
r
e
d
b
y
a
c
o
m
m
e
n
t
o
r
s
t
a
t
e
m
e
n
t
m
a
d
e
b
y
a
c
h
a
r
a
c
t
e
r
o
r
b
y
c
h
a
r
a
c
t
e
r
s
e
n
g
a
g
i
n
g
i
n
c
o
n
v
e
r
s
a
t
i
o
n
.
N
o
d
i
a
l
o
g
u
e
O
n
e
c
h
a
r
a
c
t
e
r
m
a
k
e
s
a
c
o
m
m
e
n
t
o
r
s
t
a
t
e
m
e
n
t
E
x
a
m
p
l
e
s
a
.
H
e
s
a
i
d

O
w

b
.
H
e
s
a
i
d

D
o
n

t
c
o
m
e
o
v
e
r
h
e
r
e

T
w
o
o
r
m
o
r
e
c
h
a
r
a
c
t
e
r
s
e
n
g
a
g
e
i
n
c
o
n
v
e
r
s
a
t
i
o
n
E
x
a
m
p
l
e
H
e
s
a
i
d

O
h
l
o
o
k
,
t
h
e
r
e
i
s
a
n
a
l
i
e
n

a
n
d
s
h
e
s
a
i
d

O
h
,
l
e
t
s
g
o
s
e
e
t
h
e
m
.

N
a
r
r
a
t
o
r
e
v
a
l
u
a
t
i
o
n
s
N
a
r
r
a
t
o
r
e
v
a
l
u
a
t
i
o
n
s
a
r
e
a
n
y
e
x
p
l
a
n
a
t
i
o
n
p
r
o
v
i
d
e
d
i
n
t
h
e
s
t
o
r
y
t
o
j
u
s
t
i
f
y
w
h
y
a
n
a
c
t
i
o
n
o
r
e
v
e
n
t
t
o
o
k
p
l
a
c
e
.
N
o
n
a
r
r
a
t
o
r
e
v
a
l
u
a
t
i
o
n
s
O
n
e
n
a
r
r
a
t
o
r
e
v
a
l
u
a
t
i
o
n
E
x
a
m
p
l
e
S
h
e
r
a
n
u
p
t
o
s
a
y
h
e
l
l
o
t
o
t
h
e
a
l
i
e
n
.
S
h
e
a
l
w
a
y
s
w
a
n
t
e
d
t
o
m
e
e
t
o
n
e
.
T
w
o
o
r
m
o
r
e
n
a
r
r
a
t
o
r
e
v
a
l
u
a
t
i
o
n
s
E
x
a
m
p
l
e
s
a
.
S
h
e
k
n
e
w
t
h
a
t
i
t
w
a
s
a
n
a
l
i
e
n
s
p
a
c
e
s
h
i
p
.
E
v
e
r
y
o
n
e
k
n
o
w
s
a
b
o
u
t
U
F
O
s
.
b
.
H
e
w
a
n
t
e
d
t
o
r
u
n
f
r
o
m
t
h
e
a
l
i
e
n
s
.
T
h
e
y
w
e
r
e
h
i
s
w
o
r
s
t
n
i
g
h
t
m
a
r
e
.
126 TOPICS IN LANGUAGE DISORDERS/APRILJUNE 2008
Table 3. An example of INC scoring
Pretest 1: Alien Story INC Scoring
There is a boy (character). Character 1
There is a girl (character). Setting 0
They are playing (action). Initiating event 0
There are some funny people (character). Internal response 0
There is a dog too (character). Plan 0
Action/attempt 1
Complication 0
Consequence 0
Narrator evaluation 0
Formulaic markers 0
Temporal markers 0
Causal adverbial clauses 0
Knowledge of dialogue 0
Total INC Score 2
Posttest: Alien Story INC Scoring
The boy saw the aliens (character). Character 1
The aliens landed (IE) in Central Park (setting) and said Setting 2
Wow, would you look at this? (knowledge of dialogue). Initiating event 2
The girl was scared (IR), but she thought it might be fun to go over Internal response 2
and meet the aliens (plan). Plan 1
She ran to meet the aliens (action) before she got too scared to do Action/attempt 2
it (temporal marker). Complication 1
The boy stopped her (complication). Consequence 0
She was sad (IR) because she had always wanted to meet some Narrator evaluation 0
aliens (causal adverbial clause). Formulaic markers 0
Temporal markers 1
Causal adverbial clauses 1
Knowledge of dialogue 1
Total INC Score 14
book and to think about the story that went
with the pictures. Once the children had
looked at all of the pictures, the examiners
turned to the first page of the book and asked
the children to generate a story while looking
at the pictures. Stories were transcribed from
digital audio recordings according to SALT
transcript conventions and were analyzed by
three independent raters.
The narrative intervention program
After pretest 2, children received therapy
in groups of three or four, in 90-min sessions
four times per week for 4 weeks. Graphic or-
ganizers were used to teach character, setting,
initiating event, character response, plan, ac-
tions, consequence, complication, and res-
olution. In addition to story grammar in-
struction, causality, temporal concepts, and
dialogue were targeted throughout each ther-
apy session. Children practiced retelling sto-
ries that corresponded to wordless picture
books created by the authors. The children
also retold and created narratives on the basis
of single pictures of scenes taken from popu-
lar magazines.
Results and Discussion
Interscorer agreement is one important
index of the reliability of any progress
Emerging Procedures in Narrative Assessment 127
monitoring measure. Three raters were
trained in the use of the INC coding system
by the second author, SG, using transcripts
from children who did not participate in
the group summer intervention program.
The raters scored stories and discussed their
scoring decisions together until they were
90% reliable for scoring each individual story
element. The pretest and posttest transcripts
were equally divided and randomly assigned
to each rater (60 stories for each rater).
Interrater reliability was based on a random
sample of 20 stories (4 samples of each of the
5 elicitation contexts). Average point-by-point
agreement was 90% for One Frog Too Many;
96% for A boy, a Dog and a Frog; 94% for
Two Friends; 96% for the Late for School
story, and 92% for the Alienstory.
Content sampling reliability was assessed
by computing intercorrelations between INC
Table 4. Means, standard deviations, and Cohens d effect sizes for the pretreatment phase
(pretest 1 to pretest 2) and treatment phase (pretest 2 to posttest) of the study
Standard Standard Pooled Cohens
Deviation Deviation Standard d Effect
Measure Comparison Mean 1 Mean 1 Mean 2 Mean 2 Deviation Size
TNL NLI Pre 1 Pre 2 82.75 12.34 81.75 11.03 11.69 0.09
Pre 2 Post 81.75 11.03 97.25 12.63 11.83 1.31
TNL NC Pre 1 Pre 2 7.58 2.81 7.33 2.54 2.67 0.09
Pre 2 Post 7.33 2.54 9.5 2.84 2.69 0.81
TNL ON Pre 1 Pre 2 6.75 1.82 6.58 2.11 1.96 0.09
Pre 2 Post 6.58 2.11 9.58 1.72 1.91 1.57
Late for school Pre 1 Pre 2 6.92 3.23 4.92 2.85 3.04 0.66
Pre 2 Post 4.92 2.85 9.92 3.48 3.17 1.58
Alien story Pre 1 Pre 2 6.83 3.83 6.08 6.23 5.03 0.15
Pre 2 Post 6.08 6.23 11.00 3.44 4.83 1.02
Two friends Pre 1 Pre 2 6.92 3.68 5.75 2.45 3.07 0.38
Pre 2 Post 5.75 2.45 9.83 3.61 3.03 1.34
A Boy, a Dog Pre 1 Pre 2 10.33 4.27 10.33 3.97 4.13 0.00
And a Frog Pre 2 Post 10.33 3.99 12.17 3.51 3.75 0.49
One Frog too Pre 1 Pre 2 9.25 4.03 9.58 2.64 3.33 0.10
Many Pre 2 Post 9.58 2.64 13.08 1.93 2.29 1.53
TNL: Test of Narrative Language
NLI: Narrative language index composite score; NC: Narrative comprehension composite score; ON: Oral narration
composite score.
Pre 1: Pretest 1
Pre 2: Pretest 2
Post.: Posttest.
scores for each of the five story elicitation con-
texts at pretest 1 and pretest 2. Nineteen of
the 20 correlations were significant at the .05
level. The correlations ranged between r =
.604 and r = .898 (unadjusted). These results
indicate that the INC scoring system yielded
similar scores across static picture, sequenced
picture, and wordless picture book sampling
contexts.
The INC scoring procedure was specifically
designed to measure change in narrative struc-
tures that would complement and correlate
with performance on the TNL. The progress
monitoring function of the INC was assessed
by comparing the means, standard deviations,
and Cohens d effect sizes for the pretreat-
ment phase (pretest 1 to pretest 2) and treat-
ment phase (pretest 2 to posttest) for all sto-
ries. The relevant values are shown in Table 4.
A repeated-measures analysis of variance was
128 TOPICS IN LANGUAGE DISORDERS/APRILJUNE 2008
used to determine whether childrens narra-
tive performance significantly changed over
the three assessment periods. For each sam-
pling context, the within subjects variable
was the three testing points (pretest 1, pretest
2, and posttest). The analysis of variance
for the Two Friends stories yielded a sig-
nificant main effect for time, F(2, 22) =
6.93, p < .01;
2
= .387. Pairwise compar-
isons indicated that there were no significant
differences between pretest 1 and pretest 2.
However, the posttest was significantly higher
than both pretest 1 and pretest 2 (the treat-
ment phase of the study). These results indi-
cate that childrens narration did not change
appreciably during the month that they were
not enrolled in school and were not receiving
intervention. Following 1 month of interven-
tion, INC scores changed significantly. This
same pattern of results was obtained for each
of the INC measures and for the TNL.
To evaluate the clinical significance of this
change, Cohens d effect sizes were calcu-
lated for the pretreatment and posttreatment
phases of the study (Table 4). Cohens d
values indicate the extent in change from
one time to another in standard deviation
units. A full explanation of this procedure
can be found in Dollaghan (2007). As is
shown in Table 4, the INC scores did not
change appreciably from pretest 1 to pretest
2, when they were not receiving therapy.
Cohens d values of 0.09 to 0.10 demon-
strate 90% or greater overlap in the distribu-
tions of the scores at pretest 1 and pretest
2. The INC scoring system was sensitive to
changes in narrative ability after treatment.
Cohens d calculations for differences be-
tween pretest 2 and posttest (the treatment
phase of the study) yielded moderate to large
effect sizes, with d values ranging from 0.49
to 1.6 across the five sampling conditions.
These results validate the use of the INC scor-
ing system as a reliable progress monitoring
measure when the same procedures and the
same or equivalent stories are used to elicit
each narrative. In this study, the INC scores
reflected no improvements in narration dur-
ing the month that children did not receive
intervention. The INC scores reflected sig-
nificant improvements in narrative ability af-
ter only 1 month of intervention. Taken to-
gether, these findings suggest that the INC
is highly sensitive to changes in narrative
production.
Finally, to assess criterion-prediction valid-
ity, the extent to which the INC scores pre-
dicted performance on the TNL was explored.
The INC scores for each of the five sampling
conditions were highly correlated at each
testing period (pretest 1, pretest 2, posttest)
with the total raw score on the TNL Oral
Narration scale. The Pearson correlations be-
tween the INC scores and the TNL ranged be-
tween 0.602 and 0.828. These high correla-
tions demonstrate that the INCscoring system
yielded values across the five story conditions
that were consistently related to the TNL.
Precautions
The validity of the INC as a means for mon-
itoring progress is dependent upon the use of
materials and standard elicitation procedures
that produce equivalent stories from one time
to another. Likewise, the context under which
these stories are elicited must be similar. For
example, if a clinician asks a child to tell a
story about a wordless picture book before in-
tervention, the clinician needs to use a simi-
lar wordless picture book at posttest. Differ-
ent yet equivalent books or story elicitation
procedures would be ideal for progress mon-
itoring purposes. Currently such stimuli and
procedures are being developed.
CONCLUSIONS
A new criterion-referenced narrative scor-
ing system called the INC was designed to
evaluate changes in a variety of narrative skills.
A preliminary study of the use of this scor-
ing system yielded evidence suggesting that
the INC could be scored consistently, yielded
similar scores across three sampling contexts,
was sensitive to change after intervention, and
assessed narrative constructs that were similar
to those assessed by the TNL. These results
Emerging Procedures in Narrative Assessment 129
support the use of the INC as a clinical tool
for evaluating the outcomes of narrative in-
tervention given that important guidelines are
followed.
REFERENCES
Anastasi, A., & Urbina, S. (1997). Psychological testing
(7th ed.). Upper Saddle River, NJ: Prentice Hall.
Bain, S. K. (2001). Review of the Renfrew bus story. In B.
S. Plake & J. C. Impara (Eds.), The fourteenth mental
measurements yearbook (pp. 621624). Lincoln, NE:
Buros Institute of Mental Measurements.
Balason, D. V., & Dollaghan, C. A. (2002). Grammatical
morpheme production in 4-year-old children. Journal
of Speech, Language, and Hearing Research, 45(5),
961969.
Baxter, A. (2006). Review of the test of narrative lan-
guage. In K. F. Geisinger, R. A., Spies, J. F Carlson., &B.
S. Plake (Eds.), The sixteenth mental measurements
yearbook (pp. 383384). Lincoln, NE: Buros Institute
of Mental Measurements.
Bennett-Kastor, T. (1988). Analyzing childrens lan-
guage: Methods and theories. New York: Blackwell.
Berman, R. A., & Slobin, D. I. (1994). Relating events
in narrative: A crosslinguistic developmental study.
Hillsdale, NJ: Erlbaum.
Bishop, D. V. M., & Adams, C. (1992). Comprehen-
sion problems in children with specific language im-
pairment: Literal and inferential meaning. Journal of
Speech and Hearing Research, 35(1), 119129.
Bishop, D. V. M., & Edmundson, A. (1987). Language-
impaired 4-year-olds: Distinguishing transient from
persistent impairment. Journal of Speech and Hear-
ing Disorders, 52(2), 156173.
Bond, L. A. (1996). Norm- and criterion-referenced
testing. Practical Assessment, Research & Evalu-
ation, 5(2). Retrieved February 28, 2008, from
http://PAREonline.net/getvn.asp?v=5&n=2.
Boting, N. (2002). Narrative as a tool for the assessment
of linguistic and pragmatic impairments. Child Lan-
guage Teaching & Therapy, 18(1), 122.
Boudreau, D. M., & Chapman, R. S. (2000). The relation-
ship between event representation and linguistic skill
in narratives of children and adolescents with Down
syndrome. Journal of Speech, Language, and Hear-
ing Research, 43(5), 11461159.
Brown, R. (1973). A first language: The early stages. Lon-
don: George Allen & Unwin.
Chapman, S. B., Watkins, R., Gustafson, C., Moore, S.,
Levin, H. S., &Kufera, J. A. (1997). Narrative discourse
in children with closed head injury, children with
language impairment, and typically developing chil-
dren. American Journal of Speech-Language Pathol-
ogy, 6(2), 6676.
Cowley, J., & Glasgow, C. (1994). The Renfrew bus story.
Centreville, DE: The Centreville School.
Deno, S. L. (2003). Developments in curriculum-based
measurement. Journal of Special Education, 37, 184
192.
Dollaghan, C. A. (2007). The handbook for evidence-
based practice in communication disorders. Balti-
more, MD: Paul H. Brookes.
Evans, J. L., & Craig, H. (1992). Language sampling col-
lection and analysis: Interview compared to freeplay
assessment contexts. Journal of Speech and Hearing
Research, 35, 343353.
Fey, M. E., Catts, H. W., Proctor-Williams, K., Tomblin, J.
B., & Zhang, X. (2004). Oral and written story com-
position skills of children with language impairment.
Journal of Speech, Language, and Hearing Research,
47(6), 13011318.
Fuchs, L. S., Deno, S., & Mirkin, P. (1984). Effects of
frequent curriculum-based measurement and evalua-
tion on pedagogy, student achievement, and student
awareness of learning. American Educational Re-
search Journal, 21, 449460.
Gillam, R. B., &Johnston, J. R. (1992). Spoken and written
language relationships in language/learning-impaired
and normally achieving school-age children. Jour-
nal of Speech and Hearing Research, 35(6), 1303
1315.
Gillam, R. B., & Pearson, N. (2004). Test of narrative lan-
guage. Austin, TX: PRO-ED.
Gillam, R. B., Loeb, D. F., Hoffman, L. M., Bohman, T.,
Champlin, C., Thibodean, L., et al. (2008). The effi-
cacy of fast forward language intervention in school-
age children with language impairment: A randomized
controlled trial. Journal of Speech, Language, and
Hearing Research, 51, 97119.
Greenhalgh, K. S., & Strong, C. J. (2001). Literate lan-
guage features in spoken narratives of children with
typical language and children with language impair-
ments. Language, Speech, and Hearing Services in
Schools, 32(2), 114125.
Guthrie, L., &Hall, W. (1983). Continuity/discontinuity in
the function and use of language. Review of Research
in Education, 10, 5577.
Hughes, D. L., McGillvray, L., & Schmidek, M. (1997).
Guide to narrative language: Procedures for assess-
ment. Eau Claire, WI: Thinking Publications.
John, S. F., Lui, M., & Tannock, R. (2003). Childrens story
retelling and comprehension using a new narrative
resource. Canadian Journal of School Psychology,
18(12), 91113.
Justice, L. M., Bowles, R., Eisenberg, S. L., Kaderavek,
J. N., Ukrainetz, T. A., & Gillam, R. B. (2006). The
index of narrative micro-structure (INMIS): A clini-
cal tool for analyzing school-aged childrens narrative
130 TOPICS IN LANGUAGE DISORDERS/APRILJUNE 2008
performance. American Journal of Speech-Language
Pathology, 15, 177191.
Labov, W. (1972). Sociolinguistic patterns. Philadelphia:
University of Pennsylvania Press.
Lester, H. (1986). A porcupine named Fluffy. New York:
Houghton Mifflin.
Lester, H. (1987). Pookins gets her way. New York:
Houghton Mifflin.
Linn, R. L., & Gronlund, N. E. (2000). Measurement and
assessment in teaching. Upper Saddle River, NJ: Pren-
tice Hall.
Mayer, M. (1967). A boy, a dog and a frog. New York:
Penguin.
Mayer, M. (1969). Frog, where are you? New York:
Penguin.
Mayer, M. (1974). Frog goes to dinner. New York:
Penguin.
Mayer, M., & Mayer, M. (1975). One frog too many.
New York: Penguin.
McCabe, A., & Bliss, L. S. (2003). Patterns of narra-
tive discourse: A multicultural, life span approach.
Boston: Allyn & Bacon.
McCauley, R. J., & Swisher, L. (1984). Use and misuse of
norm-referenced tests in clinical assessment: A hypo-
thetical case. Journal of Speech and Hearing Disor-
ders, 49(4), 338348.
Mehta, P. D., Foorman, B. R., Branum-Martin, L., & Taylor,
P. W. (2005). Literacy as a unidimensional multilevel
construct: Validation, sources of influence, and impli-
cations in a longitudinal study in grades 1 to 4. Scien-
tific Studies of Reading, 9(2), 85116.
Miller, J. F., & Chapman, R. S. (2004). Systematic analysis
of language transcripts (version 8.0) [Computer soft-
ware]. Madison: Language Analysis Laboratory, Wais-
man Center, University of Wisconsin-Madison.
Miller, L. S. (2000). Bird and his ring. Austin, TX: Neon
Rose.
Mu

noz, M. L., Gillam, R. B., Pe

na, E. D., & Gulley-Faehnle,

A. (2003). Measures of language development in fic-
tional narratives of Latino children. Language, Speech,
and Hearing Services in Schools, 34(4), 332342.
Nation, N., Clarke, P., & Marshall, C. M. (2004). Hidden
language impairments in children: Parallels between
poor reading comprehension and specific language
impairment? Journal of Speech, Language, and Hear-
ing Research, 47, 199211.
Newman, R. M., & McGregor, K. K. (2006). Teachers
and laypersons discern quality differences between
narratives produced by children with or without SLI.
Journal of Speech, Language, and Hearing Research,
49(5), 10221036.
Pankratz, M. E., Plante, E., Vance, R., & Insalaco, D. M.
(2007). The diagnostic and predictive validity of the
renfrew bus story. Language, Speech, and Hearing
Services in Schools, 38(4), 390399.
Pe

na, E. D., Gillam, R. B., Malek, M., Ruiz-Felter, R.,

Resendiz, M., Fiestas, C., et al. (2006). Dynamic as-
sessment of school-age childrens narrative ability: An
experimental investigation of classification accuracy.
Journal of Speech, Language, and Hearing Research,
49, 10371057.
Peterson, C., & McCabe, A. (1983). Developmental psy-
cholinguistics: Three ways of looking at a childs nar-
rative. New York: Plenum.
Renfrew, C. E. (1969). The bus story: A test of continuous
speech. North Place, Old Headington: Oxford.
Scott, C. M., & Windsor, J. (2000). General language per-
formance measures in spoken and written narrative
and expository discourse of school-age children with
language learning disabilities. Journal of Speech, Lan-
guage, and Hearing Research, 43(2), 324340.
Snyder, L. S., & Downey, D. M. (1991). The language-
reading relationship in normal and reading-disabled
children. Journal of Speech and Hearing Research,
34, 129140.
Spaulding, T. J., Plante, E., & Farinella, K. A. (2006). Eligi-
bility criteria for language impairment: Is the low end
of normal always appropriate? Language, Speech, and
Hearing Services in Schools, 37(1), 6172.
Stein, N. L., & Glenn, C. G. (1978). An analysis of story
comprehension in elementary school children. In R.
O. Freedle (Ed.), New directions in discourse process-
ing (pp. 53120). Norwood, NJ: Ablex.
Stein, N. L., & Glenn, C. G. (1982). Childrens concept of
time: The development of a story schema. In W. Fried-
man (Ed.), The developmental psychology of time
(pp. 255282). New York: Academic Press.
Stothard, S. E., Snowling, M. J., Bishop, D. V. M.,
Chipchase, B. B., & Kaplan, C. A. (1998). Language-
impaired preschoolers: A follow-up into adolescence.
Journal of Speech, Language, and Hearing Research,
41(2), 407418.
Strong, C. J. (1998). The strong narrative assessment pro-
cedure. Eau Claire, WI: Thinking Publications.
Ukrainetz, T. A. (Ed.) (2006). Contextualized language
intervention: Scaffolding pre-K-12 literacy achieve-
ment. Eau Claire, WI: Thinking Publications.
Ukrainetz, T. A., Justice, L. M., Kaderavek, J. N., Eisenberg,
S. L., Gillam, R. B., &Harm, H. M. (2005). The develop-
ment of expressive elaboration in fictional narratives.
Journal of Speech, Language, and Hearing Research,
48, 13631377.
Zhang, X., &Tomblin, J. B. (2003). Explaining and control-
ling regression to the mean in longitudinal research
designs. Journal of Speech, Language, and Hearing
Research, 46(6), 13401351.