ARTICLE

Screening for Developmental Delay in the Setting of

a Community Pediatric Clinic: A Prospective
Assessment of Parent-Report Questionnaires
David Rydz, MSca, Myriam Srour, MD, CMa,b, Maryam Oskoui, MD, CMa,b, Nancy Marget, MScc, Mitchell Shiller, MD, CMb,
Rena Birnbaum, MSc, OTd, Annette Majnemer, PhD, OTa,d, Michael I. Shevell, MD, CMa,b

Departments of aNeurology/Neurosurgery, bPediatrics, and cEducational and Counseling Psychology and dSchool of Physical and Occupational Therapy, McGill
University, Montreal, Quebec, Canada

The authors have indicated they have no financial relationships relevant to this article to disclose.

ABSTRACT
OBJECTIVES. Our goal for this study was to prospectively test whether parent-com-
pleted questionnaires can be effectively used in the setting of a busy ambulatory
www.pediatrics.org/cgi/doi/10.1542/
pediatric clinic to accurately screen for developmental impairments. Specific ob- peds.2006-0466
jectives included (1) assessing the feasibility of using parent-report instruments in doi:10.1542/peds.2006-0466
the setting of a community pediatric clinic, (2) evaluating the accuracy of 2
Key Words
available screening tests (the Ages and Stages Questionnaire and Child Develop- developmental screening, parent-report,
ment Inventory), and (3) ascertaining if the pediatrician’s clinical judgment could questionnaires, infant

be used as a potential modifier. Abbreviations

AAP—American Academy of Pediatrics
METHODS. Subjects were recruited from the patient population of a community clinic CCC—Children’s Care Clinic
BDI—Battelle Developmental Inventory
providing primary ambulatory pediatric care. Subjects without previous develop- ASQ—Ages and Stages Questionnaire
mental delay or concerns noted were contacted at the time of their routine CDI—Child Development Inventory
18-month-old visit. Those subjects who agreed to participate were randomly Accepted for publication May 23, 2006

assigned to 1 of 2 groups and completed either the Ages and Stages Questionnaire Address correspondence to Michael I. Shevell,
MD, CM, Montreal Children’s Hospital, Room
or Child Development Inventory. The child’s pediatrician also completed a brief A-514, 2300 Tupper, Montreal, Quebec,
questionnaire regarding his or her opinion of the child’s development. Those Canada H3H 6P3. E-mail: michael.shevell@
muhc.mcgill.ca
children for whom concerns were identified by either questionnaire underwent PEDIATRICS (ISSN Numbers: Print, 0031-4005;
additional detailed testing by the Battelle Development Inventory, the “gold Online, 1098-4275). Copyright © 2006 by the
American Academy of Pediatrics
standard” for the purposes of this study. An equal number of children scoring
within the norms of the screening measures also underwent testing with the
Battelle Development Inventory.

RESULTS. Of the 356 parents contacted, 317 parents (90%) agreed to participate.
Most parents correctly completed the Ages and Stages Questionnaire (81%) and
the Child Development Inventory (75%). Predictive values were calculated for the
Ages and Stages Questionnaire and the Child Development Inventory (sensitivity:
0.67 and 0.50; specificity: 0.39 and 0.86; positive predictive value: 34% and 50%;
negative predictive value: 71% and 86%, respectively). Incorporating the physi-

e1178 RYDZ et al
Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015
cian’s opinion regarding the developmental status of the
child did not improve the accuracy of the screening
questionnaires.
CONCLUSIONS. Three important conclusions were reached:
(1) parent-completed questionnaires can be feasibly
used in the setting of a pediatric clinic; (2) the pediatri-
cian’s opinion had little effect in ameliorating the accu-
racy of either questionnaire; and (3) single-point accu-
racy of these screening instruments in a community
setting did not meet the requisite standard for develop-
ment screening tests as set by current recommendations.
This study raises important questions about how devel-
opmental screening can be performed, and we recom-
mend additional research to elucidate a successful
screening procedure.
A N ESTIMATED 12% to 16% of the US pediatric pop-
ulation has a developmental disability.1 There has
been increasing pressure to identify these children at an
earlier age, with the current focus being on infants.2 Part
of the reason is that research has demonstrated not only
that intervention programs are cost-effective and have
lifelong benefits but also that developmental attainment
is maximized when intervention is commenced earlier.
Consequently, professional societies such as the Ameri-
can Academy of Pediatrics (AAP) advocate the identifi-
cation of developmentally delayed children before 2
years of age.2
The most effective manner to identify children with
developmental delay still remains elusive, although the
need to find a practical screening method has been in-
dicated in the literature as early as 1979. Shonkoff et al
concluded that “more precise techniques for pediatric
developmental assessment and conclusive evaluations of
specific interventions will have to be produced. . . . The
current difficulty in defining criteria for optimal pediatric
management emphasizes the need for creative, method-
ologically sophisticated research in the area.”3 More re-
cently, Sices et al4 stated, “The AAP . . . does not provide
specific guidance on how a primary care physician is to
perform developmental surveillance and screening. Re-
search on how these guidelines can be best implemented
in the context of primary care practice would help stan-
dardize and enhance the value of the experience for
patients and families.”4 These authors call for the formu-
lation of guidelines, which can be used by primary care
practitioners in a time- and cost-efficient manner.
Currently, most primary care practitioners incorpo-
rate developmental surveillance that relies heavily on a
pediatrician’s intuition and clinical judgment. Although
such a method might be accurate given enough time,
training, supplementary resources, and expertise, stud-
ies have demonstrated that clinical judgment alone does
not accurately detect developmental delay. For example,
Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015
a study from 1987 demonstrated that only 20% to 30%
of developmentally impaired children were being iden-
tified before school age by primary care practitioners
using developmental surveillance.5 For this reason, pro-
fessional organizations advocate the use of standardized
developmental screening tests.2 Developmental screen-
ing has been shown to improve the accuracy with which
children’s developmental delays are identified when
compared with decisions based only on clinical judg-
ment.6–8 Indeed, the sensitivity and specificity of screen-
ing instruments are usually reported between 70% and
90%.9–12 Although the use of screening tests would im-
prove the rate and accuracy of identification, a recent
survey demonstrated that only 23% of primary care
clinicians used a standardized screening tool, leading the
authors to conclude: “Our findings do raise. . .the con-
cerns that systems of care that foster the proper use of
adequate detection methods in the primary care setting
continue to be elusive.”13 A previous survey documented
some of the obstacles faced by primary care practitioners,
with time constraints being voiced by 82% of them.14
The lack of medical staff (48%) and the burden of cost
(44%) in monitoring developmental delay, for which
they are not well compensated, were also listed as sig-
nificant preventive factors.14
As a substitute to these time-intensive, pediatrician-
administered screening tests, a role for standardized
parent-completed questionnaires can be considered.
Parent-completed questionnaires are as accurate as de-
velopmental screening instruments, because current re-
search strongly supports the observation that parents,
regardless of differences in socioeconomic status, geo-
graphical location, or parental well-being, can give ac-
curate information about their child’s development.15–21
Moreover, parent-completed screening tools are cost-
effective in the short-term, can be completed over the
telephone, in the waiting room, or by mail, and are
time-efficient.22 Realizing the advantages of parent-com-
pleted questionnaires, Regalado and Halfon wrote: “The
available evidence suggests that assessment of develop-
mental issues might benefit from the wider use of struc-
tured, validated approaches. . . . [S]ystemic assessment
of parent’s concerns can play a role in identifying chil-
dren with developmental problems, replacing or supple-
menting longer and more costly developmental screening
assessments.”23 Consequently, the use of parent-com-
pleted questionnaires might be more practical in the
busy pediatric clinic.
The literature concerning the topic of developmental
screening with parent-completed questionnaires has
several shortfalls that limit the widespread applicability
of the results. Previous studies have focused on specific
cohorts that had already been flagged as being delayed
or consisted of high-risk infants (very low birth weight
and prematurity).24–26 Sample sizes in the majority of
studies, which rely on specific cohorts, are small, and the
PEDIATRICS Volume 118, Number 4, October 2006 e1179
applicability of these results to the general pediatric pop-
ulation as a whole is uncertain.24–26 Studies, using other
screening instruments as the gold standard, must also be
analyzed skeptically, because validating a screening in-
strument with another parent-completed screening in-
strument can skew results.25 These studies do not offer
guidelines that primary care practitioners can use to
screen for developmental delays.7,24–26 Part of the reason
might be that these tests are not designed with an em-
phasis on keeping the methods in line with the actual
reality of the clinical setting. The majority of research
has focused on validation or predictive value and not on
actual methods or practicality. Indeed, Sonnander states
that “few . . . studies focusing on evaluation of develop-
mental screening programs conducted within a clinical
setting were found,” whereas “empirical research into
child development and the predictive value of develop-
mental tests is extensive.”27 Thus, there is a lack of direct
pragmatic research addressing the area of how a primary
care practitioner should screen for developmental delay
in a community.
To create an efficient system for recognition of devel-
opmental problems, we defined 3 objectives for our
study: (1) to test whether questionnaires can be feasibly
completed by the caregiver in the waiting room of a busy
pediatric clinic; (2) to examine the accuracy of these
parent-completed questionnaires when used in such a
setting; and (3) to assess what role the practitioner plays
in ameliorating the questionnaire’s accuracy. These as-
pects of parent-completed screening tests need to be
examined systematically to provide a rational founda-
tion to the formulation of a feasible, clear, and accurate
screening protocol.
METHODS
Subjects
Subjects were recruited from the patient population of
the Children’s Care Clinic (CCC). Located in Pierrefonds,
Quebec, Canada, the CCC is an exclusively pediatrics
group practice, incorporating 7 full-time pediatricians. It
is community based, drawing from a suburban, largely
middle class population and providing comprehensive
primary pediatric care to its clientele. Its patient demo-
graphic and appointment systems are fully computer-
ized.
Subjects were recruited at the time of their 18-
month-old visit. This is a standard routine visit that
coincides with the administration of a number of vac-
cines. The 18-month timing was also chosen because it
allows for assessment of motor, language, social, and
cognitive skills by standardized developmental screening
instruments. Furthermore, successful identification of
delays at this point in time would represent a substantial
improvement over what is currently achieved (ie, refer-
ral to pediatric subspecialists and rehabilitation services
e1180 RYDZ et al
Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015
typically occurs at ⱖ2–3 years).18 All subjects turning 18
months were contacted and eligible to participate unless
there was an established significant developmental dis-
ability. Recruitment was terminated after 101 assess-
ments were completed with the Battelle Development
Inventory (BDI) because of funding constraints. Also,
parents completing the questionnaire needed to have a
fourth-grade reading level and understanding of English,
because these questionnaires are presently available in
English only.
Procedures
The database at the CCC was scanned continually for
subjects who were turning 18 months of age in the
forthcoming 2 months. The identified caregivers were
sent a cover letter, signed by the pediatricians at the CCC
indicating their support, that described the study and
included study contact information. A telephone call
from a research assistant was made soon after receipt of
the letter, during which any questions or concerns re-
garding the study were addressed and verbal consent or
refusal was noted. If a parent refused participation, the
reason for refusal was obtained and noted. All parents
who refused were also asked to complete a simple de-
mographic questionnaire over the telephone that gath-
ered basic demographic information on employment,
income, and education. If verbal consent was given, the
subject’s appointment date was noted, and a dossier was
placed in the child’s folder at the clinic. Each dossier
contained:
1. One of 2 selected parental-report measures (the Ages
and Stages Questionnaire [ASQ] or Child Develop-
ment Inventory [CDI]) depending on group assign-
ment, which was performed using random-number
tables and blocking to ensure equal distribution.
2. A sheet with the question “Do you have any present
developmental concerns regarding your child (yes/
no)?” If answering “yes,” the caregiver chose from a
checklist to specify if this concern related to motor
(gross and/or fine), language, social, or cognitive do-
mains or to ⬎1 domain; the caregiver was provided
space to provide any additional comments.
3. A simple demographic questionnaire requesting in-
formation on employment, education and income.
4. A consent form for participation.
5. A Likert-type questionnaire regarding the ease of use
of the questionnaire they had completed.
Completed questionnaires were scored by a research
assistant (D.R.) according to established procedures for
each measure. The number of incorrectly completed
questionnaires for each measure was noted. Those chil-
dren who failed the parent-completed questionnaire
(⬍2 SDs on ⱖ1 domain) underwent BDI testing (the
gold standard for the purpose of this study), which was
administered approximately 3 months after the initial
screen by an experienced pediatric occupational thera-
pist who was blinded to both the screening instruments
used and the results obtained. The 3-month interval
used in this study represents the 3-month referral wait
typically experienced by families awaiting a more thor-
ough developmental assessment, and at 21 months of
age, the child falls in the middle of the 18- to 23-month
marking scheme of the BDI, giving the most accurate
estimation of his or her real abilities according to the
parameters of this measure.
To investigate false-negative estimates, an equal
number of children scoring within the norms of the
screening questionnaires were selected as controls and
also underwent BDI testing. Controls were the next child
of the same gender participating in the study who had
scored normally on the parent-completed screening in-
strument. A false-negative estimate for this approach
was also obtained in a likewise manner. In this fashion,
the control group was selected in an unbiased manner.
Because of constraints posed by funding and time avail-
able from the occupational therapist, only 101 children
underwent BDI testing, after which ongoing study re-
cruitment was terminated. Although this is not as favor-
able as testing all the participants with the BDI, 101
completed BDIs represents a large enough sample to be
a good indicator of the accuracy of the ASQ and CDI.
The pediatrician who provided care to the child was
blinded to the actual questionnaire used and the results
obtained but was asked as part of the study to answer a
question identical to the one given to the parent regard-
ing any possible concerns about the child’s development.
This question was answered at the time of the subject’s
visit by simply checking the appropriate response
box(es). The results of this simple questionnaire were
used in the analysis to establish what role the pediatri-
cian could play in influencing the accuracy of the ASQ
and CDI.
Parents who had agreed over the telephone to partic-
ipate but did not return the questionnaire were con-
tacted by telephone and reminded to return the ques-
tionnaire. If the questionnaire had not been returned 3
months after it was due, the parents were sent a demo-
graphic questionnaire to assess whether any parameters
differentiated these parents from those who actually
completed the questionnaire. Families with whom we
had no personal contact by telephone were not included
in the study.
Throughout this study, data obtained were stored and
managed on secure computers within the Montreal
Children’s Hospital Division of Pediatric Neurology with
separation of subject identifier variables. Ethical ap-
proval for conduct of the study was obtained from the
Montreal Children’s Hospital–McGill University Health
Centre Research Ethics Board. Data storage and analysis
Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015
were performed by using SPSS 12.0 (SPSS Inc, Chicago,
IL), and all data entered were verified for errors.
Measures
BDI
The BDI,28 a psychometrically sound developmental as-
sessment for children from birth to 8 years, is widely
used in studies evaluating developmental delay and in
US-mandated early intervention programs. The BDI
gathers information, via a structured test format, to in-
clude interviews with the caregiver and direct observa-
tion of the child. Items are assigned an age level at which
75% of the norming population was able to perform. It
is composed of 341 items that evaluate 5 different do-
mains: personal/social, adaptive, motor, communica-
tion, and cognitive, with each domain further subdi-
vided into subdomains. Items are scored as typically
signifying fully developed skills, sometimes signifying
emerging abilities, or rarely signifying absent skills and
are scored at values of 2, 1, or 0, respectively. The
authors of the test recommend the test for many func-
tions including general screening, detailed assessment of
children who have previously been identified as having
developmental delay, or identifying strengths and weak-
nesses of normal or developmentally impaired children.
The manual states that the assessment may take up to 2
hours to administer with children ⬍3 years of age.
ASQ
The ASQs29 are 19 parent-report questionnaires that
span the age range of 4 to 60 months. Questionnaire
points include 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 27,
30, 33, 36, 42, 48, 54, and 60 months of age. Each
questionnaire is composed of 3 sections: a brief set of
demographic items; 30 questions about the infant’s or
child’s development assessing 5 different domains
equally (communication, gross motor, fine motor, prob-
lem-solving, and personal/social); and 7 open-ended
questions eliciting parental concerns. The choice of re-
sponses for each item is “yes,” “sometimes,” or “not yet,”
which are scored as 10, 5, or 0, respectively. The test is
graded according to the domain tested and compared
with an empirically derived screening cutoff score and
takes ⬃10 to 15 minutes to complete and ⬃5 minutes to
score.
Although the ASQ states that the questionnaire can
be used in-clinic, the ASQ is meant for mail-out pur-
poses to be completed at home at the specific age inter-
vals listed above. Because of this, the instructions ask the
parents to attempt every activity with their child and, if
the child is noncompliant, to try at a later time. Unfor-
tunately, this is not feasible in the context of a busy
waiting room. Thus, these instructions were abridged,
and parents were asked to base their responses on their
observations and previous experiences with their child.
PEDIATRICS Volume 118, Number 4, October 2006 e1181
The ASQ manual does state that completion of the ques-
tionnaire in the waiting room is the least accurate
method, because the parent has a limited time to com-
plete the activities, and the appointment might not fall at
an age at which the best results are achieved.
CDI
The CDI30 is useful for testing children whose ages range
from birth to 6 years. It consists of 300 items: 270 items
composed of yes/no statements about the child’s devel-
opment and 30 items that evaluate various potential
sensory, physical, motor, language, and behavioral prob-
lems. These items are grouped into 8 subscales: social,
self-help, gross motor, fine motor, expressive language,
comprehension, letters, and numbers. Each scale is
scored by tallying the “yes” answers; a child who re-
ceives a score that is 1.5 SDs below the mean is graded as
borderline, whereas a child scoring ⬎2 SDs below the
mean is graded as delayed. The parent needs ⬃10 to 15
minutes to complete this screening instrument and ⬃5
minutes to score it.
The CDI was reformatted for the purposes of this
study. In its original form, the CDI is answered on a
separate marking sheet such that the questions are
found in one booklet and the answers are written in
another. To make the process easier for parents, the CDI
was retyped and presented in a format such that the
parent can immediately read the question and then an-
swer the item in the same booklet.
RESULTS
Descriptive Data
Over the recruitment period we received the names of
532 possible subjects from the database at the CCC.
From this list, 84 children were excluded because (1) the
children were no longer being followed at the clinic (60
children), (2) the children had moved (12 children), (3)
the parents did not speak or read English (8 children),
(4) the children had an established developmental delay
(2 children), (5) the study terminated before the 18-
month appointment had been booked (1 child), or (6)
the child had passed away (1 child). Of the 448 remain-
ing children, we were unable to contact by telephone 92
of them; therefore, they also were not included in the
study. We successfully contacted the remaining 356 chil-
dren, and of these, 317 families (90%) agreed to partic-
ipate over the telephone and 39 families (10%) refused.
Finally, of the 317 who had agreed over the telephone to
participate at the forthcoming clinic visit, 5 returned the
questionnaire incomplete and 64 did not return the
questionnaire at all despite regular subsequent prompt-
ing as described. Those who had originally agreed to
participate over the telephone but never returned the
questionnaire were labeled “false participants.” Thus,
the total number that completed the questionnaire was
248. Table 1 shows the demographic data of these 3
groups: true participants, false participants, and refusals.
The average age of the children who participated in
the study was 18.4 ⫾ 0.64 months. The data for gesta-
tional ages and gender of the children are summarized in
Table 2.
Performance on Screening Tools
Of the 134 ASQs that were returned completed, 53
(40%) children failed at least one domain. The domain
that the infant failed most often on the questionnaire
was communication (46), followed by the problem solv-
ing, (7) gross motor, (4) social, (4) and fine motor (3)

TABLE 1 Demographic Data for 3 Populations

Characteristics True Participants, False Participants, Refusal,
n (%) n (%) n (%)
Responding parent
Mother 223 (90) NA NA
Father 19 (8) NA NA
Other 6 (2) NA NA
Education, last year of school completed, mother/father N ⫽ 243 N ⫽ 17 N ⫽ 23
High school incomplete 3 (1)/2(1) 2 (12)/0 (0) 1 (4)/1 (4)
High school 18 (7)/28 (12) 3 (18)/2 (12) 3 (13)/3 (13)
CEGEP/college 80 (33)/76 (32) 4 (24)/3 (18) 4 (17)/8 (35)
University 116 (47)/102 (42) 7 (41)/11 (65) 10 (44)/10 (44)
Graduate school 26 (11)/33 (14) 1 (6)/1 (6) 5 (22)/1 (4)
Working mother N ⫽ 248 N ⫽ 17 N ⫽ 23
Yes 159 (64) 10 (59) 13 (57)
Combined income for the household, $ N ⫽ 228 N ⫽ 12 N ⫽ 21
0–19 000 6 (3) 0 (0) 2 (10)
20 000–39 999 14 (7) 1 (8) 0 (0)
40 000–59 999 23 (10) 2 (17) 8 (37)
60 000–79 999 45 (19) 1 (8) 2 (10)
ⱖ80 000 140 (61) 8 (67) 9 (43)
The 3 populations were true participants, false participants (those who agreed to participate over the telephone but never returned the
questionnaire), and refusals. NA indicates not applicable; CEGEP, collège d’enseignement général et professionnel.

e1182 RYDZ et al
Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015
 TABLE 2 Descriptive Data for 3 Populations ASQ and the CDI being negligible. Table 3 summarizes
True False Refusal these data.
Participants Participants
Child gender, n (%) Psychometric Values
Male 132 (53) 34 (53) 22 (56)
Female 116 (47) 30 (47) 17 (44) Psychometric Values of the Screening Questionnaires
Age Using the BDI as the gold standard, we assessed the ASQ
Mean ⫾ SD, mo 18.4 ⫾ .64 NA NA and the CDI for their psychometric properties: sensitiv-
Range, mo 17.03–20.47 NA NA ity, specificity, positive predictive value, and negative
Gestational age, mean ⫾ SD, wk 38.6 ⫾ 2.0 39.0 ⫾ 2.7 38.5 ⫾ 2.5
predictive value. Not one of these questionnaires proved
Preterm (⬍36 wk gestation), n (%) 20 (8) 3 (5) 2 (6)
Term (⬎36 wk gestation), n (%) 218 (92) 56 (95) 33 (94) to be an ideal screening instrument. The ASQ had mod-
NA indicates not applicable.
erate sensitivity (0.67) but poor specificity (0.39). Con-
versely, the CDI had poor sensitivity (0.50) but excellent
specificity (0.86). Table 4 summarizes the predictive val-
ues for the ASQ, CDI, and pediatrician’s questionnaire.
domains. Of the 53 children who failed the screening
tool, 41 continued study participation and underwent
Psychometric Values of Incorporating the Pediatrician’s
BDI assessment (ie, 12 dropped out).
Opinion With the Parent-Completed Questionnaires (ASQ
Conversely, of the 114 children who were assessed by
and CDI)
the CDI, 11 failed (failure rate: 9%). Expressive language
The physician’s opinion was incorporated with the par-
was again the most likely domain to be failed (5), fol-
ent-completed questionnaires to determine if this could
lowed by the gross motor, (5) fine motor, (1) self-help
be used as a potential modifier. As a result, those chil-
(1), and language comprehension (1 failure) domains.
dren (1) who had failed the parent-completed question-
Of the 11 children who failed the CDI, 8 continued study
naire and (2) for whom concern was listed by the pedi-
participation and undertook BDI assessment (ie, 3
atrician were grouped into a new category as those who
dropped out).
had failed both screening instruments. Likewise, those
In total, 101 BDI assessments were performed. Twen-
(1) who had passed the parent-completed questionnaire
ty-nine children received a failing score. Similarly, the
and (2) for whom no concern was listed by the pedia-
domain that was failed most often was communication
(17), followed by the gross motor (11), cognition (6),
personal-social (3), and adaptive (1) domains. Of the
101 BDIs performed, 49 were children who had previ- TABLE 3 Assessment of the Questionnaires’ Feasibility
ously failed one of the questionnaires, whereas 52 in- ASQ
fants were control subjects. Of the 101 BDI tests per- N 183
No. of questionnaires distributed 165
formed, 41 failed ASQs and 8 failed CDIs were assessed
No. (%) completed 134 (81)
with the BDI, and the remaining 52 BDI assessments No. returned late 37
were performed on control subjects. No. (%) of questionnaires not returned 28 (17)
No. (%) of questionnaires incomplete 3 (2)
Ease of Questionnaire Completion Parent’s opinion of questionnaire (N ⫽ 1; mean: 1.5 ⫾ .6), n (%)
Very easy 75 (57)
Closer examination was given to the rates of completion
Easy 50 (38)
and the parents’ opinion of the questionnaires to assess Neutral 6 (5)
feasibility and ease of use. Overall, the CDI was more Difficult 1 (1)
likely to be either returned late (27% for CDI vs 22% for Very difficult 0
ASQ) or not returned at all (23% for CDI vs 17% for CDI
N 171
ASQ). This difference was to be expected, because the
No. of questionnaires distributed 152
CDI contains more items to answer and, thus, is more No. (%) completed 114 (75)
labor intensive. Overall, 81% of the ASQs were returned No. returned late 41
completed, and 75% of the CDIs were returned com- No. (%) of questionnaires not returned 36 (23)
pleted. No. (%) of questionnaires incomplete 2 (2)
Parent’s opinion of questionnaire (N ⫽ 112; mean: 1.6 ⫾ .7), n (%)
The Likert-type questionnaire was used to assess the
Very easy 54 (48)
parent’s opinion of the questionnaires. The options (very Easy 49 (44)
easy, easy, neutral, difficult, and very difficult) were Neutral 8 (7)
given the numerical values 1 through 5, respectively. Difficult 1 (1)
The ASQ received a mean value of 1.5 ⫾ 0.6, whereas Very difficult 0
the CDI received a mean value of 1.6 ⫾ 0.7. Therefore, Completion rate and the parental ranking of the ASQ and the CDI. (To collect data on the
parent’s opinion of the questionnaire, the question “Did you find this questionnaire easy to
the majority of the parents ranked the questionnaires as complete? Place a check mark in the box that corresponds to your answer” was asked. The
either very easy or easy, with the difference between the number and types of responses are listed.)

PEDIATRICS Volume 118, Number 4, October 2006 e1183

Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015
 TABLE 4 Psychometric Values (Sensitivity, Specificity, Positive (1) the feasibility of using parent-completed question-
Predictive Value, and Negative Predictive Value) for the naires in the waiting room; (2) the role of the pediatri-
ASQ and CDI cian’s impression in the screening process; and (3) the
BDI Total Sensitivity Specificity PV⫹ PV⫺ accuracy of 2 parent-completed screening measures (the
ASQ and CDI). Each of these issues can be addressed
Fail Pass
separately in the context of our results.
ASQ 0.67 0.39 0.34 0.71
Fail 14 27 41
Pass 7 17 24 Feasibility
Total 21 44 65 The participation rate was encouraging. The majority of
CDI 4 4 8 0.50 0.86 0.50 0.86 parents (ASQ: 95%; CDI: 92%) found the questionnaire
Fail either easy or very easy to complete. The completion
Pass 4 24 28
Total 8 28 36
rate was also high for both questionnaires. Observed
PV⫹ indicates positive predictive value; PV⫺, negative predictive value.
completion was lower for the CDI (75%) than the ASQ
(81%), which was to be expected because the CDI is
longer and more detailed.
trician were grouped into another category as those More than 75% of the contacted population was as-
scoring normally on both tests. Children who did not fit sessed with a standardized screening tool, representing
the criteria for either of these 2 groups (ie, they had an improved rate over what is currently being achieved
passed the questionnaire and failed the pediatrician’s with practitioner-administered tests. (Only 23% of pri-
questionnaire or vice versa) were not included in the mary care physicians are regularly using a standardized
calculation. The 2 groups (one representing children screening tool.)13 Most important to note is that these
who had passed both questionnaires and the other rep- questionnaires were administered in the clinic at the
resenting the children who had failed both question- time of the patient’s appointment and did not need the
naires) were compared with the results of the BDI to assistance of either office staff or the physician. Because
obtain predictive values. Incorporating the pediatrician’s these tests are also cost-effective, the 3 most frequently
opinion with the results of the questionnaire did not mentioned complaints associated with a screening test
improve the predictive accuracy of either questionnaire administered by health care professionals (lack of time,
substantially. Table 5 summarizes the results of compar- lack of staff, and cost) were overcome.
ing the combined result of the pediatrician’s opinion and It is also important to note that the overall completion
the screening tool with the BDI. rate may have been decreased by the additional burden
To be comprehensive, different analyses, such as placed on the parent from the 4 additional forms placed
moving the BDI cutoff to 1 SD below the mean, remov- in the dossier to be completed. Nonetheless, simple en-
ing failed communication domains on the ASQ as a deavors can be conceptualized to increase the comple-
potential confounder, and redefining a failing score for tion rate with active endorsement and encouragement
the ASQ as 2 SDs below mean on ⱖ2 domains, were by the primary care physician and clinic staff. Thus, it
attempted. None of these posthoc manipulations had a seems feasible for parents to complete developmental
beneficial effect on predictive values, and the results are screening questionnaires in the waiting room while
not presented herein. waiting to be seen by their child’s physician.

DISCUSSION Pediatrician’s Opinion as a Potential Modifier

To determine the usefulness of parent-completed ques- The pediatrician’s opinion did not improve the predictive
tionnaires in a practical setting, 3 issues were examined: values of the questionnaires used. The pediatrician’s
opinion had good specificity but poor sensitivity; thus,
TABLE 5 Psychometric Values With the Addition of the the use of their clinical judgment tended to underiden-
Pediatrician’s Opinion for the ASQ and CDI tify developmental impairments. Indeed, the pediatri-
BDI Total Sensitivity Specificity PV⫹ PV⫺ cian’s questionnaire and the CDI had very similar prop-
Fail Pass erties.
It is disconcerting that combining the results of a
Ped and ASQ 0.60 0.73 0.60 0.73
Fail 9 6 15 standardized screening tool with the pediatrician’s opin-
Pass 6 16 22 ion could not produce enhanced and ultimately accept-
Total 15 22 36 able sensitivity and specificity values. Perhaps this indi-
Ped and CDI 0.40 0.89 0.40 0.89 cates the lack of proper health care training in regards to
Fail 2 3 5
childhood development. Indeed, the literature has em-
Pass 3 23 26
Total 5 26 31 phasized that there is a gap between the knowledge and
Ped indicates pediatrician’s opinion; PV⫹, positive predictive value; PV⫺, negative predictive skills required in providing developmental services and
value. the limited training that many clinicians receive in this

e1184 RYDZ et al
Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015
area. This is a problem that seems to have persisted over
decades. In 1979 Shonkoff et al lamented that “four
fifths of physicians . . . viewed their formal training in
this area as inadequate. . . . [A]lmost 2/3 did not feel
that practical experience was an adequate substitute for
formal training in developmental assessment skills.”3
This reality still persists ⬎20 years later. A 2000 study
performed by the AAP found that 64% of primary care
physicians reported inadequate training in developmen-
tal assessment.31 As a result, an improvement in screen-
ing accuracy does not just necessitate more accurate
developmental screening tests but also a joint effort from
different facets of the professional, public, and research
community advocating for more-thorough training in
childhood development and impairments.
Accuracy of the ASQ and CDI
The psychometric properties of the ASQ and CDI were
surprising, because several studies for both the ASQ and
the CDI have advocated their accuracy in screening in-
fants for developmental delay. A factor that may have
reduced the accuracy of the ASQ and CDI in our study
was the time difference between screening the child and
performing the full assessment. Unlike other projects in
which the gold standard was applied immediately after
the screening instrument, our study imposed a 3-month
waiting period representing the “real-life” delay typically
experienced between screening and referral. Previous
studies have demonstrated that 3 months represents a
time interval within which an infant’s developmental
status may change. Darrah et al32 found that, when
assessing fine motor skills, gross motor skills, and com-
munication, the majority of children tested serially had
40 to 60 percentile fluctuations in performance. Changes
of this magnitude took place at least once within 5
testing points (ie, 9, 11, 13, 16, and 21 months of age).
Consequently, a wait of 3 months could result in a
different developmental estimate. Because of the insta-
bility and discontinuity of a child’s development, what
might have been identified as a transient weakness at 18
months could have resolved itself by 21 months. The
developmental domain of communication also seems to
be too variable at the 18-month age to provide a reliable
indicator between normality and abnormality. At this
age, the “normal” differences in the actual times that
communication milestones are reached by children are
too variable,2 which demonstrates the importance of the
time of actual assessment and the advantage of regular,
ongoing multiple screening efforts.
Overall, neither test provided acceptable screening
properties at 18 months of age; therefore, a screening
protocol suitable to the setting of a clinic cannot consist
solely of either instrument applied at one point in time.
However, our results demonstrate that the CDI identi-
fied children with normal development with high accu-
racy (ie, high negative predictive value). Future studies
Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015
can incorporate this particular strength within a screen-
ing procedure. Moreover, we found that a “one-shot”
screening protocol might not be sufficient to identify
developmental delay accurately. Indeed, the AAP31 and
other researchers have come to the same conclusion. For
example, Darrah et al32 concluded that screening should
involve multiple time points and multiple domains be-
fore referrals are made. A similar statement was issued
by the AAP Committee on Children With Disabilities.2
This study has several limitations. First, the ASQ was
used as an in-clinic questionnaire; although the manual
states that it can be used in this fashion, this screening
instrument was originally constructed as a take-home
questionnaire so that the parent could attempt to ad-
dress the items over several days with the child. By
limiting the time to ⬃15 minutes, with the parent rely-
ing on past experiences to answer the questions, the
accuracy of the instrument could have been somewhat
compromised. Second, only 36 subjects underwent both
the CDI and BDI. This limits the ability to evaluate the
accuracy of the CDI, in particular with respect to sensi-
tivity, because there were few subjects who actually
failed the CDI. Thus, a larger sample size may be needed
before drawing clearer conclusions regarding the CDI’s
performance as a screening tool. Finally, the community
in which we were conducting the study was predomi-
nantly middle class with a high school education and
some postsecondary attendance. Our ability to general-
ize the results to the general population, particularly to
disadvantaged populations, is limited.
This study does lay the foundation for future studies
on screening. First, and most important, there is the
possibility of assessing the benefits of ⱖ2 serial screening
tests at different points in time. Because the develop-
mental trajectory, especially at younger ages, is subject
to large variability, a second screening effort (either
completed in-clinic or sent by mail) has the potential to
improve the accuracy of the initial identification process.
To decrease the amount of time used to score the ques-
tionnaire, application of computer software could be
implemented (ie, questionnaire completed and scored
on a hand-held device). This would reinforce the prac-
ticality of these types of screening instruments by de-
creasing the effort required from office staff and person-
nel. Finally, a study conducted on a more representative
population incorporating a larger sample size would
clarify issues of applicability, feasibility, and yield. With
additional research, the knowledge gap between avail-
able screening tests and developing a practical protocol
for their application can be bridged. Such efforts would
have considerable clinical implications with respect to
systematically enhancing the capability for earlier iden-
tification of children with developmental delays, which
should expedite intervention and, theoretically, opti-
mize eventual outcome.
PEDIATRICS Volume 118, Number 4, October 2006 e1185
ACKNOWLEDGMENTS
Dr Shevell is grateful for the support of the Montreal
Children’s Hospital foundation during the writing of this
manuscript. This project was supported by a clinical
research project grant from the Montreal Children’s
Hospital Research Institute.
REFERENCES
1. Boyle CA, Decoufle P, Yeargin-Allsoop MY. Prevalence and
health impact of developmental disabilities. Pediatrics. 1994;93:
863– 865
2. American Academy of Pediatrics, Committee on Children With
Disabilities. Developmental surveillance and screening of in-
fants and young children. Pediatrics. 2001;108:192–196
3. Shonkoff JP, Dworkin PH, Leviton A, Levine MD. Primary
approaches to developmental disabilities. Pediatrics. 1979;64:
506 –514
4. Sices L, Feudtner C, McLaughlin J, Drotar D, Williams M. How
do primary care physicians identify young children with devel-
opmental delays? A national survey. J Dev Behav Pediatr. 2003;
24:409 – 417
5. Palfrey JS, Singer JD, Walker DK, Butler JA. Early identifica-
tion of children’s special needs: a study in five metropolitan
communities. J Pediatr. 1987;111:651– 659
6. Ireton H, Glascoe FP. Assessing children’s development using
parent’s reports: the Child Developmental Inventory. Clin Pe-
diatr (Phila). 1995;34:248 –255
7. Dulcan MK, Costello EJ, Costello AJ, Edelbrock C, Brent D,
Janiszewski S. The pediatrician as gatekeeper to mental health
care for children: do parents’ concerns open the gates? J Am
Acad Child Adolesc Psychiatry. 1990;29:453– 458
8. Dearlove J, Kearney D. How good is general practice develop-
mental screening? BMJ. 1990;300:1177–1180
9. Glascoe FP. Parents’ concerns about children’s development:
prescreening technique or screening test. Pediatrics. 1997;99:
522–528
10. Glascoe FP, Byrne KE, Ashford LG, Johnson KL, Chang B,
Strickland B. Accuracy of the Denver-II in developmental
screening. Pediatrics. 1992;89:1221–1225
11. Macias MM, Saylor CF, Greer MK, Charles JM, Bell N, Kati-
kaneni LD. Infant screening: the usefulness of the Bayley In-
fant Neurodevelopmental Screener and the Clinical Adaptive
Test/Clinical Linguistic Auditory Milestone Scale. J Dev Behav
Pediatr. 1998;19:155–161
12. Squires J, Bricker D, Potter L. Revision of a parent-completed
developmental screening tool: Ages and Stages Questionnaires.
J Pediatr Psychol. 1997;22:313–328
13. Sand N, Silverstein M, Galscoe FP, Gupta VB, Tonniges TP,
O’Connor KG. Pediatricians’ reported practices regarding de-
velopmental screening: do guidelines work? Do they help? Pe-
diatrics. 2005;116:174 –179
14. Academy of Pediatrics, Division of Health Policy Research.
Periodic survey #53: identification of children ⬍36 months at
risk for developmental problems and referral to early identifi-
e1186 RYDZ et al
Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015
cation programs. 2003. Available at: www.aap.org/research/
periodicsurvey/ps53exs.htm. Accessed July 14, 2004
15. Knobloch H, Stevens F, Malone A, Ellison P, Risemberg H. The
validity of parental reporting of infant development. Pediatrics.
1979;63:872– 878
16. Glascoe FP, Alteimer WA, MacLean WE. The importance of
parent’s concerns about their child’s development. Am J Dis
Child. 1989;143:955–958
17. Glascoe FP, MacLean WE, Stone WL. The importance of par-
ent’s concerns about their child’s behavior. Clin Pediatr (Phila).
1991;30:8 –14
18. Glascoe FP. It’s not what it seems: the relationship between
parent’s concerns and children with global delays. Clin Pediatr
(Phila). 1994;33:292–296
19. Glascoe FP, Dworkin PH. The role of parents in the detection of
developmental and behavioral problems. Pediatrics. 1995;95:
829 – 836
20. Glascoe FP, Foster M, Wolraich W. An economic analysis of
developmental detection methods. Pediatrics. 1997;99:830 – 837
21. Glascoe FP. Parents’ evaluation of developmental status: how
well do parents’ concerns identify children with behavioral and
emotional problems? Clin Pediatr (Phila). 2003;42:133–138
22. Dobrez D, Sasso AL, Holl J, Shalowitz M, Leon S, Budetti P.
Estimating the cost of developmental and behavioral screening
of preschool children in general pediatric practice. Pediatrics.
2001;108:913–922
23. Regalado M, Halfon N. Primary care services promoting opti-
mal child development from birth to age 3 years. Arch Pediatr
Adolesc Med. 2001;155:1311–1322
24. Skellern CY, Rogers Y, O’Callaghan MJ. A parent-completed
developmental questionnaire: follow up of ex-premature in-
fants. J Paediatr Child Health. 2001;37:125–129
25. Montgomery ML, Saylor CF, Bell NL, Macias MM, Charles JM,
Katikaneni LD. Use of the Child Development Inventory to
screen high-risk populations. Clin Pediatr (Phila). 1999;38:
535–539
26. Doig KB, Macias MM, Saylor CF, Craver JR, Ingram PE. The
Child Development Inventory: a developmental outcome mea-
sure for follow up of the high-risk infant. J Pediatr. 1999;135:
358 –362
27. Sonnander K. Early identification of children with develop-
mental disabilities. Acta Paediatr Suppl. 2000;89:17–23
28. Newborg J, Stock JR, Wnek L, Guidubaldi J, Svinicki J. Battelle
Developmental Inventory: Examiner’s Manual. Allen, TX: DLM
LINC Associates; 1984
29. Squires J, Potter L, Mounts L. ASQ User’s Guide for the Ages and
Stages Questionnaire. 2nd ed. Baltimore, MD: Brookes; 1997
30. Ireton H. Child Development Inventories Manual. Minneapolis,
MN: Behavior Science Systems; 1992
31. Centers for Disease Control and Prevention. Barriers to devel-
opmental screening according to pediatricians: results from
AAP surveys of pediatricians. Available at: www.cdc.gov/
ncbddd/child/DSbarriersrpt.pdf. Accessed August 11, 2006
32. Darrah J, Hodge M, Magill-Evans J, Kembhavi G. Stability of
serial assessments of motor and communication abilities in
typically developing infants: implications for screening. Early
Hum Dev. 2003;72:97–110
 Screening for Developmental Delay in the Setting of a Community Pediatric
Clinic: A Prospective Assessment of Parent-Report Questionnaires
David Rydz, Myriam Srour, Maryam Oskoui, Nancy Marget, Mitchell Shiller, Rena
Birnbaum, Annette Majnemer and Michael I. Shevell
Pediatrics 2006;118;e1178
DOI: 10.1542/peds.2006-0466
Updated Information & including high resolution figures, can be found at:
Services http://pediatrics.aappublications.org/content/118/4/e1178.full.
html
References This article cites 27 articles, 17 of which can be accessed free
at:
http://pediatrics.aappublications.org/content/118/4/e1178.full.
html#ref-list-1
Citations This article has been cited by 12 HighWire-hosted articles:
http://pediatrics.aappublications.org/content/118/4/e1178.full.
html#related-urls
Permissions & Licensing Information about reproducing this article in parts (figures,
tables) or in its entirety can be found online at:
http://pediatrics.aappublications.org/site/misc/Permissions.xht
ml
Reprints Information about ordering reprints can be found online:
http://pediatrics.aappublications.org/site/misc/reprints.xhtml

PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly

publication, it has been published continuously since 1948. PEDIATRICS is owned, published,
and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk
Grove Village, Illinois, 60007. Copyright © 2006 by the American Academy of Pediatrics. All
rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.

Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015

 Screening for Developmental Delay in the Setting of a Community Pediatric
Clinic: A Prospective Assessment of Parent-Report Questionnaires
David Rydz, Myriam Srour, Maryam Oskoui, Nancy Marget, Mitchell Shiller, Rena
Birnbaum, Annette Majnemer and Michael I. Shevell
Pediatrics 2006;118;e1178
DOI: 10.1542/peds.2006-0466

The online version of this article, along with updated information and services, is
located on the World Wide Web at:
http://pediatrics.aappublications.org/content/118/4/e1178.full.html

PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly

publication, it has been published continuously since 1948. PEDIATRICS is owned,
published, and trademarked by the American Academy of Pediatrics, 141 Northwest Point
Boulevard, Elk Grove Village, Illinois, 60007. Copyright © 2006 by the American Academy
of Pediatrics. All rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.

Downloaded from pediatrics.aappublications.org at Nanyang Technological University on June 2, 2015