PUBLIC HEALTH BRIEFS
should be investigated more fully; however, the absence of
association between THM concentrations and incidence of
other cancers and the additional problem of lack of associa-
tion between THM concentrations and pancreatic cancer in
any other sex- or race-specific group raises doubts as to the
validity of this finding. Since ecological studies are theoreti-
cally biased toward Type I error, there is the distinct possi-
bility that this finding is a spurious product of confounding
by unmeasured variables.
Concurrent with the Type I bias, is a lower probability
of missing a true association. Given a sound methodology,
this no-effect phenomenon could be due to the modest range
of THM values (71 ppb) or to factors actually inhibiting the
disease process; an actual association might be masked by a
competing risk, (e.g., the population has a high incidence of
another disease which diminishes the susceptibility pool for
these cancers, or a competing exposure, or the environmen-
tal "noise" in Erie County is too great to allow the surfacing
of a modest THM effect).
One must also add that information gathered from dis-
ease registries is subject to biases such as differences in path-
ological diagnosis, and incomplete reporting or non-
compliance, and that the site-specific age-adjusted incidence
Evaluation of Utilization of Laboratory Tests
In a Hospital Emergency Room
ZACHARY BLOOMGARDEN, MD, AND VICTOR W. SIDEL, MD
Abstract: Visits to a metropolitan teaching hospital
emergency room were evaluated for laboratory test utiliza-
tion and quality of care by chart review. Of 630 tests, 197
(32 per cent), clustered in 44 out of 476 (9 per cent) patient
visits, were considered unnecessary. The number of tests per
visit showed a strong negative correlation with the necessity
of tests and with the quality of care. Results suggest the use-
fulness of identifying a subgroup of patients with excessive
tests and implementing measures to alter testing behavior for
this subgroup. (Am J Public Health 70:525-528, 1980.)
From the Departments of Medicine and Social Medicine, Mon-
tefiore Hospital and Medical Center, Albert Einstein College of
Medicine. Address reprint requests to Zachary Bloomgarden, MD,
Department of Social Medicine, Montefiore Hospital and Medical
Center, 111 East 210th Street, Bronx, NY 10467. This paper, sub-
mitted to the Journal April 4, 1979, was revised and accepted for
publication December 26, 1979
AJPH May 1980, Vol. 70, No. 5
rates by census tract are subject to substantial variability
since many rates are based on few cases and the occurrence
of one or two tumor cases in a particular census tract greatly
influences the incidence rate. Furthermore, use of adjusted
rates glosses over associations that may be present in specif-
ic categories. Measurements of the independent variables
were taken after the appearance of the diseases and, al-
though chlorination practices in Erie County have not
changed appreciably in four decades, this theoretically does
not account for the long latency period usually associated
with cancer. Patterns of population migration out of Erie
County have not been quantified and differential migration
by water quality could be a source of bias due to incomplete
cohort followup.
On the other hand, this research does not present meth-
odology inferior to previous research reporting positive as-
sociations, and in some ways it may be considered more re-
fined.
REFERENCE
1. Wilkins JR, Reiches, NA and Kruse CW: Organic chemical con-
taminants in drinking water and cancer. Am J Epidemiology.
110:420-448, 1979.
The hospital emergency room (ER) has become an in-
creasingly important source of ambulatory care for the com-
munity, often for patients with self-limited and non-urgent
problems or for the "worried well."' 4 There appears also to
be a marked increase in the number of laboratory tests ob-
tained in the ER, as in other areas of medical practice.56
The phenomenon of excessive laboratory testing, well docu-
mented in hospitalized patients,7-"l has received less atten-
tion in the ER setting. In this communication we report an
analysis of utilization of laboratory tests in a metropolitan
hospital ER.
Methods
We reviewed the charts of every fifth adult visit over a
three-week period to the ER of a 700-bed teaching hospital.
Without reference to the results of the tests, a panel of six
reviewers (three senior resident physicians and three attend-
ing physicians in the Department of Medicine) evaluated
each test performed with respect to: 1) necessity of the test
525
PUBLIC HEALTH BRIEFS

TABLE 1-Relationship of Necessity of Tests Performed to Quality of Care

Classification of Tests
Assessment of
Quality of Care Necessary* Relevant" Not Indicated Total

Optimal 339 (77) 44 (10) 57 (13) 440 (100)

Less than optimal 77 (48) 27 (17) 56 (35) 160 (100)
Inadequate 17(57) 1 ( 3) 12(40) 30(100)
TOTAL 433 (68) 72 (12) 125 (20) 630 (100)
Note: Percentages shown in parentheses.
*X2 (for deviation from null hypothesis that per cent necessary is independent of quality of care) = 15.2; d.f. = 2;
p < .01)
"Includes tests classified as "medico-legally required"

for clinical decision-making during the ER visit; 2) relevance 100-

of the test to the patient's clinical condition; and 3) medical- (82)
legal requirement for the test.* The reviewers also classified (40)
90-
the overall care on each visit as: 1) optimal; 2) less than opti-
mal but adequate; or 3) inadequate.** La
tn
to
a) 80- (20)
L)
z (21)
Results
70-
I-
A total of 476 visits were sampled. After excluding visits
during which no laboratory tests were ordered and visits 0

whose laboratory evaluation consisted only of roentgeno- c

grams of bones with suspected fracture, 218 patient-visits
with 630 laboratory tests or roentgenograms remained. Of
these 630 tests, 433 (68 per cent) were classified as "neces-
sary" for clinical decision-making in the ER, an additional 68
(11 per cent) as "relevant", and 4 (1 per cent) as "medico-
legally required," leaving 125 tests (20 per cent) classified as
"not indicated" (Table 1).
The correlation between the number of tests done per
patient-visit and the evaluation of necessity of the tests is
shown in Figure 1. For the 82 visits in which only one labora-
tory test was performed, fully 94 per cent of these tests were
considered necessary. In visits with two to four tests, 79 per
cent were classified as necessary, while only 55 per cent of
the tests in patients with five or more tests per visit were so
classified. The trend for decreasing percentage of necessary
*Each test was coded separately for the attending and house-
staff reviewers as being:
(1) Necessary: classified as necessary for clinical decision-
making during the ER visit by two or more of the three reviewers;
(2) Relevant: not coded as "necessary" but classified as rele-
vant or necessary by two or more reviewers;
(3) Medico-legally required: not coded as "necessary" or
"relevant" but classified as necessary, relevant, or midicol-legally
required by two or more reviewers;
(4) Not indicated: not fulfilling any of the above criteria.
The combined coding for the attending and housestaff reviewers
was based on the same principles, with equal weight given to each
reviewer.
**The overall evaluation of patient care was coded as "in-
adequate" if two or more of the six reviewers classified it as such,
regardless of the classification assigned by other reviewers; care was
coded as "less than optimal" if three or more reviewers classified it
as either "less than optimal" or "inadequate"; the remaining visits
were coded as "optimal".
tL 60- (22)
a. (18) (15)
50-
2
E3
3 4 5
5 6m
6
e
->7
Number of Tests/Patient Visit
FIGURE 1-Relationship of Necessity of Tests to Total Number of
Tests Performed during the Visit. The Number of Patient Visits in
Each Category is Shown in Parentheses.
tests with increasing number of tests given is statistically
highly significant. It is noteworthy that the total of 197 tests
not considered necessary were obtained in only 44 patient
visits.
The necessity for individual types of laboratory tests is
examined in Table 2; 12 types of tests were done with suf-
ficient frequency for evaluation; for nine types, more than
three-fourths were classified as necessary. The SMA-6***
and SMA-12t and prothrombin time, however, were eval-
uated as necessary in less than one-half of the visits in which
they were done, a finding partly explained by the practice of
ordering an "admission survey" (CBC, SMA-6, SMA-12,
and prothrombin time) done customarily for all patients hos-
pitalized on the Medical Service at the hospital in which the
***The SMA-6 is an automated analysis of serum sodium, po-
tassium, chloride, bicarbonate, urea nitrogen, and glucose.
tThe SMA-12 is an automated analysis of serum calcium, phos-
phate, uric acid, albumin, total protein, lactate dehydrogenase,
creatinine phosphokinase, glutamate-oxalate transaminase, alkaline
phosphatase, creatinine, and total and direct bilirubin.

526 AJPH May 1980, Vol. 70, No. 5

 PUBLIC HEALTH BRIEFS

TABLE 2-Evaluation of Necessity of Individual Laboratory Tests

No. of times Per Cent Per Cent Per Cent Per Cent
Test Test Done Necessary Relevant* Not Indicated Total

CBC 122 80 10 10 100

SMA-6 80 55 19 26 100
SMA-12 56 17 36 47 100
Prothrombin Time 55 17 14 69 100
Urine Analysis 49 88 3 9 100
Amylase 20 80 3 17 100
Arterial Blood Gas 17 79 6 15 100
Electrocardiogram 91 96 2 2 100
Chest Roentgenogram 57 91 4 5 100
Abdominal Roentgenogram 23 82 11 7 100
Other Roentgenogram 26 87 2 11 100
Culture 28 73 14 13 100
Other Tests 15 60 17 23 100

X2 (for deviation from null hypothesis that per cent of tests necessary is independent of type of test) = 69.3; d.f. =
12; p < 0.001
*Includes tests "medico-legally required"

study was performed. Fifty such four-test "'surveys" were
ordered, but in only two visits were all the survey tests con-
sidered necessary by both housestaff and attending review-
ers. Twenty of the 50 surveys of four tests each were consid-
ered by the reviewers to have included three or more unnec-
essary tests. In all, 81 (41 per cent) of these 200 tests
performed as surveys were considered "not necessary"
compared with the 116 (23 per cent) unnecessary tests
among the group of 430 tests not performed as surveys (X2 =
5.76; p < .02).
Of the 218 patient visits evaluated, the care was classi-
fied as "inadequate" in nine (4 per cent), and "less than opti-
mal" in 33 (15 per cent). Visits in which care was felt to be
"'inadequate" or "less than optimal" had, respectively,
means of 3.3 and 4.9 tests per visit, while those with "ade-
quate" care had 2.5 tests per visit. As shown in Table 1, 77
per cent of tests done on patients with "optimal" care were
considered necessary, compared to 48 per cent and 57 per
cent of tests done on patients with "less than optimal" or
"inadequate" over-all care.
No clear differences emerged between the evaluation
patterns of housestaff and attending reviewers, with similar
judgments of necessity and relevance of the entire group of
laboratory tests. The attending physicians' evaluations are
compared with the housestaff members' evaluations on all of
the 630 tests in the study in Table 3. Both groups agreed on
their assessment of "necessity" in 531 instances (84 per
cent). Of the 99 tests (16 per cent) in which there was dis-
agreement, housestaff members were almost twice as likely
as attending physicians to assess a test as being necessary.
Discussion
Of 630 laboratory tests evaluated by the reviewers, 32
per cent were considered not necessary for clinical decision-
X2 for deviation from the null hypothesis that per cent neces-
sary is independent of the number of tests done = 69.1; d.f. = 6;
p < 0.0001.
making, and 20 per cent were considered not indicated on
any grounds; 42 per cent of tests considered not necessary
were ordered as parts of screening surveys. These were clus-
tered in 44 patient visits, 20 per cent of the 218 visits eval-
uated and only 9 per cent of the total of 476 sampled visits.
Unnecessary tests were correlated with the total number of
tests per visit, and with the reviewers' judgment of quality of
care delivered. These correlations may be confounded by
the reviewers including the total number of tests as an ele-
ment in judging necessity, despite instructions not to do so,
or their view of the necessity of testing as an element in judg-
ing the overall quality of care.
This study did not evaluate illness outcome or patient-
physician interaction; it was designed to use reviewer judg-
ment as a normative standard. Such "'implicit" standards are
frequently used in evaluating diagnostic processes, although
they have not been shown to be directly relevant to improve-
ment of outcome.'2-'5 Since outcome may be remote or
unrelated to treatment, "implicit" criteria of the type em-
ployed here'6 can be used to produce alterations in the
existing patterns of establishing diagnoses, specifically test-
ordering behavior. Interventions to alter physician test-
ordering might include requiring explicit presentation of the
indications for the test and regular review of physicians'
testing patterns.
It is of interest that an assessment of care of adult wom-
en with symptoms of urinary tract infection in the same ER
showed a significant correlation of physician performance
TABLE 3-Comparison of Attending Physician and House Staff
Evaluation of Tests Performed
Attending Physician Evaluations
House Staff
Evaluations "Necessary" "Not Necessary" Total
"Necessary" 389 60 449
"Not Necessary" 39 142 181
TOTAL 428 202 630

AJPH May 1980, Vol. 70, No. 5 527

PUBLIC HEALTH BRIEFS
with outcome.'9 It would appear useful, based on the results
of this study, to isolate the relatively small number of pa-
tient-visits which led to inappropriate use of laboratory tests,
and, by making a determined effort to alter testing behavior
in these visits in a controlled experiment, determine the rela-
tionship between change in this aspect of the diagnostic pro-
cess and change in the outcome of care.
REFERENCES
1. Weinerman ER, Ratner RS, Robbins R, et al: Yale studies in
ambulatory medical care: V. Determinants of use of hospital
emergency service. Am J Public Health 56:1037-1056, 1966.
2. Garfield S: The delivery of medical care. Scientific American
222:15-23, 1970.
3. Gibsen G: Status of urban services. Hospitals JAHA 45:49-54,
1971.
4. Jonas S, Flesh R, Brook R, et al: Maintaining utilization of a
municipal hospital emergency department. Hospital Topics
54:43-48, 1976.
5. Carter PM, Davison AJ, Wickens I, et al: The horns of the path-
ological dilemma. Hosp and Health Serv Rev, pp 346-350, Oc-
tober 1975.
6. Scitovsky AA and McCall N: Changes in the Costs of Treatment
of Selected Illnesses, 1951-1964-1971. U.S. DHEW, U.S. Pub-
lic Health Service, DHEW Publication No. (HRA) 77-3161,
1977.
7. Griner PF and Liptziu B: Use of the laboratory in a teaching
hospital: Implications for patient care, education, and hospital
costs. Ann Internal Med 75:157-163, 1971.
8. Griner PF: Treatment of acute pulmonary edema: Conventional
or intensive care? Ann Internal Med 77:501-506, 1972.
9. Griner PF: Medical intensive care in the teaching hospital:
Costs versus benefits: The need for an assessment. Ann Internal
Med 78:581-585, 1973.
The Achievement of Continuity of Care In a
Primary Care Training Program
JOHN ROGERS, MD, AND PETER CURTIS, MRCP, MRCGP, DOBST
Although the importance of continuity in primary care is
intuitively accepted by many authorities in medical educa-
tion and health services, its value in terms of outcomes is not
well proven. In spite of the relative lack of evidence, resi-
dency training programs in primary care are committed by
accreditation boards to the implementation of a continuous
clinical experience.'
Address reprint requests to Dr. Peter Curtis, Assistant Profes-
sor, Department of Family Medicine, School of Medicine, Universi-
ty of North Carolina, Chapel Hill, NC 27514. Dr. Rogers is a Fellow,
Department of Family Medicine, University of Washington School
of Medicine. This paper, submitted to the Journal August 1, 1979,
was revised and accepted for publication December 26, 1979.
528
10. Edwards LD, Levin S, Balagtus R, et al: Ordering patterns and
utilization of bacteriologic culture reports. Ann Internal Med
132:678-682, 1973.
11. Dixon RH and Lazzio J: Utilization of clinical chemistry serv-
ices by medical house staff: An analysis. Ann Internal Med
134:1064-1067, 1974.
12. Sidel VW: Evaluation of the quality of medical practice. JAMA
198:763-764, 1966.
13. Lewis CE: The state of the art of quality assessment-1973.
Medical Care 12:799-806, 1974.
14. Sidel, VW: Quality for whom? Effects of professional responsi-
bility for quality of health care on equity. Bull NY Acad Med
52:164-176, 1976.
15. Eyes B and Evans AF: Post-traumatic skull radiographs: Timn
for a reappraisal. Lancet 2:85-86, 1978.
16. Donabedian A: The quality of medical care. Science 200:856-
864, 1978.
17. Donabedian A: Promoting quality through evaluating the pro-
cess of patient care. Medical Care 6:181-202, 1968.
18. Brook RH and Appel FA: Quality-of-care assessment: Choosing
a method for peer review. N Engl J Med 288:1323-1329, 1973.
19. Rubenstein L, Mates S and Sidel VW: Quality of care assess-
ment by process and outcome scoring: Use of weighted algorith-
mic assessment criteria for evaluation of emergency room care
of women with symptoms of urinary tract infection. Ann Inter-
nal Med 86:617-625, 1977.
ACKNOWLEDGMENTS
The authors are grateful to: Drs. David Hamerman, David
Kindig, and Isidore Levine for their advice on and support of this
study; Dr. Herbert Levine for help with the statistical evaluation;
Drs. Harold Keltz, Stephen Brenner, Philip Lief, Michael Seidman,
Mary Ann Chase, and James Bradof for acting as reviewers; Drs.
Lewis Goldfrank, Leo Koss, Susan Mates, and Thomas Ben-
eventano, and Daniel Drosness and Mo Katz for reviewing the
manuscript; and Jean Nardelli, Eve Teitelbaum, and Edythe Weber
for their patient and meticulous work on the manuscript.
The extent to which physicians in primary care training
programs achieve continuity with their patients is not well
established.
This study, undertaken at the Family Practice Center at
the University of North Carolina, was designed to measure
the degree of continuity of care achieved by resident and
faculty physicians over a one year period (July 1976 to June
1977).
The Family Practice Center served an enrolled popu-
lation of 5,020 patients (1,980 families) over the 12 months.
Each physician was responsible for a specific "practice" of
assigned patients or families on a continuing basis. In addi-
tion, the physicians worked within three defined medical
teams.
AJPH May 1980, Vol. 70, No. 5