Toxicology 313 (2013) 134–144

Contents lists available at ScienceDirect

Toxicology
journal homepage: www.elsevier.com/locate/toxicol

A four-step approach to evaluate mixtures for consistency with dose addition夽

Richard C. Hertzberg a,∗ , Yi Pan b,1 , Ruosha Li b,2 , Lynne T. Haber c , Robert H. Lyles b , David W. Herr d ,
Virginia C. Moser d , Jane Ellen Simmons d
a
Biomathematics Consulting, 1932 Grist Stone Court NE, Atlanta, GA 30307, USA
b
Dept. of Biostatistics and Bioinformatics, Emory University, 1518 Clifton Road, Atlanta, GA 30322, USA
c
Toxicology Excellence for Risk Assessment (TERA), 2300 Montana Avenue, Suite 409, Cincinnati, OH 45211, USA
d
National Health and Environmental Effects Research Laboratory, US EPA, TW Alexander Drive, Research Triangle Park, NC 27709, USA

a r t i c l e i n f o a b s t r a c t

Article history: Mixture risk assessment is often hampered by the lack of dose–response information on the mixture
Received 12 June 2012 being assessed, forcing reliance on component formulas such as dose addition. We present a four-step
Received in revised form 7 October 2012 approach for evaluating chemical mixture data for consistency with dose addition for use in suppor-
Accepted 8 October 2012
ting a component based mixture risk assessment. Following the concepts in the U.S. EPA mixture risk
Available online 9 November 2012
guidance (U.S. EPA, 2000a,b), toxicological interaction for a defined mixture (all components known)
is departure from a clearly articulated definition of component additivity. For the common approach
Keywords:
of dose additivity, the EPA guidance identifies three desirable characteristics, foremost of which is that
Toxicological interaction
Carbamates
the component chemicals are toxicologically similar. The other two characteristics are empirical: the
Nonadditive mixture components have toxic potencies that are fixed proportions of each other (throughout the dose
Synergy range of interest), and the mixture dose term in the dose additive prediction formula, which we call the
Antagonism combined prediction model (CPM), can be represented by a linear combination of the component doses.
Dose–response model A consequent property of the proportional toxic potencies is that the component chemicals must share
a common dose–response model, where only the dose coefficients depend on the chemical components.
A further consequence is that the mixture data must be described by the same mathematical function
(“mixture model”) as the components, but with a distinct coefficient for the total mixture dose. The mix-
ture response is predicted from the component dose–response curves by using the dose additive CPM
and the prediction is then compared with the observed mixture results. The four steps are to evaluate: (1)
toxic proportionality by determining how well the CPM matches the single chemical models regarding
mean and variance; (2) fit of the mixture model to the mixture data; (3) agreement between the mixture
data and the CPM prediction; and (4) consistency between the CPM and the mixture model. Because
there are four evaluations instead of one, some involving many parameters or dose groups, there are
more opportunities to reject statistical hypotheses about dose addition, thus statistical adjustment for
multiple comparisons is necessary. These four steps contribute different pieces of information about the
consistency of the component and mixture data with the two empirical characteristics of dose additivity.
We examine this four-step approach in how it can show empirical support for dose addition as a predictor
for an untested mixture in a screening level risk assessment. The decision whether to apply dose addition
should be based on all four of those evidentiary pieces as well as toxicological understanding of these
chemicals and should include interpretations of the numerical and toxicological issues that arise during
the evaluation. This approach is demonstrated with neurotoxicity data on carbamate mixtures.
© 2012 Elsevier Ireland Ltd. All rights reserved.

夽 Disclaimer: The views expressed in this paper are those of the authors and do not necessarily represent the views or policies of the U.S. Environmental Protection Agency.
Mention of trade names or commercial products does not constitute endorsement or recommendation for use.
∗ Corresponding author. Tel.: +1 404 291 0087.
E-mail address: hertzberg rc@bellsouth.net (R.C. Hertzberg).
1
Present address: Division of Laboratory Science, National Center for Environmental Health, 1600 Clifton Road, Atlanta, GA 30333, USA.
2
Present address: Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh, 130 DeSoto Street, Pittsburgh, PA 15261, USA.

0300-483X/$ – see front matter © 2012 Elsevier Ireland Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.tox.2012.10.016
 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144
1. Background
Interest in mixtures risk assessment is not new, with explicit ref-
erence to mixtures in early laws such as the 1976 Toxic Substances
Control Act (15 U.S.C. §2601 et seq.) and the Comprehensive Envi-
ronmental Response, Compensation, and Liability Act of 1980 (42
U.S.C. §9601 et. seq.). Since 1980, dose addition has been the most
common approach for estimating a mixture response using only
dose–response information on the component chemicals (ATSDR,
2004; U.S. EPA, 1986, 1989b, 2000b). Dose addition has had sev-
eral definitions since its early presentations, with some based on
evaluation of a single mixture point and others involving com-
parisons of dose–response curves throughout the entire response
range of interest (Berenbaum, 1985; Bliss, 1939; Gennings et al.,
2004; Sühnel, 1998). As applied by the U.S. Environmental Pro-
tection Agency (EPA) for risk assessment, dose addition is deemed
appropriate for groups of chemicals that are toxicologically simi-
lar, where depending on the application, “similar” can range from
having the same mode of action (MOA) to merely affecting the
same target organs. EPA’s use of dose addition in dose–response
assessment also involves two main quantitative concepts: pro-
portional toxicity of the component chemicals, and additivity of
potency-weighted doses for estimating the mixture response. EPA
initially applied a rough version of this dose addition approach
in the 1980s for the evaluation of noncancer risk at Superfund
hazardous waste sites by using a hazard index, where the only tox-
icological requirement was similarity of target organs (U.S. EPA,
1989b). EPA also applied a more rigorous version of dose addition
to the dose–response prediction of mixtures of dioxins and furans
using toxicity equivalence factors (TEFs) keyed to the well studied
dioxin congener, 2,3,7,8-TCDD (U.S. EPA, 1989a, 2010). Presently,
EPA and the Agency for Toxic Substances Disease Registry advo-
cate dose addition as a default in the case of multiple chemicals
with similar target organs that lack interaction data (ATSDR, 2004;
U.S. EPA, 2000b). The World Health Organization (WHO/IPCS) also
developed a TEF approach for dioxin-like chemicals (Van den Berg
et al., 1998, 2006) and recently adopted dose addition in their tiered
framework for mixture assessment (Meek et al., 2011).
Published approaches to the evaluation of mixtures in terms
of dose addition vary considerably. One traditional approach is the
use of isoboles, i.e., the graphical displays of isotoxic dose combina-
tions for binary mixtures (Rodea-Palomares et al., 2010; Tallarida,
2006). More advanced and complex analyses use statistical eval-
uations of nonlinear dose–response models along with optimal
experimental designs for mixture data, some with full or fractional
factorial designs (Groten et al., 1996), and others with more effi-
cient experiments using variable mixture dose but fixed component
proportions (Gennings et al., 1997). Some studies only compare
the combination model with mixture data using methods such as
response surface analysis (Casey et al., 2004). We describe in this
paper a four-step approach for evaluating consistency with dose
addition as defined by EPA (U.S. EPA, 2000b), focusing on how
well the mixture dose–response can be predicted using only the
dose–response data on the component chemicals. This approach
begins with finding a common dose–response model for the com-
ponent chemicals and ends with comparisons of the prediction
model with the mixture response.
Dose addition is defined here in terms of a model for predicting
mixture response that is based on the concept of toxicologi-
cal similarity of the mixture’s component chemicals, historically
termed simple similar action (Finney, 1971). That model only
requires the combined information from the dose–response data
of the component chemicals. Such an approach is highly desir-
able for mixtures risk assessment because there is a fair amount
of dose–response information on hundreds of single chemicals,
e.g., EPA’s IRIS database represents 557 substances (accessed online
135
May 21, 2012), but comparatively few studies on mixtures, espe-
cially considering the “multitude of combinations possible” (Boobis
et al., 2011) in the environment. We refer to this prediction model in
general as the combined prediction model (CPM), a name that high-
lights its use in predicting the mixture response whereas a model
derived from the mixture data would describe the mixture response.
The EPA mixture risk methodology (U.S. EPA, 2000b) describes one
aspect of dose addition as: “two chemicals are dose additive if
chemical B is functionally a clone of chemical A.” On the same page
is the notion of toxic proportionality: “for equal effects, the dose
of chemical B is a constant multiple of the dose of chemical A”.
EPA and others have applied this toxic proportionality concept to a
few groups of toxically similar chemicals by estimating the relative
potency factor (RPF) for every chemical in each group (Budinsky
et al., 2006; Müller et al., 2009; U.S. EPA, 2000b, 2003). One com-
mon approach for determining an RPF is to use the ratio of isotoxic
doses, e.g., the dose of an index chemical divided by the equivalent
dose of the chemical of interest. The index chemical is usually the
chemical in the group with the best dose–response data. When one
chemical in the group has limited dose–response data, its response
at any dose can be estimated from the dose–response data of the
index chemical, once its dose is scaled by the RPF. We note here
that the RPF has fewer requirements than the TEF because an RPF
can be restricted to a single exposure route, or duration, or single
toxic effect. The TEF requires same MOA or toxic mechanism and so
a single TEF value applies to all effects and exposure situations. An
additional ramification of the EPA approach implied by the clone
concept and the RPF definition is that for the dose range of interest
the toxic proportionality across components is constant.
There are some important logical consequences of these toxico-
logical similarity descriptions in terms of empirical characteristics
of the dose–response data. First, all components should elicit the
same toxic effect. Second, all components of the mixture should
have geometrically congruent dose–response curves, which means
that once each component dose (on the x-axis) has been scaled
for toxic potency, the component response curves will be identical
in shape. This is the toxic proportionality assumption, illustrated
most famously by isobole plots and by parallel lines in probit by
log(dose) plots (Bliss, 1939; Carter, 1995). One traditional interpre-
tation of toxic proportionality is that the components can be viewed
as dilutions or concentrations of each other. In this paper, we are
extending the usual interpretation of the dilution/concentration
concept to imply that the scatter in the response should only
depend on the magnitude of response, neither on the chemical
nor on the dose. We thus extend the similarity to include the
variance in the response data and are therefore defining toxic
proportionality as toxic surrogacy, where the dose–response rela-
tion for a poorly studied chemical can be estimated by that of
a well-studied chemical. That is in fact the application of index
chemical-based approaches, i.e., TEFs and RPFs (Hertzberg et al.,
1999; Rice et al., 2005). With the extended surrogacy concept to
include the response variance, the response for a poorly studied
chemical can be estimated by the index chemical not only for the
mean response but also its confidence intervals. We then include
a common response variance in the dose additive CPM for the
mixture. This focus on the component modeling and the CPM to
establish toxicological proportionality and the inclusion of the vari-
ance in the definition of dose addition are new aspects to mixtures
dose–response evaluation. We acknowledge that this definition of
dose additivity is more restrictive than previously published ver-
sions, such as those that only consider expected values (means), and
those that allow relative potencies to change for different response
levels, i.e., linear isoboles (for n = 2 chemicals) that are not neces-
sarily parallel (Berenbaum, 1985). There are also other definitions
that are more oriented toward toxicological understanding of a
mixture. Our application is mixture dose–response estimation for
136 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144
use in health risk assessment. Unless otherwise specified, in the
remainder of this paper dose addition refers to the following defi-
nition. When data are only available for the component chemicals,
a chemical mixture is operationally treated as dose additive if the
following properties hold:
(1) The components are toxicologically similar. This is a subjective
determination based on biological information, not on statisti-
cal evaluations.
(2) The mixture components have toxic potencies that are fixed
proportions of each other throughout the dose range of
interest. Quantitatively the components behave as chemical
clones of each other (dilutions or concentrations). Thus their
dose–response curves follow the same dose–response model
and are geometrically congruent, i.e., once each component
dose is scaled for relative potency the curves are theoreti-
cally identical, and the response variance only depends on the
response value, not the chemical.
(3) The predicted mixture response follows the same
dose–response model as the components but with the
dose term replaced by a linear combination of the component
doses. This prediction model is the dose addition formula.
When data do exist for the actual mixture of interest, the final
desirable property is consistency between the prediction model
and the mixture dose–response. For most mixture assessments,
those data will not be available. Supporting information might exist
for a similar mixture, i.e., one with nearly the same composition and
for which the data reflect nearly the same exposure setting (Stork
et al., 2008; U.S. EPA, 2000b). In this paper we evaluate both the
quality of the component-based prediction and the quality of the
consistency with the mixture data.
One often preferred biological support for dose addition is that
the critical toxic effects arise from one toxic mechanism, or at
least from the same mode of action (MOA). Such is the basis of
the TEFs that EPA developed for dioxins and furans and is a main
constraint in the Food Quality Protection Act of 1996 as imple-
mented by EPA for pesticide risk assessment using RPFs (U.S. EPA,
2002a). The TEF concept used by EPA for dioxin-like chemicals actu-
ally extends to all toxic effects of concern and all exposure routes
through the assumption of a common mechanism as the root cause.
That allows one index chemical to be used for prediction of the mix-
ture response for any of those effects, not just for one critical effect.
A common MOA, however, can be insufficient for understanding
interaction potential, because it fails to consider toxicokinetic inter-
ference (Lambert and Lipscomb, 2007). The RPF approach, because
its scope of application can be limited to certain endpoints and
exposure settings, allows it to be applicable to many more chemical
groups than the TEF approach.
In the present study, the goal is to evaluate the prediction pro-
cess outlined in the EPA mixture risk guidance (U.S. EPA, 2000b),
whereby component data are used to predict the response of the
mixture under an assumption of dose addition as defined above.
Part of that evaluation involves comparing the prediction with
actual mixture response data. While toxicological insight helps
one understand data variation and curve shape, we focus here
on empirical support for dose–response similarity of components
and for dose addition. Statistical deviation from such similarity
could then be evidenced by inconsistency in the mean response,
in the response variance, in model parameters, and in other
dose–response characteristics. Our evaluation of consistency with
dose additivity considers similarity across the component chemi-
cals as well as similarity between the components and the mixture,
including comparisons between the component based CPM and
the dose–response model derived from the mixture data (mix-
ture model). We demonstrate these proposed assessment steps
using data from two studies on toxicity of carbamates, neuro-
toxic chemicals exhibiting fairly similar effects and MOAs (U.S. EPA,
2007). The first involves acute exposure to a binary mixture, from
which we use data on inhibition of cholinesterase activity in brain
(Mwanza et al., 2012). The second involves acute exposure to a
mixture of seven carbamates, from which we use data on motor
activity (Moser et al., 2012). Both of those mixture studies used a
ray design (fixed component proportions) so the only measure of
mixture exposure needed is the total dose, i.e., sum of the compo-
nent doses. It must be emphasized that those two articles report
toxicological evaluations of and insights about specific mixtures
regarding the evidence for dose additivity, of which the empiri-
cal analyses are one part. In contrast, the present paper focuses on
the empirical approach itself, regarding its potential for estimating
risks for an untested mixture of the same component chemicals.
While any health risk assessment of a mixture must include toxico-
logical information, in this paper we present and examine only the
empirical analyses, particularly numerical issues that can impact
the interpretation about dose addition.
2. Methods
The purpose of this paper is to describe and critique an approach for evaluating
the evidence for dose additivity of a specified chemical mixture. The evaluation of
dose addition that we propose involves a synthesis of four steps:
(1) evaluate how well the CPM fits the combined single chemical data set,
(2) evaluate how well the mixture model fits the mixture data,
(3) evaluate the agreement between the CPM and the mixture data,
(4) evaluate the consistency between the CPM and mixture model.
2.1. The four steps for analyzing consistency with dose additivity
2.1.1. Step 1: evaluate how well the CPM fits the combined single chemical data
set
In this step, the CPM is determined by regression applied to the data set formed
by merging all the component dose–response data. The resulting model is then
statistically compared with all the component models to decide if the combined
model satisfactorily describes each component’s dose–response data. That statisti-
cal determination of the CPM using only component data is a standard approach to
considerations of dose additivity. What is new here to additivity determinations is
the emphasis on model selection for the components individually, and the inclu-
sion of a common variance in the concept of toxicological similarity. A subsequent
successful fit of the CPM to the merged component data supports the toxicological
proportionality across the mixture’s component chemicals, such as is required by
relative potency factors (Hertzberg et al., 1999; Rice et al., 2005). That proportional-
ity will also be shown by good estimates of any parameters that are common across
the components, such as a background response (at dose 0) or a plateau response
(a high dose asymptote). For example, when the mixture components differ widely
in their plateau response values, the CPM will not show a good fit because its sin-
gle plateau value will be significantly different from many of the plateau values of
the components. Because the CPM is developed using only the component data, a
good CPM fit to the composite single chemical data is necessary but not sufficient
for consistency with EPA’s definition of dose additivity; consistency also requires a
good fit of the CPM to the mixture response (covered in Steps 3 and 4).
Before the CPM can be developed, a model must be selected for the
dose–response data of the components. This model selection must be carefully
performed because the subsequent evaluations regarding dose addition use the
selected model function. Most guidance on nonlinear regression emphasizes that
model selection should be based as much as possible on the understanding of the
underlying physical processes (Bates and Watts, 1988; Conolly, 2001). As is true
for most chemicals of environmental concern, there are insufficient details on the
toxicokinetics and toxicological mechanisms of carbamates to allow biologically
based mathematical models for the endpoints in this study. Our selection of can-
didate mathematical functions is then based on preliminary reviews of the shapes
of all of the component dose–response data sets along with examples from the
dose–response literature (Gennings et al., 2002; Lyles et al., 2008). We used five
representative models (Table 1) that can describe somewhat different curve shapes:
two exponential models, two logistic models and a Gompertz model. Each of these
models can smoothly decrease with increasing dose, all except Exp2 have a plateau
parameter, and the more complex logistic and Gompertz models can have a low
dose shoulder to approximate a region below a threshold dose (i.e., of no statistical
difference from controls). While others have used generalized linear modeling for
such fitting, where the data are transformed to make a linear dose–response curve,
we have followed the advice of modelers to stay in the original data domain (i.e., data
are not transformed) to better understand errors (Bates and Watts, 1988; Garson,
 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144 137

Table 1 where LL is the log likelihood, N is the total sample size and k is the number of
Dose–response models evaluated for the carbamates. model parameters (including the variance). The AICc numerical value has no intrinsic
interpretation, but it is useful for comparing two models, where the smaller AICc
Model namea Formulab Background
indicates the preferred model. No numerical guidance has been found on how much
Exp2 y(d) = exp(˛ + ˇd) exp(˛) smaller the AICc value must be to determine the preferred model, but common
advice is to prefer the simplest model that fits the data (Saffron et al., 2006). We
Exp3 y(d) =  + exp(˛ + ˇd)  + exp(˛)
decided to choose the simpler model (fewer parameters) unless the more complex
1 − 1 − model has an AICc value that is lower by at least 2, regardless of the actual value of
Log3 y(d) =  + 1+exp(d−˛)
+ 1+exp(˛)
each model’s AICc. For each component, the adequate models are identified based on
ϕ ϕ
Log4 y(d) =  + 1+exp(˛+ˇd)
+ 1+exp(˛) AICc and extent of fit based on the p-value, and a preferred model is then selected.
ϕ ϕ The preferred model overall is then the model that performs best across all the
Gomp4 y(d) =  + +
exp(exp(−(˛+ˇd))) exp(exp(−(˛))) components.
a
The model name includes the number of parameters to be estimated. Once the preferred model function has been selected, the CPM is formed. As
b
y = response, d = dose,  = plateau response at high dose. Note that ˛,  1 and originally shown by Berenbaum and explained in detail by Meadows and colleagues
ϕ do not have any real world interpretation. The y(0) in the model results is the (Berenbaum, 1985; Meadows et al., 2002), the dose additive mixture prediction
background response from the model and not necessarily in agreement with the formula (CPM) replaces the dose term, ˇd, used in the component dose–response
control group mean. function by the combined dose term, sum[ˇi di ], i.e., a linear combination of all the
component doses. For a mixture of c chemicals, the CPM is then fitted to the compos-
ite data set formed by merging the c individual component data sets. For example,
2009), and so we use nonlinear regression with the selected models. Each candidate with a binary mixture and the Exp3 model (decreasing exponential with a plateau
model was fitted by nonlinear regression using PROC NLMIXED (SAS Institute Inc., in the higher dose range), the CPM that corresponds to the model in Table 1 is:
2008), with model fit evaluated using the F statistic: y = g(d1 , d2 ) =  + exp(˛ + ˇ1 d1 + ˇ2 d2 ) (3)
SSLF/(c − k) To facilitate understanding of the different models, we are using this simpler func-
F= (1)
SSPE/(n − c) tion notation where the subscripts refer to the data set (e.g., d1 for dose of component
1) instead of the standard statistical notation of a model where dosei refers to the
where SSLF is the lack-of-fit sum of squares, SSPE is the pure error sum of squares, c
ith data point. In Eq. (3), we see the common plateau response () and common
is the number of unique dose values, k is the number of parameters that need to be
background response at zero dose ( + exp(˛)). One can visualize the CPM fit with
estimated and n is the number of observations (Kutner et al., 2005). Consistent with
two component chemicals by a graph showing the CPM and data on the walls of a
the EPA benchmark dose guidance, acceptable fit is when the p-value >0.1 (Graves
box, which represent the single chemical dose–response planes (Fig. 1).
et al., 2001; U.S. EPA, 2000a). Model quality was also judged using the corrected
The CPM is then evaluated for fit using only the single chemical response data;
Akaike Information Criterion (AICc), which is bias-corrected for small sample sizes.
there are no interior mixture data (where both components are at nonzero doses).
It penalizes a model for having too many parameters (adds 2 for each parameter)
Any conclusion at this point about dose addition is based on the consistency between
and rewards a model for good fit, thus balancing fit and simplicity (Burnham and
the CPM and the single chemical models. That consistency is evaluated using a like-
Anderson, 2004). The formula for this criterion is
lihood ratio test because the models are nested, i.e., each component model is a
2k(k + 1) special case of the CPM where the doses are zero for all other components. For
AICC = −2LL(|data) + 2k + (2) example, in comparing these models for a binary mixture, the number of degrees of
N−k−1
freedom for the chi-square test statistic is the difference between the total number

Fig. 1. Results of fitting a dose-additive model to the composite of the two single chemical data sets. This example shows the Exp3 model simultaneously fitted to the
response data for carbaryl and propoxur. Response: brain ChE activity as percent of control mean.
138 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144
of parameters in the two component models and the number of parameters in the
CPM. Because we estimate the mean and the variance, the number of parameters
estimated from each model-fitting step is the number of model parameters plus
one for the variance. For a binary mixture with the three-parameter Exp3 model,
this test statistic then follows a chi-square distribution with 3 (=4 + 4 − 5) degrees
of freedom.
2.1.2. Step 2: evaluate how well the mixture model fits the mixture data
Under dose addition, the dose–response model common across the component
chemicals must also be the model for the mixture, i.e., the same mathematical func-
tion but different parameters. When the test mixture has a constant composition
across all dose groups, the component fractions are fixed so each component dose
is the total mixture dose (sum of component doses) multiplied by that fraction. It
is easy to show that in such cases, the mixture model will be the same function as
the component models but with its mixture specific dose coefficient and with dose
given as the total mixture dose (Meadows et al., 2002). For example, a simple expo-
nential model for component 1, y1 (d) = exp(˛ + ˇ1 d) becomes the mixture model,
ymix (dmix ) = exp(˛ + ˇmix dmix ) where dmix is the sum of the component doses. Aside
from the dose measure, the only change is the dose coefficient, ˇ. Model fitting and
the evaluation of lack of fit is performed as with the component models and with
the CPM. A good fit usually supports consistency of dose–response shape across the
component and mixture data sets, and thus provides another piece of evidence in
support of dose addition.
2.1.3. Step 3: evaluate the agreement between the mixture data and the CPM
Each response mean, ȳ, for the mixture data is compared with the CPM by use of
prediction intervals, which account for variation in the CPM as well as in the mixture
data. Using a large sample approximation, the 95% prediction interval is:
 dose–response assessment (U.S. EPA, 2000b). For this study, the evaluation and con-
ŷ ± t0.975,N−k var(ŷ) + var(ȳ) (4)
where k is the number of model parameters in the CPM and N is the total number
of data points used to estimate the model parameters (Gennings et al., 1997). Most
useful for displaying the predictive ability of a model in a mixture risk assessment,
this is the classic comparison of model with actual data. If the mixture means lie
outside the prediction interval, then a reasonable conclusion is that the component
and mixture data are not consistent with dose addition. An adequate prediction is
consistent with dose addition, but the model could also capture the mixture data if
the CPM variance is high so the prediction intervals are large. Because of the empha-
sis in this paper on model comparisons, the CPM is only presented graphically with
the mixture means; statistical tests comparing the data means with the prediction
intervals are not performed.
2.1.4. Step 4: evaluate the consistency between the CPM and mixture model
This fourth step is to statistically compare the models: the CPM with the mixture
model. This comparison involves all model parameters, separately and collectively,
and also identifies the doses at which the models are significantly different. These
models were fitted using distinct data sets, i.e., components or the mixture. Because
they share a common model formula, this comparison can directly evaluate consis-
tency in terms of the observable parameters, e.g., those representing the background
and plateau responses. If dose addition is consistent with the data, then the param-
eters of the CPM should be close to those of the mixture model. For example, with a
binary mixture of carbaryl and propoxur and the Exp3 model, the Wald test for dose
additivity evaluates the composite hypothesis of all three parameter differences:
H0 : ˇmix = fˇ21 + (1 − f )ˇ22 ; mix =  and ˛mix = ˛ (5)
where each mixture parameter has a mix subscript, unsubscripted parameters are
those of the CPM, and f represents the mixture fraction that is carbaryl. Because
the comparison involves models, it is more robust than comparing the CPM with
the mixture data (Step 3) because it covers the full dose range, is less sensitive to
data outliers, and identifies any differences in measurable “real world” values such
as the magnitude of the high-dose plateau response. This comparison, however, is
dependent on how well the CPM and mixture model fit their respective data. For
example, when fits are poor, biologically important differences could be statistically
similar because of the high variances.
2.2. Evaluation of modeling results for consistency with dose addition
Adequacy of the dose addition formula for predicting mixture response is then
gauged by qualitatively considering together the results from all four steps. Conclu-
sions about adequacy can range from full empirical support for the EPA definition to
various types of partial support, e.g., consistency of means but not of variability in
response, adequate prediction in a restricted dose range (e.g., near the lowest sig-
nificant responses), or toxic proportionality for some but not all components. There
can also be complete lack of support, e.g., in spite of the toxicological similarity
across the components there is no empirical support from any of the four evalua-
tions in our methodology. These conclusions are primarily empirical, and thus must
then be interpreted regarding toxicological importance and possible toxicological
implications (e.g., toxicokinetic interactions, differences in MOAs).
There is also a final operation, which should be present in any dose–response
assessment, namely a summary of uncertainties. Certainly uncertainties have been
Table 2
Summary of the four evaluations of consistency with dose addition for the binary
mixture with the endpoint of brain ChE activity as percent of control mean,
model = Exp3.
1. CPM vs. single chemicalsa Yes, means agree; No, variances differ
2. Mixture modelb Yes
3. CPM vs. mixture datac No, less than dose additive at two middle
doses (mixture mean activity less inhibited
than predicted by the CPM)
4. CPM vs. mixture modeld No, slope parameters differ (adjusted
p = 0.006)
a
In Step 1, the statistical evaluation compares the CPM with the component
models, looking for consistency of the means and variances.
b
In Step 2, the mixture model, which is the same as the components’ models but
with parameters derived from fitting to the mixture data, is evaluated for goodness
of fit to the mixture data.
c
In Step 3, the CPM is evaluated for consistency with the mixture data, by dis-
playing prediction intervals for the CPM and dose group means for the mixture
data.
d
In Step 4, the CPM is compared with the mixture model globally, by parameter,
and at each dose. Only global and parameter comparisons are shown here.
included in the initial deliberations of toxicological similarity, the gathering of
the dose–response data on the components and the mixture, the dose–response
modeling and the statistical comparisons. Here we are emphasizing the need for
a summary discussion of uncertainties when presenting the conclusions of the
clusions are examined for weaknesses in the modeling processes, particularly any
numerical properties that would interfere with the ability to determine consistency
with or departure from dose additivity. One particular approach we recommend
for gauging the possible impact of a component’s model lack of fit is to calculate
an expected component contribution (ECC), which we define as the percentage of
the mixture response under dose addition that is expected from each component.
For example, if a component’s expected contribution to the mixture is small, then
uncertainties in its dose–response modeling are not likely to be important. Because
this study uses oral exposures, each ECC estimate is calculated as the product of
the component’s fraction in the mixture and the component’s oral relative potency
factor (RPF), but scaled so the sum of scores is 100. The oral RPFs used here are the
values employed in the study designs that provided our example data set (Setzer,
U.S. EPA, personal communication, Moser et al., 2012).
2.3. Example data sets
The data in the examples used here come from two EPA toxicology studies of
rats with acute oral exposure to carbamates (singly and in mixtures): one on binary
mixtures (Mwanza et al., 2012) and one on a seven-chemical mixture (Moser et al.,
2012). The chemical proportions in this latter mixture were derived from a surro-
gate for relative exposure: the relative amounts of each chemical sold in the state of
California in 2005. Both studies measured several endpoints. The data we are using
here include: for the binary mixture, cholinesterase (ChE) activity in brain in terms
of percentage of the control mean; and for the seven-chemical mixture, the number
of light-beam breaks in a 20-min interval as a measure of motor activity. The tox-
icological aspects of those two studies, including the toxicological interpretations
of the evaluations of dose additivity, are presented in those two publications. We
present here the details of the statistical methods used in those studies to evaluate
consistency with dose additivity.
Statistical analyses were mainly conducted using Proc NLMIXED of SAS 9.2 (SAS
Institute Inc., 2008). Supplemental analyses and graphics were conducted using JMP
8 (SAS Institute Inc., 2009).
3. Results with example data sets
3.1. Example of a binary mixture of carbaryl and propoxur, brain
ChE activity
The two chemicals in this mixture, carbaryl and propoxur, are
carbamate pesticides. A full description of the component and mix-
ture toxicology studies and additivity results has been previously
published (Mwanza et al., 2008, 2012) and those results for brain
ChE activity are summarized in Table 2. The toxic endpoint of
concern discussed here is relative cholinesterase inhibition (as per-
cent of control mean), with increasing inhibition as dose increases.
Cholinesterase activity then decreases with dose. This example
focuses on brain cholinesterase activity when represented as per-
cent of the control mean.
 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144 139

Fig. 2. CPM with component data and dose group mean (circle) for binary mixture study, brain ChE as percent of control mean. Left: carbaryl and right: propoxur.

Step 1: The initial step is to evaluate the fit of the CPM to the com-
ponent data sets. Data for cholinesterase activity show a smooth
decrease with dose for each chemical (Fig. 2), with a plateau
response at high dose, suggesting the Exp3 model. Indeed, the
evaluation of lack-of-fit significance as well as the corrected Akaike
Information Criterion indicates that the Exp3 model performs best
of the five models tested (Table 3). Tests for equality of the variance
across dose showed no significant differences for carbaryl and for
the binary mixture, but dose dependence for propoxur. Because
our dose additivity definition includes a common variance, the
most consistent approach to apply both to components and to the
mixture was to assume no difference across dose. The CPM fits very
well to the data and means for both components (Table 4, Fig. 2).
The likelihood ratio test comparing the CPM to the
two single chemical models yields a test statistic of
759.0 − (379.5 + 366.6) = 12.9. Under the null hypothesis that
the two chemicals are toxicologically equivalent, the statistics
follow a chi-square distribution with 3 degrees of freedom
(p = 0.005), showing significant differences among the three
models (Mwanza et al., 2012). The data for brain ChE inhibition
by carbaryl and propoxur would then not be consistent with the
toxic proportionality assumption. How can this be when the CPM
fits the component data so well (Fig. 2)? The statistical difference
is mainly caused by the differing variances of the carbaryl and
propoxur models, 122.0 vs. 228.9, respectively. Based on the CPM
fitting, these single chemical data sets are then not consistent with
toxic proportionality in terms of congruence of the dose–response
data (e.g., confidence limits), although they are highly consistent
in terms of the estimated mean response.
Step 2: The next step is to fit the same model formula, Exp3, to the
mixture data to evaluate curve shape similarity between the mix-
ture and the components. Dose for the mixture is simply the total
mixture dose (sum of component doses). This simplified represen-
tation can be used because the data are for a fixed-ratio ray design,
i.e., the total dose uniquely defines the component doses. The data
are sparse near the point of maximum curvature (near 20 mg/kg)
but the overall fit of Exp3 to the mixture data is good (Fig. 3), with
a lack of fit p-value of 0.17 and good parameter estimates, shown
by the small standard errors and p-values in Table 5.
Step 3: Next is to compare the CPM directly with the mixture
data. This is performed here using prediction intervals for the CPM
and dose group means for the mixture response. Prediction inter-
vals are more informative than confidence intervals because they
incorporate the observed variation in the mean response in the
mixture data, so that means outside the prediction intervals indi-
cate statistical departure from dose additivity (Gennings, 1995).
While the plateau response of the mixture data is not apparent
from the means, the mean responses at the highest two doses

Table 3
Model comparisons for the binary mixture components carbaryl and propoxur for
brain ChE activity as percent of control mean.

Component Model AICc Lack of fit

Exp2 483.2 <0.001

Exp3 387.9 0.945
Carbaryl Log3 403.2 0.004
Log4 389.9 0.846
Gomp 389.9 0.847

Exp2 410.8 <0.001

Exp3 375.0 0.483
Propoxur Log3 379.8 0.066
Log4 411.3 <0.001
Gomp 413.8 <0.001 Fig. 3. Fit of the Exp3 mixture model to the binary mixture data for brain ChE as
percent of the control mean. Symbol for dose group means (triangle) is enlarged for
improved legibility. Mixture dose is the sum of component doses.
Table 4
Estimates of background, plateau and slopes of the Exp3 CPM for the binary mixture Table 5
for brain ChE activity as percent of control mean. Estimate of background, plateau and slope of the mixture model: the Exp3 model
fitted to the binary mixture data for brain ChE activity as percent of control mean.
Estimate Lower conf. limit Upper conf. limit
Estimate Lower conf. limit Upper conf. limit
Background 98.647 95.398 101.89
Plateau 40.922 38.023 43.821 Background 100.44 95.822 105.07
Carbaryl slope −0.144 −0.183 −0.105 Plateau 39.544 35.815 43.272
Propoxur slope −0.258 −0.347 −0.168 Mixture slope −0.103 −0.130 −0.076
140 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144

Table 7
Carbamate components of the seven-chemical mixture and their EPA relative
potency factors.

Chemical Mixture fraction Oral RPFa Expected component

contribution (percent)b

Carbaryl 0.39 0.15 4.2

Carbofuran 0.038 2.4 11.8
Formetanate 0.033 2.18 9.3
Methiocarb 0.0029 0.18 0.1
Methomyl 0.41 0.67 36.2
Oxamyl 0.13 1.00 38.4
Propoxur 0.0018 0.11 0.0
a
Oral RPFs were calculated from doses causing 10% inhibition of brain ChE activity
(W. Setzer, U.S. EPA, personal communication).
b
Expected component contribution is equal to the percentage of the mixture
toxicity expected from each component based on dose additivity, calculated as the
Fig. 4. Comparison of the CPM with the binary mixture response data from the ray
product of RPF and mixture fraction then scaled so the sum equals 100.
design for brain ChE activity as percent of control mean, showing the CPM with 95%
prediction intervals (horizontal bars) and dose group means (triangle) at each dose.
Symbol for means (plus) is enlarged for improved legibility. Mixture dose is the sum Table 8
of component doses. The four evaluations of consistency with dose addition for the seven chemical mix-
ture with the endpoint of motor activity, model = Exp3.

are within the prediction intervals for the CPM plateau. Thus the
mixture data suggest less ChE inhibition than predicted at 3 and
10 mg/kg, but are not statistically different from the CPM predic-
tion at the highest doses (Fig. 4).
Step 4: The fourth step is to statistically compare the models: the
CPM with the mixture model. These models were fitted using dis-
tinct data sets, i.e., components or the mixture. Because they share
a common model formula, this comparison can directly evaluate
consistency in terms of the observable parameters, in this case
those representing the background and plateau responses. The
early leveling off of the CPM compared with the mixture model
(compare Figs. 2 and 3) forces the CPM to drop more sharply and
underestimate the mixture ChE activity at the middle doses. From
Step 3, for those doses the mixture means display statistically less
toxicity than predicted by the CPM. Combining that result with the
smoothness of the mixture data, we expect the slope parameters
to be different. In this step, inconsistency with dose addition is
shown when the parameters of the CPM are not close to those of
the mixture model.
The result from the Wald test (Eq. (5)) is a p = 0.0013, indicat-
ing significant differences between the models. If we separately
test the three parts of the null hypothesis, which represent the
slope, plateau and background of the models, and then adjust the
p-values for multiple comparisons by the Hochberg and Stepdown
Bonferroni (Holm’s correction) methods (Hochberg and Benjamini,
1990), we find that only the slope parameters are significantly dif-
ferent (Table 6, adjusted p = 0.006). The estimate of the plateau
response from the CPM is very close to that of the mixture model,
40.9 vs. 39.5, respectively, as percent of the control mean for brain
ChE activity. The difference in slopes is confirmed by comparing
model predictions at each tested dose. Only at the middle doses of 3
and 10 mg/kg are the predicted means different (adjusted p < 0.001)
between the two models.
1. CPM vs. single chemicalsa No. plateaus differ, poor fit of preferred model
to two chemicals, differently shaped
component dose–response curves
2. Mixture modela No. curve fit, plateau estimate not statistically
significant (p > 0.8)
3. CPM vs. mixture dataa No. less than dose additive at third dose,
greater than dose additive at three highest
doses
4. CPM vs. mixture modela No. slope and alpha parameters differ, plateau
parameters marginally different (p = 0.08)
a
The explanations of each column are the same as those given for Table 2.
Having illustrated in detail how the four steps apply in the first
example, the remaining example focuses on selected steps. The
goal in the following example is to illustrate key strengths and
complications of the analyses.
3.2. Example of a seven carbamate mixture, motor activity
This example demonstrates the four-step evaluation with a
more complex mixture composed of seven carbamate pesticides
(Table 7). A full description of the component and mixture toxico-
logy studies and additivity results is found elsewhere (Moser et al.,
2012) and the results are summarized in Table 8. We highlight here
the main additivity calculations and some issues that arose during
the analysis. In contrast with the first example, the toxic endpoint
of concern discussed here is decrease in motor activity. Photobeams
spaced around an activity chamber detected rat movement by pho-
tobeam breaks during a 20-min session. Because ChE inhibition can
contribute to decreased motor activity, a similar dose–response
shape is expected, i.e., greater suppression as dose increases and
an exponential decay curve with motor activity (as counts of pho-
tobeam breaks) decreasing with dose. The response, however, is
at a higher organizational level, based on whole animal behavior
instead of enzyme activity, so greater differences across the mixture

Table 6
Significance evaluation for the multiple tests of the model disagreement (binary mixture model vs. CPM) for brain ChE response as percent of control mean.

Evaluationa Estimateb Standard error DF t-Value p-Value Adjusted p-valuec

ˇmix − fˇCPM1 − (1 − f )ˇCPM2 0.0876 0.0280 285 3.13 0.0019 0.006

mix −  −0.1378 0.2375 285 −0.58 0.5621 0.729
˛mix − ˛ 0.0536 0.0590 285 0.91 0.3644 0.729
a
ˇCPM1 and ˇCPM2 represent the slope parameters in the CPM for the first and second components, and f represents the mixture fraction that is component 1.
b
Response data were scaled by 10 to improve the numerical convergence, so plateau =  × 10, i.e., the observed mixture plateau response is smaller than predicted by
1.378 (39.544 vs. 40.922).
c
Adjusted p-values from stepdown Bonferroni (Holms correction) method for multiple comparisons. Hochberg adjustment produced identical result for slope parameters
and slightly smaller values (0.562) for the other two parameters.
 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144
Fig. 5. Evaluation of statistical fit to the motor activity count measure for the five
models where acceptable lack of fit is when p ≥ 0.1. The chemicals are ordered
according to their expected contributions under dose addition, with Methiocarb
and Propoxur expected to contribute little to the joint toxicity in the seven-chemical
mixture.
components and more complex dose–response shapes would not
be surprising results.
With seven component chemicals, the component model fitting
produces 35 regression analyses (7 chemicals × 5 models). Selec-
tion of the preferred common model for the components (Step 1)
was facilitated by a composite bar chart of the lack of fit p-values
(Fig. 5). The Exp3 model (exponential with plateau) describes well
the data for five of the components, with the worst lack of fit values
for carbofuran (p = 0.049) and oxamyl (p = 0.004). The plateau esti-
mates for the 20-min count vary across the components. Because
the Exp3 model had unconstrained parameters, the plateau param-
eter estimate could be less than zero, a biological impossibility for
this endpoint. This occurred for oxamyl (plateau estimate = −243),
whose data do not follow a smoothly declining exponential curve
(Fig. 6, right side), especially in the low dose region. Methomyl
also had a negative plateau (−824) and showed acceptable fit by
the Exp3 model, but minimal curvature: a linear model fit very
well (p = 0.9). The plateau estimates for the other five components
ranged from 12 to 57. At the highest tested dose, oxamyl and
methomyl had relatively high mean response values of 42 and 65,
respectively. For comparison, propoxur had a mean response of 44
while the other four components had response values of 33 or less.
For methomyl and oxamyl, their motor activity data do not ade-
quately define the high dose curvature needed for a reasonable
plateau estimate. The plateau estimates for the other five compo-
nents ranged from 12 to 57. The CPM differed statistically from
the seven single chemical models (p < 0.005), thus the empirical
support for toxicological proportionality is quite weak. This result
Fig. 6. Motor activity data for carbofuran (left) and oxamyl (right), the components of the seven-chemical mixture with the poorest fit by the preferred Exp3 model (see Fig.
5), showing somewhat different curve shapes. Symbols for means (triangles) are enlarged and connected by dashed line for improved legibility. Curve: CPM based on the
Exp3 model.
141
Fig. 7. Goodness of fit p-value for the Exp3 model vs. the expected component con-
tribution (ECC) in the seven-chemical mixture from the ray design for motor activity.
ECC is equal to the percentage of the mixture toxicity expected from each compo-
nent based on dose additivity. Horizontal line: 10%, the p-value minimum used by
EPA for acceptable dose–response model fit. ECC = 14% if all chemicals were in equi-
toxic proportions. Oxamyl is expected to contribute most to the mixture toxicity yet
has the worst fit of the Exp3 model. Cr, carbaryl; Cf, carbofuran; Fm, formetanate;
Mc, methiocarb; Mm, methomyl; Ox, oxamyl; Pr, propoxur.
is consistent with the differences in estimated plateau response
and the poor fit of the Exp3 function to the data of two compo-
nents (Fig. 6). The importance to the mixture prediction of the poor
oxamyl fit can be suggested by its expected component contribu-
tion of 38.36% (Fig. 7), which indicates oxamyl is expected to be the
most influential component on the mixture response.
The dose additive CPM prediction of the mixture differed sub-
stantially from both the observed mixture response means and
the mixture model (Wald test comparing CPM and mixture model,
p < 0.0001) (Fig. 8). The plateau estimates for the motor activity
counts were 22.9 (CPM) vs. 1.7 (mixture model). Consistent with
that result, the mixture showed (Steps 3 and 4) statistically lower
response values, reflecting greater than dose additive suppression
of motor activity, at the three highest doses.
4. Discussion
The weakest part of a mixture risk assessment is often the
lack of dose–response data on the particular mixture of concern.
The purpose of the present work was to examine a proposed
dose–response assessment approach based on dose additivity to
determine its usefulness in estimating mixture response using
information only on the component chemicals. Once the com-
ponent chemicals are deemed to be toxicologically similar, the
proposed four-step approach is mainly empirical, with conclusions
derived from statistical analysis of dose–response data and models.
142 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144
Fig. 8. Comparison for the seven-chemical mixture of the Exp3-based CPM (solid)
with the Exp3-based mixture model (dashed) along with the response data and dose
group means from the ray design for motor activity. Symbols for means (triangles)
are enlarged for improved legibility. The two models are statistically different at all
four highest doses. Mixture dose is the sum of component doses.
These four evaluation steps provide somewhat different lines of
evidence about the support for dose additivity. When deciding
whether to apply dose addition to a particular set of chemicals, the
purpose of that application should be matched to the most appro-
priate lines of evidence. As this is a new approach, it is not yet
known, but presumed unlikely that many mixtures and their com-
ponent chemicals will have strong evidence across all four areas.
Inconsistency with one area is not necessarily grounds for aban-
donment of dose addition. For example, if the goal is to predict
the mean mixture toxicity over dose ranges close to those in the
underlying data sets, then the ability of the CPM to predict the mix-
ture mean would rank first in importance, with its consistency with
component means a close second. If the goal is to predict the mix-
ture response in the low dose region, which is more relevant to most
environmental exposures, then support for dose addition should be
examined for the low doses and consistency of high dose plateau
estimates should not be a deciding factor. A conclusion of strong
empirical support for dose addition from all four evaluations pro-
vides a firm foundation for the dose–response assessment phase
of the mixture risk assessment, i.e., estimating the response of the
untested mixture using dose addition is scientifically sound.
A conclusion that dose addition is not supported by any of the
evaluations for a particular set of chemicals implies that dose addi-
tion should not be applied to that set, and therein lies the primary
failing of yes/no decisions: rejection of a procedure without any
measure of the error in its use. For joint toxicity, the measure we
recommend is the interaction magnitude, which is defined here
as the proportional change in isoeffective dose (Hertzberg and
Teuschler, 2002). For example, if the isoeffective dose is the ED10
(dose causing 10% response), then the interaction magnitude is the
ratio of the ED10 predicted under dose addition to the ED10 esti-
mated from the mixture data (or the inverse, if the ratio is less
than one). For example, if the interaction magnitude is 3, then an
estimated “safe” exposure level based on dose addition would be
in error by 3-fold, either too high or too low based on whether
the predicted ED10 is higher or lower than observed, respectively.
While it is extremely rare for joint toxicity studies to quantify inter-
actions (Boobis et al., 2011), the approach described here provides
that information in the two models, i.e., the combined prediction
model (component data) and the mixture model (mixture data).
Instead of merely deciding for or against dose addition, the ana-
lyst should quantify the interaction. A good example is Moser
et al. (2012) where the interaction magnitude was determined for
multiple toxic endpoints for each of two different mixtures of the
same components.
The main advantage of the proposed four-step approach is clar-
ity in interpreting the results, some of which could be missed
by a simple yes/no evaluation of dose additivity. For example,
if all components show adequate model fit and good evidence
of toxicological proportionality, that result alone is valuable for
risk assessors because it supports the use of chemical surrogates
based on RPFs (or TEFs) for those many chemicals that do not have
dose–response models.
Further, if the component models are only consistent when the
common variance requirement is dropped (allowing a separate
variance parameter for each component term in the CPM), then
toxicological proportionality holds for estimating mean responses.
For example, in the binary carbamate mixture and brain ChE activ-
ity results discussed above (Mwanza et al., 2012), the component
variances were not consistent. A reanalysis without the common
variance gives the conclusion that carbaryl can serve as a surrogate
for estimating the ChE response for propoxur (the RPF approach)
but only for the mean response. Because the component variances
were not consistent, risk methods that use confidence limits on the
dose–response curves, such as the lower bound on the benchmark
dose (for EPA’s reference dose) or the dose at an upper bound on a
specified cancer risk level (U.S. EPA, 2002b, 2005), would then have
weaker empirical support. In this example, if the propoxur BMDL
were estimated from the Carbaryl BMDL using the RPF approach,
there would be increased uncertainty because of the different vari-
ances. Note that these conclusions would be missed in an evaluation
that only fit an additive model to the mixture.
Now consider a thought experiment where toxic proportional-
ity is established, and then the mixture shows significant deviation
from dose additivity in the mean response at certain combination
doses. That result could lead to particularly strong toxicological
insight. If those deviations are toxicologically important, then a
follow up investigation should focus on changes in the toxicological
processes, e.g., alterations in toxicity, repair, or in tissue concentra-
tions, and why they occur at those identified doses. Such a situation
is important to understand because dose addition is incorrect yet
it is the approach that would have been used if no mixture data
were available because of the components’ toxicological similar-
ity. This thought experiment also emphasizes why toxicological
judgment must be combined with the statistical characterization
to properly interpret the model comparisons in terms of potential
toxicity (Brown et al., 1988).
The successful fitting of the mixture data with the preferred
model function is necessary for supporting dose addition, as is the
CPM fitting well the component data, but those steps are not suf-
ficient to confirm consistency with dose addition. The third and
fourth steps are needed because those steps compare the predic-
tion with the observations, i.e., model vs. data (Step 3) and model vs.
model (Step 4). The model comparisons give a more encompassing
result because the entire data sets are involved in the modeling, so
dose by dose fluctuations are somewhat smoothed out. In addition,
the full mixture dose range is used in the evaluation, so dose-
related trends (and model weaknesses) should be easier to identify,
improving the understanding of how far the results can be gener-
alized to other mixtures of the tested components.
The final caution is that the evaluation steps discussed here only
establish consistency with dose additivity for the data evaluated. If
the exposure scenario of the application differs too much from the
exposures evaluated for dose additivity, extrapolation uncertainty
can be substantial. That uncertainty is not merely the numerical
consequence of modeling outside the range of the data. Mixtures
can cause several types of toxic effects (Carpenter et al., 2002),
and dose additivity for one effect does not confer dose additivity
for other effects. Mixtures of the same compositions can show
different effects or interactions if administered by different routes
(Choudhury and Mudipalli, 2008). Moser et al. (2012) show how
 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144
two mixtures of the same seven components but different indi-
vidual doses and different proportions can produce quite different
results. Even binary mixtures of the same components but different
proportions (mixing ratios) can show different types of toxico-
logical interaction, e.g., the classic chloral hydrate and ethanol
isobole (Gessner, 1995). Consider a hypothetical example mixture
of ten components, where one chemical is at a toxicologically small
proportion and has a very different curve shape than the other com-
ponents. Dose addition might still adequately predict the response
of the tested mixture because that component contributes little
to the mixture response (an extreme case of the seven-chemical
example given above). A different mixture where that component
is dominant could show substantial deviation from dose addition
because of the change in mixture curve shape. This example shows
why the toxicological proportionality (Step 1) must be evaluated;
the comparison of component model fits would have detected the
outlier chemical and raised a caution when extrapolating to a mix-
ture where that chemical was dominant.
We conclude that empirical interaction studies relating interac-
tion to administered doses do offer utility for risk-based screening.
The four-step approach presented here offers a reasonable evalu-
ation of consistency with dose addition. Good consistency would
then support the use of dose addition to predict the dose–response
of an untested mixture of the same component chemicals. How-
ever, to go beyond a screening level assessment and actually predict
the array of mixture effects, a more complete understanding of the
toxicological processes is needed, particularly for generalization to
other mixtures of the same chemicals with widely different pro-
portions or total doses. Physiologically based toxicokinetic (PBTK)
models show potential in this regard. For example, PBTK mod-
els derived from kinetics studies on binary mixtures have been
shown to predict toxicity for more complex mixtures (Krishnan
et al., 2002). Further, research on integrating exposure assessments
with PBTK models offers a promising approach for extending sin-
gle route results to address multiroute scenarios (Arnold and Price,
2007). Unfortunately such studies are rare. Until such biologically
based approaches are more broadly applied, reliance on simpler,
scenario-specific component based formulas such as dose addi-
tivity will continue to provide important information to inform
mixture dose–response and risk assessment.
Conflict of interest statement
All authors state that there is no conflict of interest.
Funding
This work was supported by intramural funding of the US
Environmental Protection Agency. Statistical analysis was funded
by EPA contract GS-10F0369N to Toxicology Excellence for Risk
Assessment and EPA Purchase Order EP-D-08-085 to R.C. Hertzberg.
References
Arnold, S.F., Price, P.S., 2007. Modeling mixtures resulting from concurrent expo-
sures to multiple sources. Toxicol. Appl. Pharmacol. 223, 121–124.
ATSDR, 2004. Guidance Manual for the Assessment of Joint Toxic Action of Chemical
Mixtures. Agency for Toxic Substances and Disease Registry, U.S. Department of
Health and Human Services, Public Health Service, Atlanta.
Bates, D., Watts, D., 1988. Nonlinear Regression Analysis. John Wiley and Sons, New
York.
Berenbaum, M.C., 1985. The expected effect of a combination of agents: the general
solution. J. Theor. Biol. 114, 413–431.
Bliss, C.I., 1939. The toxicity of poisons applied jointly. Ann. Appl. Biol. 26, 585–615.
Boobis, A., Budinsky, R., Collie, S., Crofton, K., Embry, M., Felter, S., Hertzberg, R., Kopp,
D., Mihlan, G., Mumtaz, M., Price, P., Solomon, K., Teuschler, L., Yang, R., Zaleski,
R., 2011. Critical analysis of literature on low-dose synergy for use in screening
chemical mixtures for risk assessment. Crit. Rev. Toxicol. 41, 369–383.
143
Brown, S.L., Brett, S.M., Gough, M., Rodricks, J.V., Tardiff, R.G., Turnbull, D.,
1988. Review of interspecies risk comparisons. Regul. Toxicol. Pharm. 8,
191–206.
Budinsky, R.A., Paustenbach, D., Fontaine, D., Landenberger, B., Starr, T.B., 2006. Rec-
ommended relative potency factors for 2,3,4,7,8-pentachlorodibenzofuran: the
impact of different dose metrics. Toxicol. Sci. 91, 275–285.
Burnham, K.P., Anderson, D.R., 2004. Multimodel inference: understanding AIC and
BIC in model selection. Sociol. Methods Res. 33, 261–304.
Carpenter, D.O., Arcaro, K., Spink, D.C., 2002. Understanding the human health effects
of chemical mixtures. Environ. Health Perspect. 110 (Suppl. 1), 25–42.
Carter, W.H., 1995. Relating isobolograms to response surfaces. Toxicology 105,
181–188.
Casey, M., Gennings, C., Carter, W.H., Moser, V.C., Simmons, J.E., 2004. Detecting
interaction(s) and assessing the impact of component subsets in a chemical
mixture using fixed-ratio mixture ray designs. J. Agric. Biol. Environ. Stat. 9,
339.
Choudhury, H., Mudipalli, A., 2008. Potential considerations & concerns in the risk
characterization for the interaction profiles of metals. Indian J. Med. Res. 128,
132–153.
Conolly, R.B., 2001. Biologically motivated quantitative models and the mixture
toxicity problem. Toxicol. Sci. 63, 1–2.
Finney, D.J., 1971. Probit Analysis. Cambridge University Press, Cambridge.
Garson, G.D., 2009. Nonlinear Regression Statnotes: Topics in Multivariate Analysis.
Course notes for Public Administration, Raleigh, NC, p. 575.
Gennings, C., 1995. An efficient experimental design for detecting departure from
additivity in mixtures of many chemicals. Toxicology 105, 189–197.
Gennings, C., Carter, W.H., Carney, E., Charles, G., Gollapudi, B., Carchman, R., 2004.
A novel flexible approach for evaluating fixed ratio mixtures of full and partial
agonists. Toxicol. Sci. 80, 134–150.
Gennings, C., Carter Jr., W.H., Campain, J.A., Bae, D.-s., Yang, R.S.H., 2002. Sta-
tistical analysis of interactive cytotoxicity in human epidermal keratinocytes
following exposure to a mixture of four metals. J. Agric. Biol. Environ. Stat. 7,
58–73.
Gennings, C., Schwartz, P., Carter Jr., W.H., Simmons, J.E., 1997. Detection of depar-
tures from additivity in mixtures of many chemicals with a threshold model. J.
Agric. Biol. Environ. Stat. 2, 198–211.
Gessner, P.K., 1995. Isobolographic analysis of interactions: an update on applica-
tions and utility. Toxicology 105, 161–179.
Graves, C.G., Matanoski, G.M., Tardiff, R.G., 2001. Weight of evidence for an
association between adverse reproductive and developmental effects and expo-
sure to disinfection by-products: a critical review. Regul. Toxicol. Pharm. 34,
103–124.
Groten, J.P., Schoen, E.D., Feron, V.J., 1996. Use of factorial designs in combination
toxicity studies. Food Chem. Toxicol. 34, 1083–1089.
Hertzberg, R.C., Rice, G.E., Teuschler, L.K., 1999. Methods for health risk assessment
of combustion mixtures. In: Roberts, S., Teaf, C., Bean, J. (Eds.), Hazardous Waste
Incineration: Evaluating the Human Health and Environmental Risks. CRC Press,
Boca Raton, pp. 105–148.
Hertzberg, R.C., Teuschler, L.K., 2002. Evaluating quantitative formulas for
dose–response assessment of chemical mixtures. Environ. Health Perspect. 110
(Suppl. 6), 965–970.
Hochberg, Y., Benjamini, Y., 1990. More powerful procedures for multiple signifi-
cance testing. Stat. Med. 9, 811–818.
Krishnan, K., Haddad, S., Beliveau, M., Tardif, R., 2002. Physiological mod-
eling and extrapolation of pharmacokinetic interactions from binary to
more complex chemical mixtures. Environ. Health Perspect. 110 (Supp 6),
989–994.
Kutner, M., Neter, J., Nachtsheim, C., Li, W., 2005. Applied Linear Statistical Models.
McGraw-Hill/Irwin, Boston.
Lambert, J.C., Lipscomb, J.C., 2007. Mode of action as a determining factor in addi-
tivity models for chemical mixture risk assessment. Regul. Toxicol. Pharm. 49,
183–194.
Lyles, R.H., Poindexter, C., Evans, A., Brown, M., Cooper, C.R., 2008. Nonlinear
model-based estimates of IC50 for studies involving continuous therapeutic
dose–response data. Contemp. Clin. Trials 29, 878–886.
Meadows, S., Gennings, C., Carter, W.H., Bae, D.-S., 2002. Experimental designs for
mixtures of chemicals along fixed ratio rays. Environ. Health Perspect. 110,
979–983.
Meek, B., Boobis, A., Crofton, K.M., Heinemeyer, G., Van Raaij, M., Vickers, C., 2011.
Risk assessment of combined exposure to multiple chemicals: a WHO/IPCS
framework. Regul. Toxicol. Pharm. 60, S1–S14.
Moser, V.C., Padilla, S., Simmons, J.E., Haber, L.T., Hertzberg, R.C., 2012. Impact of
chemical proportions on the acute neurotoxicity of a mixture of seven carba-
mates in preweanling and adult rats. Toxicol. Sci. 129, 126–134.
Müller, A.K., Bosgra, S., Boon, P.E., Voet H. v. d. Nielsen, E., Ladefoged, O., 2009. Prob-
abilistic cumulative risk assessment of anti-androgenic pesticides in food. Food
Chem. Toxicol. 47, 2951–2962.
Mwanza, J.-C., Finley, D., Spivey, C.L., Graff, J.E., Herr, D.W., 2008. Depression of the
photic after discharge of flash evoked potentials by physostigmine, carbaryl and
propoxur, and the relationship to inhibition of brain cholinesterase. Neurotoxi-
cology 29, 87–100.
Mwanza, J.-C., Lyke, D.F., Hertzberg, R.C., Haber, L., Kohrman-Vincent, M., Li, R., Pan,
Y., Lyles, R.H., Simmons, J.E., MacMillan, D.K., Zehr, R.D., Swank, A.E., Herr, D.W.,
2012. Cholinesterase inhibition and depression of the photic after discharge of
flash evoked potentials following acute or repeated exposures to a mixture of
carbaryl and propoxur. Neurotoxicology 33, 332–346.
144 R.C. Hertzberg et al. / Toxicology 313 (2013) 134–144
Rice, G., Teuschler, L., Simmons, J.E., Hertzberg, R., 2005. Mixtures, toxicology and
risk assessment. In: Wexler, P. (Ed.), Encyclopedia of Toxicology. Academic Press,
San Diego, pp. 120–123.
Rodea-Palomares, I., Petre, A.L., Boltes, K., Leganes, F., Perdigon-Melon, J.A., Rosal,
R., Fernandez-Pinas, F., 2010. Application of the combination index (CI)-
isobologram equation to study the toxicological interactions of lipid regulators
in two aquatic bioluminescent organisms. Water Res. 44, 427–438.
Saffron, C.M., Park, J.-H., Dale, B.E., Voice, T.C., 2006. Kinetics of contaminant desorp-
tion from soil: comparison of model formulations using the Akaike Information
Criterion. Environ. Sci. Technol. 40, 7662–7667.
SAS Institute Inc., 2008. SAS/STAT 9.2 User’s Guide. SAS Institute Inc., Cary, NC.
SAS Institute Inc., 2009. JMP Release 8 Statistics and Graphics Guide, 2nd edition.
SAS Institute Inc., Cary, NC.
Stork, L.G., Gennings, C., Carter, W.H., Teuschler, L.K., Carney, E.W., 2008. Empiri-
cal evaluation of sufficient similarity in dose–response for environmental risk
assessment of chemical mixtures. J. Agric. Biol. Environ. Stat. 13, 313.
Sühnel, J., 1998. Parallel dose–response curves in combination experiments. Bull.
Math. Biol. 60, 197–213.
Tallarida, R.J., 2006. An overview of drug combination analysis with isobolograms.
J. Pharmacol. Exp. Ther. 319, 1–7.
U.S. EPA, 1986. Guidelines for the Health Risk Assessment of Chemical Mixtures. Risk
Assessment Forum, Washington, DC, p. 38.
U.S. EPA, 1989a. Interim Procedures for Estimating Risks Associated with Expo-
sures to Mixtures of Chlorinated Dibenzo-p-Dioxins and -Dibenzofurans
(CDDs and CDFs) and 1989 Update. Risk Assessment Forum, Washington,
DC.
U.S. EPA, 1989b. Risk Assessment Guidance for Superfund. Vol. 1. Human Health
Evaluation Manual (Part A). Office of Solid Waste and Emergency Response,
Washington, DC.
U.S. EPA, 2000a. Benchmark Dose Technical Guidance Document External Review
Draft. Risk Assessment Forum, Washington, DC.
U.S. EPA, 2000b. Supplementary Guidance for Conducting Health Risk Assessment
of Chemical Mixtures. Risk Assessment Forum, Washington, DC.
U.S. EPA, 2002a. Guidance on Cumulative Risk Assessment of Pesticide Chemicals
That Have a Common Mechanism of Toxicity. Office of Pesticide Programs,
Washington, DC.
U.S. EPA, 2002b. A Review of the Reference Dose and Reference Concentration Pro-
cesses. U.S. EPA Risk Assessment Forum, Washington, DC.
U.S. EPA, 2003. Developing Relative Potency Factors for Pesticide Mixtures: Biosta-
tistical Analyses of Joint Dose–Response. Office of Research and Development,
National Center for Environmental Assessment, Cincinnati, OH, p. 115.
U.S. EPA, 2005. Guidelines for Carcinogen Risk Assessment. Risk Assessment Forum,
Washington, DC, p. 166.
U.S. EPA, 2007. Revised N-Methyl Carbamate Cumulative Risk Assessment. Office of
Pesticide Programs, Washington, DC, p. 277.
U.S. EPA, 2010. Recommended Toxicity Equivalence Factors (TEFs) for Human
Health Risk Assessments of 2,3,7,8-Tetrachlorodibenzo-p-Dioxin and Dioxin-
Like Compounds. Office of Research and Development, Risk Assessment Forum,
Washington, DC.
Van den Berg, M., Birnbaum, L., Bosveld, T., Brunstrom, B., et al., 1998. Toxic equiv-
alency factors (TEFs) for PCBs, PCDDs, PCDFs for humans and wildlife. Environ.
Health Perspect. 106, 775–792.
Van den Berg, M., Birnbaum, L.S., Denison, M., De Vito, M., Farland, W., Feeley, M.,
Fiedler, H., Hakansson, H., Hanberg, A., Haws, L., Rose, M., Safe, S., Schrenk, D.,
Tohyama, C., Tritscher, A., Tuomisto, J., Tysklind, M., Walker, N., Peterson, R.E.,
2006. The 2005 world health organization reevaluation of human and mam-
malian toxic equivalency factors for dioxins and dioxin-like compounds. Toxicol.
Sci. 93, 223–241.