Você está na página 1de 13

1492 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 63, NO.

7, JULY 2016

Robust PBPK/PD-Based Model Predictive


Control of Blood Glucose
Stephan Schaller∗ , Jörg Lippert, Lukas Schaupp, Thomas R. Pieber, Andreas Schuppert, and Thomas Eissing

Abstract—Goal: Automated glucose control (AGC) has not yet million patients worldwide [5]. Type-2 diabetes (T2DM) ac-
reached the point where it can be applied clinically [3]. Chal- counts for 85% to 95% of all diabetes. It is caused by a defect
lenges are accuracy of subcutaneous (SC) glucose sensors, phys- in the body’s autonomous regulation of blood glucose, which
iological lag times, and both inter- and intraindividual variability.
To address above issues, we developed a novel scheme for MPC is normally managed by complex interactions among neuronal,
that can be applied to AGC. Results: An individualizable generic hormonal, and metabolic-signaling networks. Type-1 diabetes
whole-body physiology-based pharmacokinetic and dynamics (T1DM), although less common, has a high prevalence in chil-
(PBPK/PD) model of the glucose, insulin, and glucagon metabolism dren and adolescents and is caused by an autoimmune reaction
has been used as the predictive kernel. The high level of mecha- damaging the insulin producing beta cells of the pancreas. Cur-
nistic detail represented by the model takes full advantage of the
potential of MPC and may make long-term prediction possible as rently, T1DM and severe cases of T2DM can only be controlled
it captures at least some relevant sources of variability [4]. Robust- by the affected individual, through constant vigilance and man-
ness against uncertainties was increased by a control cascade rely- ual insulin therapy.
ing on proportional-integrative derivative-based offset control. The Effective automated control of blood-glucose levels could
performance of this AGC scheme was evaluated in silico and retro-
theoretically reduce the burden of manual therapy and improve
spectively using data from clinical trials. This analysis revealed that
our approach handles sensor noise with a MARD of 10%–14%, and the risk profiles of most patients with T1DM. Additional ben-
model uncertainties and disturbances. Conclusion: The results sug- efits would include improved quality of life and reduced costs
gest that PBPK/PD models are well suited for MPC in a glucose con- [6]. An integrated system that combines a subcutaneously (SC)
trol setting, and that their predictive power in combination with the implanted continuous glucose monitor (CGM) and an insulin in-
integrated database-driven (a priori individualizable) model frame-
fusion pump (IIP) are often referred to as an “artificial pancreas”
work will help overcome current challenges in the development of
AGC systems. Significance: This study provides a new, generic, and (AP).
robust mechanistic approach to AGC using a PBPK platform with Although AP systems have been under development for more
extensive a priori (database) knowledge for individualization. than 50 years, they are still not capable of orchestrating every-
Index Terms—Artificial pancreas (AC), decision support, dia- day control of blood glucose. This is due to both technical and
betes, glucose metabolism, patient variability, physiology-based system-inherent hurdles, including: 1) insufficient accuracy of
pharmacokinetics and pharamcodynamics (PBPK/PD), predictive CGM devices, 2) the lag times of SC glucose measurements
control, predictive models. (SCGM) when changes in blood-glucose levels are rapid, 3 the
lag time for the onset of insulin action after SC insulin adminis-
I. INTRODUCTION
tration, and 4) the lingering/tailing action of insulin action after
IABETES mellitus is a chronic disease that affects a
D continuously growing population of currently over 380
SC administration [3], [7], [8].
Recent improvements in the accuracy of SC CGM devices,
IIPs, and safety systems have enhanced conditions for develop-
Manuscript received May 20, 2014; revised April 7, 2015, June 7, 2015, and ing a fully integrated AP system [8], [9]. Also, the superiority
September 15, 2015; accepted October 27, 2015. Date of publication November of AGC involving both SC measurement of glucose and SC
2, 2015; date of current version June 16, 2016. The work of S. Schaller, J.
Lippert, L. Schaupp, T. R. Pieber, and T. Eissing was supported in part by infusion of insulin (the SC–SC route) over manual control of
the European Commission under Grant Agreement 248590 (FP7 Integrated glucose levels has been demonstrated in clinical trials [10].
Project Reaction; Remote Accessibility to Diabetes Management and Therapy Various strategies for controlling blood glucose, ranging from
in Operational Healthcare Networks). Asterisk indicates corresponding author.
* S. Schaller is with the Department of Computational Systems Biology, proportional-integrated derivative (PID) [11]–[17] to complex
Bayer Technology Services GmbH, Leverkusen 51368, Germany, and also with fuzzy logic methods [18], [19], have been used in designs for the
the Aachen Institute for Advanced Study in Computational Engineering Sci- AP. Nevertheless, physiologic lag times remain a core problem
ences, RWTH Aachen, Aachen 52062, Germany (e-mail: stephan.schaller@
rwth-aachen.de). in reactive (i.e., PID) feedback-control solutions [20]. These lag
J. Lippert is with the Department of Computational Systems Biology, Bayer times are better addressed using predictive control solutions,
Technology Services GmbH, and also with Clinical Pharmacometrics, Bayer and model predictive control (MPC) is the most widely used
Pharma AG.
L. Schaupp and T. R. Pieber are with the Department of Internal Medicine, among these [21]–[27].
Medical University of Graz Key components of such MPC approaches are mathematical
A. Schuppert is with the Department of Computational Systems Biology, models of the biological processes relevant to glucose control.
Bayer Technology Services GmbH, and also with the Aachen Institute for
Advanced Study in Computational Engineering Sciences, RWTH Aachen Early models only vaguely reflected the physiological structures
T. Eissing is with the Department of Computational Systems Biology, Bayer involved, in spite of the fact that these are vital to comprehend-
Technology Services GmbH ing the influence of lag times. Research to improve these models
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org. focused on more physiology-based model concepts, as model-
Digital Object Identifier 10.1109/TBME.2015.2497273 based glucose control requires reliable models with longer term
0018-9294 © 2015 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution
requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
SCHALLER∗ et al.: ROBUST PBPK/PD-BASED MODEL PREDICTIVE CONTROL OF BLOOD GLUCOSE 1493

predictive capabilities. Recently, the UVa/Padova simulator, A. Individualized Physiology-Based Models and Software
which is based on a model by Dalla Man et al. [28], proved suffi-
PBPK models describe the mechanisms underlying the ab-
ciently effective to receive approval from the U.S. Federal Drug sorption, distribution, metabolism, and excretion of a substance
Administration as a replacement for animal testing of glucose
within the body in great detail. Such models are based to a large
controllers [29]. In addition, the Cambridge Model developed
extent on prior information. This information is taken from col-
by Hovorka et al. [30] for closed-loop glucose control incorpo- lections of anatomical and physiological data, and calculations
rates physiological aspects and represents the current state of
are made based on the drug’s physiochemical properties and
the art in glucose modeling as it applies to model-based glucose
their effects on various organs and tissues, taking into account
control. However, even these more physiological models fail to an organism’s anatomy and physiology and its changes with
scale key parameters such as blood flow, organ volume, and sub-
age, as well as its weight, height, gender, and race (e.g., age de-
compartment size to an individual’s anthropologic properties. A pendence of organ weights, blood flows). Also, the information
fully physiological modeling framework such as that described about how active processes (e.g., metabolic rates, transport, and
in [4] allows for a priori individualization of these properties of
clearance rates) scale with relation to these properties is inte-
the model with respect to age, weight, height, gender, and race. grated. Generic adult PK prediction models can automatically
We have developed a control approach that combines, for be parameterized based on the basic physicochemical param-
the first time, a detailed a priori individualizable generic
eters of a substance, and can then be used to simulate drug-
whole-body physiology-based pharmacokinetic and dynamics concentration profiles in various organs and tissues [31]. Once
(PBPK/PD) model of the glucose-insulin metabolism (GIM) an adult PBPK model has been established, it can be extrapo-
[4] with a robust MPC algorithm for AGC in a post hoc in sil-
lated to individuals—be it adults, children, or elderly—outside
ico study. Based on this accurate model for predicting the core of the selected cohort [32]. The PBPK models for glucose, in-
dynamics of blood-glucose levels (i.e., in the undisturbed state)
sulin, and glucagon [4] were established in the PK-Sim 5.1,
in an individual and adapting over time by adding continuously
and coupled in the MoBi 3.1, commercial software packages
gathered patient data, the MPC computes an optimal feedfor- for PBPK modeling and molecular biology modeling, respec-
ward control input. To increase closed-loop stability as well as
tively [1], [31]. Model identification, model parameterization,
robustness against both disturbances and model uncertainties,
and the development of control algorithms were conducted us-
we use a PID-based feedback controller that compensates for ing the MoBi Toolbox for MATLAB 3 and MATLAB 2012 (The
prediction errors (offset).
MathWorks, Natick, MA, USA).
Here we test, in silico, the performance of the control algo-
rithm in a simulated control trial, in the face of measurement
error, model uncertainty, and disturbances.
B. Generating Virtual Patients
The model used to generate the in silico subjects in the sim-
ulation environment was previously described in detail and val-
II. MATERIALS AND METHODS
idated, including how the parameters for the generic model
The glucose control framework developed here allows both were set [4]. This model was fitted to a dataset collected for
in silico evaluation of controller concepts and control of blood 12 T1DM patients studied in a 2-phase randomized crossover
glucose in T1DM patients in a clinical setting. The interac- trial (2PRCT, from the project Advanced Insulin Infusion using
tions among the components of the integrated system are based a Control Loop [33], and the effectiveness of continuous SC
on the modular approach described in [8]. The three interact- insulin infusion was compared to that of an MPC system (Cam-
ing layers work on different timescales (see Appendix Fig. 5). bridge Algorithm [33])), generating 12 virtual patients. In the
The outer layer is on a slow timescale and adjusts the param- absence of SC glucagon infusion and mean meal-independent
eters of the inner fast layer. In the system described here, the parameters for oral glucose absorption, the algorithm identifies
outer layer is represented by model adaptation, i.e., “offline op- six parameters (first six in Table II, Appendix) for model indi-
timization,” using glucose measurement data to adjust the model vidualization (also in [4], supporting information Table S3) from
kernel of the MPC (middle layer). The MPC, i.e., “online sim- the blood-glucose measurement data, and then fits the generic
ulation and control,” in turn, calculates insulin delivery based model to the respective patient data to generate a virtual patient.
on the prediction of blood-glucose levels and meal information. Thus, the in silico subjects used for the in silico validation and
The middle layer is further restricted by the inner layer, i.e., the MPC algorithm presented here use the exact same model as
the robustness layer, which is comprised of the offset-controller presented in [4]. The models may only differ by their physio-
(PID). It constantly adjusts 1) the blood-glucose target value logic values (e.g., physiology-dependent (calculated) properties
for the MPC and 2) the insulin doses calculated by the MPC. like blood volume) and the six identified parameters used for
The insulin adjustments are based on the latest CGM data (and model individualizations (the MPC is initialized with mean pop-
the predictive error), incorporating algorithms for pump shutoff, ulation values for these six parameters).
and insulin-on-board constraints. To parameterize the model, the fmincon function of the MAT-
Once the integrated system is in place, various increasingly LAB optimization toolbox was used with a quadratic penalty
complex configurations of an AP system become possible. function.
1494 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 63, NO. 7, JULY 2016

C. Design of In Silico Clinical Trial best-fit parameter set was used to initiate the MPC at 7:30 p.m.,
The in silico trials evaluating the integrated algorithms were with the optimal insulin dose, uM PC (t) , calculated based on
carried out using the protocol described above for the 2PRCT. predictions of the model (see Appendix Fig. 5).
The AGC continuously improves the model kernel used by
The control inputs used were constant insulin infusions over
the MPC over the course of the trial, as more data becomes
15 min, corresponding to the sampling time for blood-glucose
values and CGM readouts (note that although CGM readouts available. The individual parameter set is constantly optimized
based on all measurement data collected up to that point. A
were taken every 5 min, due to the need for manual handling
“growing horizon estimatior” (see Appendix Fig. 7) is used to
every third readout was deemed sufficient). In the 2PRCT proto-
col, the subjects were admitted to the clinical research center at this end.
However, this feature was deactivated for the purposes of the
the Medical University of Graz (MUG) at 2 p.m. Prior to the first
meal on the day of arrival, from 2 p.m. until 5 p.m. (t = 0–180 in silico analysis described here. The evaluation of controller
min), the study started with a glucose clamp and subjects were performance in the context of suboptimal fit of the model, i.e.,
in the presence of model uncertainty, required that the model
adjusted to 100 mg/dl blood glucose and received basal insulin
via a pump until closed-loop control was initiated following the was not adapted following the initial matching. This initially
first meal. Over the course of the trial, the subjects (N = 12) re- matched model is suboptimal in the sense that the match from
the initial population is based on a coarse grid search over the
ceived four meals. Dinner (the first meal) was provided at 6 p.m.
(t = 240 min), and closed-loop glucose control was initiated at range of the parameters, and does not necessarily reflect the
7:30 p.m. (t = 330 min, end of observation phase) and contin- optimal parameters for the individual.
The algorithm, which was run on a standard laptop PC, was
ued until 10 p.m. the following day. On the second day, the
subjects received meals at 7 a.m. (t = 1020 min, breakfast), 12 implemented in MATLAB and controlled via a graphical user
interface (GUI using MATLAB GUIDE). For model adapta-
p.m. (t = 1320 min, lunch), and 6 p.m. (t = 1680 min, dinner).
tion and input optimization, we use the functions fmincon and
Meal information was transferred to the controller at the time of
meal onset, such that prandial insulin was not infused prior to fminbnd, respectively, of the MATLAB optimization toolbox.
The parameters used to configure the controller are provided in
the meals.
Table III (Appendix).
The target range for insulin was defined as 70—180-mg/dl 3-h
postprandial and 70–140 mg/dl at all other times. “OVERALL” 1) Model Predictive Control: MPC is a form of control in
which the “current” control action is determined by solving,
refers to the duration of the trial (7:30 p.m., Day 1 to 10 p.m., Day
2), “DAYTIME” to the time from the end of breakfast to the end at each sampling instant, a finite-horizon open-loop optimal
of trial (8 a.m., Day 2 to 10 p.m., Day 2), and “OVERNIGHT” control problem P (x, k). A schematic of the general concept
is shown in Appendix Fig. 7. Optimization of the cost function
to the nighttime period during the trial (10 p.m., Day 1 to 8 a.m.,
Day 2). Throughout the trial, the control input was calculated V (x, k, u) at time k = tk , subject to any imposed control, state,
every 15 min, when the blood glucose was sampled. and terminal constraints, yields an optimal control sequence u of
N piecewise constant control inputs, and the first control in this
sequence at time k = tk is applied to the controlled system [2],
D. Control Algorithm [34]. This produces a closed-loop control strategy, solving an
The effectiveness of model-based glucose control algorithms open-loop optimization problem. The general control problem
depends on the model kernel being tuned to the system it con- is defined as
trols. In the case of blood-glucose control, the model is tuned for
each patient using experimental data gathered over time [12]. P (x, k) = min V (x, k, u) (1)
u
The study described here tests our robust MPC algorithm in
an in silico trial, using the protocol described above. During this
with the discretized point in time tk , and the general cost func-
trial, glucose measurements from simulated virtual patients were
tion
used to individualize the kernel on which the MPC operates.
The algorithm first establishes a mean model of GIM, and this k
+N
is then scaled based on the physiologic properties (age, weight, V (x, k, u) = l (xi , ui ) + F (xk +N ) (2)
height, gender, and race) of the enrolled patient. Model individu- i=k
alization is started with the first blood-glucose measurement at 2
p.m. on Day 1. As the efficient use of time for model individual- with stage cost l, the optimized input sequence u =
ization (finding a timely solution to the optimization problem) is {uk , uk +1 , . . . , uk +N −1 }, system state xi [corresponds to ei-
paramount, a virtual population was generated a priori based on ther plasma (IV) glucose or SC glucose], and terminal cost
the known physiologic properties (e.g., age, weight, and height) F (xk +N ).
of the patient, but with variations the individual parameter set However, this is a multidimensional optimal control prob-
(first six parameters in Table II, Appendix) taken from distri- lem that uses a sequence of piecewise constant inputs u =
butions precalculated based on known model parameterizations {uk , uk +1 , . . . , uk +N −1 }, and, thus, solving it may become
[4]. The resulting virtual population was then screened for the computationally demanding. A common approach to save com-
best fit to data collected during the clamp phase (observation putational time and increasing the speed of convergence of the
phase), i.e., from 2 p.m. until 7:30 p.m. on the first day. The optimization problem is to restrict optimization to the first input
SCHALLER∗ et al.: ROBUST PBPK/PD-BASED MODEL PREDICTIVE CONTROL OF BLOOD GLUCOSE 1495

of the sequence. The cost function is then defined as 


k 
k
−W p (k −j )
≈ Kp e ej + Ki ej (5)
k
+N j =0 j =0
V (x, k, u) = l (xi ) + uk (3)
i=k
with the controller gains K and the forgetting factor Wp , in the
case of the proportional component, where Wp determines how
yielding the feedback control law rapidly past values fade from memory. We chose to use an expo-
nentially decaying fading function. The derivative component
u0 = u0k . (4) was not included in calculating ΔTV, as this would increase the
influence of sensor noise. The integral component was included
to accommodate for the effects of model nonlinearity, ensuring
2) Offset-Control/Error Compensation (PID): To increase
that the set-point (dynamic target value) error would be zero.
the robustness of the chosen MPC AGC approach, we use
A drawback of dynamic shifting of the target value is that it
a PID-based offset controller here. The development of such
could lead to a nonlinear behavior of some components of the
a controller is based on the basic concepts of feedback con-
GIM model. This could “distort” the estimate of the insulin dose
trol [35], [36]. In general, a PID controller is based on three
required, due to nonlinearities inherent to the model. However,
characteristics of the output tracking error e (t) in relation to
individualization of the model kernel ultimately reduces the
a (constant) target value: the weighted current absolute error
required shift, diminishing the influences of nonlinearities. The
(proportional component, P); the weighted sum of all past er-
reason for using only the proportional and integral components
rors (integral component, I); and the weighted change in error
in calculating ΔTV is that this value should compensate solely
compared to the error at the last sampling point (derivative com-
for the overall drift in predictive error; short-term deviations and
ponent, D). These three components are summed in calculating
disturbances are corrected by the input correction, in particular
the control input (see Appendix Fig. 5).
by its derivative component.
The offset controller is introduced for two reasons: to compen-
The second component, Δu (t), of the offset controller cor-
sate for inaccuracies of the model fit and to reject disturbances.
rects the input MPC dose calculations, uM PC (t). The calcula-
Even a detailed PBPK/PD model does not fully account for all
tion of this error also takes into account the shift in target value
dynamics that may cause an observed change in blood-glucose
for the MPC controller, correcting only any remaining devia-
levels. The offset controller filters the residual error of the model
tions. This prediction error of ê (t) = ΔTV − e (t) takes into
predictions made by the MPC (see Appendix Fig. 5). Thus, the
consideration the dynamic shift ΔTV of the MPC such that the
tracking error e (t) corresponds to a prediction error and is cal-
input correction Δu (t) approaches zero once the target value
culated as deviation of the simulated blood-glucose prediction,
is shifted by the value of the prediction error (e (t) = ΔTV). A
x (t), from the actual blood-glucose measurements, y (t) (for
generalized formulation of the FMPD control for the calculation
more details on simulated concentrations see Appendix A).
of the input correction Δu (t) is
Based on these values, the offset-controller calculates two cor-
rections via a feedback loop based on the prediction error e (t). 
k 
k
The first is the set-point correction ΔTV (i.e., a shift in target Δuk = Kp e−W p (k −j ) êj + Ki êj
value) of the MPC. The second is the control input correction, j =0 j =0
Δu (t) , which corrects the insulin infusion recommendations
uM PC (t) made by the MPC. 
k
+Kd e−W d (k −j ) (êj − êj −1 ) . (6)
To increase the robustness of offset control against sensor
j =0
noise and disturbances, the fading memory principle from fad-
ing memory proportional derivative control (FMPD) [12–15] The input correction was tuned to respond more rapidly to
was adapted for the P and D components. The reason a dynamic changes in e (t). The tuning of the forgetting factors Wp and
target value correction, ΔTV, is used is that in spite of model Wd within a real control setting is always a tradeoff between
inaccuracy (i.e., a model drift), most of the insulin input will be susceptibility to sensor error/noise and an increase in time delay
calculated by the MPC rather than by the feedback controller. or “sluggishness” of the controller. As the derivative component
This is to ensure timely control action, as the MPC is a (feedfor- represents the rapid first-response characteristic of a beta cell, a
ward) predictive controller, and shifting influence to the reactive larger factor (and thus faster memory decay) was chosen.
FMPD feedback component (i.e., the input correction Δu (t)) Both MPC (penalty function) and offset control are sub-
as the determinant of input would lead to a reduction in overall ject to additional safety constraints and parameterizations, as
response time. documented in the Appendix.
ΔTV is calculated from the relative prediction error Although our results indicate that the MPC control algorithm
e (t) = x (t) − y (t), such that DynamicTV = TV + ΔTV is robust even in the context of significant prediction error, a key
and ΔTV (t = 0) = 0. In the context of fading memory, this feature is that when the model kernel is used in a clinical-trial
gives setting it will be continuously updated as new measurements
become available. A standard optimization routine (fmincon,
 t  t
−W p (t−τ ) MATLAB) was chosen for optimizing the individual parameter
ΔTV = Kp e e (τ ) dτ + Ki e (τ ) dτ set [4] in this context.
0 0
1496 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 63, NO. 7, JULY 2016

Fig. 1. Effects of model uncertainty on glucose concentration, insulin in- Fig. 2. Effects of disturbances on glucose concentration, insulin infusion, and
fusion, and insulin concentration. Representative in silico run, carried out for insulin concentration. Representative in silico run, carried out for Subject 06.
Subject 08. In plots of glucose (top) and insulin (bottom) concentrations (C.), red Symbols and conditions of experiment are as described for Fig. 1.
triangles represent (simulated) measurements from the in silico individual, and
blue curves represent predictions made by the model. In plot of insulin infusion
(Inf.; centre), blue bars represent doses calculated by the MPC, with green being
the component added by the offset controller, and red the component subtracted
measurement error for such measurements is <2% [37]) as it
by the offset controller. The target value (TV) for the blood-glucose concentra- will be used for IV measurments (for safety reasons) in the first
tion was 110 mg/dl. DynamicTV = TV + ΔTV. Light and darker purple clinical trial at MUG.
backgrounds indicate an extended (70–180 mg/dl and tight (70–140 mg/dl) glu-
cose target range. The black broken vertical line (t = 330 min) represents start
In the second scenario, simulated SCGMs from CGM devices
of control. Only carbohydrates (meals; IV and oral glucose) and insulin during were used to simulate, e.g., the Dexcom G4 Platinum CGM de-
clamping (prior to t = 330 min) were administered based on the original pro- vice (with mean absolute relative error (MARE) of 10.8 ± 9.9%
tocol of the clinical trial. Information on carbohydrate intake was passed to the
controller upon start of intake, except for extra orange juice (12 g) on day 2 (4
[38]), as it will be used in the second clinical trial to evaluate the
p.m., t = 1560 min). The optimal insulin dose was calculated by the controller. performance in a state-of-the-art setting. The simulated noise
More details on control performance are given in Tables I and II and Fig. 4. for the CGM sensor is documented in the appendix (15).
In addition, a retrospective validation of the controller was
conducted offline. Here, the same protocol was used, but actual
E. Method for In Silico Controller Evaluation measurements from the 2PRCT trial were used instead of those
from the in silico patients.
We have tested our control system against 12 in silico in-
dividuals, with the model parameterized based on the 2PRCT
data. We evaluated the robustness of the controller in the setting III. RESULTS
of three potential sources of variability: 1) model uncertainties, In silico evaluation is used to verify and validate closed-loop
using a suboptimal fit of the internal MPC model kernel; 2) dis- control algorithms, and is a prerequisite for in vivo testing of
turbances, e.g., the unanticipated intake of carbohydrates; and AP systems during clinical trials [29]. This approach provides
3) sensor error, e.g., simulating sensor noise of both intravenous valuable information about controller performance (with respect
(IV) and SCGMs. to both safety and limitations) and is useful in optimizing the
For the MPC algorithm, a prediction horizon of N = 240 clinical study design.
min (6 h) was chosen. The dose of insulin, which was infused at
constant rate over a 15 min period, was calculated by solving the
nonlinear constrained optimization problem described in (1). A. In Silico Evaluation of Online Controller (IV Measurement
The in silico trials for evaluating the integrated algorithms of Glucose)
were carried out using the same protocol as in the 2PRCT. 1) Effects of Model Uncertainties: The glucose control al-
As mentioned in Section II-D2, we did not adapt the model gorithm relies on an accurate internal model of the controlled
following the initial matching with the starting population. subject, because inaccurate parameterizations can lead to mis-
Given that this study was carried out in preparation for in calculation of insulin doses by the MPC component. To assess
vivo clinical trials, we evaluated two scenarios—one in which the implications of model uncertainty, we deactivated model
IV measurement of glucose was simulated, and another in which adaptation and deliberately conducted our in silico evaluation
CGM (SC) measurement of glucose was simulated. based on the initial matches of model parameterization that
The first scenario is for controller evaluation using simulated took place during the clamp phase. Also, the parameters that
IV measurements of glucose with a mean standard deviation affect meal absorption and result in the highest interoccasion
of 5%. This case was used to simulate the Super GL 2 instru- variability within the model (caloric content, fraction solid, and
ment (Dr. Müller Gerätebau GmbH, Freital, Germany; expected meal volume [4]) were changed at each meal, with the maximal
SCHALLER∗ et al.: ROBUST PBPK/PD-BASED MODEL PREDICTIVE CONTROL OF BLOOD GLUCOSE 1497

Fig. 3. In silico evaluation of controller across the virtual cohort by control


variability grid analysis (CVGA, 100%tile). The CVGA displays the minimal
(x-axis) to maximal (y-axis) glucose values (range of control) for all (simulated)
measurements for each individual. Left: Variability in control for duration of the
trial (7:30 p.m., Day 1 to 10 p.m., Day 2). Right: Variability in control overnight
(10 p.m., Day 1 to 8 a.m., Day 2). More details on control performance are given Fig. 5. Schematic of the work and information flow of an integrated system
in Tables I and II and Fig. 4. as applied in a clinical environment to provide continuous closed-loop glucose
control. The PBPK/PD model kernel is initialized with patient data (green;
physiological parameters, e.g., weight, height, gender). Blood glucose is mea-
TABLE I sured frequently (blue) the values are stored, and the most recent are delivered
In Silico CONTROL TRIAL STATISTICS (IV GLUCOSE TRIAL) to the controller. The process works on both extended and short timeframes.
The extended timeframe applies to the offline “optimization/model adaptation”
(blue), which is based on the full measurement data history (outer layer). The
Target Overall Daytime Overnight short timeframe applies to online calculation (red) of the optimal insulin dose
(u (t)) based on recent glucose measurements (middle layer). The MPC with
t in target [%] 84 (72 to 92) 79 (67 to 90) 94 (83 to 100) PID offset control is embedded in the online simulation and control workflow.
t below 52 mg/dl [%] 0∗ (0 to 0) 0 (0 to 0) 0 (0 to 0) Based on the tracking error (e (t)), the weighted actual absolute error (P), the
t below 70 mg/dl [%] 0∗ (0 to 0) 0 (0 to 0) 0 (0 to 0) weighted sum of all past errors (I), and the weighted change in error relative
t above 140 mg/dl [%] 26 (19 to 41) 34 (26 to 46) 6 (0 to 17) to that at the previous sampling point (D) are summarized in the calculating
Mean glucose [mg/dl] 128 132 117 the change in target value for the MPC controller (ΔTV) and the correction in
Glucose stdv [mg/dl] 28 32 11 insulin dose, Δu (t) (innermost layer).
Max. glucose range amplitude [mg/dl] 115 (95 to 134) 115 (95 to 134) 43 (32 to 65)

Targets defined in Section II.


*
Values are given as mean values, and the 5%–95% tile in parentheses. For details, see
relative change of the uniform distribution being 30%. Fig. 1
Table II. shows the in silico trial for Subject 08.
Values for zero are absolute, not rounded. Especially during the phase immediately following clamping,
omitting the adaptation step leads to a greater likelihood of
model uncertainties. As the offset controller takes a short time
to adjust for prediction errors, the MPC starts (at t = 330 min)
with a high target value (150 mg/dl), which is gradually reduced
toward the desired target level (110 mg/dl).
For the virtual patient used as an example in Fig. 1, the pre-
dicted half-life for insulin was underestimated (see Table II; SC
insulin binding factor Qfac was underestimated resulting in a
faster release of insulin from the SC injection depot). Thus, the
predicted insulin levels fell more rapidly than the actual lev-
els, and, consequently, the predicted glucose levels rose more
rapidly than the measured levels (e.g., at t = 400 min). This
caused the MPC to supplement insulin prematurely. However,
Fig. 4. Summary of sensor error-dependent performance of control. Each
large circle represents a trial using a virtual cohort (n = 12) to evaluate the
given that the insulin level in the in silico patient was sufficient
effects of error (expressed as MARD in%) and the delay (up to 30 min) in and the glucose level was already below target, the offset con-
SC measurements. The left part (x-axis points 3–17) represents in silico trials troller corrected for (reduced) the premature dose calculated by
with artificially generated CGM sensor noise, and the right part (x-axis point 5)
represents the in silico trial with simulated IV measurements (no delay). Each
the MPC, i.e., the next time glucose rose above the target level
chart shows (as% of total time) the time at target concentration, the time above (t > 500 min), the offset controller did not reduce the insulin
140 mg/dl (hyperglycemia), the time below 70 mg/dl (mild hypoglycaemia), dose. This reactive control behavior slightly restricts the predic-
and the time below 50 mg/dl (hypoglycaemia). The colors indicate whether
control performance is good (green), suboptimal (orange), or critical (red).
tive capabilities of the MPC and causes the glucose trajectories
Specifically, for each sector, the colors reflect the following: time at target con- to oscillate. Especially in the context of high insulin doses (e.g.,
centration: green >80%, orange 60%–80%, red <60%; time above 140 mg/dl: prandial insulin), an underestimate of the half-life of insulin ac-
green <30%, orange 30%–80%, red >50%; time below 70 mg/dl: green <1%,
orange 1%–3%, red >3%; time below 50 mg/dl: green = 0% (absolute value,
tion (i.e., overestimation of insulin clearance) can have serious
not rounded), orange >0% but <1%; red >1%. All values marked as orange consequences; in this example, the large insulin dose at break-
are acceptable under controlled conditions. fast at time t = 1100 min resulted in postprandial low glucose
1498 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 63, NO. 7, JULY 2016

TABLE II
INDIVIDUAL PARAMETERS AND PERFORMANCE INDICATORS FOR THE IV CONTROL RUN

Individual parameters for in-silico subjects (S1–12) and the corresponding identified values used by the MPC (in parentheses)

Parameter Unit S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12


SI a [−] 1.3 (1.2) 1.8 (1.7) 2.6 (1.7) 1.1 (0.8) 1.5 (1.2) 1.3 (1.1) 1 (1) 2.2 (1.7) 1.5 (1.7) 1.3 (1.4) 1.5 (1.2) 1.7 (1.4)
SN b [−] 0.4 (0.5) 2 (0.5) 2 (1) 2 (1) 1.7 (1) 1 (1) 1.1 (0.5) 1 (0.5) 1.1 (1) 1.1 (1) 1.1 (1) 1.1 (1)
Qfa c c [IU/μmol] 1000 1000 3000 2000 1000 1000 2000 3000 2000 3000 3000 2000
(500) (500) (2000) (500) (500) (2000) (2000) (500) (2000) (1000) (2000) (500)
k SI C D d [1/min] 2E−6 2E−6 1E−5 1E−5 5E−6 1E−5 2E−6 8E−6 7E−6 5E−6 1E−5 9E−6
(1E−6) (7E−6) (1E−5) (7E−6) (7E−6) (7E−6) (7E−6) (7E−6) (1E−5) (7E−6) (1E−5) (7E−6)
G F R fIr a c e [−] 2 (1) 2 (1) 3 (1) 1 (1) 5 (1) 2 (1) 2 (1) 1 (1) 1 (1) 1 (1) 5 (1) 2 (1)
G F R fNr a c f [−] 15 (10) 5.5 (10) 10 (10) 10 (10) 10 (10) 10 (10) 10 (10) 10 (10) 10 (10) 10 (10) 10 (10) 10 (10)
G en der [−] Male Female Male Male Male Female Male Male Female Female Male Male
Age [years] 64 43 40 60 40 42 38 57 40 22 41 21
W eig ht [kg] 74 62 73 88 79 66 89 74 55 65 75 75
H eig ht [cm] 173 168 176 180 169 165 190 174 165 166 171 180
BM I [cm/m2 ] 24.73 21.97 23.57 27.16 27.66 24.24 24.65 24.44 20.20 23.59 25.65 23.15
Resulting individual mean and range of blood-glucose values for the IV control run
M ean [mg/dl] 123.3 127.3 126.5 123.3 123.9 137.6 123.0 125.3 132.9 124.6 129.2 123.6
M ax. [mg/dl] 200.8 218.3 220.2 189.2 181.9 205.6 193.5 207.8 212.5 207.6 213.8 186.5
M in . [mg/dl] 89.0 70.2 83.7 82.3 84.0 98.4 78.4 71.4 86.5 76.7 91.6 83.1
t in target∗ [%] 84.9 81.5 86.6 90.8 92.4 73.1 89.9 88.2 76.5 84.0 84.0 91.6

a
Insulin Sensitivity.
b
Glucagon Sensitivity.
c
Subcutaneous insulin binding factor.
d
Subcutaneous insulin degradation rate constant.
e
Renal insulin clearance factor.
f
Renal glucagon clearance factor.
*
No hypoglycemia, thus the remaining time was spent > 180 mg/dl.

levels at t = 1250 min as the decline of insulin action/levels the evaluation conservative). As our results show (see Figs. 1
was slower than that predicted by the MPC. However, over- and 2), noise at the level chosen for the simulation did not have
all the glucose trajectories stayed well within the target range, a significant adverse effect on the glucose control.
even after a glucose disturbance occurred at t = 1560 min (as To assess overall performance of the control system through-
elaborated in the next section). out the cohort, we performed control variability grid analysis on
2) Effects of Disturbances: Even with improvements in the the control results (CVGA, Fig. 3) [39]. This revealed that the
sensor and pump technologies, AGC continues to rely on the glucose trajectories were well within the normoglycemic range;
controller being able to cope with both inaccuracies in glucose episodes of hypoglycemia were absent.
sensing and delays in insulin action. These issues are particularly Also, the nighttime control (see Fig. 3, right panel) was com-
problematic when a system disturbance, e.g., an unannounced pletely within the target range, with the mean blood-glucose
meal, occurs and triggers a rise in glucose that is substantially level only slightly above target value. More details on control
faster than insulin can be absorbed and exert its activity [8]. performance are given in Tables I and II and Fig. 4.
Another type of disturbance that can trigger such problems is a
change in metabolism, e.g., as a consequence of exercise, stress,
or medical treatment.
For the purpose of evaluating the model’s ability to reject dis- B. In Silico Evaluation of an Online Controller (SC
turbances, an intake of one glass of orange juice (12 g glucose) Measurement of Glucose)
was triggered at 4 p.m. on day 2 (t = 1560 min). As shown for As a step forward and for the sake of compatibility with cur-
Subject 06 in Fig. 2, the resulting discrepancies between the rent state-of-the-art systems, the algorithm for glucose control
model predictions and measurements trigger both a rapid reac- in subjects with T1DM was adapted to work with SC mea-
tion by the offset controller and a slow and steady shift from surements of glucose from CGM devices (e.g., Dexcom G4
the dynamic target value. In spite of an overprediction of the Platinum, MARE of 10.8 ± 9.9% [38], with glucose readouts
half-life of insulin, the disturbance is managed well and does every 5 min). This integrated system has been thoroughly eval-
not result in a controller overreaction. uated in silico in an extensive robustness analysis that assesses
3) Effects of Error/Noise and Overall Performance: The con- measurement errors and potential time delays associated with
trol algorithm for glucose measurements was tested on measure- SC methods of measurement. The ultimate aim is to define ac-
ments with a MARE of 5%. Although the algorithm will first be curacy criteria for the required sensor system.
evaluated in a clinical setting in which measurement will be IV, For each of the 12 in silico patients, we tested 15 artificial
dosing will be SC and the expected MARE will be below 2% (a error/time-delay sets, producing a total of 180 trial runs. The
higher simulated MARE was chosen for the purpose of making error/time-delay sets were assembled based on the assumption
SCHALLER∗ et al.: ROBUST PBPK/PD-BASED MODEL PREDICTIVE CONTROL OF BLOOD GLUCOSE 1499

evaluated the effectiveness of this approach in controlling glu-


cose by carrying out a post hoc in silico study. The algorithm
was tested against model uncertainty using suboptimal model
fits, with the scenarios tested incorporating glucose-sensor noise
and unprogrammed carbohydrate disturbances. As input, the al-
gorithm requires only the individual’s physiologic properties
(height, weight, gender, and age) and the quantities of carbohy-
drates consumed. We found that the combination of a predictive
system (MPC) with a reactive system (FMPD) significantly in-
creases the controller’s robustness with respect to uncertainty.
All control insulin inputs are calculated and no additional pre-
meal insulin bolus is required. Overall, the controller performs
Fig. 6. General mechanistic principles of a PBPK organ representation (in PK- well. Although most systems currently available have been de-
Sim) with four subcompartments [intracellular (cell), interstitial (int), plasma veloped for the SC measurement of glucose, the controller de-
(pl) and blood cells (bc)] and the corresponding flow rates and transports. C:
concentrations, Q: flow rates, P ∗ SA: permeability surface area products, K: veloped here uses a novel PBPK/PD model kernel, and because
partition coefficient, V ma x , Km : Michaelis–Menten constants, for exam- its initial clinical testing will be done using measurements of
ple, transport or metabolization processes (figure taken from [1]. For explicit plasma glucose (for safety reasons), our in silico evaluation
representations in the glucose-insulin model please refer to [4]).
included this procedure.
Many algorithms have been developed for AGC [40]–[42].
Among these, MPC is currently favored for a number of reasons:
it is a predictive approach that can manage time delays inherent
to the system (SC glucose monitoring and SC insulin infusion);
it is model-based, which allows for straightforward personal-
ization using patient-specific parameterization [43]; and it is
optimization-based, making it possible to integrate performance
specifications by constraining states (i.e., penalizing for hypo-
glycaemia) and inputs (insulin or glucagon infusion).
The common approach of AGC using MPC is to adapt, in
addition to global model adaptation (time-invariant patient-
specific parameterization), a single input–output sensitive pa-
rameter per sampling time. This accounts for process variability
over time (intraindividual or intraday variability), and results in
Fig. 7. Schematic of the MPC. Solving the optimization problem P at time a discrete time-dependent profile for this parameter. Usually, a
k over the prediction horizont N (here, N = 8 sampling points) results in the parameter representing insulin sensitivity in the model is cho-
(online) feedback control law uk . The collected data before time k resulting
from past control is used for internal model adaptation [calculated offline, here: sen for this time-dependent adaptation [22], [44]. Feedforward
growing horizon estimator (GHE)] of the MPC by solving the optimization calculation of the control input is then also based on sensitivity
problem J with respect to the model parameters p using a growing horizon identified using past measurement data.
(N = k) or a moving horizon (N = constant) estimator (not described here,
see [2]). Once control is initiated, MPC is continuously updated with new In this study, only a global time-invariant patient-specific
model parameters to improve the feedback control, whereas P is solved at adaptation of the model was conducted (model individualiza-
every sampling point, J is generally solved only once every few sampling tion using all available measurement data) to improve prediction
points (here at every fourth).
of the individual’s core dynamics. Unlike other systems [22],
[44], the model adaptation used here does not accommodate for
that sensor error would range from 3% to 17% MARE, and that short-term changes in insulin sensitivity, as this is thought to
the delay would range from 0 to 30 min. be at least partly captured by the mechanistic structure of the
The results of the robustness analysis are depicted in a traffic- internal model itself, e.g., through insulin receptor and glucagon
light plot (see Fig. 4). They indicate that an increase in MARE dynamics [4]. A different approach was chosen for managing
leads to an overall reduction in glucose levels as the % of time the remaining intraindividual variability because establishing a
at target increases (for the zero time-delay cases), but also to an time-dependent insulin sensitivity profile within the PBPK/PD
increase in the prevalence of hypoglycaemic events. In contrast, framework is too time consuming. The intraday variability (un-
increasing the time delay increases the prevalence of hypergly- certainties and disturbances, i.e., the prediction/measurement
caemia due to increased glucose fluctuations. mismatch [45] also on a shorter timescale) is, thus, handled by
the dynamic shift in the target value and the dose-correction
IV. DISCUSSION components of the (FMPD) offset controller. In contrast to
We have developed a novel approach to individualized AGC other set-point optimization approaches [46], the offset con-
by integrating, for the first time, a detailed generic whole- troller is used only to improve the robustness of the approach;
body PBPK/PD model [4] into a robust MPC algorithm. We it is not used to anticipate behavioral patterns of the controlled
1500 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 63, NO. 7, JULY 2016

TABLE III and an accompanying loss of system flexibility, and, thus, a re-
PARAMETERS FOR THE CONTROL ALGORITHM
duced ability to accommodate for extreme short-term changes in
patient behavior.
Parameter Value Dimension Description Source
MPC
V. CONCLUSION
UT 140 [mg/dl] Upper glucose target range limit GCP
LT 70 [mg/dl] Lower glucose target range limit GCP Over the years, MPC has established itself as the benchmark
WTV 100 [−] Weight for target value cost function fit in AGC systems research.
c 2 [−] Ordinate shift for value cost function fit
WN G S 1/2000 [−] Weight for negative slope penalty fit
To address relevant issues in blood-glucose control such as
kd 1/100 [−] Gain for negative slope penalty fit system lag time and variability, as well as sensor inaccuracy,
W E IO B 3000 [−] Weight of insulin-on-board constraint fit we have developed a robust nonlinear MPC using a PBPK/PD
Offset Controller (FMPD) model that captures absorption, distribution, and metabolization
K pΔ T V 0.2 [−] Controller gain for P-controller fit
K iΔ T V 0.02 [−] Controller gain for I-controller fit of glucose, insulin, and glucagon, as well as interactions among
W pΔ T V 0.02 [−] Forgetting factor for P-controller fit them.
K pΔ u 4e-8 [U/(mg/dl)] Controller gain for P-controller fit To ensure robustness of the developed algorithm, a thorough
K iΔ u 1e-9 [U/(mg/dl)] Controller gain for I-controller fit
K dΔ u 5e-7 [U/(mg/dl)] Controller gain for D-controller fit robustness evaluation was conducted. This analysis revealed
W pΔ u 0.02 [−] Forgetting factor for P-controller fit that our approach could cope with sensor noise of a MARD of
W dΔ u 0.1 [−] Forgetting factor for D-controller fit 10%–14% which is sufficient to cover the measurement errors
W E IO B 30 [−] Weight of insulin-on-board constraint fit
kp , d 3 [−] Constant for gain scheduling fit made by current state-of-the-art CGMs.
The new approach, which uses a generic PBPK platform,
also allows seamless integration, i.e., extension, of the AGC
with other PBPK diabetes modeling and simulation work. It is
individual. Also, gain scheduling is applied for safety reasons possible to integrate established PBPK models for treatment of
(“pump shutoff”) but not to adapt to patient characteristics [14]. diabetes with different types of insulin, treatments not involving
Although controller performance was good, our analysis in- insulin, or treatment of comorbidities [4].
dicates opportunities for improvement in future versions. First, In summary, the current study describes and evaluates, both
model uncertainties in predicting insulin dynamics (due to er- in silico and through retrospective analysis, a blood-glucose
rors in the estimated half-life of insulin) result in inverse os- control system that combines a highly predictive whole-body
cillation of patient plasma glucose levels and rates of insulin PBPK/PD model [4] with an MPC framework and a dose-
infusion. This phenomenon can have serious consequences in correction module capable of reacting to unpredictable patient
the case of high (e.g., prandial) insulin doses but can be at- behavior and handling uncertainty in model predictions and
tenuated by model adaptation. Second, state-of-the-art systems measurements. This approach holds great promise for AGC, and
work through the SC–SC route, which is explicitly mapped by its effectiveness will be further evaluated in a clinical setting.
the current model kernel (interstitial fluid kinetics of glucose in
skin, muscle, and fat, and insulin absorption in SC tissue [4])
APPENDIX
but subject to physiological (plasma to interstitial) and technical
(CGM signal processing) time delays. Although interstitial fluid Here, we describe the components and constraints relevant to
kinetics of the model and the simulated CGM sensor noise were the interactions among components of the integrated modular
evaluated in silico here, it remains to be determined (within approach as depicted in Fig. 5.
a real-clinical trial) whether their modeling is sufficiently ac- A. PBPK Model Basis and Simulation of Blood-Glucose
curate for adequate description of SCGM by CGM devices or Concentrations: The basis for the model is the combination of
if associated interoccasion variability in the measurements can formalisms for drug distribution with the spatial structure of
critically affect controller performance. the physiology-based compartmentalization [1]. Here, a brief
The MPC-based AGC system tested here has been devel- description of whole-body PBPK modeling is given. A more
oped for use in a clinical environment. As such, the next step in-depth description is provided in a published description of a
would be to go beyond evaluation with respect to model un- workshop [47]; this provides insight into the essential structural
certainty and carbohydrate disturbances, and to establish an elements and theoretical concepts that underlie a PBPK model,
understanding of how well the system copes with illness, med- with additional in-depth technical detail in [48]–[50] and detail
ication, stress (e.g., in an intensive-care setting), and physical on the database in [51].
exercise. Although the unprogrammed carbohydrate challenge In short, within a PBPK model, compound concentrations in
experiment indicates that the controller can manage significant subcompartments (vascular, interstitial, and cellular space) of
disturbances, it remains to be determined how predictive (i.e., all major organs are calculated. Compounds distribute via blood
elaborate) such a system should be if it does not explicitly ac- flows and diffusion processes or active transport processes me-
count for all external or internal disturbances to glucose dynam- diated by transport protein as shown in Fig. 6. This approach is
ics. Although the predictive control approach is essential to cope also used to calculate the IV (muscle/fat vascular space) and SC
with time delays in SC administration of insulin, the tradeoff for (fat interstitial space) glucose concentrations to be used as ob-
high-level prediction is the need for high computational power served variables [simulated blood-glucose prediction, x (t)) for
SCHALLER∗ et al.: ROBUST PBPK/PD-BASED MODEL PREDICTIVE CONTROL OF BLOOD GLUCOSE 1501

blood-glucose control]. The PBPK/PD model used for these cal- the one penalizing high glucose levels (upper threshold value
culations is based on coupling PBPK models of glucose, insulin, function UTHV)] defined as:
and glucagon via their interactions, i.e., pharmacodynamics [4].  mg 
The PD functions of the model are directly adapted from UTHVi = e1/70(x i −U T ) , with U T = DT V − 10 (9)
dl
Sorensen [52] with the modifications (parameterization) and
and the stage cost being
extensions (insulin receptor model and incretin effect model)
documented in the supplementary information of [4] (the sup- l (xi ) = WTV (LTHVi + UTHVi − c) (10)
plementary information also includes a model file; the soft-
with weight WTV and the correction constant c chosen to set
ware (PKSim/MoBi) and MATLAB are both required to run
l (xi = DT V ) = 0.
the provided model. The software is available free of charge
(after registration) for academic use (http://www.systems-
biology.com/uc/download.html). 1) Effect of Insulin on Board (MPC): In the case of insulin
PBPK models are extensively used in various applications, oversaturation, the MPC cost function is penalized, with the
including drug–drug interaction studies and pediatric trial sim- EIOB value (7) added to the cost function.
ulations, as well as in regulatory submissions (US Food and
Drug Administration) during the development of pharmaceuti-
cals [53]–[57]. 2) Negative Slope: A negative slope of the blood-glucose
B. Effect of Insulin on Board (EIOB): We designed a trajectory at glucose concentrations around or below the target
penalty function to account for the EIOB. Its functions are value is undesirable, as this indicates that glucose levels continue
parameterized such that only those insulin doses that lead to to fall. Thus, where the slope was negative, a penalty was added
oversaturation of the pharmacodynamic effect at the target tis- to the cost function V (x, k, u). This penalty was defined as
sue are penalized. The EIOB functions for hepatic (EIOBliv ) k
+N
and peripheral/muscular (EIOBmus ) insulin action are calcu- N GS = WN G S −ẽi · exp (−kd (ẽi − ẽi−1 ))
lated based on the individual’s insulin sensitivity (SI ) and the i=k
amount of phosphorylated insulin receptor, as follows: with ẽi = (xi − DT V ) , if (ẽi − ẽi−1 ) < 0 (11)
WEIOB (SI IReff )8 with the penalty weight WN G S and the gain constant kd . This
EIOB (IReff ) = (7) results in the objective function
8 + (S IR )8
Km I eff
k+N 
with the relative effect of phosphorylated insulin receptors in the V (x, k, u) = WTV (LTHVi + UTHVi − c)
respective tissue represented by IReff = IRp /IRp0 , where the i=k
number of phosphorylated insulin receptors is IRp , the number 
of basal phosphorylated insulin receptors IRp0 , the threshold −WN G S (ẽi · exp (−kd (ẽi − ẽi−1 )))
value Km , and the weight of the constraint WEIOB . When the
PBPK/PD model is individualized, the pharmacodynamic func- +EIOBmus
i + EIOBliv
i + uk . (12)
tions are automatically adapted via the insulin sensitivity factor
(parameter SI ), and, thus, the EIOB for each subject is also
individualized. D. FMPD Constraints:
Km is calculated based on the patient’s basal state at the
beginning of the clamp phase, and EIOB is solved for Km 1) Effect of Insulin on Board (the FMPD Controller): Con-
with EIOB (IReff = Km ) = 1. The constraint for EIOB was trolling blood glucose via the SC route introduces significant
used for both the FMPD controller and the MPC algorithm. delays between that time at which the controller input is applied
The details of how the EIOB was applied within the respective and the time at which its effects are initiated. These delays pose
control system are provided below. a major challenge in glucose control, as they generally cause
C. MPC Cost Function: The performance of the MPC the system to both overshoot with respect to insulin levels, and,
algorithm strongly depends on the design of the cost function more importantly, undershoot with respect to glucose levels. If
V (x, k, u) (see Fig. 7), and that of the stage cost l (xi ). A not managed properly, these effects can result in severe episodes
general approach would be to use a sum-of-squares function to of hypoglycemia. Feedback systems with delays are especially
penalize the control errorl (xi ) = f (DT V, xi ) (dynamic target prone to overshoot following severe disturbances. In the case
value versus predicted glucose). However, as hypoglycemia is of glucose control, this can be after a meal is consumed or a
more critical than hyperglycemia, an asymmetrical cost function physical activity is undertaken.
was chosen. Specifically, we chose two overlapping exponential The FMPD controller used to circumvent these problems is
functions, with the one penalizing low glucose levels (lower combined with the EIOB constraint by weighing the dose cor-
threshold value function (LTHV)] defined as rection Δuk with the sum of the weighted EIOB constraints,
 mg  resulting in the effective value for the dose correction Δuk , as
LTHVi = e1/8(L T −x i ) , with LT = DT V − 40 (8) shown in (13) at the top of the next page.
dl
1502 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 63, NO. 7, JULY 2016

k  
Kp j =0 e−W p (k −j ) êj + Ki kj=0 êj + Kd kj=0 e−W d (k −j ) (êj − êj −1 )
Δuk = (13)
1 + EIOBliv + EIOBmus

2) Gain Scheduling: In general, a standard PD or PID con- REFERENCES


troller is symmetric, i.e., both positive and negative deviations [1] S. Willmann et al., “PK-Sim R
: A physiologically based pharmacokinetic
from the target value are treated equally by counteracting con- ‘whole-body’ model,” Biosilico, vol. 1, pp. 121–124, 2003.
trol actions. Here, however, control is one-sided, because insulin [2] J. B. Rawlings, and D. Q. Mayne, Model Predictive Control Theory and
Design. Madison, WI, USA: Nob Hill, 2009.
is infused without any counteracting control input. Below-target [3] R. Hovorka, “Closed-loop insulin delivery: From bench to clini-
episodes of glucose levels are commonly managed by fully sus- cal practice,” Nature Rev. Endocrinol., vol. 7, no. 7, pp. 385–395,
pending activity of the insulin pump whenever blood-glucose 2011.
[4] S. Schaller et al., “A generic integrated physiologically-based whole-
levels dip below the target value. However, our experience with body model of the glucose-insulin-glucagon regulatory system,” CPT,
the dataset at hand showed that extremely low insulin levels Pharmacometrics Syst Pharmacol., vol. 2, no. 8, art. no. e65, 2013.
(which can be reached when the pump is fully suspended) trig- doi:10.1038/psp.2013.40
[5] International-Diabetes-Federation, IDF Diabetes Atlas, Brussels,
ger excessive hepatic glucose output and the release of large Belgium, 2013.
amounts of glucose. Thus, rather than completely suspend- [6] B. Kovatchev, “Closed loop control for type 1 diabetes,” BMJ, vol. 342,
ing pump activity, we reduced the input gain for insulin infu- art. no. d1911, 2011. doi: 10.1136/bmj.d1911
[7] B. W. Bequette, “A critical assessment of algorithms and challenges in
sions below target value to avoid depleting the insulin pool. We the development of a closed-loop artificial pancreas,” Diabetes Technol.
calculated the adjusted controller gains as Ther., vol. 7, no. 1, pp. 28–47, Feb. 2005.
[8] C. Cobelli et al., “Artificial pancreas: Past, present, future,” Diabetes,
vol. 60, no. 11, pp. 2672–2682, Nov. 1, 2011.
K̃p,d = Kp,d (1 + 3ē (t)) , if ē (t) < 0, with ē (t) [9] E. Dassau et al., “Closing the loop,” Int. J. Clin. Pract. Suppl., vol. 65,
no. 166, pp. 20–25, Feb. 2010.
= (y (t) − TV) /TV. (14) [10] R. Hovorka et al., “Overnight closed loop insulin delivery (artificial
pancreas) in adults with type 1 diabetes: Crossover randomised controlled
All control inputs calculated as negative were set to zero. studies,” BMJ, vol. 342, art. no. d1855, Apr. 2011. doi: 10.1136/bmj.d1855
[11] G. M. Steil et al., “Closed-loop insulin delivery-the path to physiological
E. CGM In Silico Sensor Noise: Artificial error signal was glucose control,” Adv. Drug Del. Rev., vol. 56, no. 2, pp. 125–144, Feb.
calculated using the method described by Fachinetti et al. [58]. 10, 2004.
The SC measurement (SCGM) signal is calculated as [12] B. Gopakumaran et al., “A novel insulin delivery algorithm in rats with
type 1 diabetes: The fading memory proportional-derivative method,” Ar-
tif. Organs, vol. 29, no. 8, pp. 599–607, Aug. 2005.
SCGM (t) = (1 + s (t)) IG (t) + v (t) (15) [13] J. R. Castle et al., “Novel use of glucagon in a closed-loop system
for prevention of hypoglycemia in type 1 diabetes,” Diabetes Care,
vol. 33, no. 6, pp. 1282–1287, Jun. 2010.
with v (t) representing a Gaussian noise signal (zero mean, max. [14] P. G. Jacobs et al., “Automated control of an adaptive bihormonal,
rel. error 35%), IG (t) the actual interstitial (i.e., SC) glucose dual-sensor artificial pancreas and evaluation during inpatient stud-
levels, and s (t) the error drift factor generated from integration ies,” IEEE Trans. Biomed. Eng., vol. 61, no. 10, pp. 2569–2581,
Oct. 2014.
of a Gaussian noise signal, as [15] P. G. Jacobs et al., “Development of a fully automated closed loop ar-
tificial pancreas control system with dual pump delivery of insulin and
s (t + 1) = 3s (t) − 3s (t − 1) + s (t − 2) + w (t) (16) glucagon,” in Proc. IEEE Annu. Int. Conf. Eng. Med. Biol. Soc., 2011,
pp. 397–400.
[16] A. Dauber et al., “Closed-loop insulin therapy improves glycemic control
with w(t) as the Gaussian noise (zero mean and max 1% random in children aged <7 years: A randomized controlled trial,” Diabetes Care,
drift per time step of 15 min) that results in a maximal relative vol. 36, no. 2, pp. 222–227, Feb. 1, 2013.
[17] M. J. O’Grady et al., “The use of an automated, portable glucose control
drift of max(s) = 80% within a 24-h time interval. system for overnight glucose control in adolescents and young adults with
The artificial SCGM (SCGM(t)) was, thus, generated by su- type 1 diabetes,” Diabetes Care, vol. 35, no. 11, pp. 2182–2187, Nov. 1,
perimposing the single artificial sensor noise signal profile on the 2012.
[18] E. Atlas et al., “MD-logic artificial pancreas system: A pilot study in adults
raw glucose trajectories from the SC compartment (interstitial with type 1 diabetes,” Diabetes Care, vol. 33, no. 5, pp. 1072–1076, May
skin) of the simulated individual. 2010.
[19] S. Miller et al., “Automatic learning algorithm for the MD-logic ar-
tificial pancreas system,” Diabetes Technol. Ther., vol. 13, no. 10,
ACKNOWLEDGMENT pp. 983–990, Oct. 2011.
[20] J. El Youssef et al., “A review of closed-loop algorithms for glycemic
The authors would like to thank S. Willmann and control in the treatment of type 1 diabetes,” Algorithms, vol. 2, no. 1,
pp. 518–532, 2009.
M. Krauss for fruitful discussion and T. Gaub, J. Solodenko, [21] R. S. Parker et al., “A model-based algorithm for blood glucose control
and K. Coboeken for the strong technical support as platform in type I diabetic patients,” IEEE Trans. Biomed. Eng., vol. 46, no. 2,
developers. S. Schaller designed and performed the research, pp. 148–157, Feb. 1999.
[22] R. Hovorka et al., “Nonlinear model predictive control of glucose con-
analyzed the data, and developed the control algorithm, the centration in subjects with type 1 diabetes,” Physiol. Meas., vol. 25,
model kernel, and modeling tools, and J. Lippert, A. Schuppert, no. 4, pp. 905–920, Aug. 2004.
and T. Eissing designed research and developed the modeling [23] L. Magni et al., “Run-to-run tuning of model predictive control for type
1 diabetes subjects: In silico trial,” J. Diabetes Sci. Technol., vol. 3, no. 5,
tools. L. Schaupp and T. R. Pieber contributed the clinical data. pp. 1091–1098, 2009.
All authors helped in preparing this paper.
SCHALLER∗ et al.: ROBUST PBPK/PD-BASED MODEL PREDICTIVE CONTROL OF BLOOD GLUCOSE 1503

[24] S. J. Russell et al., “Blood glucose control in type 1 diabetes with a [49] T. Rodgers, and M. Rowland, “Physiologically based pharmacokinetic
bihormonal bionic endocrine pancreas,” Diabetes Care, vol. 35, no. 11, modelling 2: Predicting the tissue distribution of acids, very weak
pp. 2148–2155, Nov. 2012. bases, neutrals and zwitterions,” J. Pharmaceutical Sci., vol. 95, no. 6,
[25] P. Soru et al., “MPC based artificial pancreas: Strategies for individ- pp. 1238–1257, Jun. 2006.
ualization and meal compensation,” Annu. Rev. Control, vol. 36, no. 1, [50] W. Schmitt, “General approach for the calculation of tissue to plasma
pp. 118–128, 2012. partition coefficients,” Toxicol. In Vitro, vol. 22, no. 2, pp. 457–467, Mar.
[26] L. Leelarathna et al., “Day and night home closed-loop insulin delivery 2008.
in adults with type 1 diabetes: Three-center randomized crossover study,” [51] S. Willmann et al., “Development of a physiology-based whole-body
Diabetes Care, vol. 37, no. 7, pp. 1931–1937, Jul. 2014. population model for assessing the influence of individual variability
[27] S. Del Favero et al., “First use of model predictive control in out- on the pharmacokinetics of drugs,” J. Pharmacokinet. Pharmacodyn.,
patient wearable artificial pancreas,” Diabetes Care, vol. 37, no. 5, vol. 34, no. 3, pp. 401–431, Jun. 2007.
pp. 1212–1215, May 2014. [52] J. T. Sorensen, “A physiologic model of glucose metabolism in man and
[28] C. Dalla Man et al., “Meal simulation model of the glucose-insulin its use to design and assess improved insulin therapies for diabetes,” Ph.D.
system,” IEEE Trans. Biomed. Eng., vol. 54, no. 10, pp. 1740–1749, Oct. dissertation, Dept. Chem. Eng., Massachusetts Inst. Technol., Cambridge,
2007. MA, USA, 1985.
[29] B. P. Kovatchev et al., “In silico preclinical trials: A proof of concept [53] R. Leong et al., “Regulatory experience with physiologically based phar-
in closed-loop control of type 1 diabetes,” J. Diabetes Sci. Technol., macokinetic modeling for pediatric drug trials,” Clin. Pharmacol. Thera-
vol. 3, no. 1, pp. 44–55, 2009. peutics, vol. 91, no. 5, pp. 926–931, 2012.
[30] R. Hovorka et al., “A simulation model of glucose regulation in the [54] P. Zhao et al., “Applications of physiologically based pharmacokinetic
critically ill,” Physiol. Meas., vol. 29, no. 8, pp. 959–978, Aug. 2008. (PBPK) modeling and simulation during regulatory review,” Clin. Phar-
[31] T. Eissing et al., “A computational systems biology software platform for macol. Therapeutics, vol. 89, no. 2, pp. 259–267, 2011.
multiscale modeling and simulation: Integrating whole-body physiology, [55] A. R. Maharaj and A. N. Edginton, “Physiologically based pharmacoki-
disease biology, and molecular reaction networks,” Front. Physiol., vol. 2, netic modeling and simulation in pediatric drug development,” CPT, Phar-
no. 4, pp. 1–10, 2011. doi:10.3389/fphys.2011.00004 macometrics Syst. Pharmacol., vol. 3, no. 11, pp. 1–13, 2014.
[32] A. Strougo et al., “First dose in children: Physiological insights into [56] A. R. Maharaj et al., “A workflow example of PBPK modeling to support
pharmacokinetic scaling approaches and their implications in paedi- pediatric research and development: Case study with lorazepam,” AAPS
atric drug development,” J. Pharmacokinet. Pharmacodyn., vol. 39, J., vol. 15, no. 2, pp. 455–464, Apr. 1, 2013.
pp. 195–203, Feb. 5, 2012. [57] H. M. Jones et al., “Physiologically based pharmacokinetic modeling in
[33] R. Hovorka et al., “Closing the loop: The adicol experience,” Diabetes drug discovery and development: A pharmaceutical industry perspective,”
Technol. Ther., vol. 6, no. 3, pp. 307–318, Jun. 2004. Clin. Pharmacol. Ther., vol. 97, no. 3, pp. 247–262, Mar. 2015.
[34] D. Q. Mayne et al., “Constrained model predictive control: Stability and [58] A. Facchinetti et al., “Reconstruction of glucose in plasma from interstitial
optimality,” Automatica, vol. 36, no. 6, pp. 789–814, 2000. fluid continuous glucose monitoring data: Role of sensor calibration,”
[35] R. C. Dorf, and R. H. Bishop, Modern Control Systems, 12th ed. Engle- J. Diabetes Sci. Technol., vol. 1, no. 5, pp. 617–623, Sep. 2007.
wood Cliffs, NJ, USA: Prentice-Hall, 2011.
[36] G. F. Franklin et al., Feedback Control of Dynamic Systems, 4th ed.
Englewood Cliffs, NJ, USA: Prentice-Hall, 2002.
[37] Mueller. Dr. Mueller Geraetebau. (2015). [Online]. Available: http://www. Stephan Schaller was born in Stuttgart, Germany, in
dr-mueller-geraetebau.de/ 1984. He received the Diploma degree in control sys-
[38] E. R. Damiano et al., “A comparative effectiveness analysis of three tems engineering from the University of Stuttgart,
continuous glucose monitors: The navigator, G4 Platinum, and enlite,” J. Stuttgart, in 2009, and the Ph.D. degree in com-
Diabetes Sci. Technol., vol. 8, pp. 699–708, Apr. 21, 2014. putational engineering from the Aachen Institute
[39] L. Magni et al., “Evaluating the efficacy of closed-loop glucose regulation for Advanced Studies in Computational Engineer-
via control-variability grid analysis,” J. Diabetes Sci. Technol., vol. 2, no. ing Science, RWTH Technical University of Aachen,
4, pp. 630–635, Jul. 2008. Aachen, Germany, in 2014.
[40] F. J. Doyle et al., “Closed-loop artificial pancreas systems: Engineering Since 2013, he has been a Biological Modeling
the algorithms,” Diabetes Care, vol. 37, no. 5, pp. 1191–1197, May 1, Expert at Bayer Technology Services, GmbH, Lev-
2014. erkusen, Germany. Since 2015, he has also been a
[41] P. A. Bakhtiani et al., “A review of artificial pancreas technologies Senior Systems Pharmacology Scientist at Sanofi Deutschland GmbH, Frank-
with an emphasis on bi-hormonal therapy,” Diabetes Obesity Metab., furt, Germany. Besides carrying out PBPK/PD and systems biology modeling
vol. 15, no. 12, pp. 1065–1070, Dec. 2013. and consulting in this area, he is developing an algorithm for automated blood-
[42] R. Hovorka et al., “Assessing performance of closed-loop insulin delivery glucose control using physiologically based pharmacokinetic/dynamic model
systems by continuous glucose monitoring: Drawbacks and way forward,” kernels within a model predictive control framework.
Diabetes Technol. Ther., vol. 15, no. 1, pp. 4–12, Jan. 2013. Mr. Schaller received the “Hans Günter Schäfer-PK/PD-Science-Award” in
[43] B. P. Kovatchev, “Diabetes technology: Markers, monitoring, assess- 2014.
ment, and control of blood glucose fluctuations in diabetes,” Scien-
tifica, Cairo, vol. 2012, art. no. 283821(14 pages), pp. 1–14, 2012.
doi:10.6064/2012/283821
[44] R. Hovorka et al., “Blood glucose control by a model predictive control Jörg Lippert, photograph and biography not available at the time of publication.
algorithm with variable sampling rate versus a routine glucose manage-
ment protocol in cardiac surgery patients: A randomized controlled trial,”
J. Clin. Endocrinol. Metab., vol. 92, no. 8, pp. 2960–2964, Aug. 2007.
[45] B. W. Bequette, “Algorithms for a closed-loop artificial pancreas: The Lukas Schaupp received the Diploma and Ph.D. de-
case for model predictive control,” J. Diabetes Sci. Technol., vol. 7, no. 6, grees in electrical and biomedical engineering from
pp. 1632–1643, Nov. 2013. the Graz University of Technology, Graz, Austria, in
[46] Y. Wang et al., “Model predictive control with learning-type set-point: 1992 and 1998, respectively.
Application to artificial pancreatic β-cell,” AIChE J., vol. 56, no. 6, In 1995, he attended the Postgraduate Internship
pp. 1510–1518, 2010. Program with the Whiting School of Engineering,
[47] H. M. Jones and K. Rowland-Yeo, “Basic concepts in physiologically Johns Hopkins University, Baltimore, USA (Applied
based pharmacokinetic modeling in drug discovery and development,” Physics Laboratory). From 1998–2001, he was a Re-
CPT, Pharmacometrics Syst. Pharmacol., vol. 2, no. 8, art. no. e63, search Fellow with Karl Franzens University, Graz.
pp. 1–12, 2013. doi:10.1038/psp.2013.41 From 2001 to 2009, he was with the Joanneum Re-
[48] T. Rodgers et al., “Physiologically based pharmacokinetic modeling 1: search Institute of Medical Technologies and Health
Predicting the tissue distribution of moderate-to-strong bases,” J. Phar- Management, Austria. Since 2001, he has been a Senior Researcher at the
maceutical Sci., vol. 94, no. 6, pp. 1259–1276, Jun. 2005. Medical University of Graz, Graz. His main research interests include medical
technologies and devices, with particular focus in diabetes research.
1504 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 63, NO. 7, JULY 2016

Thomas R. Pieber received the M.D. degree from Andreas Schuppert, photograph and biography not available at the time of
the Karl-Franzens-University School of Medicine, publication.
Graz, Austria, in 1987.
He was trained in intensive care medicine, en-
docrinology, and metabolism at the Karl-Franzens- Thomas Eissing studied life sciences and biotech-
University, Graz, at the Heinrich-Heine-University, nology at the University of Stuttgart, Stuttgart, Ger-
Düsseldorf, Germany, and at the University of Texas, many, and at the University of New South Wales,
Dallas, USA. From 2005 to 2008, he was the Med- Sydney, N.S.W., Australia, and received a diploma in
ical Director with the University Hospital Graz, technical biology from the Univeryity of Stuttgart in
and he is currently a Head at the Department of 2002. He received his Ph.D. degree from the Institute
Internal Medicine, Division of Endocrinology and of Systems Theory and Automatic Control, Stuttgart
Metabolism, Medical University of Graz, Graz. Since 2001, he has been a Direc- in 2007.
tor at the HEALTH—Institute for Biomedicine and Health Sciences, Joanneum In 2007, he joined Bayer Technology Services,
Research, Graz, which carries out research for the development and evaluation Leverkusen, Germany, where he is involved in phar-
of medical systems, data and health management, and bioanalytics for medical macokinetic, pharmacodynamic, and systems biol-
research. He is an Opinion Leader in the field of diabetes, heading a multidisci- ogy modeling and consulting. In 2012, he became the Group Head of Systems
plinary team of 140 employees, of which approximately half are researchers and Biology, and in 2014, he became the Group Head of Systems Pharmacology
half are healthcare professionals working in the field of diabetes and associated CV.
disorders. Dr. Eissing diploma thesis received an award for the best technical biology
thesis of the year, and his Ph.D. thesis received the MTZ award for medical
systems biology.