1

Introduction

During an experiment, measurement of certain variables e.g. concentrations,

pressures, temperatures, etc. is conducted. Let y = [ y i , y 2 > " - > y m ] b
me m

dimensional vector of measured variables during an experiment. In addition, during each experiment, certain conditions are set or fixed by the experimentalist e.g. substrate concentration in an enzyme kinetics experiment, time and temperature in a kinetics experiment, etc. Let x = [x], x 2 ,..., x n ] r be the n-dimensional vector of
these input variables that can be assumed to be known precisely.

The experimentalist often formulates a mathematical model in order to describe the observed behavior. In general, the model consists of a set of equations based on the principles of chemistry, physics, thermodynamics, kinetics and transport phenomena and attempts to predict the variables, y, that are being measured. In general, the measured variables y are a function of x. Thus, the model has the following form
y = Function of(\, k) + Random Error (1.1)

The random error arises from the measurement of y the true value of which
is not known. The measurements are assumed to be free of systematic errors. The

modeling equations contain adjustable parameters to account for the fact that the models are phenomenological. For example, kinetic rate expressions contain rate constants (parameters) the value of which is unknown and not possible to be obtained from fundamental principles.

Copyright 2001 by Taylor & Francis Group, LLC

Chapter 1

Parameter estimation is one of the steps involved in the formulation and validation of a mathematical model that describes a process of interest. Parameter estimation refers to the process of obtaining values of the parameters from the matching of the model-based calculated values to the set of measurements (data). This is the classic parameter estimation or model fitting problem and it should be distinguished from the identification problem. The latter involves the development of a model from input/output data only. This case arises when there is no a priori information about the form of the model i.e. it is a black box. When the model equations are linear functions of the parameters the problem is called linear estimation. Nonlinear estimation refers to the more general and most frequently encountered situation where the model equations are nonlinear functions of the parameters. Parameter estimation and identification are an essential step in the development of mathematical models that describe the behavior of physical processes (Seinfeld and Lapidus, 1974; Aris, 1994). The reader is strongly advised to consult the above references for discussions on what is a model, types of models, model formulation and evaluation. The paper by Plackett that presents the history on the discovery of the least squares method is also recommended (Plackett, 1972). A smooth function f(x) is used quite often to describe a set of data, (x^y^, ( X 2>y2)> j( x N>yN)- Fitting of a smooth curve, f(x), through these points is usually done for interpolation or visual purposes (Sutton and MacGregor, 1977). This is called curve fitting. The parameters defining the curve are calculated by minimiz-

ing a measure of the closeness of fit such as the function S(k)= /^[y; -f(xj)] .
The parameters have no physical significance. The scope of this book deals primarily with the parameter estimation problem. Our focus will be on the estimation of adjustable parameters in nonlinear models described by algebraic or ordinary differential equations. The models describe processes and thus explain the behavior of the observed data. It is assumed that the structure of the model is known. The best parameters are estimated in order to be used in the model for predictive purposes at other conditions where the model is called to describe process behavior. The unknown model parameters will be obtained by minimizing a suitable objective function. The objective function is a measure of the discrepancy or the departure of the data from the model i.e., the lack of fit (Bard, 1974; Seinfeld and Lapidus, 1974). Thus, our problem can also be viewed as an optimization problem and one can in principle employ a variety of solution methods available for such problems (Edgar and Himmelblau, 1988; Gill et al. 1981; Reklaitis, 1983; Scales, 1985). Finally it should be noted that engineers use the term parameter estimation whereas statisticians use such terms as nonlinear or linear regression analysis to describe the subject presented in this book. In parameter estimation, the general problem we have to solve is:
Copyright 2001 by Taylor & Francis Group, LLC

Introduction

Given the structure of the model (i.e. the governing model equations) and a set of measured data points, the problem is to find the unknown model parameters so that the values calculated by the model match the data in some optimal manner (e.g., by minimizing the sum of squares of errors)

The specific issues that we have tried to address in this book are:

(i)
(ii)

Structure of the Model ("What kind of models can be used? Linear or nonlinear? Algebraic or differential equation models?")
Selection of the Objective Function ("What do we minimize to estimate the parameters?")

(Hi)

Solution Techniques ("How do we minimize the objective function?")

(iv)

Statistical Properties of Parameter Estimates ("How accurate are the estimated parameters?")
Statistical Properties of Model-Based Calculated Values ("Given the uncertainty in the model parameters, what is the uncertainty in the calculated values?")
Tests for Model Adequacy ("Is the model good enough?")

(v)

(vi)

(vii)
(viii)

Tests for Model Discrimination ("Among several rival models, which is the best one?")
Factorial Experimental Design ("What is the first set of experiments I should run?")
Sequential Experimental Design ("What should my next experiment be so that I gather maximum information?") for model discrimination (to select the best model among several ones that fit the data,) or for precise parameter estimation (to minimize further the parameter uncertainty in a particular model).

(ix)

Issues such as the ones above are of paramount importance and interest to practicing engineers, researchers and graduate students. In the next paragraphs we mention several examples that cover many important areas of chemical, biochemical and petroleum engineering. Chemical kinetics is an area that received perhaps most of the attention of chemical engineers from a parameter estimation point of view. Chemical engineers need mathematical expressions for the intrinsic rate of chemical reactions
Copyright 2001 by Taylor & Francis Group, LLC

Chapter 1

in order to design reactors that produce chemicals in an industrial scale. For example, let us consider the following consecutive first order reactions that take place (Smith, 1981; Froment and Bischoff, 1990)
k] k2
(1.2) A -> B -> D

During an experiment, the concentrations of A and B are usually measured. If it is assumed that only component A is present in the reactor initially (CA0^0,
CBO=0, CDO=0), we do not need to measure the concentration of chemical D because C A +CB+C D =CAO- The rate equations for A and B are given by
, A = k,C

dt dC R dt .

where CA and CB are the concentrations of A and B, t is reaction time, and k h k2

are the rate constants. The above rate equations can be easily integrated to yield the following algebraic equations

(L4b)

We can then obtain C D from the concentration invariant CD=CAO-CA-CB

c n -cJ.+-" -!"''"
The values of the rate constants are estimated by fitting equations 1.4a and 1.4b to the concentration versus time data. It should be noted that there are kinetic models that are more complex and integration of the rate equations can only be done numerically. We shall see such models in Chapter 6. An example is given next. Consider the gas phase reaction of NO with O2 (Bellman et al. 1967):
Copyright 2001 by Taylor & Francis Group, LLC

Introduction

2NO + O2 <-> 2NO2

(1.5)

The model is given by the following kinetic rate equation

C N02 (0) = 0

(1.6)

where a= 126.2, p=91.9, and k,, k2 are the rate constants to be estimated. The data consists of a set of measurements of the concentration of NO 2 versus time.
In this case, fitting of the data requires the integration of the governing differential equation. In biochemical engineering one often employs parametric models to describe enzyme-catalyzed reactions, microbial growth, nutrient transport and metabolic rates in biological systems. The parameters in these models can be readily obtained though the methods described in this book. In addition, we present shortcut estimation methods for the estimation of the average specific uptake or production rates during cell cultivation in batch, fed-batch, chemostat or perfusion cultures. For example, such an analysis is particularly important to scientists screening cell lines to identify high producers. Petroleum and chemical engineers perform oil reservoir simulation to optimize the production of oil and gas. Black-oil, compositional or thermal oil reservoir models are described by sets of differential equations. The measurements consist of the pressure at the wells, water-oil ratios, gas-oil ratios etc. The objective is to estimate through histoiy matching of the reservoir unknown reservoir properties such as porosity and permeability. Volumetric equations of state (EoS) are employed for the calculation of fluid phase equilibrium and thermo-physical properties required in the design of processes involving non-ideal fluid mixtures in the oil, gas and chemical industries. Mathematically, a volumetric EoS expresses the relationship among pressure, volume, temperature, and composition for a fluid mixture. The next equation gives the Peng-Robinson equation of state, which is perhaps the most widely used EoS in industrial practice (Peng and Robinson, 1976). (1.7)

v- b

v(v + b) + b(v - b)

where P is the pressure, T the temperature and v the molar volume. The mixture

parameters a and b are defined as follows

Copyright 2001 by Taylor & Francis Group, LLC

Chapter 1

i b' i
where at, b, and 3j are parameters specific for the individual components i and j, and kjj is an empirical interaction parameter characterizing the binary formed by

component i and component j. It is well known that the introduction of empirical parameters, k,j, in the equations of state enhances their ability as tools for process design. The interaction parameters are usually estimated from the regression of binary vapor-liquid equilibrium (VLB) data. Such data consist of sets of temperature, pressure and liquid and vapor phase mole fractions for one of the components. Typical examples such as the ones mentioned above, are used throughout this book and they cover most of the applications chemical engineers are faced with. In addition to the problem definition, the mathematical development and the numerical results, the implementation of each algorithm is presented in detail and computer listings of selected problems are given in the attached CD. Finally, it is noted that even though we are not concerned with model development in this book, we should always keep in mind that: (i) The development of models is not necessarily in the direction of greater
complexity or an increasing number of adjustable parameters.

(ii)

A higher degree of confidence is gained if the adjustable parameters are found from independent sources.

Copyright 2001 by Taylor & Francis Group, LLC