Você está na página 1de 552

Advanced Control using

M ATLAB

or Stabilising the
unstabilisable

David I. Wilson
Auckland University of Technology
New Zealand
July 27, 2016
Copyright 2016 David I. Wilson
Auckland University of Technology
New Zealand

Creation date: July, 2016.

All rights reserved. No part of this work may be reproduced, stored in a retrieval system, or
transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or
otherwise, without prior permission.
C ONTENTS

1 Introduction 1
1.1 Notation and colour conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Matlab for computer aided control design . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.1 Alternative computer design aids . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Laboratory equipment for control tests . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.1 Plants with one input and one output . . . . . . . . . . . . . . . . . . . . . . 5
1.3.2 Multi-input and multi-output plants . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.3 Slowing down Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Economics of control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 From differential to difference equations 13


2.1 Computer in the loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.1.1 Sampling an analogue signal . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.2 Selecting a sample rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.3 The sampling theorem and aliases . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.4 Discrete frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Finite difference models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.1 Difference equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 The z transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.1 z-transforms of common functions . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4 Inversion of z-transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4.1 Inverting z-transforms with symbolically . . . . . . . . . . . . . . . . . . . . 24
2.4.2 The partial fraction method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4.3 Long division . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4.4 Computational approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4.5 Numerically inverting the Laplace transform . . . . . . . . . . . . . . . . . . 31
2.5 Discretising with a sample and hold . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.5.1 Converting Laplace transforms to z-transforms . . . . . . . . . . . . . . . . 38
2.5.2 The bilinear transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.6 Discrete root locus diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.7 Multivariable control and state space analysis . . . . . . . . . . . . . . . . . . . . . . 44
2.7.1 States and state space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.7.2 Converting differential equations to state space form . . . . . . . . . . . . . 49
2.7.3 Interconverting between state space and transfer functions . . . . . . . . . . 51
2.7.4 Similarity transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.7.5 Interconverting between transfer functions forms . . . . . . . . . . . . . . . 57
2.7.6 The steady state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.7.7 Poles and zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.8 Solving the vector differential equation . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.8.1 Numerically computing the discrete transformation . . . . . . . . . . . . . . 64
2.8.2 Using M ATLAB to discretise systems . . . . . . . . . . . . . . . . . . . . . . . 66
2.8.3 Time delay in state space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
2.9 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
iii
iv CONTENTS

2.9.1 Stability in the continuous domain . . . . . . . . . . . . . . . . . . . . . . . . 72


2.9.2 Stability of the closed loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.9.3 Stability of discrete time systems . . . . . . . . . . . . . . . . . . . . . . . . . 76
2.9.4 Stability of nonlinear differential equations . . . . . . . . . . . . . . . . . . . 77
2.9.5 Expressing matrix equations succinctly using Kronecker products . . . . . . 83
2.9.6 Summary of stability analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
2.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3 Modelling dynamic systems with differential equations 87


3.1 Dynamic system models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.1.1 Steady state and dynamic models . . . . . . . . . . . . . . . . . . . . . . . . 88
3.2 A collection of illustrative dynamic models . . . . . . . . . . . . . . . . . . . . . . . 88
3.2.1 Simple models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.2.2 A continuously-stirred tank reactor . . . . . . . . . . . . . . . . . . . . . . . 93
3.2.3 A forced circulation evaporator . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.2.4 Binary distillation column models . . . . . . . . . . . . . . . . . . . . . . . . 96
3.2.5 Rigorous tray-by-tray distillation column models . . . . . . . . . . . . . . . 100
3.2.6 Interaction and the Relative Gain Array . . . . . . . . . . . . . . . . . . . . . 104
3.3 Curve fitting experimental data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.3.1 Least-squares polynomial regression . . . . . . . . . . . . . . . . . . . . . . . 109
3.3.2 Least squares improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.3.3 Identification of nonlinear models . . . . . . . . . . . . . . . . . . . . . . . . 114
3.3.4 Parameter confidence intervals . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.4 Numerical tools for modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
3.4.1 The solution of nonlinear ordinary differential equations . . . . . . . . . . . 123
3.4.2 Differential/Algebraic equation systems and algebraic loops . . . . . . . . . 127
3.5 Linearisation of nonlinear dynamic equations . . . . . . . . . . . . . . . . . . . . . . 128
3.5.1 Linearisation using the S YMBOLIC toolbox. . . . . . . . . . . . . . . . . . . . 130
3.5.2 Using linmod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
3.5.3 Quantifying the extent of the nonlinearity . . . . . . . . . . . . . . . . . . . . 132
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

4 The PID controller 135


4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.1.1 P, PI or PID control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.2 The industrial PID algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.2.1 Implementing the derivative component . . . . . . . . . . . . . . . . . . . . 136
4.2.2 Variations of the PID algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.2.3 Integral only control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.3 Simulating a PID process in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.4 Extensions to the PID algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.4.1 Avoiding derivative kick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
4.4.2 Input saturation and integral windup . . . . . . . . . . . . . . . . . . . . . . 145
4.5 Discrete PID controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.5.1 Discretising continuous PID controllers . . . . . . . . . . . . . . . . . . . . . 149
4.5.2 Simulating a PID controlled response in Matlab . . . . . . . . . . . . . . . . 150
4.5.3 Controller performance as a function of sample time . . . . . . . . . . . . . 151
4.6 PID tuning methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
4.6.1 Open loop tuning methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
4.6.2 Closed loop tuning methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.6.3 Closed loop single-test tuning methods . . . . . . . . . . . . . . . . . . . . . 162
4.6.4 Summary on closed loop tuning schemes . . . . . . . . . . . . . . . . . . . . 169
4.7 Automated tuning by relay feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
4.7.1 Describing functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
4.7.2 An example of relay tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
CONTENTS v

4.7.3 Self-tuning with noise disturbances . . . . . . . . . . . . . . . . . . . . . . . 176


4.7.4 Modifications to the relay feedback estimation algorithm . . . . . . . . . . . 178
4.8 Drawbacks with PID controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
4.8.1 Inverse response processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
4.8.2 Approximating inverse-response systems with additional deadtime . . . . 187
4.9 Dead time compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
4.10 Tuning and sensitivity of control loops . . . . . . . . . . . . . . . . . . . . . . . . . . 191
4.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

5 Digital filtering and smoothing 195


5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
5.1.1 The nature of industrial noise . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
5.1.2 Differentiating without smoothing . . . . . . . . . . . . . . . . . . . . . . . . 197
5.2 Smoothing measured data using analogue filters . . . . . . . . . . . . . . . . . . . . 198
5.2.1 A smoothing application to find the peaks and troughs . . . . . . . . . . . . 199
5.2.2 Filter types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
5.2.3 Classical analogue filter families . . . . . . . . . . . . . . . . . . . . . . . . . 203
5.3 Discrete filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
5.3.1 A low-pass filtering application . . . . . . . . . . . . . . . . . . . . . . . . . . 211
5.3.2 Digital filter approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5.3.3 Efficient hardware implementation of discrete filters . . . . . . . . . . . . . 216
5.3.4 Numerical and quantisation effects for high-order filters . . . . . . . . . . . 218
5.3.5 Filter visualisation tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
5.4 The Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.4.1 Fourier transform definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
5.4.2 Orthogonality and frequency spotting . . . . . . . . . . . . . . . . . . . . . . 225
5.4.3 Power spectral density estimates from time series data . . . . . . . . . . . . 227
5.4.4 Fourier smoothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
5.5 Numerically differentiating industrial data . . . . . . . . . . . . . . . . . . . . . . . 231
5.5.1 Establishing feedrates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

6 Identification of dynamic models 237


6.1 The importance of system identification . . . . . . . . . . . . . . . . . . . . . . . . . 237
6.1.1 Basic definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
6.1.2 Black, white and grey box models . . . . . . . . . . . . . . . . . . . . . . . . 239
6.1.3 Techniques for identification . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
6.2 Graphical and non-parametric model identification . . . . . . . . . . . . . . . . . . 241
6.2.1 Time domain identification using graphical techniques . . . . . . . . . . . . 241
6.2.2 Experimental frequency response analysis . . . . . . . . . . . . . . . . . . . 247
6.2.3 An alternative empirical transfer function estimate . . . . . . . . . . . . . . 253
6.3 Continuous model identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6.3.1 Fitting transfer functions using nonlinear least-squares . . . . . . . . . . . . 256
6.3.2 Identification of continuous models using successive differentiation . . . . 257
6.3.3 Practical continuous model identification . . . . . . . . . . . . . . . . . . . . 260
6.4 Popular discrete-time models structures . . . . . . . . . . . . . . . . . . . . . . . . . 261
6.4.1 Extending the linear model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
6.4.2 Output error model structures . . . . . . . . . . . . . . . . . . . . . . . . . . 265
6.4.3 General input/output models . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6.4.4 Common nonlinear model structures . . . . . . . . . . . . . . . . . . . . . . 267
6.5 Regressing discrete model parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 268
6.5.1 Simple offline ARX model identification routines . . . . . . . . . . . . . . . 271
6.5.2 Bias in the parameter estimates with noise models . . . . . . . . . . . . . . . 272
6.5.3 Using the System Identification toolbox . . . . . . . . . . . . . . . . . . . . . 273
6.5.4 Estimating noise models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
vi CONTENTS

6.5.5 Fitting parameters to state space models . . . . . . . . . . . . . . . . . . . . 277


6.6 Model structure determination and validation . . . . . . . . . . . . . . . . . . . . . 280
6.6.1 Estimating model order and deadtime . . . . . . . . . . . . . . . . . . . . . . 282
6.6.2 Robust model fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
6.7 Online model identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
6.7.1 Recursive least squares estimation . . . . . . . . . . . . . . . . . . . . . . . . 286
6.7.2 Recursive least-squares in M ATLAB . . . . . . . . . . . . . . . . . . . . . . . 290
6.7.3 Recursive least-squares in S IMULINK . . . . . . . . . . . . . . . . . . . . . . 292
6.7.4 Tracking the precision of the estimates . . . . . . . . . . . . . . . . . . . . . . 295
6.8 Modifications to the basic RLS scheme . . . . . . . . . . . . . . . . . . . . . . . . . . 297
6.8.1 The forgetting factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
6.8.2 The influence of the forgetting factor . . . . . . . . . . . . . . . . . . . . . . . 299
6.8.3 Covariance wind-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
6.8.4 Maintaining a symmetric positive definite covariance matrix . . . . . . . . . 302
6.9 Online estimating of noise models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
6.9.1 A recursive extended least-squares example . . . . . . . . . . . . . . . . . . 305
6.9.2 Recursive identification using the SI toolbox . . . . . . . . . . . . . . . . . . 308
6.9.3 Simplified RLS algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
6.10 Identification by parameter optimisation . . . . . . . . . . . . . . . . . . . . . . . . . 310
6.11 Closed loop identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
6.11.1 Closed loop RLS in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
6.12 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

7 Adaptive Control 321


7.1 Why adapt? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
7.1.1 The adaption scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
7.1.2 Classification of adaptive controllers . . . . . . . . . . . . . . . . . . . . . . . 323
7.1.3 A simple example of adaption: Gain scheduling . . . . . . . . . . . . . . . . 323
7.1.4 Identify and then control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
7.2 Self tuning regulators (STRs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
7.2.1 Simple minimum variance control . . . . . . . . . . . . . . . . . . . . . . . . 327
7.3 Adaptive pole-placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
7.3.1 The Diophantine equation and the closed loop . . . . . . . . . . . . . . . . . 330
7.3.2 Solving the Diophantine equation in Matlab . . . . . . . . . . . . . . . . . . 331
7.3.3 Adaptive pole-placement with identification . . . . . . . . . . . . . . . . . . 334
7.3.4 Adaptive pole-placement in S IMULINK . . . . . . . . . . . . . . . . . . . . . 337
7.4 Practical adaptive pole-placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
7.4.1 Dealing with non-minimum phase systems . . . . . . . . . . . . . . . . . . . 340
7.4.2 Separating stable and unstable factors . . . . . . . . . . . . . . . . . . . . . . 343
7.4.3 Experimental adaptive pole-placement . . . . . . . . . . . . . . . . . . . . . 345
7.4.4 Minimum variance control with dead time . . . . . . . . . . . . . . . . . . . 346
7.5 Summary of adaptive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

8 Multivariable controller design 355


8.1 Controllability and observability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
8.1.1 Controllability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
8.1.2 Observability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
8.1.3 Computing controllability and observability . . . . . . . . . . . . . . . . . . 359
8.1.4 State reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
8.2 State space pole-placement controller design . . . . . . . . . . . . . . . . . . . . . . 364
8.2.1 Poles and where to place them . . . . . . . . . . . . . . . . . . . . . . . . . . 367
8.2.2 Deadbeat control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
8.2.3 Pole-placement for tracking control . . . . . . . . . . . . . . . . . . . . . . . 371
8.3 Estimating the unmeasured states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
8.4 Combining estimation and state feedback . . . . . . . . . . . . . . . . . . . . . . . . 375
CONTENTS vii

8.5 Generic model control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377


8.5.1 The tuning parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
8.5.2 GMC control of a linear model . . . . . . . . . . . . . . . . . . . . . . . . . . 380
8.5.3 GMC applied to a nonlinear plant . . . . . . . . . . . . . . . . . . . . . . . . 381
8.6 Exact feedback linearisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
8.6.1 The nonlinear system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
8.6.2 The input/output feedback linearisation control law . . . . . . . . . . . . . 388
8.6.3 Exact feedback example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
8.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394

9 Classical optimal control 395


9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
9.2 Parametric optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
9.2.1 Choosing a performance indicator . . . . . . . . . . . . . . . . . . . . . . . . 396
9.2.2 Optimal tuning of a PID regulator . . . . . . . . . . . . . . . . . . . . . . . . 399
9.2.3 Calling S IMULINK from inside an optimiser . . . . . . . . . . . . . . . . . . . 402
9.2.4 An optimal batch reactor temperature policy . . . . . . . . . . . . . . . . . . 404
9.3 The general optimal control problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
9.3.1 A revision of multivariable calculus . . . . . . . . . . . . . . . . . . . . . . . 407
9.3.2 The optimal control formulation . . . . . . . . . . . . . . . . . . . . . . . . . 408
9.3.3 The two-point boundary problem . . . . . . . . . . . . . . . . . . . . . . . . 411
9.3.4 Optimal control examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
9.3.5 Problems with a specified target set . . . . . . . . . . . . . . . . . . . . . . . 419
9.4 Linear quadratic control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
9.4.1 Continuous linear quadratic regulators . . . . . . . . . . . . . . . . . . . . . 422
9.4.2 Analytical solution to the LQR problem . . . . . . . . . . . . . . . . . . . . . 424
9.4.3 The steady-state solution to the matrix Riccati equation . . . . . . . . . . . . 427
9.4.4 The discrete LQR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
9.4.5 A numerical validation of the optimality of LQR . . . . . . . . . . . . . . . . 438
9.4.6 Tracking LQR control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
9.5 Estimation of state variables in stochastic systems . . . . . . . . . . . . . . . . . . . 443
9.5.1 Random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
9.5.2 Covariance matrices and confidence ellipses of random sequences . . . . . 449
9.5.3 Combining deterministic and stochastic processes . . . . . . . . . . . . . . . 452
9.5.4 The Kalman filter estimation scheme . . . . . . . . . . . . . . . . . . . . . . . 454
9.5.5 The steady-state form of the Kalman filter . . . . . . . . . . . . . . . . . . . . 458
9.5.6 Current and future prediction forms . . . . . . . . . . . . . . . . . . . . . . . 459
9.5.7 An application of the Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . 462
9.5.8 The role of the Q and R noise covariance matrices . . . . . . . . . . . . . . . 464
9.5.9 Extensions to the basic Kalman filter algorithm . . . . . . . . . . . . . . . . . 467
9.5.10 The Extended Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
9.5.11 Combining state estimation and state feedback . . . . . . . . . . . . . . . . . 472
9.5.12 Optimal control using only measured outputs . . . . . . . . . . . . . . . . . 473
9.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475

10 Predictive control 477


10.1 Model predictive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
10.1.1 Constrained predictive control . . . . . . . . . . . . . . . . . . . . . . . . . . 480
10.1.2 Dynamic matrix control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
10.2 A Model Predictive Control Toolbox . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
10.2.1 A model predictive control GUI . . . . . . . . . . . . . . . . . . . . . . . . . 489
10.2.2 MPC toolbox in M ATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490
10.2.3 Using the MPC toolbox in S IMULINK . . . . . . . . . . . . . . . . . . . . . . 492
10.2.4 MPC control of an unstable 3 degree of freedom helicopter . . . . . . . . . . 493
10.2.5 Further readings on MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
viii CONTENTS

10.3 Optimal control using linear programming . . . . . . . . . . . . . . . . . . . . . . . 498


10.3.1 Development of the LP problem . . . . . . . . . . . . . . . . . . . . . . . . . 498

A List of symbols 507

B Useful utility functions in Matlab 509

C Transform pairs 511

D Useful test models 513


D.1 A forced circulation evaporator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513
D.2 Aircraft model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
L IST OF F IGURES

1.1 Standard color conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2


1.2 Closed loop color conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 The standard colour and plot layout for a controlled response used in this book. . 3
1.4 Black Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 The Black-box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6 Balance arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.7 Fan and flapper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.8 Flapper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.9 Helicopter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.10 Helicopter control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.11 Helicopter flying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.12 Real-time Simulink simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.13 Traditional vs. Advanced control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.14 Economic improvements of better control . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1 The computer in the control loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14


2.2 3 bit sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 A time series with unknown frequency components . . . . . . . . . . . . . . . . . . 18
2.4 The frequency component of a sampled signal . . . . . . . . . . . . . . . . . . . . . 18
2.5 Frequency aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.6 The Scarlet Letter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.7 Henons attractor in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.8 Henons attractor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.9 Inverting z-transforms using dimpulse . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.10 Numerically inverting the Laplace transform using the Bromwich integral . . . . . 32
2.11 Visualising the Bromwich integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.12 Numerically inverting the Laplace transform . . . . . . . . . . . . . . . . . . . . . . 35
2.13 Numerically inverting Laplace transforms . . . . . . . . . . . . . . . . . . . . . . . . 35
2.14 Ideal sampler and zeroth-order hold. . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.15 Rectangular pulse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.16 Zeroth-order hold effects on the discrete Bode diagram . . . . . . . . . . . . . . . . 42
2.17 Bode plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.18 The discrete root locus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.19 Various discrete closed loop responses . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.20 A binary distillation column block diagram . . . . . . . . . . . . . . . . . . . . . . . 46
2.21 Block diagrams of continuous and discrete state-space systems . . . . . . . . . . . . 48
2.22 A block diagram of a state-space system with direct feed-through . . . . . . . . . . 48
2.23 Unsteady and steady states for level systems . . . . . . . . . . . . . . . . . . . . . . 59
2.24 Submarine step response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
2.25 Issues and approaches in assessing system stability . . . . . . . . . . . . . . . . . . 72
2.26 Nyquist diagram of Eqn. 2.102 in (a) three dimensions and (b) as typically pre-
sented in two dimensions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.27 Liapunov (18571918) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
ix
x LIST OF FIGURES

2.28 Regions of stability for continuous and discrete systems . . . . . . . . . . . . . . . . 85

3.1 A stable and unstable pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89


3.2 Simple buffer tank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.3 The UK growth based on the GDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.4 A continually stirred tank reactor (CSTR) reactor . . . . . . . . . . . . . . . . . . . . 94
3.5 A forced circulation evaporator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.6 Distillation column model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
3.7 Wood-Berry column in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.8 Wood-Berry step response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.9 Distillation tower . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.10 Sparsity of the distillation column model . . . . . . . . . . . . . . . . . . . . . . . . 103
3.11 Open loop distillation column control . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.12 Distillation column control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.13 Distillation column control (in detail) . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.14 Distillation interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.15 Dynamic RGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.16 Dynamic RGA for a 3 term system . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.17 Density of Air . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.18 Fitting a high-order polynomial to some physical data . . . . . . . . . . . . . . . . . 113
3.19 A bio-chemical reaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.20 Model of compressed water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.21 Experimental pressure-rate data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.22 t distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
3.23 Experimental pressure-rate data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.24 Parameter confidence regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
3.25 Linear and nonlinear trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
3.26 Linearising a nonlinear tank model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

4.1 PID block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137


4.2 A series PID controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.3 Comparing PI and integral-only control for the real-time control of a noisy flapper
plant with sampling time T = 0.08. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.4 PID simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.5 PID internals in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
4.6 PID controller variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
4.7 Realisable PID controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.8 PID controller in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.9 PID controller with anti-derivative kick. . . . . . . . . . . . . . . . . . . . . . . . . . 142
4.10 Avoiding derivative kick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
4.11 Illustrating the improvement of anti-derivative kick schemes for PID controllers
when applied to the experimental electromagnetic balance. . . . . . . . . . . . . . . 144
4.12 Derivative control and noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.13 Anti-windup comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
4.14 Discrete PID controller in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
4.15 Headbox control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
4.16 Headbox controlled response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
4.17 A PID controlled process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
4.18 Sample time and discrete PID control . . . . . . . . . . . . . . . . . . . . . . . . . . 152
4.19 ZN open loop tuning method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
4.20 Cohen-Coon model fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
4.21 Cohen-Coon tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.22 PID tuning using a GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
4.23 Solving for the ultimate frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
4.24 Ziegler-Nichols tuned responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
LIST OF FIGURES xi

4.25 A typical closed loop response of a stable system to a P-controller . . . . . . . . . . 163


4.26 A Yuwana-Seborg closed loop step test . . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.27 Closed loop responses using the YS scheme . . . . . . . . . . . . . . . . . . . . . . . 167
4.28 A self-tuning PID controlled process . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.29 A process under relay tuning with the PID regulator disabled. . . . . . . . . . . . . 170
4.30 An unknown plant under relay feedback . . . . . . . . . . . . . . . . . . . . . . . . 172
4.31 Nyquist & Bode diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
4.32 PID Relay tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
4.33 Relay tuning with noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
4.34 Relay tuning of the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
4.35 Relay tuning results of the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
4.36 A relay with hysteresis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
4.37 Relay with hysteresis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
4.38 Relay feedback with hysteresis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
4.39 Relay feedback with an integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.40 2-point Relay identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
4.41 Monotonicity index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
4.42 The J curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
4.43 An inverse response process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
4.44 A NMP plant controlled with a PI controller . . . . . . . . . . . . . . . . . . . . . . 187
4.45 Approximating inverse-response systems with additional deadtime . . . . . . . . . 188
4.46 Smith Predictor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
4.47 Smith Predictor with no mismatch . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
4.48 Smith predictor in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
4.49 Dead time compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
4.50 Deadtime compensation applied to the blackbox . . . . . . . . . . . . . . . . . . . . 192
4.51 Closed loop with plant subjected to disturbances . . . . . . . . . . . . . . . . . . . . 192
4.52 Sensitivity transfer functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
4.53 Sensitivity robustness measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

5.1 A filter as a transfer function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195


5.2 A noisy measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
5.3 Noise added to a true signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
5.4 Derivative action given noisy data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
5.5 Smoothing industrial data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
5.6 Low-pass filter specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
5.7 Three single low-pass filters cascaded together to make a third-order filter. . . . . . 201
5.8 Amplitude response of filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
5.9 Analogue Butterworth filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
5.10 Analogue Chebyshev filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
5.11 Butterworth and Chebyshev filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
5.12 Using a Butterworth filter to smooth noisy data . . . . . . . . . . . . . . . . . . . . . 212
5.13 The frequency response for Butterworth filters . . . . . . . . . . . . . . . . . . . . . 213
5.14 Various Butterworth filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
5.15 Advantages of frequency pre-warping . . . . . . . . . . . . . . . . . . . . . . . . . . 215
5.16 Hardware difference equation in Direct Form I . . . . . . . . . . . . . . . . . . . . . 216
5.17 An IIR filter with a minimal number of delays . . . . . . . . . . . . . . . . . . . . . 217
5.18 Cascaded second-order sections to realise a high-order filter . . . . . . . . . . . . . 219
5.19 A second-order section (SOS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
5.20 Comparing single precision second-order sections with filters in direct form II trans-
posed form. Note that the direct form II filter is actually unstable when run in single
precision. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.21 Magnitude response from fvtool . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.22 The Fourier approximation to a square wave . . . . . . . . . . . . . . . . . . . . . . 225
5.23 Two signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
xii LIST OF FIGURES

5.24 Critical radio frequencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229


5.25 Power spectrum for a signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
5.26 Smoothing by Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
5.27 Differentiating and smoothing noisy measurement . . . . . . . . . . . . . . . . . . . 233
5.28 Filtering industrial data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

6.1 The prediction problem for modelling . . . . . . . . . . . . . . . . . . . . . . . . . . 237


6.2 Comparing model and plant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
6.3 An experimental setup for input/output identification . . . . . . . . . . . . . . . . 240
6.4 Typical open loop step tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6.5 Areas method for model identification . . . . . . . . . . . . . . . . . . . . . . . . . . 243
6.6 Examples of the Areas method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
6.7 Identification of the Blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
6.8 Balance arm step test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
6.9 Random signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
6.10 A 5-element binary shift register to generate a pseudo-random binary sequence. . 246
6.11 Pseudo-random binary sequence generator in S IMULINK . . . . . . . . . . . . . . . 247
6.12 Black box experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
6.13 Black box response analysis using a series of sine waves . . . . . . . . . . . . . . . . 249
6.14 Black box response using an input chirp signal. . . . . . . . . . . . . . . . . . . . . . 250
6.15 Black box frequency response analysis using a chirp signal. . . . . . . . . . . . . . . 251
6.16 Flapper response to a chirp signal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
6.17 Experimental setup to subject a random input into an unknown plant. The in-
put/output data was collected, processed through Listing 6.2 to give the frequency
response shown in Fig. 6.18. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
6.18 The experimental frequency response compared to the true analytical Bode dia-
gram. See the routine in Listing 6.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
6.19 Black box response given a pseudo-random input sequence. . . . . . . . . . . . . . 253
6.20 Black box frequency response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
6.21 Empirical transfer function estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6.22 Experimental data from a continuous plant . . . . . . . . . . . . . . . . . . . . . . . 256
6.23 A continuous-time model fitted to input/output data . . . . . . . . . . . . . . . . . 257
6.24 Continuous model identification strategy . . . . . . . . . . . . . . . . . . . . . . . . 259
6.25 Continuous model identification simulation . . . . . . . . . . . . . . . . . . . . . . . 260
6.26 Continuous model identification of the blackbox . . . . . . . . . . . . . . . . . . . . 261
6.27 Identification using Laguerre functions . . . . . . . . . . . . . . . . . . . . . . . . . 262
6.28 A signal flow diagram of an auto-regressive model with exogenous input or ARX
model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
6.29 A signal flow diagram of a ARMAX model . . . . . . . . . . . . . . . . . . . . . . . 264
6.30 Discrete time models implemented in S IMULINK . . . . . . . . . . . . . . . . . . . . 266
6.31 A signal flow diagram of an output-error model . . . . . . . . . . . . . . . . . . . . 266
6.32 A general input/output model structure . . . . . . . . . . . . . . . . . . . . . . . . . 267
6.33 Hammerstein and Wiener model structures . . . . . . . . . . . . . . . . . . . . . . . 268
6.34 Kaos, Control and regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
6.35 ARX estimation exhibiting bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
6.36 Offline system identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
6.37 An AR only model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
6.38 Fitting a polynomial to some (x, y) data . . . . . . . . . . . . . . . . . . . . . . . . . 281
6.39 Using the AIC to establish model structure . . . . . . . . . . . . . . . . . . . . . . . 281
6.40 Using the AIC to establish model structure using separate validation data . . . . . 282
6.41 Identification of deadtime from the step response . . . . . . . . . . . . . . . . . . . 283
6.42 Deadtime estimation at fast sampling . . . . . . . . . . . . . . . . . . . . . . . . . . 283
6.43 Deadtime estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
6.44 Blackbox step model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
6.45 Ideal RLS parameter estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
LIST OF FIGURES xiii

6.46 Recursive least squares estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292


6.47 RLS under S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
6.48 RLS under S IMULINK (version 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
6.49 Estimation of a two parameter plant . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
6.50 Confidence limits for estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
6.51 RLS and an abrupt plant change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
6.52 The memory of the forgetting factor . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
6.53 Identification using various forgetting factors . . . . . . . . . . . . . . . . . . . . . . 300
6.54 Covariance wind-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
6.55 Variable forgetting factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
6.56 RLS with coloured noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
6.57 Recursive extended least-squares estimation . . . . . . . . . . . . . . . . . . . . . . 307
6.58 Recursive extended least-squares estimation . . . . . . . . . . . . . . . . . . . . . . 308
6.59 A simplified recursive least squares algorithm . . . . . . . . . . . . . . . . . . . . . 309
6.60 Furnace input/output data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
6.61 The MIT rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
6.62 Optimising the adaptation gain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
6.63 Closed loop estimation using Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . 315
6.64 RLS in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

7.1 Adaptive control structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323


7.2 Varying process gain of a spherical tank . . . . . . . . . . . . . . . . . . . . . . . . . 324
7.3 Simple minimum variance control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328
7.4 Simple minimum variance control (zoomed) . . . . . . . . . . . . . . . . . . . . . . 329
7.5 Pole-placement control structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
7.6 Adaptive pole-placement control structure with RLS identification . . . . . . . . . 335
7.7 Control of multiple plants with an adapting controller. We desire the same closed
loop response irrespective of the choice of plant. . . . . . . . . . . . . . . . . . . . . 335
7.8 Three open loop plants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
7.9 Desired closed loop response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
7.10 Adaptive pole-placement with identification . . . . . . . . . . . . . . . . . . . . . . 338
7.11 Comparing the adaptive pole-placement with the reference trajectory . . . . . . . . 338
7.12 Adaptive pole-placement with identification in S IMULINK . . . . . . . . . . . . . . 339
7.13 Internals of the Adaptive pole-placement design routine . . . . . . . . . . . . . . . 339
7.14 Bursting in adaptive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
7.15 A plant with poorly damped zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
7.16 Adaptive pole-placement with an unstable B . . . . . . . . . . . . . . . . . . . . . . 344
7.17 Migration of poles and zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
7.18 Areas of well-damped poles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
7.19 Adaptive pole-placement of the black-box . . . . . . . . . . . . . . . . . . . . . . . . 347
7.20 Adaptive pole-placement of the black-box (zoomed portion) . . . . . . . . . . . . . 348
7.21 A non-minimum phase plant with an unstable zero which causes an inverse step
response. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
7.22 Moving average control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

8.1 Reconstructing states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364


8.2 Pole-placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
8.3 Pole-placement degradation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
8.4 Deadbeat control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
8.5 State feedback control system with an integral output state. . . . . . . . . . . . . . . 372
8.6 State feedback with integral states . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
8.7 Simultaneous control and state estimation . . . . . . . . . . . . . . . . . . . . . . . . 375
8.8 Control and estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
8.9 GMC tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
8.10 GMC tuning characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
xiv LIST OF FIGURES

8.11 Linear GMC response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382


8.12 Linear GMC comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
8.13 GMC CSTR control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
8.14 A CSTR phase plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
8.15 The configuration of an input/output feedback linearisation control law . . . . . . 388
8.16 Exact feedback linearisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392

9.1 IAE areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397


9.2 ITAE breakdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
9.3 Optimal responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
9.4 Optimal PID tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
9.5 Optimum PI tuning of the blackbox plant . . . . . . . . . . . . . . . . . . . . . . . . 402
9.6 S IMULINK model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
9.7 Production of a valuable chemical in a batch reactor. . . . . . . . . . . . . . . . . . . 404
9.8 Temperature profile optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
9.9 Optimum temperature profile comparison for different number of temperatures . 406
9.10 Optimal control for a simple cart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
9.11 Optimal control for the Rayleigh problem . . . . . . . . . . . . . . . . . . . . . . . . 416
9.12 Optimal control for the batch reactor . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
9.13 Optimal control with targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
9.14 Steady-state and time-varying LQR control . . . . . . . . . . . . . . . . . . . . . . . 426
9.15 Steady-state continuous LQR controller . . . . . . . . . . . . . . . . . . . . . . . . . 429
9.16 A block diagram of LQR control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
9.17 Comparing discrete and continuous LQR controllers . . . . . . . . . . . . . . . . . . 435
9.18 LQR control with varying tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
9.19 Pole-placement and LQR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
9.20 Pole-placement and LQR showing the input . . . . . . . . . . . . . . . . . . . . . . 439
9.21 Trial pole locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
9.22 Trial pole-placement performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
9.23 LQR servo control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
9.24 LQI servo control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
9.25 Black box servo control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
9.26 A block diagram of a state estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
9.27 PDF of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
9.28 Correlated noisy x, y data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
9.29 2D ellipse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
9.30 2D ellipse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452
9.31 A block diagram of a stochastic state-space system . . . . . . . . . . . . . . . . . . . 452
9.32 Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458
9.33 Prediction-type Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460
9.34 Current-type Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
9.35 Kalman filter demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
9.36 The performance of a Kalman filter for different q/r ratios . . . . . . . . . . . . . . 467
9.37 A random walk process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
9.38 LQG in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
9.39 LQG of an aircraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474

10.1 Horizons used in model predictive control . . . . . . . . . . . . . . . . . . . . . . . 478


10.2 Predictions of the Reserve Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478
10.3 Inverse plant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
10.4 Predictive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
10.5 Acausal response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
10.6 Varying the horizons of predictive control . . . . . . . . . . . . . . . . . . . . . . . . 484
10.7 MPC on the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
10.8 Step response coefficients for a stable system . . . . . . . . . . . . . . . . . . . . . . 485
LIST OF FIGURES xv

10.9 DMC control details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487


10.10DMC control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488
10.11Adaptive DMC of the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490
10.12An MPC graphical user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
10.13Multivariable MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492
10.14S IMULINK and MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493
10.15MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
10.16A 3 degree of freedom helicopter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
10.17MPC control of a helicopter structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 495
10.18MPC control of a helicopter results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
10.19LP constraint matrix dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500
10.20LP optimal control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504
10.21LP optimal control with active constraints . . . . . . . . . . . . . . . . . . . . . . . . 504
10.22Non-square LP optimal control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
10.23LP optimal control showing acausal behaviour . . . . . . . . . . . . . . . . . . . . . 505
xvi LIST OF FIGURES
L IST OF TABLES

1.1 Computer aids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1 Final and initial value theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23


2.2 Inverting a z transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3 Laplace transform pairs used for testing . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.1 Standard nomenclature used in modelling dynamic systems . . . . . . . . . . . . . 92


3.2 Parameters of the CSTR model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.3 The important variables in the forced circulation evaporator from Fig. 3.5. . . . . . 95
3.4 Compressed water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.5 The parameter values for the CSTR model . . . . . . . . . . . . . . . . . . . . . . . . 126
3.6 The initial state and manipulated variables for the CSTR simulation . . . . . . . . . 126

4.1 Alternative PID tuning parameter conventions . . . . . . . . . . . . . . . . . . . . . 136


4.2 Ziegler-Nichols open-loop PID tuning rules . . . . . . . . . . . . . . . . . . . . . . . 156
4.3 PID controller settings based on IMC for a small selection of common plants where
the control engineer gets to chose a desired closed loop time constant, c . . . . . . . 157
4.4 Various alternative Ziegler-Nichols type PID tuning rules as a function of the ul-
timate gain, Ku , and ultimate period, Pu . . . . . . . . . . . . . . . . . . . . . . . . . 159
4.5 Closed-loop single-test PID design rules . . . . . . . . . . . . . . . . . . . . . . . . . 165
4.6 Relay based PID tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

5.1 Filter transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

6.1 Experimentally determined frequency response of the blackbox . . . . . . . . . . . 249


6.2 Bias in estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
6.3 Identification in state-space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

8.1 The relationship between regulation and estimation . . . . . . . . . . . . . . . . . . 374


8.2 Litchfield nonlinear CSTR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

9.1 Common integral performance indices . . . . . . . . . . . . . . . . . . . . . . . . . . 398

xvii
xviii LIST OF TABLES
L ISTINGS

2.1 Numerically inverting Laplace transforms using the Gaver-Stehfest algorithm . . . 33


2.2 Symbolic Laplace to z-transform conversion . . . . . . . . . . . . . . . . . . . . . . 38
2.3 Symbolic Laplace to z-transform conversion with ZOH . . . . . . . . . . . . . . . . 38
2.4 Extracting the gain, time constants and numerator time constants from an arbitrary
transfer function format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.5 Find the transmission zeros for a MIMO system . . . . . . . . . . . . . . . . . . . . 60
2.6 Submarine simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.7 Example of the Routh array using the symbolic toolbox . . . . . . . . . . . . . . . . 74
2.8 Solve the continuous matrix Lyapunov equation using Kronecker products . . . . 80
2.9 Solve the matrix Lyapunov equation using the lyap routine . . . . . . . . . . . . . 80
2.10 Using Lyapunov to establish stability of a linear system . . . . . . . . . . . . . . . . 81
2.11 Solve the discrete matrix Lyapunov equation using Kronecker products . . . . . . 82
3.1 Computing the dynamic relative gain array analytically . . . . . . . . . . . . . . . . 107
3.2 Computing the dynamic relative gain array numerically as a function of . See also
Listing 3.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.3 Curve fitting using polynomial least-squares . . . . . . . . . . . . . . . . . . . . . . 111
3.4 Polynomial least-squares using singular value decomposition. This routine follows
from, and provides an alternative to, Listing 3.3. . . . . . . . . . . . . . . . . . . . . 113
3.5 Curve fitting using a generic nonlinear optimiser . . . . . . . . . . . . . . . . . . . . 115
3.6 Curve fitting using the O PTI optimisation toolbox. (Compare with Listing 3.5.) . . 116
3.7 Fitting water density as a function of temperature and pressure . . . . . . . . . . . 117
3.8 Finding the optimum parameter for a nonlinear reaction rate model . . . . . . . . . 118
3.9 Routines for the cumulative probability distribution, pt, and the inverse CDF, qt,
for the t-distribution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
3.10 Parameter confidence limits for a nonlinear reaction rate model. This routines fol-
lows from Listing 3.8 and uses the statistical functions from Listing 3.9. . . . . . . . 120
3.11 Comparing the dynamic response of a pendulum to the linear approximation . . . 124
3.12 The dynamics of a chaotic pendulum from Eqn. 3.69. . . . . . . . . . . . . . . . . . 125
3.13 Using linmod to linearise an arbitrary S IMULINK module. . . . . . . . . . . . . . . 131
4.1 Constructing a transfer function of a PID controller . . . . . . . . . . . . . . . . . . 137
4.2 Constructing a discrete (filtered) PID controller . . . . . . . . . . . . . . . . . . . . . 149
4.3 A simple continuous-time PID controller . . . . . . . . . . . . . . . . . . . . . . . . 150
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
4.4 Ziegler-Nichols PID tuning rules for an arbitrary transfer function . . . . . . . . . . 162
4.5 Identifies the characteristic points for the Yuwana-Seborg PID tuner from a trial
closed loop response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
4.6 Compute the closed loop model from peak and trough data . . . . . . . . . . . . . 168
4.7 Compute the ultimate gain and frequency from the closed loop model parameters. 168
4.8 Compute the open loop model, Gm , Eqn. 4.31. . . . . . . . . . . . . . . . . . . . . . 168
4.9 Compute appropriate PI or PID tuning constants based on a plant model, Gm , us-
ing the IMC schemes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
4.10 Calculates the period and amplitude of a sinusoidal time series using least-squares. 177
5.1 Designing Butterworth Filters using Eqn. 5.4. . . . . . . . . . . . . . . . . . . . . . . 204
5.2 Designing a low-pass Butterworth filter with a cut-off frequency of fc = 800 Hz. . 205
xix
xx LISTINGS

5.3 Designing a high-pass Butterworth filter with a cut-off frequency of fc = 800 Hz. . 205
5.4 Designing Chebyshev Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
5.5 Computing a Chebyshev Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
5.6 Converting a 7th-order Butterworth filter to 4 second-order sections . . . . . . . . . 219
5.7 Comparing DFII and SOS digital filters in single precision. . . . . . . . . . . . . . . 220
5.8 Designing and visualising a 5th order elliptic band-stop filter. . . . . . . . . . . . . 221
5.9 Routine to compute the power spectral density plot of a time series . . . . . . . . . 227
5.10 Smoothing and differentiating a noisy signal . . . . . . . . . . . . . . . . . . . . . . 233
6.1 Identification of a first-order plant with deadtime from an openloop step response
using the Areas method from Algorithm 6.1. . . . . . . . . . . . . . . . . . . . . . . 243
6.2 Frequency response identification of an unknown plant directly from input/output
data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
6.3 Non-parametric frequency response identification using etfe. . . . . . . . . . . . . 253
6.4 Function to generate output predictions given a trial model and input data. . . . . 256
6.5 Optimising the model parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
6.6 Validating the fitted model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
6.7 Continuous model identification of a non-minimum phase system . . . . . . . . . . 258
6.8 Generate some input/output data for model identification . . . . . . . . . . . . . . 271
6.9 Estimate an ARX model from an input/output data series using least-squares . . . 272
6.10 An alternative way to construct the data matrix for ARX estimation using Toeplitz
matrices. See also Listing 6.9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
6.11 Offline system identification using arx from the System Identification Toolbox . . 274
6.12 Offline system identification with no model/plant mismatch . . . . . . . . . . . . . 275
6.13 Demonstrate the fitting of an AR model. . . . . . . . . . . . . . . . . . . . . . . . . . 275
6.14 Create an input/output sequence from an output-error plant. . . . . . . . . . . . . 276
6.15 Parameter identification of an output error process using oe and arx. . . . . . . . 277
6.16 A basic recursive least-squares (RLS) update (without forgetting factor) . . . . . . . 289
6.17 Tests the RLS identification scheme using Listing 6.16. . . . . . . . . . . . . . . . . . 291
6.18 A recursive least-squares (RLS) update with a forgetting factor. (See also Listing 6.16.)299
6.19 Create an ARMAX process and generate some input/output data suitable for sub-
sequent identification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
6.20 Identify an ARMAX process from the data generated in Listing 6.19. . . . . . . . . 306
6.21 Recursively identify an ARMAX process. . . . . . . . . . . . . . . . . . . . . . . . . 307
6.22 Kaczmarzs algorithm for identification . . . . . . . . . . . . . . . . . . . . . . . . . 309
6.23 Adaption of the plant gain using steepest descent . . . . . . . . . . . . . . . . . . . 313
7.1 Simple minimum variance control where the plant has no time delay . . . . . . . . 327
7.2 A Diophantine routine to solve F A + BG = T for the polynomials F and G. . . . . 332
7.3 Alternative Diophantine routine to solve F A + BG = T for the polynomials F and
G. Compare with Listing 7.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
7.4 Constructing polynomials for the Diophantine equation example . . . . . . . . . . 333
7.5 Solving the Diophantine equation using polynomials generated from Listing 7.4. . 334
7.6 Adaptive pole-placement control with 3 different plants . . . . . . . . . . . . . . . . 336
7.7 The pole-placement control law when H = 1/B . . . . . . . . . . . . . . . . . . . . 341
7.8 Factorising an arbitrary polynomial B(q) into stable, B + (q), and unstable and poorly
damped, B (q), factors such that B = B + B and B + is defined as monic. . . . . . 343
7.9 Minimum variance control design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
8.1 A simple state reconstructor following Algorithm 8.1. . . . . . . . . . . . . . . . . . 363
8.2 Pole-placement control of a well-behaved system . . . . . . . . . . . . . . . . . . . . 367
8.3 A deadbeat controller simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
8.4 Pole placement for controllers and estimators . . . . . . . . . . . . . . . . . . . . . . 376
8.5 GMC on a Linear Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
8.6 GMC for a batch reactor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
8.7 The dynamic equations of a batch reactor . . . . . . . . . . . . . . . . . . . . . . . . 385
8.8 Find the Lie derivative for a symbolic system . . . . . . . . . . . . . . . . . . . . . . 390
8.9 Establish relative degree, r (ignore degree 0 possibility) . . . . . . . . . . . . . . . . 391
LISTINGS xxi

8.10 Design Butterworth filter of order r. . . . . . . . . . . . . . . . . . . . . . . . . . . . 391


8.11 Symbolically create the closed loop expression . . . . . . . . . . . . . . . . . . . . . 391
9.1 Returns the IAE performance for a given tuning. . . . . . . . . . . . . . . . . . . . . 400
9.2 Optimal tuning of a PID controller for a non-minimum phase plant. This script file
uses the objective function given in Listing 9.1. . . . . . . . . . . . . . . . . . . . . . 400
9.3 Returns the ITSE using a S IMULINK model. . . . . . . . . . . . . . . . . . . . . . . . 403
9.4 Solving the optimal control problem for a simple cart . . . . . . . . . . . . . . . . . 413
9.5 Analytically computing the co-state dynamics and optimum input trajectory as a
function of states and co-states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
9.6 Solving the reaction profile boundary value problem using the boundary value
problem solver, bvp4c.m. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
9.7 Computes the full time-evolving LQR solution . . . . . . . . . . . . . . . . . . . . . 425
9.8 The continuous time differential Riccati equation. This routine is called from List-
ing 9.9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
9.9 Solves the continuous time differential Riccati equation using a numerical ODE
integrator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
9.10 Calculate the continuous optimal steady-state controller gain. . . . . . . . . . . . . 428
9.11 Closed loop simulation using an optimal steady-state controller gain. . . . . . . . . 429
9.12 Solving the algebraic Riccati equation for P using Kronecker products and vec-
torisation given matrices A, B, Q and R. . . . . . . . . . . . . . . . . . . . . . . . . 431
9.13 Calculate the discrete optimal steady-state gain by iterating until exhaustion. Note
it is preferable for numerical reasons to use lqr for this computation. . . . . . . . . 433
9.14 Comparing the continuous and discrete LQR controllers. . . . . . . . . . . . . . . . 434
9.15 An LQR controller for the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
9.16 Comparing an LQR controller from Listing 9.15 with a pole-placement controller . 438
9.17 Computing the closed loop poles from the optimal LQR controller from Listing 9.15. 440
9.18 Tracking LQR controller with integral states . . . . . . . . . . . . . . . . . . . . . . . 442
9.19 Comparing the actual normally distributed random numbers with the theoretical
probability density function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
9.20 Generating correlated noise sequences with a known covariance, Q. . . . . . . . . . 448
9.21 Probability and inverse probability distributions for the F -distribution. . . . . . . . 449
9.22 Generate some correlated random data. . . . . . . . . . . . . . . . . . . . . . . . . . 449
9.23 Plot a 3D histogram of the random data from Listing 9.22. . . . . . . . . . . . . . . 450
9.24 Compute the uncertainty regions from the random data from Listing 9.23. . . . . . 450
9.25 Validating the uncertainty regions computed theoretically from Listing 9.24. . . . . 451
9.26 Solving the discrete time Riccati equation using exhaustive iteration around Eqn. 9.106
or alternatively using the dare routine. . . . . . . . . . . . . . . . . . . . . . . . . . 458
9.27 Alternative ways to compute the Kalman gain . . . . . . . . . . . . . . . . . . . . . 460
9.28 State estimation of a randomly generated discrete model using a Kalman filter. . . 463
9.29 Computing the Kalman gain using dlqe. . . . . . . . . . . . . . . . . . . . . . . . . 464
9.30 Demonstrating the optimality of the Kalman filter. . . . . . . . . . . . . . . . . . . . 465
9.31 Potters algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469
10.1 Predictive control with input saturation constraints using a generic nonlinear opti-
miser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
10.2 Objective function to be minimised for the predictive control algorithm with input
saturation constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
10.3 Dynamic Matric Control (DMC) control . . . . . . . . . . . . . . . . . . . . . . . . . 488
10.4 Setting up an MPC controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490
10.5 Nonlinear dynamics of a helicopter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495
10.6 Optimal control using linear programming . . . . . . . . . . . . . . . . . . . . . . . 502
B.1 Polynomial addition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
B.2 Convolution (polynomial multiplication) of two or more polynomials. . . . . . . . 509
B.3 Strip leading zeros from a polynomial. . . . . . . . . . . . . . . . . . . . . . . . . . . 510
xxii LISTINGS
C HAPTER 1

I NTRODUCTION

Mathematicians may flatter themselves that they posses new ideas which mere human language is as yet
unable to express. Let them make the effort to express those ideas in appropriate words without the aid of
symbols, and if they succeed, they will not only lay us laymen under a lasting obligation but, we venture to
say, they will find themselves very much enlightened during the process, and will even be doubtful
whether the ideas expressed as symbols had ever quite found their way out of the equations into their
minds.
James Clerk Maxwell, 1890

Control, in an engineering sense, is where actions are taken to ensure that a particular physi-
cal process responds in some desired manner. Automatic control is where we have relieved the
human operator from the tedium of consistently monitoring the process and supplying the nec-
essary corrections. Control as a technical discipline is therefore important not only in the fields
of engineering, but also in economics, sociology and indeed in most aspects of our life. When
studying control, we naturally assume that we do conceivably have some chance of influencing
things. For example, it is worthwhile to study the operation of a coal fired power plant in order
to minimise possibly polluting emissions, but it is not worth our time to save the world from the
next ice age, or as the results of a special study group who investigated methods designed to pro-
tect the world from a stray comet (such as the one postulated to have wiped out the dinosaurs 80
million years ago) concluded, there was nothing feasible we could do, such as change the earths
orbit, or blast the asteroid, to avoid the collision. In these latter examples, the problem exists, but
our influence is negligible.

The teaching of control has changed in emphasis over the last decade from that of linear single-
input/single-output systems elegantly described in the Laplace domain, to general nonlinear
multiple-input/multiple-output systems best analysed in the state space domain. This change
has been motivated by the increasing demands by industry and public to produce more, faster or
cleaner and is now much more attractive due to the impressive improvements in computer aided
tools, such as M ATLAB used in these notes. This new emphasis is called advanced (or modern)
control as opposed to the traditional or classical control. This set of notes is intended for students
who have previously attended a first course in automatic control covering the usual continuous
control concepts like Laplace transforms, Bode diagrams, stability of linear differential equations,
PID controllers, and perhaps some exposure to discrete time control topics like z transforms and
state space.

This book attempts to describe what advanced control is, and how it is applied in engineering
applications with emphasis on the construction of controllers using computer aided design tools
such as the numerical programming environment M ATLAB from the MathWorks, [138]. With
this tool, we can concentrate on the intentions behind the design procedures, rather than the
1
2 CHAPTER 1. INTRODUCTION

mechanics to follow them.

Part one contains some revision material in z-transforms, modelling and PID controller tuning.
The discrete domain, z-transforms, and stability concepts with a brief discussion of appropriate
numerical methods are introduced in chapter 2. A brief potpourri of modelling is summarised
in chapter 3. Chapter 4 is devoted to the most common industrial controller, the three term PID
controller with emphasis on tuning, implementation and limitations. Some basic concepts from
signal processing such as filtering and smoothing are introduced in chapter 5. Identification and
the closely related adaptive control are together in chapters 6 and 7. State space analysis and
optimal control design are given in the later chapters 8 and 9.

1.1 N OTATION AND COLOUR CONVENTIONS

Throughout these notes I have used some typographical and colour conventions. In mathematical
expressions, scalar variables are written in italic such as a, b, c, or if Greek, , while vectors x, y
are upright bold lower case and matrices, A, are bold upper case. More notation is introduced
as required.

M ATLAB instructions to be typed in as a command are given in a fixed-width font as A=chol(B*B)


and it is important to carefully note the typographical difference between similar symbols such
as the number zero, 0 and the capital letter O, the number one (1) and the letter l, and the Greek
letters , and the English italic letters v and p.

A section of M ATLAB code and the associated output is given as

1
1 >> G = pole(tf(1,[1 3])) % Find the poles of G(s) = s+3
.
G =
-3

However in many cases the output is not explicitly given and the M ATLAB prompt is omitted.

The book is primarily concerned with control loops for which we will use, where practical, a
consistent colour convention. The inputs or manipulated variables to the plant are coloured blue,
while the plant outputs or measurements are coloured red. A suitable mnemonic is to think of
a simple domestic hot water heater where the industrial plant being dirty brown heats up the
incoming input cold water (blue) to produce an output hot water (red) as shown in Fig. 1.1.

Input u Plant output, y

cold water Domestic hot water


Hot water heater

Figure 1.1: Standard colour conventions used for inputs (blue) and outputs (red) in this book.

The default colouring convention for a closed loop control system is shown in Fig. 1.2. Here we
colour the reference or setpoint in red (or pink), and disturbing signals such as noise (perhaps
arising from nature) are coloured green.

Fig. 1.3 shows an example of a typical controlled response given in this book. The top plot shows
the output in red, with the setpoint in dotted pink. The lower trend shows the manipulated
input in blue. Other information such as the constraint bounds for the manipulated variable are
overlayed in grey.
1.2. MATLAB FOR COMPUTER AIDED CONTROL DESIGN 3

noise, e

error
e u
setpoint, r + controller Plant b output, y

Figure 1.2: Standard colour conventions used for closed loop block diagrams.

setpoint
1
output y(t)

0.5

0 y(t)

1
Input u(t)

0.5

0 discrete input
Figure 1.3: The standard colour and plot lay-
0 10 20 30 40 out for a controlled response used in this
time book.

1.2 M ATLAB FOR COMPUTER AIDED CONTROL DESIGN

Modern control design has heavy computing requirements. In particular one needs to:

1. manipulate symbolic algebraic expressions, and

2. perform intensive numerical calculations and simulations for proto-typing and testing quickly
and reliably, and finally

3. to implement the controller at high speed in special hardware such as an embedded con-
troller or a digital signal processing (DSP) chip perhaps using assembler.

To use this new theory, it is essential to use computer aided design (CAD) tools efficiently as real-
world problems can rarely be solved manually. But as [173] point out, the use of computers in the
design of control systems has a long and fairly distinguished history. This book uses M ATLAB
for the design, simulation and prototyping of controllers.

M ATLAB, (which is short for MATrix LABoratory), is a programming environment that grew out
of an effort to create an easy user-interface to the very popular and well regarded public domain
F ORTRAN linear algebra collection of programmes, L INPACK and E ISPACK. With this direct inter-
pretive interface, one can write quite sophisticated algorithms in a very high level language, that
are consequently easy to read and maintain. M ATLAB is supported with a variety of toolboxes
4 CHAPTER 1. INTRODUCTION

comprising of collections of source code subroutines organised in areas of specific interest. The
toolboxes we are most interested in, and used in this book are:

Control toolbox containing functions for controller design, frequency domain analysis, conver-
sions between various models forms, pole placement, optimal control etc. (Used through-
out)
Symbolic toolbox which contains a gateway to the symbolic capabilities of M APLE.
Signal processing toolbox containing filters, wave form generation and spectral analysis. (Used
principally in chapter 5.)
System identification toolbox for identifying the parameters of various dynamic model types.
(Used in chapter 6.) You may also find the following free statistics toolbox useful available
at: www.maths.lth.se/matstat/stixbox/, or the R package from [165].
Optimisation which contains routines to maximise or minimise algebraic or dynamic functions.
One suitable free toolbox is available from www.i2c2.aut.ac.nz/Wiki/OPTI/. This
toolbox is used primarily in chapters 9 and 10.
Real-time toolbox can be used to interface M ATLAB to various analogue to digital converters. A
free version suitable with Measurement Computings MCC DAQs is available from
www.i2c2.aut.ac.nz/Resources/Software/SysIDToolbox.html.

Additional documentation to that supplied with M ATLAB is the concise and free summary notes
[187] or the more recent [69]. Recently there has been exponential growth of other texts that heav-
ily use M ATLAB (such as this one), and a current list is available from the Mathworks anonymous
ftp server at www.mathworks.com. This server also contains many user contributed codes, as
well as updates, bug fixes etc.

If M ATLAB, or even programming a high level language is new to you, then [206] is a cheap
recommended compendium, similar in form to this, covering topics in numerical analysis, again
with many M ATLAB examples.

1.2.1 A LTERNATIVE COMPUTER DESIGN AIDS

Table 1.1 lists a number of alternatives computer-aided design and modelling environments sim-
ilar and complimentary to M ATLAB.

Product WWW site comment


S CI L AB www.scilab.org Free Matlab/Simulink clone
P YTHON code.google.com/p/pythonxy Free programming environment
R www.r-project.org/ Free statistical computing
O CTAVE www.octave.org Free Matlab clone, inactive
RL AB rlabplus.sourceforge.net Matlab clone, Linux
V ISUAL M ODEL Q www.qxdesign.com shareware Simulink clone
M U PAD www.mupad.de Interfaces with S CI L AB
M APLE www.maplesoft.com commercial CAS
M ATHEMATICA www.mathematica.com commercial CAS

Table 1.1: Shareware or freeware Matlab lookalikes and computer algebra systems

Unlike M ATLAB, symbolic manipulators are computer programs that by manipulating symbols
can perform algebra. Such programs are alternatively known as computer algebra systems or
1.3. LABORATORY EQUIPMENT FOR CONTROL TESTS 5

CAS. The most well known examples are M ATHEMATICA, M APLE, M U PAD, and M ACSYMA, (see
Table 1.1). These programs can find analytical solutions to many mathematical problems involv-
ing integrals, limits, special functions and so forth. They are particularly useful in the controller
design stage.

The Numerics in Control1 group in Europe have collect together a freeware F ORTRAN subroutine
library S LICOT for routines relevant in systems and control.

Problem 1.1 1. Familiarise yourself with the fundamentals of M ATLAB. Run the M ATLAB
demo by typing demo once inside M ATLAB.
2. Try the M ATLAB tutorial (part 1).
3. Read through the M ATLAB primer, [187] or [69], and you should get acquainted with the
M ATLAB users manual.

1.3 L ABORATORY EQUIPMENT FOR CONTROL TESTS

Obviously if we are to study automatic control with the aim to control eventually chemical plants,
manufacturing processes, robots, undertake filtering to do active noise cancellation and so forth,
we should practice, preferably on simpler, more well understood, and potentially less hazardous
equipment.

In the Automatic Control Laboratory in the Department of Electrical Engineering at the Karlstad
University, Sweden we have a number of simple bench-scale plants to test identification and
control algorithms on.

1.3.1 P LANTS WITH ONE INPUT AND ONE OUTPUT

T HE BLACKBOX

Fig. 1.4 and Fig. 1.5(a) shows what we perhaps unimaginatively refer to as a black-box. It is a
box, and it is coloured black. Subjecting the box to an input voltage from 0 to 5 volts delivers an
output voltage also spanning from around 0 to 5 volts, but lagging behind the input voltage since
the internals of the blackbox are simply either 7 or 9 (depending on the switch position) low-pass
passive filters cascaded together.

The blackbox is a relatively well behaved underdamped stable system with dominant time con-
stants of around 5 to 10 seconds. Fig. 1.5(b) shows the response of the blackbox to two input steps.
The chief disadvantage of using this device for control studies is that the output response is not
visible to the naked eye, and that we cannot manually introduce disturbances. One complication
you can do is to cascade two blackboxes together to modify the dynamics.

E LECTRO - MAGNETIC BALANCE ARM

The electromagnetic balance arm shown in Fig. 1.6(a) is a fast-acting, highly oscillatory, plant
with little noise. The aim is to accurately weigh small samples by measuring the current required
to keep the balance arm level, or alternatively just to position the arm at different angles. The
output response to a step in input shown in Fig. 1.6(b) indicates how long it would take for the
oscillations to die away.
1 The home page is located at http://www.win.tue.nl/niconet/niconet.html
6 CHAPTER 1. INTRODUCTION

Input Output
From computer B LACK B OX To computer
sluggish
D/A A/D

Fast
Input indicator

GND

Figure 1.4: Blackbox configuration. The manual switch marked will toggle between either 7 or 9
low-pass filters.

Blackbox step response

0.45

0.4

0.35

0.3
ipnut/output

0.25

0.2

0.15 input
output
0.1

0.05

0.05
15 20 25 30 35 40 45 50
time (s)

(a) Black-box wiring to the National Instruments (b) The response of the blackbox to 2 step inputs
LabPC terminator.

Figure 1.5: The Black-box

Step response of the balance arm


1

0.8

0.6
arm position

0.4

0.2

0.2

0.4
20 30 40 50 60 70 80 90 100

0.3

0.2

0.1
input

0.1

0.2

20 30 40 50 60 70 80 90 100
time T=0.05 seconds

(a) The electromagnetic balance arm (b) The response of the arm to a step changes in input.

Figure 1.6: The electromagnetic balance arm


1.3. LABORATORY EQUIPMENT FOR CONTROL TESTS 7

FAN AND F LAPPER

Contrary to the balance arm, the fan and flapper in Fig. 1.7 and Fig. 1.8(b) has few dynamics,
but significant low-pass filtered measurement noise. The counter weight can be moved along the
lever arm to mimic a disturbance variable. An interesting exercise is to place two flapper units
in close proximity. The air from one then disturbs the flapper from the other which makes an
interacting multivariable plant.

adjustable counter weight

angle transducer

output angle

Input, u(t) Motor

Fan

Figure 1.7: The fan and flapper with an adjustable counter weight as a disturbance

S TEPPER MOTORS

A stepper motor is an example of a totally discrete system.

1.3.2 M ULTI - INPUT AND MULTI - OUTPUT PLANTS

It is possible to construct multivariable interacting plants by physically locating two plants close
to each other. One possibility is to locate two flappers adjacent to each other, another possibility
is one flapper and one balance arm. The extent of the interaction can be varied by adjusting the
relative position of the two plants.

H ELICOPTER

The model helicopter, Fig. 1.9, is an example of a highly unstable, multivariable (3 inputs, 2 out-
puts) nonlinear, strongly interacting plant. It is a good example of where we must apply control
(or crash and burn). Fig. 1.10(b) shows the controlled response using 2 PID controllers to control
the direction and altitude. Fig. 1.11(a) shows a 3 dimensional view of the desired and actual flight
path. Section 10.2.4 illustrates the control of a different three degree of freedom helicopter.
8 CHAPTER 1. INTRODUCTION

Step response of the flapper


0.35

0.3

0.25

output
0.2

0.15

0.1
0 5 10 15 20 25 30 35 40 45 50

0.4

0.3
input
0.2

0.1

0.1
0 5 10 15 20 25 30 35 40 45 50
time (sec) dt=0.05

(a) The fan/flapper equipment (b) Step response of the flapper

Figure 1.8: The Fan and flapper.

top rotor
side rotor

moveable counter weight


plant outputs

elevation angle

azimuth angle

support stand

Figure 1.9: Helicopter plant with 2 degrees of freedom. See also Fig. 1.10(a).

1.3.3 S LOWING DOWN S IMULINK

For some applications like the development of PID controllers you want to be able to slow down
the S IMULINK simulation to have time to manually introduce step changes, add disturbances,
switch from automatic to manual etc. If left alone in simulation mode, S IMULINK will run as fast
as possible, but it will slow down when it needs to sub-sample the integrator around discontinu-
ities or periods when the system is very stiff.

The S IMULINK Execution Control block allows you to specify that the simulation runs at a multi-
ple of real-time. This is most useful when you want to slow down a simulation, or ensure that it
runs at a constant rate.
1.3. LABORATORY EQUIPMENT FOR CONTROL TESTS 9

(a) The flying helicopter balanced using 2 PID controllers since it is openloop unstable and would otherwise crash.

Helicopter
1.5

0.5
output

0.5

1.5
0 10 20 30 40 50 60 70 80 90 100

0.5
input

0.5

1
0 10 20 30 40 50 60 70 80 90 100
Time (sec)
(b) Multivariable PID control of the helicopter exhibiting mediocre controlled response and severe
derivative kick.

Figure 1.10: Multivariable PID control of an unstable helicopter


10 CHAPTER 1. INTRODUCTION

Helicopter flight path

0.5

0
Up/Down

0.5

1
1.5
1
0.5
0 250
200
0.5
150
1 100
50
East/West 1.5 0
time

(a) Multivariable helicopter control (b) Model helicopter in the trees

Figure 1.11: Helicopter flying results

The block is available from: http://www.mathworks.com/matlabcentral/fileexchange

The implementation is shown in Fig. 1.12 where the Simulation Execution Control block looks
like an A/D card but in fact is not connected to anything, although it does export some diagnostic
timing information.

Scope1
Simulink
Execution
Control

1
s+1
Signal Transfer Fcn Scope
Generator

Figure 1.12: Real-time Simulink simulations. A parameter in the S IMULINK E XECUTION C ON -


TROL block sets the speed of the simulation.

An alternative method to slow down a Simulink simulation and force it to run at some set rate is
to use the commercial Humusoft real-time toolbox, but not with the A/D card actually interfaced
to anything.

1.4 E CONOMICS OF CONTROL

Most people would agree that Engineers apply technology, but what do these two words really
mean? Technology is derived from two Greek words, techne which means skill or art, and logia
which means science or study. The interesting point here is that the art component is included.
The English language unfortunately confuses the word engine with engineering so that many peo-
ple have the mistaken view that engineers drive engines (mostly). Actually engineer is derived
from the Latin ingeniatorium which means one who is ingenious at devising. A far cry from the
relatively simple act of piloting jumbos. An interesting American perspective of the professional
engineer and modern technology is given as light reading in [2] and Flormans The Existential
1.4. ECONOMICS OF CONTROL 11

Pleasures of Engineering, [67].

Chemical engineering is, succinctly put, chemistry constrained by cost. The chemist wants the
reaction to proceed, the chemical engineer takes that for granted, but is interested in increasing
the rate, or pushing the equilibrium, or in most cases both at the same time. As the incentive to
produce better products increases, accompanied by an awareness of potent global competition
driving one to reduce costs, process control becomes an important aspect of engineering.

Obviously modern computerised process control systems are expensive. They are especially ex-
pensive compared with other computers such as office or financial computers because the market
is smaller, the environment harsher, the graphical requirements more critical, the duties more var-
ied, and the potential payoffs larger. Process control has at least two main duties to perform; first
to ensure that the plant is operated safely (that is protect the plant, environment and people), and
second that the product quality is consistent with some customer or regulatory body demanded
specifications. There is always a trade off between how much control you apply and the bene-
fits that result. An automobile manufacturer could produce an almost totally indestructible car
(ie, tank), but the expense of raw materials required, and the high running costs would certainly
deem the project an economic failure.

On the other hand, in 1965 Ralph Nader complained in the aptly named Unsafe at Any Speed about
the poor quality of American automotive engineering, the lack of controls and the unsafe result.
This influential book challenged the balance from between commercial profits and more quality
control. A product with less variation in quality may be worth more than a product that has
a higher average quality, but more variation. A potentially dangerous production facility that
regularly destroys equipment, people or surroundings is not usually tolerated by the licensing
authorities.

Safety concerns motivate


better control.

Fig. 1.13, adapted from [7], gives an industrial perspective of the status of process control in 1994.
The techniques are divided into those considered classical or traditional, which demand only
modest digital online computing power, (if any), little in the way of explicit process models or
understanding, and those termed loosely advanced.

One of the major concerns for the process control engineer is to reduce the output variance. If
the variation about the setpoint is small, then the setpoint can be shifted closer to the operating
constraint, without increasing the frequency of alarms. Fig. 1.14 demonstrates the ideal case that
while popular in the advertising literature, is harder to achieve unambiguously in practice.

Many text books in the control field are very vague about the actual configuration of real process
control systems used today. Other books that take a more trade oriented approach, are vague
about the academic side of the control performance. There are a number of reasons for this. First
many texts try hard to describe only the theoretical aspects of digital control, and anything re-
motely applied is not considered worthy of their attention. Secondly, the control systems are
rapidly changing as the cost of micro-processors drop in price, and different programming meth-
ods come into flavour. Thirdly many industries are deliberately vague about publishing the de-
tails of their control system since they perceive that this information could help their competitors.
12 CHAPTER 1. INTRODUCTION

Advanced control

Optimisation direct search rule definition online simulation Model Predictive Control

Constraint steady state dynamic methods single variable multivariate control

Regulatory Signal condition PID Deadtime compensation feedforward control

Basic Analog control PLCs DCS process computer

Field Valves transmitters smart transmitters Onstream analysers

Traditional control

Figure 1.13: A comparison of traditional vs. advanced process control techniques. Adapted from
[7].

process output violations


upper quality constraint

setpoint #3
loss ($)

setpoint #1 advanced control

regulatory control

manual control
time

Figure 1.14: Economic improvements owing to better control. If the control scheme can reduce
the variance, the setpoint can be shifted closer to the operating or quality constraint, thereby
decreasing operating costs.

However some information of this type is given in [64, pp131-149] and [18]. One good balance
for the practitioner is [147].
C HAPTER 2

F ROM DIFFERENTIAL TO DIFFERENCE


EQUATIONS

Excerpt from What a sorry state of affairs, Martin Walker, Guardian Weekly, June 29, 1997.

. . . But then he said something sensible, as he quite often does. . . .

We need to treat individuals as individuals and we need to address discrete problems for what they are,
and not presume them to be part of some intractable racial issue. Gingrich, properly understood, is a
national treasure, and not only because he is one of the few Americans who understand the difference
between discreet and discrete.

2.1 C OMPUTER IN THE LOOP

The cost and flexibility advantages of implementing a control scheme in software rather than
fabricating it in discrete components today are simply too large to ignore. However by inserting a
computer to run the software necessitates that we work with discrete regularly sampled signals.
This added complexity, which by the way is more than compensated by the above mentioned
advantages, introduces a whole new control discipline, that of discrete control.

Fig. 2.1 shows a common configuration of the computer in the loop. For the computer to respond
to any outside events, the signals must first be converted from an analogue form to a digital
signal, say 1 to 5 Volts, which can, with suitable processing be wired to an input port of the
computers processor. This device that accomplishes this conversion is called an Analogue to
Digital converter or A/D. Similarly, any binary output from the pins on a processor must first be
converted to an analogue signal using a Digital to Analogue converter, or D/A. In some micro-
controllers, (rather than micro-processors), such as Intels 8048 or some versions of the 8051, these
converters may be implemented on the microcontroller chip.

Digital to analogue conversion is easy and cheap. One simply loads each bit across different re-
sistors, and sums the resultant voltages. The conversion is essentially instantaneous. Analogue
to digital is not nearly as easy nor cheap, and this is the reason that the common data acquisition
cards you can purchase for your PC will often multiplex the analogue input channel. There are
various schemes for the A/D, one using a D/A inside a loop using a binary search algorithm. Ob-
viously this conversion is not instantaneous, although this is not normally considered a problem
for process control applications. Any introductory electrical engineering text such as [174] will
give further details on the implementation details.
13
14 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Process control computer

setpoint, r(t)
+ A/D Computer D/A Plant output, y(t)

Figure 2.1: The computer in the control loop.

2.1.1 S AMPLING AN ANALOGUE SIGNAL

It is the A/D converter that is the most interesting for our analysis of the discrete control loop. The
A/D converter will at periodic intervals defined by the computers clock sample the continuous
analogue input signal. The value obtained is typically stored in a device called a zeroth-order
hold until it is eventually replaced by the new sample collected one sample time later. Given
constraints on cost, the A/D converter will only have a limited precision, or limited number of
bits in which to store the incoming discrete value. Common A/D cards such as the PCLabs card,
[3], use 12 bits giving 212 or slightly over 4 thousand discretisation levels. The residual chopped
away is referred to as the quantisation error. For a given number of bits, b, used in the converter,
the amplitude quantisation is
= 2b
Low cost analogue converters may only use 8 bits, while digital audio equipment use between 16
and 18 bits.

Fig. 2.2 shows the steps in sampling an analogue signal with a three bit (8 discrete levels), A/D
sampler. The dashed stair plot gives an accurate representation of the sampled signal, but owing
to the quantisation error, we are left with the solid stair plot. You can reproduce Fig. 2.2 in M AT-
LAB using the fix command to do the chopping, and stairs to construct the stair plot. While

Sampling a continuous signal


8
Analogue
7 Sampled
quantisized
6

4
Signal

3
Figure 2.2: Sampling an analogue signal 2
(heavy solid) with a three bit (8 discrete lev- 1
els) A/D converter and zeroth-order hold. 0
The sampled values, , are chopped to the 1
next lowest discrete integer level giving the 2
0 2 4 6 8 10 12
sampled and quantisied output Sample time, (kT)

other types of hold are possible, anything higher than a first-order hold is rarely used.

2.1.2 S ELECTING A SAMPLE RATE

Once we have decided to implement discrete control, rather than continuous control, we must
decided on a reasonable sampling rate. This is a crucial parameter in discrete control systems.
2.1. COMPUTER IN THE LOOP 15

The sample time, T , (or sometimes denoted t ), is measured in time units, say seconds or in
industrial applications, minutes. The reciprocal of the sample time is the sample frequency, f ,
and is usually measured in cycles/second or Hertz. The radial or angular velocity (which some
confusingly also term frequency) is denoted and is measured in radians/second. The inter-
relationships between these quantities are:
   
1 cycles s radians/s
fs = = (2.1)
T second 2 radians/cycle
The faster the sampling rate, (the smaller the sampling time, T ), the better our discretised signal
approximates the real continuous signal. However, it is uneconomic to sample too fast, as the
computing and memory hardware may become too expensive. When selecting an appropriate
sampling interval, or sample rate, we should consider the following issues:

The maximum frequency of interest in the underlying signal


The sampling theorem which specifies a lower limit required on the sampling rate to resolve
any particular frequency unambiguously. (See 2.1.3 following.)
Any analogue filtering that may be required (to reduce the problem of aliasing)
The cost of the hardware and the speed of the A/D converters.

Ogata discusses the selection of a sample time qualitatively in [152, p38]. However for most
chemical engineering processes, which are dominated by relatively slow and overdamped pro-
cesses, the sample time should lie somewhere between ten times the computational delay of the
hardware tc and some small fraction of the process dominant time constant , say

10tc T (2.2)
10
For most chemical engineering applications, the computational delay is negligible compared with
the process time constant, (tc 0), so we often choose T /10. Thus for a simple first order
rise, we would expect to have about 2030 data samples from 0 to 99%. Some may argue that
even this sampling rate is too high, and opt for a more conservative (larger) sample time down
to /6. Note that commonly used guidelines such as presented in Table 22.1 in [183, p535] span a
wide range of recommended sample times.

O VERLY FAST SAMPLING

Apart from the high cost of fast A/D converters, there is another argument against fast sampling.
When one samples a continuous system, the poles of a stable continuous system map to the poles
of a stable discrete system as T goes to zero. However the zeros in the LHP of a continuous
system may not map to zeros inside the unit disk of a discrete system as T tends to zero. This
nonintuitive result could create problems if one samples a system, and then uses the inverse of this
system within a one step controller, since now the zeros outside the unit circle become unstable
poles inside the controller for small sample times. Note that the inverse continuous system is
stable, and the discrete inverse system will be stable for large sample times, and it will only be
unstable for small sample times.

2.1.3 T HE SAMPLING THEOREM AND ALIASES

Dinsdale was a gentleman. And whats more he knew how to treat a female impersonator
John Cleese in Monty Python (15-9-1970)
16 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

The sampling theorem gives the conditions necessary to ensure that the inverse sampling pro-
cedure is a one to one relationship. To demonstrate the potential problems when sampling,
consider the case where we have two sinusoidal signals, but at different frequencies.1
   
7 1
y1 = sin 2 t and y2 = sin 2 t
8 8
If we sample these two signals relatively rapidly, say at 25Hz (T =40ms) then we can easily see
two distinct sine curves. However if we sample at T = 1, we obtain identical points.

Aliasing
1

t = [0:0.04:8]'; 0.5
2 y1= -sin(2*pi*7/8*t);
signal

y2= sin(2*pi/8*t);
plot(t,[y1, y2]) 0

As shown opposite, we note that y1 0.5


(solid) and y2 (dashed) just happen to
coincide at t = 0, 1, 2, 3, . . . (). 1
0 2 4 6 8
time (s)

Consequently, at the slower sampling rate of 1 Hz, we cannot distinguish between the two dif-
ferent signals. This is the phenomenon known as aliasing since one of the frequencies pretends
to be another. In conclusion, two specific sinusoids of different frequencies can have identical
sampled signals. Thus in the act of taking samples from a continuous measurement, we have lost
some information. Since we have experienced a problem when we sample too slowly, it is reason-
able to ask what the minimum rate is so that no aliasing occurs. This question is answered by the
sampling theorem which states:

To recover a signal from its sample, one must sample at least two times a period, or
alternatively sample at a rate twice the highest frequency of interest in the signal,
fs > f. (2.3)

Alternatively, the highest frequency we can unambiguously reconstruct for a given sampling rate,
1/T , is half of this, or 1/(2T ). This is called the Nyquist frequency, fN .

In the second example above when we sampled at 1 Hz, the sampling radial velocity was s =
2 = 6.28 rad/s. This was satisfactory to reconstruct the low frequency signal (f1 = 1/8 Hz) since
21 = 1.58 rad/s. We are sampling faster than this minimum, so we can reconstruct this signal.
However for the faster signal (f2 = 7/8 Hz), we cannot reconstruct this signal since 22 = 11.0
rad/s, which is faster than the sampling radial velocity.

2.1.4 D ISCRETE FREQUENCY

If we sample the continuous signal (which comprises of a single frequency f = 2),


x(t) = A cos(t)
with a sample time of T ,
x(kT ) = A cos(kT ) = A cos(k)
1 This example was adapted from Franklin & Powell p81
2.1. COMPUTER IN THE LOOP 17

where the digital frequency, , is defined as

def f
= T = 2f T = 2
fs

then the range of analogue frequencies is 0 < f < , while the range of digital frequencies is
limited by the Nyquist sampling limit, fs /2 giving the allowable range for the digital frequency
as
0

M ATLAB unfortunately decided on a slightly different standard in the S IGNAL P ROCESSING tool-
box. Instead of a range from zero to , M ATLAB uses a range from zero to 2 where 1 corresponds
to half the sampling frequency or the Nyquist frequency. See [44].

In summary:

symbol units Range


sample time T or t s 0
sampling frequency fs = T1 Hz 0
angular velocity = 2f rad/s 0
digital frequency = T = 2ffs dim. 0

fs 1
Nyquist frequency fN = = Hz
2 2T
2fN
dim Nyquist frequency N = = dim.
fs

It is practically impossible to avoid aliasing problems when sampling, using only digital filters.
Almost all measured signals are corrupted by noise, and this noise has usually some high or even
infinite frequency components. Thus the noise is not band limited. With this noise, no matter
how fast we sample, we will always have some reflection of a higher frequency component that
appears as an impostor or alias frequency.

If aliasing is still a problem, and you cannot sample at a higher rate, then you can insert a low
pass analogue filter between the measurement and the analogue to digital converter (sampler).
The analogue filter or in this case known as an anti-aliasing filter, will band-limit the signal, but
not corrupt it with any aliasing. Expensive high fidelity audio equipment will still use analogue
filters in this capacity. Analogue and digital filters are discussed in more detail in chapter 5.

D ETECTING ALIASES

Consider the trend y(t) in Fig. 2.3 where we wish to estimate the important frequency components
of the signal. It is evident that y(t) is comprised of one or two dominating harmonics.

The spectral density when sampling at T = 0.7s ( in Fig.2.3) given in the upper trend of Fig. 2.4
exhibits three distinct peaks. These peaks are the principle frequency components of the sig-
nal and are obtained by plotting the absolute value of the Fourier transform of the time signal2 ,
|DF T {y(t)}|. Reading off the peak positions, and for the moment overlooking any potential
problems with undersampling, we would expect y(t) to be something like

y(t) sin(20.1t) + sin(20.5t) + sin(20.63t)


2 More about the spectral analysis or the power spectral density of signals comes in chapter 5.
18 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

3
continuous
Ts=0.7
2
Ts=1.05

output
0

Figure 2.3: Part of a noisy time series with


1
unknown frequency components. The con-
tinuous underlying signal (solid) is sam- 2
0 1 2 3 4 5 6 7
pled at T = 0.7, (), and at T = 1.05 s (). time (s)

2
10 Ts = 0.7 s
power

0
10
fN=0.71

2
10 Ts = 1.05 s
power

0
10
fN=0.48

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8


frequency (Hz)

Figure 2.4: The frequency component of a signal sampled at Ts = 0.7s (upper) and Ts = 1.05s
(lower). The Nyquist frequencies for both cases are shown as vertical dashed lines. See also
Fig. 2.5.

However in order to construct Fig. 2.4 we had to sample the original time series y(t) possibly
introducing spurious frequency content. The Nyquist frequency fN = 1/(2Ts ) is 0.7143 and is
shown as a vertical dashed line in Fig. 2.4(top). The power spectrum is reflected in this line, but
is not shown in Fig. 2.4.

If we were to re-sample the process at a different frequency and re-plot the power density plot
then the frequencies that were aliased will move in this second plot. The points in Fig. 2.3 are
sampled at t = 1.05s with corresponding spectral power plot in Fig. 2.4. The important data
from Fig. 2.4 is repeated below.

Curve Ts (s) fN (Hz) peak 1 peak 2 peak 3


top 0.7 0.7143 0.1 0.50 0.63
bottom 1.05 0.4762 0.1 0.152 0.452

First we note that the low frequency peak (f1 = 0.1Hz) has not shifted from curve a (top) to curve
b (bottom), so we would be reasonably confident that f1 = 0.1Hz and is not corrupted by the
sampling process.

However, the other two peaks have shifted, and this shift must be due to the sampling process.
Let us hypothesize that f2 = 0.5Hz. If this is the case, then it will appear as an alias in curve b
since the Nyquist frequency for curve b (fN (b) = 0.48) is less than the proposed f2 = 0.5, but it
will appear in the correct position on curve a. The apparent frequency f2 (b) on curve b will be

f2 (b) = 2fN (b) f2 = 2 0.4762 0.5 = 0.4524


2.2. FINITE DIFFERENCE MODELS 19

which corresponds to the third peak on curve b. This would seem to indicate that our hypothesis
is correct for f2 . Fig. 2.5 shows this reflection.

2
10 T = 0.7 s
s
power

0
10
f =0.71
N

2
10 T = 1.05 s
s
Figure 2.5: Reflecting the frequency re-
power

sponse in the Nyquist frequency from


0
10 Fig. 2.4 shows why one of the peaks is evi-
f =0.48
N
dent in the frequency spectrum at Ts = 1.05,
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
frequency (Hz) but what about the other peak?

Now we turn our attention to the final peak. f3 (a) = 0.63 and f2 (b) = 0.152. Let us again try
the fact that f3 (a) is the true frequency = 0.63Hz. If this were the case the apparent frequency
on curve b would be f3 (b) = 2fN (b) f2 = 0.3224Hz. There is no peak on curve b at this
frequency so our guess is probably wrong. Let us suppose that the peak at f3 (a) is the first
harmonic. In that case the true frequency will be f3 = 2fN (a) f3 (a) = 0.8Hz. Now we check
using curve b. If the true third frequency is 0.8Hz, then the apparent frequency on curve b will
be f3 (b) = 2fN (b) f3 = 0.153Hz. We have a peak here which indicates that our guess is a good
one. In summary, a reasonable guess for the unknown underlying function is

y(t) sin(20.1t) + sin(20.5t) + sin(20.8t)

although we can never be totally sure of the validity of this model. At best we could either re-
sample at a much higher frequency, say fs > 10Hz, or introduce an analogue low pass filter to cut
out, or at least substantially reduce, any high frequencies that may be reflected.

2.2 F INITE DIFFERENCE MODELS

To compute complex mathematical operations such as differentiation or integration, on a digital


computer, we must first discretise the continuous equation into a discrete form. Discretising is
where we convert a continuous time differential equation into a difference equation that is then
possible to solve on a digital computer.

In control applications, we are often required to solve


ordinary differential equations such as

dy
= f (t, y) (2.4)
dt
for y(t). The derivative can be approximated by the sim-
ple backward difference formula

dy yt ytT
(2.5)
dt T
where T is the step size in time or the sampling inter-
val. Provided T is kept sufficiently small, (but not zero)
Figure 2.6: In Nathaniel Hawthornes The
the discrete approximation to the continuous derivative Scarlet Letter, Hester Prynne is forced to
is reasonable. Inserting this approximation, Eqn. 2.5 wear a large, red A on her chest when
she is found guilty of adultery and refuses
to name the father of her illegitimate child.
20 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

into our original differential equation, Eqn. 2.4, and re-


arranging for yt results in the famous Euler backward
difference scheme;
yt = ytT + T f (t T, ytT ) (2.6)
Such a scheme is called a recurrence scheme since a previous solution, ytT , is used to calculate yt ,
and is suitable for computer implementation. The beauty of this crude method of differentiation
is the simplicity and versatility. Note that we can discretise almost any function, linear or non-
linear without needing to solve analytically the original differential equation. Of course whether
our approximation to the original problem is adequate is a different story and this complex, but
important issue is addressed in the field of numerical analysis.

Problem 2.1 1. What is the finite difference approximation for a third order derivative?
2. Write down the second order finite difference approximation to
d2 y dy
2 + 2 + y = ku
dt2 dt

2.2.1 D IFFERENCE EQUATIONS

Difference equations are the discrete counterpart to continuous differential equations. Often the
difference equations we are interested in are the result of the discretisation of a continuous dif-
ferential equation, but sometimes, as in the case below, we may be interested in the difference
equation in its own right. Difference equations are much easier to simulate and study in a digital
computer than a differential equation for example, since they are already written in a form where
we can just step along following a relation

xk+1 = f (xk )

rather than resorting to integration. This step by step procedure is also known as a mathematical
mapping since we map the old vector xk to a new vector xk+1 .

H ENON S CHAOTIC ATTRACTOR

The system of discrete equations known as Henons attractor,

xk+1 = (yk + 1) 1.4x2k , x0 = 1 (2.7)


yk+1 = 0.3xk , y0 = 1 (2.8)

is an interesting example of a discrete mapping which exhibits chaotic behaviour.

We will start from point x0 = (x0 , y0 ) = (1, 1) and investigate the subsequent behaviour using
S IMULINK. Since Henons attractor is a discrete system, we will use unit delays to shift back
from xk+1 xk and to better visualise the results we will plot an (x, y) phase plot. The S IMULINK
simulation is given in Fig. 2.7. Compare this S IMULINK construction (which I have drawn de-
liberately to flow from right to left) so that it matches the way we read the system equations,
Eqn. 2.7-Eqn. 2.8.

The time response of x and y, (left figures in Fig. 2.8), of this mapping is not particularly inter-
esting, but the result of the phase plot (without joining the dots), (right figure in Fig. 2.8), is an
interesting mapping.

Actually what the chaotic attractor means is not important at this stage, this is after all only a
demonstration of a difference equation. However the points to note in this exercise are that we
2.3. THE Z TRANSFORM 21

1.4
Product
Gain2

x 1/z 1
To Workspace x1 Sum1 Constant

XY Graph
y 1/z 0.3
To Workspace2 y1
Gain

Figure 2.7: Henons attractor, (Eqns 2.72.8), modelled in S IMULINK. See also Fig. 2.8.

2
1
1 Start point
k

0
x

1 0.5
yk

2
1

0.5 0
yk

0.5 0.5
0 5 10 15 20 25 30 35 1.5 1 0.5 0 0.5 1 1.5
time counter xk

(a) The time response (b) The (x, y) or phase response

Figure 2.8: The time response (left) and phase plot (right) of Henons attractor as computed by
the S IMULINK block diagram in Fig. 2.7.

never actually integrated any differential equations, but only stepped along using the unit delay
blocks in S IMULINK. Consequently these types of simulations are trivial to program, and run very
fast in a computer. Another example of a purely discrete system is given in later Fig. 6.10.

2.3 T HE z TRANSFORM

The sampling of a continuous function f (t) at regular sampling intervals equally spaced T time
units apart creates a new function f (t),

X
f (t) = f (kT ) (t kT ) (2.9)
n=0

where (x) is the Dirac delta function which is defined as being a perfect impulse at x = 0,
and zero elsewhere. Note that the actual value of (0) is infinity, but the integral of (t) is 1.0.
Expanding the summation term in Eqn 2.9 gives
f (t) = f0 (t) + fT (t T ) + f2T (t 2T ) + f3T (t 3T ) +
22 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Now suppose we wish to know the value of the third sampled function f (t = 3T ). For simplicity
we will assume the sample time is T = 1.

f (3) = f0 (3) + f1 (2) + f2 (1) + f3 (0) + f4 (1) +


| {z } | {z }
all zero all zero

All the terms except for the term containing (0) are zero, while (0) = . Thus the value of the
function f (3) = . Often you will see a graph of f (t) depicted where the heights of the values
at the sample times are the same as the height of the continuous distribution such as sketched in
Fig. 2.14. Strictly this is incorrect, as it is the strength or integral of f (t) which is the same as the
value of the continuous distribution f (t). I think of the function f (t) as a series of impulses
whose integral is equal to the value of the continuous function f (t).

If we take the Laplace transform of the sampled function f (t), we get


(
)
X

L {f (t)} = L fkT (t kT ) (2.10)
n=0

Now the function fkT is assumed constant so it can be factored out of the Laplace transform, and
the Laplace transform of the (0) is simply 1.0. If the impulse is delayed kT units, then the Laplace
transform is ekT 1. Thus Eqn. 2.10 simplifies to


X
X
L {f (t)} = fkT ekT s = fk z k
n=0 k=0

where we have defined


def
z = esT (2.11)

In summary, the z-transform of the sampled function f (t) is defined as


X
def
F (z) = Z {f (t)} = fk z k (2.12)
k=0

The function F (z) is an infinite series, but can be written in a closed form if f (t) is a rational
function.

2.3.1 z- TRANSFORMS OF COMMON FUNCTIONS

We can use the definition of the z-transform, Eqn. 2.12, to compute the z-transform of common
functions such as steps, ramps, sinusoids etc. In this way we can build for ourselves a table of
z-transforms such as those found in many mathematical or engineering handbooks.

S AMPLED STEP FUNCTION

The unit sampled step function is simply s(kT ) = 1, k 0. The z-transform of s(kT ) following
the definition of Eqn. 2.12 is


X
S(z) = kn z k = 1 + 1 z 1 + 1 z 2 + 1 z 3 + (2.13)
k=0
2.4. INVERSION OF Z -TRANSFORMS 23

By using the sum to infinity for a geometric series,3 we obtain the closed form equivalent for
Eqn. 2.13 as
1
S(z) = (2.14)
1 z 1

R AMP AND EXPONENTIAL FUNCTIONS

For the ramp function, x(k) = n for k 0, and the exponential function, x(k) = ean , k 0, we
could go through the same formal analytical procedure, but in this case we can use the ztrans
command from the symbolic toolbox in M ATLAB.

 z-transform
The of the exponential function,
The z-transform of the ramp function, Z {k}, Z eak is
using the Symbolic toolbox for M ATLAB is
1 >> syms k T a
>> ztrans(exp(-a*k*T))
1 >> syms k T
ans =
>> ztrans(k*T)
z/exp(-a*T)/(z/exp(-a*T)-1)
ans =
>> simplify(ans)
T*z/(z-1)2
6 ans =
z*exp(a*T)/(z*exp(a*T)-1)

In a similar manner we can build our own table of z-transforms of common functions.

F INAL AND INITIAL VALUE THEOREMS

Tables and properties of z-transforms of common functions are given in many handbooks and
texts such as [152, p49,67], but two of the more useful theorems are the initial value and the final
value theorems given in Table 2.1. As in the continuous case, the final value theorem is only
applicable if the transfer function is stable.

Table 2.1: Comparing final and initial value theorems for continuous and discrete systems

Continuous Discrete
Initial value, limt0 y(t) lims sY (s) lim
 z Y (z)
 
Final value, limt y(t) lims0 sY (s) limz1 1 z 1 Y (z)

2.4 I NVERSION OF z- TRANSFORMS

The usefulness of the z-transform is that we can do algebraic manipulations on discrete difference
equations in the same way we can do algebraic manipulations on differential equations using the
Laplace transform. Generally we:
3 The sum of a geometric series of n terms is
a (1 r n )
a + ar + ar 2 + + ar n1 = ; r 6= 1
1r
and if |r| < 1, then the sum for an infinite number of terms converges to
a
S =
1r
24 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

1. Convert to z-transforms, then

2. do some relatively simple algebraic manipulations to make it easier to

3. invert back to the time domain.

The final step, the inversion, is the tricky part. The process of obtaining f (t) back from F (z) is
called inverting the z-transform, and is written as

f (t) = Z 1 {F (z)} (2.15)

Note that only the sampled function is returned f (t), not the original f (t). Thus the full inversion
process to the continuous domain f (t) is a one to many operation.

There are various ways to invert z-transforms:

1. Use tables of z-transforms and their inverses in handbooks, or use a symbolic manipulator.
(See 2.4.1.)

2. Use partial-fraction expansion and tables. In M ATLAB use residue to compute the partial
fractions. (See 2.4.2.)

3. Long division for rational polynomial expressions in the discrete domain. In M ATLAB use
deconv to do the polynomial division. (See 2.4.3.)

4. Computational approach while suitable for computers, has the disadvantage that the an-
swer is not in a closed form. (See 2.4.4.)

5. Analytical formula from complex variable theory. (Not useful for engineering applications,
see 2.4.5.)

2.4.1 I NVERTING z- TRANSFORMS WITH SYMBOLICALLY

The easiest, but perhaps not the most instructive, way to invert z-transforms is to use a computer
algebra package or symbolic manipulator such as the symbolic toolbox or M U PAD. One simply
enters the z-transform, then requests the inverse using the iztrans function in a manner similar
to the forward direction shown on page 23.

>> syms z % Define a symbolic variable z


>> G = (10*z+5)/(z-1)/(z-1/4) % Construct G(z) = (10z + 5)/(z 1)/(z 1/4)
3 G =
(10*z+5)/(z-1)/(z-1/4)
>> pretty(G) % check it
10 z + 5
-----------------
8 (z - 1) (z - 1/4)
>> iztrans(G) % Invert the z-transform, Z 1 {G(z)}
ans =
20*kroneckerDelta(n, 0) - 40*(1/4)n + 20

The Kronecker function, or kroneckerDelta(n, 0) is a shorthand way of expressing piece-


wise functions in M ATLAB. The expression kroneckerDelta(n, 0) returns 1 when n = 0, and
0 for all other values.
2.4. INVERSION OF Z -TRANSFORMS 25

(n)
0.5

0
5 0 5
n

This is a rather messy and needlessly complicated artifact due to the fact that the symbolic toolbox
does not know that n is defined as positive. Also M ATLAB automatically choose n for the indexing
variable, but we might prefer to use k and explicitly inform M ATLAB that k > 0. This gives us a
cleaner solution.

>> syms z
>> syms k positive
>> G = (10*z+5)/(z-1)/(z-1/4);
4

>> y=iztrans(G,k) % Invert to get y[k] = Z 1 {G(z)}


y =
20-40*(1/4)k

9 >> limit(y,k,inf) % Compute final value: y[]


ans =
20

The last command computed the steady-state by taking the limit of y[k] as k .

2.4.2 T HE PARTIAL FRACTION METHOD

Inverting transforms by partial fractions is applicable for both discrete z-transforms and con-
tinuous Laplace transforms. In both cases, we find an equivalent expression for the transform
in simple terms that are summed together. Hopefully we can then consult a set of tables, such
as [152, Table 2.1 p49], to invert each term individually. The inverse of the sum, is the sum of
the inverses meaning that these expressions when summed together give the full inversion. The
M ATLAB function residue forms partial fractions from a continuous transfer function, although
the help file warns that this operation is numerically poorly conditioned. Likewise the routine
residuez (that is residue with a trailing z) is designed to extract partial fractions in a form
suitable when using ztransforms.

As an example, suppose we wish to invert

4s2 58s 24 4s2 58s 24


G(s) = 3 2
=
s + 2s 24s s(s + 6)(s 4)

using partial fractions noting that it contains no multiple or complex roots.

It is easy to spot the pole at s = 0, and the two remaining can be found by synthetic division, or
by factorising using the roots([1 2 -24 0]) command. The partial fraction decomposition
of G(s) can be written as

4s2 58s 24 A B C
G(s) = = + +
s(s + 6)(s 4) s s+6 s4

where the coefficients A, B and C can be found either by equating the coefficients or using the
26 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

cover-up rule shown below.



4s2 58s 24 4s2 58s 24
A= = 1, B= =3
(s + 6)(s 4) s=0 s(s 4)
s=6

4s2 58s 24
C= = 8
s(s + 6)
s=4

In M ATLAB we can use the residue command to extract the partial fractions.

>> B = [-4 -58 -24]; % Numerator of G(s) = (4s2 58s 24)/(s(s + 6)(s 4))
>> A = [ 1 2 -24 0]; % Denominator of G(s)
>> [r,p,k] = residue(B,A) % Find partial fractions
4 r = % residues (top line)
3
-8
1
p = % factors or poles
9 -6
4
0
k = % No extra parts
[]

The order of the residue and pole coefficient produced by residue is the same for each, so the
partial fraction decomposition is again
4s2 58s 24 3 8 1
G(s) = = + +
s3 2s 24s s+6 s4 s
and we can invert each term individually perhaps using tables to obtain the time domain solution
g(t) = 3e6t 8e4t + 1
If the rational polynomial has repeated roots or complex poles, slight modifications to the proce-
dure are necessary so you may need to consult the help file.

S PECIAL CASES FOR z- TRANSFORMS

We can invert z-transforms in a manner similar to Laplace transforms, but if you consult a table
of z-transforms and their inverses in standard control textbooks such as Table 2-1 in [153, p49],
you will discover that the table is written in terms of factors of the form z/(z + a) or alternatively
1/(1 z 1 ) rather than say 1/(z + a). This means that we should first perform the partial fraction
decomposition on X(z)/z, rather than just X(z) as in the continuous case. An outline to follow is
given in Algorithm 2.1.

Algorithm 2.1 Inverting z-transforms by partial fractions

To invert a z transform, X(z), using partial fractions, do the following:

1. Divide X(z) by z obtaining X(z)/z.


2. Find the partial fractions of X(z)/z perhaps using residue.
3. Multiply each partial fraction by z to obtain a fraction of the form z/(z + a)
4. Find the inverse of each fraction separately using tables, and sum together to find x(t).
2.4. INVERSION OF Z -TRANSFORMS 27

S YMBOLIC PARTIAL FRACTIONS IN M ATLAB

Suppose we want to invert


10z
G(z) =
z2 + 4z + 3
to g[k] using partial fractions. Unfortunately there is no direct symbolic partial fraction command,
but Scott Budge from Utah University found the following sneaky trick which involves first in-
tegrating, then differentiating the polynomial. This round-about way works because M ATLAB
integrates the expression by internally forming partial fractions and then integrating term by
term. Taking the derivative brings you back to the original expression, but now in partial fraction
form.

>> syms z
2 >> G = 10*z/(z2 + 4*z + 3);
>> G= diff(int(G/z)) % Extract partial fractions of G(z)/z
G =
5/(z+1)-5/(z+3)
>> G=expand(G*z)
7 G =
5z 5z
5*z/(z+1)-5*z/(z+3) % Gives us G(z)/z = z+1 z+3

>> syms k positive; iztrans(G,z,k)


ans =
12 -5*(-3)k+5*(-1)k

While this strategy currently works, we should take note that in future versions of the symbolic
toolbox this may not continue to work. In the above example we also have the option of using
residuez.

2.4.3 L ONG DIVISION

Consider the special case of the rational polynomial where the denominator is simply 1,

c0 + c1 z 1 + c2 z 2 + c3 z 3 +
F (z) =
1
The inverse of this transform is trivial since the time solution at sample number k or t = kT is
simply the coefficient of the term z k . Consequently it follows that f (0) = c0 , f (T ) = c1 , f (2T ) =
c2 , . . . , f (kT ) = ck etc. Thus if we have a rational transfer function, all we must do is to synthet-
ically divide out the two polynomials to obtain a single, albeit possibly infinite, polynomial in
z.

This approach is known as direct division [152, p69] and in general will result in an infinite
power series in z. As such it is not a particularly elegant closed-form solution, but for most stable
systems, the power series can be truncated eventually with little error.

So to invert a z-transform using long-division, we:

1. Convert the z-transform to a (possibly infinite) power series by dividing the denominator
into the numerator using long division or deconv.

2. The coefficients of z k are the solutions at the sample times kT .


28 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Long division by hand is tedious and error-prone, but we can use the polynomial division capa-
bilities of M ATLABs deconv to do this long division automatically. Since deconv only performs
integer division returning a quotient and remainder, we must fool it into giving us the infinite
series by padding the numerator with zeros.

To invert
z2 + z z2 + z
Y (z) = 2
= 3
(z 1)(z 1.1z + 1) z 2.1z 2 + 2.1z 1

using deconv for the long division, we pad the numerator on the right with say 5 zeros, (that is,
we multiply by z 5 ), and then do the division.

z7 + z6
= z 4 + 3.1z 3 + 4.41z 2 + 3.751z + 1.7161 + remainder
z 3 2.1z 2 + 2.1z 1 | {z }
Q

We are not particularly interested in the remainder polynomial. The more zeros we add, (5 in the
above example), the more solution points we get.

>> Yn = [1,1,0]; % Numerator of G(z)


>> Yd = conv([1,-1],[1 -1.1 1]); % Denominator of G(z) is (z 1)(z 2 1.1z + 1)
3 >> [Q,R] = deconv([Yn,0,0,0,0,0],Yd) % Multiply by z 5 to zero pad, then do long division
Q =
1.0000 3.1000 4.4100 3.7510 1.7161

>> dimpulse(Yn,Yd,7); % Matlab's check

We can also numerically verify the inversion using dimpulse for six samples as shown in Fig. 2.9.

3
Output

1
Figure 2.9: Inverting z-transforms using dimpulse for
the first six samples. It is possible also to construct a 0
discrete transfer function object and then use the generic
0 1 2 3 4 5 6
impulse routine. sample #

Problem 2.2 1. The pulse transfer function of a process is given by

Y (z) 4(z + 0.3)


= 2
X(z) z z + 0.4

Calculate the response of y(nT ) to a unit step change in x using long division.

2. Determine the inverse by long division of G(z)

z(z + 1)
G(z) =
(z 1)(z 2 z + 1)
2.4. INVERSION OF Z -TRANSFORMS 29

2.4.4 C OMPUTATIONAL APPROACH

The computational method is so-called because it is convenient for a computer solution technique
such as M ATLAB as opposed to an analytical explicit solution. In the computational approach we
convert the z-transform to a difference equation and use a recurrence solution.

Consider the following transform (used in the example from page 24) which we wish to invert
back to the time domain.
X(z) 10z + 5 10z + 5 10z 1 + 5z 2
= = 2 = (2.16)
U (z) (z 1)(z 0.25) z 1.25z + 0.25 1 1.25z 1 + 0.25z 2

where we have coloured the input values blue and the output values red. (We will again use this
colour convention in chapter 6).

The inverse is equivalent to solving for x(t) when u(t) is an impulse function. The transform can
be expanded and written as a difference equation

xk = 1.25xk1 0.25xk2 + 10uk1 + 5uk2 (2.17)

Since U (z) is an impulse, U (z) = 1 which means that only u(0) = 1 while all other samples
u(k) = 0, k 6= 0. Now we substitute k = 0 to start, and note that u(k) = x(k) = 0 when k < 0 by
definition. We now have enough information to compute x(0).

x0 = 1.25x1 0.25x2 + 10u1 + 5u2


= 1.25 0 0.25 0 + 10 0 + 5 0
=0

Continuing we substitute k = 1 into Eqn. 2.17 and using our previously computed x0 and noting
that now u0 is nonzero, we can find the next term.

x1 = 1.25x0 0.25x1 + 10u0 + 5u1


= 1.25 0 0.25 0 + 10 1 + 5 0
= 10

Just to clarify this recurrence process further, we can try the next iteration of Eqn. 2.17 with k = 2,

x2 = 1.25x1 0.25x0 + 10u1 + 5u0


= 1.25 10 0.2 0 + 10 0 + 5 1
= 17.5

Now we can use the full recurrence relation, (Eqn 2.17), to obtain the solution in a stepwise man-
ner to build up the solution as shown in Table 2.2. All the terms on the right hand side of Eqn 2.17
are either known initially (u(t)), or involve past information, hence it is possible to solve. Note
that in this inversion scheme, u(t) can be any known input series.

M ATLAB is well suited for this type of computation. To invert Eqn. 2.16 we can use the discrete
filter command. We rewrite Eqn. 2.16 in the form of a rational polynomial in z 1 ,

b0 + b1 z 1 + b2 z 2
G(z) =
1 + a1 z 1 + a2 z 2

where the numerator and denominator are entered as row vectors in decreasing powers of z. We
then create an impulse input vector u with say 6 samples and then compute the output of the
system
30 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Table 2.2: The inversion of a z transform using the computation method. Only the first 5 samples
and the final value are calculated. The final value is as calculated using the symbolic manipulator
earlier on page 25.

time (kT ) u(k) x(k)


2 0 0
1 0 0
0 1 0
1 0 10
2 0 17.5
3 0 19.375
4 0 19.8438
.. .. ..
. . .
0 20

>> b = [0.0, 10.0, 5.0] % Numerator (10z 1 + 5z 2 )


>> a = [1.0, -1.25, 0.25] % Denominator (1 1.25z 1 + 0.25z 2 )
3

>> u = [1, zeros(1,5)] % impulse input, u(k) = (k)


u =
1 0 0 0 0 0
>> x = filter(b,a,u) % compute y(k)
8 x =
10.0000 17.5000 19.3750 19.8438 19.9609 19.9902

which gives the same results as Table 2.2.

The control toolbox has a special object for linear time invariant (LTI) systems. We can create a
discrete system with the transfer function command, tf, and by specifying a sampling time, we
imply to M ATLAB that it is a discrete system.

1 >> sysd = tf([10 5],[1 -1.25 0.25],1) % System of interest (10z + 5)/(z 2 1.25z + 0.25)

Transfer function:
10 z + 5
-------------------
6 z2 - 1.25 z + 0.25

Sampling time: 1

>> y=impulse(sysd) % compute the impulse response


11 y =
0
10.0000
17.5000
19.3750
16 19.8438
19.9609

Finally, the contour integral method can also be used to invert z-transforms, [152, 33], but Se-
borg et al maintains it is seldom used in engineering practice, [183, p571] because it is fraught
with numerical implementation difficulties. These difficulties are also present in the continuous
equivalent; some of which are illustrated in the next section.
2.4. INVERSION OF Z -TRANSFORMS 31

2.4.5 N UMERICALLY INVERTING THE L APLACE TRANSFORM

Surprisingly, the numerical inversion of continuous transfer functions is considered far less impor-
tant than the computation of the inverse of discrete transfer functions. This is fortunate because
the numerical inversion of Laplace transforms is devilishly tricky. Furthermore, it is unlikely that
you would ever want to numerically invert a Laplace transform in control applications, since most
problems involver little more than a rational polynomial with a possible exponential term. For
these problems we can easily factorise the polynomial and use partial fractions or use the step
orimpulse routines for continuous linear responses.

However in the rare cases where we have a particularly unusual F (s) which we wish to convert
back to the time domain, we might be tempted to use the analytical expression for the inverse
directly
Z +j
1
f (t) = F (s) est ds (2.18)
2j j
where is chosen to be larger than the real part of any singularity of F (s). Eqn. 2.18 is sometimes
known as the Bromwich integral or Mellins inverse formula.

The following example illustrates a straight forward application of Eqn. 2.18 to invert a Laplace
transform. However be warned that for all but the most well behaved rational polynomial exam-
ples this strategy is not to be recommended as it results in severe numerical roundoff error due to
poor conditioning.

Suppose we try to numerically invert a simple Laplace transform,


3 3 
F (s) = f (t) = 1 e5t
s(s + 5) 5
with a corresponding simple time solution. We can start by defining an anonymous function of
the Laplace transform to be inverted.

Fs = @(s) 3.0./(s+5)./s; % Laplace function to invert F (s) = 3/(s(s + 5)) . . .


Fsexp = @(s,t) Fs(s).*exp(s*t) % and associated integrand F (s)est

Now since the largest singularity of F (s) is 0, we can choose the contour path safely to be say
= 0.1. We can also approximate the infinite integration interval by reasonably large numbers.
It is convenient that the M ATLAB routine integral works directly in the complex domain.

c = 0.1; % Value > than any singularities of F (s)


a = c-100j; b = c+200j; % Contour path approx: j to + j
3

t=linspace(0,8); % time range of interest

% Numerically approximate Eqn. 2.18.


ft = 1./(2*pi*1j)*arrayfun(@(t) integral(@(x) Fsexp(x,t),a,b), ti);
8 plot(t,ft) % Compare results in Fig. 2.10(a).

Due to small numerical roundoff however, the returned solution has a small imaginary compo-
nent, which we can in this instance safely ignore. In this rather benign example, where we know
the analytical solution, we can validate the accuracy as shown in Fig. 2.10(a), which it has to be
admitted, is not that wonderful. We should also note in the simplistic implementation above we
have not adequately validated that our algorithmic choices such as the finite integration limits
are appropriate, so we should treat any subsequent result with caution. In fact if you repeat this
calculation with a larger value of , or a smaller integration range such as say 1 10j to 1 + 10j
32 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

0.7 2

0.6 1.5 n o
0.5 L1 1
1
s 2 +1

0.4
f(t)

f(t)
0.5
0.3
0
0.2 n o
3
L 1
s(s+5) 0.5
0.1

0 1
0.02 2

1
error

error
0
0

0.02 1
0 2 4 6 8 0 5 10 15 20
t t

(a) Inverting
n o a benign transfer function, (b) Inverting
 a challenging transfer function,
3
L1 s(s+5) L 1 1
s2 +1

Figure 2.10: Numerically inverting Laplace transforms by direct evaluation of the Bromwich in-
tegral. Since this strategy is suspectable to considerable numerical errors, it is not to be recom-
mended.

then the quality of the solution drops alarmingly. One way to improve the integration accuracy is
to use the waypoints option in the integral routine.

How the Bromwich integral, Eqn. 2.18, actually works can be illustrated with the somewhat un-
usual pair
 
e s 1
F (s) = f (t) = erfc (2.19)
s 2 t
The top two plots in Fig. 2.11(a) plot the two surfaces of the complex function F (s)est (which is the
integrand in Eqn. 2.18, the thing we need to integrate) for the specific case when t = 1.5. We need
to plot two surfaces, because since s is a complex variable, F (s)est will in general be a complex
variable as well. Consequently we can plot the real surface on one plot and the imaginary surface
on another.

Now to perform the integration in Eqn. 2.18, we need to integrate this complex function along
the imaginary axis, or in this case just slightly east of the imaginary axis in order to miss the
singularity in F (s) at the origin. The two semi-transparent windows vertically cutting the surfaces
show the path along which we need to integrate.

To better visualise exactly what we are integrating, the plots of the function just along the wall are
repeated in the figures below in Fig. 2.11(b). Note how the integrand for {G(s)} is odd, so the
definite integral for symmetric limits will turn out to be zero. However the integral for the real
part (left hand plot in Fig. 2.11(b)) is definitely not zero, but clearly seems to converge as s tends
towards i. The total integral will therefore be a pure imaginary number which is scaled by
1/(2i) to give a pure real number representing the time domain solution y(t) at the specific time
t = 1.5.

Over the years there have been many attempts to improve the inversion strategy as reviewed
in [1], but nearly all of the proposals run into numerical problems of precision. One popular
approximation of order M for inverse Laplace transforms introduced by Gaver and subsequently
2.4. INVERSION OF Z -TRANSFORMS 33


(a) The real and imaginary surfaces of the complex function F (s)est where F (s) = e s /s for the
case when t = 1.5. Note the singularity at the origin.

2.5 0.6

0.4
2

0.2
1.5

0
(G(s))

(G(s))

1
0.2

0.5
0.4

0
0.6

0.5 0.8
5 0 5 5 0 5
{s} {s}

(b) The shaded part represents the integral of F (s)est along the imaginary axis. The real part is
even (so the integral is non-zero), while the imaginary part is odd, so the integral is zero.


Figure 2.11: Visualising the Bromwich integral for F (s) = e s
/s for the case when t = 1.5.

improved by Stehfest is
2M  
ln(2) X k ln(2)
f (t) = L1 {F (s)} wk F (2.20)
t t
k=1

where the weights, wk , are given by


min(k,M)    
(1)M+k X M 2j j
wk = j M+1 (2.21)
M! j j kj
j=(k+1)/2

and the notation


 x means the greatest integer less than or equal to x, [112]. The binomial coeffi-
cients nk can be generated in M ATLAB using nchoosek(n,k).

Listing 2.1 implements this algorithm for scalar t. The routine could be made more efficient by
pre-computing the weights just once for a given approximation order M so in the common case
where t is a vector, we do not waste time recomputing the weights. Note also that the approx-
imation assumes that t is strictly greater than zero, so for t = 0 we could use the initial value
theorem.

Listing 2.1: Numerically inverting Laplace transforms using the Gaver-Stehfest algorithm
34 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

function [y, A] = GSinvLT(Fs,t,M)


2 % Invert a laplace transform, F (s), to f (t) using the Gaver-Stehfest algorithm of order M

% Compute weights Ak = f (M ) following Eqn. 2.21


A = zeros(2*M,1); % placeholder for weights
for k=1:2*M
7 B=0;
for j=floor((k+1)/2):min([k,M])
B=B+j(M+1)*nchoosek(M,j)*nchoosek(2*j,j)*nchoosek(j,k-j);
end % j
A(k) = (-1)(M+k)*B/factorial(M);
12 end % k

ln2t = log(2)/t; % Now compute solution


k = [1:2*M]';
y = ln2t*sum(A.*Fs(k*ln2t)); % Eqn. 2.20
17 return

The performance of this approximation


is given in Fig. 2.12 which shows the problem when at-
tempting to invert F (s) = 1/ s2 + 1 with a varying number of terms.

The Gaver-Stehfest family of schemes are well known to produce inaccurate results for under-
damped systems, [112, p2986], but when we increase the number of terms in an attempt to im-
prove the accuracy, the results deteriorate due to numerical round off. This is because the weights
grow rapidly and have alternating signs. Directly applying the Bromwich integral to this trans-
form as shown in Fig. 2.10(b) is no better. While the strategy suggested in [1] seemingly circum-
vents the problem of precision, it does so by brute-force using multi-precision arithmetic which is
hardly elegant.

An alternative numerical algorithm is the inverse Laplace transform function invlap function
contributed by Karl Hollenbeck from the Technical University of Denmark. You can test this
routine using one of the following Laplace transform pairs in Table 2.3. The first two are standard,
those following are more unusual. A more challenging collection of test transforms is given in [1].

Table 2.3: A collection of Laplace transform pairs suitable for testing numerical inversion strate-
gies.

Description F (s) f (t) Comment


2
Easy test 1/s t
1 1

Control TF 2 et e3t
(s + 2)(s + 3)
1
Oscillatory J0 (t) Bessel function, see Fig. 2.10(b) & Fig. 2.12.
s2 + 1
1
Nasty e1/s 1
t
cos( 4t), See Fig. 2.13.
s

n o
The numerical solution to L1 1 e1/s is compared with the analytical solution from Table 2.3
s
in Fig. 2.13 using a modest tolerance of 102 . The routine splits the time axis up into decades, and
inverts each separately which is why we can see some numerical noise starting at t = 10.

All of these numerical inversion strategies, starting from the direct application of the Bromwich
integral in Eqn. 2.18, the Gaver-Stehfest algorithm and variants and the implementation invlap
require some care to get reasonable results. Unfortunately in practical cases it can be difficult to
2.4. INVERSION OF Z -TRANSFORMS 35

# of terms = 8
0.5

# of terms = 16
0.5

# of terms = 20
0.5

0.5 # of terms = 26

0
Figure 2.12: Demonstrating the precision problems when
0.5 numerically inverting the Laplace transform using the
Gaver-Stehfest algorithm with a varying number of
0 5 10 terms. The exact inversion is the red solid line while the
time, t approximate inversion is given by .

1.5

n o
exp(1/s)
f(t)

0.5 L1
s

0
0.5
10

2
10
error

4
10

Figure 2.13: Testing the numerical inversion


6
10
0 5 10 15 20 25 30 35 40
of the Laplace transform e1/s / s using the
time invlap routine.
36 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

choose suitable algorithmic tuning constants, which mean that it is difficult to generate any sort
of error bound on the computed curve.

samhold

2.5 D ISCRETISING WITH A SAMPLE AND HOLD

In 2.1.1 we saw how the continuous analogue signal must be sampled to be digestible by the pro-
cess control computer. To analyse this sampling operation, we can approximate the real sampler
with an ideal sampler in cascade with a hold element such as given in Fig. 2.14.

f (t)
f (t) b b
Hold fh (t)
Ideal sampler

continuous signal sampled signal sampled & held signal

Figure 2.14: Ideal sampler and zeroth-order hold.

When we sample the continuous function f (t), we get a sampled function f (t), that is a series of
spikes that exist only at the sampling instant as shown in the middle plot of Fig. 2.14. The sampled
function, f (t), is undefined for the time between the sampling instants which is inconvenient
given that this is the bulk of the time. Obviously one sensible solution is to retain the last sampled
value until a new one is collected. During the sample period, the sampled value f (t) is the same
as the last sampled value of the continuous function; f (kT + t) = f (kT ) where 0 t < T shown
diagrammatically in the third plot in Fig. 2.14. Since the last value collected is stored or held, this
sampling scheme is referred to as a zeroth-order hold. The zeroth-order refers to the fact that the
interpolating function used between adjacent values is just a horizontal line. Higher order holds
are possible, but the added expense is not typically justified even given the improved accuracy.

We can find the Laplace transform of the zeroth-order hold element by noting that the input is an
impulse function, and the output is a rectangular pulse of T duration, and y(t) high, as shown in
Fig. 2.15.
output rectangular pulse

y(t)

time, t
sample time, T
0

Figure 2.15: A Rectangular pulse.


2.5. DISCRETISING WITH A SAMPLE AND HOLD 37

Directly applying the definition of the Laplace transform,


Z Z T
L{zoh} = y(t) est dt = yt 1 est dt
0 0
1 esT
= yt
| {zs }
zoh

we obtain the transform for the zeroth-order hold. In summary, the transformation of a continu-
ous plant G(s) with zero-order hold is
 
G(s)
G(z) = (1 z 1 ) Z (2.22)
s

Note that this is not the same as simply the z-transform of G(s).

Suppose we wish to discretise the continuous plant


8
G(s) = (2.23)
s+3
with a zero-order hold at a sample time of T = 0.1 seconds. We can do this analytically using
tables, or we can trust it to M ATLAB. First we try analytically and apply Eqn 2.22 to the continuous
process. The difficult bit is the z-transformation. We note that
   
G(s) 8 3
Z = Z (2.24)
s 3 s(s + 3)

and that this is in the form given in tables of z-transforms such as [152, p50], part of which is
repeated here:

X(s) X(z)
a (1 eaT )z 1
s(s + a) (1 z 1 )(1 eaT z 1 )

Inverting using the tables, and substituting in Eqn. 2.22 gives


   
G(s) 8 3
G(z) = (1 z 1 ) = (1 z 1 ) Z
s 3 s(s + 3)
8 (1 e3T )z 1
= (1 z 1 )
3 (1 z 1 )(1 e3T z 1 )
0.3 1
8 (1 e )z
=
3 (1 e0.3 z 1 )
0.6912z 1
= , for T = 0.1 (2.25)
1 0.7408z 1
We can repeat this conversion numerically using the continuous-to-discrete function, c2d. We
first define the continuous system as an LTI object, and then convert to the discrete domain using
a zeroth order hold.

>>sysc = tf(8,[1 3]) % Continuous system: Gc (s) = 8/(s + 3)

3 Transfer function:
8
38 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

-----
s + 3

8 >> sysd = c2d(sysc,0.1,'zoh') % convert to discrete using a zoh

Transfer function:
0.6912
----------
13 z - 0.7408

Sampling time: 0.1

which once again is Eqn. 2.25. In fact we need not specify the zoh option when constructing a
discrete model using c2d or even in S IMULINK, since a zeroth-order hold is employed by default.

Methods for doing the conversion symbolically are given next in section 2.5.1.

2.5.1 C ONVERTING L APLACE TRANSFORMS TO z- TRANSFORMS

In many practical cases we may already have a continuous transfer function of a plant or con-
troller which we wish to discretise. In these circumstances we would like to convert from a con-
tinuous transfer function in s, (such as say a Butterworth filter) to an equivalent discrete transfer
function in z. The two common ways to do this are:

1. Analytically by first inverting the Laplace transform back to the time domain, and then take
the (forward) z-transform.
def 
G(z) = Z L1 {G(s)} (2.26)

2. or by using an approximate method known as the bilinear transform.

The bilinear method whilst approximate has the advantage that it just involves a simple algebraic
substitution for s in terms of z. The bilinear transform is further covered in 2.5.2.

The formal method we have already seen previously, but since it is such a common operation, we
can write a symbolic M ATLAB script to do it automatically for us for any given transfer function.
However we should be aware whether the transformation includes the sample and hold, or if that
inclusion is left up to the user.

Listing 2.2 converts a Laplace expression to a z-transform expression without a zeroth-order hold.
Including the zeroth-order hold option is given in Listing 2.3.

Listing 2.2: Symbolic Laplace to z-transform conversion


function Gz = lap2ztran(G)
% Convert symbolically G(s) to G(z)
syms t k T z
Gz = simplify(ztrans(subs(ilaplace(G),t,k*T),k,z));
5 return

Listing 2.3: Symbolic Laplace to z-transform conversion with ZOH


function Gz = lap2ztranzoh(G)
% Convert symbolically G(s) to G(z) with a ZOH.
2.5. DISCRETISING WITH A SAMPLE AND HOLD 39

syms s t k T z
Gz = simplify((1-1/z)*ztrans(subs(ilaplace(G/s),t,k*T),k,z));
5 return

We can test the conversion routines in Listings 2.2 and 2.3 on the trial transfer function G(s) =
1/s2 .

>> Gz =lap2ztran(1/s2); % Convert G(s) = 1/s2 to the discrete domain G(z).


>> pretty(Gz)
T z
--------
5 2
(z - 1)

>> Gz = lap2ztranzoh(1/s2);% Do the conversion again, but this time include a ZOH.
>> pretty(Gz)
10 2
(z + 1) T
1/2 ----------
2
(z - 1)

2.5.2 T HE BILINEAR TRANSFORM

The analytical one step backone step forward procedure while strictly correct, is a little tedious,
so a simpler, albeit approximate, way to transform between the Laplace domain and the z-domain
is to use the bilinear transform method, sometimes known as Tustins method. By definition
def
z = esT or z 1 = esT , (from Eqn. 2.11). If we substituted directly natural logs would appear in
the rational polynomial in z,
ln(z)
s= (2.27)
T
making the subsequent analysis difficult. For example the resulting expression could not be trans-
formed into a difference equation which is what we desire.

We can avoid the troublesome logarithmic terms by using a Pade approximation for the exponen-
tial term as
2 sT
esT = z 1 (2.28)
2 + sT
or alternatively
2 (1 z 1 )
s (2.29)
T (1 + z 1 )
This allows us to transform a continuous time transfer function G(s) to a discrete time transfer
function G(z),
G(z) = G(s)|s= 2 1z1 (2.30)
T 1+z 1

Eqn. 2.30 is called the bilinear transform owing to the linear terms in both the numerator and
denominator and it has the advantage that a stable continuous time filter will be stable in the dis-
crete domain. The disadvantage is that the algebra required soon becomes unwieldy if attempted
manually. Other transforms are possible and are discussed in 4-2 p308 of Ogata [152]. Always
remember that this transformation is approximate, being equivalent to a trapezoidal integration.
40 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Here we wish to approximately discretise the continuous plant

1
G(s) =
(s + 1)(s + 2)

at a sample time of T = 0.1 using the bilinear transform. The discrete approximate transfer
function is obtained by substituting Eqn. 2.29 for s and simplifying.

1
G(z) =
(s + 1)(s + 2) s= 2 1z1
T 1+z 1

1 1 T 2 (z + 1)2
=   =
2 1z 1
+1 2 1z 1
+2 2 (2T + 6T + 4)z + (4T 2 8)z + 2T 2 6T + 4
2 2
T 1+z 1 T 1+z 1

0.0022z 2 + 0.0043z + 0.0022


= for T = 0.1
z 2 1.723z + 0.74

Since this transformation is just an algebraic substitution, it is easy to execute it symbolically in


M ATLAB.

1 >> syms s T z
>> G = 1/(s+1)/(s+2);
>> Gd = simplify(subs(G,s,2/T*(1-1/z)/(1+1/z))); % Substitute Eqn. 2.29 for s.
>> pretty(Gd)
2 2
6 T (z + 1)
1/2 -------------------------------------
(2 z - 2 + T z + T) (z - 1 + T z + T)

Alternatively we could use M ATLAB to numerically verify our symbolic solution given that T =
0.1.

>> G = zpk([],[-1 -2],1)


2 Zero/pole/gain:
1
-----------
(s+1) (s+2)

7 >> T=0.1;
>> Gd = c2d(G,T,'tustin')

Zero/pole/gain:
0.0021645 (z+1)2
12 ---------------------
(z-0.9048) (z-0.8182)

Sampling time: 0.1


>> tf(Gd)
17

Transfer function:
0.002165 z2 + 0.004329 z + 0.002165
------------------------------------
z2 - 1.723 z + 0.7403
22

Sampling time: 0.1


2.5. DISCRETISING WITH A SAMPLE AND HOLD 41

The bilinear command in the S IGNAL P ROCESSING toolbox automatically performs this map-
ping from s to z. This is used for example in the design of discrete versions of common analogue
filters such as the Butterworth or Chebyshev filters. These are further described in 5.2.3.

Problem 2.3 1. Use Tustins method (approximate z-transform) to determine the discrete time
response of
4
G(s) =
(s + 4)(s + 2)
to a unit step change in input by long division. The sample time T = 1 and solve for 7 time
steps. Compare with the exact solution.

T HE FREQUENCY RESPONSE CHARACTERISTICS OF A HOLD ELEMENT

The zeroth order hold element modifies the transfer function, so consequently influences the
closed loop stability, and frequency characteristics of the discrete process. We can investigate
this influence by plotting the discrete Bode and Nyquist plots for a sampled process with and
without a zeroth order hold.

Suppose we have the continuous process

1.57
Gp (s) = (2.31)
s(s + 1)

which is sampled at a frequency of = 4 rad/s. (This corresponds to a sample time of T = /2


seconds.) First we will ignore the zeroth order hold and transform Eqn 2.31 to the z domain by
using tables such as say Table 21, p49 #8 in DCS.

(1 eaT )z 1
Gp (z) = 1.57 (2.32)
(1 z 1 )(1 eaT z 1 )
1.243z
= (2.33)
(z 1)(z 0.208)

To get the discrete model with the zeroth order hold, we again use Eqn 2.22, and the tables. Doing
this, in a similar manner to what was done for Eqn 2.25, we get

1.2215z + 0.7306
Gh0 Gp (z) = (2.34)
(z 1)(z 0.208)

Now we can plot the discrete frequency responses using M ATLAB to duplicate the figure given in
[110, p649] as shown in Fig. 2.16.

In the listing below, I first construct the symbolic discrete transfer function, substitute the sam-
pling time, then convert to a numeric rational polynomial. This rational polynomial is now in a
suitable form to be fed to the control toolbox routines such as bode.

syms s T z
2 G = 1.57/(s*(s+1))
Gd = lap2ztran(G) % convert to a ztransform without ZOH
Gd2 = subs(Gd,T,pi/2)
[num,den] = numden(Gd2) % extract top & bottom lines

7 B= sym2poly(num); A = sym2poly(den); % convert to polynomials

Gd = tf(B,A,pi/2) % construct a transfer function


42 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

To generate the discretisation including the zeroth-order hold, we could use the symbolic lap2ztranzoh
routine given in Listing 2.3, or we could use the built-in c2d function.

1 [num,den] = numden(G) % extract numerator & denominator


Bc= sym2poly(num); Ac = sym2poly(den);
Gc = tf(Bc,Ac)

Gdzoh = c2d(Gc,pi/2,'zoh')
6

bode(Gc,Gd,Gdzoh)
legend('Continuous','No ZOH','with ZOH')

In this case I have used vanilla Bode function which will automatically recognise the difference
between discrete and continuous transfer functions. Note that it also automatically selects both
a reasonable frequency spacing, and inserts the Nyquist frequency at fN = 1/2T = 1/ Hz or
N = 2 rad/s.

These Bode plots shown in Fig. 2.16, or the equivalent Nyquist plots, show that the zeroth order
hold destabilises the system. The process with the hold has a larger peak resonance, and smaller
gain and phase margins.

Bode Diagram

100

50
Magnitude (dB)

50

100
90
Continuous
No ZOH
with ZOH
Phase (deg)

135

180
Figure 2.16: The Bode dia-
gram showing the difference
between the continuous plant, 225
2 1 0 1 2
a discretisation with and with- 10 10 10 10 10
out the zeroth-order hold. Frequency (rad/sec)

Of course we should compare the discrete Bode diagrams with that for the original continuous
process. The M ATLAB bode function is in this instance the right tool for the job, but I will con-
struct the plot manually just to demonstrate how trivial it is to substitute s = i, and compute the
magnitude and phase of G(i) for all frequencies of interest.

num = 1.57; % Plant of interest G(s) = 1.57/(s2 + s + 0)


2 den = [1 1 0];
w = logspace(-2,1)'; % Frequency range 102 < < 101 rad/s.
iw = 1i*w; % s = j
2.6. DISCRETE ROOT LOCUS DIAGRAMS 43

G = polyval(num,iw)./polyval(den,iw); % G(s = j)
loglog(w,abs(G)) % Plot magnitude |G(i)|, see Fig. 2.17.
7 semilogx(w,angle(G)*180/pi) % and phase (i)

We can compare this frequency response of the continuous system with the discretised version in
Fig. 2.16.

10 4

10 2
AR

10 0

10 -2
-80
phase angle, [deg]

-100

-120

-140

-160
Figure 2.17: A frequency response plot of
-180
10 -2 10 -1
10 0
10 1 G(s) where s = j constructed manually
frequency [rad/s] without using the bode command.

2.6 D ISCRETE ROOT LOCUS DIAGRAMS

The root locus diagram is a classical technique used to study the characteristic response of trans-
fer functions to a single varying parameter such as different controller gains. Traditionally the
construction of root locus diagrams required tedious manual algebra. However now modern
programs such as M ATLAB can easily construct reliable root locus diagrams which makes the
design procedure far more attractable.

For many controlled systems, the loop is stable for small values of controller gain Kc , but unstable
for a gain above some critical value Ku . It is therefore natural to ask how does the stability (and
response) vary with controller gain? The root locus gives this information in a plot form which
shows how the poles of a closed loop system change as a function of the controller gain Kc . Given
a process transfer function G and a controller Kc Q, the closed loop transfer function is the familiar
Kc Q(z)G(z)
Gcl = (2.35)
1 + Kc Q(z)G(z)
where the stability is dependent on the denominator (characteristic equation) 1 + Kc Q(z)G(z),
which is of course itself dependent on the controller gain Kc . Plotting the roots of

1 + Kc Q(z)G(z) = 0 (2.36)

as a function of Kc creates the root locus diagram.

In this section4 we will plot the discrete root locus diagram for the process
0.5(z + 0.6)
Gp (z) = (2.37)
z 2 (z 0.4)
4 Adapted from [71, p124]
44 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

which is a discrete description of a first order process with dead time. We also add a discrete
integral controller of the form
z
Gc (z) = Kc
z1
We wish to investigate the stability of the characteristic equation as a function of controller gain.

With no additional arguments apart from the transfer function, rlocus will draw the root locus
selecting sensible values for Kc automatically. For this example, I will constrain the plots to have
a square aspect ratio, and I will overlay a grid of constant shape factor and natural frequency
n using the zgrid function.

Q = tf([1 0],[1 -1],-1); % controller without gain, Q(z)


G = tf(0.5*[1 0.6],[1 -0.4 0 0],-1); % plant, Gp (z)
3

Gol = Q*G; % open loop Q(z)Gp (z)


zgrid('new'); xlim([-1.5 1.5]); axis equal
rlocus(Gol) % plot root locus, see Fig. 2.18.
rlocfind(Gol)

Once the plot such as Fig. 2.18 is on the screen, I can use rlocfind interactively to establish the
values of Kc at different critical points in Fig. 2.18. Note that the values obtained are approximate.

Pole description Name Kc


border-line stability ultimate gain 0.6
cross the = 0.5 line design gain 0.2
critically damped breakaway gain 0.097
overdamped, n = 36 0.09

Once I select the gain that corresponds to the = 0.5 crossing, I can simulate the closed loop step
response.

Gol.Ts = 1; % Set sampling time T = 1


Kc = 0.2 % A gain where we expect = 0.5
3 step(Kc*Gol/(1 + Kc*Gol),50) % Simulate closed loop, see Fig. 2.19.

A comparison of step responses for various controller gains is given in Fig. 2.19. The curve in
Fig. 2.19 with Kc = 0.2 where 0.5 exhibits an overshoot of about 17% which agrees well with
the expected value for a second order response with a shape factor of = 0.5. The other curves
do not behave exactly as one would expect since our process is not exactly a second order transfer
function.

2.7 M ULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

In the mid 1970s, the western world suffered an oil shock when the petroleum producers and
consumers alike realised that oil was a scarce, finite and therefore valuable commodity. This had
a number of important implications, and one of these in the chemical processing industries was
the increased use of integrated heat plants. This integration physically ties separate parts of the
plant together and demands a corresponding integrated or plant-wide control system. Clas-
sical single-input/single-output (SISO) controller design techniques such as transfer functions,
frequency analysis or root locus were found to be deficient, and with the accompanying devel-
opment of affordable computer control systems that could administer hundreds of inputs and
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 45

Root Locus

1.5

1
0.5/T Ultimate
0.6/T 0.4/T gain
0.7/T 0.1
0.3/T
0.2
0.3
0.8/T 0.4 0.2/T Design gain
0.5 0.5
0.6
0.7
0.9/T 0.8 0.1/T
Imaginary Axis

0.9
/T
0
/T

0.9/T 0.1/T
breakaway
0.5 0.8/T 0.2/T gain

0.7/T 0.3/T
0.6/T 0.4/T
0.5/T
1

1.5

1.5 1 0.5 0 0.5 1 1.5


Real Axis

Figure 2.18: The discrete root locus of Eqn. 2.37 plotted using rlocus. See also Fig. 2.19 for the
resultant step responses for various controller gains.

1
K =0.1
c
0
2

1
K =0.2
c
0
2

Figure 2.19: Various discrete closed loop re-


1 sponses for gains Kc = 0.6, 0.2 and 0.1. Note that
K =0.6 with Kc = 0.6 we observe a response on the sta-
c
bility boundary, while for Kc = 0.1 we observe a
0
0 10 20 30 40 50 critically damped response as anticipated from the
time [s] root locus diagram given in Fig. 2.18.
46 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

outputs, many of which were interacting, nonlinear and time varying, new tools were required
and the systematic approach offered by state space analysis became popular.

Many physical processes are multivariable. A binary distillation column such as given in Fig. 2.20
typically has four inputs; the feed flow and composition and reflux and boil-up rates, and at least
four outputs, the product flow and compositions. To control such an interacting system, mul-
tivariable control such as state space analysis is necessary. The Wood-Berry distillation column
model (Eqn 3.24) and the Newell & Lee evaporator model are other examples of industrial process
orientated multivariable models.
Disturbances

feed rate distillate composition, xD

distillate rate, D
feed composition
Distillation Tray 5 temperature, T5
Column Tray 15 temperature, T15
Manipulated

reflux rate
Bottoms flowrate, B

reboiler heat Bottoms composition, xB

Plant Inputs Plant Plant Outputs

Figure 2.20: A binary distillation column with multiple inputs, disturbances, and multiple outputs

State space analysis only considers first order differential equations. To model higher order sys-
tems, one needs only to build systems of first order equations. These equations are conveniently
collected, if linear, in one large matrix. The advantage of this approach is a compact represen-
tation, and a wide variety of good mathematical and robust numerical tools to analyse such a
system.

A FEW WORDS OF CAUTION

The state space analysis is sometimes referred to as the modern control theory, despite the fact
that is has been around since the early 1960s. However by the end of the 1970s, the promise of
the academic advances made in the previous decade were turning out to be ill-founded, and it
was felt that this new theory was ill-equipped to cope with the practical problems of industrial
control. In many industries therefore, Advanced Process Control attracted a bad smell. Writing
a decade still later in 1987, Morari in [140] attempts to rationalise why this disillusionment was
around at the time, and whether the subsequent decade of activity alleviated any of the concerns.
Morari summarises that commentators such as [70] considered theory such as linear multivariable
control theory, (i.e. this chapter) which seemed to promise so much, actually delivered very little
and had virtually no impact on industrial practice. There were other major concerns such as
the scarcity of good process models, the increasing importance of operating constraints, operator
acceptance etc, but the poor track record of linear multivariable control theory landed top billing.
Incidentally, Morari writing almost a decade later still in 1994, [143], revisits the same topic giving
the reader a nice linear perspective.
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 47

2.7.1 S TATES AND STATE SPACE

State space equations are just a mathematical equivalent way of writing the original common dif-
ferential equation. While the vector/matrix construction of the state space approach may initially
appear intimidating compared with high order differential equations, it turns out to be more con-
venient and numerically more robust to manipulate these equations when using programs such
as M ATLAB. In addition the state space form can be readily expanded into multivariable systems
and even nonlinear systems.

The state of a system is the smallest set of values such that knowledge of these values, knowledge
of any future inputs and the governing dynamic equations, it is possible to predict everything
about the future (and past) output of the system. The state variables are often written as a column
vector of length n and denoted x. The state space is the n dimensional coordinate space that all
possible state vectors must lie in.

The input to a dynamic system is the vector of m input variables, u, that affect the state variables.
Historically control engineers further subdivided the input variables into those they could eas-
ily and consciously adjust (known as control or manipulated variable inputs), and those variables
that may change, but are outside the engineers immediate sphere of influence (like the weather)
known as disturbance variables. The number of input variables need not be the same as the num-
ber of state variables, and indeed m is typically less than n.

The state space equations are the n ordinary differential equations that relate the state derivatives
to the state themselves, the inputs if any, and time. We can write these equations as

x = f (x, u, t), x(t = 0) = x0 , x n1 (2.38)

where f () is a vector function, the n initial conditions at t = 0 for the system is some known
point, x0 , and that the vector x has dimensions (n 1).

In control applications, the input is given by a control law which is often itself a function of the
states,
u = h(x) (2.39)
which if we substitute into Eqn. 2.38, we get the closed loop response

x = f (x, h(x), t)

which is now no longer an explicit function of the input u. For autonomous closed loop systems
there is no explicit dependence on time, so the differential equation is simply

x = f (x) (2.40)

For the continuous linear time invariant case, Eqn. 2.38 simplifies to

x = Ax + Bu (2.41)

or in discrete form at time t = kT


xk+1 = xk + uk (2.42)
where we have taken the convention that we use Roman variable names, A, B, for the continuous
case, and Greek variable names, , , in the discrete case. (Note that M ATLAB does not follow
this convention.)

Block diagrams of both the continuous and discrete formulations of the state space model are
shown in Fig. 2.21 and both are conceptually very similar. Such a form is suitable to implement
state space systems in a simulator such as S IMULINK for example.
48 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

x R x(t) xk+1 xk
+ uk +
u(t) B z 1

Continuous plant Discrete plant

Figure 2.21: A block diagram of a state space dynamic system, (a) continuous system: x = Ax +
Bu, and (b) discrete system: xk+1 = xk + uk . (See also Fig. 2.22.)

The output or measurement vector, y, are the p variables that are directly measured from the oper-
ating plant, since often the true states cannot themselves be directly measured. These outputs are
related to the states by
y = g(x), y p1 (2.43)
or if the measurement relation is linear, then

y = Cx. (2.44)

Sometimes the input may directly affect the output bypassing the dynamic elements, in which
case the full linear system in state space is described by

x = Ax + Bu
(2.45)
y = Cx + Du

which is the standard starting point for the analysis of continuous linear dynamic systems, and is
shown in block diagram form in Fig. 2.22.

x R x(t)
u(t) B + C + y(t)

Figure 2.22: A complete block diagram of a continuous state space dynamic system with outputs
and direct measurement feed-through, Eqn. 2.45.

Note that in Fig. 2.22, the internal state vector (bold lines), has typically more elements than either
the input or output vectors (thin lines). The diagram also highlights the fact that as an observer to
the plant, we can relatively easily measure our outputs (or appropriately called measurements),
we presumably know our inputs to the plant, but we do not necessarily have access to the internal
state variables since they are always contained inside the dashed box in Fig. 2.22. Strategies to
estimate these hidden internal states is known as state estimation and are described in section 9.5.
Fig. 2.22 also introduces a convention we will use later in system identification, where I colour
the input signals blue, and the outputs red.
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 49

D IMENSIONS OF THE STATE - SPACE DESCRIPTION

It is important to be clear on the dimensions of the standard state-space system description given
by Eqn. 2.45. We will take the convention that the state vector x is an (n 1) column vector, or
alternatively written as x (n1) . That means we have n states, m inputs, and p measurements,
or
x (n1) , u (m1) , u (p1) (2.46)
Consequently, the dimensions of the model (transition) matrix, A, the input matrix, B, and the
output or measurement matrix, C are

A (nn) , B (nm) , C (pn) , D (pm) . (2.47)

These four matrices in Eqn. 2.45 can be concatenated into one large block thus obtaining a short-
hand way of storing these numbers

# of inputs
n m

A B
n # of states
(n n) (n m)

C D p # of measurements
(p n) (p m)

which is often called a packed matrix notation of the quadruplet (A,B,C,D). If you get confused
about the appropriate dimensions of A, B etc, then this is a handy check, or alternatively you
can use the diagnostic routine abcdchk. Note that in practice, the D matrix is normally the
zero matrix since the input does not generally effect the output immediately without first passing
through the system dynamics.

Following our coloring convention introduced previously, we have coloured the control matrix B
blue in the packed matrix because it is associated with the input, coloured the output matrix C
red because it is associated with the outputs, and the direct term D purple because it is associated
with both inputs and outputs.

2.7.2 C ONVERTING DIFFERENTIAL EQUATIONS TO STATE SPACE FORM

If we have a collection or system of interlinked differential equations, it is often convenient and


succinct to group them together in a vector/matrix collection. Alternatively if we start with an
nth order differential equation, for the same reasons it is advisable to rewrite this as a collection
of n first order differential equations known as Cauchy form.

Given a general linear nth order differential equation

y (n) + a1 y (n1) + a2 y (n2) + + an1 y + an y = b0 u(n) + b1 u(n1) + + bn1 u + bn u (2.48)


50 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

by inspection, we can create the equivalent transfer function as

Y (s) b0 sn + b1 sn1 + + bn1 s + bn


= n (2.49)
U (s) s + a1 sn1 + + an1 s + an
We can cast this transfer function (or alternatively the original differential equation) into state
space in a number of equivalent forms. They are equivalent in the sense that the input/output
behaviour is identical, but not necessarily the internal state behaviour.

The controllable canonical form is



x1 0 1 0 0 x1 0
x2 0
0 1 0


x2
0

.. .. .. .. .. .. .. ..
. = . . . . . . + . u (2.50)

xn1 0 0 0 1 xn1 0
xn a0 a1 a2 an1 xn 1

x1
h

i x2

.. .. . ..
y= b n an b 0 . bn1 an1 b0 . .. b 1 a1 b 0
. + b0 u

(2.51)
xn1
xn

which is useful when designing pole-placement controllers. The observable canonical form is

x1 0 0 0 an x1 b n an b 0
x2 1 0 0 an1 x2 bn1 an1 b0

.. = .. . . .. .. .. .. + .. u (2.52)
. . . . . . . .
xn 0 0 1 a1 xn b 1 a1 b 0

x1
x2
 
..


y= 0 0 0 1 . + b0 u (2.53)

xn1
xn

and is useful when designing observers.

If the transfer function defined by Eqn. 2.49 has real and distinct poles,

Y (s) b0 sn + b1 sn1 + + bn1 s + bn


=
U (s) (s + p1 )(s + p2 ) (s + pn )
c1 c2 cn
= b0 + + + +
s + p1 s + p2 s + pn
then we can derive an especially elegant state space form

x1 p1 0 x1 1
x2
p2


x2
1

.. = . .. .. + .. u (2.54)
. . .
xn 0 pn xn 1

x1
  x2


y= c1 c2 cn .. + b0 u (2.55)
.
xn
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 51

that is purely diagonal. As evident from the diagonal system matrix, this system is totally decou-
pled and possess the best numerical properties for simulation. For systems with repeated roots,
or complex roots, or both, the closest we can come to a diagonal form is the block Jordan form.

We can interconvert between all these above forms using the M ATLAB canon command.

>> G = tf([5 6],[1 2 3 4]); % Define transfer function G = (5s2 + 6s)/(s3 + 2s2 + 3s + 4)
2

>> canon(ss(G),'compan') % Convert TF to the companion or observable canonical form

a =
x1 x2 x3
7 x1 0 0 -4
x2 1 0 -3
x3 0 1 -2

b =
12 u1
x1 1
x2 0
x3 0

17 c =
x1 x2 x3
y1 0 5 -4

d =
22 u1
y1 0

These transformations are discussed further in [152, p515]. M ATLAB uses a variation of this form
when converting from a transfer function to state space in the tf2ss function.

2.7.3 I NTERCONVERTING BETWEEN STATE SPACE AND TRANSFER FUNCTIONS

The transfer function form and the state space form are, by in large, equivalent, and we can con-
vert from one representation to the other. Starting with our generic state space model, Eqn. 2.41,

x = Ax + Bu (2.56)

with initial condition x(0) = 0 and taking Laplace transforms gives

sx(s) = Ax(s) + Bu(s)


x(s) (sI A) = Bu(s)
1
x(s) = (sI A) B u(s) (2.57)
| {z }
Gp (s)

where Gp (s) is a matrix of expressions in s and is referred to as the transfer function matrix. The
M ATLAB command ss2tf (state space to transfer function) performs this conversion.

In the common single-input/single output case,

x = Ax + Bu
y = Cx
52 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

then the single, scalar transfer function is

G(s) = C (sI A)1 B


Now if we express this transfer function as a numerator and a denominator, G(s) = N (s)/D(s),
then the denominator polynomial is
D(s) = det (sI A) (2.58)
which we recognise as the characteristic polynomial for the matrix A. The roots of this polynomial
are the poles of G(s) and are given by the eigenvalues of A.

The numerator polynomial is


N (s) = C adj (sI A) B (2.59)
where adj() is the adjugate, (or sometimes known as the adjoint), of a matrix. The roots of this
polynomial are the zeros of G(s). In M ATLAB we can use the zero command for SISO systems or
tzero for the transmission zeros for MIMO systems.

It is interesting to note that the stability, oscillation and decay rate are governed only by the poles
and hence the system dynamic matrix A, while the zeros are determined by the dynamics as well
as the B and C matrices which in turn are a function of the sensor and manipulated variable
positions, [87, p46].

State space to transfer function conversion

Suppose we wish to convert the following state space description to transfer function form.
   
7 10 4 1
x = x+ u (2.60)
3 4 2 3
From Eqn. 2.57, the transfer function matrix is defined as
1
Gp (s) = (sI A) B
    1  
1 0 7 10 4 1
= s
0 1 3 4 2 3
 1  
(s + 7) 10 4 1
= (2.61)
3 (s 4) 2 3
Now at this point, the matrix inversion in Eqn. 2.61 becomes involved because of the presence of
the symbolic s in the matrix. Inverting the symbolic matrix expression gives the transfer function
matrix.
  
1 s 4 10 4 1
Gp (s) =
(s + 2)(s + 1) 3 s + 7 2 3

4 s + 26
2
s+2 s + 3s + 2
= 2

3(6 + s)
2
s+2 s + 3s + 2
We can directly apply Eqn. 2.57 using the symbolic toolbox in M ATLAB.

>> syms s
2 >> A = [-7 10; -3 4]; B = [4 -1; 2 -3];
>> G = (s*eye(2)-A)\B % Pulse transfer function Gp (s) = (sI A)1 B.
G =
[ 4/(s+2), -(s+26)/(2+s2+3*s)]
[ 2/(s+2), -3*(s+6)/(2+s2+3*s)]
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 53

The method for converting from a state space description to a transfer function matrix described
by Eqn. 2.57 is not very suitable for numerical computation owing to the symbolic matrix in-
version required. However [177, p35] describes a method due to Faddeeva that is suitable for
numerical computation in M ATLAB.

Faddeevas algorithm to convert a state space description, x = Ax + Bu into transfer function


form, Gp (s).

1. Compute a, the characteristic equation of the n n matrix A. (Use poly in M ATLAB.)


2. Set En1 = I.
3. Compute recursively the following n 1 matrices

En1k = AEnk + ank I, k = 1, 2, , n 1

4. The transfer function matrix is then given by

sn1 En1 + sn2 En2 + + E0


Gp (s) = B (2.62)
an sn + an1 sn1 + + a1 s + a0

We can repeat the state space to transfer function example, Eqn. 2.60, given on page 52 using
Faddeevas algorithm starting by computing the characteristic polynomial of A

a(s) = s2 + 3s + 2 = (s + 1)(s + 2)

and with n = 2 we can compute


   
1 0 5 1
E1 = , and E0 =
0 1 2 2

Following Eqn. 2.62 gives a matrix of polynomials in s as,


    
1 1 0 4 10
(sI A)1 = 2 s +
s + 3s + 2 0 1 3 7
 
1 s4 10
=
(s + 1)(s + 2) 3 s+7

which we can recognise as the same expression as we found using the symbolic matrix inverse on
page 52. All that remains is to post multiply by B to obtain the pulse transfer function matrix.

We can implement Faddeevas algorithm using the symbolic toolbox as

syms s
a = poly(A); n = size(A,1);
as = poly2sym(a,'s'); % Construct characteristic polynomial, a(s).
4 E = eye(n,n); I = eye(n,n);
Gp = s(n-1)*E; % En1 = I
for i=n-2:-1:0
E = A*E + a(i+2)*I;
Gp = Gp + si*E;
9 end
Gp = simplify(Gp*B/as); % Eqn. 2.62.

Finally, we can also use the control toolbox for the conversion from state space to transfer function
form. First we construct the state space form,
54 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

A = [-7 10; -3,4]; B = [4 -1; 2 -3]; % Continuous state space example from Eqn. 2.60.
C = eye(size(A));
sys = ss(A,B,C,[]) % construct system object

where in this case if leave out the D matrix, M ATLAB assumes no direct feed though path. We
now can convert to the transfer function form:

>> sys_tf = minreal(tf(sys)) % Convert to TF & remember to cancel common factors


2

Transfer function from input 1 to output...


4
#1: -----
s + 2
7

2
#2: -----
s + 2

12 Transfer function from input 2 to output...


-s - 26
#1: -------------
s2 + 3 s + 2

17 -3 s - 18
#2: -------------
s2 + 3 s + 2

Once again we get the same pulse transfer function matrix. Note however that it is also possible
to use ss2tf.

T RANSFER FUNCTION TO STATE SPACE

To convert in the reverse direction, we can use the M ATLAB function tf2ss (transfer function to
state space) thus converting an arbitrary transfer function description to the state space format of
Eqn 2.41. For example starting with the transfer function

(s + 3)(s + 4) s2 + 7s + 12
G(s) = = 3
(s + 1)(s + 2)(s + 5) s + 8s + 17s + 10

we can convert to state space form using tf2ss.

1 >>num = [1 7 12]; % Numerator: s2 + 7s + 12


>>den = [1 8 17 10]; % Denominator: s3 + 8s + 17s + 10
>>[A,B,C,D] = tf2ss(num,den) % convert to state space

which returns the following four matrices



8 17 10 1
A= 1 0 0 , B= 0
0 1 0 0
 
C = 1 7 12 , D=0
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 55

This form is a variation of the controllable canonical form described previously in 2.7.2. This
is not the only state space realisation possible however as the C ONTROL T OOLBOX will return a
slightly different ABCD package

>> Gc = tf([1 7 12],[1 8 17 10]) % Transfer function in polynomial form


2 Transfer function:
s2 + 7 s + 12
-----------------------
s3 + 8 s2 + 17 s + 10

7 >>[A,B,C,D] = ssdata(Gc) % extract state space matrices


A =
-8.0000 -4.2500 -1.2500
4.0000 0 0
0 2.0000 0
12 B =
2
0
0
C =
17 0.5000 0.8750 0.7500
D =
0

Alternatively we could start with zero-pole-gain form and obtain yet another equivalent state
space form.

1 G = zpk([-3 -4],[-1 -2 -5],1) % Transfer function model in factored format


Gss = ss(G)

2.7.4 S IMILARITY TRANSFORMATIONS

The state space description of a differential equation is not unique. We can transform from one
description to another by using a linear invertible transformation such as x = Tz. Geometrically
in 2 dimensions, this is equivalent to rotating the axes or plane. When one rotates the axes, the
inter-relationship between the states do not change, so the transformation preserves the dynamic
model.

Suppose we have the dynamic system

x = Ax + Bu (2.63)

which we wish to transform in some manner using the non-singular transformation matrix, T,
where x = Tz. Naturally the reverse transformation z = T1 x exists because we have restricted
ourselves to consider only the cases when the inverse of T exists. Writing Eqn. 2.63 in terms of
our new variable z we get

d(Tz)
= ATz + Bu
dt
z = T1 ATz + T1 Bu (2.64)

Eqn 2.64 and Eqn 2.63 represent the same dynamic system. They have the same eigenvalues hence
the similarity transform, but just a different viewpoint. The mapping from A to T1 AT is called
56 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

a similarity transform and preserves the eigenvalues. These two matrices are said to be similar.
The proof of this is detailed in [85, p300] and [152, p513514].

The usefulness of these types of transformations is that the dynamics of the states are preserved
(since the eigenvalues are the same), but the shape and structure of the system has changed.
The motivation is that for certain operations (control, estimation, modelling), different shapes are
more convenient. A pure, (or nearly), diagonal shape of the A matrix for example has much better
numerical properties than a full matrix. This also has the advantage that less computer storage
and fewer operations are needed.

To convert a system to a diagonal form, we use the transformation matrix T, where the columns
of T are the eigenvectors of A. Systems where the model (A) matrix is diagonal are especially
easy to manipulate. We can find the eigenvectors T and the eigenvalues e of a matrix A using the
eig command, and construct the new transformed, and hopefully diagonal system matrix,

[T,e] = eig(A) % find eigenvectors & eigenvalues of A


V = T\A*T % New system matrix, T1 AT, or use canon

C REATING RANDOM MATRICES WITH SPECIFIED EIGENVALUES

Another use is when testing new control or estimation algorithms, it is sometimes instructive to
devise non-trivial test systems with specified properties. For example you may wish to test an
interacting plant that is stable with both overdamped and underdamped oscillatory modes. That
is we wish to construct a full random A matrix, but with specified eigenvalues. We can use
the similarity transformations to obtain these models by starting with a simple diagonal matrix
containing the desired eigenvalues.

>> n = 4;
>> D = diag([1:n]); % Specify known eigenvalues: 1,2,3,4
3 >> T = randn(n,n); % Generate a random 4*4 matrix
>> A = T\D*T % Form a random matrix with the specified eigenvalues
A =
1.9092 -2.9399 -0.1916 4.0369
-0.8617 3.9709 -0.3746 -3.1316
8 0.5293 1.5593 2.9287 -1.8495
0.4539 0.7573 0.5479 1.1912

We can check that the matrix A has the required eigenvalues,

1 >> eig(A) % should be the same as D.


ans =
4.0000
1.0000
3.0000
6 2.0000

>>cond(A)
ans =
13.6753

which it does. However we notice that the condition number of A is not the same as the condition
number of D which is simply 4, (maximum eigenvalue over minimum eigenvalue). One way to
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 57

generate a random matrix with the same condition number is to force the random matrix to be
orthogonal which has a condition number of 1. That strategy also has the added benefit that we
avoid a matrix inversion when constructing matrix A.

>> V = orth(randn(n)) % Random orthogonal matrix where VT = V1


>> A2 = V'*D*V
A2 =
3.2095 -0.1270 0.6526 -0.6599
5 -0.1270 2.2961 -0.3519 -0.4404
0.6526 -0.3519 1.2703 -0.1186
-0.6599 -0.4404 -0.1186 3.2240
>> eig(A2) % Eigenvalues should be 1,2,3,4
ans =
10 4.0000
3.0000
1.0000
2.0000
>> cond(A2) % condition # should be same as D which is 4
15 ans =
4.0000

Other useful transformations such as the controllable and observable canonical forms are covered
in [152, 64 p646 ]. The M ATLAB function canon can convert state space models to diagonal or
observable canonical form (sometimes known as the companion form). Note however the help file
for this routine discourages the use of the companion form due to its numerical ill-conditioning.

M ATLAB comes with some utility functions to generate test models such as ord2 which generates
stable second order models of a specified damping and natural frequency, and rmodel is a flexible
general stable random model builder or arbitrary order.

2.7.5 I NTERCONVERTING BETWEEN TRANSFER FUNCTIONS FORMS

The previous section described how to convert between different representations of linear dy-
namic systems such as differential equations, transfer functions and state space descriptions. This
section describes the much simpler task of converting between the different ways we can write
transfer functions.

Modellers tend to think in continuous time systems, G(s), and in terms of process gain and time
constants, so will naturally construct transfer functions in factored form as

K i (i s + 1)
(2.65)
j (j s + 1)

where the all the variables of interest such as time constants j are immediately apparent. On
the other hand, system engineers tend to think in terms of poles and zeros, so naturally construct
transfer functions also in factored form, but with a different scaling

K i (s + zi )
(2.66)
j (s + pi )

so where now the poles, pj , and zeros zi are immediately apparent. This is the form that M ATLAB
uses in the zeros-pole-gain format, zpk.

Finally the hardware engineer would prefer to operate in the expanded polynomial form, (partic-
58 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

ularly in discrete cases), where the transfer function is of the form

bm sm + bm1 sm1 + + b0
(2.67)
sn + an1 sn1 + + a0

This is the form that M ATLAB uses in the transfer-function format, tf. Note that the leading
coefficient in the denominator is set to 1. As a M ATLAB user, you can define a transfer function
that does not follow this convention, but M ATLAB will quietly convert to this normalised form if
you type something like G = tf(zpk(G)).

The inter-conversions between the forms is not difficult; between expressions Eqn. 2.65 and Eqn. 2.66
simply require some adjusting of the gains and factors, while to convert from Eqn. 2.67 requires
one to factorise polynomials.

For example, the following three transfer function descriptions are all equivalent

2 (10s + 1)(3s + 1) 1.5 (s 0.3333)(s + 0.1) 60s2 + 14s + 2


= =
(20s + 1)(2s + 1)(s + 1) (s + 1)(s + 0.5)(s + 0.05) 40s3 + 62s2 + 23s + 1
| {z } | {z } | {z }
time constant zero-pole-gain expanded polynomial

It is trivial to interconvert between the zero-pole-gain, Eqn. 2.66, and the transfer function for-
mats, Eqn. 2.67, in M ATLAB, but it is less easy to convert to the time constant description. List-
ing 2.4 extracts from an arbitrary transfer function form the time constants, , the numerator time
constants, , and the plant gain K.

Listing 2.4: Extracting the gain, time constants and numerator time constants from an arbitrary
transfer function format
Gplant = tf(2*mconv([10 1],[-3 1]),mconv([20 1],[2 1],[1 1])) % TF of interest
G = zpk(Gplant); % Convert TF description to zero-pole-gain

4 Kp = G.k;
p = cell2mat(G.p); z = cell2mat(G.z); % Extract poles & zeros
delay0 = G.iodelay ; % Extract deadtime (if any)

tau = sort(-1./p,'descend'); % Convert (& sort) to time constants, 1/(j s + 1)


9 ntau = sort(-1./z,'descend'); % Convert to numerator time constants, (i s + 1)
K = Kp*prod(tau)/prod(ntau); % Adjust plant gain

We could of course use the control toolbox functions pole and zero to extract the poles and
zeros from an arbitrary LTI model.

2.7.6 T HE STEADY STATE

The steady-state, xss , of a general nonlinear system x = f (x, u) is the point in state space such
that all the derivatives are zero, or the solution of

0 = f (xss , u) (2.68)

If the system is linear, x = Ax + Bu, then the steady state can be evaluated algebraically in closed
form
xss = A1 Bu (2.69)
Consequently to solve for the steady-state one could invert the model matrix A but which may be
illconditioned or computationally time consuming. Using M ATLAB one should use xss = -A\B*u.
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 59

If A has no inverse, then no (or alternatively infinite) steady states exist. An example of a process
that has no steady state could be a tank-flow system that has a pump withdrawing fluid from
the outlet at a constant rate independent of liquid height say just exactly balancing an input flow
shown in the left hand schematic in Fig. 2.23. If the input flow suddenly increased, then the level
will rise until the tank eventually overflows. If instead the tank was drained by a valve partially
open at a constant value, then as the level rises, the increased pressure (head) will force more
material out through the valve, (right-hand side of Fig. 2.23.) Eventually the system will rise to a
new steady state. It may however overflow before the new steady state is reached, but that is a
constraint on the physical system that is outside the scope of the simple mathematical description
used at this time.
Flow in

height, h
h

flow out is constant flow out = f(height)

constant flow pump restriction

Figure 2.23: Unsteady (left) and steady (right) states for level systems

If the system is nonlinear, there is the possibility that multiple steady states may exist. To solve for
the steady state of a nonlinear system, one must use a nonlinear algebraic solver such as described
in chapter 3.

Example The steady state of the differential equation

d2 y dy
2
+7 + 12y = 3u
dt dt
where dydt = y = 0 at t = 0 and u = 1 , t 0 can be evaluated using Laplace transforms and the
final value theorem. Transforming to Laplace transforms we get

s2 Y (s) + 7sY (s) + 12Y (s) = 3U (s)


Y (s) 3 3
= 2 =
U (s) s + 7s + 12 (s + 3)(s + 4)

while for a step input

1 3
Y (s) = 2
s s + 7s + 12
60 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

The final value theorem is only applicable if the system is stable. To check, we require that the
roots of the denominator, s2 + 7s + 12, lie in the left hand plane,

6 72 4 12
s= = 4 and 3
2
Given that both roots have negative real parts, we have verified that our system is stable and we
are allowed to apply the final value theorem to solve for the steady-state, y(),
3 3
lim y(t) = lim sY (s) = lim = = 0.25
t s0 s2 + 7s + 12
s0 12
Using the state space approach to replicate the above, we first cast the second order differential
equation into two first order differential equations using the controllable canonical form given in
Eqn. 2.51. Let z1 = y and z2 = dydt , then
   
0 1 0
z = z+ u
12 7 3

and the steady state is now


    
7/12 1/12 0 0.25
zss = A1 Bu = 1=
1 0 3 0
Noting that z1 = y, we see that the steady state is also at 0.25. Furthermore, the derivative term
(z2 ) is zero, which is as expected at steady state.

2.7.7 P OLES AND ZEROS

As mentioned in section 2.7.3, the poles of a state space dynamic system are the roots of the
polynomial det(sI A)1 and the zeros are the roots of C adj(sI A)1 B. The concept of zeros,
however in multivariable systems is more complicated than poles. Because forming the adjugate
is awkward, it is easier to note the identity
C adj(sI A)1 B = det(R(s))
where  
def sI A B
R(s) =
C 0
That means we can find the transmission zeros of a state space system by computing generalised
eigenvalues of the matrix pencil
    
A B I 0
, ,
C 0 0 0
as detailed in [87, p55]. Listing 2.5 does this calculation which essentially duplicates tzero in
M ATLAB.

Listing 2.5: Find the transmission zeros for a MIMO system


function [tzeros ] = tzero2(G)
% Transmission zeros for a MIMO system (See also tzero.m)

[A,B,C,D] = ssdata(ss(G)); % convert to state space


5 R1 = [-A, B; C, D]; R2 = [-eye(size(A)), zeros(size(B)); ...
zeros(size(C)), zeros(size(D))];

tzeros = eig(R1,R2); % Extract generalised eigenvalues


tzeros(isinf(tzeros)) = []; % Drop zeros at
10 end
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 61

2.8 S OLVING THE VECTOR DIFFERENTIAL EQUATION

Since we can solve differential equations by inverting Laplace transforms, we would expect to be
able to solve state space differential equations such as Eqn. 2.41 in a similar manner. If we look
at the Laplace transform of a simple linear scalar differential equation, x = ax + bu we find two
terms,

sX(s) x0 = aX(s) + bU (s)


x0 b
X(s) = + U (s) (2.70)
sa sa
One of these terms is the response of the system owing to the initial condition, x0 eat , and is called
the homogeneous solution, and the other term is due to the particular input we happen to be
using. This is called the particular integral, and we must know the form of the input, u(t), before
we solve this part of the problem. The total solution is the sum of the homogeneous and particular
components.

T HE HOMOGENEOUS SOLUTION

For the moment we will consider just the homogeneous solution to our vector differential equa-
tion. That is, we will assume no driving input, or u(t) = 0. (In the following section we will add
in the particular integral due to a non-zero input.)

Our vector differential equation, ignoring any input, is simply

x = Ax, x(t = 0) = x0 (2.71)

Taking Laplace transforms, and not forgetting the initial conditions, we have

sx(s) x0 = A x(s)
x(s) (sI A) = x0 (s)
x(s) = (sI A)1 x0

Finally inverting the Laplace transform back to the time domain gives
n o
x(t) = L1 (sI A)1 x0 (2.72)

Alternatively we can solve Eqn. 2.71 by separating the variables and integrating obtaining

x(t) = eAt x0 (2.73)

where the exponent of a matrix, eAt , is itself a matrix of the same size as A. We call this matrix
exponential the transition matrix because it transforms the state vector at some initial time x0 to
some point in the future, xt . We will give it the symbol (t).

The matrix exponential is defined just as in the scalar case as a Taylor series expansion,

A2 t2 A3 t3
eAt or (t) = I + At +
def
+ +
2! 3!

X Ak tk
= (2.74)
k!
k=0

although this series expansion method is not recommended as a reliable computational strategy.
Better strategies are outlined on page 65.
62 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS
n o
Comparing Eqn. 2.72 with Eqn. 2.73 we can see that the matrix eAt = L1 (sI A)1 .

So to compute the solution, x(t), we need to know the initial condition and a strategy to numeri-
cally compute a matrix exponential.

T HE PARTICULAR SOLUTION

Now we consider the full differential equation with nonzero input x = Ax + Bu. Building on the
solution to the homogeneous part in Eqn. 2.73, we get
Z t
x(t) = t x0 + t Bu d (2.75)
0

where now the second term accounts for the particular input vector u.

Eqn. 2.75 is not particularly useful as written as both terms are time varying. However the con-
tinuous time differential equation can be converted to a discrete time differential equation that is
suitable for computer control implementation provided the sampling rate is fixed, and the input
is held constant between the sampling intervals. We would like to convert Eqn. 2.41 to the discrete
time equivalent, Eqn. 2.42, repeated here

xk+1 = xk + uk (2.76)

where xk is the state vector x at time t = kT where T is the sample period. Once the sample period
is fixed, then and are also constant matrices. We have also assumed here that the input vector
u is constant, or held, over the sample interval, which is the norm for control applications.

So starting with a known xk at time t = kT , we desire the state vector at the next sample time,
xk+1 , or
Z (k+1)T
A T
xk+1 = e xk + e A (k+1)T
eA Bu dt (2.77)
kT

But as we have assumed that the input u is constant using a zeroth-order hold between the sam-
pling intervals kT and (k + 1)T , Eqn. 2.77 simplifies to
Z T
xk+1 = eAT xk + eA Buk d (2.78)
0

where = T t. For convenience, we can define two new matrices as

= eAT
def
(2.79)
Z T !
=
def
e A
d B (2.80)
0

which gives us our desired transformation in the form of Eqn. 2.76. In summary, to discretise
x = Ax+Bu at sample interval T , we must compute a matrix exponential, Eqn. 2.79, and integrate
a matrix exponential, Eqn. 2.80.

Note that Eqn. 2.78 involves no approximation to the continuous differential equation provided
the input is constant over the sampling interval. Also note that as the sample time tends to zero,
the state transition matrix tends to the identity, I.

If the matrix A is non-singular, then


 
= eAT I A1 B (2.81)
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 63

is a simpler expression than the more general Eqn. 2.80.

Double integrator example. The single-input/single output double integrator system, G(s) =
1/s2 , can be represented in continuous state space using two states as
   
0 0 1
x = x+ u (2.82)
1 0 0
 
y= 0 1 x (2.83)

At sample time T , the state transition matrix is from Eqn. 2.79,


   
0 0 1 0
= eAT = exp =
T 0 T 1

and the control matrix is given by Eqn. 2.80


Z !
T
= e A
d B
0
Z T   ! 
1 0 1
= d
0 1 0

T 0  1  T
= T2 = T2
T 0
2 2
For small problems such as this, the symbolic toolbox helps do the computation.

>> A = [0 0; 1 0]; B = [1; 0];


>> syms T lambda % Symbolic sample time T and
>> Phi = expm(A*T) % (T )
Phi =
5 [ 1, 0]
[ T, 1]
>> Delta = int(expm(A*lambda),lambda,0,T)*B % (T )
Delta =
[ T]
10 [ 1/2*T2]

Symbolic example with invertible A matrix. We can discretise the continuous state space system
   
1.5 0.5 1
x = x+ u
1 0 0

analytically at a sample time T by computing matrix exponentials symbolically.

>> A = [-3/2, -1/2; 1, 0]; B = [1;0];

>> syms T
>> Phi = expm(A*T)
5 Phi =
[ -exp(-1/2*T)+2*exp(-T), exp(-T)-exp(-1/2*T)]
[ -2*exp(-T)+2*exp(-1/2*T), 2*exp(-1/2*T)-exp(-T)]

Since A is invertible in this example, we can use the simpler Eqn. 2.81
64 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

 
>> Delta = (Phi - eye(2))*A\B % = eAT I A1 B
Delta =
3 [ 2/(exp(-1/2*T)*exp(-T)-exp(-T)-exp(-1/2*T)+1)*(exp(-T)-exp(-1/2*T))]
[ -2*(-exp(-1/2*T)+2*exp(-T)-1)/(exp(-1/2*T)*exp(-T)-exp(-T)-exp(-1/2*T)+1)]
>> simplify(Delta)
ans =
[ 2*exp(-1/2*T)/(exp(-T)-1)]
8 [ -2*(2*exp(-1/2*T)+1)/(exp(-T)-1)]

Of course it is evident from the above example that the symbolic expressions for (T ) and (T )
rapidly become unwieldy for dimensions much larger than about 2. For this reason, analytical
expressions are of limited practical worth. The alternative numerical schemes are discussed in
the following section.

2.8.1 N UMERICALLY COMPUTING THE DISCRETE TRANSFORMATION

Calculating numerical values for the matrices and can be done by hand for small dimensions
by converting to a diagonal or Jordan form, or numerically using the exponential of a matrix.
Manual calculations are neither advisable nor enjoyable, but [20, p35] mention that if you first
compute
Z T
AT 2 A2 T 3 Ak1 T k
= eA d = IT + + + + +
0 2! 3! k!
then
= I + A and = B (2.84)
A better approach, at least when using M ATLAB, follows from Eqn. 2.79 and Eqn. 2.80 where
d
= A = A (2.85)
dt
d
= B (2.86)
dt
These two equations can be concatenated to give
    
d A B
= (2.87)
dt 0 I 0 I 0 0
R R
which is in the same form as da/dt = ab. Rearranging this to da/a = b dt leads to the analytical
solution     
A B
= exp T (2.88)
0 I 0 0
enabling us to extract the required and matrices provided we can reliably compute the ex-
ponential of a matrix. The M ATLAB C ONTROL T OOLBOX routine to convert from continuous to
discrete systems, c2d, essentially follows this algorithm.

We could try the augmented version of Eqn. 2.88 to compute both and with one call to the
matrix exponential function for the example started on page 63.

>> A = [-3/2, -1/2; 1, 0]; B = [1;0];


2 >> syms T

>> [n,m] = size(B); % extract dimensions of A & B.


 
A B
>> Aa = [A,B;zeros(m,n+m)] % Augmented A matrix, A =
0 0
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 65

Aa =
7 -1.5 -0.5 1.0
1.0 0.0 0.0
0.0 0.0 0.0
>> Phi_a = expm(Aa*T); % compute exponential, = exp(AT )
>> Phi = Phi_a(1:n,1:n) % Extract upper left block of composite solution
12 Phi =
[ -exp(-1/2*T)+2*exp(-T), exp(-T)-exp(-1/2*T)]
[ -2*exp(-T)+2*exp(-1/2*T), 2*exp(-1/2*T)-exp(-T)]
>> Delta = Phi_a(1:n,n+1:end)
Delta =
17 [ -2*exp(-T)+2*exp(-1/2*T)]
[ 2*exp(-T)-4*exp(-1/2*T)+2]

R ELIABLY COMPUTING MATRIX EXPONENTIALS

There are in fact several ways to numerically compute a matrix exponential. Ogata gives three
computational techniques, [152, pp526533], and a paper by Cleve Moler (one of the original
M ATLAB authors) and Charles Van Loan is titled Nineteen dubious ways to compute the exponential
of a matrix5 , and contrasts methods involving approximation theory, differential equations, matrix
eigenvalues and others. Of these 19 methods, two found their way into M ATLAB, namely expm,
which is the recommended default strategy, and expm1 intended to compute ex 1 for small x .

The one time when matrix exponentials are trivial to compute is when the matrix is diagonal.
Physically this implies that the system is totally decoupled since the matrix A is comprised of
only diagonal elements and the corresponding exponential matrix is simply the exponent of the
individual elements. So given the diagonal matrix

1 0 0
D = 0 2 0
0 0 3
then the matrix exponential is simply

e1 0 0
D
e = 0 e2 0
0 0 e3

which is trivial and reliable to compute. So one strategy then is to transform our system to a
diagonal form (if possible), and then simply find the standard scalar exponential of the individual
elements. However some matrices, such as those with multiple eigenvalues, is is impossible to
convert to diagonal form, so in those cases the best we can do is convert the matrix to a Jordan
block form as described in [152, p527], perhaps using the jordan command from the Symbolic
toolbox.

However this transformation is very sensitive to numerical roundoff and for that reason is not
used for serious computation. For example the matrix
 
1 1
A=
1
with = 0 has a Jordan form of  
1 1
0 1
5 SIAM Review, vol. 20, A 1978, pp802836 and reprinted in [160, pp649680]. In fact 25 years later, an update was

published with some recent developments.


66 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

but for 6= 0, then the Jordan form drastically changes to the diagonal matrix
 
1+ 0
0 1

In summary, for serious numerical calculation we should use the matrix exponential function
expm. Remember not to confuse finding the exponent of a matrix, expm, with the M ATLAB func-
tion exp which simply finds the exponent of all individual elements in the matrix.

2.8.2 U SING M ATLAB TO DISCRETISE SYSTEMS

All of the complications of discretising continuous systems to their discrete equivalent can be
circumvented by using the M ATLAB command c2d which is short for continuous to discrete.
Here we need only to pass the continuous system of interest, and the sampling time. As an
example we can verify the conversion of the double integrator system shown on page 2.8.

G = tf(1,[1 0 0]) % Continuous system G(s) = 1/s2 in transfer function form


2 Gc = ss(G) % Convert to continuous state space
Gd = c2d(Gc,2) % Convert to discrete state space with a sample time of T = 2

a =
x1 x2
7 x1 1 0
x2 2 1

b =
u1
12 x1 2
x2 2

c =
x1 x2
17 y1 0 1

d =
u1
y1 0
22

Sampling time (seconds): 2


Discrete-time state space model.

Unlike in the analytical case presented previously, here we must specify a numerical value for the
sample time, T .

Example: Discretising an underdamped second order system with a sample time of T = 3 fol-
lowing Eqn. 2.88 using M ATLAB to compute the matrix exponential of the augmented system.

1 [A,B,C,D] = ord2(3, 0.5); % generate a second order system


Ts = 3.0; % sample time

[na,nb] = size(B);
X = expm([A,B; zeros(nb,na+nb)]*Ts); % matrix exponential
6 Phi = X(1:na,1:na); % Pull off blocks to obtain & .
Del = X(1:na,na+1:end);
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 67

We can use M ATLAB to numerically verify the expression found for the discretised double inte-
grator on page 63 at a specific sample time, say T = 4.

The double integrator in input/output form


is
1 >>Gc = tf(1,[1,0,0])% G(s) = 1/s2
G(s) = 2 >>Gc_ss = ss(Gc);% state space
s
3 >>dt = 4; % sample time
which in packed state space notation (refer
page 49) is >>Gd_ss = c2d(Gc_ss,dt)
a =
  0 0 1 x1 x2
A B
= 1 0 0 8 x1 1.000 0
C D
0 1 0 x2 4.000 1.000

Using a sample time of T = 4, we can com- b =


pute u1
    13 x1 4.000
1 0 1 0 x2 8.000
= =
T 1 4 1
    c =
T 4 x1 x2
= T 2 = 16
2 2 18 y1 0 1.000

We can verify this using c2d in M ATLAB as d =


demonstrated in the M ATLAB script oppo- u1
site. y1 0

Once the two system matrices and are known, solving the difference equation for further val-
ues of k just requires the repeated application of Eqn. 2.42. Starting from known initial conditions
x0 , the state vector x at each sample instant is calculated thus

x1 = x0 + u0
x2 = x1 + u1 = 2 x0 + u0 + u1
x3 = x2 + u2 = 3 x0 + 2 u0 + u1 + u2
.. ..
. .
n1
X
n
xn = x0 + n1k uk (2.89)
k=0

The M ATLAB function ltitr which stands for linear time invariant time response or the more
general dlsim will solve the general linear discrete time model as in 2.89. In special cases such
as step or impulse tests, you can use dstep, or dimpulse.

Problem 2.4 1. Evaluate An where  


1 1
A=
1 0
for different values of n. What is so special about the elements of An ? (Hint: find the
eigenvalues of A.)

2. What is the determinant of An ?

3. Write a couple of M ATLAB m-functions to convert between the A, B, C and D form and the
packed matrix form.
68 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

4. Complete the state space to transfer function conversion analytically started in 2.7.3,
Eqn. 2.61. Compare your answer with using ss2tf.

A submarine example A third-order model of a submarine from [63, p416 Ex9.17] is



0 1 0 0
x = 0.0071 0.111 0.12 x + 0.095 u (2.90)
0 0.07 0.3 0.072

where the state vector x is defined as


 
x = d
dt

The state is the inclination of the submarine and is the angle of attack above the horizontal.
The scalar manipulated variable u is the deflection of the stern plane. We will assume that of the
three states, we can only actually measure two of them, and , thus
 
1 0 0
C=
0 0 1

In this example, we consider just three states x, one manipulated variable u, and two outputs, y.

The stability of the open-loop system is given by the eigenvalues of A. In M ATLAB, the command
eig(A) returns
0.0383 + 0.07i
eig(A) = 0.0383 0.07i
0.3343
showing that all eigenvalues have negative real parts, indicating that the submarine is oscillatory,
though stable. In addition, the complex conjugate pair indicates that there will be some oscillation
to the response of a step change in stern plane.

We can simulate the response of our submarine system (Eqn 2.90) to a step change in stern plane
movement, u, using the step command. The smooth plot in Fig. 2.24 shows the result of this
continuous simulation, while the staired plot shows the result of the discrete simulation using a
sample time of T = 5. We see two curves corresponding to the two outputs.

Listing 2.6: Submarine simulation


A = [0,1,0;-0.0071,-0.111,0.12;0,0.07,-0.3]; B = [0,-0.095,0.072]';
C = [1 0 0;0 0 1];
3

Gc = ss(A,B,C,0) % Continuous plant, Gc (s)

dt = 5; % sample time T = 5 (rather coarse)


Gd = c2d(Gc,dt) % Create discrete model
8

step(Gc,Gd); % Do step response and see Fig. 2.24.

Fig. 2.24 affirms that the openloop process response is stable, supporting the eigenvalue analysis.
Notice how the step command automatically selected appropriate time scales for the simulation.
How did it do this?

The steady state of the system given that u = 1 is, using Eqn 2.69,

1 >> u = 1;
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 69

0
G
c
G
5 d
x1

10

15

0.4

0.3
2

0.2
x

0.1
Figure 2.24: Comparing the continuous and
0
0 20 40 60 80 discrete step responses of the submarine
time model.

>>xss = -A\B*u; % Steady-states given by xss = ABu.


>>yss = C*xss
yss = % See steady-state outputs in Fig. 2.24.
-9.3239
6 0.2400

which corresponds to the final values given in Fig. 2.24.

We can verify the results from the c2d routine using Eqns 2.79 and 2.80, or in this case since A is
invertible, Eqn. 2.81.

>> Phi = expm(A*5.0); % Dont forget the m in expmatrix()


 
>> Delta = (Phi-eye(size(A)))*(A\B) % = eAT I A1 B

This script should give the same and matrices as the c2d function since A is non-singular.

Given a discrete or continuous model, we can compute the response to an arbitrary input se-
quence using lsim:

t = [0:dt:100]'; U = sin(t/20); % Arbitrary input U (t)

3 lsim(Gc,U,t) % continuous system simulation, y(t)


lsim(Gd,U) % discrete system simulation, yk

We could explicitly demand a first-order-hold as opposed to the default zeroth order by setting
the options for c2d.

1 optn = c2dOptions('Method','foh') % Ask for a first-order hold


Gd2 = c2d(Gc,5,optn) % Compare the step response of this with a zeroth-order hold
70 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Problem 2.5 By using a suitable state transformation, convert the following openloop system

x1 = x2
x2 = 10 2x2 + u

where the input u is the following function of the reference r,


 
u = 10 + 9r 9 4 x

into a closed loop form suitable for simulation in M ATLAB using the lsim function. (ie: x =
Ax + Br) Write down the relevant M ATLAB code segment. (Note: In practice, it is advisable to
use lsim whenever possible for speed reasons.)

Problem 2.6 1. Write an m-file that returns the state space system in packed matrix form that
connects the two dynamic systems G1 and G2 together,

Gall = G1 G2

Assume that G1 and G2 are already given in packed matrix form.

2. (Problem from [186, p126]) Show that the packed matrix form of the inverse of Eqn. 2.45,
u = G1 y, is
 
A BD1 C BD1
G1 = (2.91)
D1 C D1
assuming D is invertible, and we have the same number of inputs as outputs.

2.8.3 T HE DISCRETE STATE SPACE EQUATION WITH TIME DELAY

The discrete state space equation, Eqn. 2.41, does not account for any time delay or dead time
between the input u and the output x. This makes it difficult to model systems with delay. How-
ever, by introducing new variables we can accommodate dead time or time delays. Consider the
continuous differential equation,
xt = Axt + But (2.92)
where is the dead time and in this particular case, is exactly equal to 2 sample times ( = 2T ).
The discrete time equivalent to Eqn 2.92 for a two sample time delay is

xk+3 = xk+2 + uk (2.93)

which is not quite in our standard state space form of Eqn. 2.42 owing to the difference in time
subscripts between u and x.

Now let us introduce a new vector of state variables, z where



xk
def
zk = xk+1 (2.94)
xk+2

is sometimes known as a shift register or tapped delay line of states. Using this new state vector,
we can write the dynamic system, Eqn. 2.93 compactly as

xk+1 Ixk+1
zk+1 = xk+2 = Ixk+2 (2.95)
xk+3 xk+2 + uk
2.9. STABILITY 71

or in standard state space form as



0 I 0 0
zk+1 = 0 0 I z k + 0 uk (2.96)
0 0

This augmented system, Eqn. 2.96 (with 2 units of dead time) is now larger than the original
Eqn. 2.93 given that we have 2n extra states. Furthermore, note that the new augmented transition
matrix in Eqn. 2.96 is now no longer of full rank (provided it was in the first place). If we wish to
incorporate a delay , of sample times ( = T ), then we must introduce dummy state vectors
into the system creating a new state vector z with dimensions (n 1);
 
zk = xk xk+1 xk+2 xk+

The augmented state transition matrix and augmented control matrix are of the form

0 I 0 0 0
0 0 I 0 0

0 0 0 0 0

= . . . . , and = 0 (2.97)
.. .. .. . . ...

..
0 0 0 I .
0 0 0

Now the discrete time equation is


zk+1 = zk + uk (2.98)
Since all the introduced states are unobservable, we have
 
xk = I 0 0 0 zk (2.99)

and that the output equation is amended to


 
yk = C I 0 0 0 zk (2.100)

If the dead time is not an exact multiple of the sample time, then more sophisticated analysis is
required. See [71, p174]. Systems with a large value of dead time become very unwieldy as a
dummy vector is required for each multiple of the sample time. This creates large systems with
many states that possibly become numerically ill-conditioned and difficult to manipulate.

Problem 2.7 1. Simulate the submarine example (Eqn 2.90) but now introducing a dead time
of 2 sample time units.
2. Explain how the functions step and dstep manage to guess appropriate sampling rates,
and simulation horizons. Under what circumstances will the heuristics fail?

2.9 S TABILITY

Stability is a most desirable characteristic of a control system. As a control designer, you should
only design control systems that are stable. For this reason, one must at least be able to analyse
a potential control system in order to see if it is indeed theoretically stable at least. Once the
controller has been implemented, it may be too late and costly to discover that the system was
actually unstable.

One definition of stability is that a system of differential equations is stable if the output is
bounded for all time for any given bounded input, otherwise it is unstable. This is referred to
as bounded input bounded output (BIBO) stability.
72 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

While for linear systems, the concept of stability is well defined and relatively straight forward
to evaluate6 , this is not the case for nonlinear systems which can exhibit complex behaviour. The
next two sections discuss the stability for linear systems. Section 2.9.4 discusses briefly techniques
that can be used to assess stability for nonlinear systems.

Before any type of stability analysis can be carried out, it is important to define clearly what sort
of analysis is desired. For example is a Yes/No result acceptable, or do you want to quantify
how close to instability the current operating point is? Is the system linear? Are the nonlinearities
hard or soft? Fig. 2.25 highlights some of these concerns.

System stability

Transfer function Nonlinear


LTIs

Linearise
non-
polynomial Accuracy ?
yes/no
Pade

Routh array marginal


Lyapunov
Jury Test Simulate
eigenvalues
roots of denom. May be hard
Extensive testing?
Bode plots
Gain & phase margins

Figure 2.25: Issues and approaches in assessing system stability

2.9.1 S TABILITY IN THE CONTINUOUS DOMAIN

The most important requirement for any controller is that the controller is stable. Here we de-
scribe how one can analyse a continuous transfer function to determine whether it is stable or
not.

First recall the conditions for BIBO stability. In the Laplace domain, the transfer function system
is stable if all the poles (roots of the denominator) have negative real parts. Thus if the roots of the
denominator are plotted on the complex plane, then they must all lie to the left of the imaginary
axis or in the Left Hand Plane (LHP). In other words, the time domain solution of the differential
equation must contain only eat terms and no terms of the form e+at , assuming a is positive.

Why is the stability only determined by the denominator of the transfer function and not by the
numerator or the input?

1. First the input is assumed bounded and stable (by BIBO definition).
6 Actually, the stability criterion can be divided into 3 categories; Stable, unstable and critically (un)stable. Physically

critical stability never really occurs in practice.


2.9. STABILITY 73

2. If the transfer function is expanded as a sum of partial fractions, then the time solution terms
will be comprised of elements that are summed together. For the system to be unstable,
at least one of these individual terms must be unstable. Conversely if all the terms are
stable themselves, then the summation of these terms will also be stable. The input will
also contribute separate fractions that are summed, but these are assumed to be stable by
definition, (part 1).

Note that it is assumed for the purposes for this analysis, that the transfer function is written in
its simplest form, that it is physically realisable, and that any time delays are expanded as a Pade
approximation.

In summary to establish the stability of a transfer function, we could factorise the denomina-
tor polynomial using a computer program to compute numerically the roots such as M ATLABs
roots routine. Alternatively, we can just hunt for the signs of the real part of the roots, a much
simpler operation, and this is the approach taken by the Routh and Jury tests described next.

R OUTH STABILITY CRITERION

The easiest way to establish the absolute stability of a transfer function is simply to extract the
roots of the denominator perhaps using M ATLAB S root command. In fact this is an overkill
since to establish absolute stability we need only to know the presence of any roots with positive
real parts, not the actual value.

Aside from efficiency, there are two cases where this strategy falls down. One is where we do
not have access to M ATLAB or the polynomial is particularly ill-conditioned such that simple root
extraction techniques fail for numerical reasons, and the other case is where we may have a free
parameter such as a controller gain in the transfer function. This has prompted the development
of simpler exact algebraic methods such as the Routh array or Lyapunovs method to assess sta-
bility.

The Routh criterion states that all the roots of the polynomial characteristic equation,

an sn + an1 sn1 + an2 sn2 + an3 sn3 + + a1 s + a0 = 0 (2.101)

have negative real parts if and only if the first column of the Routh table (as described below)
have the same sign. Otherwise the number of sign changes is equal to the number of right-hand
plane roots.

The first two rows of the Routh table is simply formed by listing the coefficients of the character-
istic polynomial, Eqn. 2.101, in the manner following
sn an an2 an4
sn1 an1 an3 an5
sn2 b1 b2 b3
sn3 c1 c2 c3
.. .. ..
. . .

and where the new entries b and c in the subsequent rows are defined as

an1 an2 an an3 an1 an4 an an5


b1 = , b2 = , etc.
an1 an1
b1 an3 an1 b2 b1 an5 an1 b3
c1 = , c2 = , etc.
b1 b1

The Routh table is continued until only zeros remain.


74 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

The Routh array is only applicable for continuous time systems, but for discrete time systems,
the Jury test can be used in a manner similar to the Routh table for continuous systems. The
construction of the Jury table is slightly more complicated than the Routh table and is described
in [61, 117118].

The Routh array is most useful when investigating the stability as a function of a variable such
as controller gain. Rivera-Santos has written a M ATLAB routine to generate the Routh Array with
the possibility of including symbolic variables.7

Suppose we wish to determine the range of stability for a closed loop transfer function with
characteristic equation
s4 + 3s3 + 3s2 + 2s + K = 0
as a function of the unknown gain K.8

Listing 2.7: Example of the Routh array using the symbolic toolbox
>> syms K % Define symbolic gain K
>> ra = routh([1 3 3 2 K],K) % Build Routh array for s4 + 3s3 + 3s2 + 2s + K
3 ra =
[ 1, 3, K]
[ 3, 2, 0]
[ 7/3, K, 0]
[ -9/7*K+2, 0, 0]
8 [ K, 0, 0]

Since all the elements in the first column of the table must be positive, we know that for stability
(9/7)K + 2 > 0 and that K > 0, or
0 < K < 14/7

S TABILITY AND TIME DELAYS

Time delay or dead time does not affect the stability of the open loop response since it does not
change the shape of the output curve, but only shifts it to the right. This is due to the non-
polynomial term es in the numerator of the transfer function. Hence dead time can be ignored
for stability considerations for the open loop. However in the closed loop the dead time term ap-
pears in both the numerator and the denominator and now does affect the stability characteristics
and can now no longer be ignored. Since dead time is a nonpolynomial term, we cannot simply
find the roots of the denominator, since now there are an infinite number of them, but instead we
must approximate the exponential term with a truncated polynomial such as a Pade approxima-
tion and then apply either a Routh array in the continuous time case, or a Jury test in the discrete
time case.

Note however that the Nyquist stability criteria yields exact results, even for those systems with
time delays. The drawback is that the computation is tedious without a computer.

Problem 2.8 Given


15(4s + 1)es
G(s) =
(3s + 1)(7s + 1)
Find the value of deadtime such that the closed-loop is just stable using both a Pade approxi-
mation and a Bode or Nyquist diagram.
7 The routine, routh.m, is available from the MathWorks users group collection at
www.mathworks.com/matlabcentral/fileexchange/.
8 Problem adapted from [154, p288]
2.9. STABILITY 75

A first order Pade approximation is


1 2 s
es =
1 + 2 s

2.9.2 S TABILITY OF THE CLOSED LOOP

Prior to the days of cheap computers that can readily manipulate polynomial expressions and
extract the roots of polynomials, one could infer the stability of the closed loop from a Bode or
Nyquist diagram of the open loop.

The openloop system


100(s + 2)
G(s) = e0.6s (2.102)
(s + 5)(s + 4)(s + 3)
is clearly stable, but the closed loop system, G/(1 + G) may, or may not be stable. If we substitute
s = i, and compute the complex quantity G(i) as a function of , we can apply either the Bode
stability criteria, or the equivalent Nyquist stability criteria to establish the stability of the closed
loop without deriving an expression for the closed loop, and subsequently solving for the closed
loop poles.

The Bode diagram consists of two plots; the magnitude of G(i) versus , and the phase of G(i)
versus . Alternatively we could plot the real and imaginary parts of G(i) as a function of fre-
quency, but this results in a three dimensional plot. Such a plot for the system given in Eqn. 2.102
is given in Fig. 2.26(a).

(a) The Nyquist diagram with frequency information (b) A 2D Nyquist diagram which shows the closed
presented in the third dimension. A flagpole is loop is unstable since it encircles the (1, 0i) point.
drawn at (1, 0i). Since the curve encircles the flag- In this case both positive and negative frequencies are
pole, the closed loop is unstable. plotted.

Figure 2.26: Nyquist diagram of Eqn. 2.102 in (a) three dimensions and (b) as typically presented
in two dimensions.

However three dimensional plots are difficult to manipulate, so we normally ignore the frequency
component, and just view the shadow of the plot on the real/imaginary plane. The two dimen-
sional Nyquist curve corresponding to Fig. 2.26(a) is given in Fig. 2.26(b). It is evident from either
plot that the curve does encircle the critical (0, 1i) point, so the closed loop system will be un-
stable.

Establishing the closed loop stability using the Nyquist criteria is slightly more general than when
using the Bode criteria. This is because for systems that exhibit a non-monotonically decreasing
curves may cross the critical lines more than once leading to misleading results. An interesting
example from [80] is use to illustrate this potential problem.
76 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

2.9.3 S TABILITY OF DISCRETE TIME SYSTEMS

The definition of stability for a discrete time system is similar to that of the continuous time system
except that the definition is only concerned with the values of the output and input at the sample
times. Thus it is possible for a system to be stable at the sample points, but be unstable between
the sample points. This is called a hidden oscillation, although rarely occurs in practice. Also note
that the stability of a discrete time system is dependent on the sample time, T .

Recall that at sample time T , by definition z = exp(sT ), the poles of the continuous transfer
function pi map to exp(pi T ) in discrete space. If pi is negative, then exp(pi T ) will be less than
1.0. If pi is positive, then the corresponding discrete pole zi will be larger than 1.0. Strictly if pi is
complex, then it is the magnitude of zi that must be less than 1.0 for the system to be stable.

In summary, the discrete time transfer function G(z) is stable if G(z) has no discrete poles on, or
outside, the unit circle. This circle of radius 1 is called the unit circle.

Now the evaluation of stability procedure is the same as that for the continuous case. One simply
factorises the characteristic equation of the transfer function and inspects the roots. If any of the
roots zi lie outside the unit circle, the system will be unstable. If any of the poles lie exactly on
the unit circle, there can be some ambiguity about the stability, see [120, p120]. For example the
transfer function which has poles on the unit circle,

1
G(z) =
z2 + 1

given a bounded input such as a step, u(k) = {1, 1, 1, 1 . . .} produces

y(k) = {0, 0, 1, 1, 0, 0, 1, 1, 0, 0, . . .}

which is bounded and stable. However if we try a different bounded input such as a cycle u(k) =
{1, 0, 1, 0, 1, 0, 1, . . .} we observe

y(k) = {0, 0, 1, 0, 2, 0, 3, 0, 4, 0, 5, 0, 6, 0, . . .}

which is unbounded.

Just as in the continuous case, we can establish the absolute stability of a discrete transfer function
without explicitly solving for the roots of the denominator D(z) = 0, which, in the days before
computers was a main consideration. The main methods for discrete systems are the Jury test,
the bilinear transform coupled with the Routh array, and Lyapunovs method. All these methods
are analytical and hence exact. The Jury test is the discrete equivalent to the Routh array and
is covered in Ogata [152, p242]. Lyapunovs method has the special distinction that it is also
applicable to nonlinear ODEs and is covered in 2.9.4.

Suppose we wish to establish the stability of the discretised version of

1
G(s) =
6s2 + 5s + 1

at a sampling period of T = 2 and a first-order hold.


2.9. STABILITY 77

The discrete transfer function is


0.0746 + 0.2005z 1 + 0.0324z 2
G(z) = >> G = tf(1,[6 5 1]);
1 0.8813z 1 + 0.1889z 2
2 >> Gd = c2d(G,2,'foh')
with discrete poles given by the solution of >> pole(Gd)
ans =
1 0.8813z 1 + 0.1889z 2 = 0 0.5134
0.3679
or  7 >> pzmap(Gd); axis('equal');
0.5134
z=
0.3679

Since both discrete poles lie inside the unit circle, the discretised transfer function with a first-
order hold is still stable.

If you are using the state space description, then simply check the eigenvalues of the homoge-
neous part of the equation (the A or matrix).

2.9.4 S TABILITY OF NONLINEAR DIFFERENTIAL EQUATIONS

Despite the fact that nonlinear differential equations are in general very difficult, if not impossible
to solve, one can attempt to establish the stability without needing to find the solution. Studies
of this sort fall into the realm of nonlinear systems analysis which demands a high degree of
mathematical insight and competence. [191] is a good introductory text for this subject that avoids
much of the sophisticated mathematics.

Two methods due to the Russian mathematician Lyapunov9 address this nonlinear system sta-
bility problem. The indirect or linearisation method is based on the fact that the stability near
the equilibrium point will be closely approximated to the linearised approximation at the point.
However it is the second method, or direct method that being exact is a far more interesting and
powerful analysis. Since the Lyapunov stability method is applicable for the general nonlinear
differential equation, it is of course applicable for linear systems as well.

Figure 2.27: Alexander Mikhailovich Liapunov (18571918) was a Russian


mathematician and mechanical engineer. He had the very rare merit of produc-
ing a doctoral dissertation of lasting value. This classic work was originally pub-
lished in 1892 in Russian but is now available in an English translation Stability of
Motion, Academic Press, NY, 1966. Liapunov died by violence in Odessa, which
cannot be considered a surprising fate for a middle class intellectual in the chaotic
aftermath of the Russian revolution.
Excerpt from Differential Equations and historical applications, Simmons, p465.

The Lyapunov stability theorem says that if such a differential system exists

x = f (x, t) (2.103)

where f (0, t) = 0 for all t, and if there exists a scalar function V (x) having continuous first partial
derivatives satisfying V (x) is positive definite and the time derivative, V (x), is negative definite,
then the equilibrium position at the origin is uniformly asymptotically stable. If V (x, t) as
x , then it is stable in the large. V (x) is called a Lyapunov function of the system.
9 Some authors, e.g. Ogata, tend to it spell as Liapunov. Note however that M ATLAB uses the lyap spelling.
78 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

If the Lyapunov function is thought of as the total energy of the system, then we know that the
total energy must both always be positive, hence the restriction that V (x) is positive definite, and
also for the system to be asymptotically stable, this energy must slowly with time die away, hence
the requirement that V (x) < 0. The hard part about this method is finding a suitable Lyapunov
function in the first place. Testing the conditions is easy, but note that if your particular candidate
V (x) function failed the requirements for stability, this either means that your system is actually
unstable, or that you just have not found the right Lyapunov function yet. The problem is that
you dont know which. In some cases, particularly vibrating mechanical ones, good candidate
Lyapunov functions can be found by using the energy analogy, however in other cases this may
not be productive.

Algorithm 2.2 Lyapunov stability analysis of nonlinear continuous and discrete systems.

The second method of Lyapunov is applicable for both continuous and discrete systems, [191,
p65], [152, p557].

Discrete difference equations,

Given a nonlinear difference equation,


Continuous differential equations xk+1 = f (xk ), f (0) = 0
Given a nonlinear differential equation, and a scalar function V (x), continuous in x, then
if
x = f (x, t)
1. V (x) > 0 for x 6= 0 (or V (x) is positive def-
and a scalar function V (x), then if inite), and
1. V (x) is positive definite, and 2. V (x), the difference of V (x), is negative
2. V (x), the derivative of V (x), is negative definite, where
definite, and V (xk ) = V (xk+1 ) V (xk )
3. V (x) as kxk = V (f (xk )) V (xk )
then the system x = f (x, t) is globally asymptoti-
cally stable. and
3. V (x) as kxk
then the system xk+1 = f (xk ) is globally asymp-
totically stable.

Example of assessing the stability of a discrete nonlinear system using Lyapunovs second method.

Consider the dynamic equation from Ogata [154, Ex 9-18, p729]



x1 = x2 x1 x21 + x22

x2 = x2 x2 x21 + x22

and we want to establish if it is stable or not. The system is nonlinear and the origin is the only
equilibrium state. We can propose a trial Lyapunov function V (x) = x21 + x22 which is positive
definite. (Note that that was the hard bit!) Now differentiating gives
dx1 dx2 
V (x) = 2x1 + 2x2 = 2 x21 + x22
dt dt
which is negative definite. Since V (x) as x , then the system is stable in the large.
Relevant problems in [154, p771] include B-9-15, B-9-16 and B-9-17.
2.9. STABILITY 79

The problem with assessing the stability of nonlinear systems is that the stability can change
depending on the values of the states and/or inputs. This is not the case for linear systems,
where the stability is a function of the system, not the operating point. An important and relevant
chemical engineering example of a nonlinear system that can either be stable or unstable is a
continuously stirred tank reactor. The reactor can either be operated in the stable or unstable
regime depending on the heat transfer characteristics. The stability of such reactors is important
for exothermic reactions, since the thermal run-away is potentially very hazardous.

S TABILITY OF LINEAR TIME INVARIANT SYSTEMS USING LYAPUNOV

Sections 2.9.1 and 2.9.3 established stability criteria for linear systems. In addition to these meth-
ods, the second method of Lyapunov can be used since the linear system is simply a special case
of the general nonlinear system. The advantages of using the method of Lyapunov are:

1. The Lyapunov method is analytical even for nonlinear systems and so the criteria is, in prin-
ciple, exact.

2. It is computationally simple and robust and does not require the solution to the differential
equation.

3. We do not need to extract eigenvalues of an n n matrix which is an expensive numeri-


cal computation, (although we may need to check the definiteness of a matrix which does
require at least a Cholesky decomposition).

Of course similar advantages apply to the Routh array procedure, but unlike the Routh array, this
approach can be extended to nonlinear problems.

To derive the necessary condition for stability of x = Ax using the method of Lyapunov, we
follow the procedure outlined in Algorithm 2.2 where we choose a possible Lyapunov function
in quadratic form,
V (x) = xT Px
The Lyapunov function will be positive definite if the matrix P is positive definite. The time
derivative is given by

V (x) = xT Px + xT Px
= (Ax)T Px + xT P(Ax)
 
= xT AT P + PA x
| {z }
Q

where we have defined for convenience the matrix Q = (AT P + PA).

So if Q is negative definite, (or alternatively Q is positive definite, Q 0), then the system is
stable at the origin and hence asymptotically stable in the large. Note that this solution procedure
is analytical.

Unfortunately there is no cheap and easy computational way to establish if an arbitrary matrix is
positive definite or not, but using M ATLAB we would usually attempt a Cholesky factorisation.
Apart from the difficulty in deciding if a matrix is positive definite or not, we also have the prob-
lem that a failure of Q being positive definite does not necessarily imply that the original system is
unstable. All it means is that if the original system does turn out to be stable, then the postulated
P was actually not a Lyapunov function.
80 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

A solution procedure that avoids us testing many different Lyapunov candidates is to proceed in
reverse; namely choose an arbitrary symmetric positive definite Q (say the identity matrix), and
then solve
AT P + PA = Q (2.104)
for the matrix P. If P is positive definite, then the system x = Ax is stable.

Eqn. 2.104 is known as the matrix Lyapunov equation or the more general form, AP + PB = Q,
is known as Sylvesters equation.

One obvious way to solve for the symmetric matrix P in Eqn. 2.104 is by equating the coefficients
which results in a n(n + 1)/2 system of linear algebraic equations. It is convenient in M ATLAB
to use the Kronecker tensor product to set up the equations and then use M ATLABs backslash
operator to solve them. This has the advantage that it is expressed succinctly in M ATLAB, but it
has poor storage and algorithmic characteristics and is recommended only for small dimensioned
problems. Section 2.9.5 explains Kronecker tensor products and this analytical solution strategy
in further detail.

Listing 2.8: Solve the continuous matrix Lyapunov equation using Kronecker products
n= size(Q,1); % Solve AT P + PA = Q, Q = QT
I= eye(n); % Identity In , the same size as A
3 P=reshape(-(kron(I,A')+kron(A',I))\Q(:),n,n);% vec(IAT +AT I)vec(P) = vec(Q)
P= (P+P')/2; % force symmetric (hopefully unnecessary)

The final line, while strictly unnecessary, simply forces the computed result to be symmetric,
which of course it should be in the first case.

Less restrictive conditions on the form of Q exist that could make the solution procedure easier to
do manually and are discussed in [154, p766], and [152, p554]. Note however, using a computa-
tional tool such as M ATLAB, we would not normally bother with these modifications.

The C ONTROL T OOLBOX for M ATLAB includes the function lyap that solves the matrix Lyapunov
equation which is more numerically robust and efficient that equating the coefficients or using
the Kronecker tensor product method given on page 80. However note that the definition of the
Lyapunov matrix equation used by M ATLAB,

AP + PAT = Q (2.105)

is not exactly the same as that defined by Ogata and used previously in Eqn. 2.104. (See the
M ATLAB help file for clarification of this and compare with example 920 in [154, p734].)

Suppose we wish to establish the stability of the continuous system

x = Ax
 
0 1
= x
1 0.5

using lyap, as opposed to say extracting eigenvalues.

Now we wish to solve A P + PA = Q for P where Q is some conveniently chosen positive


definite matrix such as say Q = I. Since M ATLABs lyap function solves the equivalent matrix
equation given by Eqn. 2.105, we must pass A rather than A as the first argument to lyap.

Listing 2.9: Solve the matrix Lyapunov equation using the lyap routine
1 >> A = [0,1;-1,-0.4];
2.9. STABILITY 81

>> P = lyap(A',eye(size(A))) % Use AT instead of A


P =
2.7000 0.5000
0.5000 2.5000
6 >> A'*P + P*A + eye(size(A)) % Does this equal zero ?
ans =
1.0e-015 *
0 -0.4441
-0.4441 0.2220
11

>> Q = eye(size(A)); % alternative method


>> n = max(size(Q)); % solve AT X + XA = C where C 0
>> I = eye(n);
>> P=reshape(-(kron(I,A')+ kron(A',I))\Q(:),n,n)
16 P =
2.7000 0.5000
0.5000 2.5000

Both methods return the same matrix P,


 
2.7 0.5
P=
0.5 2.5

which has minor determinants of 2.7 and 6.5. Since both are positive, following Sylvesters crite-
ria, P is positive definite and so the system is stable.

While we can use Sylvesters criteria to establish if P is positive definite or not, and hence the
systems stability, it is not efficient for large systems. To establish if a symmetric matrix is pos-
itive definite, the easiest way is to look at the eigenvalues. If they are all positive, the matrix is
positive definite. Unfortunately we were originally trying to avoid to solve for the eigenvalues,
so this defeats the original purpose somewhat. Another efficient strategy to check for positive
definiteness with M ATLAB is to attempt a Cholesky decomposition using the [R,p]=chol(A)
command. The decomposition will be successful if A is positive definite, and will terminate early
with a suitable error message if not.

For example suppose we wanted to establish the stability of a transfer function


s1
G=
(s + 1)(s + 2)(s 3)
which is clearly unstable due to the pole at s = +3. In this case we convert to state space, solve
for P and then attempt a Cholesky decomposition to establish if P 0.

Listing 2.10: Using Lyapunov to establish stability of a linear system


>> G = ss(zpk(1,[-1 -2 3],1)); % System is clearly unstable since one pole is s = +3
2 >> P = lyap(G.a, eye(size(G.a))) % Extract the A matrix & solve Lyapunov
P =
0.6667 0.1179 -0.1179
0.1179 0.3333 0.1667
-0.1179 0.1667 -0.1667
7

>> chol(P)
Error using chol
Matrix must be positive definite.

>> [,p] = chol(P) % If p 6= 0, then P is not positive definite.


12 p =
3
82 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

As a final check, we could evaluate the eigenvalues of A in M ATLAB by typing eig(A). Of course
in the above case we will get 1, 2 and +3.

D ISCRETE TIME SYSTEMS

For linear discrete time systems, the establishment of stability is similar to the continuous case
given previously. Given the discrete time system xk+1 = xk , we will again choose a positive
definite quadratic function for the trial Lyapunov function, V (x) = xT Px where the matrix P is
positive definite, and compute the forward difference
V (x) = V (xk+1 ) V (xk )
= xTk+1 Pxk+1 xTk Pxk
= (xk )T Pxk xTk Pxk
 
= xTk T P P xk
| {z }
Q

Now if the matrix Q defined as  


Q = T P P (2.106)
is positive definite, then the system is stable.

The reverse solution procedure, i.e. to solve for the matrix P given Q and is analogous to the
continuous time case given in Listing 2.8 using the Kronecker tensor product.

Listing 2.11: Solve the discrete matrix Lyapunov equation using Kronecker products
1 n = max(size(Q)); % Solve AT PA P = Q, with Q 0
I = eye(n); % identity
P = reshape((kron(I,I)-kron(Phi',Phi'))\Q(:),n,n)
P = (P+P')/2; % force symmetric (hopefully unnecessary)

Once again, the more efficient dlyap routine from the C ONTROL T OOLBOX can also be used. In
fact dlyap simply calls lyap after some minor pre-processing.

We can demonstrate this establishing the stability of


 
3 0.5
xk+1 = xk
0 0.8
In this case since is upper triangular, we can read the eigenvalues by inspection, (they are the
diagonal elements, 3 and 0.8). Since one of the eigenvalues is outside the unit circle we know im-
mediately that the process is unstable. Notwithstanding, by following the method of Lyapunov,

1 >> Phi = [3, -0.5; 0, 0.8]


>> Q = eye(size(Phi)); % Ensure Q 0
>> P = dlyap(Phi',Q) % Note is transposed.
P =
-0.1250 -0.1339
6 -0.1339 2.9886
>> [R,rc] = chol(P) % Is P +ve definite?
R =
[]
rc =
11 1 % No it isnt, unstable!
2.9. STABILITY 83

we verify that the system is indeed unstable.

We can both check the definiteness of P using Sylvesters criteria [154, pp924-926], or by using
the chol function as in the above example, or even by finding the eigenvalues of P. We then can
compare the stability criteria thus established with that obtained by solving for the eigenvalues
of .

Algorithm 2.3 Stability of linear systems using Lyapunov


We wish to establish the absolute stability of the dynamic linear continuous system, x = Ax or
the discrete counterpart, xk+1 = xk

1. Choose a convenient positive definite n n matrix Q, say the identity, Inn , then

2. Solve either
AT P + PA = Q
in the case of a continuous system, or

T P P = Q

in the case of a discrete system for P by either equating the coefficients or using lyap or
dlyap.

3. The Lyapunov function is V (x) = xT Px, so check the definiteness of P. The system is stable
if and only if P is positive definite. Check using Sylvesters criteria or attempt a Cholesky
factorisation.

2.9.5 E XPRESSING MATRIX EQUATIONS SUCCINCTLY USING K RONECKER PROD -


UCTS

The strategy employed in Listings 2.8 and 2.11 to solve the Lyapunov equation used Kronecker
products and vectorisation or stacking to express the matrix equation succinctly. This made it easy
to solve since the resulting expression was now reformulated into a system of linear equations
which can be solved using standard linear algebra techniques. Further details on the uses and
properties of Kronecker products (or tensor products) are given in [88, p256], [116, Chpt 13] and
[132]. The review in [36] concentrates specifically on Kronecker products used in control applica-
tions.

The Kronecker product, given the symbol , of two arbitrarily sized matrices, A mn and
B pq , results in a new (large) matrix

a11 B a1n B
AB=
.. .. .. mpnq
. . . (2.107)
am1 B amn B

of size (mp nq). In M ATLAB we would write kron(A,B). Note in general A B 6= B A.

The vectorisation of a matrix A is an operation that converts a rectangular matrix into a single
column by stacking the columns of A on top of each other. In other words, if A is an (n m)
matrix, then vec(A) is the resulting (nm1) column vector. In M ATLAB we convert block matrices
or row vectors to columns simply using the colon operator, or A(:).

We can combine this vectorisation operation with Kronecker products to express matrix multi-
plication as a linear transformation. For example, for the two matrices A and B of compatible
84 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

dimensions, then

vec (AB) = (I A) vec(B)


  (2.108)
= BT I vec(A)

and for the three matrices A, B and C of compatible dimensions, then


 
vec (ABC) = CT A vec(B)
= (I AB) vec(C) (2.109)
 
= CT BT I vec(A)

Table II in [36] summarises these and many other properties of the algebra of Kronecker products
and sums.

This gives us an alternative way to express matrix expressions such as the Sylvester equation
AX XA = Q where we wish to solve for the matrix X given matrix A. In this case, using
Eqn. 2.108, we can write
 
vec(AX XA) = I A AT I vec(X) = vec(Q)

which is in the form of a system of linear equations Gx = q where the vectors 


x and q are simply

the stacked columns of the matrices X, and Q, and the matrix G is given by I A AT I .
We first solve for the unknown vector x using say x = G1 q or some numerically sound equiv-
alent, and then we reassemble the matrix X by un-stacking the columns from x. Of course this
strategy is memory intensive because the size of the matrix G is (n2 n2 ). However [36] describes
some modifications to this approach to reduce the dimensionality of the problem.

U SING fsolve TO SOLVE MATRIX EQUATIONS

The general nonlinear algebraic equation solver fsolve within the O PTIMISATION T OOLBOX has
the nice feature that it can solve matrix equations such as the continuous time Lyapunov equation
directly. Here we wish to find the square matrix X such that

AX + XAT + Q = 0

given an arbitrary square matrix A and a positive definite matrix Q. For an initial estimate for X
we should start with a positive definite matrix such as I. We can compare this solution with the
one generated by the dedicated lyap routine.

>> n = 4;
>> A = rand(n); % Create a random matrix A of dimensions (n n)
>> Q1 = randn(n); Q = Q1'*Q1; % Create a random positive definite matrix Q
4

>> LyEqn = @(X) A*X+X*A'+Q; % Matrix equation to be solved: AX + XAT + Q = 0


>> X = fsolve(LyEqn, eye(n)); % Solve the Lyapunov using fsolve
X1 =
-0.8574 -0.6776 1.0713
9 -0.6776 2.7742 -2.6581
1.0713 -2.6581 0.7611

>> Xx = lyap(A,Q); % Solve the Lyapunov using the control toolbox


Xx =
2.10. SUMMARY 85

14 -0.8574 -0.6776 1.0713


-0.6776 2.7742 -2.6581
1.0713 -2.6581 0.7611

2.9.6 S UMMARY OF STABILITY ANALYSIS

The stability of a linear continuous time transfer function is determined by the sign of the real part
of the poles. For the transfer function to be stable, all the poles must lie strictly in the left-hand-
plane. In the discrete case, the stability is determined by the magnitude of the possibly complex
poles. To be stable, the discrete poles must lie within the unit circle. (See Fig. 2.28.) The key
difference between the stability of discrete and continuous transfer functions is that the sample
time plays an important role. Generally as one increases the sample time, the discrete system tend
to instability.


stability boundary

1 Unstable
Stable Unstable
entire LHP Stable
0 1 0 1

unit circle

Continuous s-plane Discrete z-plane

Figure 2.28: Regions of stability for the poles of continuous (left) and discrete (right) systems

To establish the stability of the transfer functions, one need not solve for the roots of the de-
nominator polynomial since exact algebraic methods such as the Routh Array, Jury blocks and
Lyapunov methods exist. However with the current computer aided tools such as M ATLAB, the
task of reliably extracting the roots of high order polynomials is not considered the hurdle it once
was. The Lyapunov method is also applicable for nonlinear systems. See Ogatas comments, [152,
p250].

2.10 S UMMARY

This chapter briefly developed the tools needed to analyse discrete dynamic systems. While most
physical plants are continuous systems, most control is performed on a digital computer, and
therefore the discrete description is far more natural for computer implementation or simulation.
Converting a continuous differential equation to a discrete differential equation can be done either
by using a backward difference approximation such as Eulers scheme, or by using z transforms.
The z transform is the discrete equivalent to the Laplace transform in the continuous domain.

A vector/matrix approach to systems modelling and control has the following characteristics:
86 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

1. We can convert any linear time invariant differential equation into the state space form
x = Ax + Bu.
2. Once we have selected a sampling time, T , we can convert from the continuous time domain
to the discrete form equivalent; xk+1 = xk + uk .
3. The stability of both the discrete and continuous time systems is determined by the eigen-
values of the A or matrix.
4. We can transform the mathematical description of the process to dynamically similar de-
scriptions by adjusting our base coordinates, which may make certain computations easier.

Stability is one of the most important concepts for the control engineer. Continuous linear systems
are stable if all the poles lie in the left hand side of the complex plane, ie they have negative real
parts. For discrete linear systems, the poles must lie inside the unit circle. No easy check can be
made for nonlinear systems, although the method due to Lyapunov can possibly be used, or one
can approximate the nonlinear as a linear system and check the stability of the latter.

When we deal with discrete systems, we must sample the continuous function. Sampling can
introduce problems in that we may miss interesting information if we dont sample fast enough.
The sampling theorem tells us how fast we must sample to reconstruct particular frequencies.
Over sampling is expensive, and could possibly introduce unwanted artifacts such as RHP zeros
etc.
C HAPTER 3

M ODELLING DYNAMIC SYSTEMS


WITH DIFFERENTIAL EQUATIONS

The ability to solve so many problems with a small arsenal of mathematical methods gave rise to
a very optimistic world view which in the 18th Century came to be called the Age of
Reason. The essential point is that the world was felt to be predictable.
Hans Mark

3.1 D YNAMIC SYSTEM MODELS

Dynamic system models are groups of equations that involve time derivative terms that attempt
to reproduce behaviour that we observe around us. I admire the sense of optimism in the quote
given above by Hans Mark: if we know the governing equations, and the initial conditions, then
the future is assured. We now know that the 18th century thinkers were misguided in this sense,
but nevertheless engineers are a conservative lot, and still today with similar tools, they aim to
predict the known universe.

It is important to realise that with proven process models, control system design becomes system-
atic, hence the importance of modelling.

Physically dynamic systems are those that change with time. Examples include vibrating struc-
tures (bridges and buildings in earthquakes), bodies in motion (satellites in orbit, fingers on a
typewriter), changing compositions (chemical reactions, nuclear decay) and even human charac-
teristics (attitudes to religion or communism throughout a generation, or attitudes to the neigh-
bouring country throughout a football match) and of course, there are many other examples. The
aim of many modellers is to predict the future. For many of these types of questions, it is infea-
sible to experiment to destruction, but it is feasible to construct a model and test this. Today, it is
easiest to construct a simulation model and test this on a computer. The types of models referred
to in this context are mathematically based simulation models.

Black-box or heuristic models, where we just fit any old curve to the experimental data, and White-
box or fundamental models, where the curves are fitted to well established physical laws represent
two possible extremes in modelling philosophy. In practice, most engineering models lie some-
where in the grey zone, known appropriately as grey-box models where we combine our partial
prior knowledge with black-box components to fit any residuals. Computer tools for grey-box
modelling are developed in [32].
87
88 Modelling

3.1.1 S TEADY STATE AND DYNAMIC MODELS

Steady state models only involve algebraic equations. As such they are more useful for design
rather than control tasks. However they can be used for control to give broad guidelines about
the operating characteristics of the process. One example is Bristols relative gain array ( 3.2.6)
which uses steady state data to analyse for multivariable interaction. But it is the dynamic model
that is of most importance to the control engineer. Dynamic models involve differential equations.
Solving systems involving differential equations is much harder than for algebraic equations, but
is essential for the study and application of control. A good reference for techniques of modelling
dynamic chemical engineering applications is [200].

3.2 A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

An economist is an expert who will know tomorrow why the things he predicted yesterday didnt happen
today.
Laurence J. Peter

This section reviews the basic steps for developing dynamic process models and gives a few ex-
amples of common dynamic models. Modelling is particularly important in the capital-intensive
world of chemical manufacture. Perhaps there is no better example of the benefits of modelling
than in distillation. One industry magazine1 reported that distillation accounted for around 3% of
the total world energy consumption. Clearly this provides a large economic incentive to improve
operation.

When modelling chemical engineering unit operations, we can consider (in increasing complex-
ity):

1. Lumped parameters systems such as well-mixed reactors or evaporators

2. Staged processes such as distillation columns, multiple effect evaporators, floatation.

3. Distributed parameter systems such as packed columns, poorly mixed reactors, dispersion
in lakes & ponds.

An evaporator is an example of a lumped parameter system and is described in 3.2.3 while a


rigorous distillation column model described on page 100 is an example of a staged process.

At present we will now investigate only lumped parameter systems and staged processes which
involve only ordinary differential equations, as opposed to partial differential equations, albeit
a large number of them. Problems which involve algebraic equations coupled to the ordinary
differential equations are briefly considered in section 3.4.2.

A collection of industrial process models is available in the nonlinear model library from www.hedengren.net/resear

There are many text books devoted to general modelling for engineers, notably [35, 115] for gen-
eral issues, [45, 199] for solution techniques and [133, 200] for chemical engineering applications.
I recommend the following stages in the model building process:

1. Draw a diagram of the system with a boundary and labelling the input/output flows that
cross the boundary.
1 The Chemical Engineer, 21st October, 1999, p16
3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 89

2. Decide on what are state variables, what are parameters, manipulated variables, uncon-
trolled disturbances etc.
3. Write down any governing equations that may be relevant such as conservation of mass and
energy.
4. Massage these equations in a standard form suitable for simulation.

The following sections will describe models for a flow system into and out of a tank with external
heating, a double integrator such as an inverted pendulum or satellite control, a forced circulation
evaporator, and a binary distillation column.

3.2.1 S IMPLE MODELS

The best material model of a cat is another, or preferably the same, cat.

Arturo Rosenblueth, Philosophy of Science, 1945

P ENDULUMS

Pendulums provide an easy and fruitful target to model. They can easily be built, are visually
impressive, and the governing differential equations derived from the physics are simple but
nonlinear. Furthermore, if you invert the pendulum such as in a rocket, the system is unstable,
and needs an overlaying control scheme.

Torque, Tc


len
gth
,l


mg

mg

Pendulum (stable) Inverted Pendulum (unstable)

Figure 3.1: A stable and unstable pendulum

Fig. 3.1 shows the two possible orientations. For the classical stable case, the equation of motion
for the pendulum is
d2
ml2 2 + mgl sin = Tc (3.1)
dt
where is the angle of inclination, and Tc is the applied torque. We can try to solve Eqn. 3.1
analytically using dsolve from the symbolic toolbox,
90 Modelling

>> syms m l g positive


>> syms Tc theta
>> dsolve('m*l2*D2theta + m*g*l*sin(theta) = Tc') % Eqn. 3.1
4

Warning: Explicit solution could not be found; implicit solution returned.

but the symbolic engine fails to find an analytical solution.

Eqn. 3.1 is a second order nonlinear differential equation, but by defining a new state variable
vector, x, as
def
x1 = (3.2)
def d
x2 = x1 = (3.3)
dt
then we note that x2 = x1 , so the single second order differential equation can now be written as
two coupled first order equations
x1 = x2 (3.4)
g Tc
x2 = sin x1 + (3.5)
l ml2
Linearising Eqn. 3.5 is trivial since sin for small angles. Consequently, both linear and
nonlinear expressions are compared below
" # " # " # " #
0 x1 0 0 1 0
x = g + Tc , x = g x+ Tc (3.6)
sin(x1 ) 0 2
0 2
| l {z ml } | l {z ml }
nonlinear linear

while the numerical results are compared on page 124.

In the inverted case, the equation for x2 is almost the same,


g Tc
x2 = + sin x1 + (3.7)
l ml2
but the sign change is enough to place the linearised pole in the right-hand plane.

A DOUBLE INTEGRATOR

The double integrator is a simple linear example from classical mechanics. It can describe, amongst
other things, a satellite attitude control system,
d2
J =u (3.8)
dt2
where J is the moment of inertia, is the attitude angle, u is the control torque (produced by
small attitude rockets mounted on the satellites side). We can convert the second order system in
Eqn. 3.8 to two first order systems by again defining a new state vector
   
x1 def
x= = (3.9)
x2
Substituting this definition into Eqn. 3.8, we get J x2 = u and x1 = x2 , or in a matrix-vector form
     
x1 0 1 0
= x 1 u (3.10)
x2 0 0 J
3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 91

A LIQUID LEVEL TANK

Buffer storage vessels such as shown in Fig. 3.2 are commonly used in chemical plants between
various pieces of equipment such as reactors and separators to dampen out fluctuations in pro-
duction flows.
Flow in, Fin
Tank with cross sectional area, A

measured height, h P
true height, h

flow out, Fout

valve

Figure 3.2: Simple buffer tank

Our system boundary is essentially the figure boundary, so we have one input, the input flow
rate, Fin , and one output, the output flow rate, Fout . What interests us is the amount of fluid in
the tank, M , which physically must always be less than the full tank Mmax , and greater than 0.

The governing relation is the conservation of mass,


dM
= Fin Fout (3.11)
dt
If the tank has a constant cross sectional area, then the amount of water in the tank M is pro-
portional to the height of water h, since most fluids are assumed incompressible. In addition the
mass M is proportional to the volume, M = V = Ah. Rewriting Eqn. 3.11 in terms of height
dh Fin Fout
= (3.12)
dt A
since the area and density are constant. If the liquid passes though a valve at the bottom of the
tank, the flowrate out will be a function of height. For many processes, the flow is proportional
to the square root of the pressure drop, (Bernoullis equation)

Fout P = k h
This square root relation introduces a mild nonlinearity into the model.

The tank level is not actually measured in the tank itself, but in a level leg connected to the
tank. The level in this level leg lags behind the true tank level owing to the restriction valve on
the interconnecting line. This makes the system slightly more difficult to control owing to the
added measurement dynamics. We assume, that the level leg can be modelled using a first order
dynamic equation, with a gain of 1, and a time constant of about 2 seconds. I can estimate this by
simply watching the apparatus. Thus
dh h h
= (3.13)
dt 2
92 Modelling

The level signal is a current, I, between 4 and 20 mA. This current is algebraically related (by
some function f () although often approximately linearly) to the level in the level leg. Thus the
variable that is actually measured is:
I = f (h) (3.14)
The current I is called the output or measured variable. In summary, the full model equations
are;

dh Fin k h
= (3.15)
dt A
dh h h
= (3.16)
dt 2
I = f (h) (3.17)

The input variable (which we may have some hope of changing) is the input flow rate, Fin , and
the exit valve position which changes k. The dependent (state) variables are the actual tank level,
h, and the measurable level in the level leg, h. The output variable is the current, I. Constant
parameters are density, and tank cross-sectional area. In Table 3.1 which summarises this nomen-
clature, I have written vectors in a bold face, and scalars in italics. The input variables can be
further separated into variables that can be easily changed or changed on demand such as the
outlet valve position (manipulated variables), and inputs that change outside our control (distur-
bance variables). In this case the inlet flowrate.

Table 3.1: Standard nomenclature used in modelling dynamic systems

type symbol variable


independent t time, t
states x h, h
inputs u Fin , k
outputs y I
parameters , A

A STIRRED TANK HEATER

Suppose an electrical heater is added causing an increase in water temperature. Now in addition
to the mass balance, we have an energy balance. The energy balance in words is the rate of
temperature rise of the contents in the tank is proportional to the amount of heat that the heater is
supplying plus the amount of heat that is arriving in the incoming flow minus the heat lost with
the out flowing stream minus any other heat losses.

We note that for water the heat capacity cp , is almost constant over the limited temperature range
we are considering, and that the enthalpy of water in the tank is defined as

H = M cp T (3.18)

where T is the temperature difference between the actual tank temperature T and some refer-
ence temperature, Tref . Writing an energy balance gives
dT
M cp = Fin cp (Tin Tref ) Fout cp T + Q q (3.19)
dt
where Q is the heater power input (kW) and q is the heat loss (kW). These two equations (Eqn 3.12
and 3.19) are coupled. This means that a change in mass in the tank M, will affect the temperature
in the tank T (but not the other way around). This is called one-way coupling.
3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 93

For most purposes, this model would be quite adequate. But you may decide that the heat loss q
to the surroundings is not constant and the variation significant. A more realistic approximation
of the heat loss would be to assume that it is proportional to the temperature difference between
the vessel and the ambient (room) temperature Troom . Thus q = k(T Troom ). Note that the heat
loss now can be both positive or negative. You may also decide that the heat capacity cp , and
density of water are a function of temperature. Now you replace these constants with functions
as cp (T ) and (T ). These functions are tabulated in the steam tables2 . The more complete model
would then be

dh Fin Fout
= (3.20)
dt (T )A
dT
A(T ) = Fin cp (T ) (Tin T ref ) Fout cp (T )T + Q k (T Troom )
dt

C AVEAT E MPTOR

Finally we should always state under what conditions the model is valid and invalid. For this
example we should note that the vessel temperature should not vary outside the range 0 < T <
100C since the enthalpy equation does not account for the latent heat of freezing or boiling. We
should also note that h, Fin , Fout , M, A, k, cp , are all assumed positive.

As a summary and a check on the completeness of the model, it is wise to list all the variables,
and state whether the variable is a dependent or independent variable, whether it is a dynamic
or constant variable, or whether it is a parameter. The degree of freedom for a well posed prob-
lem should be zero. That is, the number of unknowns should equal the number of equations.
Incomplete models, and/or bad assumptions can give very misleading and bad results.

A great example highlighting the dangers of extrapolation from obviously poor models is the bi-
annual UK government forecast of Britains economic performance. This performance or growth,
is defined as the change in the Gross Domestic Product (GDP) which is the total amount of goods
and services produced in the country that year. Figure 3.3 plots the actual growth over the last
few years (solid line descending), compared with the forecasts given by the Chancellor of the Ex-
chequer, (dotted lines optimistically ascending). I obtained this information from a New Scientist
article provocatively titled Why the Chancellor is always wrong, [51]. Looking at this performance,
it is easy to be cynical about governments using enormous resources to model things that are
essentially so complicated as to be unmodellable, and then produce results that are politically
advantageous.

3.2.2 A CONTINUOUSLY- STIRRED TANK REACTOR

Tank reactors are a common unit operation in chemical processing. A model presented in [84]
considers a simple A B reaction with a cooling jacket to adjust the temperature, and hence the
reaction rate as shown in Fig. 3.4.

The reactor model has two states, the concentration of compound A, given the symbol Ca mea-
sured in mol/m3 and the temperature of the reaction vessel liquid, T measured in K. The manip-
ulated variables are the cooling jacket water temperature, Tc , the temperature of the feed Tf and

2 Steam tables for M ATLAB are available from http://www.kat.lth.se/edu/kat061/MfileLib.html


94 Modelling

119

118 Forecasts

117

Growth GDP change


116

115

114

113

Figure 3.3: The UK growth based on the 112


Actual
GDP comparing the actual growth (solid)
111
versus Treasury forecasts (dashed) from 1989 1990 1991 1992 1993
19891992. Year

Stirrer

Feed, Tf , Caf

Coolant input, Tc

AB
Coolant outflow

Product, T, Ca

Figure 3.4: A continually stirred tank reactor (CSTR) reactor

the concentration of the feed, Caf .


 
dT q E
= (Caf Ca ) k0 exp Ca (3.21)
dt V RT
 
dCa q mH E UA
= (Tf T ) + k0 exp Ca + (Tc T ) (3.22)
dt V Cp RT V Cp

The values for the states and inputs at steady-state are



    Tc 300
def T 324.5 def
xss = = , uss = Tf = 350 (3.23)
Ca 0.8772
ss Caf ss 1

The values of the model parameters such as , Cp etc. are given in Table 3.2.

Note: At a jacket temperature of Tc = 305K, the reactor model has an oscillatory response. The
oscillations are characterized by reaction run-away with a temperature spike. When the concen-
3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 95

Table 3.2: Parameters of the CSTR model


name variable unit
Volumetric Flowrate q = 100 m3 /sec
Volume of CSTR V = 100 m3
Density of A-B Mixture = 1000 kg/m3
Heat capacity of A-B Mixture Cp = .239 J/kg-K
Heat of reaction for AB mH = 5 104 J/mol
E/R = 8750 K
Pre-exponential factor k0 = 7.2 1010 1/sec
Overall Heat Transfer Coefficient Area U A = 5 104 W/K

tration drops to a low value, the reactor cools until the concentration builds and there is another
run-away reaction.

3.2.3 A FORCED CIRCULATION EVAPORATOR

Evaporators such as illustrated in Fig. 3.5 are used to concentrate fruit juices, caustic soda, alu-
mina and many other mixtures of a solvent and a solute. The incentive for modelling such unit
operations is that the model can provide insight towards better operating procedures or future
design alternatives. A relatively simple dynamic model of a forced circulation evaporator is de-
veloped in [149, chapter 2] is reproduced here as a test process for a wide variety of control tech-
niques of interest to chemical engineers. Other similar evaporator models have been reported
in the literature; [66] documents research over a number of years at the University of Alberta
on a pilot scale double effect evaporator, and [42] describe a linearised pilot scale double effect
evaporator used in multivariable optimal selftuning control studies.

The nonlinear evaporator model can be linearised into a state space form with the variables de-
scribed in Table 3.3. The parameters of the linearised state space model are given in Appendix D.1.

Table 3.3: The important variables in the forced circulation evaporator from Fig. 3.5.

type name variable unit range


level L2 m 02
State steam pressure P2 kPa unspecified
product conc x2 % 0100
product flow F2 kg/min 0-10
Manipulated steam pressure P100 kPa 100350
c/w flow F200 kg/min 0300
circulating flow F3 kg/min 30-70
feed flow F1 kg/min 5-15
Disturbance feed composition x1 % 0-10

feed temperature T1 C 20-60

c/w temperature T200 C 15-35

The distinguishing characteristics of this evaporator system from a control point are that the de-
tailed plant model is nonlinear, the model is non-square (the number of manipulated variables,
u, do not equal the number of state variables, x), and that one of the states, level, is an integra-
tor. The dynamic model is both observable and controllable, which means that by measuring the
outputs only, we can at least in theory, control all the important state variables by changing the
inputs.
96 Modelling

cooling water flow, F200


condensor
Vapour Cooling water
T200

condensate

L level, L2
Pressure, P2 P
separator
Steam
Steam pressure, P100

Evaporator

Condensate F3
Feed
F1 , x1 , T1 composition, x2
circulation pump
A
Product
Product flow, F2

Figure 3.5: A forced circulation evaporator from [149, p7] showing state (black), manipulated
(blue) and disturbance variables (green). See also Table 3.3.

Problem 3.1 1. Look at the development of the nonlinear model given in [149, chapter 2].
What extensions to the model would you suggest? Particularly consider the assumption of
constant mass (M = 50kg) in the evaporator circuit and the observation that the level in the
separator changes. Do you think this change will make a significant difference?
2. Describe in detail what tests you would perform experimentally to verify this model if you
had an actual evaporator available.
3. Construct a M ATLAB simulation of the nonlinear evaporator. The relevant equations can be
found in [149, Chpt 2].

3.2.4 B INARY DISTILLATION COLUMN MODELS

Distillation columns, which are used to separate mixtures of different vapour pressures into al-
most pure components, are an important chemical unit operation. The columns are expensive
to manufacture, and the running costs are high owing to the high heating requirement. Hence
there is much incentive to model them with the aim to operate them efficiently. Schematics of two
simple columns are given in Figures 3.6 and 3.9.

Columns such as given in Fig. 3.6 typically have at least 5 control valves, but because the hydro-
dynamics and the pressure loops are much faster than the composition dynamics, we can use the
bottoms exit valve to control the level at the bottom of the column, use the distillate exit valve
3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 97

c/w

Reflux, R
Distillate, D, xD

Feed, F, xF

Distillation column

Steam, S
Reboiler
Bottoms, B, xB

Figure 3.6: Schematic of a distillation column

to control the level in the separator, and the condenser cooling water valve to control the column
pressure. This leaves the two remaining valves (steam reboiler and reflux) to control the top and
bottoms composition.

Distillation column models that are not obtained experimentally are called rigorous models and
are based on fundamental physical and chemical relations. A more recent distillation column
model, developed by Shell and used as a test model is discussed in [141]. However most sim-
ple column models have been developed from experimental step tests. Two such models are
described next.

T HE W OOD -B ERRY COLUMN MODEL

Wood and Berry experimentally modelled a 9 inch diameter, 8-tray binary distillation column in
[207] that separated methanol from water as shown in Fig. 3.6. The transfer function model they
derived by step testing a real column has been extensively studied, although some authors, such
as [189], point out that the practical impact of all these studies has been, in all honesty, minimal.

The Wood-Berry model is written as a matrix of Laplace transforms in deviation variables as



12.8es 18.9e3s 3.8e8s
16.7s + 1 21s + 1 u + 14.9s + 1 d

y= 6.6e7s 3s 3.4s
(3.24)
19.4e 4.9e
10.9s + 1 14.4s + 1 13.2s + 1
where the inputs u1 and u2 are the reflux and reboiler steam flowrates (in lb/min) respectively, the
outputs y1 and y2 are the weight % of ethanol in the distillate and bottoms, and the disturbance
variable, d is the feed flowrate. The time scale is in minutes.

It is evident from the transfer function model structure in Eqn. 3.24 that the plant will be inter-
acting and that all the transfer functions have some time delay associated with them, but the
98 Modelling

off-diagonal terms have the slightly larger time delays. Both these characteristics will make con-
trolling the column difficult.

In S IMULINK, we can represent Eqn. 3.24 as a collection of transfer functions as shown in Fig. 3.7.
In this graphical representation, the interactions are easy to see.

reflux reflux
distillate

steam distillate

steam
bottoms
feed

1 bottoms
Wood Berry Column
feedrate

(a) Overall column mask.

12.8
1 1
16.7s+1
reflux distillate
G11 Transport
Delay
6.6
10.9s+1
G21 Transport
Delay1
-18.9
21s+1
G12 Transport
Delay2

-19.4
2 2
14.1s+1
steam bottoms
G22 Transport
Delay3

3.8
14.9s+1
D1 Transport
3 Delay5
feed

4.9
13.2s+1
D2 Transport
Delay4

(b) Inside the Wood Berry column mask. Compare this with Eqn. 3.24.

Figure 3.7: Wood-Berry column model implemented in S IMULINK

In M ATLAB, we can elegantly describe Eqn. 3.24 as a matrix of transfer functions including the
feed disturbances with a matrix of deadtimes as follows:

>> G = tf({12.8, -18.9, 3.8; 6.6 -19.4, 4.9}, ...


{ [16.7 1],[21 1], [14.9 1]; [10.9 1],[14.4 1],[13.2 1]}, ...
'ioDelayMatrix',[1,3 8; 7,3, 3.4], ...
'InputName',{'Reflux','Steam', 'feed'}, ...
5 'OutputName',{'Distillate','bottoms'})
3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 99

G =
From input "Reflux" to output...
12.8
10 Distillate: exp(-1*s) * ----------
16.7 s + 1

6.6
bottoms: exp(-7*s) * ----------
15 10.9 s + 1

From input "Steam" to output...


-18.9
Distillate: exp(-3*s) * --------
20 21 s + 1

-19.4
bottoms: exp(-3*s) * ----------
14.4 s + 1
25

From input "feed" to output...


3.8
Distillate: exp(-8*s) * ----------
14.9 s + 1
30

4.9
bottoms: exp(-3.4*s) * ----------
13.2 s + 1

35 Continuous-time transfer function.

Once we have formulated the transfer function model G(s), we can perform the usual types of
analysis such as step tests etc as shown in Fig. 3.8. Alternatively we could have use the S IMULINK
model in Fig. 3.7 to get the same result.

Step Response
From: Reflux From: Steam From: feed

10
To: Distillate

-10
Amplitude

-20

0
To: bottoms

-10
Figure 3.8: Step responses of the 6
transfer functions that make up the
-20 Wood-Berry binary distillation col-
0 50 100 0 50 100 0 50 100
Time (seconds) umn model in Eqn. 3.24.
100 Modelling

T HE OLMR MODEL

A 3-input/3 output matrix of transfer functions model of an 19 plate ethanol/water distillation


column model with a variable side stream draw off from [155] is

0.66e2.6s 0.61e3.5s 0.0049es
6.7s + 1 8.64s + 1 9.06s + 1

y1 6.5s 3s u1
0.012e1.2s

y2 = 1.11e 2.36e
u2
3.25s + 1 5s + 1 7.09s + 1
y3 u3
34.68e9.2s 46.2e 9.4s
0.87(11.61s + 1)e2.6s
8.15s + 1 10.9s + 1 (3.89s + 1)(18.8s + 1)
1.2s
0.14e 0.0011(26.32s + 1)e2.66s

6.2s + 1 (3.89s + 1)(14.63s + 1)

10.5s 3.44s

0.53e 0.0032(19.62s + 1)e d1
+ (3.25)
6.9s + 1 (7.29s + 1)(8.94s + 1) d2

11.54e0.6s 0.32e 2.6s
7.01s + 1 7.76s + 1

which provides an alternative model for testing multivariable control schemes. In this model
the three outputs are the overhead ethanol mole fraction, the side stream ethanol mole fraction,
and the temperature on tray 19, the three inputs are the reflux flow rate, the side stream product
rate and the reboiler steam pressure, and the two disturbances are the feed flow rate and feed
temperature. This system is sometimes known as the OLMR after the initials of the authors.

Problem 3.2 1. Assuming no disturbances (d = 0), what are the steady state gains of the
Wood-Berry column model? (Eqn 3.24). Use the final value theorem.

2. Sketch the response for y1 and y2 for;

(a) a change in reflux flow of +0.02


(b) A change in the reflux flow of 0.02 and a change in reboiler steam flow of +0.025
simultaneously.

3. Modify the S IMULINK simulation to incorporate the feed dynamics.

3.2.5 R IGOROUS TRAY- BY- TRAY DISTILLATION COLUMN MODELS

Distillation column models are important to chemical engineers involved in the operation and
maintenance of these expensive and complicated units. While the behaviour of the actual multi-
component tower is very complicated, models that assume ideal binary systems are often good
approximations for many columns. We will deal in mole fractions of the more volatile component,
x for liquid, y for vapour and develop a column model following [134, p69].

A generic simple binary component column model of a distillation column such as shown in
Fig. 3.9 assumes:

1. Equal molar overflow applies (heats of vapourisation of both components are equal, and
mixing and sensible heats are negligible.)

2. Liquid level on every tray remains above weir height.


3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 101

3. Relative volatility and the heat of vapourisation are constant. In fact we assume a constant
relative volatility, . This simplifies the vapour-liquid equilibria (VLE) model to
xn
yn =
1 + ( 1)xn
on tray n with typically 1.22.
4. Vapour phase holdup is negligible and the feed is a saturated liquid at the bubble point.

c/w
condensor

condensate collector

Distillate, D, xD
tray #N
Reflux, R, xD

Feed, F, xF
feed tray, NF

tray #1
reboiler, V

Bottoms, B, xB

Figure 3.9: A binary distillation tower

The N trays are numbered from the bottom to top, (tray # 0 is the reboiler and tray # N + 1 is
the condenser). We will develop separate model equations for the following parts of the column,
namely:

Condenser is a total condenser, where the reflux is a liquid and the reflux drum is perfectly
mixed.
General tray has liquid flows from above, and vapour flows from below. It is assumed to be
perfectly mixed with a variable liquid hold up, but no vapour hold up as it is assumed very
fast.
Feed tray Same as a general tray, but with an extra (liquid) feed term.
Top & bottom trays Same as a general tray, but with one of the liquid (top) or vapour (bottom)
flows missing, but recycle/reflux added.
Reboiler in a perfectly mixed thermo-siphon reboiler with hold up Mb .

The total mass balance in the condenser and reflux drum is


dMd
=V RD
dt
102 Modelling

and the component balance on the top tray


dMd xd
= V yNT (R + D)xD (3.26)
dt
For the general nth tray, that is trays #2 through N 1, excepting the feed tray, the total mass
balance is
dMn
= Ln+1 Ln + Vn1 Vn
dt | {z }
0
= Ln+1 Ln (3.27)

and the component balance

dMn xn
= Ln+1 xn+1 Ln xn + V yn1 V yn (3.28)
dt
For the special case of the feed nF th tray,

dMnF
= LnF +1 LnF + F, mass balance
dt
dMnF xnF
= LnF +1 xnF +1 LnF xnF + F xF + V ynF 1 V ynF , component balance
dt
and for the reboiler and column base
dMB
= L1 V B
dt
dMB xB
= L1 x1 BxB V xB
dt
In summary the number of variables for the distillation column model are:

Tray compositions, xn , yn 2NT


Tray liquid flows, Ln NT
VARIABLES Tray liquid holdups, Mn NT
Reflux comp., flow & hold up, xD , R, D, MD 4
Base comp., flow & hold up, xB , yB , V, B, MB 5
Total # of equations 4NT + 9

and the number of equations are:

Tray component balance, NT


Tray mass balance NT
Equilibrium (tray + base) NT + 1
E QUATIONS
hydraulic NT
Reflux comp. & flow 2
Base comp & flow 2
Total # of equations 4NT + 7

Which leaves two degrees of freedom. From a control point of view we normally fix the boildup
rate, Q, and the reflux flow rate (or ratio) with some sort of controller,

R = f (xD ), V Q = f (xB )

Our dynamic model of a binary distillation column is relatively large with two inputs (R, V ), two
outputs (xB , xD ) and 4N states. Since a typical column has about 20 trays, we will have around
3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 103

n = 44 states. which means 44 ordinary differential equations However the (linearised) Jacobian
for this system while a large 44 44 matrix is sparse. In our case the percentage of non-zero
elements or sparsity is 9%.

x .. x
x .
|{z} u
|{z}
4444 442

The structure of the A and B matrices are, using the spy command, given in Fig. 3.10.

10

15
ODE equations

20

25

30

35

40

45
0 10 20 30 40
nz = 25

Figure 3.10: The incidence of the Jacobian (blue) and B matrix (red) for the ideal binary distillation
column model. Over 90% of the elements in the matrix are zero.

There are many other examples of distillation column models around (e.g. [144, pp 459]) This
model has about 40 trays, and assumes a binary mixture at constant pressure and constant relative
volatility.

S IMULATION OF THE DISTILLATION COLUMN

The simple nonlinear dynamic simulation of the binary distillation column model can be used in
a number of ways including investigating the openloop response, interactions and quantify the
extent of the nonlinearities. It can be used to develop simple linear approximate transfer function
models, or we could pose What if? type questions such as quantifying the response given feed
disturbances.

An openloop simulation of a distillation column gives some idea of the dominant time constants
and the possible nonlinearities. Fig. 3.11 shows an example of one such simulation where we step
change: the reflux from R = 128 to R = 128 + 0.2, and the reboiler from V = 178 to V = 178 + 0.2.

From Fig. 3.11 we can note that the open loop step results are overdamped, and that steady-state
gains are very similar in magnitude. Furthermore the response looks very like a 2 2 matrix of
second order overdamped transfer functions,
    
xD (s) G11 G12 R(s)
=
xB (s) G21 G22 V (s)
104 Modelling

0.995

Distill conc
0.99

0.985

0.98

0.975

0.1

0.08
base conc

0.06

0.04
Figure 3.11: Open loop re-
0.02 Step change in Reflux
sponses of the 20-tray binary
Step change in Vapour
distillation column for a step
0
change in reflux, and a step 0 5 10 15 20 25 30
change in vapour boil-off rate. time (min)

So it is natural to wonder at this point if it is possible to approximate this response with a low-
order model rather than requiring the entire 44 states and associated nonlinear differential equa-
tions.

C ONTROLLED SIMULATION OF A DISTILLATION COLUMN

A closed loop simulation of the 20 tray binary distillation column with a feed disturbance from
xF = 0.5 to xF = 0.54 at t = 10 minutes is given in Fig. 3.12. Note that while we have plotted the
liquid concentrations on all the 20 trays, we are really only interested in the very top (distillate)
and bottom (reboiler) trends. However simply to obtain the top and bottom trends, we are forced
to calculate all the tray concentrations in this rigorous model. We can look more closely in Fig. 3.13
at the distillate and base compositions to see if they really are in control, namely at xb = 0.02, xd =
0.98.

Distillation columns are well known to interact, and these interactions cause difficulties in tuning.
We will simulate in Fig. 3.14 a step change in the distillate setpoint from xD = 0.98 to xD = 0.985
at t = 10 min and a step change in bottoms concentration at 150 minutes. The interactions
are evident in the base composition transients owing to the changing distillate composition and
visa versa. These interactions can be minimised by either tightly tuning one of the loops so con-
sequently leaving the other loose, or by using a steady-state or dynamic decoupler, or even a
multivariable controller.

3.2.6 I NTERACTION AND THE R ELATIVE G AIN A RRAY

The Wood-Berry distillation column model is an example of a multivariable, interacting process.


This interaction is evident from the presence of the non-zero off diagonal terms in Eqn. 3.24. A
quantitative measure of the extent of interaction in a multivariable process is the Relative Gain
3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 105

Binary distillation column model


1

0.9

0.8

0.7
Liquid concentration

0.6

0.5

0.4

0.3

0.2

0.1

135
R

130

125
190 Figure 3.12: A closed loop simulation of a
binary distillation column given a feed dis-
180 turbance. The upper plot shows the con-
V

centration on all the 20 trays, the lower


170 plots shows the manipulated variables (re-
0 20 40 60 80 100
time (min) flux and boil-up) response.

0.99
Distillate

0.985

0.98

0.975
0.04
Bottoms

0.03

0.02 Figure 3.13: Detail of the distillate and base


concentrations from a closed loop simula-
0.01 tion of a binary distillation column given a
0 20 40 60 80 100
time [mins] feed disturbance. (See also Fig. 3.12.)

Array (RGA) due to Bristol, [37]. The RGA is only applicable to square systems (the number of
manipulated variables equal the number of controlled variables), and is a steady-state measure.

The RGA, given the symbol , is an (n n) matrix where each element is the ratio of the open
loop gain divided by the closed loop gain. Ideally, if you are the sort who do not like interactions,
you would like to be diagonally dominant. Since all the rows and all the columns must sum to
1.0, then a system with no interaction will be such that = I.

The open loop steady-state gain is easy to determine. Taking the Wood-Berry model, Eqn. 3.24, as
an example, the final steady-state as a function of the manipulated variable u, is
 
12.8 18.9
yss = Gss u = u (3.29)
6.6 19.4
106 Modelling

Distillate xD
0.99

0.98

0.04

0.03

Base, xB
0.02

0.01

0
160

Reflux
140

120
200
reboiler
180
Figure 3.14: Distillation interactions are evi-
dent when we step change the distillate and 160
0 50 100 150 200 250 300
bottoms setpoints independently. time (min)

The ijth element of Gss is the open loop gain between yi and uj . The Gss matrix can be evaluated
experimentally or formed by applying the final value theorem to Eqn 3.24. Now the closed loop
gain is a function of the open loop gain, so only the open loop gains are needed to evaluate the
RGA. Mathematically the relative gain array, , is formed by multiplying together Gss and G ss
elementwise,3

= Gss G1 ss (3.30)
where the special symbol means to take the Hardamard product (also known as the Schur
product), or simply the elementwise product of two equally dimensioned matrices as opposed to
the normal matrix multiplication.

In M ATLAB, the evaluation of the relative gain array is easy.

>>Gss = [12.8, -18.9; 6.6, -19.4] % Steady-state gain from Eqn. 3.29
>>L = Gss.*inv(Gss)' % See Eqn. 3.30. Dont forget the dot-times (.*)

which should return something like


 
2.0094 1.0094
=
1.0094 2.0094

Note that all the columns and rows sum to 1.0. We would expect this system to exhibit severe
interactions, although the reverse pairing would be worse.

The usefulness of the RGA is in choosing which manipulated variable should go with which
control variable in a decentralised control scheme. We juggle the manipulated/control variable
parings until most approaches the identity matrix. This is an important and sensitive topic in
process control and is discussed in more detail in [195, pp494503], and in [183, p457]. The most
frequently discussed drawback of the relative gain array, is that the technique only addresses the
steady state behaviour, and ignores the dynamics. This can lead to poor manipulated/output
pairing in some circumstances where the dynamic interaction is particularly strong. The next
section further illustrates this point.
 
3 Note that this does not imply that = Gss G1
ss , i.e. without the Hardamard product. See [183, p456] for further
details.
3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS 107

T HE DYNAMIC RELATIVE GAIN ARRAY

As mentioned above, the RGA is only a steady-state interaction indicator. However we could use
the same idea to generate an interaction matrix, but this time consider the elements of the transfer
function matrices as a function of frequency by substituting s = j. This now means that the
dynamic relative gain array, (), is a matrix where the elements are functions of as opposed to
being constants.

Consider the (2 2) system from [185]



  2.5e5s 5  
y1 (15s + 1)(2s + 1) u1
4s + 1
= (3.31)
y2 1 4e5s u2
3s + 1 20s + 1
| {z }
G(s)

which has as its distinguishing feature significant time delays on the diagonal elements. We could
compute the dynamic relative gain array matrix using the definition

(s) = G(s) G(s)T (3.32)

then substituting j for s, and perhaps using the symbolic toolbox to help us with the possibly
unwieldy algebra.

Listing 3.1: Computing the dynamic relative gain array analytically


>>syms s
>>syms w real
3

>>G=[2.5*exp(-5*s)/(15*s+1)/(2*s+1), 5/(4*s+1); ...


1/(3*s+1), -4*exp(-5*s)/(20*s+1)] % Plant from Eqn. 3.31

>>RGA = G.*inv(G') % (s), Eqn. 3.32


8 >>DRGA = subs(RGA,'s',1j*w) % Substitute s = j

>> abs(subs(DRGA,w,1)) % Magnitude of the DRGA matrix at = 1 rad/s


ans =
0.0379 0.9793
13 0.9793 0.0379

You can see that from the numerical values of the elements of at = 1 rad/s that this system is
not diagonally dominant at this important frequency. Fig. 3.15 validates this observation.

An alternative numerical way to generate the elements of the DRGA matrix as a function of fre-
quency is to compute the Bode diagram for the multivariable system, extract the current gains,
and then form the RGA from Eqn. 3.30.

Listing 3.2: Computing the dynamic relative gain array numerically as a function of . See also
Listing 3.1.
G = tf({2.5, 5; 1 -4}, ...
2 {conv([15 1],[2,1]),[4 1]; [3 1],[20 1]}, ...
'ioDelayMatrix',[5,0; 0,5]) % Plant from Eqn. 3.31

[,,K] = zpkdata(G); sK =sign(K); % get sign of the gains


[Mag,Ph,w] = bode(G); % Compute Bode diagram
108 Modelling

7 DRGA = NaN(size(Mag));

for i=1:length(w)
K = Mag(:,:,i).*sK; % Gain, including signs
DRGA(:,:,i) = K.*(inv(K))'; % () = K KT
12 end

OnDiag = squeeze(DRGA(1,1,:)); OffDiag = squeeze(DRGA(1,2,:));


semilogx(w, OnDiag,'-',w,OffDiag,'--'); % See Fig. 3.15

The trends of the diagonal and off-diagonal elements of are plotting in Fig. 3.15. What is in-
teresting about this example is that if we only consider the steady-state case, = 0, then is
diagonally dominant, and our pairing looks suitable. However what we should be really con-
centrating on are the values around the corner frequency at 0.1 where now the off-diagonal
terms start to dominate.

1
1,1
0.8 1,2

0.6

0.4

0.2

Figure 3.15: The diagonal and off-diagonal el-


ements of the RGA matrix as a function of 0
3 2 1 0 1
10 10 10 10 10
frequency. frequency

For comparison, Fig. 3.16 shows the dynamic RGA for the (33) OLMR distillation column model
from Eqn. 3.25. In this case the off-diagonal elements do not dominate at any frequency.

2.5

11
2

22 Diagonal elements
1.5

elements of ()

33
1

0.5

0 23

0.5 13 Offdiagonal elements
12
1
Figure 3.16: Comparing the diagonal and
off-diagonal elements of the (3 3) RGA 1.5
3 2 1 0 1
10 10 10 10 10
matrix from Eqn. 3.25. frequency

3.3 C URVE FITTING EXPERIMENTAL DATA

In most cases of industrial importance, the models are never completely white-box, meaning that
there will always be some parameters that need to be fitted, or at least fine-tuned to experimental
data. Generally the following three steps are necessary:
3.3. CURVE FITTING EXPERIMENTAL DATA 109

1. Collect N experimental independent xi , and dependent yi data points.


2. Select a model structure, f (, x) with a vector of parameters that are to be estimated.
3. Search for the parameters that optimise some sort performance index such as to maximise
the goodness of fit, or equivalently minimise the sum of squared residuals.

In this, and the following sections, we will colour the parameters to be estimated in green to
distinguish them clearly from the experimental (x, y) data pairs.

A common performance index or loss function is to select the adjustable parameters so that the
sum of difference squared between the raw data and your model predictions is minimised. This
is the classical Gaussian least-squares minimisation problem. Formally we can write the perfor-
mance index as,
XN
2
J = (yi yi ) (3.33)
i=1

where the ith model prediction, yi is a function of the model, parameters, , and input data,

yi = f (, xi ) (3.34)

We want to find the best set of parameters, or in other words the parameter vector that min-
imises the cost function J in Eqn. 3.33,
(N )
X
2
= arg min (yi yi ) (3.35)
i=1

The term arg min part of Eqn. 3.35 can be read as the argument who minimises since we are
not particularly interested in the actual value of the performance function at the optimum, J , but
rather the value of the parameters at the optimum, . This least-squared estimation is an intuitive
performance index, and has certain attractable analytical properties. The German mathematician
Gauss is credited with popularising this approach, and indeed using it for analysing data taken
while surveying Germany and making astronomical observations.

Slight modifications to the cost function J in Eqn. 3.33 include weighting the individual errors,
or adding a regularisation component which is sometimes known as ridge regression in statis-
tics or Tikhonov regulisation. These modifications can improve estimation when the data is ill-
conditioned.

Note however other possible objectives rather than Eqn. 3.33 are possible, such as minimising the
sum of the absolute value of the deviations, or minimising the single maximum deviation. Both
these latter approaches have seen a resurgence of interest of the last few years since the computer
tools have enabled investigators to by-pass the difficulty in analysis.

There are many ways to search for the parameters, and any statistical text book will cover these.
However the least squares approach is popular because it is simple, requires few prior assump-
tions, and typically gives good results.

3.3.1 L EAST- SQUARES POLYNOMIAL REGRESSION

A popular way to fit smooth curves to experimental data is to find a polynomial that passes
through the cloud of experimental data. This is known as polynomial regression. The nth order
polynomial to be regressed is

y = n xn + n1 xn1 + + 1 x + 0 (3.36)
110 Modelling

where we try different values of the (n + 1) parameters in the vector until the calculated depen-
dent variable y is close to the actual measured value y. Mathematically the vector of parameters
is obtained by solving a least squares optimisation problem.

Given a set of parameters, we can predict the ith model observation using

n
n1

x 1 ...
 n n1
yi = x x (3.37)

1
0

or written in compact matrix notation


yi = xTi
where xi is the independent (column) data vector for the ith observation. If all n observations are
stacked vertically together, we obtain the matrix system
n
y1 x1 x1n1 x1 1
y2 xn2 x1n1 x2 1

.. = . .. .. .. .. (3.38)
. .. . . . .
n1
yN xn xN xN 1
| N {z }
X
or y = X in a more compact matrix notation. The matrix of comprised of the stacked rows of
measured independent data, X, in Eqn. 3.38 is called the Vandermonde matrix or data matrix and
can be easily constructed in M ATLAB using the vander command, although it is well known that
this matrix can become very illconditioned.

We can search for the parameters in a number of ways, but finding the parameters that minimise
the sum of squared errors is a common and easy approach. In matrix notation, the error vector is
defined as
def
= y X (3.39)
and in this case our scalar objective function that we want to minimise (sometimes called the cost
function), is the sum of squared errors,

J = = (y X) (y X)
= y y X y y X + X X

We want to choose such that J is minimised, ie a stationary point, thus we can set the partial
derivatives of J with respect to the parameters to the zero row vector,4
J
=0

 T
= XT y yT X 2T XT X (3.40)
= 2yT X + 2T X X (3.41)

which we can now solve for the unknown vector as


 1
= X X X y (3.42)
| {z }
pseudo-inverse
4 See also section 9.3.1. Ogata, [152, p938], also gives some helpful rules when using matrix-vector differentials, but

uses a slightly different nomenclature standard.


3.3. CURVE FITTING EXPERIMENTAL DATA 111

As a consequence of the fact that we carefully chose our model structure in Eqn. 3.36 to be linear
in the parameters, , then the solution given by Eqn. 3.42 is analytic and therefore very reliable,
and straight forward to implement as opposed to nonlinear regression which require iterative
solution techniques.

This method of fitting a polynomial through experimental data is called polynomial least-squares
regression. In general, the number of measurements must be greater (or equal) to the number
of parameters. Even with that proviso, the data matrix X X can get very ill-conditioned, and
hence it becomes hard to invert in a satisfactorily manner. This problem occurs more often when
high order polynomials are used or when you are trying to over-parameterise the problem. One
solution to this problem is given in section 3.3.2.
 1
The matrix X X X is called the left pseudo-inverse of X, and is sometimes denoted X+ .
The pseudo inverse is a generalised inverse applicable for even non-square matrices and is dis-
 1
cussed in [154, p928]. M ATLAB can compute X X X with the pseudo inverse command,
pinv, enabling the parameters to be evaluated simply by typing theta = pinv(X)*y, al-
though it is more efficient to simply use the backslash command, theta = X\y.

A N EXAMPLE OF POLYNOMIAL CURVE FITTING USING LEAST- SQUARES

Tabulated below is the density of air as a function of temperature. We wish to fit a smooth
quadratic curve to this data.

Temperature [ C] 100 50 0 60 100 160 250 350


Air density [kg/m3 ] 1.98 1.53 1.30 1.067 0.946 0.815 0.675 0.566

2.5
Raw Data
fitted curve
2
Air density, [kg/m3]

1.5

Figure 3.17: The density of air as a function


of temperature. Experimental data, , and
0.5
200 100 0 100 200 300 400 a fitted quadratic curve. (See also Fig. 3.18
Temperature [C] following for a higher-order polynomial fit.)

To compute the three model parameters from the air density data we can run the following m-file.

Listing 3.3: Curve fitting using polynomial least-squares


T= [-100 -50 0 60 100, 160 250 350]'; % Temperature, T
rho= [1.98, 1.53, 1.30, 1.067 0.946, 0.815, 0.675, 0.566]'; % Air density,

X =vander(T);X =X(:,end-2:end); % Keep only last 3 columns of the Vandermonde matrix


5 theta = X\rho; % Solve for , where = 1 T 2 + 2 T + 3

Ti = linspace(-130,400); % validate with a plot


rho_pred = polyval(theta,Ti);
112 Modelling

plot(T,rho,'o',Ti,rho_pred,'r-') % See Fig. 3.17.

The resulting curve is compared with the experimental data in Fig. 3.17.

In the example above, we constructed the Vandemonde data matrix explicitly, and solved for the
parameters using the pseudo-inverse. In practice however, we would normally simply use the
built-in M ATLAB command polyfit which essentially does the same thing.

3.3.2 I MPROVEMENTS TO THE LEAST SQUARES ESTIMATION ALGORITHM

There are many extensions to this simple multi-linear regression algorithm that try to avoid the
poor numerical properties of the scheme given above. If you consider pure computational speed,
then solving a set of linear equations is about twice as fast as inverting a matrix. Therefore instead
of Eqn. 3.42, the equivalent
X X = X y (3.43)
is the preferred scheme to calculate . M ATLAB S backslash operator, \, used in the example
above follows this scheme internally. We write it as if we expect an inverse, but it actually solves
a system of linear equations.

A numerical technique that uses singular value decomposition (SVD) gives better accuracy (for
the same number of bits to represent the data), than just applying Eqn 3.42, [163, 180]. Singular
values stem from the property that we can decompose any matrix T into the product of two
orthogonal matrices (U, V) and a diagonal matrix ,
T = UV (3.44)
where due to the orthogonality,
UU = I = U U and VV = I = V V (3.45)
The diagonal matrix consists of the square roots of the eigenvalues of T T which are called
the singular values of T, and the number that differ significantly from zero is the rank of T. We
can make one further modification to Eqn 3.42 by adding a weighting matrix W. This makes the
solution more general, but often in practice one sets W = I. Starting with Eqn 3.43 including the
weighting matrix W1 = G G,
X W1 X = X W1 y (3.46)

X G GX = X G Gy (3.47)
def def
Now we define T = GX and z = Gy gives
T T = T z (3.48)
Now we take the singular value decomposition (SVD) of T in Eqn 3.48 giving
  
VU UV = VU z (3.49)
V2 V = (3.50)
Multiplying both sides by V and noting that is diagonal we get,
2 V = U z (3.51)
1
| {z } = V
VV U z (3.52)
identity
1
= V
| {z U } z (3.53)
pseudo inverse
3.3. CURVE FITTING EXPERIMENTAL DATA 113

The inverse of is simply the inverse of each of the individual diagonal (and non-zero) elements,
since it is diagonal. The key point here is that we never in the algorithm needed to calculate the
possibly ill-conditioned matrix T T. Consequently we should always use Eqn. 3.53 in preference
to Eqn. 3.42 due to the more robust numerical properties as shown in Listing 3.4.

Listing 3.4: Polynomial least-squares using singular value decomposition. This routine follows
from, and provides an alternative to, Listing 3.3.
1 [U,S,V] = svd(X,0); % Use the economy sized SVD, UV = X
theta2 = V*(S\U')*rho % = V1 U , Eqn. 3.53.

It does not take much for the troublesome XT X matrix to get ill-conditioned. Consider the tem-
perature/density data for air from Fig. 3.17, but in this case we will use temperature in degrees
Kelvin (as opposed to Celsius), and we will take data over slightly larger range of temperatures.
This is a reasonable experimental request, and so we should expect to be able to fit a polynomial
to this new data in much the same way we did in Listing 3.3.

However as shown in Fig. 3.18, we cannot reliably fit a fifth-order polynomial using standard
least-squares to this data set, although we can reliably fit such a 5th-order polynomial using the
SVD strategy of Eqn. 3.53. Note that M ATLABs polyfit uses the reliable SVD strategy internally.

Polynomial order = 5
3.5
Raw Data
standard LS
3 Standard leastsquares SVD fitted
= (X X)1 X y
2.5
Air density, [kg/m3]

2 LS using SVD
= V 1 U T y
1.5

1 Figure 3.18: Comparing vari-


ous ways to fit a 5th order
0.5 polynomial to the density of
air as a function of tempera-
0 ture in degrees Kelvin. (See
100 200 300 400 500 600 700
Temperature [degrees Kelvin] also Fig. 3.17.)

A further refined regression technique is termed partial least squares (PLS), and is summarised
in [74]. Typically these schemes decompose the data matrix X, into other matrices that are
ordered in some manner that roughly corresponds to information. The matrices with the least
information (and typically most noise) are omitted, and the subsequent regression uses only the
remaining information.

The example on page 111 used the least squares method for estimating the parameters of an
algebraic equation. However the procedure for estimating a dynamic equation remains the same.
This will be demonstrated later in 6.7.

Problem 3.3 Accuracy and numerical stability are very important when we are dealing with com-
puted solutions. Suppose we wish to invert
 
1 1
A=
1 1+
where is a small quantity (such as nearly the machine eps)
114 Modelling

1. What is the (algebraic) inverse of A?

2. Obviously if = 0 we will have trouble inverting since A is singular, but what does the
pseudo-inverse, A+ , converge to as 0?

3.3.3 I DENTIFICATION OF NONLINEAR MODELS

If the model equation is nonlinear in the parameters, then the solution procedure to find the opti-
mum parameters requires a nonlinear optimiser rather than the relatively robust explicit relation
given by Eqn. 3.42 or equivalent. Nonlinear optimisers are usually based on iterative schemes
additionally often requiring good initial parameter estimates and even then may quite possibly
fail to converge to a sensible solution. There are many algorithms for nonlinear optimisation
problems including exhaustive search, the simplex method due to Nelder-Mead, and gradient
methods.

M ATLAB provides a simple unconstrained optimiser, fminsearch, which uses the Nelder-Mead
simplex algorithm. A collection of more robust algorithms and algorithms to optimise constrained
systems possibly with integer constraints is the O PTI toolbox5 .

The following biochemical example illustrates the solution of a nonlinear algebraic optimisation
problem. Many biological reactions are of the Michalis-Menton form where the cell number y(t)
is given at time t by the relation
t
y(t) = (3.54)
t + 1

In this model, time is the independent variable, y(t) is the observed variable, and the two param-
eters are
 
def
=

Suppose we have some experimental data for a particular reaction given as

time, t 0 0.2 0.5 1 1.5 2 2.5


cell count, y 0 1.2 2.1 2.43 2.52 2.58 2.62

where we wish to estimate the parameters and in Eqn. 3.54 using nonlinear optimisation tech-
niques. Note that the model equation (3.54) is nonlinear in the parameters. While it is not possible
to write the equation in the linear form y = f (t) as done in 3.3, it is possible to linearise the
equation (Lineweaver-Burke plot) by transforming the equation. However this will also trans-
form and bias the errors around the parameters. This is not good practice as it introduces bias
in the estimates, but does give good starting estimates for the nonlinear estimator. Transforming
Eqn. 3.54 we get
y
y=
t

Thus plotting y/t against y should result in a straight line with an intercept of / and a slope of
1/.

5 The O PTI toolbox is available from www.i2c2.aut.ac.nz


3.3. CURVE FITTING EXPERIMENTAL DATA 115

3.5

Assuming we have the experimental data stored as


3
column vectors [t,y] in M ATLAB, we could plot

cell count, y
2.5

t = [0, 0.2, 0.5:0.5:2.5]';


2
y = [0,1.2,2.1,2.43,2.52,2.58,2.62]';
3 % ignore divide-by-zero 1.5
plot(y./t,y,'o-')
1
0 1 2 3 4 5 6
y/t
which should given an approximately straight line (ignoring the first and possibly the second
points). To find the slope and intercept, we use the polyfit function to fit a line and we get a
slope = 0.2686 and intercept = 2.98. This corresponds to = 11.1 and = 3.723.

Now we can refine these estimates using the fminsearch Nelder-Mead nonlinear optimiser.
We must first construct a a small objection function file which evaluates the sum of the squared
error for a given set of trial parameters and the experimental data. This is succinctly coded as an
anonymous function in Listing 3.5 following.

Since we require the experimental data variables y and t in the objective function, but we are not
optimising with respect to them, we must pass these variables as additional parameters to the
optimiser.

Listing 3.5: Curve fitting using a generic nonlinear optimiser


1 % Compute sum of square errors given trial and (t, y) data.
sse= @(theta,t,y) sum((y-theta(1)*t./(theta(2)*t+1)).2);

optns = optimset('Diagnostics','on','Tolx',1e-5);
theta = fminsearch(sse,[11.1 3.723]',optns,t,y) % Polish the estimates.
6 plot(t,y,'o',t, theta(1)*t./(theta(2)*t+1)) % Check final fit in Fig. 3.19.

Listing 3.5 returns the refined estimates for as


   
12.05
= =
4.10

and a comparison of both the experimental data , and the models predictions is given in Fig. 3.19.

3 /

2.5

2
cell count

1.5

0.5 Exptal data Figure 3.19: The fitted (dashed) and experi-
Best fit curve mental, , data for a bio-chemical reaction.
0
0 0.5 1 1.5 2 2.5 3 3.5
The asymptotic final cell count, /, is given
time by the dashed horizontal line.
116 Modelling

C URVE FITTING USING THE O PTI OPTIMISATION TOOLBOX

The M ATLAB O PTIMISATION toolbox contains more sophisticated routines specifically intended
for least-squares curve fitting. Rather than write an objective function to compute the sum of
squares and then subsequently call a generic optimiser as we did in Listing 3.5, we can solve the
problem in a much more direct manner. The opti_lsqcurvefit is the equivalent routine in the
O PTI toolbox that solves least-squares regression problems.

Listing 3.6: Curve fitting using the O PTI optimisation toolbox. (Compare with Listing 3.5.)
>> theta0 = [11,3]'; % Initial guess 0
>> F = @(x,t) x(1)*t./(x(2)*t+1); % f (, t) = 1 t/(2 t + 1)
>> >> theta = opti_lsqcurvefit(F,theta0,t,y)
4 theta =
12.0440
4.1035

Note that in Listing 3.6 we encapsulate the function in an anonymous function which is then
passed to the least-squares curve fit routine, opti_lsqcurvefit.

H IGHER DIMENSIONAL MODEL FITTING

Searching for parameters where we have made two independent measurements is just like search-
ing when we have made one independent measurement, except that to plot the results we must
resort to contour or three-dimensional plots. This makes the visualisation a little more difficult,
but changes nothing in the general technique.

We will aim to fit a simple four parameter, two variable function to model the compressibility
of water. Water, contrary to what we taught in school, is compressible, although this is only
noticeable at very high pressures. If we look up the physical properties for compressed water
in steam tables,6 we will find something like table 3.4. Fig. 3.20(a) graphically illustrates the

Table 3.4: Density of compressed water, ( 103 kg/m3 )

Pressure (bar) Temperature, C


0.01 100 200 250 300 350 374.15
100 1.0050 0.9634 0.8711 0.8065 0.7158
221.2 1.0111 0.9690 0.8795 0.8183 0.7391 0.6120
500 1.0235 0.9804 0.8969 0.8425 0.7770 0.6930 0.6410
1000 1.0460 1.0000 0.9242 0.8772 0.8244 0.7610 0.7299

strong temperature influence on the density compared with pressure. Note how the missing data
represented by NaNs in M ATLAB is ignored in the plot.

Our model relates the density of water as a function of temperature and pressure. The proposed
model structure is  
b c
= P k exp a + + 3 (3.55)
T T
where the model parameters
def  
= a b c k (3.56)
6 Rogers & Mayhew, Thermodynamic and Transport properties of Fluids, 3rd Edition, (1980), p11
3.3. CURVE FITTING EXPERIMENTAL DATA 117

density (kg/m3)
1100

1000
1000
1100 900
950 850
800
3

1000
True density kg/m

775
900

Pressure (bar)
700
900
850
600
800
800 500
700
750 400 700
600
0
1000 700 300
100 700
200 200 1e+003 1e+003 925
925 850 775
500 650 1e+003
300 100
400 0 Pressure (bar) 0 50 100 150 200 250 300 350 400
Temperature, C Temperature (deg C)

(a) True density of compressed water as a function of tem- (b) Model fit
perature and pressure.

Figure 3.20: A 2D model for the density of compressed water. In Fig. 3.20(b), the mark experi-
mental data points used to construct the 2-D model given as contour lines.

need to be determined.

Again the approach is to minimise numerically the sum of the squared errors using an optimiser
such as fminsearch. We can check the results of the optimiser using a contour plot with the
experimental data from Table 3.4 superimposed.

The script file in Listing 3.7 calls the minimiser which in turn calls the anonymous function
J_rhowat which given the experimental data and proposed parameters returns the sum of squared
errors. This particular problem is tricky, since it is difficult to know appropriate starting guesses
for , and the missing data must eliminated before the optimisation. Before embarking on the full
nonlinear minimisation, I first try a linear fitting to obtain good starting estimates for . I also
scale the parameters so that the optimiser deals with numbers around unity.

Listing 3.7: Fitting water density as a function of temperature and pressure


rhowat = @(a,P,T) P.a(1).*exp(a(2) + ...
1e2*a(3)./T + 1e7*a(4)./T.3) % Assumed model = P k exp a + Tb + Tc3


J_rhowat = @(a,P,T,rho) ...


norm(reshape(rho,[],1)-reshape(rhowat(a,P,T),[],1)); % SSE J = ( )2
P
4

T = [0.01, 100, 200, 250,300,350, 374.15]'; % Temp [deg C]


P = [100 221.2 500 1000]'; % Pressure [Bar]
rhof = 1.0e3*[1.00250, 0.9634, 0.8711, 0.8065, 0.7158, NaN, NaN; ...
9 1.0111, 0.9690, 0.8795, 0.8183, 0.7391, 0.6120, NaN; ...
1.0235, 0.9804, 0.8969, 0.8425, 0.7770, 0.6930, 0.6410; ...
1.0460, 1.0000, 0.9242, 0.8772, 0.8244, 0.7610, 0.7299]; % density kg/m3

[TT,PP] = meshgrid(T+273,P); Tv = TT(:); Pv = PP(:); % vectorise


14 A = [ones(size(Tv)), 1.0e2 ./Tv, 1.0e7 ./Tv.3, log(Pv)]; % Scaled data matrix
idx = isnan(rhof(:)); % find missing data points
rhofv = rhof(:); rhofv(idx) = []; % remove bad points
Tv(idx) = []; Pv(idx) = []; A(idx,:) = [];
theta = A\log(rhofv); % first (linear) estimate
19

% Do nonlinear fit
118 Modelling

theta_opt = fminsearch(@(theta) J_rhowat(theta,Pv,Tv,rhofv),theta);

[Ti,Pi] = meshgrid([0.01:10:370]+273,[100:100:1000]); % Compare fit with data


24 rho_est = rhowat(theta_opt,Pi,Ti);

Fig. 3.20(b) compares contour plots of the density of water as a function of pressure and temper-
ature derived from the experimental data and the fitted model. The shows the location of the
experimental data points. The solid contour lines give the predicted density of water compared
with the contours derived from the experimental data (dashed lines). The optimum parameters
found above are
a 5.5383
b 5.8133 102
= c = 1.8726 107
(3.57)
k 0.0305

3.3.4 T HE CONFIDENCE OF THE OPTIMISED PARAMETERS

Finding the optimum model parameters to fit the data is only part of the task. A potentially
more difficult objective is to try to establish how precise these optimum parameters are, or what
the confidence regions of the parameters are. simply expressing the uncertainty of your model as
parameter values some interval is a good first approximation, but it does neglect the interaction
of the other parameters, the so-called correlation effect.

The following data was taken from [86, p198], but with some scaling, modifications and correc-
tions. Himmelblau, [86, p198], gives some reaction rate data, y, as a function of pressure, x, as:

Pressure, x 2 3 3.5 4 5 5.5 6


rate, y 0.68 0.858 0.939 0.999 1.130 1.162 1.190

and proposes a two parameter nonlinear model of the form


1 x
y(x) = (3.58)
1 + 2 x
In this regression problem we have n = 2 parameters, and m = 7 experimental observations.

Now suppose that we use a nonlinear optimiser to search for the parameters such that the sum
of squared error is minimised,
n
!

X
2
= arg min (yi yi ) (3.59)
i=1

If we do this optimisation using say the same technique as described in 3.3.3 calling lsqcurvefit
with an initial guess of say T = [1, 1],

Listing 3.8: Finding the optimum parameter for a nonlinear reaction rate model
1 x = [2,3,3.5,4,5,5.5,6]'; % Pressure, x
y = [0.68,0.858,0.939,0.999,1.130,1.162,1.190]'; % Reaction rate, y(x)

1 x
Rxn_rate = @(p,x) p(1)*x./(1+p(2)*x); % y(x) = 1+2 x
theta0 = [1,1]'; % initial guess
6 theta = lsqcurvefit(Rxn_rate,theta0,x,y); % Find optimum using lsqcurvefit
3.3. CURVE FITTING EXPERIMENTAL DATA 119

then we find that the optimum parameters are approximately


 
0.5154
= (3.60)
0.2628

and the raw data and model predictions are compared in Fig. 3.23.

Experimental data (o) & model fit


1.3
Data
1.2 model

1.1

1
rate

0.9

0.8

0.7

0.6

0.5 Figure 3.21: Fitting experimental raw data


1 2 3 4 5 6 7
pressure (), to a nonlinear model.

The fit in Fig. 3.21 looks reasonable, but it would be prudent to be able to statistically defend the
quality of the fit. To do that, we need to establish the individual confidence limits of the parameters,
and to do that, we need to know the following:

1. The n optimum parameters obtained from the m experimental observations. (For the
nonlinear case, this can be found using a numerical optimiser as shown above in Listing 3.8.)
2. A linearised (m n) data matrix, also known as a Jacobian, X, centered about the optimised
parameters at each observation defined as

def yi
Xi,j = (3.61)
j
Thus each of the m rows of X are the partial derivatives of that particular observation with
respect to each of the n parameters. Note that the dimensions of X should be taller than
wide.
It is possible that the numerical optimiser can export X at the solution, or we could compute
it using either an analytical expression or finite differences.
3. We can now form the (scaled) covariance matrix
 1
def
P = X X (3.62)

from the data matrix.


Being a covariance matrix, P must be symmetric positive definite although in practice this
requirement may not always hold owing to poor numerical conditioning. The diagonal
elements are the variances we will use for the individual confidence limits. When using
M ATLAB, it is better to look at the singular values, or check the rank of X X before doing
the inversion.
4. An estimate of the measurement noise sYi . This may be known from prior knowledge, or
simply approximated by s
p
sYi s2r = (3.63)
mn
120 Modelling

where the is the sum of squared residuals, and the positive integer = (mn) is known
as the degree of freedom.
5. Finally we also need to decide on a suitable confidence interval, typically 90%, 95% or 99%.
One this is decided, we can compute the value of the inverse cumulative t-distribution at
degrees of freedom using appropriate statistical tables, or the simple statistical functions
given in Listing 3.9.

Listing 3.9: Routines for the cumulative probability distribution, pt, and the inverse CDF,
qt, for the t-distribution.
pt = @(x,v) (x0).*(1-0.5*betainc(v./(v+x.2),v/2,0.5)) + ...
(x<0).*(0.5*betainc(v./(v+x.2),v/2,0.5))
qt = @(P,v) fsolve(@(x) pt(x,v)-P,sqrt(2)*erfinv(2*P-1), ...
4 optimset('display','off'));

Once we have computed all of the above, the confidence interval for parameter i is given by
p
i t(1/2) sYi Pi,i (3.64)

where t(1/2) is the inverse of the cumulative Students t distribution evaluated at = n


m (number of observations less the number of parameters) degrees of freedom and Pi,i are the
diagonal elements of the covariance matrix P.

Note that if we assume we have perfect measurements and a perfect model, then we would expect
that the sum of the errors will be exactly zero given the true parameters, and hence the uncertainty
interval to be zero. This is consistent with Eqn. 3.63 when = 0 although of course this occurs
rarely in practice!

The t-statistic in Eqn. 3.64, t(1/2) is the value of x to which only /2 of the integral lies to the
right. This quantity can easily obtained from statistical tables or using the qt routine from the
S TIXBOX collection mentioned on page 4 for m-file implementations of some commonly used
statistical functions, or tinv from the Statistics toolbox.

Fig. 3.22 plots the inverse of the cumulative Students t distribution as a function of degrees of
freedom for the three commonly used confidence limits: 90%, 95% and 99%. Note that in the
case of 95% ( = 5%), then provided we have sufficient degree of freedoms, (i.e. we are not trying
to identify n parameters with the minimum of n experiments), then the t statistic both converges
to the inverse of the cumulative normal distribution, and furthermore, is approximately equal to
2.

For the regression problem started on page 118, we can compute the Jacobian X in closed form.
The partial derivatives of the model Eqn. 3.58 with respect to the parameters are

y x y 1 x2
= , =
1 1 + 2 x 2 (1 + 2 x)2

Given the optimum parameters established in Listing 3.8, the individual uncertainties are calcu-
lated in Listing 3.10. The cumulative density function for the t-distribution, pt and its inverse,
qt could be substituted with tcdf and tinv from the Statistical toolbox respectively, or using
routines from the R statistical package, [165].

Listing 3.10: Parameter confidence limits for a nonlinear reaction rate model. This routines fol-
lows from Listing 3.8 and uses the statistical functions from Listing 3.9.
3.3. CURVE FITTING EXPERIMENTAL DATA 121

Inverse of the cumulative tdistribution, qt(P,)


12

10

6
x

4 Figure 3.22: The inverse of the cumu-


99%
lative density function of the t dis-
2 95% tribution for varying degrees of free-
90%
dom for 90%, 95% and 99% confi-
dence limits. Note that t(1/2) 2
0
0 5 10 15 20 for = 5% for all but very low de-
degree of freedom, grees of freedom.

1 mobs = length(x); npar = length(theta); % # of observations & # of parameters


y_est = Rxn_rate(theta,x);% Predicted reaction rate y(, P
x)
j = sum((y-y_est).2); % sum of squared errors J = i 2

d1= 1+theta(2)*x;
6 X = [x./d1, -theta(1)*x.*x./d1.2]; % Data gradient matrix
P = inv(X'*X); % not numerically sound ??

alpha = 0.05; % CI level (97.5%)


nu = mobs-npar; % Degrees of freedom = m n
11 t = qt(1-alpha/2,nu); % t statistic from approx function

sy = sqrt(j/nu); % approx measurement noise std


b_ci = t*sy*sqrt(diag(P)); % confidence regions

This gives the 95% confidence limits for the parameters as7
   
0.4808 0.5501
<< (3.65)
0.2301 0.2956
What is important to check in the an expression like Eqn. 3.65 is that none of the parameter bounds
span zero. If they did, then this suggests that the parameter should be dropped.

While the bounds of the parameters are useful, what is more useful is the corresponding bounds
on the predictions. The non-simultaneous prediction bounds are
p
B = y t1/2 s2 + xSxT (3.66)
| {z }
bounds

T 1
where S = (X X) sY and x is the ith row vector of the Jacobian associated with measurement
yi .

A plot of the raw data (), the model with the optimum estimated parameters and the associated
error bounds using Eqn. 3.65 is given in Fig. 3.23.

The confidence region (an ellipse in the two variable case) can also be plotted. This gives a deeper
understanding of the parameter interactions, but the visualisation becomes near impossible as
7 These figures differ from the worked example in Himmelblau. I think he swapped two matrices by mistake, then

continued with this error.


122 Modelling

95% Confidence bounds


1.3
Data
1.2 model
95% bounds
1.1

rate, y(x)
0.9

0.8

0.7

Figure 3.23: The same experimental 0.6


data and model from Fig. 3.21, but
0.5
with the associated 95% error bounds 1 2 3 4 5 6 7
on the predictions. pressure, x

the number of parameters gets much larger than 3. The region for this example is evaluated the
following section.

T HE CONFIDENCE REGIONS OF THE PARAMETERS

An approximate confidence region can be constructed by linearising the nonlinear model about
the optimum parameters. The covariance of the parameter estimate is
 1
cov(a) X X Y2 i (3.67)

where a are the estimated parameters and is the true, but unknown, parameters. Now the
confidence region, an ellipse in two parameter space, or an ellipsoid in higher dimensions, is
the region that with a confidence limit of say 95% we are certain that the true parameter () lies
within.

( a) (X X) ( a) = s2Yi m F1 [m, n m] (3.68)

where F [m, n m] is the upper limit of the F distribution for m and n m degrees of freedom
where m is the number of parameters and n is the number of data
 points.
 This value can be found
in statistical tables. Note that the middle term of Eqn. 3.68 is X X and not the inverse of this.

For the previous example, we can plot the ellipse about the true optimised parameters. Fig 3.24
shows the 95% confidence ellipse about the optimised parameter (marked with a ) and the indi-
vidual confidence lengths superimposed from Eqn. 3.65. Since the ellipse is narrow and slanted,
we note that the parameters are strongly correlated with each other.

The ellipse defines an area in which the probability that the true parameter vector lies is better
than 95%. If we use the parameter points marked as a in Fig. 3.24, then the model prediction
will differ from the prediction using the optimised parameter vector. This difference is shown as
the errorbars in the lower plot of Fig. 3.24. Note that these errorbars are much smaller than the
errorbars I obtained from considering only the confidence interval for each individual parameter
given in Fig 3.23. Here, owing to the correlation (slanting of the ellipse in Fig 3.24), the parameters
obtained when neglecting the correlation lie far outside the true 95% confidence region. Hence
the bigger errorbars.
3.4. NUMERICAL TOOLS FOR MODELLING 123

Optimum solution with 95% confidence

0.3

0.29

0.28

0.27
2

0.26

0.25

0.24
Figure 3.24: The approximate 95% confi-
dence elliptical region in 2 parameter space
0.23
compared with the individual rectangular
0.22
0.46 0.48 0.5 0.52 0.54 0.56
confidence bounds with the individual con-
1 fidence bounds.

3.4 N UMERICAL TOOLS FOR MODELLING

Unfortunately there are no known methods of solving Eqn 1 (a nonlinear differential equation).
This, of course, is very disappointing.
M. Braun, [35, p493]

Since differential equations are difficult to solve analytically, we typically need to resort to numer-
ical methods as described in many texts such as [41, 43, 48, 94, 103, 172] in addition to [206].

Differential equations are broadly separated into two families:

ODEs Ordinary differential equations (ODEs) where time is the only independent variable. The
solution to ODEs can be described using standard plots where the dependent variables are
plotted on the ordinate or y axis against time on the x axis. Ways to numerically solve these
equations are described in section 3.4.1.
PDEs Partial differential equations (PDEs) are where space (and perhaps time) are independent
variables. To display the solution of PDEs, contour maps or 3D plots are required in general.
Generally PDEs are much harder to solve than ODEs and are beyond the scope of this text.

3.4.1 T HE SOLUTION OF NONLINEAR ORDINARY DIFFERENTIAL EQUATIONS

One of the most important tools in the desk-top experimenters collection, is a good numerical
integrator. We need numerical integrators whenever we are faced with a nonlinear differential
equation, the solutions of which is intractable by any other means. M ATLAB supplies a number of
different numerical integrators optimised for different classess of problems. Typically the 4th/5th
dual order Runge-Kutta implementation with adaptive stepsizing, ode45, is a good first choice.

To use the integrators, you must write a function subroutine that contains your differential system
to be solved which will be called by the numerical integration routine.

1. The script file that calls the integrator solver (eg: ode45) with name of the function subrou-
tine to be integrated and parameters including initial conditions, tolerance and time span.
2. The function subroutine (named in the script file) that calculates the derivatives, y, as a
function of the independent variable (usually time), and the dependent variable, f (t, y). In
many cases we are solving a system of ordinary differential equations where the variables
124 Modelling

are coupled. In this case we often use the nomenclature x(t) to represent the vector of
dependent variables.

We will demonstrate this procedure by constructing templates of the above two files to solve a
small nonlinear differential equation. One can reuse this template as the base for solving other
ODE systems.

ODE TEMPLATE EXAMPLE

Suppose our task is to investigate the response of the nonlinear pendulum system given in Eqn. 3.6
and compare this with the linearised version. The script file given in Listing 3.11 computes the
angle = x1 (t) trajectory for both the nonlinear and linear approximation.

Listing 3.11: Comparing the dynamic response of a pendulum to the linear approximation
1 g=8.81; l=1; m = 5; T_c=0; % System constants
xdot = @(t,x) [x(2); -g/l*sin(x(1)) + T_c/m/l2]; % dx/dt

x0 = [1,1]'; % start position x(0)


[t,x]= ode45(xdot,[0,10],x0); % Integrate nonlinear
6

% Now compare with a linearised system


A = [0 1; -g/l 0]; B = [0;0]; C = [1,0];D=0;
tv = linspace(0,10)';
[y2,t2]=lsim(ss(A,B,C,D),0*tv,tv,x0);
11

plot(t,x(:,1),t2,y2) % See Fig. 3.25.

Note how the trajectories for the nonlinear (solid) result as calculated from ode45 and linear
approximation calculated using lsim gradually diverge in Fig. 3.25.

1.5

0.5

0
x(t)

0.5

1
Non Linear
Figure 3.25: The trajectory of the true non- 1.5 Linearised
linear pendulum compared with the lin- 0 2 4 6 8 10
earised approximation. time
3.4. NUMERICAL TOOLS FOR MODELLING 125

A CHAOTIC PENDULUM

A second example of an interesting dynamic system is a chaotic forced pendulum from [24]. The
dynamic equation system is
d
= sin + g cos
dt q
d (3.69)
=
dt
d
= d
dt
where the three states are angular frequency , angular position, , and the phase of the forcing
term, . We are particularly interested in investigating the behaviour of this system as a function
of the three key parameters, g, q and d .

Listing 3.12: The dynamics of a chaotic pendulum from Eqn. 3.69.


xdot = @(t,x,g,q,wd) [ -x(1)/q - sin(x(2))+g*cos(x(3)); ...
x(1); ...
3 wd];
x0 = [-1 2 0.3]'; % Initial conditions

Note how in the definition above for the dynamic system, we have included the three key param-
eters as trailing arguments so that we can easily change them. For example for the case where we
are interested in g = 1.15, q = 2 and d = 2/3, we simulate

1 g=1.15; q=2; wd = 2/3; % Set parameters to g = 1.15, q = 2 and d = 2/3


[t,x] = ode45(@(t,x) xdot(t,x,g,q,wd),[0,500],x0); % Solve ODE

plot(x(:,2), x(:,1)); % Plot phase plot


ylabel('\omega'); xlabel('\theta')
6 axis([-3,3,-3,3])

or to investigate the case where we change g to 1.07, we need only to re-run

g=1.07; % Change parameters g = 1.07 and re-solve


[t,x] = ode45(@(t,x) xdot(t,x,g,q,wd),[0,500],x0);

and we do not need to edit the dynamic system.

Problem 3.4 1. One of my favourite differential equations is called the Van der Pol equation;
d2 y  dy
m 2
b 1 y2 + ky = 0
dt dt
where m = 1, b = 3 and k = 2. Solve the equation for 0 < t < 20 using both ode23 and
ode45. Sketch the solution. Which routine is better? You will need to create a function
called vandepol(t,y) which contains the system of equations to be solved.

Problem 3.5 A particularly nasty nonlinear chemical engineering type model is found in [82]. This
is a model of two CSTRs8 in series, cooled with a co-current cooling coil. The model equations
8 continuously stirred tank reactors
126 Modelling

are:
 
q E
Ca1 = (Caf Ca1 ) k0 Ca1 exp
V1 RT1
 
q (H)k0 Ca1 E
T1 = (Tf T1 ) + exp +
V1 cp RT1
  
c cpc hA1
qc 1 exp (Tcf T1 )
cp V1 qc c cpc
 
q E
Ca2 = (Ca1 Ca2 ) k0 Ca2 exp
V2 RT2
 
q (H)k0 Ca2 E
T2 = (T1 T2 ) + exp +
V2 cp RT2
      
c cpc hA2 hA1
qc 1 exp T1 T2 + exp (Tcf T1 )
cp V2 qc c cpc qc c cpc

where the state, control, disturbance and measured variables are defined as
def   def def  
x= Ca1 T1 Ca2 T2 , u = qc , d= Caf Tcf
def
y = Ca2

and the parameters values are given in Table 3.5.

Table 3.5: The parameter values for the CSTR model

description variables value units


reactant flow q 100 l/min
conc. of feed reactant A Caf 1.0 mol/l
temp. of feed Tf 350 K
temp. of cooling feed Tcf 350 K
volume of vessels V1 = V2 100 l
heat transfer coeff. hA1 = hA2 1.67 105 J/min.K
pre-exp. constant k0 7.2 1010 min1
E/R E/R 1.0 104 K
reaction enthalpy H 4.78 104 J/mol
fluid density c = 1.0 g/l
heat capacity cp = cpc 0.239 J/g.K

Note that the model equations are nonlinear since the state variable T1 (amongst others) is a
appears nonlinearly in the differential equation. In addition, this model is referred to as control
nonlinear, since the manipulated variable enters nonlinearly. Simulate the response of the con-
centration in the second tank (Ca2 ) to a step change of 10% in the cooling flow using the initial
conditions given in Table 3.6.

Table 3.6: The initial state and manipulated variables for the CSTR simulation

description variable value units


coolant flow qc 99.06 l/min
conc. in vessel 1 Ca1 8.53 102 mol/l
temp. in vessel 1 T1 441.9 K
conc. in vessel 2 Ca2 5.0 103 mol/l
temp. in vessel 2 T2 450 K
3.4. NUMERICAL TOOLS FOR MODELLING 127

You will need to write a .m file containing the model equations, and use the integrator ode45.
Solve the system over a time scale of about 20 minutes. What is unusual about this response?
How would you expect a linear system to behave in similar circumstances?

Problem 3.6 Develop a M ATLAB simulation for the high purity distillation column model given in
[144, pp459]. Verify the open loop responses given on page 463. Ensure that your simulation is
easily expanded to accommodate a different number of trays, different relative volatility etc.

Problem 3.7 A slight complexity to the simple ODE with specified initial conditions is where we
still have the ODE system, but not all the initial conditions. In this case we may know some of the
end conditions instead, thus the system is in principle able to be solved, but not in the standard
manner; we require a trial and error approach. These types of problems are called two point
boundary problems and we see them arise in heat transfer problems and optimal control. They
are so called two point boundary problems because for ODEs we have two boundaries for the
independent variable; the start and the finish. Try to solve the following from [200, p178].

A fluid enters an immersed cooling coil 10m long at 200 C and is required to leave at 40 C. The
cooling medium is at 20 C. The heat balance is

d2 T
= 0.01(T 20)1.4 (3.70)
dx2
where the initial and final conditions are;

Tx=0 = 200, Tx=10 = 40

To solve the system, we must rewrite Eqn 3.70 as a system of two first order differential equations,
and supply a guess for the missing initial condition dTdx x=0 =? We can then integrate the system
until x = 10 and check that the final condition is as required. Solve the system.
dT
Hint: Try 47 < dx x=0 < 42.

3.4.2 D IFFERENTIAL /A LGEBRAIC EQUATION SYSTEMS AND ALGEBRAIC LOOPS

In many cases when deriving models from first principals, we end up with a coupled system
of some differential equations, and some algebraic equations. This is often due to our practice
of writing down conservation type equations (typically dynamic), and constraint equations, say
thermodynamic, which are typically algebraic. Such a system in general can be written as

f (x, x, t) = 0 (3.71)

and is termed a DAE or differential/algebraic equation system. If we assume that our model is
well posed, that is we have some hope of finding a solution, then we expect that we have the
same number of variables as equations, and it follows that some of the variables will not appear
in the differential part, but only in the algebraic part. We may be able to substitute those algebraic
variables out, such that we are left with only ODEs, which can then be solved using normal
backward difference formula numerical schemes such as rk2.m. However it is more likely that
we cannot extract the algebraic variables out, and thus we need special techniques to solve these
sorts of problems (Eqn. 3.71) as one.

In a now classic article titled Differential/Algebraic Equations are not ODEs, [161] gives some insight
into the problems. It turns out that even for linear DAE systems, the estimate of the error, which is
typically derived from the difference between the predictor and the corrector, does not decrease as
the step size is reduced. Since most normal ODE schemes are built around this assumption, they
128 Modelling

will fail. This state of affairs will only occur with certain DAE structures where the nilpotency or
index is 3. Index 2 systems also tend to cause BDF schemes to fail, but can be tackled using other
methods. The index problem is important because in many times it can be changed (preferably
reduced to less than two), by rearranging the equations. Automated modelling tools tend, if left
alone, to create overly complex models with a high index that are impossible to solve. However
if we either by using an intelligent symbolic manipulator or human, change these equations, we
may be able to reduce the index.

T HE DIFFICULTIES OF ALGEBRAIC LOOPS

One reason that computer aided modelling tools such as S IMULINK have taken so long to mature
is the problem of algebraic loops. This is a particular problem when the job of assembling the
many different differential and algebraic equations in an efficient way is left to a computer.

Suppose we want to simulate a simple feedback process where the gain is a saturated function of
the output, say
K(y) = max(min(y, 5), 0.1)
If we simulate this in Simulink using
1
1
s+1
Step Gain Transfer Fcn
Product Scope

Saturation Gain1

we run into an Algebraic Loop error. M ATLAB returns the following error diagnostic (or something
similar):

Warning: Block diagram sgainae contains 1 algebraic loop(s).


Found algebraic loop containing block(s):
sgainae/Gain1
sgainae/Saturation (discontinuity)
sgainae/Product (algebraic variable)
Discontinuities detected within algebraic loop(s), may have trouble solving

and the solution stalls.

The simplest way to avoid these types of problems is to insert some dynamics into the feedback
loop. In the example above, we could place a transfer function with a unit gain and very small
time constant in place of the algebraic gain in the feedback loop. While we desire the dynamics
of the gain to be very fast so that it approximates the original algebraic gain, overly fast dynamics
cause numerical stability problems, hence there is a trade off.

3.5 L INEARISATION OF NONLINEAR DYNAMIC EQUATIONS

While most practical engineering problems are nonlinear to some degree, it is often useful to be
able to approximate the dynamics with a linear differential equation which means we can apply
linear control theory. While it is possible to design compensators for the nonlinear system directly,
this is in general far more complicated, and one has far fewer reliable guidelines and recipes to
3.5. LINEARISATION OF NONLINEAR DYNAMIC EQUATIONS 129

follow. One particular version of nonlinear controller design called exact nonlinear feedback is
discussed briefly in 8.6.

Common nonlinearities can be divided into two types; hard nonlinearities such as hysteresis,
stiction, and dead zones, and soft nonlinearities such as the Arrhenius temperature depen-
dency, power laws, etc. Hard nonlinearities are characterised by functions that are not differen-
tiable, while soft nonlinearities are. Many strategies for compensating for nonlinearities are only
applicable for systems which exhibit soft nonlinearities.

We can approximate soft nonlinearities by truncating a Taylor series approximation of the original
system. The success of this approximation depends on whether the original function has any non-
differentiable terms such as hysteresis, or saturation elements, and how far we deviate from the
point of linearisation. This section follows the notation of [76, 3.10].

Suppose we have the general nonlinear dynamic plant


x = f (x(t), u(t)) (3.72)
y = g(x, u) (3.73)
and we wish to find an approximate linear model about some predefined operating point of in-
terest (xa , ua ). A first-order Taylor series expansion of Eqns 3.723.73 about this point is

f f
x f (xa , ua ) + =xa
(x(t) xa ) + =xa
(u(t) ua ) (3.74)
x x u=ua u x u=ua

g g
y g(xa , ua ) + =xa
(x(t) xa ) + =xa
(u(t) ua ) (3.75)
x u=ua
x u x u=ua
where the ijth element of the Jacobian matrix f /x is f i /xj . Note that for linear systems, the
Jacobian is simply A in this notation, although some authors define the Jacobian as the transpose
of this, or AT .

The linearised system Eqn 3.743.75 can be written as


x = Ax(t) + Bu(t) + E (3.76)
y = Cx(t) + Du(t) + F (3.77)
where the constant matrices A, B, C and D are defined as

f f
A= =xa
, B= =xa
x xu =ua u x
u=ua
g g
C= =xa
, D= =xa
x xu=ua u x u=ua
and the bias vectors E and F are

f f
E = f (xa , ua ) =xa
xa =xa
ua
x x u=ua u xu =ua

g g
F = g(xa , ua ) =xa
xa =xa
ua
x x u=ua u x u=ua

Note that Eqns 3.763.77 are almost in the standard state-space form, but they include the extra
bias constant matrices E and F. It is possible by introducing a dummy unit input to convert this
form into the standard state-space form,
 
  u
x = Ax + B E (3.78)
1
 
  u
y = Cx + D F (3.79)
1
130 Modelling

which we can then directly use in standard linear controller design routines such as lsim.

In summary, the linearisation requires one to construct matrices of partial derivatives with respect
to state and input. In principle this can be automated using a symbolic manipulator provided the
derivatives exist and are well behaved. In both the S YMBOLIC T OOLBOX and in M APLE we can
use the jacobian command as illustrated in section 3.5.1. Alternatively we could construct the
Jacobians using finite differences which is the approach that linmod uses when given an arbitrary
blackbox S IMULINK model as described in section 3.5.2.

3.5.1 L INEARISATION USING THE S YMBOLIC TOOLBOX .

Suppose we want to linearise the two-state nonlinear system


  "   #
x1 ax1 exp 1 xb2
x = =
x2 cx1 (x2 u2 )
| {z }
f (x)

at an operating point, xa = [1, 2]T , ua = 20.

First we start by defining the nonlinear plant of interest, f (x, u),

>> syms x1 x2 a b c u real


>> x = [x1 x2]';
3 >> fx = [a*x(1)*exp(1-b/x(2)); c*x(1)*(x(2)-u2)]
fx =
[ a*x1*exp(1-b/x2)]
[ c*x1*(x2-u2)]

Now we are ready to construct the symbolic matrix of partial derivatives, A(x, u), B(x, u) and
E(x, u).

>> Avar = jacobian(fx,x) % A = f /x. Note this is an (n n) matrix


Avar =
[ a*exp(1-b/x2), a*x1*b/x22*exp(1-b/x2)]
4 [ c*(x2-u2), c*x1]
>> Bvar = jacobian(fx,u) % B = f /u. In this case, B is a column vector.
Bvar =
[ 0]
[ -2*c*x1*u]
9 >> Evar = fx - Avar*x - Bvar*u
Evar =
[ -a*x1*b/x2*exp(1-b/x2)]
[ -c*x1*x2+2*u2*c*x1]

We can substitute a specific set of constants, say, a = 5, b = 6, c = 7 and also substitute the
operating point of interest, x = [1, 2]T , u = 20, into the symbolic matrices above to obtain the now
constant numeric matrices.

>> A = subs(Avar,{a,b,c,x1,x2,u},{5,6,7,1,2,20})
A =
3 1.0e+003 *
0.0007 0.0010
3.5. LINEARISATION OF NONLINEAR DYNAMIC EQUATIONS 131

-2.7860 0.0070
>> B = subs(Bvar,{a,b,c,x1,x2,u},{5,6,7,1,2,20})
B =
8 [ 0]
[ -280]
>> E = subs(Evar,{a,b,c,x1,x2,u},{5,6,7,1,2,20})
E =
1.0e+003 *
13 -0.0020
5.5860

At this point we could compare in simulation the linearised version with the full nonlinear model.

E XAMPLE LINEARISING A NONLINEAR TANK MODEL

Suppose we wish to linearise the model of the level in a tank given on page 91 where the tank
geometry is such that A = 1. The nonlinear dynamic system for the level h is then simplified to
dh
= k h + Fin
dt
For a constant flow in, Finss , which we will term steady state and denote with the superscript ss,
the resulting steady state level is given by
 2
Finss
hss =
k

noting that dh/dt = 0. We wish to linearise the system about this steady-state, so we will actually
work with deviation variables,
def def
x = h hss , u = Fin Finss
Now following Eqn. 3.74, we have
f f
h = x = f (hss , Finss ) + (h hss ) + (Fin Finss )
| {z } h Fin
=0

Note that since f /h = k/(2 hss ), then our linearised model about (Finss , hss ) is
 
k
x = x+u
2 hss
which is in state-space form in terms of deviation variables x and u.

We can compare the linearised model with the nonlinear model in Fig. 3.26 about a nominal input
flow of Finss = 2 and k = 2 giving a steady-state level of hss = 4. Note that we must subtract and
add the relevant biases when using the linearised model.

3.5.2 U SING LINMOD

An alternative, and much simpler way to linearise a dynamic model is to use linmod which
extracts the Jacobians from a S IMULINK model by finite differences.

Listing 3.13: Using linmod to linearise an arbitrary S IMULINK module.


132 Modelling

Repeating
Sequence
Stair 1 level, h
s
Add1 Add
level
Fin_ss
Math
Fin_ss Gain Function

k sqrt

Scope

simout
x = Ax+Bu To Workspace
uFin_ss u+Lss
y = Cx+Du
Bias StateSpace Bias1

(a) A S IMULINK nonlinear tank model and linear state-space model for comparison

10
full nonlinear response
9

7
level

6
linearised response

3
0 10 20 30 40 50
time

(b) Nonlinear and linearised model comparison

Figure 3.26: Comparing the linearised model with the full nonlinear tank level system

1 k=1; Fin_ss = 2; % Model parameters and steady-state input


hss = (Fin_ss/k)2 % Steady-state level, hss
[A,B,C,D] = linmod('sNL_tank_linmod',hss,Fin_ss) % Linearise

A =
6 -0.2500 % Note k/(2 hss ) = 1/4
B =
1
C =
1
11 D =
0

3.5.3 Q UANTIFYING THE EXTENT OF THE NONLINEARITY

It is important for the control designer to be able to quantify, if only approximately, the extent
of the open-loop process nonlinearity. If for example the process was deemed only marginally
nonlinear, then one would be confident a controller designed assuming a linear underlying plant
3.6. SUMMARY 133

would perform satisfactory. On the other hand, if the plant was strongly nonlinear, then such a
linear controller may not be suitable. Ideally we would like to be able to compute a nonlinear
metric, say from 0 (identically linear) to 1 (wildly nonlinear) that quantifies this idea simply by
measuring the openloop input/output data. This of course is a complex task, and is going to be
a function of the type input signals, duration of the experiment, whether the plant is stable or
unstable, and if feedback is present.

One such strategy is proposed in [81] and used to assess the suitability of linear control schemes
in [181]. The idea is to compute the norm of the difference between the best linear approxima-
tion and the true nonlinear response for the worst input trajectory within a predetermined set
of trajectories. This is a nested optimisation problem with a min-max construction. Clearly the
choice of linear model family from which to choose the best one, and the choice of the set of input
trajectories will have an effect on the final computed nonlinear measure.

3.6 S UMMARY

Stuart Kauffman, in a New Scientist article,9 succinctly paraphrased the basis of the scientific
method. He said:

. . . state the variables, the laws linking the variables, and the initial and boundary
conditions, and from these compute the forward trajectory of the biosphere.

In actual fact he was lamenting that this strategy, sometimes known as scientific determinism
voiced by Laplace in the early 19th century was not always applicable to our world as we under-
stand it today. Nonetheless, for our aim of modeling for control purposes, this philosophy has
been, and will remain for some time I suspect, to be remarkably successful.

Modelling of dynamic systems is important in industry. These types of models can be used for
design and or control. Effective modelling is an art. It requires mathematical skill and engineering
judgement. The scope and complexity of the model is dependent on the purpose of the model. For
design studies a detailed model is usually required, although normally only steady state models
are needed at this stage. For control, simpler models (although dynamic) can be used since the
feedback component of the controller will compensate for any model-plant mismatch. However
most control schemes require dynamic models which are more complicated than the steady state
equivalent.

Many of the dynamic models used in chemical engineering applications are built from conser-
vation laws with thermodynamic constraints. These are often expressed as ordinary differential
equations where we equate the rate of accumulation of something (mass or energy) to the inputs,
outputs and generation in a defined control volume. In addition there may be some restrictions
on allowable states, which introduces some accompanying algebraic equations. Thus general
dynamic models can be expressed as a combination of dynamic and algebraic relations
dx
= f (x, u, , t) (3.80)
dt
0 = g(x, u , t) (3.81)
which are termed DAEs (differential and algebraic equations), and special techniques have been
developed to solve them efficiently. DAEs crop up frequently in automated computer modelling
packages, and can be numerically difficult to solve. [137] provides more details in this field.

Steady state models are a subset of the general dynamic model where the dynamic term, Eqn
3.80, is set to equal zero. We now have an augmented problem of the form Eqn 3.81 only. Linear
9 Stuart Kauffman, God of creativity, 10 May 2008, New Scientist, pp52-53
134 Modelling

dynamic models are useful in control because of the many simple design techniques that exist.
These models can be written in the form

x = Ax + Bu (3.82)

where the model structure and parameters are linear and often time invariant.

Models are obtained, at least in part, by writing the governing equations of the process. If these
are not known, experiments are needed to characterise fully the process. If experiments are used
the model is said to be heuristic. If the model has been obtained from detailed chemical and
physical laws, then the model is said to be mechanistic. In practice, most models are a mixture of
these two extremes. However what ever model is used, it still is only an approximation to the real
world. For this reason, the assumptions that are used in the model development must be clearly
stated and understood before the model is used.
C HAPTER 4

T HE PID CONTROLLER

4.1 I NTRODUCTION

The PID controller is the most common general purpose controller in the chemical process indus-
try today. It can be used as a stand-alone unit, or it can be part of a distributed computer control
system. Over 30 years ago, PID controllers were pneumatic-mechanical devices, whereas nowa-
days they are implemented in software in electronic controllers. The electronic implementation
is much more flexible than the pneumatic devices since the engineer can easily re-program it to
change the configuration of things like alarm settings, tuning constants etc.

Once we have programmed the PID controller, and have constructed something, either in soft-
ware or hardware to control, we must tune the controller. This is surprisingly tricky to do success-
fully, but some general hints and guidelines will be presented in 4.6. Finally, the PID controller is
not perfect for everything, and some examples of common pitfalls when using the PID are given
in 4.8.

4.1.1 P, PI OR PID CONTROL

For many industrial process control requirements, proportional only control is unsatisfactory
since the offset cannot be tolerated. Consequently the PI controller is probably the most com-
mon controller, and is adequate when the dynamics of the process are essentially first or damped
second order. PID is satisfactory when the dynamics are second or higher order. However the
derivative component can introduce problems if the measured signal is noisy. If the process has
a large time delay (dead time), derivative action does not seem to help much. In fact PID control
finds it difficult to control processes of this nature, and generally a more sophisticated controller
such as a dead time compensator or a predictive controller is required. Processes that are highly
underdamped with complex conjugate poles close to the imaginary axis are also difficult to con-
trol with a PID controller. Processes with this type of dynamic characteristics are rare in the chem-
ical processing industries, although more common in mechanical or robotic systems comprising
of flexible structures.

My own impressions are that the derivative action is of limited use since industrial measurements
such as level, pressure, temperature are usually very noisy. As a first step, I generally only use a PI
controller, the integral part removes any offset, and the two parameter tuning space is sufficiently
small, that one has a chance to find reasonable values for them.

135
136 CHAPTER 4. THE PID CONTROLLER

4.2 T HE INDUSTRIAL PID ALGORITHM

This section describes how to implement a simple continuous-time PID controller. We will start
with the classical textbook algorithm, although for practical and implementation reasons indus-
trially available PID controllers are never this simple for reasons which will soon become clear.
Further details on the subtlety of implementing a practical useful PID controller are described in
[55] and the texts [14, 16, 98].

The purpose of the PID controller is to measure a process error and calculate a manipulated
action u. Note that while u is referred to as an input to the process, it is the output of the controller.

The textbook non-interacting continuous PID controller follows the equation


 Z 
1 d
u = Kc + dt + d (4.1)
i dt
where the three tuning constants are the controller proportional gain, Kc , the integral time, i and
the derivative time, d , the latter two constants having units of time, often minutes for industrial
controllers. Personally, I find it more intuitive to use the reciprocal of the integral time, 1/i , which
is called reset and has units of repeats per unit time. This nomenclature scheme has the advantage
that no integral action equates to zero reset, rather than the cumbersome infinite integral time. Just
to confuse you further, for some industrial controllers to turn off the integral component, rather
than type in something like 99999, you just type in zero. It is rare in engineering where we get to
approximate infinity with zero! Table 4.1 summarises these alternatives.

Table 4.1: Alternative PID tuning parameter conventions

Parameter symbol units alternative symbol units


Gain Kc input/output Proportional band PB %
integral time i seconds reset 1/i seconds1
derivative time d seconds

The Laplace transform of the ideal PID controller given in Eqn. 4.1 is
 
U (s) 1
= C(s) = Kc 1 + + d s (4.2)
E(s) i s
and the equivalent block diagram in parallel form is give in Fig. 4.1 where is is clear that the three
terms are computed in parallel which is why this form of the PID controller is sometimes also
known as the parallel PID form.

We could rearrange Eqn. 4.2 in a more familiar numerator/denominator transfer function format,

Kc i d s2 + i s + 1
C(s) = (4.3)
i s
where we can clearly see that the ideal PID controller is not proper, that is, the order of the nu-
merator (2), is larger than the order of the denominator (1) and that we have a pole at s = 0. We
shall see in section 4.2.1 that when we come to fabricate these controllers we must physically have
a proper transfer function, and so we will need to modify the ideal PID transfer function slightly.

4.2.1 I MPLEMENTING THE DERIVATIVE COMPONENT

The textbook PID algorithm of Eqn. 4.2 includes a pure derivative term d s. Such a term is not
physically realisable, and nor would we really want to implement it anyway, since abrupt changes
4.2. THE INDUSTRIAL PID ALGORITHM 137

error input integrator Controller output


1
Kc + u
i s

d s

differentiator

Figure 4.1: PID block diagram.

in setpoint would cause extreme values in the manipulated variable.

There are several approximations that are used in commercial controllers to address this problem.
Most schemes simply add a small factory-set lag term to the derivative term. So instead of simply
d s, we would use
d s
derivative term = d (4.4)
s+1
N
where N is a large value, typically set to a large number somewhere between 10 and 100. Using
this derivative term modifies the textbook PID transfer function of Eqn. 4.2 or Eqn. 4.3 to
 
1 d s
C(s) = Kc 1 + + d (4.5)
i s Ns+1
 
(i d + i d N ) s2 + (i N + d )s + N
= Kc (4.6)
i s (d s + N )
which is now physically realisable. Note that as N , the practical PID controller of Eqn. 4.6
converges to the textbook version of Eqn. 4.3. The derivative-filtered PID controller of Eqn. 4.6
collapses to a standard PI controller if d = 0.

We can fabricate in M ATLAB the transfer function of this D-filtered PID controller with the pidstd
command which M ATLAB refers to as a PID controller in standard form.

Listing 4.1: Constructing a transfer function of a PID controller


>> C = pidstd(1,2,3,100) % Construct a standard form PID controller
Continuous-time PIDF controller in standard form:
3
 
1 d s
% C(s) = Kp 1 + i s
+ d
s+1
N

with Kp = 1, Ti = 2, Td = 3, N = 100

The generated controller is of a special class known as a pidstd, but we can convert that to the
more familiar transfer function with the now overlayed tf command.

>> tf(C)

Transfer function:
4 101 s2 + 33.83 s + 16.67
-------------------------
s2 + 33.33 s
138 CHAPTER 4. THE PID CONTROLLER

A related M ATLAB function, pid, constructs a slight modification of the above PID controller in
what M ATLAB refers to as parallel form,

s
Cparallel (s) = P + I/s + D
f s + 1

where in this case as the derivative filter time constant, f , approaches zero, the derivative term
approaches the pure differentiator.

4.2.2 VARIATIONS OF THE PID ALGORITHM

The textbook algorithm of the PID controller given in Eqn. 4.2 is sometimes known as the par-
allel or non-interacting form, however due to historical reasons there is another form of the PID
controller that is sometimes used. This is known as the series, cascade or interacting form
 
1
Gc (s) = Kc 1+ (1 + d s) (4.7)
i s

where the three series PID controller tuning constants, Kc , i and d are related to, but distinct
from, the original PID tuning constants, Kc , i and d . A block diagram of the series PID controller
is given in Fig. 4.2.

Differentiator Integrator
1
d s
i s
u
Kc b
+ b
+
error controller MV

Figure 4.2: A series PID controller

A series PID controller in the form of Eqn. 4.7 can always be represented in parallel form

i + d i d
Kc = Kc , i = i + d , d = (4.8)
i i+ d

but not necessarily the reverse

Kc  p 
Kc = 1 + 1 4d /i (4.9)
2
i  p 
i = 1 + 1 4d /i (4.10)
2
i  p 
d = 1 1 4d /i (4.11)
2

since a series form only exists if i > 4d . For this reason, the series form of the PID controller is
less commonly used, although some argue that it is easier to tune, [190]. Note that both controller
forms are the same if the derivative component is not used. A more detailed discussion on the
various industrial controllers and associated nomenclatures is given in [98, pp32-33].
4.3. SIMULATING A PID PROCESS IN SIMULINK 139

PI controlled response Integral only control


0.5 0.4

0.4 0.3
output

output
0.3 0.2

0.2 0.1

0.1 0
0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200

0.8 0.6

0.6 0.4
input

input
0.4 0.2

0.2 0

0 0.2
0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200
time (sec) dt=0.08 time (sec) dt=0.08

(a) PI control of a flapper with K = 1 and reset= 0.2 (b) Integral-only control of a flapper.

Figure 4.3: Comparing PI and integral-only control for the real-time control of a noisy flapper
plant with sampling time T = 0.08.

4.2.3 I NTEGRAL ONLY CONTROL

For some types of control, integral only action is desired. Fig. 4.3(a) shows the controlled response
of the laboratory flapper, (1.3.1), controlled with a PI controller. The interesting characteristic of
this plants behaviour is the significant disturbances. Such disturbances make it both difficult to
control and demand excessive manipulated variable action.

If however we drop the proportional term, and use only integral control we obtain much the same
response but with a far better behaved input signal as shown in Fig. 4.3(b). This will decrease the
wear on the actuator.

If you are using Eqn. 4.1 with the gain Kc set to zero, then no control at all will result irrespective
of the integral time. For this reason, controllers either have four parameters as opposed to the
three in Eqn. 4.1, or we can follow the S IMULINK convention shown in Fig. 4.6.

4.3 S IMULATING A PID PROCESS IN S IMULINK

S IMULINK is an ideal platform to rapidly simulate control problems. While it does not have the
complete flexibility of raw M ATLAB, this is more than compensated for in the ease of construction
and good visual feedback necessary for rapid prototyping. Fig. 4.4 shows a S IMULINK block
diagram for the continuous PID control of a third order plant.

3
PID
3s3 +5s2 +6s+1.6
Step PID Controller Plant output

Figure 4.4: A S IMULINK block diagram of a PID controller and a third-order plant.

The PID controller block supplied as part of S IMULINK is slightly different from the classical de-
140 CHAPTER 4. THE PID CONTROLLER

scription given in Eqn. 4.2. S IMULINKs continuous PID block uses the complete parallel skeleton,
I Ns
Gc (s) = P + +D (4.12)
s s+N
where we choose the three tuning constants, P, I and D, and optionally the filter coefficient N
which typically lies between 10 and 100. You can verify this by unmasking the PID controller (via
the options menu) block to exhibit the internals as shown in Fig. 4.5.

Proportional Gain

1
1 I 1
s
u Sum y
Integral Gain
Integrator

D N
SumD
Figure 4.5: An unmasked view of a PID Derivative Gain Filter Coefficient

controller block that comes supplied in


1
S IMULINK. Note how the configuration dif- s

fers from the classical form. See also Fig. 4.6. Filter

Note how the derivative component of the S IMULINK PID controller in Eqn. 4.12 follows the
realisable approximation given in Eqn. 4.4 by using a feedback loop with an integrator and gain
of N = 100 as a default.

Block diagrams of both the S IMULINK implementation and the classical PID scheme are compared
in Fig. 4.6. Clearly the tuning constants for both schemes are related as follows:
Kc
P = Kc , I= , D = Kc d (4.13)
i
or alternatively
Kc D
Kc = P, i = , d = (4.14)
I Kc
The S IMULINK scheme has the advantage of allowing integral-only control without modification.

I 1 u
b

s
+ u Kc b
i s +

Ds d s

Simulink PID controller block Classical PID controller block

Figure 4.6: Block diagram of PID controllers as implemented in S IMULINK (left) and classical text
books (right).

If you would rather use the classical textbook style PID controller, then it is easy to modify the
S IMULINK PID controller block. Fig. 4.7 shows a S IMULINK implementation which includes the
filter on the derivative component following Eqn. 4.6. Since PID controllers are very common,
you may like to mask the controller as illustrated in Fig. 4.8, add a suitable graphic, and add this
to your S IMULINK library.
4.4. EXTENSIONS TO THE PID ALGORITHM 141

1
1 Kc 1/taui 1
s
error Add 4 u
Pgain reset Integrator

Figure 4.7: A realisable con-


s
taud tinuous PID controller imple-
taud /N.s+1
mented in S IMULINK with a fil-
deriv filtered deriv ter on the derivative action.

3
3s3 +5s2 +6s+1.6
Step Plant output
PID Controller

Figure 4.8: A S IMULINK block diagram of a classical PID controller as a mask

Note that to have a reverse acting controller, we either require all three constants to be negative,
or just add a negative gain to the output of the controller.

Reasonable S IMULINK controller tuning


constants for this example are: 1.5

1
y & ref

P = 2, I = 1.5, D=2
0.5
or in classical form
0

Kc = 2, i = 1.33, d = 1 4

2
u

which gives a step response as shown op-


posite. 0

0 2 4 6 8 10
time

Because the derivative term in the PID controller acts on the error rather than the output, we see
a large derivative kick in the controller output1 . We can avoid this by using the PID block with
anti-windup, or by modifying the PID block itself. Section 4.4.1 shows how this modification
works.

4.4 E XTENSIONS TO THE PID ALGORITHM

Industrial PID controllers are in fact considerably more complicated than the textbook formula
of Eqn. 4.1 would lead you to believe. Industrial PID controllers have typically between 15 and
25 parameters. The following describes some of the extra functionality one needs in an effective
commercial PID controller.

1 Note that I have re-scaled the axis in the simulation results


142 CHAPTER 4. THE PID CONTROLLER

4.4.1 AVOIDING DERIVATIVE KICK

If we implement the classical PID controller such as Eqn. 4.3 with significant derivative action, the
input will show jump excessively every time either the output or the setpoint changes abruptly.
Under normal operations conditions, the output is unlikely to change rapidly, but during a set-
point change, the setpoint will naturally change abruptly, and this causes a large, though brief,
spike in the derivative of the error. This spike is fed to the derivative part of the PID controller,
and causes unpleasant transients in the manipulated variable. If left unmodified, this may cause
excessive wear in the actuator. Industrial controllers and derivative kick is further covered in
[183, p191].

It is clear that there is a problem with the controller giving a large kick when the setpoint abruptly
changes. This is referred to as derivative kick and is due to the near infinite derivative of the error
when the setpoint instantaneously changes as when in a setpoint change. One way to avoid
problems of this nature is to use the derivative of the measurement y, rather than the derivative
of the error e = y y. If we do this, the derivative kick is eliminated, and the input is much less
excited. Equations of both the classical and the anti-kick PID controller equations are compared
below.
 Z 
1 t d
Normal: Gc () = Kc + d + d (4.15)
i 0 dt
 Z 
1 t dy
Anti-Kick: Gc (, y) = Kc + d + d (4.16)
i 0 dt

The anti-derivative kick controller is sometimes known as a PI-D controller with the dash indicat-
ing that the PI part acts on the error, and the D part acts on the output measurement. In S IMULINK
you can build a PID controller with anti-kick by modifying the standard PID controller block as
shown in Fig. 4.9.

input/output

1 3
1/taui Kc
s (s+4)(s+1)(s+0.4)
Sum1 Sum
Pulse Integrator gain Plant
reset
Generator
s
taud
taud/N.s+1
derivative filter
deriv

Figure 4.9: PID controller with anti-derivative kick. See also Fig. 4.10.

Fig. 4.10 compares the controlled performance for the third order plant and tuning constants
given previously in Fig. 4.4 where the derivative term uses the error (lefthand simulation), with
the modification where the derivative term uses the measured variable (righthand simulation).

Evident from Fig 4.10 is that the PID controller using the measurement rather than the error
behaves better, with much less controller action. Of course, the performance improvement is
only evident when the setpoint is normally stationary, rather than a trajectory following problem.
Stationary setpoints are the norm for industrial applications, but if for example, we subjected the
4.4. EXTENSIONS TO THE PID ALGORITHM 143

Normal PID No derivative Kick


1.5 1.5

output & setpoint


1 1

0.5 0.5

0 0

0.5 0.5
0 10 20 30 40 0 10 20 30 40

40 40

20 20
input

0 0

20 20

40 40
0 10 20 30 40 0 10 20 30 40
time time

Figure 4.10: PID control of a plant given in Fig. 4.9. The derivative kick is evident in the input
(lower trend of the left-hand simulation) of the standard PID controller. We can avoid this kick
by using the derivative of the output rather than the error as seen in the right-hand trend. Note
that the output controlled performance is similar in both cases

closed loop to a sine wave setpoint, then the PID that employed the error in the derivative term
would perform better than the anti-kick version.

A N ELECTROMAGNETIC BALANCE ARM

The electromagnetic balance arm described previously in 1.3.1 is extremely oscillatory as shown
in Fig. 1.6(b). The overshoot is about 75% which corresponds to a damping ratio of 0.1
assuming a proto-type second order process model.

To stabilise a system with poles so close to the imaginary axis requires substantial derivative
action. Without derivative action, the integral component needed to reduce the offset would
cause instability. Unfortunately however the derivative action causes problems with noise and
abrupt setpoint changes as shown in Fig. 4.11(a).

The controller output, u(t) exhibits a kick every time the setpoint is changed. So instead of the
normal PID controller used in Fig. 4.11(a), an anti-kick version is tried in Fig. 4.11(b). Clearly
there are no spikes in the input signal when Eqn. 4.16 is used, and the controlled response is
slightly improved.

Abrupt setpoint changes are not the only thing that can trip up a PID controller. Fig. 4.12 shows
another response from the electromagnetic balance, this time with even more derivative action.
At low levels the balance arm is very oscillatory, although this behaviour tends to disappear at
the higher levels owing to the nonlinear friction effects.
144 CHAPTER 4. THE PID CONTROLLER

No derivative kick
Derivative kick 2200
2500
2000
Flapper angle

2000
1800

1500 1600

1400
1000

1600
3000
1400
2000
input

1200
1000 1000

0 800
0 10 20 30 40 50 60 0 10 20 30 40 50
time [s] time [s]
(a) PID control exhibiting significant derivative kick. (b) PID control with anti-kick.

Figure 4.11: Illustrating the improvement of anti-derivative kick schemes for PID controllers
when applied to the experimental electromagnetic balance.

4000

3500
Setpoint & output

3000

2500

2000

1500
5000

4000
Figure 4.12: Derivative 3000
input

control and noise. Note


the difference in re- 2000
sponse character once
the setpoint is increased 1000
over 3000. This is due 0
to nonlinear friction 0 5 10 15 20 25 30 35
effects. time [s]
4.5. DISCRETE PID CONTROLLERS 145

4.4.2 I NPUT SATURATION AND INTEGRAL WINDUP

Invariably in practical cases the actual manipulated variable value, u, demanded by the PID con-
troller is impossible to implement owing to physical constraints on the system. A control valve for
example cannot shut less than 0% open or open more than 100%. Normally clamps are placed
on the manipulated variable to prevent unrealisable input demands occurring such as

umin < u < umax , or u = MIN(max u,MAX(min u,u))

or in other words, if the input u is larger than the maximum input allowed max_u, it will be
reset to that maximum input, and similarly the input is saturated if it is less than the minimum
allowable limit. In addition to an absolute limit on the position of u, the manipulated variable
can not instantaneously change from one value to another. This can be expressed as a limit on the
derivative of u, such as | du
dt | < cd .

However the PID control algorithm presented so far assumes that the manipulated variable is
unconstrained so when the manipulated variable does saturate, the integral error term continues
to grow without bound. When finally the plant catches up and the error is reduced, the integral
term is still large, and the controller must eat up this error. The result is an overly oscillatory
controlled response.

This is known as integral windup. Historically with the analogue controllers, integral windup
was not much of a problem since the pneumatic controllers had only a limited integral capacity.
However, this limit is effectively infinite in a digital implementation.

There are a number of ways to prevent integral windup and these are discussed in [16, pp1014]
and more recently in [142, p60] and [33]. The easiest way is to check the manipulated variable
position, and ignore the integral term if the manipulated variable is saturated.

An alternative modification is to compute the difference between the desired manipulated vari-
able and the saturated version and to feed this value back to the integrator within the controller.
This is known as anti-windup tracking and is shown in Fig. 4.13(a). When the manipulated input is
not saturated, there is no change to normal PID algorithm. Note that in this modified PID config-
uration we differentiate the measured value (not the error), we approximate the derivative term
as discussed in section 4.2.1, and we can turn off the anti-windup component with the manual
switch.

As an example (adapted from [20, Fig 8.9]), suppose we are to control an integrator plant, G(s) =
1/s, with tight limits on the manipulated variable, |u| < 0.1. Without anti-windup, the controller
output rapidly saturates, and the uncompensated response shown in Fig. 4.13(b) is very oscilla-
tory. However with anti-windup enabled, the controlled response is much improved.

4.5 D ISCRETE PID CONTROLLERS

To implement a PID controller as given in Eqn. 4.1 on a computer, one must first discretise or
approximate the continuous controller equation. Assuming we are operating at a If the error at
time t = kT is k , then the continuous expression

 Z 
1 de(t)
u(t) = Kc e(t) + e(t) dt + d (4.17)
i dt
146 CHAPTER 4. THE PID CONTROLLER

Manual Switch Constant


0

Sum3
1/taui 1/s
Sum2
Integrator
reset
Sum1
1 Kc 1
error u
gain actuator model

taud.s+1
2 Kc*taud
taud/N.s+1
y
Kc*deriv approx Deriv
(a) A PID controller with anti-windup.

Antiwindup demonstration
3
2
output & setpoint

1
0
1
2 Antiwindup off Antiwindup on
3

0.15
0.1
0.05
input

0
0.05
0.1

0 100 200 300 400 500 600 700 800 900 1000
time
(b) Turning on the anti-windup controller at t = 660 dramatically improves the controlled response.

Figure 4.13: The advantages of using anti-windup are evident after t = 660.
4.5. DISCRETE PID CONTROLLERS 147

can be approximated with




k
T X d
uk = Kc k + j + (k k1 ) (4.18)
i j=0 T
| {z }

| {z } differential
integral

The integral in Eqn. 4.17 is approximated in Eqn. 4.18 by the rectangular rule and the derivative
is approximated as a first order difference, although other discretisations are possible. Normally
the sample time, T , used by the controller is much faster than the dominant time constants of the
process so the approximation is satisfactory and the discrete PID controller is, to all intents and
purposes, indistinguishable from a continuous controller.

It is considerably more convenient to consider the discrete PID controller as a rational polynomial
expression in z. Taking z-transforms of the discrete position form of the PID controller, Eqn. 4.18,
we get

T i+1  d
+ z i+2 + + z 1 + 1 E(z) + (1 z 1 )E(z)

U (z) = Kc E(z) + z
i T
| {z }
| {z }
integral differential

This shows that the transfer function of the PID controller is


 
T 1 d 1
GPID (z) = Kc 1 + + (1 z ) (4.19)
i (1 z 1 ) T
 !
Kc T + i T + d i i (T 2d ) z 1 + i d z 2
2
= (4.20)
T i 1 z 1

Eqn. 4.19 is the discrete approximation to Eqn. 4.2 and the approximation is reasonable provided
the sample time, T , is sufficiently short. A block diagram of this discrete approximation is

T 1
Kc b
i 1z 1 + u

d
T 1 z 1

which we could further manipulate this using (block diagram algebra rules) to use only delay
elements to give

z 1

T
+
Kc b
i + b
+ u

d
T
b
+

z 1

There are other alternatives for a discrete PID controller depending on how we approximate the
148 CHAPTER 4. THE PID CONTROLLER

integral part. For example we could use a backward difference,

ukT = u(k1)T + T y(k1)T


| {z } | {z }
old integral add on
T
Gi (z) =
z1

or a forward difference,

ukT = u(k1)T + T ykT


| {z } | {z }
old integral add on

Tz
Gi (z) =
z1

(which is not to be recommended due to stability problems), or the trapezoidal approximation.



ukT = u(k1)T + T /2 ykT + y(k1)T
| {z } | {z }
old integral add on

T (z + 1)
Gi (z) =
2(z 1)

We can insert any one of these alternatives to give the overall discrete z-domain PID controller,
although the trapeziodal scheme
 
T (1 + z 1 ) d 1
GPID (z) = Kc 1 + + (1 z ) (4.21)
2i (1 z 1 ) T
  !
Kc T + 2i T + 2d i + T 2 2i T 4i d z 1 + 2i d z 2
2
= (4.22)
2T i 1 z 1

is the most accurate and therefore the preferred implementation.

A S IMULINK discrete PID controller with sample time T using the trapeziodal approximation of
Eqn. 4.21 is given in Fig. 4.14. Note that now, as opposed to the continuous time implementation
of the PID controller, the derivative and integral gain values are now a function of sample time, T .
While continuous versions of PID controllers exist in S IMULINK, discrete versions simulate much
faster.

1+z 1
1 Kc T/taui/2 1
error 1z1 u
Sum1
gain Discrete Filter1
integral

1z1
taud/T
1
Discrete Filter2
derivative

Figure 4.14: A discrete PID controller in S IMULINK using a trapezoidal approximation for the
integral with sample time T following Eqn. 4.21. This controller block is used in the simulation
presented in Fig. 4.15.
4.5. DISCRETE PID CONTROLLERS 149

4.5.1 D ISCRETISING CONTINUOUS PID CONTROLLERS

The easiest way to generate a discrete PID controller is to simply call the M ATLAB standard PID
pidstd with a trailing argument to indicate that you want a discrete controller. Since the default
discretisation strategy is using the forward Euler, it would be prudent to use explicitly state the
stable backward Euler option for both the integration and differentiation.

Listing 4.2: Constructing a discrete (filtered) PID controller


>> C = pidstd(1,2,3,100,0.1, ...
'IFormula','BackwardEuler','DFormula','BackwardEuler')
Discrete-time PIDF controller in standard form:
4
 
1 Ts z 1
% C(s) = Kp 1 + i
z1
+ d Ts z
d /N+ z1

with Kp = 1, Ti = 2, Td = 3, N = 100, Ts = 0.1

9 >> tf(C)

Transfer function:
24.13 z2 - 47.4 z + 23.31
--------------------------
14 z2 - 1.231 z + 0.2308

Sampling time: 0.1

V ELOCITY FORM OF THE PID CONTROLLER

Eqn. 4.18 is called the position form implementation of the PID controller. An alternative form
is the velocity form which is obtained by subtracting the previous control input uk1 from the
current input uk to get

uk = uk uk1
    
T d 2d d
= Kc 1 + + k 1 + k1 + k2 (4.23)
i T T T

The velocity form in Eqn. 4.23 has three advantages over the position form: (see [195, pp636-637]),
namely it requires no need for initialisation, (the computer does not need to know the current u),
no integral windup problems, and the algorithm offers some protection against computer failure
in that if the computer crashes, the input remains at the previous, presumably reasonable, value.
One drawback, however, is that it should not be used for P or PD controllers since the controller
is unable to maintain the reference value.

Fig. 4.15 shows a S IMULINK block diagram of a pressure control scheme on a headbox of a paper
machine. The problem was that the pressure sensor was very slow, only delivering a new pressure
reading every 5 seconds. The digital PID controller however was running with a frequency of
1 Hz. The control engineer in this application faced with poor closed loop performance, was
concerned that the pressure sensor was too slow and therefore should be changed. In Fig. 4.15,
the plant is a continuous transfer function while the discrete PID controller runs with sample
period of T = 1 second, and the pressure sensor is modelled with a zeroth-order hold with T = 5
seconds. Consequently the simulation is a mixed continuous/discrete example, with more than
one discrete sample time.
150 CHAPTER 4. THE PID CONTROLLER

3.5
error ctrl output
200s2 +30s+1
Signal headbox Scope
Generator Discrete PID

pressure sensor

Figure 4.15: Headbox control with a slow pressure transducer. The discrete PID controller was
given in Fig. 4.14.

Fig. 4.16 shows the controlled results. Note how the pressure signal to the controller lags behind
the true pressure, but the controller still manages to control the plant.

2
y
1.5 y sampled
setpoint
1
y & setpoint

0.5

0.5

1.5
4

0
u

4
0 50 100 150
time [s]

Figure 4.16: Headbox control with a slow pressure transducer measurement sample time of T = 5
while the control sample time is T = 1. Upper: The pressure setpoint (dotted), the actual pressure
and the sampled-and-held pressure fed back to the PID controller. Lower: The controller output.

4.5.2 S IMULATING A PID CONTROLLED RESPONSE IN M ATLAB

We can simulate a PID controlled plant in M ATLAB by writing a small script file that calls a general
PID controller function. The PID controller is written in the discrete velocity form following
Eqn. 4.23 in the M ATLAB function file pidctr.m shown in listing 4.3.

Listing 4.3: A simple continuous-time PID controller


function [u,e] = pidctr(err,u,dt,e,pidt)
4.5. DISCRETE PID CONTROLLERS 151

% [u,e] = pidctr(err,u,dt,e,pidt)
% PID controller
4 % err: = , current error, u = u(t), current manipulated variable
% dt = T sample time, e: row vector of past 3 errors
% pidt = [Kc , 1/I , d ] = tuning constants

k = pidt(1); rs = pidt(2); td2=pidt(3)/dt;


9 e = [err,e(1:2)]; % error shift register
du = k*[1+rs*dt+td2,-1-2*td2,td2]*e';
u = du + u; % Update control value unew = uold + u
return

This simple PID controller function is a very naive implementation of a PID controller without any
of the necessary modifications common in robust commercial industrial controllers as described
in section 4.4.

The plant to be controlled for this simulation is


1.2q 1
Gp (q 1 ) = q d (4.24)
1 0.25q 1 0.5q 2
where the sample time is T = 2 seconds and the dead time is d = 3 sample time units, and the
setpoint is a long period square wave. For this simulation, we will try out the PID controller with
tuning constants of Kc = 0.3, 1/i = 0.2 and d = 0.1. How I arrived at these tuning constants is
discussed later in 4.6.

The M ATLAB simulation using the PID function from Listing 4.3 is given by the following script
file:

a=[0.25,0.5];b=1.2;theta=[a b]'; dead = 3; % Plant G(q), Eqn. 4.24.

3 dt = 2.0; t = dt*[0:100]'; yspt = square(t/25); % time vector & setpoint


y = zeros(size(yspt));u=zeros(size(y)); % initialise y(t) & u(t)
pid = [0.3, 0.2, 0.1]; % PID tuning constants: Kc = 0.3, 1/I = 0.2, d = 0.1
e = zeros(1,3); % initialise error integral

8 for i=3+dead:length(y)
X = [y(i-1), y(i-2), u(i-1-dead)]; % collect i/o
y(i) = X*theta; % system prediction
err = yspt(i) - y(i); % current error
[u(i),e] = pidctr(err,u(i-1),dt,e,pid); % PID controller from Listing 4.3.
13 end % for
plot(t,yspt,'--',t,[y,u]) % Plot results in Fig. 4.17.

Figure 4.17 shows the controlled response of this simulation. Note how I have plotted the in-
put (lower plot of Fig. 4.17) as a series of horizontal lines that show that the input is actually a
piecewise zeroth-order hold for this discrete case using the stairs function.

4.5.3 C ONTROLLER PERFORMANCE AS A FUNCTION OF SAMPLE TIME

Given that the discrete PID controller is an approximation to the continuous controller, we must
expect a deterioration in performance with increasing sample time. Our motivation to use coarse
sampling times is to reduce the computational load. Fig. 4.18 compares the controlled response
of the continuous plant,
s+3
Gp (s) =
(s + 4)( 2 s2 + 2 s + 1)
152 CHAPTER 4. THE PID CONTROLLER

PID control
2

y & setpoint
0

2
1

0.5
Input

0.5
Figure 4.17: The output (up-
per), and discretised input
1
(lower) of a PID controlled 0 50 100 150 200
process. Time

Ts = 6 Ts = 4 Ts = 2 Ts = 1

1 1 1 1
y & setpoint

0.5 0.5 0.5 0.5


discrete
continuous PID
0 PID 0 0 0

2 2 2 2

1.5 1.5 1.5 1.5


input

1 1 1 1

0.5 0.5 0.5 0.5

0 0 0 0
0 20 40 60 0 20 40 60 0 20 40 60 0 20 40 60

Figure 4.18: The effect of varying sampling time, T , when using a discrete PID controller com-
pared to a continuous PID controller. As T 0, the discrete PID controller converges to the
continuous PID controller.

with = 4, = 0.4 given the same continuous controller settings at different sampling rates.
Note that the reset and derivative controller settings for the discrete controller are functions of
time, and must be adjusted accordingly. Fig. 4.18 shows that the controller performance improves
as the sampling time is decreased and converges to the continuous case. However be warned,
if the sampling time is too small, the discrete PID controller is then suspectable to numerical
problems.
4.6. PID TUNING METHODS 153

4.6 PID TUNING METHODS

Tuning PID controllers is generally considered an art and is an active research topic both in
academia and in industry. Typical chemical processing plants have hundreds or perhaps thou-
sands of control loops of which the majority of these loops are non-critical and are of the PID
type and all these loops require tuning. Arzen [9] asserts that it is a well known fact that many
control loops in (the) process industry are badly tuned, or run in manual mode. Supporting this
claim, here is a summary of the rather surprising results that Ender, [65], found after investigating
thousands of control loops over hundreds of plants2 ;

More than 30% of the controllers are operating in manual.

More than 60% of all installed loops produce less variance in manual than automatic.

The average loop costs $40,000 to design, instrument and install.

Note however that it is not only industry that seems unable to tune PID controllers since many
publications in the academic world also give mis-tuned PID controllers. This is most common
when comparing the PID with some other sort of more sophisticated (and therefore hopefully
better performing), controller. So it appears that it would be worthwhile to look more closely at
the tuning of PID regulators.

There are two possibilities that we face when tuning PID controllers. One is that we have a model
of the plant to be controlled, perhaps as a transfer function, so then we need to establish a suitable
PID controller such that when combined with the plant, we obtain an acceptable response. The
second case is where we dont even have a model of the plant to be controlled, so our task is also
to identify (implicitly or explicitly) this plant model as well.

Tuning PID controllers can be performed in the time domain or in the frequency domain with the
controller either operating (closed loop), or disconnected (open loop), and the tuning parameter
calculations can be performed either online or offline. The online tuning technique is the central
component of automated self tuners, discussed more in chapter 7. Section 4.6.1 considers the two
classical time domain tuning methods, one closed loop, the other open loop.

4.6.1 O PEN LOOP TUNING METHODS

Open loop tuning methods are where the feedback controller is disconnected and the experi-
menter excites the plant and measures the response. They key point here is that since the con-
troller is now disconnected the plant is clearly now no longer strictly under control. If the loop is
critical, then this test could be hazardous. Indeed if the process is open-loop unstable, then you
will be in trouble before you begin. Notwithstanding for many process control applications, open
loop type experiments are usually quick to perform, and deliver informative results.

To obtain any information about a dynamic process, one must excite it in some manner. If the
system is steady at setpoint, and remains so, then you have no information about how the process
behaves. (However you do have good control so why not quit while you are ahead?) The type
of excitation is again a matter of choice. For the time domain analysis, there are two common
types of excitation signal; the step change, and the impulse test, and for more sophisticated
analysis, one can try a random input test. Each of the three basic alternatives has advantages and
disadvantages associated with them, and the choice is a matter for the practitioner.
2 mainly in the US
154 CHAPTER 4. THE PID CONTROLLER

Step change The step change method is where the experimenter abruptly changes the input to
the process. For example, if you wanted to tune a level control of a buffer tank, you could
sharply increase (or decrease) the flow into the tank. The controlled variable then slowly
rises (or falls) to the new operating level. When I want a quick feel for a new process, I
like to perform a step test and this quickly gives me a graphical indication of the degree of
damping, overshoot, rise time and time constants better than any other technique.
Impulse test The impulse test method is where the input signal is abruptly changed to a new
value, then immediately equally abruptly changed back to the old value. Essentially you
are trying to physically alter the input such that it approximates a Dirac delta function.
Technically both types of inputs are impossible to perform perfectly, although the step test
is probably easier to approximate experimentally.
The impulse test has some advantages over the step test. First, since the input after the ex-
periment is the same as before the experiment, the process should return to the same process
value. This means that the time spent producing off-specification (off-spec) product is min-
imised. If the process does not return to the same operating point, then this indicates that
the process probably contains an integrator. Secondly the impulse test (if done perfectly)
contains a wider range of excitation frequencies than the step test. An excitation signal with
a wide frequency range gives more information about the process. However the impulse
test requires slightly more complicated analysis.
Random input The random input technique assumes that the input is a random variable approx-
imating white noise. Pure white noise has a wide (theoretically infinite) frequency range,
and can therefore excite the process over a similarly wide frequency range. The step test,
even a perfect step test, has a limited frequency range. The subsequent analysis of this type
of data is now much more tedious, though not really any more difficult, but it does require
a data logger (rather than just a chart recorder) and a computer with simple regression
software. Building up on this type of process identification where the input is assumed,
within reason, arbitrary, are methods referred to as Time Series Analysis (TSA), or spectral
analysis; both of which are dealt with in more detail in chapter 6.

O PEN - LOOP OR PROCESS REACTION CURVE TUNING METHODS

There are various tuning strategies based on an open-loop step response. While they all follow
the same basic idea, they differ in slightly in how they extract the model parameters from the
recorded response, and also differ slightly as to relate appropriate tuning constants to the model
parameters. This section describes three alternatives, the classic Ziegler-Nichols open loop test,
the Cohen-Coon test, and the Astrom-Hagglund suggestion.

The classic way of open loop time domain tuning was first published in the early 40s by Ziegler
and Nichols3 , and is further described in [154, pp596597] and in [183, chap 13]. Their scheme re-
quires you to apply a unit step to the open-loop process and record the output. From the response,
you graphically estimate the two parameters T and L as shown in Fig. 4.19. Naturally if your
response is not sigmoidal or S shaped such as that sketched in Fig. 4.19 and exhibits overshoot,
or an integrator, then this tuning method is not applicable.

This method implicitly assumes the plant can be adequately approximated by a first order transfer
function with time delay,
Kes
Gp (4.25)
s + 1
where L is approximately the dead time , and T is the open loop process time constant . Once
you have recorded the openloop input/output data, and subsequently measured the times T and
3 Ziegler & Nichols actually published two methods, one open loop, and one closed loop, [209]. However it is only the

second closed loop method that is generally remembered today as the ZieglerNichols tuning method.
4.6. PID TUNING METHODS 155

Input Output
Plant

tangent at inflection point

K
b

L T time

Figure 4.19: The parameters T and L to be graphically estimated for the openloop tuning method
relations given in Table 4.2.

L, the PID tuning parameters can be obtained directly from Table 4.2.

A similar open loop step tuning strategy due to Cohen and Coon published in the early 1950s is
where you record the time taken to reach 50% of the final output value, t2 , and the time taken to
reach 63% of the final value, t3 . You then calculate the effective deadtime with
t2 ln(2)t3
=
1 ln(2)
and time constant,
= t3 t1
The open loop gain can be calculated by dividing the final change in output over the change in
the input step.

Once again, now that you have a model of the plant to be controlled in the form of Eqn. 4.25,
you can use one the alternative heuristics given in Table 4.2. The recommended range of values
for the deadtime ratio for the Cohen-Coon values is 0.1 < / < 1. Also listed in Table 4.2 are
the empirical suggestions from [17] known as AMIGO, or approximate M -constrained integral
gain optimisation. These values have the same form as the Cohen-Coon suggestions but perform
slightly better.

Fig. 4.20 illustrates the approximate first-order plus deadtime model fitted to a higher-order over-
damped process using the two points at 50% and 63%. The subsequent controlled response using
the values derived from in Table 4.2 is given in Fig. 4.21.

Conventional thought now considers that both the Zeigler-Nichols scheme in Table 4.2 and the
Cohen-Coon scheme gives controller constants that are too oscillatory and hence other modified
tuning parameters exist, [182, p329]. Problem 4.1 demonstrates this tuning method.

Problem 4.1 Suppose you have a process that can be described by the transfer function
K
Gp =
(3s + 1)(6s + 1)(0.2s + 1)
Evaluate the time domain response to a unit step change in input and graphically estimate the
parameters L and T . Design a PI and PID controller for this process using Table 4.2.
156 CHAPTER 4. THE PID CONTROLLER

Table 4.2: The PID tuning parameters as a function of the openloop model parameters K, and
from Eqn. 4.25 as derived by Ziegler-Nichols (open loop method), Cohen and Coon, or alterna-
tively the AMIGO rules from [17].

Controller Kc i d

P
Ziegler-Nichols K
(Open loop) 0.9
PI
K 0.3
1.2
PID 2 0.5
K
 
1
P 1+
K 3
Cohen-Coon  
1 30 + 3/
PI 0.9 +
K 12 9 + 20/
 
1 4 32 + 6/ 4
PID +
K 3 4 13 + 8/ 11 + 2/

1   0.4 + 0.8 0.5


AMIGO PID 0.2 + 0.45
K + 0.1 0.3 +

1.5

fitted model
1
y

Figure 4.20: Fitting a first-order model with


actual
deadtime using the Cohen-Coon scheme.
Note how the fitted model is a reason- 0.5
able approximation to the actual response
t2 t3
just using the two data points and gain.
See Fig. 4.21 for the subsequent controlled 0
0 5 10 15 20 25 30 35
response. time

C ONTROLLER SETTINGS BASED ON THE OPEN LOOP MODEL

If we have gone to all the trouble of estimating a model of the process, then we could in principle
use this model for controller design in a more formal method than just rely on the suggestions
given in Table 4.2. This is the thinking behind the Internal Model Control or IMC controller
design strategy. The IMC controller is a very general controller, but if we restrict our attention
to just controllers of the PID form, we can derive simple relations between the model parameters
and appropriate controller settings.

The nice feature of the IMC strategy is that it provides the scope to adjust the tuning with a single
parameter, the desired closed loop time constant, c , something that is missing from the strategies
given previously in Table 4.2. A suitable starting guess for the desired closed loop time constant
is to set it equal to the dominant open loop time constant.

Table 4.3 gives the PID controller settings based on various common process models. For a more
complete table containing a larger selection of transfer functions, consult [183, p308].
4.6. PID TUNING METHODS 157

Ponly PIcontrol PID control


y & setpoint 2 2 2

0 0 0

2 2 2

5 5 5

0 0 0
u

5 5 5
0 100 200 0 100 200 0 100 200
time time time

Figure 4.21: The closed loop response for a P, PI and PID controlled system using the Cohen-Coon
strategy from Fig. 4.20.

Table 4.3: PID controller settings based on IMC for a small selection of common plants where the
control engineer gets to chose a desired closed loop time constant, c .

Plant PID constants


Kc K i d

K/( s + 1)
c
K 1 + 2
1 + 2
(1 s + 1)(2 s + 1) c
2
K/s 2c
c
K 2c + 2c
2c +
s( s + 1) c2 2c +

Kes
c +
s + 1 + /2
+ /2
c + /2 2 +

An simplification of the this IMC idea in an effort to make the tuning as effortless as possible is
given in [190].

Perhaps the easiest way to tune a plant when the transfer function is known is to use the M ATLAB
function pidtune, or the GUI, pidtool as shown in Fig. 4.22.

4.6.2 C LOSED LOOP TUNING METHODS

The main disadvantage of the open loop tuning methods is that it is performed with the controller
switched to manual, i.e. leaving the output uncontrolled in open loop. This is often unreason-
able for systems that are openloop unstable, and impractical for plants where there is a large
invested interest, and the operating engineers are nervous. The Ziegler-Nichols continuous cy-
cling method described next is a well-known closed loop tuning strategy used to address this
problem, although a more recent single response strategy given later in 4.6.3 is faster, safer, and
158 CHAPTER 4. THE PID CONTROLLER

Figure 4.22: PID tuning of an arbitrary transfer function using the M ATLAB GUI.

easier to use.

Z IEGLER -N ICHOLS CONTINUOUS CYCLING METHOD

The Ziegler-Nichols continuous cycling method is one of the best known closed loop tuning
strategies. The controller is left on automatic, but the reset and derivative components are turned
off. The controller gain is then gradually increased (or decreased) until the process output contin-
uously cycles after a small step change or disturbance. At this point, the controller gain you have
selected is the ultimate gain, Ku , and the observed period of oscillation is the ultimate period, Pu .

Ziegler and Nichols originally suggested in 1942 PID tuning constants as a function of the ultimate
gain and ultimate period as shown in the first three rows of Table 4.4. While these values give
near-optimum responses fort load changes, practical experience and theoretical considerations
(i.e. [16, 31]) have shown that these tuning values tend to give responses that are too oscillatory
for step-point changes due to the small phase margin. For this reason, various people have sub-
sequently modified the heuristics slightly as listed in the remaining rows of Table 4.4 which is
expanded from that given in [182, p318].

Experience has shown that the Chien-Hrones-Reswick values give an improved response on the
original Ziegler-Nichols, but the Astrom-Hagglund tend, like the ZN, to be overly oscillatory.
While the Tyreus-Luyben values deliver very sluggish responses, they exhibit very little overshoot
and are favoured by process engineers for that reason.

Algorithm 4.1 summarises the ZN ultimate oscillation tuning procedure.

Algorithm 4.1 Ziegler-Nichols closed loop PID tuning


4.6. PID TUNING METHODS 159

Table 4.4: Various alternative Ziegler-Nichols type PID tuning rules as a function of the ultimate
gain, Ku , and ultimate period, Pu .
Response type PID constants
Kc i d
P 0.5Ku
Ziegler-Nichols PI 0.45Ku Pu /1.2
PID 0.6Ku Pu /2 Pu /8
No overshoot 0.2Ku Pu /2 Pu /2
Modified ZN Some overshoot 0.33Ku Pu /2 Pu /3
PI 0.31Ku 2.2Pu
Tyreus-Luyben PID 0.45Ku 2.2Pu Pu /6.3
Chien-Hrones-Reswick PI 0.47Ku 1
Astrom-Hagglund PI 0.32Ku 0.94
Specified phase-margin, m PID Ku cos(m ) f d Eqn. 4.58

To tune a PID controller using the closed-loop Ziegler-Nichols method, do the following:

1. Connect a proportional controller to the plant to be controlled. I.e. turn the controller on
automatic, but turn off the derivative and integral action. (If your controller uses integral
time, you will need to set i to the maximum allowable value.)
2. Choose a trial sensible controller gain, Kc to start.
3. Disturb the process slightly and record the output.
4. If the response is unstable, decrease Kc and go back to step 3, otherwise increase Kc and
repeat step 3 until the output response is a steady sinusoidal oscillation. Once a gain Kc has
been found to give a stable oscillation proceed to step 5.
5. Record the current gain, and measure the period of oscillation in time units (say seconds).
These are the ultimate gain, Ku and corresponding ultimate period, Pu .
6. Use Table 4.4 to establish the P, PI, or PID tuning constants.
7. Test the closed loop response with the new PID values. If the response is not satisfactory,
further manual fine-tuning may be necessary.

This method has proved so popular, that automatic tuning procedures have been developed that
are based around this theory as detailed in chapter 7. Despite the fact that this closed loop test
is difficult to apply experimentally, gives only marginal tuning results in many cases, it is widely
used and very well known. Much of the academic control literature uses the Z-N tuning method
as a basis on which to compare more sophisticated schemes but often conveniently forgetting
in the process that the Z-N scheme was really developed for load disturbances as opposed to
setpoint changes. Finally many practicing instrument engineers (who are the ones actually tuning
the plant) know only one formal tuning method this one.

Consequently it is interesting to read the following quote from a review of a textbook in Process
control:4 . . . The inclusion of tuning methods based on the control loop cycling (Ziegler-Nichols
method) without a severe health warning to the user reveals a lack of control room experience on
behalf of the author.
4 Review of A Real-Time Approach to Process Control by Svrcek, Mahoney & Young, reviewed by Patrick Thorpe in The

Chemical Engineer, January 2002, p30.


160 CHAPTER 4. THE PID CONTROLLER

Z IEGLER -N ICHOLS CONTINUOUS CYCLING EXAMPLE

Finding the ultimate gain and frequency in practice requires a tedious trial-and-error approach.
If however, we already have the model of the process, say in the form of a transfer function, then
establishing the critical frequency analytically is much easier, although we may still need to solve
a nonlinear algebraic equation.

Suppose we have identified a model of our plant as

1
G(s) = e3s (4.26)
6s2 + 7s + 1
and we want to control this plant using a PID controller. To use Ziegler-Nichols settings we need
to establish the ultimate frequency, u , where the angle of G(s = iu ) = radians, or solve the
nonlinear expression
6 G(iu ) = (4.27)
for u . In the specific case given in Eqn. 4.26, we have

6 e3iu 6 2
16u +7iu =
 
7u
3u tan1 = (4.28)
1 6u2

which is a non-trivial function of ultimate frequency, u . However it is easy to graph this relation
as a function of u and look for zero crossing such as shown in Fig. 4.23, or use a numerical
technique such as Newton-Rhapson to establish that u 0.4839 rad/s implying an ultimate
period of Pu = 2/u = 12.98 seconds per cycle.

2
3 watan2(7 w,16 w )+

1
f()

1
Figure 4.23: Solving F () = 0 for the
2
ultimate frequency. In this case the ul-
0 0.2 0.4 0.6 0.8 1
timate frequency is u 0.48 rad/s.

A quick way to numerically solve Eqn. 4.28 is to first define an anonymous function and then to
use fzero to find the root.

 
1 >> F = @(w) -3*w - atan2(7*w,1-6*w.2)+pi; % F () = 3 tan1 7
16 2
+
>> ezplot(F,[0 1]) % See Fig. 4.23.
>> wu = fzero(F,0.1) % Solve F () = 0 for
wu =
0.4839

Once we have found the critical frequency, u , we can establish the magnitude at this frequency
by substituting s = iu into the process transfer function, Eqn. 4.26,

|G(iu )| = |G(0.48i)| 0.2931

which gives a critical gain, Ku = 1/0.2931 = 3.412. An easy way to do this calculation in M ATLAB
is to simply use bode at a single frequency, as in
4.6. PID TUNING METHODS 161

>> [M,ph] = bode(G,wu) % Compute G(s = ju ), should obtain = 180 , and Ku .


M =
0.2931
ph =
10 -180.0000

Now that we have both the ultimate gain and frequency we can use the classic Zielger-Nichols
rules in Table 4.4 to obtain our tuning constants and simulate the controlled process as shown in
Fig. 4.24.

P control PI control PID control


1.5 1.5 1.5
output & setpoint

1 1 1

0.5 0.5 0.5

0 0 0

2 2 2
input

1 1 1

0 0 0
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
time time time

Figure 4.24: The step and load (at t = 100) responses of a P (left) and PI (middle) and PID (right)
controlled process using the closed loop Ziegler-Nichols suggestions from Table 4.4.

The tuned response to both a setpoint change and a load disturbance at t = 100 of all three
candidates shown in Fig. 4.24 is reasonable, but as expected the P response exhibits offset, and
the PI is sluggish. We would expect a good response from the PID controller because the actual
plant model structure (second order plus deadtime), is similar to the assumed model structure by
Ziegler and Nichols, which is first order plus deadtime. However while it could be argued that
the step response is still too oscillatory, the response to the load disturbance is not too bad.

In the very common case of a first order plus deadtime structure, Eqn. 4.25, we can find the
ultimate frequency by solving the nonlinear equation

u tan1 ( u ) + = 0 (4.29)

for u , and then calculate the ultimate gain from a direct substitution

1 1 + 22
Ku = = (4.30)
|G(iu )| K

The tedious part of the above procedure is generating the nonlinear equation in u for the spe-
cific plant structure of interest. We can however automate this equation generation for standard
polynomial transfer functions with deadtime as shown in the listing below.

[num,den] = tfdata(G,'v') % Or use: cell2mat(G.num)


162 CHAPTER 4. THE PID CONTROLLER

iodelay = G.iodelay;

% Construct the equation 6 G(i) = and solve for


15 F = @(w) angle(polyval(num,1i*w))-iodelay*w-angle(polyval(den,1i*w))+pi;
wc = fzero(F,0) % Now solve for c

Rather than solve for the critical frequency algebraically, we could alternatively rely on the M AT-
LAB margin routine which tries to extract the gain and phase margins and associated frequencies
for arbitrary transfer functions. Listing 4.4 shows how we can tune a PID controller for an ar-
bitrary transfer function, although note that because this strategy attempts to solve a nonlinear
expression for the critival frequency using a numerical iterative scheme, this routine is not infal-
lible.

Listing 4.4: Ziegler-Nichols PID tuning rules for an arbitrary transfer function
G = tf(1,conv([6 1],[1 1]),'iodelay',5); % Plant of interest G = 1
(6s+1)(s+1)
e5s

[Gm,Pm,Wcg,Wcp] = margin(G); % Establish critical points


20 Ku = Gm; Pu = 2*pi/Wcg; % Critical gain, Ku & period, Pu

Kc = 0.6*Ku; taui = Pu/2; taud = Pu/8; % PID tuning rules (Ziegler-Nichols)


Gc = tf(Kc*[taui*taud taui 1],[taui 0]); % PID controller Kc (i d s2 + i s + 1)/(i s)

Note that while the use of the ultimate oscillation Ziegler-Nichols tuning strategy is generally
discouraged by practicitioners, it is the oscillation at near instability which is the chief concern,
not the general idea of relating the tuning constants to the ultimate gain and frequency. If for
example, we already have a plant model in the form of a transfer function perhaps derived from
first principles, or system identification techniques, then we can by-pass the potentially hazardous
oscillation step, and compute directly the tuning constants as a function of ultimate gain and
frequency.

4.6.3 C LOSED LOOP SINGLE - TEST TUNING METHODS

Despite the fact that Ziegler-Nichols continuous cycling tuning method is performed in closed
loop, the experiment is both tedious and dangerous. The Yuwana-Seborg tuning method de-
scribed here, [208], retains the advantages of the ZN tuning method, but avoids many of the the
disadvantages. Given the attractions of this closed-loop reaction curve tuning methodology, there
have been subsequently many extensions proposed, some of which are summarised in [50, 99].
The following development is based on the modifications of [49] with the corrections noted by
[197].

The Yuwana-Seborg (YS) tuning technique is based on the assumption that the plant transfer
function, while unknown, can be approximated by the first order plus deadtime structure,

Km em s
Gm (s) = (4.31)
m s + 1
with three plant parameters, process gain Km , time constant, m , and deadtime m .

Surrounding this process with a trial, but known, proportional-only controller Kc , gives a closed
loop transfer function between reference r(t) and output y(t) as

Y (s) Kc Km em s
= (4.32)
R(s) 1 + s + Kc Km em s
4.6. PID TUNING METHODS 163

A Pade approximation can be used to expand the non-polynomial term in the denominator, al-
though various people have subsequently modified this aspect of the method such as using a
different deadtime polynomial approximation. Using some approximation for the deadtime, we
can approximate the closed loop response of Eqn. 4.32 with the more convenient

Kes
Gcl (4.33)
2 s2 + 2 s + 1
and we can extract the model parameters, K, , and in Eqn. 4.33 from a single closed loop
control step test.

Suppose we have a trial proportional-only feedback controlled response with a known controller
gain Kc . It does not really matter what this gain is so long as the response oscillates sufficiently
as shown in Fig. 4.25.

If we step change the reference variable setpoint from r0 to r1 , we are likely to see an under-
damped response such as shown in Fig. 4.25. From this response we measure the initial output y0 ,
the first peak yp1 , the first trough, ym1 and the second peak yp2 , and associated times. Under these
conditions we expect to see some offset given the proportional-only controlled loop, and that the
controller gain should be sufficiently high such that the response exhibits some underdamped
(oscillatory) behaviour.

peak #1
yp1
peak #2
setpoint
yp2
r1
offset
y
setpoint change


ym
trough #1 typical output response

period, P
r0 y0

time

Figure 4.25: A typical closed loop response of a stable system to a P-controller

The final value of the output, y , is approximately


2
yp2 yp1 ym
y = (4.34)
yp1 + yp2 2ym

or alternatively, if the experimental test is of sufficient duration, y could simply be just the last
value of y collected. Now we are ready to graphically fit the parameters to the closed loop model,
Eqn. 4.33.

The closed loop gain K is given by


y
K= (4.35)
r1 r0
and the closed loop overshoot is given by
 
1 yp1 y y ym yp2 y
H= + + . (4.36)
3 y yp1 y y ym
164 CHAPTER 4. THE PID CONTROLLER

The closed loop deadtime is


= 2tp1 tm , (4.37)
the closed loop shape factor
ln(H)
= q , (4.38)
2 + ln2 (H)
and the closed loop time constant
p
(tm tp1 ) 1 2
= . (4.39)

Now that we have fitted a closed loop model, Eqn. 4.33, we can compute the ultimate gain, Ku ,
and ultimate frequency, u , by solving the nonlinear expression
 
2 u
u tan1 = (4.40)
1 2 u2

for u , perhaps using an iterative scheme starting with u = 0. It follows then that the ultimate
gain is  
1
Ku = Kc 1 + (4.41)
|Gcl (iu )|
where the magnitude
K
|Gcl (iu )| = p (4.42)
(1 u )2 + (2 u )2
2 2

Alternatively of course we could try to use the M ATLAB routine margin to extract Ku and u .

At this point, given the key design parameters Ku and u , we can tune PID controllers using the
ZN scheme directly following Table 4.4. Note that we now have achieved our aim of tuning a PID
controller using the Ziegler-Nichols rules without undertaking a repetitive, tedious, and possibly
dangerous, experiment searching for the ultimate oscillations.

Alternatively we can derive the parameters for the assumed open loop plant model, Eqn. 4.31, as

|y y0 |
Km = (4.43)
Kc (|r1 r0 | |y y0 |)
1 p 2 2
m = Ku Km 1 (4.44)
u
tan1 (m u )
m = (4.45)
u
and then subsequently we can tune it either using the ZN strategy given in Listing 4.4, or use the
internal model controller strategy from Table 4.3, or perhaps using ITAE optimal tuning constants.
The optimal ITAE tuning parameters for P, PI and PID controllers are derived from
 B  D  F
A m 1 1 m m
Kc = , = , d = E m (4.46)
Km m i Cm m m

where the controller structure-specific constants A, B, C, D, E, F are given in Table 4.5. We are
now in a position to replace our trial Kc with the hopefully better controller tuning constants.

Note that the change in setpoint, |r1 r0 | must be larger than the actual response |y y0 | owing
to the offset exhibited by a proportional only controller. If the system does not respond in this
manner, then a negative process gain is implied, which in turn implies nonsensical imaginary
time constants. Another often overlooked point is that if the controlled process response does not
oscillate, then the trial gain Kc is simply not high enough. This gain should be increased, and the
4.6. PID TUNING METHODS 165

Table 4.5: The ITAE optimal tuning constants A through E in Eqn. 4.46 for P, PI, and PID con-
trollers.
mode A B C D E F
P 0.49 1.084
PI 0.859 0.977 1.484 0.680
PID 1.357 0.947 1.176 0.738 0.381 0.99

test repeated. If the process never oscillates, no matter how high the gain is, then one can use an
infinite gain controller at least in theory.

Algorithm 4.2 summarises this single-test closed loop tuning strategy.

Algorithm 4.2 Yuwana-Seborg closed loop PID tuning

Tuning a PID controller using the closed-loop Yuwana-Seborg method is very similar to Algo-
rithm 4.1, but avoids the trial and error search for the ultimate gain and period.

1. Connect a proportional only controller to the plant.


2. Choose a trial sensible controller gain, Kc and introduce a set point change and record the
output.
3. Check that the output is underdamped and shaped similar to Fig. 4.25, and if not increase
the trial controller gain, Kc , and repeat step 2.
4. From the recorded underdamped response, measure the peaks and troughs, and note the
demanded setpoint change. You may find Listing 4.5 helpful here.
5. Compute the parameters of the closed loop plant and solve for the ultimate frequency,
Eqn. 4.40 and gain, Eqn. 4.41.
6. Optionally compute the open loop plant model using Eqn. 4.43Eqn. 4.45.
7. Compute the P, PI or PID tuning constants using Table 4.5 or perhaps using the model and
the ZN strategy in Listing 4.4.
8. Test the closed loop response with the new PID values. If the response is not satisfactory,
further manual fine-tuning may be necessary.

A M ATLAB implementation to compute the tuning constants given response data is given in list-
ings starting on page 167.

A consequence of the assumption that the plant is adequately modelled using first-order dy-
namics with deadtime, the Yuwana-Seborg tuning scheme is not suitable for highly oscillatory
processes, nor plants with inverse response behaviour as illustrated in section 4.8.1. Furthermore,
a consequence of the low order Pade approximation is that the strategy fails for processes with a
large relative deadtime. Ways to use this tuning scheme in the presence of excessive measurement
noise is described in section 5.2.1.

An example of a closed loop response tuner

Suppose we are to tune a PID controller for an inverse response process,


s + 1
G(s) = (4.47)
(s + 1)2 (2s + 1)
166 CHAPTER 4. THE PID CONTROLLER

Such processes with right-hand plane zeros are challenging because of the potential instability
owing to the initial response heading off into the opposite direction as described further in 4.8.1.

The response to a unit setpoint change is given in the top figure of Fig. 4.26. By recording the
values of the peaks and troughs, we can estimate the parameters of an approximate model of the
closed loop response, Eqn. 4.33. A comparison of this model with the actual data is given in the
middle plot of Fig. 4.26. It is not a perfect fit because the underlying plant is not a pure first order
with deadtime, and because Eqn. 4.33 is only approximate anyway.

Trial closed loop step test, K = 1


c

0.8

0.6

y
0.4

0.2

0 t t t
p1 m1 p2

Closed loop model


1

0.8

0.6
y &y

0.4
m

0.2

0 Actual
Fitted
0.2
Open loop model

Figure 4.26: A Yuwana-Seborg closed loop 0.8

step test for plant Eqn. 4.47. The top plot 0.6
y &y

shows the response given a unit change in 0.4


m

setpoint with the key points marked. The 0.2


middle plot compares the estimated with Actual
0
the actual closed loop data, while the bot- fitted
tom plot compares the estimated openloop 0.2
0 5 10 15 20
plant model with the true plant. time [s]

Given the closed loop model, we can now extract the open loop model of the plant which is
compared with the true plant in the bottom trend of Fig. 4.26. Note how the first-order plus
deadtime model, specifically in this case
1.003
G = e2.24s , (4.48)
3.128s + 1
approximates, within reason, the higher order true plant dynamics.

Finally now that we have a plant model, G, we can either use the ITAE optimal or the IMC tuning
scheme as shown in Fig. 4.27.

As evident in Fig. 4.27, the ITAE responses are overly oscillatory which tends to be a common
failing of PID tuning schemes that are designed to satisfy a performance objective.

In the IMC case, the desired closed loop time constant was set equal to the sum of the deadtime
and the open loop time constant, c = m +m , which as it turns out is perhaps overly conservative
4.6. PID TUNING METHODS 167

ITAE PI control ITAE PID control


1.5 1.5

1 1
y&r
0.5 0.5

0 0
2 2

0 0
u

2 2

IMC PI control IMC PID control


1.5 1.5

1 1
y&r

0.5 0.5

0 0
2 2

0 0
u

2 2
0 10 20 30 40 50 0 10 20 30 40 50

Figure 4.27: The PI (left) and PID (right) closed loop responses using the ITAE (upper) and IMC
(lower) tuning schemes based on the identified model, Eqn. 4.48, but applied to the actual plant,
Eqn. 4.47.

in this case. Better performance would be achieved by reducing c .

A UTOMATING THE Y UWANA -S EBORG TUNER

Because this is such a useful strategy for PID tuning, it is helpful to have a semi-automated pro-
cedure. The first step once you have collected the closed loop data with a trial gain is to identify
the values and timing of the peaks and troughs, yp1 , ym1 and yp2 . Listing 4.5 is a simple routine
that attempts to automatically extract these values. As with all small programs of this nature, the
identification part can fail, particularly given excessive noise or unusual response curves.

Listing 4.5: Identifies the characteristic points for the Yuwana-Seborg PID tuner from a trial closed
loop response
function [yp1,tp1,ym1,tm1,yp2,tp2,yss_est] = ys_ident(t,Y);
% Identify the peaks & troughs from an underdamped closed loop test
3 % Used for Yuwana-Seborg PID tuning

y0 = Y(1); n = length(Y); % start point & total # of points


if n < 20
fprintf('Probably need more points\n');
8 end % if
Y = Y*((Y(n) > y0)-0.5)*2; % Establish direction, 1 if positive)
yss_est = mean(Y(n:-1:n-round(n/50))); % average pt at the end (robust)

[yp1,idx] = max(Y); tp1 = t(idx); % 1st maximum point & time at occurrence
13 Y = Y(idx:n); t = t(idx:n); % chop off to keep searching
[ym1,idx] = min(Y); tm1 = t(idx); % 1st minimum point & time
168 CHAPTER 4. THE PID CONTROLLER

Y(1:idx) = []; t(1:idx)=[];


[yp2,idx] = max(Y); tp2 = t(idx); % 2nd maximum point & time at occurrence

18 return % end ys_ident.m

Using this peak and trough data, we can compute the parameters in the closed loop model fol-
lowing Listing 4.6.

Listing 4.6: Compute the closed loop model from peak and trough data
yinf = (yp1*yp2 - ym12)/(yp1+yp2-2*ym1); % Steady-state output, y
2 K = yinf/A; % Closed loop gain, K, Eqn. 4.35 & Overshoot, H
H = 1/3*((yp1-yinf)/yinf+(yinf-ym1)/(yp1-yinf)+(yp2-yinf)/(yinf-ym1));

d = 2*tp1 - tm1; % Deadtime, , Eqn. 4.37


zeta = -log(H)/sqrt(pi2+log(H)2); % shape factor, , Eqn. 4.38
7 tau = (tm1-tp1)*sqrt(1-zeta2)/pi; % time constant, , Eqn. 4.39

Gcl = tf(K,[tau2 2*zeta*tau 1],'iodelay',d); % Closed loop model, Gcl (s), Eqn. 4.33

Now that we have the closed loop model, we can compute the ultimate gain, Ku , and ultimate
frequency, u .

Listing 4.7: Compute the ultimate gain and frequency from the closed loop model parameters.
10 fwu = @(w) pi - d*w - atan2(2*zeta*tau*w,1-tau2*w.2); %

[Gm,Pm,Wcg,Wcp] = margin(Gcl) % Use as a good starting estimate for u


wu = fsolve(fwu,Wcg); % Solve nonlinear Eqn. 4.40 for u

15 B= K/sqrt((1-tau2*wu2)2+(2*tau*zeta*wu)2);
Kcu = Kc*(1+1/B); % Ultimate gain, Ku , Eqn. 4.41

Finally we can now extract the open loop first-order plus deadtime model using Listing 4.8.

Listing 4.8: Compute the open loop model, Gm , Eqn. 4.31.


Km = yinf/(Kc*(A-yinf)); % Plant gain, Gm , Eqn. 4.43
taum = 1/wu*sqrt(Kcu2*Km2-1); % Plant time constant, m , Eqn. 4.44
dm = 1/wu*(pi - atan(taum*wu)); % Plant delay, m , Eqn. 4.45
20 Gm = tf(Km,[taum 1],'iodelay',dm);% Open loop model estimate, Gm (s), Eqn. 4.31

Now that we have an estimate of the plant model, it is trivial to use say the IMC relations to
compute reasonable PI or PID tuning constants. We do need however to decide on an appropriate
desired closed loop time constant.

Listing 4.9: Compute appropriate PI or PID tuning constants based on a plant model, Gm , using
the IMC schemes.
tauc = (tau+dm)/3; % IMC desired closed loop time constant, c
controller = 'PID'; % Type of desired controller, PI or PID

switch controller % See IMC tuning schemes in Table 4.3.


25 case 'PI'
Kc = 1/Km*(tau)/(tauc+dm);
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 169

taui = taum; rs = 1/taui; % Integral time, i , and reset, 1/i .


taud = 0;
case 'PID'
30 Kc = 1/Km*(tau+dm/2)/(tauc+dm/2);
taui = tau+dm/2; rs = 1/taui;
taud = tau*dm/(2*tau+dm);
end % switch

Now all that is left to do is load the controller tuning constants into the PID controller, and set the
controller to automatic.

4.6.4 S UMMARY ON CLOSED LOOP TUNING SCHEMES

These automated PID tuners are, not surprisingly, quite popular in industry and also in academic
circles. Most of the schemes are based on what was described above, that is the unknown process
is approximated by a simple transfer function, where the ultimate gain and frequency is known
as a function of parameters. Once these parameters are curve fitted, then the PID controller is
designed via a ZieglerNichols technique. There are numerous modifications to this basic tech-
nique, although most of them are minor. One extension is reported in [50] while [197] summarises
subsequent modifications, corrects errors, and compares alternatives. The relay feedback method
of Hagglund and Astrom, [16], described in next in 4.7 is also a closed loop single test method.

Problem 4.2 1. Modify the ys_ident.m function to improve the robustness. Add improved
error checking, and try to avoid other common potential pitfalls such as negative controller
gain, integrator responses, sever nonlinearities, excessive noise, spikes etc.

2. Test your auto closed loop tuner on the following three processes Gp (s) (adapted from [50]);

e4s
G1 (s) = (4.49)
s+1
e3s
G2 (s) = (4.50)
(s + 1)2 (2s + 1)
e0.5s
G3 (s) = (4.51)
(s 1)(0.15s + 1)(0.05s + 1)

G1 and G2 show the effect of some dead time, and G3 is an open loop unstable process.

3. Create a m file similar to the one above, that implements the closed loop tuning method as
described by Chen, [50]. You will probably need to use the M ATLAB fsolve function to
solve the nonlinear equation.

4.7 A UTOMATED TUNING BY RELAY FEEDBACK

Because the manual tuning of PID controllers is troublesome and tedious, it is rarely done which
is the motivation behind the development of self-tuning controllers. This tuning on demand
behaviour after which the controller reverts back to a standard PID controller is different from
adaptive control where the controller is consistently monitoring and adjusting the controller tun-
ing parameters or even the algorithm structure. Industrial acceptance of this type of smart con-
troller was much better than for the fully adaptive controller, partly because the plant engineers
were far more confident about the inherent robustness of a selftuning controller.
170 CHAPTER 4. THE PID CONTROLLER

One method to selftune employed by the ECA family of controllers manufactured by ABB shown
in Fig. 4.28 uses a nonlinear element, the relay. The automated tuning by relays is based on the
assumption that the Ziegler-Nichols style of controller tuning is a good one, but is flawed in
practice since finding the ultimate frequency u is a potentially hazardous tedious trial and error
experiment. The development of this algorithm was a joint project between SattControl and Lund
University, Sweden [16]. This strategy of tuning by relay feedback is alternatively known as the
Auto Tune Variation or ATV method. A good tutorial for estimating models using relay feedback
is given in [125, section 4].

Figure 4.28: A commer-


cial PID controller with a
self-tuning option using a
relay from ABB.

The PID controller with selftuning capability using relay feedback is really two controllers in one
as shown in Fig. 4.29. Here the PID component is disabled, and the relay is substituted. Once the
executive software is confident that the updated controller parameters are an improvement, the
switch is toggled, and the controller reverts back to a normal PID regulator.

Relay (active)
+d
d Plant
setpoint, r u
bc
b G(s) b y

PID

(inactive)

Figure 4.29: A process under relay tuning with the PID regulator disabled.

A relay can be thought of as an on/off controller or a device that approximates a proportional


controller with an infinite gain, but hard limits on the manipulated variable. Thus the relay has
two outputs: if the error is negative, the relay sends a correcting signal of d units, and if the
error is positive, the relay sends a correcting signal of +d units.
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 171

4.7.1 D ESCRIBING FUNCTIONS

A relay is a nonlinear element that, unlike the situation with a linear component, the output
to a sinusoidal input will not be in general sinusoidal. However for many plants that are low-
pass in nature, the higher harmonics are attenuated, and we need only consider the fundamental
harmonic of the nonlinear output.

The describing function N , of a nonlinear element is defined as the complex ratio of the funda-
mental harmonic of the output to the input, or

def Y1
N = 6 (4.52)
X1

where X1 , Y1 are the amplitudes of the input and output, and is the phase shift of the fundamen-
tal harmonic component of the output, [154, p652]. If there is no energy storage in the nonlinear
element, then the describing function is a function only of the input amplitude.

By truncating a Fourier series of the output, and using Eqn. 4.52, one can show that the describing
function for a relay with amplitude d is

4d
N (a) = (4.53)
a
where a is the input amplitude.

Now a sustained oscillation or limit cycle of a linear plant G(i) together with a nonlinear element
with describing function N will occur when 1 + N G(i) = 0 or

1
G(i) = (4.54)
N
Now since in the case of a relay, N given by Eqn. 4.53 is purely real, then the intersection of G(i)
and 1/N is somewhere along the negative real axis. This will occur at a gain of a/(4d), so the
ultimate gain, Ku , which is the reciprocal of this, is

4d
Ku = (4.55)
a
The ultimate frequency, u = 2/Pu , (in rad/s) is simply the frequency of the observed output
oscillation and can be calculated from measuring the time between zero crossings to establish the
period, and the output amplitude, a, by measuring the swing of the output. We now know from
this simple experiment one point on the Nyquist curve and a reasonable PID controller can be
designed based on this point using, say, the classical Ziegler-Nichols table. A typical test setup
and response is given in Fig. 4.30.

In summary, under relay feedback control, most plants will exhibit some type of limit oscillation
such as a sinusoidal-like wave with a phase lag of 180 . This is at the point where the open loop
transfer function crosses the negative real axis on a Nyquist plot, although not necessarily at the
critical 1 point.

Given that we know a single point on the Nyquist curve, how do we obtain controller parameters?
It is convenient to restrict the controller to the PID structure since that is what is mostly about in
practice. The original ZN tuning criteria gave responses that should give a specified gain and
phase margin. For good control a gain margin of about 1.7 and a phase margin of about m = 30
is recommended. Note that by using different settings of the PID controller, one can move a single
point on the Nyquist curve to any other arbitrary point. Let us assume that we know the point
where the open loop transfer function of the process, Gp (s), crosses the negative real axis, u . This
point has a phase (or argument) of or 180. We wish the open loop transfer function, Gc Gp
172 CHAPTER 4. THE PID CONTROLLER

Relay

u
Plant b y

output, y
output swing, 2a

input, u relay swing, 2d

time, t

Period, P

Figure 4.30: An unknown plant under relay feedback exhibits an oscillation with amplitude a,
and period P .

to have a specified phase margin of m by choosing appropriate controller tuning constants. Thus
equating the arguments of the two complex numbers gives
 
1
arg 1 + + iu d = m (4.56)
iu i
which simplifies to5
1
u d = tan m
u i
Since we have two tuning parameters (i , d ), and one specified condition, (m ), we have many
solutions. Since their are many PID controller constants that could satisfy this phase margins,
Astrom, [12], chose that the integral time should be some factor of the derivative time,
i = f d (4.57)
Then the derivative time, which must be real and positive, is
p
f tan m + f 2 tan2 m + 4f
d = (4.58)
2f u
5 Remember def
that for a complex number z = x + iy, the argument of z is the angle z makes with the real axis;
y
arg(z) = arg(x + iy) = tan1
x
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 173

where the arbitrary factor, f 4. Given the ultimate gain Ku , the controller gain is

Kc = Ku cos m (4.59)

Algorithm 4.3 Relay auto-tuning


In summary, to tune a system by relay feedback, you must:

1. Turn off the current PID controller, and swap over to a relay feedback controller with known
relay gain d.
2. Apply some excitation if necessary to get the process to oscillate under relay control. Mea-
sure the resultant error amplitude a, and period Pu . Calculate the ultimate gain from
Eqn. 4.55 and associated frequency u .
3. Decide on a phase margin, (m 30 ), the factor, f 4, and calculate the PID controller
tuning constants using Eqns 4.574.59.
4. Download these parameters to the PID controller, turn off the relay feedback, and swap
back over to the PID controller.
5. Wait until the operator again initiates the self-tuning program after which return to step 1.

4.7.2 A N EXAMPLE OF RELAY TUNING

We can compare the experimentally determined ultimate gain, Ku , and ultimate frequency, u ,
from a relay experiment with the theoretical values for the plant
1
G(s) = (4.60)
( s + 1)4
where the time constant is = 0.25. The ultimate gain and ultimate frequency are analytically
obtained by solving the two equations
 
Ku Ku
arg = , and =1
( s + 1)4 s=iu ( s + 1)4
s=iu

simultaneously which is somewhat involved, (and of course requires knowledge of the plant
which in practice we would not have). Also note that due to the discontinuity in the tan1 (x)
function at the solution, we need to slightly modify the nonlinear equation for the phase angle to
be solved when using fsolve.

G = @(K,w) K./(0.25*1i*w+1).4; % Plant under consideration, Eqn. 4.60, G(s = i).


2 Mag = @(K,w) abs(G(K,w)) - 1; % Magnitude = 1
Ang = @(K,w) atan2(imag(G(K,w)), -real(G(K,w))) - 0; % Discontinuity in tan1 (x)
Fun = @(x) [Mag(x(1), x(2)); Ang(x(1), x(2))];

x0 = [0.1,1]'; % Initial guess for x = [Ku , u ]T


7 x = fsolve(Fun,x0) % Solve for Ku & u

An alternative graphical solution is obtained using Bode and/or Nyquist diagrams in M ATLAB.

G = zpk([],[-4 -4 -4 -4],44);
nyquist(G,logspace(-1,3,200));
3 bode(G) % Bode diagram or margin is an alternative
174 CHAPTER 4. THE PID CONTROLLER

The Nyquist diagram given in the upper plot of Fig Fig. 4.31 shows the curve crossing the negative
real axis at 1/Ku 0.25, thus Ku 4. To find the ultimate angular frequency, we need to plot
the phase lag of the Bode diagram shown in the bottom figure of Fig. 4.31. The phase angle crosses
the 180 radians point at a frequency of about u = 4 rad/s. Thus the ultimate period is
Pu = 2/u = 1.57 seconds. Note that it was unnecessary to plot the Nyquist diagram since we
could also extract the ultimate gain information from the magnitude part of the Bode diagram, or
even simply just use margin.

Nyquist Diagram

0.5 1/K
u

Imaginary Axis

0 6
4
0.1
3
0.5 0.5
2
1.5 1
1
1 0.5 0 0.5 1
Real Axis

Bode Diagram
Gm = 12 dB (at 4 rad/sec) , Pm = 180 deg (at 0 rad/sec)

0
Magnitude (dB)

50

100

150
0
Figure 4.31: Nyquist (top) and Bode (bot-
Phase (deg)

tom) diagrams for 1/(0.25s + 1)4 . The 180


Nyquist curve crosses the negative -axis
at about 0.25. The frequency at which the
360
Bode curve passes though = 180 is 10
1
10
0
u 10
1
10
2

4 rad/s. Frequency (rad/sec)

E STIMATING Ku AND u BY RELAY EXPERIMENT

Now we can repeat the evaluation of Ku and u , but this time by using the relay experiment.
This is the approach we would need to take in real life where we are faced not with a convenient
mathematical transfer function, but an actual physical plant.

I will use a relay with a relay gain of d = 2 but with no hysteresis (h = 0) and a sample time
of T = 0.05 seconds. To get the system moving, I will choose a starting value away from the
setpoint, y0 = 1.

G = zpk([],[-4 -4 -4 -4],44); % Plant: G(s) = 1/(0.25s + 1)4


2 Ts=0.05; t = [0:Ts:7]'; % sample time
u = 0*t; y=NaN*t; % initialise u & y
d=2; % relay gain
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 175

[Phi,Del,C,D] = ssdata(c2d(G,Ts));
x = [0,0,0,0.1]'; % some non-zero initial state
7

for i=1:length(t)-1; % start relay experiment


x = Phi*x + Del*u(i); % integrate model 1 step
y(i) = C*x; % output
u(i+1) = -d*sign(y(i)); % Relay controller
12 end % for

plot(t,y,t,u,'r--')

The result of the relay simulation is given in Fig. 4.32 where we can see that once the transients
have died out, the amplitude of the oscillation of the output (or error since the setpoint is constant)
is a = 0.64 units, with a period of P = 1.6 seconds. Using Eqn. 4.55, we get estimates for the
ultimate gain and angular frequency

4d
Pu = P = 1.6 1.57s, and Ku = = 3.978 4
a
which approximately equal the estimates that I graphically found using the Bode and Nyquist
diagrams of the true plant.

2
output & setpoint

0
a = 0.64
1
relay off
2
5
input

0
Pu = P = 1.6s
5
0 5 10 15 20 25 30
time [s]

Figure 4.32: PID tuning using relay feedback. For the first 10 seconds the relay is enabled, and the
process oscillates at the ultimate frequency. After 10s, the relay is turned off, the PID tuner with
the updated constants is enabled, and the plant is under control.

Once we have the ultimate gain and frequency, the PID controller constants are obtained from
Eqn. 4.57 to 4.59 and are shown in Table 4.6. Alternatively we can use any Z-N based tuning
scheme.

Table 4.6: The PID controller tuning parameters obtained from a relay based closed loop experi-
ment
PID parameter Hagglund Ziegler-Nichols units
Kc 3.445 0.6Ku = 3.4
i 0.866 Pu /2 = 0.8 s
d 0.217 Pu /8 = 0.2 s
176 CHAPTER 4. THE PID CONTROLLER

Now that we have the PID tuning constants, we can simulate the closed loop response to setpoint
changes. We could create the closed loop transfer function using G*Gpid/(1 + Gpid*G), how-
ever the preferred approach is to use the routines series and feedback. This has the advantage
that it will cancel common factors and perform a balanced realisation.

1 G = zpk([],[-4 -4 -4 -4],44); % Plant to be controlled, Eqn. 4.60.

K = 3.4; ti = 0.8; td = 0.2; % Controller constants from Table 4.6.


Gpid = tf(K*[ti*td ti 1],[ti 0])
Gcl = feedback(series(G,Gpid),1); % Create closed loop for simulation
6

R = [zeros(10,1);ones(100,1);-ones(100,1)]; % Design setpoint vector


T = 0.05*[0:1:length(R)-1]'; % time scale
y = lsim(Gcl,R,T); % Simulate continuous
plot(T,[y,R])

The remainder of Fig. 4.32 for the period after 10 seconds shows the quite reasonable response
using the PID constants obtained via a relay experiment. Perhaps the overshoot at around 40%
and oscillation is a little high, but this is a well known problem of controller tuning via the classical
ZN approach, and can be addressed by using the modified ZN tuning constants.

4.7.3 S ELF - TUNING WITH NOISE DISTURBANCES

The practical implementation of the relay feedback identification scheme requires that we mea-
sure the period of oscillation P , and the error amplitude a automatically. Both these parameters
are easily established manually by visual inspection if the process response is well behaved and
noise free, but noisy industrial outputs can make the identification of even these simple charac-
teristics difficult to obtain reliably and automatically.

When considering disturbances, [12] develops two modifications to the period and amplitude
identification part of the selftuning algorithm. The first involves a least squares fit, (described
below) and the second involves an extended Kalman filter described in problem 9.6. Of course
hybrids of both these schemes with additional heuristics could also be used. Using the same pro-
cess and relay as in 4.7.2, we can introduce some noise into the system as shown in the S IMULINK
model given in Fig. 4.33(a). Despite the low noise power (0.001), and the damping of the fourth-
order process, the oscillations vary in amplitude and frequency as shown in Fig. 4.33(b).

To obtain reliable estimates of the amplitude and period we can use the fact that a sampled sinu-
soidal function with period P and sample time T satisfies

y(t) 1 y(t T ) + y(t 2T ) + 2 = 0 (4.61)

with  
2T
1 = 2 cos (4.62)
P
The extra constant, 2 is introduced to take into account non-zero means. Eqn. 4.61 is linear in the
parameters, , so we can solve the linear least-squares regression problem in M ATLAB using the
backslash command.  
1  +
= = ytT 1 (yt + yt2T ) (4.63)
2
Once we have established 1 , (we are uninterested in 2 ), the period is obtained by inverting
Eqn. 4.62,
2T
P = (4.64)
cos1 (1 /2)
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 177

BandLimited
White Noise

256
0
(s+4)(s+4)(s+4)(s+4)
Constant Relay ZeroPole output

(a) Relay feedback of a fourth-order plant with band-limited noise added

2
1.5
1
input & output

0.5
0
0.5
1
1.5
2
0 2 4 6 8 10 12 14 16 18 20
time

(b) The controlled output and relay response.

Figure 4.33: Relay oscillations with noise

Now that we know the period, we can solve another linear least-squares regression problem given
the N output data points,

N      2
X 2kT 2kT
min y(kT ) 3 sin 4 cos 5 (4.65)
k=1 P P

where the amplitude is given by


q
a= 32 + 42

By doing the regression in this two step approach, we avoid the nonlinearity of Eqn. 4.65 if we
were to estimate P as well in an attempt to do the optimisation in one step.

Running the S IMULINK simulation given in Fig. 4.33(a) gives a trend something like Fig. 4.33(b).

The only small implementation problem is that S IMULINK does not necessarily return equally
sampled data for continuous system simulation. The easiest work-around is to define a regularly
spaced time vector and do a table lookup on the data using interp1. The script file given in
listing 4.10 calculates the period and amplitude using least-squares.

Listing 4.10: Calculates the period and amplitude of a sinusoidal time series using least-squares.
N = length(t); % Length of data series
tr = linspace(t(1),t(length(t)),1.5*N)'; dt = tr(2)-tr(1);
yr = interp1(t,y,tr,'linear'); % S IMULINK is not regular in time
y=yr; t = tr; % Now sample time T is regularly spaced.
5

% Construct data matrix


npts = length(y); % # of data points, N
rhs = y(3:npts) + y(1:npts-2);
X = [y(2:npts-1),-ones(size(rhs))];
178 CHAPTER 4. THE PID CONTROLLER

10 theta = X\rhs; % Solve LS problem, Eqn. 4.63.


P = 2*pi*dt/acos(theta(1)/2); % Now find period from Eqn. 4.64.

omega = 2*pi/P; % 2nd round to find amplitude, Eqn. 4.65


k = [1:npts]';
15 X = [sin(omega*k*dt) cos(omega*k*dt) ones(size(k))]; % data matrix
theta = X\y; % Solve 2nd LS problem, Eqn. 4.65.
a = norm(theta(1:2)); % Extract amplitude.

In my case, the computed results are:

P = 1.822, a = 0.616

which is not far off the values obtained with no disturbances, from page 175 of P = 1.6 and
a = 0.64.

Relay based tuning of a black-box

Using a data acquisition card and a real-time toolbox extension6 to M ATLAB, we can repeat the
trial of the relay based tuning method this time controlling a real process, in this case a black
box. The S IMULINK controller and input/output data are given in Fig. 4.34.

From Fig. 4.34, we can measure the period and amplitude of the oscillating error

Pu = 7 seconds, a = 0.125

giving an approximate ultimate gain as

4d
Ku = , with d = 1
a
4
= = 10.2
0.125
which leads to appropriate Kc , i , d , using the modified ZN rules

Kc i (s) d (s)
0.33Ku = 3.4 Pu /2 = 3.5 Pu /3 = 2.3

Since S IMULINK uses a slightly different PID controller scheme from the classical formula,

Kc
P = Kc = 3.4, I= = 0.97, D = Kc d = 7.8
i

Using these values in the PID controller with the black-box, we obtain the controlled results
shown in Fig. 4.35 which apart from the noisy input and derivative kick (which is easily removed,
refer 4.4.1), does not look too bad.

4.7.4 M ODIFICATIONS TO THE RELAY FEEDBACK ESTIMATION ALGORITHM

The algorithm using a relay under feedback as described in section 4.7 establishes just one point
on the Nyquist curve. However it is possible by using a richer set of relay feedback experiments
to build up a more complex model, and perhaps better characterise a wider collection of plants.
6 Available from Humusoft
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 179

t
Clock To Workspace1 AutoScale
Graph1
Adapter Mux yout
To Workspace
PID Mux
0.6
PID Controller
setpoint
+ +
+ Output Plug
0 RT Out RT In
Sum1 Sum
Signal Constant Switch RT Out RT In
generator1 Input Plug

Relay

(a) A S IMULINK implementation of the relay feedback experiment.

Relay based tuning (Black Box)


0.8

0.7
Output & setpoint

0.6

0.5

0.4

0.3
0 10 20 30 40 50 60 70

0.5
input

0.5

1
0 10 20 30 40 50 60 70
time (s)
(b) The output of the relay experiment

Figure 4.34: Experimental implementation of relay tuning of the blackbox

R ELAYS WITH HYSTERESIS

Physical relays always have some hysteresis h, to provide mechanical robustness when faced with
noise. By adjusting the hysteresis width we can excite the plant at different points on the Nyquist
curve.

The response of a relay with hysteresis is given in Fig. 4.36. Note that the relay will not flip to
+d from the d position until we have a small positive error, in this case h. Likewise, from the
reverse direction, the error must drop below a small negative error, h, before the relay flips from
+d to d.

The describing function for a relay with amplitude d and hysteresis width h is
1 p 2 h
= a h2 j (4.66)
N (a) 4d 4d
which is a line parallel to the real axis in the complex plane. Compare this with the describing
function for the pure relay in Eqn. 4.53.

The intersection of this line and G(i) is the resultant steady-state oscillation due to the relay,
refer Fig. 4.37. By adjusting the relay hysteresis width, h, we can move the 1/N (a) line up and
180 CHAPTER 4. THE PID CONTROLLER

PID control based on relay tune


1
Output & setpoint

0.8

0.6

0.4

0.2
0 20 40 60 80 100 120

1.5

0.5
input

0.5

1.5
0 20 40 60 80 100 120
time (s)

Figure 4.35: Experimental PID control of the blackbox using parameters obtained by the relay-
based tuning experiment. (See also Fig. 4.34.)

+d relay amplitude

output 0

h 0 +h
error

Figure 4.36: A relay with hysteresis width h and output amplitude d.


4.7. AUTOMATED TUNING BY RELAY FEEDBACK 181

down, thereby establishing different points on the Nyquist diagram. Of course if we increase the
hysteresis too much, then we may shift the 1/N (a) curve too far down so that it never intercepts
the G(i) path. In this situation, we will not observe any limit cycle in the closed loop experiment.

G(i)

1 b
N (a) increasing

operating point under relay feedback

Figure 4.37: Relay feedback with hysteresis width h.

Fig. 4.38 shows the estimated frequency response for the plant G(s) = 256/(s + 4)4 calculated
using a relay with d = 2 and varying amounts of hysteresis from h = 0 to h = 1.4. Note that the
calculated points do follow the true frequency response of the plant, at least for small values of h.
Obviously if h is too large, then the line 1/N (a) will not intersect the frequency response curve,
G(i), so there will be no oscillations.

0.4

0.2

0 0
0.2
0.4
0.2 0.6
G(i )

0.8
0.4 1
1.2
1.4
0.6

0.8
Figure 4.38: Using a relay feedback with varying
1
amounts of hysteresis from h = 0 to h = 1.4 to
1.2 obtain multiple points, , on a Nyquist curve. The
1 0.5 0 0.5 1
G(i ) true frequency response G(i) is the solid line.

I NSERTING KNOWN DYNAMICS INTO THE FEEDBACK LOOP

A second simple modification is to insert an integrator prior to the relay. The integrator subtracts
/2 or 90 from the phase, and multiplies the gain by a factor of 1/. In this way, under
relay feedback we can estimate the point where the Nyquist curve crosses the negative imaginary
axis as well as the negative real axis. Such a scheme is shown in Fig. 4.39 and is described by
[203] which is a specific case of the general schemes described by [118, 148]. A survey of other
modifications to the basic relay feedback idea for controller tuning is given in [46].
182 CHAPTER 4. THE PID CONTROLLER

Relay

K(s)
0
den(s)
Constant Manual Switch Transfer Fcn Transport output
1 Delay
s
Integrator Relay1

Figure 4.39: Relay feedback with an integrator option

Establishing multiple points on the Nyquist curve enables one to extract a transfer function mod-
els. In the case of a first-order plus deadtime model

Kp
G(s) = eLs (4.67)
s + 1
we need two arbitrary distinct points. That is we establish the magnitude and phase of G(i) at
two frequencies 1 and 2 via two relay experiments.

Defining the magnitudes and angles to be

1 1
|G(i1 )| = , |G(i2 )| =
k1 k2
6 G(i1 ) = 1 , 6 G(i2 ) = 2

then the plant gain is


s
12 22
Kp = , (4.68)
k2 12
2 k12 12
the time constant is
1 q 2 2
= k1 Kp 1, (4.69)
1
and the deadtime is
1 
L= 1 + tan1 (1 ) . (4.70)
1
In the case of a pure relay and a pure relay with an integrator,

1 = , 2 = /2.

The case for a second order plant

Kp
G(s) = eLs (4.71)
s2 + as + b
is more complicated. The following algorithm, due to [118], first estimates the deadtime via a
nonlinear function, then extracts the coefficients Kp , a and b using least-squares.

First we define the matrices



A1 12 0 1 sin L1
A2 22 1/Kp
0 1 cos L1
A= A3 = 0
, X = a/Kp , Y= (4.72)
1 0 sin L2
b/Kp
A4 0 2 0 cos L2
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 183

0.4
Step Response

2
0.2 G
G
1est
1.5 G
2est
0

1
{G(i )}

0.2

Amplitude
0.4 0.5

0.6 0

0.8 0.5

1 1
1 0.5 0 0.5 1 1.5 2 0 20 40 60 80 100 120 140 160 180
{G(i )} Time (sec)

(a) The resultant Nyquist plots (b) The resultant step responses

Figure 4.40: Identification of transfer function models using a two-point relay experiment.

and

B1 {G1 (j1 )} {G1 (j1 )} 0 0
B2 0 0 {G1 (j2 )} {G1 (j2 )}
B=
B3 = {G1 (j1 )} {G1 (j1 )}
(4.73)
0 0
B4 0 0 {G1 (j2 )} 1
{G (j2 )}

Now provided we know the deadtime L, we can solve for the remaining parameters in X using
1
A1 B1
X = A2 B2 Y (4.74)
A3 B3

However since we do not know L, we must solve the nonlinear relation

(A4 C B4 )Y = 0 (4.75)

Once given the model, any standard model-based controller tuning scheme such as internal
model control (IMC) can be used to establish appropriate PID tuning constants.

Fig. 4.40 illustrates the procedure for when the true, but unknown plant is the third-order system
1
G= e6s
(9s + 1)(6s + 1)(4s + 1)
with estimated first and second order models.

Problem 4.3 1. Draw a Nyquist diagram for the transfer function Gp (s). Investigate what hap-
pens to the Nyquist diagram when a PI controller is added. How is the curve shifted
4
Gp (s) =
( s + 1)3
where = 2.0.
2. Write down pseudo code for the automatic tuning (one button tuning) of a PID controller
using the ZN tuning criteria. Ensure that your algorithm will work under a wide variety of
conditions and will not cause too much instability if any. Consider safety and robustness
aspects for your scheme.
184 CHAPTER 4. THE PID CONTROLLER

1.5

Impulse response Impulse response Impulse response

0.5
= 0.39 = 0.71 = 1.00

0 5 10 15 0 5 10 15 0 5 10 15

Figure 4.41: The step responses and associated monotonicity index, , for three representative
systems.

3. Write a .m file to implement a relay feedback tuning scheme using the Astrom and Hagglund
tuning method. Use the pidsim file as a starting point. Pass as parameters, the relay gain
d, and the hysteresis delay h. Return the ultimate period and gain and controller tuning
parameters.

4.8 D RAWBACKS WITH PID CONTROLLERS

The PID controller despite being a simple, intuitive and flexible controller does have some draw-
backs. For example PID control is not particularly suitable for processes where the open loop step
response is highly oscillatory. A way to quantify the extent of the wobble proposed by [14, p226]
is to computing a monotonicity index
R
0
g(t) dt
= R (4.76)
0
|g(t)| dt

where g(t) is the impulse response of the system. Note that if the step response wobbles, then
the impulse response goes negative. For systems with no wobble, the area under the impulse
response and the area under the absolute value of the impulse response is the same, so = 1.
For systems that do wobble, the area under the impulse curve will be less due to the periods of
negative areas, so < 1. The step response and associated monotonicity index for three systems
are compared in Fig. 4.41. PID control is deemed suitable for process responses where > 0.8.

Further limitations of PID control are the inability to handle responses with large deadtimes, or
inverse responses. These are highlighted when trying to control an inverse response as demon-
strated in 4.8.1, and a simple add-on for deadtime processes is given in 4.9.

4.8.1 I NVERSE RESPONSE PROCESSES

An inverse response is where the response to a step change first heads in one direction, but then
eventually arrives at a steady state in the reverse direction as shown in the middle step response
of Fig. 4.41 or Fig. 4.43(b). Naturally this sort of behaviour is very difficult and un-intuitive to con-
trol, and if one tends to overreact, perhaps because the controllers gain is too high, then excessive
oscillation is likely to result. Inverse responses are common in the control of the liquid water level
in a steam drum boiler as described in [21] due to a phenomena known as the swell-and-shrink ef-
fect. For example the water level first increases when the steam valve is opened because the drum
pressure will drop causing a swelling of the steam bubbles below the surface. This undershoot
4.8. DRAWBACKS WITH PID CONTROLLERS 185

Figure 4.42: The J curve by Ian Bremmer popularised the


notion that things like private investments, or even the
openness/stability state of nations often get worse before
they get better. J curves are inverse responses.

can have important implications in economic theory as encapsulated in the popular J Curve in
Fig. 4.42, as well as attempting to reverse parking a car.

Transfer functions that have an odd number of right-hand-plane zeros have stable inverse re-
sponses. Inverse response processes belong to a wider group of transfer functions called Non-
Minimum Phase (NMP) transfer functions. Other NMP examples are those processes that have
dead time which are investigated later in 4.9. The term non-minimum phase derives from the
fact that for these transfer functions, there will exist another transfer function that has the same
amplitude ratio at all frequencies, but a smaller phase lag. The intricacies of NMP processes and
the effect of process zeros is reviewed in [87].

One interpretation of inverse response systems is as the sum of two transfer functions: one that
goes downwards a short way quickly, (an initial undershoot) and one that goes up further but much
more leisurely. The addition of these two responses gives the inverse response. For example;

K1 K2
G(s) = (4.77)
s+1 s+1
| 1 {z } | 2 {z }
G1 G2
(K1 2 K2 1 )s + K1 K2
= (4.78)
(1 s + 1)(2 s + 1)

If K1 > K2 , then for the system to have right hand plane zeros (numerator polynomial equals
zero in Eqn. 4.78), then
K1 2 < K2 1 (4.79)
If K1 < K2 , then the opposite of Eqn. 4.79 must hold.

A S IMULINK simulation of a simple inverse response process,

4 3 (5s + 1)
G(s) = = (4.80)
3s + 1 s + 1 (3s + 1)(s + 1)
| {z } | {z }
G1 G2

of the form given in Eqn. 4.77 and clearly showing the right-hand plane zero at s = 1/5 is shown
in Fig. 4.43(a). The simulation results for the two individual transfer functions that combine to
give the overall inverse response are given in Fig. 4.43(b).

C ONTROLLING AN INVERSE RESPONSE PROCESS

Suppose we wish to control the inverse response of Eqn. 4.80 with a PI controller tuned using the
ZN relations based on the ultimate gain and frequency. The following listing adapts the strategy
described in Listing 4.4 for the PID tuning of an arbitrary transfer function.
186 CHAPTER 4. THE PID CONTROLLER

(a) A S IMULINK simulation of the inverse response process de-


scribed by Eqn. 4.80

Step Response

2 G
1
Amplitude

G1
1
G +G G
1 2 2
G
2 G +G
2 1 2

0 2 4 6 8 10 12 14 16 18
Time (sec)

(b) The inverse response G made up of G1 + G2 .

Figure 4.43: An inverse response process is comprised of two component curves, G1 + G2 . Note
that the inverse response first decreases to a minimum y = 1 at around t = 1.5 but then increases
to a steady-state value of y = 1.

G= tf(4,[3 1]) - tf(3,[1 1]); % Inverse response plant, Eqn. 4.80.

3 [Ku,Pm,Wcg,Wcp] = margin(G); % Establish critical gain, Ku , and frequency, u .


Pu = 2*pi/Wcg; % Critical period, Pu
Kc = 0.45*Ku; taui = Pu/1.2; % PI tuning rules (Ziegler-Nichols)
Gc = tf(Kc*[taui 1],[taui 0]);

8 Gcl = feedback(G*Gc,1); % Closed loop


step(Gcl); % Controlled response as shown in Fig. 4.44.

The controlled performance in Fig. 4.44 is extremely sluggish taking about 100 seconds to reach
the setpoint while the open loop response as shown in Fig. 4.43(b) takes only about one tenth of
that to reach steady-state. The controlled response also exhibits an inverse response.

The crucial problem with using a PID controller for an inverse response process is that if the
controller is tuned too tight, in other words the gain is too high and the controller too fast acting,
then the controller will look at the early response of the system, which being an inverse system is
going the wrong way, and then try to correct it, thus making things worse. If the controller were
more relaxed about the correction, then it would wait and see the eventual reversal in gain, and
not give the wrong action.
4.8. DRAWBACKS WITH PID CONTROLLERS 187

0.5

Figure 4.44: A NMP plant controlled with a


0.5
0 20 40 60 80 100 120
PI controller is very sluggish but is stable, and
time [s] does eventually reach the setpoint.

In reality, when engineers are faced with inverse response systems, the gains are deliberately de-
turned giving a sluggish response. While this is the best one can do with a simple PID controller,
far better results are possible with a model based predictive controller such as Dynamic Matrix
Control (DMC). In fact this type of process behaviour is often used as the selling point for these
more advanced schemes. Processes G(s) that have RHP zeros are not open-loop unstable (they
require RHP poles for that), but the inverse process (G(s)1 ) will be unstable. This means that
some internal model controllers, that attempt to create a controller that approximates the inverse
process, will be unstable, and thus unsuitable in practice.

4.8.2 A PPROXIMATING INVERSE - RESPONSE SYSTEMS WITH ADDITIONAL DEAD -


TIME

We cannot easily remove the inverse response, but we can derive an approximate transfer function
that has no right-hand plane zeros using a Pade approximation. We may wish to do this to avoid
the problem of an unstable inverse in a controller design for example. The procedure is to replace
the non-minimum phase term (s1) in the numerator with (s1)(s+1) (s+1), and then approximate
the first term with a deadtime element. In some respects we are replacing one evil (the right-hand
plane zeros), with another, an additional deadtime.

For example the inverse response plant


(2s 1)
G= e3s (4.81)
(3s + 1)(s + 1)
has a right-hand zero at s = 1/2 which we wish to approximate as an additional deadtime ele-
ment. Now we can re-write the plant as
(2s + 1) (2s 1) 3s
G= e
(3s + 1)(s + 1) (2s + 1)
| {z }
Pade

and approximate the second factor with a Pade approximation7 in reverse


(2s + 1)
G e4s e3s
(3s + 1)(s + 1)
(2s + 1)
e7s (4.82)
(3s + 1)(s + 1)
Note that the original system had an unstable zero and 3 units of deadtime while the approximate
system has no unstable zeros, but whose deadtime has increased to 7.

We can compare the two step responses with:


7 Recall that es (1 (2/)s)/(1 + (2/)s) using a 1-1 Pade approximation.
188 CHAPTER 4. THE PID CONTROLLER

1 >> G = tf([2 -1],conv([3 1],[1 1]),'iodelay',3); % Original system, Eqn. 4.81.


>> G2 = tf(-[2 1],conv([3 1],[1 1]),'iodelay',7);% No inverse response, Eqn. 4.82.

>> step(G,G2); legend('G','G_{pade}') % Refer Fig. 4.45.

in Fig. 4.45 which, apart from the fact that the approximation version failed to capture the inverse
response, look reasonably similar.

Step Response

0.5
True system G
Gpade

0
Pade approximation
Amplitude

0.5

Figure 4.45: Approximating inverse- 1


response systems with additional 0 5 10 15 20 25
deadtime Time (sec)

Problem 4.4 A model of a flexible beam pinned to a motor shaft with a sensor at the free end
repeated in [54, p12] is approximated by the non-minimum phase model,
1 6.475s2 + 4.0302s + 175.77
Gp (s) = 3 (4.83)
s 5s + 3.5682s2 + 139.5021s + 0.0929

1. Verify that Gp (s) is non-minimum phase.


2. Plot the root locus diagram of Gp (s) for the cases (a) including the integrator, and (b) ex-
cluding the integrator. Based on the root locus, what can you say about the performance of
stabilising the system with a proportional controller?
3. Simulate the controlled response of this system with an appropriately tuned PID controller.

4.9 D EAD TIME COMPENSATION

Processes that exhibit dead time are difficult to control and unfortunately all too common in the
chemical processing industries. The PID controller, when controlling a process with deadtime
falls prey to the sorts of problems exhibited by the inverse response processes simulated in 4.8.1,
and in practice the gain must be detuned so much that the controlled performance suffers. One
successful technique for controlling processes with significant dead time, and that has subse-
quently spawned many other types of controllers, is the Smith predictor.

To implement a Smith predictor, we must have a model of the process, which we can artificially
split into two components, one being the estimated model without deadtime, G(s) and one com-
ponent being the estimated dead time,

Gp (s) = G(s) es G(s) es (4.84)


4.9. DEAD TIME COMPENSATION 189

The fundamental idea of the Smith predictor is that we do not control the actual process,Gp (s),
but we control a model of the process without the deadtime, G(s). Naturally in real life we
cannot remove the deadtime easily, (otherwise we would of course), but in a computer we can
easily remove the deadtime. Fig. 4.46 shows a block diagram of the deadtime compensator where
D(s) is our standard controller such as say a PID controller. The important feature of the Smith
predictor scheme is that the troublesome pure deadtime component is effectively removed from
the feedback loop. This makes the controllers job much easier.

Deadtime compensator
Plant

r(t) + + + D(s) G(s)es y(t)

G(s)es

G(s)

Figure 4.46: The Smith predictor controller structure

Using block algebra on the Smith predictor structure given in Fig. 4.46 we can easily show that
the closed loop transfer function is

Y (s) D(s) G(s)es


= (4.85)
R(s) 1 D(s)G(s)es + D(s)G(s)es + G(s)D(s)

and assuming our model closely apprtoximates the true plant,

D(s)G(s)
= es if G = G and = (4.86)
1 + G(s)D(s)
The nice property of the Smith predictor is now apparent in Fig. 4.47 in that the deadtime has
been cancelled from the denominator of Eqn. 4.86, and now sits outside the feedback loop.

Effective controller Plant deadtime

r(t) + D(s)G(s) b
es y(t)

Figure 4.47: The Smith predictor structure from Fig. 4.46 assuming no model/plant mis-match.

For a successful implementation, the following requirements should hold:

1. The plant must be stable in the open loop.


190 CHAPTER 4. THE PID CONTROLLER

2. There must be little or no model/plant mismatch.

3. The delay or dead time must be realisable. In a discrete controller this may just be a shift
register, but if the dead time is equivalent to a large number of sample times, or a non-
integral number of sample times, or we plan to use a continuous controller, then the imple-
mentation of the dead time element is complicated.

S MITH PREDICTORS IN S IMULINK

Suppose we are to control a plant with significant deadtime,

2
Gp = e50s (4.87)
2 s2 + 2 s + 1

where = 10, = 0.34. A PID controller with tuning constants K = 0.35, I = 0.008, D = 0.8 is
unlikely to manage very well, so we will investigate the improvement using a Smith predictor.

A S IMULINK implementation of a Smith predictor is given in Fig. 4.48 which closely follows the
block diagram of Fig. 4.46. To turn off the deadtime compensation, simply double click on the
manual switch. This will break the compensation loop leaving just the PID controller.

2
PID
100 s2 +6.8s+1
setpoint PID Controller delayp =50 Scope
process

Sum 3
Manual Switch
delay =50
2
100 s2 +6.8s+1
model

Figure 4.48: A S IMULINK implementation of a Smith predictor. As shown, the deadtime compen-
sator is activated. Refer also to Fig. 4.46.

The configuration given in Fig. 4.48 shows the predictor active, but if we change the constant
block to zero, the switch will toggle removing the compensating Smith predictor, and we are left
with the classical feedback controller. Fig. 4.49 highlights the usefulness of the Smith predictor
showing what happens when we turn on the predictor at t = 1750. Suddenly now the controller
finds it easier to drive the process since it needs not worry about the deadtime. Consequently the
controlled performance improves. We could further improve the tuning when using the Smith
predictor, but then in this case the uncompensated controlled response would be unstable. In
the simulation given in Fig. 4.49, there is no model/plant mis-match, but it is trivial to re-run
the simulation with a different dead time in the model, and test the robustness capabilities of the
Smith Predictor.

Fig. 4.50 extends this application to the control of an actual plant, (but with artificially added
deadtime simply because the plant has little natural deadtime) and a deliberately poor model
(green box in Fig. 4.50(a)). Fig. 4.50(b) demonstrates that turning on the Smith predictor at t = 100
shows a big improvement in the controlled response. However the control input shows a little
too much noise.
4.10. TUNING AND SENSITIVITY OF CONTROL LOOPS 191

Smith predictor on at t=2100


output & setpoint 2

1
Smith on
2
2

1
Input u

1
Smith on
2
0 500 1000 1500 2000 2500 3000 3500 4000
time

Figure 4.49: Dead time compensation. The Smith predictor is turned on at t = 2100 and the
controlled performance subsequently rapidly improves.

4.10 T UNING AND SENSITIVITY OF CONTROL LOOPS

Fig. 4.51 depicts a plant G controlled by a controller C to a reference r in the presence of process
disturbances v and measurement noise w. A good controller must achieve a number of differ-
ent aims. It should encourage the plant to follow a given setpoint, it should reject disturbances
introduced by possible upstream process upsets and minimise the effect of measurement noise.
Finally the performance should be robust to reasonable changes in plant dynamics.

The output Y (s) from Fig. 4.51 is dependent on the three inputs, R(s), V (s) and W (s) and using
block diagram simplification, is given by
CG G CG
Y (s) = R(s) + V (s) W (s) (4.88)
1 + CG 1 + CG 1 + CG
Ideally a successful control loop is one where the error, E = R Y , is small, or
1 G CG
E(s) = R(s) V (s) + W (s) (4.89)
1 + CG 1 + CG 1 + CG
We can simplify this expression by defining the open loop transfer function, or sometimes referred
to just as the loop transfer function, as
L(s) = C(s)G(s) (4.90)
which then means Eqn. 4.89 is now
1 G L
E(s) = R(s) V (s) + W (s) (4.91)
1+L 1+L 1+L

By rearranging the blocks in Fig. 4.51, we can derive the effect of the output on the error, the input
on the setpoint and so on. These are known as sensitivity functions and they play an important
role in controller design.

The sensitivity function is defined as the transfer function from setpoint to error
1
S(s) = = Ger (s) (4.92)
1 + L(s)
192 CHAPTER 4. THE PID CONTROLLER

PID In1 Out1

PID Controller Transport Scope


Signal
blackbox Delay
Generator

model
Delay model
Switch Step
1
s+1

(a) Deadtime compensation using S IMULINK and the real-time toolbox

Smith predictor off


0.5
setpoint & output

0.5
Smith predictor on

1
0 20 40 60 80 100 120 140 160 180 200

0.5
input

0.5

1
0 20 40 60 80 100 120 140 160 180 200
time (s)

(b) Controlled results without, and with, deadtime compensation.

Figure 4.50: Deadtime compensation applied to the blackbox. The compensator is activated at
time t = 100 after which the controlled performance improves.

process disturbance
v

u
r(t) C(s) G(s) y(t)

Controller plant
w
Measurement noise

Figure 4.51: Closed loop with plant G(s) and controller C(s) subjected to disturbances, v, and
measurement noise, w.
4.11. SUMMARY 193

The complementary sensitivity function is defined as the transfer function from reference to out-
put
L(s)
T (s) = = Gyr (s) = Gyw (s) (4.93)
1 + L(s)
The disturbance sensitivity function is

G(s)
GS(s) = = Gyv (s) (4.94)
1 + L(s)

and the control sensitivity function is

C(s)
CS(s) = = Gur (s) = Guw (s) (4.95)
1 + L(s)

The error from Eqn. 4.91 (which we desire to keep as small as practical), now in terms of the the
sensitivity and complementary sensitivity transfer functions is

E(s) = S(s)R(s) S(s)G(s)V (s) + T (s)W (s) (4.96)

If we are to keep the error small for a given plant G(s), then we need to design a controller C(s)
to keep both S(s) and T (s) small. However there is a problem because a direct consequence of
the definitions in Eqn. 4.92 and Eqn. 4.93 is that

S(s) + T (s) = 1

which means we cannot keep both small simultaneously. However we may be able to make S(s)
small over the frequencies when R(s) is large, and T (s) small when the measurement noise W (s)
dominates. In many practical control loops the servo response R(s) dominates at low frequencies,
and the measurement noise dominates the high frequencies. This is known as loop shaping.

We are primarily interested in the transfer function T (s), since it describes how the output changes
given changes in reference command. At low frequencies we demand that T (s) = 1 which means
that we have no offset at steady-state and that we have an integrator somewhere in the process.
At high frequencies we demand that T (s) 0 which means that fast changes in command signal
are ignored.

Correspondingly we demand that since S(s) + T (s) = 1, then S(s) should be small at low fre-
quencies, and 1 for high frequencies such as shown in Fig. 4.52.

The maximum values of |S| and |T | are also useful robustness measures. The maximum sensitiv-
ity function
kSk = max |S(j)| (4.97)

is shown as a in Fig. 4.52 and is inversely proportional to the minimal distance from the loop
transfer function to the critical (1, 0i) point. Note how the circle of radius 1/||S|| centered at
(1, 0i) just touches the Nyquist curve shown in Fig. 4.53. A large peak in the sensitivity plot in
Fig. 4.52, corresponds to a small distance between the critical point and the Nyquist curve which
means that the closed loop is sensitive to modelling errors and hence not very robust. Values of
kSk = 1.7 are considered reasonable, [108].

4.11 S UMMARY

PID controllers are simple and common in process control applications. The three terms of a PID
controller relate to the magnitude of the current error, the history of the error (integral), and the
194 CHAPTER 4. THE PID CONTROLLER

Bode plot of S & T


1
10
||S||

0
10

Magnitude
1
10

Sensitivity, S(s)
2
10
Complementary, T(s)

S
Figure 4.52: Typical sensitiv-
T
ity transfer functions S(s) and 3
10
T (s). The maximum of the |S| 2
10 10
1
10
0
10
1

is marked . [rad/s]

Nyquist plot of L(s)


1
circle radius=1/||S||

0.5

0.5

1
{L(i )}

1.5

2.5

3.5
Figure 4.53: A circle of radius 1/kSk cen-
tered at (1, 0i) just touches the Nyquist 4
3 2 1 0 1 2 3
curve. See also Fig. 4.52. {L(i )}

current direction of the error (derivative). The integral component is necessary to remove offset,
but can destabilise the closed loop. While this may be countered by adding derivative action,
noise and abrupt setpoint changes may cause problems. Commercial PID controllers usually add
a small time constant to the derivative component to make it physically realisable, and often only
differentiate the process output, rather than the error.

Many industrial PID controllers are poorly tuned which is clearly uneconomic. Tuning controllers
can be difficult and time consuming and there is no correct fail-safe idiot-proof procedure. The
single experiment in closed loop (Yuwana-Seborg) is the safest way to tune a critical loop, al-
though the calculations required are slightly more complicated than the traditional methods and
is designed for processes that approximate a first-order system with time delay. The tuning pa-
rameters obtained by all these methods should only be used as starting estimates. The fine tuning
of the tuning parameters is best done by an enlightened trial and error procedure on the actual
equipment.
C HAPTER 5

D IGITAL FILTERING AND


SMOOTHING

Never let idle hands get in the way of the devils work.
Basil Fawlty (circ. 75)

5.1 I NTRODUCTION

This chapter is concerned with the design and use of filters. Historically analogue filters were the
types of filters that electrical engineers have been inserting for many years in radios fabricated out
of passive components such as resisters, capacitors and inductors or more recently and reliably
out of operational amplifiers. When we filter something, be it home made wine, sump oil, or
undesirable material on the Internet, we are really interested in purifying, or separating the noise
or unwanted, from the desired.

The classical approach to filtering assumes that the useful signals lie in one frequency band, and
the unwanted noise in another. Then simply by constructing a transfer function as our filter,
we can pass the wanted band (signal), while rejecting the unwanted band (noise) as depicted in
Fig. 5.1.

input, u(t) Filter, H(s) output, y(t)


(noisy) (smooth)

low-pass filter

Figure 5.1: A filter as a low-pass transfer function

Unfortunately even if the frequency bands of the signal and noise are totally distinct, we still can-
not completely separate the signals in an online filtering operation. In practice however, we can
design the filters with such sharp cut-offs that a practical separation is achieved. More commonly
the signal is not distinct from the noise (since the noise tends to inhabit all frequency bands), so
195
196 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

now the filter designer must start to make some trade-offs.

The more sophisticated model-based filters that require computer implementation are described
in chapter 9. While the classical filters can be realised (manufactured) with passive analogue
components such as resisters, capacitors, and more rarely inductors, today they are almost always
implemented digitally. This chapter describes these methods. The S IGNAL P ROCESSING toolbox
contains many useful filter design tools which we will use in this chapter, although the workhorse
function for this chapter, filter, is part of M ATLABs kernel.

5.1.1 T HE NATURE OF INDUSTRIAL NOISE

Real industrial measurements are always corrupted with noise. This noise (or error) can be at-
tributed to many causes, but even if these causes are known, the noise is usually still unpre-
dictable. Causes for this noise can include mechanical vibrations, poor electrical connections
between instrument and transducer sensor, electrical interference of other equipment or a combi-
nation of the above. What ever the cause, these errors are usually undesirable. Remember, if it is
not unpredictable, then it is not noise.

An interesting example of the debilitating affect of noise is shown in the controlled response in
Fig. 5.2. In this real control loop, the thermocouple measuring the temperature in a sterilising
furnace was disturbed by a mobile phone a couple of meters away receiving a call. Under normal
conditions the standard deviation of the measurement is around 0.1 degrees which is acceptable,
but with the phone operating, this increases to 2 or 3 degrees.

118
temperature [deg C]

116

114

112
mobile phone range here
110

1
Power [%]

0.5

Figure 5.2: A noisy temperature mea-


surement due to a nearby mobile 0
phone causes problems in the feed- 0 5 10 15 20 25
back control loop. time [min]

We can mathematically describe the actual measurement process as

y = y + v

where y is the true value which is corrupted with some other, probably random signal which we
call v to give the as received measured value y.

Naturally we never know precisely the signals v or y . The whole point about filtering or signal
processing is to try and distinguish the underlying true signal, y from the noise, v. There is
another reason why one should study filtering, and that is to disguise horrible experimental data.
As one operator said, I control the plant by looking at trends of the unfiltered data, but I only
give filtered data to management.
5.1. INTRODUCTION 197

true signal, y b

measurement noise, v

corrupted measured signal, y

Figure 5.3: Noise added to a true, but unknown, signal

While the noise itself is unpredictable (non deterministic) it often has constant, known or pre-
dictable statistical properties. The noise is often described by a Gaussian distribution, with known
and stable statistical properties such as mean and variance.

Noise is generally considered a nuisance because

1. It is not a true reflection of the actual process variables, since the transducer, or other external
factors have introduced unwanted information.
2. It makes it difficult to differentiate the data accurately such as say for the D part of a PID
controller, or extracting the flowrate from the rate of change of a level signal.
3. Often for process management applications, only an average value or smoothed character-
istic value is required.
4. Sometimes noise is not undesirable! In some specialised circumstances noise can actually
counter-intuitively improve the strength of a signal. See the article entitled The Benefits of
Background Noise [146] for more details of this paradox.

Some care should be taken with point 1. No one really knows what the true value of the process
is (except possibly Mother Nature), so the assertion that a particular time series is not true is
difficult to rigorously justify. However for most applications, it is a reasonable and defendable
assumption. When filtering the data for process control applications, it is often assumed that the
true value is dominated by low frequency dynamics (large time constants etc), and that anything
at a high frequency is unwanted noise. This is a fair assumption for measurements taken on
most equipment in the processing industries. Large industrial sized vessels, reactors, columns etc
often have large holding volumes, large capacities, and hence slow dynamics. This tendency to
favour the low frequencies by low-pass filtering everything, while common for chemical process
engineers, is certainly not the norm in say telecommunications.

Differentiating (i.e. estimating the slope of) noisy data is difficult. Conversely, integrating noisy
data is easy. Differentiating experimental data is required for the implementation of the D part
of PID controllers, and in the analysis of rate data in chemical reaction control amongst other
things. Differentiation of actual industrial data from a batch reaction is demonstrated in 5.5, and
as a motivating example, differentiating actual data without smoothing first is given in 5.1.2.

5.1.2 D IFFERENTIATING WITHOUT SMOOTHING

As seen in chapter 4, the Proportional-Integral-Differential controller uses the derivative compo-


nent, or D part, to stabilise the control scheme and offers improved control. However in practice,
the D part is rarely used in industrial applications, because of the difficulty in differentiating the
raw measured signal in real-time.
198 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

The electromagnetic balance arm shown in Fig. 1.6 is a very oscillatory device, that if left uncom-
pensated would wobble for a considerable time. If we were to compensate it with a PID controller
such that the balance arm closely follows the desired setpoint, we find that we would need sub-
stantial derivative action to counter the oscillatory poles of the plant. The problem is how we can
reliably generate a derivative of a noisy signal for out PID controller.

Fig. 5.4 demonstrates experimentally what happens when we attempt to differentiate a raw and
noisy measurement, using a crude backward difference with a sample time T
dy yt yt1

dt T
The upper plot shows the actual position of the balance arm under real-time closed-loop PID
control using a PC with a 12 bit (4096 discrete levels) analogue input/output card. In this instance,
I tuned the PID controller with a large amount of derivative action, and it is this that dominates
the controller output (lower plot). While the controlled response is reasonable, (the arm position
follows the desired setpoint, the input is too noisy), and will eventually wear the actuators if used
for any length of time. Note that this noise is a substantial fraction of the full scale input range.

2500
setpoint & output

2000

1500

1000
5000

4000

3000
Input

2000

1000

0
0 5 10 15 20 25 30
time [s]

Figure 5.4: The significant derivative action of a PID controller given the slightly noisy data from
an electromagnetic balance arm (solid line in upper plot) causes the controller input signal (lower
plot) to go exhibit very high frequencies.

One solution to minimise the excitedness of the noisy input signal whilst still retaining the dif-
ferentiating characteristic of the controller is to low-pass filter (smooth) the measured variable
before we differentiate it. This is accomplished with a filter, and the design of these filters is what
this chapter is all about. Alternatively the controller could be completely redesigned to avoid the
derivative step.

5.2 S MOOTHING MEASURED DATA USING ANALOGUE FILTERS

If we have all the data at our disposal, we can draw a smooth curve through the data. In the
past, draughtsmen may have used a flexible ruler, or spline, to construct a smoothing curve by
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 199

eye. Techniques such as least squares splines or other classes of piecewise low-order polynomials
could be used if a more mathematically rigorous fitting is required. Techniques such as these fall
into the realm of regression and examples using M ATLAB functions such as least-squares fitting,
splines, and smoothing splines are covered more in [206]. If the noise is to be discarded and
information about the signal in the future is used, this technique is called smoothing. Naturally
this technique can only be applied offline after all the data is collected since the future data
must be known. A smoothing method using the Fourier transform is discussed in 5.4.4.

Unfortunately for real time applications such as the PID controller application in 5.1.2, we cannot
look into the future and establish what the signal will be for certain. In this case we are restricted
to smoothing the data using only output historical values. This type of noise rejection is called
real-time filtering and it is this aspect of filtering that is of most importance to control engineers.
Common analogue filters are discussed in 5.2.3, but to implement using digital hardware we
would rather use an equivalent digital description. The conversion from the classical analogue
filter to the equivalent digital filter using the bilinear transform is described in 5.3.

Finally we may wish to predict data in the future. This is called prediction.

5.2.1 A SMOOTHING APPLICATION TO FIND THE PEAKS AND TROUGHS

Section 4.6.3 described one way to tune a PID controller where we first to subject the plant to a
closed loop test with a trial gain, and then record the peaks and troughs of the resultant curve.
However as shown in the actual industrial example in Fig. 5.5(a), due to excessive noise on the
thermocouple (introduced by mobile phones in the near vicinity!), it is difficult to extract the
peaks and troughs from the measured data. It is especially difficult to write a robust algorithm
that will do this automatically with such noise.

The problem is that to achieve a reasonable smoothing we also introduce a large lag. This in turn
probably does not affect the magnitude of the peaks and troughs, but it does affect the timing, so
our estimates of the model parameters will be wrong.

One solution is to use an acausal filter which does not exhibit any phase lag. M ATLAB provides
a double-pass filter called filtfilt which is demonstrated in Fig. 5.5(b). Using the smooth
data from this acausal filter generates reasonable estimates for both the magnitude and timing of
the characteristic points needed for the tuning. The drawback is that the filtering must be done
off-line.

5.2.2 F ILTER TYPES

There are four common filter types: the low-pass, high-pass, band-pass and notch-pass. Each of
these implementations can be derived from the low-pass filter, meaning that it is only necessary
to study in detail the design of the first. A comparison of the magnitude response of these filters
is given in Fig. 5.8.

L OW- PASS FILTERS

A linear filter is mathematically equivalent to a transfer function, just like a piece of processing
equipment such as a series of mixing tanks, or distillations columns. Most of the filter applica-
tions used in process industry are low-pass in nature. That is, ideally the filter passes all the low
frequencies, and above some cut off frequency c , they attenuate the signal completely. These
200 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

Output & setpoint


4

2
raw value
0
Butterworth filter smoothed, n=5, c=0.03
Key points
2
0 100 200 300 400 500 600 700 800
time [s]

(a) To extract the key points needed for PID tuning, we must first
smooth the noisy data. Unfortunately, to achieve good smoothing
using a causal filter such as the 5th order Butterworth used here,
we introduce a large lag.

6
Output & setpoint

raw value
0
Acausal smoothed
Key points
2
0 100 200 300 400 500 600 700 800
time [s]

(b) Using an acausal filter removes the phase lag problem

Figure 5.5: In this industrial temperature control of a furnace, we need to smooth the raw mea-
surement data in order to extract characteristic points required for PID tuning.

types of filters are used to smooth out noisy data, or to retain the long term trends, rather than
the short term noisy contaminated transients.

The ideal low pass filter is a transfer function that has zero phase lag at any input frequency, and
an infinitely sharp amplitude cut off above a specified frequency c . The Bode diagram of such a
filter would have a step function down to zero on the amplitude ratio plot and a horizontal line
on the phase angle plot as shown in Fig. 5.8. Clearly it is physically impossible to build such a
filter. However, there are many alternatives for realisable filters that approximate this behaviour.
Fig. 5.6 shows a reasonable attempt to approximate the ideal low-pass filter. Typically the cus-
tomer will specify a cut-off frequency, and it is the task of the filter designer to select sensible
values for the pass and stop band oscillations or ripple and the pass and stop band frequencies to
achieve these balancing filter performance with minimal hardware cost.

The most trivial low-pass filter approximation to the ideal filter is a single lag with a cut off
frequency (c ) of 1/ . If you wanted a higher order filter, say a nth order filter, you could simply
cascade n of these filters together as shown in Fig. 5.7.

1
Gn (s) =  n (5.1)
s
c +1

Actually Eqn. 5.1 is a rather poor approximation to the ideal filter and better approximations exist
for the same complexity or filter order, although they are slightly more difficult to design and
implement in hardware. Two classic analogue filter designs are described in 5.2.3 following.
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 201

|H(i)|

1
pass band ripple
1
1+2

pass band stop band

1
A2

0
0 p s

Figure 5.6: Low-pass filter specification

cascaded 3rd order low-pass filter

1 1 1
Input, u s/c +1 s/c +1 s/c +1 output, y

single low-pass filter

Figure 5.7: Three single low-pass filters cascaded together to make a third-order filter.

H IGH - PASS FILTERS

There are however applications where high pass filters are desired, although these are far more
common in the signal processing and electronic engineering fields. These filters do the opposite
of the low pass filter; namely they attenuate the low frequencies and pass the high frequencies
unchanged. This can be likened to the old saying If Ive already seen it, Im not interested!
These remove the DC component of the signal and tend to produce the derivative of the data.
You may use a high pass filter if you wanted to remove long term trends from your data, such as
seasonal effects for example.

B AND AND NOTCH - PASS FILTERS

The third type of filter is a combination of the high and low pass filters. If these two are combined,
they can form a band pass or alternatively a notch filter. Notch filters are used to remove a par-
ticular frequency such as the common 50Hz hum caused by the mains power supply. Band pass
filters are used in the opposite manner, and are used in transistor radio sets so that the listener can
tune into one particular station without hearing all the other stations. The difference between the
202 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

expensive radios and the cheap ones, is that the expensive radios have a sharp narrow pass band
so as to reduce the amount of extra frequency. Oddly enough for some tuning situations a notch
filter is also used to remove neighbouring disturbing signals. Some satellite TV stations introduce
into the TV signal a disturbing component to prevent non-subscribers from watching their trans-
mission without paying for the secret decoder ring. This disturbance, which essentially makes
the TV unwatchable, must of course must be within the TV stations allowed frequency range.
The outer band pass filter will tune the TV to the stations signal, and a subsequent notch filter
will remove the disturbing signal. Fig. 5.8 contrasts the amplitude response of these filters.

Ideal low-pass fiter Actual low-pass filter High-pass filter Band-pass filter
AR AR AR AR

, [rad/s]
c c c l h

Figure 5.8: Amplitude response for ideal, low-pass, high pass and band-pass filters

F ILTER TRANSFORMATION

Whatever the type of filter we desire, (high-pass, band-pass notch etc), they can all be obtained
by first designing a normalised low-pass filter prototype as described by [162, p415]. This filter
is termed normalised because it has a cutoff frequency of 1 rad/s. Then the desired filter, high-
pass, band-pass or whatever, is obtained by transforming the normalised low-pass filter using
the relations given in Table 5.1 where u is the upper cutoff and l is the lower cutoff frequencies.
This is very convenient, since we need only to be able to design a low-pass filter from which all

Table 5.1: The filter transformations needed to convert from the prototype filter to other general
filters. To design a general filter, replace the s in the prototype filter with one of the following
transformed expressions.

Desired filter transformation


low pass s s/u
s2 + u l
band-pass s
s(u l )
s(u l )
band-stop s 2
s + u l
high pass s u /s

the others can be obtained using a variable transformation. Low-pass filter design is described
for the common classical analogue filter families in 5.2.3. M ATLAB also uses this approach in the
S IGNAL P ROCESSING toolbox with what it terms analogue low-pass proto-type filters.
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 203

5.2.3 C LASSICAL ANALOGUE FILTER FAMILIES

Two common families of analogue filters superior to the simple cascaded first-order filter network
are the Butterworth and the Chebyshev filters. Both these filters require slightly more computa-
tion to design, but once implemented, require no more computation time or hardware compo-
nents than any other filter of the same order. Historically these were important continuous-time
filters, and to use discrete filter designs we must convert the description into the z plane by using
the bilinear transform.

In addition to the Butterworth and Chebyshev filters, there are two slightly less common clas-
sic analogue filters; the Chebyshev type II (sometimes called the inverse Chebyshev filter), and
the elliptic or Cauer filter. All four filters are derived by approximating the sharp ideal filter in
different ways. For the Butterworth, the filter is constructed in such a way as to maximise the
number of derivatives to zero at = 0 and = . The Chebyshev filter exhibits minimum error
over the passband, while the type II version minimises the error over the stop band. Finally the
elliptic filter uses a Chebyshev mini-max approximation in both pass-band and stop-band. The
calculation of the elliptic filter requires substantially more computation than the other three. The
S IGNAL P ROCESSING toolbox contains both discrete and continuous design functions for all four
analogue filter families.

B UTTERWORTH FILTERS

The Butterworth filter, HB , is characterised by the following magnitude relation

1
|HB ()|2 =  2n (5.2)

1+ c

where n is the filter order and c is the desired cut off frequency. If we substitute s = i, we find
that the 2n poles of the squared magnitude function, Eqn. 5.2, are
1
sp = (1) 2n (ic )

That is, they are spaced equally at /n radians apart around a circle of radius c . Given that
we desire a stable filter, we choose just the n stable poles of the squared magnitude function for
HB (s). The transfer function form of the general Butterworth filter in factored form is
n
Y 1
HB (s) = (5.3)
s + pk
k=1

where   
1 2k 1
pk = c exp i + (5.4)
2 2n
Thus all the poles are equally spaced apart on the stable half-circle of radius c in the s plane.
The M ATLAB function buttap, (Butterworth analogue low pass filter prototype), uses Eqn. 5.4 to
generate the poles.

We can plot the poles of a 5th order Butterworth filter after converting the zero-pole-gain form to
a transfer function form, using the pole-zero map function, pzmap, or of course just plot the poles
directly.

1 [z,p,k] = buttap(5) % Design a 5th order Butterworth proto-type filter


pzmap(zpk(z,p,k)) % Look at the poles & zeros
204 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

Note how the poles of the Butterworth filter are equally spread around the stable half circle in
Fig. 5.9. Compare this circle with the ellipse pole-zero map for the Chebyshev filter shown in
Fig. 5.10.

0.5

(p)
0

0.5

Figure 5.9: Note how the poles of the Butterworth filter, 1


, are equally spaced on the stable half circle. See also
1 0.5 0 0.5 1
Fig. 5.10. (p)

Alternatively using only real arithmetic, the n poles of the Butterworth filter are pk = ak bk i
where
ak = c sin , bk = c cos (5.5)
where

= (2k 1) (5.6)
2n
Expanding the polynomial given in Eqn. 5.3 gives the general Butterworth filter template in the
continuous time domain,
1
HB (s) =  n  n1 (5.7)
s s
cn c + cn1 c + + c1 sc + 1

where ci are the coefficients of the filter polynomial. Using Eqn 5.4, the coefficients for the general
nth order Butterworth filter can be evaluated.

Taking advantage of the complex arithmetic abilities of M ATLAB, we can generate the poles typing
Eqn. 5.4 almost verbatim. Note how I use 1i, (that is I type the numeral 1 followed with an i
with no intervening space) to ensure I get a complex number.

Listing 5.1: Designing Butterworth Filters using Eqn. 5.4.


>>n = 4; % Filter order
>>wc = 1.0; % Cut-off frequency, c
3 >>k = [1:n]';
1 2k1
 
>>p = wc*exp(-pi*1i*(0.5 + (2*k-1)/2/n)); % Poles, pk = c exp i 2
+ 2n
>>c = poly(p); % should force real
>>c = real(c) % look at the coefficients
c =
8 1.0000 2.6131 3.4142 2.6131 1.0000

>>[B,A] = butter(n,wc,'s'); % compare with toolbox direct method

Once we have the poles, it is a simple matter to expand the polynomial out and find the coeffi-
cients. In this instance, owing to numerical roundoff, we are left with a small complex residual
which we can safely delete. The 4th order Butterworth filter with a normalised cut-off frequency
is
1
HB (s) = 4
s + 2.6131s + 3.4142s2 + 2.6131s + 1
3
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 205

Of course using the S IGNAL P ROCESSING toolbox, we could duplicate the above with a single call
to butter given as the last line in the above example.

We can design a low-pass filter at an arbitrary cut-off frequency by scaling the normalised ana-
logue proto-type filter using Table 5.1. Listing 5.2 gives the example of the design of a low-pass
filter with a cutoff of fc = 800 Hz.

Listing 5.2: Designing a low-pass Butterworth filter with a cut-off frequency of fc = 800 Hz.
ord = 2; % Desired filter order
[Z,P,K] = buttap(ord); Gc = tf(zpk(Z,P,K)); % 2nd-order Butterworth filter prototype

Fc = 800; % Desired cut-off frequency in [Hz]


5 wc = Fc*pi*2 ; % [rad/s]

n = [ord:-1:0]; % Scale filter by substituting s for s/c


d = wc.n;
Gc.den{:} = Gc.den{:}./d
10

[B,A] = butter(2,wc,'s'); Gc2 = tf(B,A); % Alternative continuous design (in Matlab)

p=bodeoptions; p.FreqUnits = 'Hz'; % Set Bode frequency axis units as Hz (not rad/s)
bode(Gc,Gc2,p)
15 hline(-90); vline(Fc)

A high pass filter can be designed in much the same way, but this time substituting c /s for s.

Listing 5.3: Designing a high-pass Butterworth filter with a cut-off frequency of fc = 800 Hz.
ord = 2; [Z,P,K] = buttap(ord); Gc = tf(zpk(Z,P,K)); % Low-pass Butterworth proto-
type

Fc = 800;wc = Fc*pi*2 ; % Frequency specifications

5 n = [0:ord]; d = wc.n; % Scale filter by converting s to c /s


Gc.den{:} = Gc.den{:}.*d;
Gc.num{:} = [1,zeros(1,ord)];

bode(Gc,p); title('High pass')

The frequency response of the Butterworth filter has a reasonably flat pass-band, and then falls
away monotonically in the stop band. The high frequency asymptote has a slope of n on a log-
log scale. Other filter types such as the Chebyshev (see following section) or elliptic filters are
contained in the S IGNAL P ROCESSING toolbox, and [44] details the use of them.

Algorithm 5.1 Butterworth filter design


Given a cut-off frequency, c , (in rad/s), and desired filter order, n,

1. Compute the n poles of the filter, HB (s), using


  
1 2k 1
pk = c exp i + , k = 1, 2, . . . , n
2 2n

2. Construct the filter HB (s) either using the poles directly in factored form, or expanding the
polynomial
1
HB (s) =
poly(pk )
206 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

3. Convert low pass filter to high-pass, band pass etc by substituting for s given in Table 5.1.
4. Convert to a discrete filter using c2dm if desired ensuring that the cut-off frequency is much
less than the Nyquist frequency; c << N = /T .

Algorithms 5.1 suffers from the flaw that it assumes the filter designer has already selected a
cut-off frequency, c , and filter order, n. However for practical cases, it is characteristics such
as band-pass ripple, stopband attenuation that are specified by the customer such as given in
Fig. 5.6, and not the filter order. We need some way to calculate the minimum filter order (to
construct the cheapest filter) from the specified magnitude response characteristics. Once that is
established, we can proceed with the filter design calculations using the above algorithm. Further
descriptions of the approximate order are given in [92, pp9305327] and [198]. The functionally
equivalent routines, buttord and cheb1ord in the S IGNAL P ROCESSING toolbox attempt to find
a suitable order for a set of specified design constraints.

In the case for Butterworth filters, the filter order is given by


&    '
log10 10Rp /10 1 10As /10 1
n= (5.8)
2 log10 (p /s )

where Rp is the pass-band ripple, As is the stopband ripple, and p , s are the pass and stop band
frequencies respectively. In general, the order n given by the value inside the brackets (without
feet) in Eqn. 5.8 will not be an exact integer, so we should select the next largest integer to ensure
that we meet or exceed the customers specifications. M ATLABs ceiling function ceil is useful
here.

Problem 5.1 1. Derive the coefficients for a sixth order Butterworth filter.
2. Draw a Bode diagram for a fourth order Butterworth filter with wc = 1 rad/s. On the same
plot, draw the Bode diagram for the filter
1
H(s) =  4
s
c +1

with the same wc . Which filter would you choose for a low pass filtering application?
Hints: Look at problem A-4-5 (DCS) and use the M ATLAB function bode. Plot the Bode
diagrams especially carefully around the corner frequency. You may wish to use the Matlab
commands freqs or freqz.
3. Find and plot the poles of a continuous time 5th order Butterworth filter. Also plot a circle
centered on the origin with a radius wc .
4. Plot the frequency response (Bode diagram) of both a high pass and low pass 4th order
Butterworth filter with a cutoff frequency of 2 rad/s.

C HEBYSHEV FILTERS

Related to the Butterworth filter family is the Chebyshev family. The squared magnitude function
of the Chebyshev filter is
1
|HC ()|2 =   (5.9)
1 + 2 Cn2 c

where again n is the filter order, c is the nominal cut-off frequency, and Cn is the nth order
Chebyshev polynomial. (For a definition of Chebyshev polynomials, see for example [206], and in
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 207

particular the cheb_pol function.) The design parameter is related to the amount of allowable
passband ripple, ,
1
=1 (5.10)
1 + 2
Alternatively the filter design specifications may be given in decibels of allowable passband rip-
ple1 , rdB ,
rdB
= 1 10 20 (5.11)
This gives the Chebyshev filter designer two degrees of freedom, (filter order n and or equiva-
lent), rather than just order as in the Butterworth case. Similar to the Butterworth filter, the poles
of the squared magnitude function, Eqn. 5.9, are located equally spaced along an ellipse in the s
plane with minor axis radius of c aligned along the real axis, where

1  1/n 
= 1/n (5.12)
2
with
1 p
= + 1 + 2 (5.13)

and a major axis radius of c aligned along t2he imaginary axis where

1  1/n 
= + 1/n (5.14)
2

Since the poles pk of the Chebyshev filter, Hc (s) are equally spaced along the stable half of the
ellipse, (refer Fig. 5.10 for a verification of this), the real and imaginary parts are given by:
 
(pk ) = c cos 2 + (2k+1) 2n


(2k+1)
 k = 0, 1, . . . , n 1 (5.15)
(pk ) = c sin 2 + 2n

and the corresponding transfer function is given by

K
Hc (s) = Qn
k=0 (s pk )

where the numerator K is chosen to make the steady-state equal to 1 if n is odd, or 1/ 1 + 2 for
even n. Algorithm 5.2 summarises the design of a Chebyshev filter with ripple in the passband.

Algorithm 5.2 Chebyshev type I filter design


Given a cut-off frequency, c , passband ripple, , or rdB , and order filter n,

1. Calculate the ripple factor


s
p 1
= 10r/10 1 = 1
(1 )2

2. Calculate the radius of the minor, , and major axis of the ellipse using Eqns 5.125.14.

3. Calculate the n stable poles from Eqn. 5.15 and expand out to form the denominator of
Hc (s).
1 The functions in the S IGNAL P ROCESSING T OOLBOX use rdB rather than .
208 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

4. Choose the normalising gain K such that at steady-state (DC gain)



1 , n even
Hc (s = 0) = 1+2
1, n odd

We can calculate the gain of a transfer function using the final value theorem which assum-
ing a unit step gives
 a0

1+2
, n even
K=
a0 , n odd
where a0 is the coefficient of s0 in the denominator of Hc (s).

We can test this algorithm by designing a Chebyshev type I filter with 3dB ripple in the passband,
(rdB = 3dB or = 1 103/20 ), and a cross-over frequency of c = 2 rad/s.

Listing 5.4: Designing Chebyshev Filters


1 n = 4; wc = 2; n, c
% Order & cut-off frequency,
r = 3; % Ripple. Note 3dB = 1/ 2
rn = 1/n;
e = sqrt(10(r/10)-1); % = 1 10r/20 , e = 1/(1 )2 1
p

6 g = 1/e + sqrt(1 + 1/e/e); minora = (grn - g(-rn))/2;


majorb = (grn + g(-rn))/2;

k = [0:n-1]'; n2 = 2*n;
realp = minora*wc*cos(pi/2 + pi*(2*k+1)/n2); % real parts of the poles
11 imagp = majorb*wc*sin(pi/2 + pi*(2*k+1)/n2); % imaginary parts
polesc = realp + 1i*imagp; % poles of the filter
Ac = real(poly(polesc)); % force real
Bc = Ac(n+1); % overall DC gain = 1 (n =odd)
if rem(n,2) % if filter order n is even
16 Bc = Bc/sqrt(1+e2); % dc gain
end % if

You may like to compare this implementation with the equivalent S IGNAL P ROCESSING toolbox
version of the type 1 Chebyshev analogue prototype by typing type cheb1ap.

We can verify that the computed poles do indeed lie on the stable half of the ellipse with major
and minor axes and respectively in Fig. 5.10. I used the axis(equal) command for
this figure forcing the x and y axes to be equally, as opposed to conveniently, spaced to highlight
the ellipse. Compare this with the poles of a Butterworth filter plotted in Fig. 5.9.

As the above relations indicate, the generation of Chebyshev filter coefficients is slightly more
complex than for Butterworth filters, although they are also part of the S IGNAL P ROCESSING
toolbox. Further design relations are given in [156, p221]. We can compare our algorithm with
the equivalent toolbox function cheby1 which also designs type I Chebyshev filters. We want the
continuous description, hence the optional s parameter. Alternatively we could use the more
specialised cheb1ap function which designs an analogue low-pass filter prototype which is in
fact called by cheby1.

Listing 5.5: Computing a Chebyshev Filter


n = 4; wc = 2; cut-off, c = 2 rad/s
% Order of Chebyshev filter;
r = 3; % dB of ripple, 3dB = 1/ 2
3 [Bc,Ac] = cheby1(n,r,wc,'s'); % design a continuous filter
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 209

1.5

0.5
(p)

0.5

1.5

2 Figure 5.10: Verifying that the poles of a fourth-order


2 1 0 1 2 analogue type 1 Chebyshev filter are equally spaced
(p) around the stable half of an ellipse.

[Hc,w] = freqs(Bc,Ac); % calculate frequency response.


semilogx(w,abs(Hc)); % magnitude characteristics

These filter coefficients should be identical to those obtained following Algorithm 5.2. You should
exercise some care with attempting to design extremely high order IIR filters. While the algorithm
is reasonably stable from a numerical point of view, if we restrict ourselves just to the factored
form, one may see unstable poles once we have expanded the polynomial owing to numerical
round-off for orders larger than about 40.

>> [Bc,Ac] = cheby1(50,3,1,'s'); % Very high order IIR filter


>> max(real(roots(Ac)))
ans =
0.0162 % +ve, unstable RHP pole
5 >> [z,p,g] = cheb1ap(50,3); % repeat, but only calculate poles
>> max(real(p)) % are any RHP?
ans =
-5.5478e-004 % All poles stable

Of course in applications while it is common to require high order for FIR filters, it is unusual that
we would need larger than about n = 10 for IIR filters.

Fig. 5.11 compares the magnitude of both the Butterworth and Chebyshev filter and was gener-
ated using, in part, the above code. For this plot, I have broken from convention, and plotted the
magnitude on a linear rather than logarithmic scale to emphasize better the asymptote towards
zero. Note that for even order Chebyshev filters such as the 4th order presented in Fig. 5.11,
the steady-state gain is not equal to 1, but rather 1 the allowable pass-band ripple. If this is a
problem, one can compensate by increasing the filter gain. For odd order Chebyshev filters, the
steady-state gain is 1 which perhaps is preferable.

The difference between a maximally flat passband filter such as the Butterworth and the equal
ripple filter such as the Chebyshev is clear. [135, p176] claim that most designers prefer the Cheby-
shev filter owing to the sharper transition across the stopband, provided the inevitable ripple is
acceptable.

In addition to the Butterworth and Chebyshev filters, there are two more classic analogue filters;
the Chebyshev type II and the elliptic or Cauer filter. All four filters are derived by approximating
the sharp ideal filter in different ways. For the Butterworth, the filter is constructed in such a way
as to maximise the number of derivatives to zero at = 0 and = . The Chebyshev filter
exhibits minimum error over the passband, while the type II version minimises the error over the
210 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

Comparison of Butterworth & Chebyshev filters

0.8
3dB

|H()|
0.6
Butterworth
Chebychev
Figure 5.11: Comparing the magni-
0.4
tude characteristics of 4th order But-
terworth and Chebyshev filters. The
0.2
Chebyshev filter is designed to have
ideal filter
a ripple equal to 3dB (dotted line)
which is approx 30%. Both filter cut- 0
10
1
10
0 1
10
c
offs are at c = 2 rad/s. [rad/s]

stop band. Finally the elliptic filter uses a Chebyshev mini-max approximation in both pass-band
and stop-band. The calculation of the elliptic filter requires substantially more computation than
the other three, although all four can be computed using the S IGNAL P ROCESSING toolbox.

5.3 D ISCRETE FILTERS

Analogue filters are fine for small applications and for where aliasing may be a problem, but
they are more expensive and more inflexible than the discrete equivalent. We would much rather
fabricate a digital filter where the only difference between filter orders is a slightly longer tapped
delay or shift register, and proportionally slower computation, albeit with a longer word length.
Discrete filters are also easy to implement in M ATLAB using the filter command.

To design a digital filter, one option is to to start the design from the classical analogue filters
and use a transformation such as the bilinear transform described in chapter 2 to the discrete
domain. In fact this is the approach that M ATLAB uses in that it first designs the filter in the
continuous domain, then employs bilinear to convert to the discrete domain. Digital filter
design is discussed further in section 5.3.2.

Traditionally workers in the digital signal processing (DSP) area have referred to two types of fil-
ters, one where the response to an impulse is finite, the other where the response is theoretically
of infinite duration. This latter recursive filter or Infinite Impulse Response (IIR), is one where
previous filtered values are used to calculate the current filtered value. The former non-recursive
filter or Finite Impulse Response, (FIR) is one where the filtered value is only dependent on past
values of the input hence A(z) = 1. FIR filters find use in some continuous signal processing ap-
plications where the operation is to be run for very long sustained periods where the alternative
IIR filters are sensitive to numerical blow-up owing to the recursive nature of the filtering algo-
rithm. FIR filters will never go unstable since they have only zeros which cannot cause instability
by themselves. FIR filters also exhibit what is known as linear phase lag, that is each of the sig-
nals frequencies are delayed by the same amount of time. This is a desirable attribute, especially
in communications and one that a causal IIR filter cannot match.

However better filter performance is generally obtained by using IIR filters for the same number
of discrete components. In applications where the FIR filter is necessary, 50 to 150 terms are com-
mon. To convert approximately from an IIR to a FIR, divide A(z) into B(z) using long division,
and truncate the series at some suitably large value. Normally, the filter would be implemented
in a computer in the digital form as a difference equation, or perhaps in hardware using a series
of delay elements and shift registers.
5.3. DISCRETE FILTERS 211

5.3.1 A LOW- PASS FILTERING APPLICATION

Consider the situation where you are asked to filter a measurement signal from a sensitive pres-
sure transducer in a level leg in a small tank. The level in the tank is oscillating about twice a
second, but this measurement is corrupted by the mains electrical frequency of 50Hz.

Suppose we are sampling at fs = 1kHz which corresponds to a sample time of T = 0.001s, and a
Nyquist frequency of fN = fs /2 = 500Hz. We should note that this sampling rate is sufficient to
avoid aliasing problems since it is above twice the highest frequency of interest, 2 50Hz.

We can simulate this scenario in M ATLAB. First we create a time vector t and then the two signals,
one is the true measurement y1 , and the other is the corrupting noise from the mains y2 . The
actual signal, as received by your data logger, is the sum of these two signals, y = y1 + y2 . The
purpose of filtering is to extract the true signal y1 from the corrupted y as closely as possible.

1.5
Ts = 1/1000; % Sample time (sec) 1
2 t = [0:Ts:1]'; % time vector
0.5
y1 = sin(2*pi*2*t); % true signal
y2 = 0.1*sin(2*pi*50*t); % noise 0

y = y1+y2; % what we read 0.5


plot(t,y) 1

1.5
0 0.2 0.4 0.6 0.8 1

Clearly even from just the time domain plot the signal y is composed of two dominant frequencies.
If a frequency domain plot were to be created, it would be clearer still. To reconstruct the true level
reading, we need to filter the measured signal y. We could design a filter with a cut-off frequency
of fc = 30Hz. This should in theory pass the slower y1 signal, but attenuate the corrupting y2 .

To implement such a digital filter we must:

1. select a filter type, say Butterworth, (refer page 203)


2. select a filter order, say 3, and
3. calculate the dimensionless frequency related to the specified frequency cutoff fc = 30Hz,
and the sample time T , (or Nyquist frequency fN ),
fc
= = 2T fc
fN
= 2 103 30 = 0.06

We are now ready to use the S IGNAL P ROCESSING T OOLBOX to design our discrete IIR filter or
perhaps build our own starting with a continuous filter and transforming to the discrete domain
as described on page 204.

>> fc = 30 % Required cut-off frequency, [Hz]


>> wc = 2*Ts*fc; % Dimensionless cut-off frequency,
>> [B,A] = butter(3,wc)% Find A(q 1 ) & B(q 1 ) polynomials
10 B =
0.0007 0.0021 0.0021 0.0007
A =
1.0000 -2.6236 2.3147 -0.6855
>> yf = filter(B,A,y); % filter the measured data
15 >> plot(t,y,t,yf,'--',t,y1) % any better ?
212 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

Once having established the discrete filter coefficients, we can proceed to filter our raw signal and
compare with the unfiltered data as shown in Fig. 5.12.

1.4

1.2

1 raw data

0.8

0.6
6th order

0.4
3rd order
Raw data
Figure 5.12: Comparing a 3rd and 6th order 0.2
3rd or Butterworth
discrete Butterworth filters to smooth noisy 6th order Butterworth
data. 0
0 0.05 0.1 0.15 0.2

The filtered data Fig. 5.12 looks smoother than the original, but it really is not smooth enough.
We should try a higher order filter, say a 6th order filter.

[B6,A6] = butter(6,wc); % 6th order, same cut-off


yf6 = filter(B6,A6,y); % smoother ?
plot(t,[y,yf,yf6]) % look any better ?

This time the 6th order filtered data in Fig. 5.12 is acceptably smooth, but there is quite a noticeable
phase lag such that the filtered signal is behind the original data. This is unfortunate, but shows
that there must be a trade off between smoothness and phase lag for all causal or online filters.

The difference in performance between the two digital filters can be shown graphically by plotting
the frequency response of the filter. The easiest way to do this is to convert the filter to a transfer
function and then use bode.

H3 = tf(B,A,Ts);
20 H6 = tf(B6,A6,Ts)
bode(H3,H6); % See Frequency response plot in Fig. 5.13.

Alternatively we could use freqz and construct the frequency response manually.

[h,w] = freqz(B,A,200); % discrete frequency response


[h6,w6] = freqz(B6,A6,200);
w = w/pi*500; % convert from rad/s to Hz
25 w6 = w6/pi*500;
loglog(w,abs(h), w6,abs(h6)) % See Magnitude plot in Fig. 5.13.

The sixth order filter in Fig 5.13 has a much steeper roll off after 30Hz, and this more approximates
the ideal filter. Given only this curve, we would select the higher order (6th) filter. To plot the
second half of the Bode diagram, we must unwrap the phase angle, and for convenience we will
also convert the phase angle from radians to degrees.

ph = unwrap(angle(h))*360/2/pi;
5.3. DISCRETE FILTERS 213

0 3rd order
10 f f
c N
Amplitude ratio

5
10

3rd order
6th order
6th order
10
10
0 1 2 3
10 10 10 10

0
Phase angle

200

400 Figure 5.13: The frequency response for a


3rd and 6th order discrete Butterworth fil-
fc fN
600
ter. The cut-off frequency for the filters is
10
0
10
1
10
2 3
10 fc = 30Hz and the Nyquist frequency is
frequency [Hz] fN = 500 Hz.

ph6 = unwrap(angle(h6))*360/2/pi;
semilogx(w,ph,w6,ph6) % plot the 2nd half of the Bode diagram

The phase lag at high frequencies for the third order filter is 3 90 = 270 , while the phase lag
for the sixth order filter is 6 90 = 540 . The ideal filter has zero phase lag at all frequencies.

The amplitude ratio plot in Fig 5.13 asymptotes to a maximum frequency of 500 Hz. This may
seem surprising since the Bode diagram of a true continuous filter does not asymptote to any
limiting frequency. What has happened, is that Fig 5.13 is the plot of a discrete filter, and that
the limiting frequency is the Nyquist frequency (2/T ). However the discrete Bode plot is a good
approximation to the continuous filter up to about half the Nyquist frequency.

A MATRIX OF FILTERS

Fig. 5.14 compares the performance of 9 filters using the 2 frequency component signal from
5.3.1. The 9 simulations in Fig 5.14 show the performance for 3 different orders (2,5 and 8) in
the columns at 3 different cut-off frequencies (20, 50 and 100 Hz) in the rows. This figure neatly
emphasizes the trade off between smoothness and phase lag.

5.3.2 A PPROXIMATING DIGITAL FILTERS FROM CONTINUOUS FILTERS

As mentioned in the introduction, one way to design digital filters, is to first start off with a
continuous design, and then transform the filter to the digital domain. The four common classical
continuous filter families: Butterworth, Chebyshev, elliptic and Bessel, are all strictly continuous
filters, and if we are to implement them in a computer or a DSP chip, we must approximate these
filters with a discrete filter.

To convert from a continuous filter H(s), to a digital filter H(z), we could use the bilinear trans-
form as described in section 2.5.2. Since this transformation method is approximate, we will find
that some frequency distortion will be introduced. However we can minimise this distortion by
214 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

order= 2
order= 5 c = 20 c = 50 c =100

raw
order= 8

filtered

Figure 5.14: The performance of various Butterworth filters of different orders 2,5, and 8 (left to
right) and different cut-off frequencies, c of 20, 50 and 100 Hz (top to bottom) of a two frequency
component signal.

adjusting the frequency response of the filter H(s) before we transform to the discrete approxi-
mation. This procedure is called using the bilinear transformation with frequency pre-warping.

F REQUENCY PRE - WARPING

The basic idea of frequency pre-warping is that we try to modify the transformation from contin-
uous to discrete so that H(z) better approximates H(s) at the frequency of interest, rather than
just at = 0. This is particularly noticeable for bandpass filtering applications.

The frequencies in the z plane (d ) are related to the frequencies in the s plane (a ) by
 
2 1 a T
d = tan (5.16)
T 2
 
2 1 d T
a = tan (5.17)
T 2

This means that the magnitude of H(s) at frequency a , |H(ia)|, is equal to the magnitude
of H(z) at frequency d , |H(eid T )|. Suppose we wish to discretise the continuous filter H(s)
using the bilinear transform, but we also would like the digital approximation to have the same
magnitude at the corner frequency (wc = 10), then we must substitute using Eqn 6.13.

Example of the advantages of frequency pre-warping.

Suppose we desire to fabricate an electronic filter for a guitar tuner. In this application the guitar
player strikes a possibly incorrectly tuned string, and the tuner box recognises which string is
struck, and then subsequently generates the correct tone. The player can then adjust the tuning of
5.3. DISCRETE FILTERS 215

the guitar. Such a device can be fabricated in a DSP chip using a series of bandpass filters tuned
to the frequencies of each string (give or take a note or two).

We will compare a continuous bandpass filter centered around a frequency of 440 Hz (concert
pitch A), with digital filters using a sampling rate of 2kHz. Since we are particularly interested
in the note A, we will take special care that the digital filter approximates the continuous filter
around f = 440Hz. We can do this by frequency pre-warping. The frequency response of the
three filters are shown in Fig. 5.15.

[B,A] = butter(4,[420 450]*2*pi,'s'); % band pass from 420 to 540 Hz


2 Gc = tf(B,A) % Continuous first order filter

Ts = 1/2000; % sample rate 2kHz


wn = pi/Ts; % Nyquist freq. (rad/s)

7 Gd = c2d(Gc,Ts,'tustin') % Now sample it using bilinear transform


Gd_pw = c2d(Gc,Ts,'prewarp',440*2*pi) % Now sample it with pre-warping

Fig. 5.15 highlights that the uncompensated digital filter (dashed) actually bandpasses signals in
the range of 350 to 400Hz rather than the 420 to 450Hz required. This significant error is addressed
by designing a filter to match the continuous filter at f = 440Hz using frequency pre-warping
(dotted).

Continuous & discrete Bode plots of a bandpass filter

0
10 Continuous
Discrete
prewarp
f
N
AR

1
10
prewarp

discrete continuous

2
10
3
10
Frequency [Hz]

Figure 5.15: Comparing a continuous bandpass filter with a discrete filter and discrete filter with
frequency pre-warping (dashed). Note the error in the passband when using the bilinear trans-
form without frequency pre-warping.

For an online filter application, we should

1. Decide on the cut-off frequency, c .

2. Select structure of the analogue filter, (Butterworth, Chebyshev, elliptical etc). Essentially
we are selecting the optimality criteria for the filter.

3. The filter order (number of poles).

4. Transform to the discrete domain using the bilinear transform.

All these considerations have advantages and disadvantages associated with them.
216 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

5.3.3 E FFICIENT HARDWARE IMPLEMENTATION OF DISCRETE FILTERS

To implement the discrete difference equation,


yk A(z 1 ) = B(z 1 )uk

in hardware, we can expand out to a difference equation


na
X nb
X
yk = ai yki + bi uki (5.18)
i=1 i=0

meaning now we can implement this using operational amplifiers and delay elements. One
scheme following Eqn. 5.18 directly given in Fig. 5.16 is known as the Direct Form I. It is easy
to verify that the Direct Form I filter has the transfer function
b0 + b1 z 1 + b2 z 2 + + bnb z nb
H(z 1 ) =
1 + a1 z 1 + a2 z 2 + + ana z na

b0
xk + + yk


1 1
z b1 a1 z
+ +

1 1
z .. a2 z
.
+ +

1 1
z .. z
.
+ +

bnb z 1
ana
| {z } +
numerator
| {z }
denominator

Figure 5.16: Hardware difference equation in Direct Form I

The square boxes in Fig. 5.16 with z 1 are the delay elements, and since we take off, or tap, the
signal after each delay, the lines are called tapped delays.

For the circuit designer who must fabricate Fig. 5.16, the expensive components are the delay
elements since they are typically specialised integrated circuits and one is economically motivated
to achieve the same result but by using fewer components. Rather than use the topology where
we need separate delay lines for both the input sequence and the output sequence separately, we
can achieve the identical result using only half the number of delays. This reduces fabrication
cost.
5.3. DISCRETE FILTERS 217

Starting with original IIR filter model

Y (z 1 ) B(z 1 )
1
= (5.19)
U (z ) A(z 1 )

and introducing an intermediate state, w,

Y (z 1 ) W (z 1 ) B(z 1 )
1
1
=
W (z ) U (z ) A(z 1 )

which means that we can describe Eqn. 5.19 as one pure infinite impulse response (IIR) and one
finite impulse response (FIR) process

Y (z 1 ) = B(z 1 )W (z 1 ) (5.20)
1 1 1
A(z )W (z ) = U (z ) (5.21)

A block diagram following this topology is given in Fig. 5.17 which is known as Direct Form II,
DFII.

wk b0
xk b
yk

z 1
a1 b1
b

z 1
a2
..
.
z 1
.. bnb
.
b

z 1
ana

Figure 5.17: An IIR filter with a minimal number of delays, Direct Form II

We could have reached the same topological result by noticing that if we swapped the order of
the tapped delay lines, in the direct form 1 structure, Fig. 5.16, then we will notice that the delay
lines are separated by a unity gain branch. Thus we can eliminate one of the branches, and save
the hardware cost of half the delays.

Further forms of the digital filter are possible and are used in special circumstances such as when
we have parallel processes or we are concerned with the quantisation of the filter coefficients.
One special case is described further in section 5.3.4.
218 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

With M ATLAB we would normally use the optimised filter command to simulate IIR filters,
but this option is not available when using say C on a microprocessor or DSP chip. The following
script implements the topology given in Fig. 5.17 for the example filter
B(z 1 ) (z 1 0.4)(z 1 0.98)
=
A(z 1 ) (z 1 0.5)(z 1 0.6)(z 1 0.7)(z 1 0.8)(z 1 0.9)
and a random input sequence.

B = poly([0.4 0.98]); % Example stable process


2 A = poly([0.5:0.1:0.9]);
U = [ones(30,1); randn(50,1)]; % example input profile
Yx = dlsim(B,A,U); % Test Matlab's version

% Version with 1 shift element -----------------


7 nb = length(B); A(1) = []; na = length(A);
nw = max(na,nb); % remember A=A-1
shiftw = zeros(size(1:nw))'; % initialise with zeros
Y = []; for i=1:length(U);
w = - A*shiftw(1:na) + U(i) ;
12 shiftw = [w; shiftw(1:nw-1)];
Y(i,1) = B*shiftw(1:nb);
end % for

t = [0:length(U)-1]'; [tt,Ys] = stairs(t,Y); [tt,Yx] =


17 stairs(t,Yx);
plot(tt,Ys,tt,Yx,'--'); % plotting verification

M ATLABs dlsim gives a similar output but is simply shifted one unit to the right.

5.3.4 N UMERICAL AND QUANTISATION EFFECTS FOR HIGH - ORDER FILTERS

Due to commercial constraints, for many digital filtering applications, we want to be able to re-
alise cheap high-quality filters at high speed. Essentially this means implementing high-order IIR
filters on low-cost hardware with short word-lengths. The problem is that high-order IIR filters
are very sensitive to quantisation effects especially when implemented in the expanded polyno-
mial form such as DFII.

One way to reduce the effects of numerical roundoff is to implement rewrite the DFII filter in fac-
tored form as a cascaded collection of second-order filters as shown in Fig. 5.18. Each individual
filter is a second-order filter as shown in Fig. 5.19. These are known as bi-quad filters, or second-
order sections (SOS). For an nth order filter, we will have n/2 sections, that is for filters with an
odd order, we will have one second order section that is actually first order.

We can convert higher-order filters to a collection of second-order sections using the tf2sos or
equivalent commands in M ATLAB. You will notice that the magnitude and range of the coef-
ficients of the SOS filter realisation is much smaller than the equivalent expanded polynomial
filter.

The separating of the factors into groups of second-order sections for a given high-order filter is
not unique, but one near-optimal way is to pair the poles nearest the unit circle with the zeros
nearest those poles and so on. This is is the strategy that tf2sos uses and is explained in further
detail in the help documentation.

Listing 5.6 demonstrates how we can optimally decompose a 7th order Butterworth filter, into
four second-order sections. We start by designing a filter in expanded form.
5.3. DISCRETE FILTERS 219

xk
yk

... SOS # n/2


SOS #1

Figure 5.18: Cascaded second-order sections to realise a high-order filter. See also Fig. 5.19.

xk b
b0 yk

z 1

a1 b
b1
z 1

a2 b2

Figure 5.19: A second-order section (SOS)

Listing 5.6: Converting a 7th-order Butterworth filter to 4 second-order sections


>> [B,A] = butter(7,0.8,'low'); H = tf(B,A,1) % Design a 7th order discrete DFII filter
2 Transfer function:

0.2363 z7 + 1.654 z6 + 4.963 z5 + 8.271 z4 + 8.271 z3 + 4.963 z2

+ 1.654 z + 0.2363
7

-------------------------------------------------------------------------
z7 + 4.182 z6 + 7.872 z5 + 8.531 z4 + 5.71 z3 + 2.349 z2

+ 0.5483 z + 0.05584

We can see the 7th order filter is rather unwieldy in its expanded form,

B 0.2363z 7 + 1.6542z 6 + 4.9626z 5 + 8.2710z 4 + 8.2710z 3 + 4.9626z 2 + 1.6542z + 0.2363


=
A z 7 + 4.1823z 6 + 7.8717z 5 + 8.5309z 4 + 5.7099z 3 + 2.3492z 2 + 0.5483z + 0.0558
but if we convert this filter to 4 second-order sections,

>> [SOS,G] = tf2sos(B,A) % Convert 7th order polynomial to 4 second-order sections


SOS =
220 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

1.0000 0.9945 0 1.0000 0.5095 0


4 1.0000 2.0095 1.0095 1.0000 1.0578 0.3076
1.0000 2.0026 1.0027 1.0000 1.1841 0.4636
1.0000 1.9934 0.9934 1.0000 1.4309 0.7687
G =
0.2363

then the numbers look far more numerically consistent. Each row of the second order section
matrix output from tf2sos is ordered as
sos = b0 b1 b2 1 a1 a2
where the associated second order section is implemented as
b0 + b1 z 1 + b2 z 2
Hi =
1 + a1 z 1 + a2 z 2
From the listing above, we can see that the first of the four second-order sections is actually a
first-order section. This is evident by noticing that the coefficients b2 and a2 are equal to zero for
the first filter section.

The advantage when using second-order sections is that we can maintain reasonable accuracy
even when using relatively short word lengths. Fig. 5.20(a), which is generated by Listing 5.7,
illustrates that we cannot successfully run a 7th-order elliptic low-pass filter in single precision in
the expanded polynomial form, (DFII), one reason being that the resultant filter is actually unsta-
ble! However we can reliably implement the same filter, in single precision if we use 4 cascaded
second-order sections. In fact for this application, the single precision filter implemented in
SOS is indistinguishable in Fig. 5.20(a) from the double precision implementation given that the
average error is always less than 106 .

Listing 5.7: Comparing DFII and SOS digital filters in single precision.
[B,A] = ellip(7,0.5,20,0.08); % Design a 7th order elliptic filter
2

U = ones(100,1); % Consider a step response


Y = filter(B,A,U); % Compute filter in double precision
Ys = filter(single(B), single(A), single(U)); % Compute filter in single precision

7 [SOS,G] = tf2sos(B,A); % Convert to SOS (in full precision)


n = size(SOS,1); % # of sections
ysos = single(G)*U; % Compute SOS filter in single precision
for i=1:n
ysos = filter(single(SOS(i,1:3)), single(SOS(i,4:end)), ysos)
12 end
stairs([Y Ys ysos]); % Compare results in Fig. 5.20(a).

It is also clear from the pole-zero plot in Fig. 5.20(b) that the single precision DFII implementation
has poles outside the unit circle due to the quantisation of the coefficients. This explains why this
single precision filter implemented in expanded DFII is unstable and should never be used.

5.3.5 F ILTER VISUALISATION TOOL

The GUI routine fvtool from the signal processing toolbox is a quick and convenient way to vi-
sualise the characteristics of various filters. Listing 5.8 shows how we can use fvtool to validate
the design of filter with a stop band between 0.45 and 0.5 and with a minimum -3 dB wobble and
-30dB in the stop band.
5.4. THE FOURIER TRANSFORM 221

zoomed region

2 0.4 PoleZero Map

single precision 0.35


filtered response
1.5 1
0.3
0.8
1
0.250.6

0.5 0.20.4
Double & single SOS
0.150.2
10 0
0 0.9 1 1.1


0
0.2
0.4
error

5
10
6 0.6
10
0.8 double
1 single
10
10 1.5 1 0.5 0 0.5 1
0 20 40 60 80 100
sample interval

(a) The time response of double and single precision filters (b) Pole-zero maps of single and double precision
to a unit step response. filters. Note that the single precision version has
poles outside the unit circle.

Figure 5.20: Comparing single precision second-order sections with filters in direct form II trans-
posed form. Note that the direct form II filter is actually unstable when run in single precision.

Listing 5.8: Designing and visualising a 5th order elliptic band-stop filter.
N = 5 % Design a 5th order elliptic filter
2 % stop band is 0.4 to 0.5 with a -3dB wobble and a -30dB in the stop band
[B,A] = ellip(N,3,30,[0.4 0.5],'stop')
h = fvtool(B,A) % See magnitude response in Fig. 5.21

The output of the visualisation tool shown in Fig. 5.21 of the magnitude response of the makes it
easy to check your filter design.

Figure 5.21: The magnitude re-


sponse of a filter generated by
fvtool.

5.4 T HE F OURIER TRANSFORM

The Fourier Transform (FT) is one of the most well known techniques in the mathematical and
scientific world today. However it is also one of the most confusing subjects in engineering,
222 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

partly owing to the plethora of definitions, normalising factors, and other incompatibilities found
in many standard text books on the subject. The Fourier transform is covered in many books on
digital signal processing, (DSP), such as [38, 135], and can also be found in mathematical physics
texts such as [107, 163]. The Fourier Transform is a mathematical transform equation that takes a
function in time and returns a function in frequency. The Fast Fourier Transform, or FFT, is simply
an efficient way to compute this transformation.

Chapter 2 briefly mentioned the term spectral analysis when discussing aliases introduced by
the sampling process. Actually a very learned authority , [179, p53], tells us that spectral for-
mally means pertaining to a spectre or ghost, and then goes on to make jokes about ghosts and
gremlins causing noisy problems. But nowadays spectral analysis means analysis based around
the spectrum of a something. We are going to use spectral analysis to investigate the frequency
components of a measured signal, since in many cases these frequency components may reveal
critical information hidden in the time domain description. This decomposition of the time do-
main signal into its frequency components is just one application of the FFT.

Commonly in mathematics, we can approximate any function f (t) using (amongst other things)
a Taylor series expansion, or a sine and cosine series expansion. If the original signal is periodic,
then the latter technique generally gives better results with fewer terms. This latter series is called
the Fourier series expansion.

The fundamental concept on which the Fourier transform (FT) is based is that almost any periodic
wave form can be represented as a sum of sine waves of different periods and amplitudes,

X
X
f (t) = An sin (2nf0 t) + Bn cos (2nf0 t) (5.22)
n=0 n=0

This means that we can decompose any periodic wave form (PW) with a fundamental frequency2
(f0 ) into a sum of sinusoidal waves, (which is known as frequency analysis) or alternatively we
can construct complicated waves from simple sine waves (which is known as frequency synthe-
sis).

Since it is impractical to have an infinite number of terms, this series is truncated, and f (t) now
only approximates the right hand side of Eqn. 5.22. Naturally, as the number of terms tends to
infinity, the approximation to the true signal f (t) improves. All we have to do now is to determine
the constants An and Bn for a given function f (t). These coefficients can be evaluated by
Z P/2 Z P/2
2 2
An = f (t) sin (2nf0 t) dt and Bn = f (t) cos (2nf0 t) dt (5.23)
P P/2 P P/2

when n 6= 0. When n = 0, we have the special case


Z P/2
1
A0 = 0, and B0 = f (t)dt (5.24)
P P/2

In summary, f (t) tells us how the signal develops in time, while the constants An , Bn give us a
method to generate f (t).

We can re-write Eqn. 5.22 as a sum of cosines and a phase angle



X
f (t) = Cn cos (2nfo t + n ) (5.25)
n=0
2 Remember the relationship between the frequency f , the period P , and the angular velocity .
1
f = =
P 2
5.4. THE FOURIER TRANSFORM 223

where the phase angle, n , and amplitude, Cn , are given by


 
An p
n = tan1 and Cn = A2n + Bn2 (5.26)
Bn
If we plot Cn vs f then this is called the frequency spectrum of f (t). Note that Cn has the same units
of An or Bn which have the same units of whatever the time series f (t) is measured in.

We can also write the Fourier series in complex quantities using the Euler relations,
ei = cos + i sin , and ei = cos i sin (5.27)

where i = 1. This implies that

ei + ei ei ei
cos = and sin = (5.28)
2 2i
So we can re-write Eqn. 5.22 in complex notation as

X
f (t) = Dn ei2nf0 t (5.29)

where the coefficients are Z P/2


1
Dn = f (t) ei2nf0 t dt (5.30)
P P/2

5.4.1 F OURIER TRANSFORM DEFINITIONS

Any function f (t) that is finite in energy can be represented in the frequency domain as F () via
the transform
Z
F () = f (t) eit dt (5.31)

Z Z
= f (t) cos(t) dt i f (t) sin(t) dt (5.32)

We can also convert back to the time domain by using the inverse Fourier transform,
Z
1
f (t) = F () eit d
2
You may notice that the transform pair is not quite symmetrical, we have a factor of 2 in the
inverse relation. If we use frequency measured in Hertz, where = 2f rather than , we obtain
a much more symmetrical, and easier to remember, pair of equations. [163, pp381382] comment
on this, and [143, p145] give misleading relations.

The spectrum of a signal f (t) is defined as the square of the absolute value of the Fourier trans-
form,
f () = |F ()|2
Parsevals equation reminds us that the energy of a signal is the same whether we describe it in
the frequency domain or the time domain,
Z Z
2 1
f (t) dt = |F ()|2 d
2

We can numerically approximate the function F () by evaluating Eqn 5.32 using a numerical
integration technique such as Eulers method. Since this numerical integration is computationally
224 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

expensive, and we must do this for every of interest, the calculation procedure is quite time
consuming. I will refer to this technique as the Slow Fourier Transform (SFT). Remember that
H() is a complex function of angular velocity (), and that the result is best displayed as a graph
with measured in radians/s as the dependent variable.

S OME HELPFUL PROPERTIES

Here are some helpful properties of the Fourier transform that may make analysis easier.

1. If f (t) = f (t) which means that f (t) is an odd function like sin t, then all the cosine terms
disappear.
2. If f (t) = f (t) which means that f (t) is an even function like cos t, then all the sine terms
disappear.
3. If f (t) = f (t + P/2), then the series will contain only odd components; n = 1, 3, 5, . . .
4. If f (t) = f (t + P/2), then the series will contain only even components; n = 2, 4, 6, . . .

Naturally we are not at liberty to change the shape of our measured time series f (t), but we may
be allowed to choose the position of the origin such that points 3 and 4 are satisfied.

A DECOMPOSITION OF A SQUARE WAVE

The square wave is a simple periodic function that can be written as a infinite sine and cosine
series.
1

-1
-2 -1 0 1 2

We note that the original wave is:

1. Periodic, with a period of P = 1 seconds, and


2. f (t) is an odd function such as sin t so we expect all the cosine terms disappear. An odd
function is where f (t) = f (t), while an even function is where f (t) = f (t).

Using Eqn. 5.23, we see that


1
An = (1 cos n)
n
which collapses to An = 2/n for n = 1, 3, 5, . . . and An = 0 for n = 2, 4, 6, . . . . This gives
the Fourier series approximation to f (t) as
2 2 2
f (t) = sin 2t + sin 6t + + sin 2(2n 1)t + (5.33)
3 (2n 1)
As the number of terms, n, in Eqn. 5.33 increase, the approximation improves as illustrated in
Fig 5.22 which shows the Fourier approximation for 1 through 4 terms. However note that the
oscillation errors at the jump discontinuities remain constant at about 9% regardless of the number
of terms taken. This counter intuitive result is known as the Gibbs phenomenon.
5.4. THE FOURIER TRANSFORM 225

# of terms = 1
1

# of terms = 2
1

# of terms = 3
1

# of terms = 4
1

0 Figure 5.22: The Fourier approxima-


tion to a square wave with varying
1 number of sinusoidal terms.
0 1 2 3 4

Later in 5.4.3 we will try to do the reverse, namely given the square wave signal, we will extract
numerically the coefficients of the sine and cosine series.

5.4.2 O RTHOGONALITY AND FREQUENCY SPOTTING

Consider the function y(t) = sin(3t) sin(5t) which is plotted for 2 full periods in Fig 5.23 (top).
If we integrate it over one full period, we can see that the positive areas, cancel out the negative
areas, hence the total integral is zero. In other words,
Z P =
y(t) dt = 0
0

The only time an integral of a full period will not be zero is when the entire curve is either all
positive or all negative. The only time that can happen, is when the two frequencies are the same
such as in Fig. 5.23 (bottom). Thus given

y = sin(nt) + sin(mt) (5.34)

then the integral over one full period


Z P
y(t) dt = 0 if n 6= m
0

will be zero if n and m are different, but will be nonzero


Z P
y(t) dt 6= 0 if n = m
0

only when n = m.

Fig. 5.23 was created by the following M ATLAB commands


226 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

1 sin(3t)*sin(5t)

0.5

0.5

1 R 2
0 sin2 (3t) =-0.000

2
1 sin (3t)

0.5

0
Figure 5.23: A plot of (top) R 2
0.5 sin2 (3t) =3.142
y = sin(3t) sin(5t) and (bottom) 0

y = sin(3t) sin(3t). In the top figure the 1


shaded regions cancel giving a total 0 /2 3/2 2
integral of zero for the two periods. time

1 t=[0:0.01:2*pi]'; % create a time vector exactly 2 Periods long


y3=sin(3*t); y5=sin(5*t);
plot(t,[y3.*y3, y3.*y5]) % plot both curves

We can numerically integrate these curves using the sum command. The integral of the top curve
in Fig. 5.23 is

>> trapz(t,y3.*y5) % Integral should be close to zero


2 ans = -1.9699e-05

R 2
which is close enough to zero, while 0
sin2 3t is evaluated as

>> trapz(t,y3.*y3) % Integral not expected to be close to zero


ans = 3.141593

and is definitely non-zero. Note that for this example we have integrated over 2 periods, so we
should divide the resultant integral by 2. This however, does not change our conclusions.

P ERIOD

The period of y = sin(nt) is P = 2/n. The period of y = sin(nt) sin(nt) can be evaluated using
an expansion for sin2 (nt) as
1
y = sin2 (nt) = (1 cos(2nt))
2
thus the period is P = /n.

The period of the general expression y = sin(nt) sin(mt) where n and m are potentially different
is found by using the expansion
1
y= [cos(n + m)t + cos(n m)t]
2
5.4. THE FOURIER TRANSFORM 227

which for the example above gives

1
y = sin(3t) sin(5t) = [ cos 8t + cos 2t] (5.35)
2
The period of the first term in Eqn 5.35 is P1 = /4 and for the second term is P2 = . The total
period is the smallest common multiple of these two periods which is for this case P = .

The property given in Eqn. 5.4.2 can be exploited for spectral analysis. Suppose we have measured
a signal x(t) that is made up of a sine wave with a particular, but unknown, frequency. However
we wish to establish this unknown frequency using spectral analysis. All we need to do is to
multiply x(t) by a trial sine wave, say sin 5t, integrate over one period, and look at the result.
If the integral is close to zero, then our original signal, x(t), had no sin 5t term, if however we
obtained a significant non-zero integral, then we would conclude that the measured signal had
a sin 5t term somewhere. Note that to obtain a complete frequency description of the signal, we
must try all possible frequencies from 0 to .

So in summary, the idea is to multiply the signal x(t) with different sine waves containing the
frequency of interest, integrate, and then see if the result is non-zero. We could plot the integral
result against the frequency thus producing a spectral plot of x(t).

5.4.3 P OWER SPECTRAL DENSITY ESTIMATES FROM TIME SERIES DATA

Instead of observing a signal in time, in many cases we may be interested in the frequency com-
ponents of the signal. That is we would like to be able to plot the power of the signal as a function
of frequency. The power of the signal x(t) over a frequency band is defined as the absolute
square of the Fourier transform
X  2
def
x () = |X()|2 = x(t) eit

When we plot the power as a function of frequency, it is much like a histogram where each fre-
quency bin contains some power. This plot is sometimes referred to as a periodogram.

M ATLAB provides a convenient way to analyse the frequency component of a finite sampled
signal by using the built-in fft function. This is termed spectral analysis and is at its simplest
just a plot of the magnitude of the Fourier transform of the time series, or abs(fft(x)). The
complexities are due to the scaling of the plot, and possible windowing to improve the frequency
resolution at the expense of the magnitude value, or vice-versa and possible adjustment for non-
periodic signals.

The function, spsd (scaled PSD) in Listing 5.9 will use the Fourier transform to decompose the
signal into different frequencies and display this information. This function was adapted from a
M ATH W ORKS Technical support note.3

Since we have assumed that the input signal is real, then we know that the Fourier transform
will be conjugate symmetrical about the Nyquist frequency, thus we can throw the upper half
away. Doing this, we must multiply the magnitude by 2 to maintain the correct size. M ATLAB
also normalises the transform by dividing by the number of data points so must also account for
this in the magnitude. Since the FFT works much faster when the series is a power of two, we
automatically pad with zeros up to the next power of 2.

Listing 5.9: Routine to compute the power spectral density plot of a time series

3 http://au.mathworks.com/help/signal/ug/psd-estimate-using-fft.html
228 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

function [Mx,f] = spsd(x,dt);


% [Mx,f] = spsd(x,[dt])
3 % Power spectral plot of x(t) sampled at dt [s], (default dt=1).
% Output: Power of x at frequency, f [Hz]
% See also: pwelch, periodogram, etfe

if nargin < 2, dt = 1.0; end % default sample time =1


8 Fs = 1/dt; Fn = Fs/2; % sampling & Nyquist frequency [Hz]
NFFT=2.nextpow2(length(x)); % NFFT=2.((log(length(x))/ log(2))) ;
FFTX=fft(x,NFFT); % Take fft, padding with zeros,
NumUniquePts = ceil((NFFT+1)/2);

13 FFTX=FFTX(1:NumUniquePts); % fft is symmetric, throw away upper half


MX=2*abs(FFTX); % Take magnitude of X & 2 since threw half away

MX(1)=MX(1)/2; % Account for endpoint uniqueness


MX(length(MX))=MX(length(MX))/2; % We know # of samples is even
18

MX=MX/length(x); % scale for length


f=(0:NumUniquePts-1)'*2*Fn/NFFT; % frequency Hz

if nargout == 0, % do plot if requested


23 plot(f,MX); xlabel('frequency [Hz]');
end % if
return % SPSD.M

To look how we can perform a spectral decomposition, let us start with a simple example where
we know the answer, namely the signal

x(t) = 4.5 sin(2 200 t)

It is immediately apparent that at a frequency of f = 200 Hz, we should observe a single Fourier
coefficient with a magnitude of 4.5 while all others are zero. Our aim is to verify this using the
power density routine from Listing 5.9.

Fs = 1000; % Sampling freq, Fs , in Hz


t=0:1/Fs:1; % Time vector sampled at Fs for 1 second
x = 4.5*sin(2*pi*t*200)'; % scaled sine wave.
5
4
3
2
1
0
0 100 200 300 400 500
spsd(x,1/Fs) % Does Fs = 200, A = 4.5 ? frequency [Hz]

We should expect to see a single distinct spike at f = 200Hz with a height of 4.5.

With the power spectrum density function spsd, we can also analyse the frequency components
of a square wave. We can generate a square wave using the square function in a manner similar
to that of generating a sine wave, or simply wrap the signum function around a sine wave.

1 Fs = 30; t=0:1/Fs:20; % Sampling freq, Fs , in Hz & time vector for 20 seconds


s = 0.5*square(2*pi*t); % Square wave
spsd(s,1/Fs) % See odd spikes ??
5.4. THE FOURIER TRANSFORM 229

Welch Power Spectral Density Estimate

Power/frequency (dB/Hz)
0

10

20

30

40

50
0 5 10 15
pwelch(s,[],[],[],Fs); % Try toolbox version Frequency (Hz)

Note that all the dominant frequency components appear at the odd frequencies (1,3,5,7. . . ), and
that the frequency components at the even frequencies are suppressed as anticipated from 5.4.1.
Of course they theoretically should be zero, but the discretisation, finite signal length and sam-
pling etc. introduce small errors here.

P RACTICAL ISSUES

In practice, constructing the periodogram from actual data can give rather messy, inconclusive
and widely fluctuating results. For a data series, you may find that different time segments of
the series give different looking spectra when you suspect that the true underlying frequency
characteristics is unchanged. One way around this is to compute the spectra or periodogram for
different segments and average the results, or you could use a windowing function.

The S IGNAL P ROCESSING toolbox includes a command periodogram and pwelch to reliably
construct power density plots. Other functions to aid the spectral analysis of systems are spa
and etfe (empirical transfer function estimate) from the S YSTEM I DENTIFICATION toolbox. All
of these routines perform basically the same task.

Problem 5.2 Fig. 5.24 trends the monthly critical radio frequencies in Washington, D.C. from May
1934 until April 1954. These frequencies reflect the highest radio frequency that can be used for
broadcasting.

15
Critical frequency

10

0
1935 1940 1945 1950 1955
Year

Figure 5.24: Critical radio frequencies

This data is available from the collection of classic time series data archived in:
http://robjhyndman.com/hyndsight/tsdl/.
(See also Problem 6.3 for further examples using data from this collection.)

1. Find the dominant periods (in days) and amplitudes of this series.
2. Can you hypothesise any natural phenomena that could account for this periodicity?

5.4.4 F OURIER SMOOTHING

It is very easy to smooth data in the Fourier domain. You simply take all your data, FFT it,
multiply this transformed data by the filter function H(f ), and then do the inverse FFT (IFFT)
230 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

to get back into the time domain. The computationally difficult parts are the FFT and IFFT, but
this is easily achieved in M ATLAB or in custom integrated circuits. Note that this section is called
smoothing as opposed to online filtering because these smoothing filters are acausal since future
data is required to smooth present data as well as past data. Filters that only operate on past
data are referred to as causal filters. Acausal filters tend to perform better than causal filters as
they exhibit less phase lag but they cannot be used in real time applications. In process control,
physical realisation constraints mean that only causal filters are possible. However in the audio
world (compact disc and digital tapes), acausal filters are used since a time delay is not noticeable
and hence allowed. Surprisingly time delays are allowed on telephone communications as well.
Numerical Recipes [163] has a good discussion of the advantages of Fourier smoothing.

Let us smooth the measurement data given in 5.3.1 and compare the smoothed signal with
the online filtered signal. This time however we will deliberately generate 210 = 1024 data pairs
rather than an even thousand so that we can take advantage of the fast Fourier Transform al-
gorithm. It is still possible to evaluate the Fourier transform of a series with a non-power of 2
number of pairs, but the calculation is much slower so its use is discouraged.

Again we will generate a measurement signal as


1
y = sin (2f1 t) + sin (2f2 t)
10
where f1 = 2 Hz and f2 = 50 Hz as before. We will generate 210 data pairs at a sampling frequency
of 200Hz.

1 t = 1/200*[0:pow2(10)-1]'; % 210 data pairs sampled at 200 Hz


1
y = sin(2*pi*2*t) + 0.1*sin(2*pi*50*t); % Signal: y = sin (2f1 t) + 10 sin (2f2 t)

Generating the Fourier transform of the data is easy. We can plot the amount or power in each
frequency bin Y(f ) by typing

yp = fft(y);
plot(abs(yp).2) % See periodogram in Fig. 5.25
5 semilogy(abs(yp).2) % clearer plot

In Fig. 5.25, we immediately note that the plot is mirrored about the central Nyquist frequency
meaning that we can ignore the right-hand side of the plot. The x-axis frequency scale is nor-
malised, but you can easily detect the two peaks corresponding to the two dominant frequencies
of 2 and 50 Hz. Since we are trying to eradicate the higher frequency, what we would desire in
the power spectrum is only one spike at a frequency of f = 2 Hz. We can generate a power curve
of this nature, by multiplying it with the appropriate filter function H(f ).

Let us generate the filter H(f ) in the frequency domain. We want to pass all frequencies less than
say 30Hz, and attenuate all frequencies above 30Hz. We do this by constructing a vector where 1
corresponds to the frequencies we wish to retain, and zero is for those we do not want. Then we
convolve the filter with the data, or elementwise multiply the two frequency series.

n = 20;
h=[ones(1,n) zeros(1,length(y)-2*n) ones(1,n)]'; % Filter H(f )
ypf = yp.*h % convolve signal & filter Y (f ) H(f )
plot([abs(yp), h, abs(ypf)])

Now we can take the inverse FFT to obtain back the transformed the time domain signal which is
now our low pass filtered signal. Technically the inverse Fourier transform of a Fourier transform
5.5. NUMERICALLY DIFFERENTIATING INDUSTRIAL DATA 231

500 Spectrum
Filter

400 zoomed
600
300
Power

400

200 200

0
100 0 50

0 Figure 5.25: The frequency power spectrum


for the level signal. Both the frequency axis
0 200 400 600 800 1000
frequency and the power axis are normalised.

of a real signal should return also a real time series of numbers. However owing to numerical
roundoff, the computed series may contain a small complex component. In practice we can safely
ignore these, and force the signal to be real.

10 yf = real(ifft(ypf));% force real signal, yf = {FFT 1 (Y (f ))}


plot(t,[y,yf]) % compare raw & smoothed in Fig. 5.26.

1.5
smoothed
1

0.5
output

0.5
original
1 Figure 5.26: The Fourier smoothed signal, yf ,
compared to the original noisy level signal, y.
1.5 Note the absence of any phase lag in the now
0.2 0 0.2 0.4 0.6 0.8
time (s) offline smoothed signal.

The result shown in Fig. 5.26 is an acceptably smooth signal with no disturbing phase lag that
was unavoidable in the online filtering example.

Problem 5.3 Design a notch filter to band-stop 50Hz. Test


 your filter using an input signal with a
time varying frequency content such as u(t) = sin 16t4 . Plot both the original and filtered signal
together. Sample around 1000 data pairs using a sampling frequency of 1000 Hz. Also plot the
apparent frequency as a function of time for u(t).

5.5 N UMERICALLY DIFFERENTIATING INDUSTRIAL DATA

In industrial applications, we often want the know the slope of a measured process variable. For
example, we may wish to know the flowrate of material into a vessel, when we only measure the
level of the vessel. In this case, we differentiate the level signal, to calculate the flowrate. However
for industrial signals, which typically have significant noise, calculating an accurate derivative is
232 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

not trivial. A crude technique such as differencing, will give intolerable error. For these types of
online applications, we must first smooth the data, then differentiate the smoothed data. This can
be accomplished using discrete filters.

5.5.1 E STABLISHING FEEDRATES

This case study demonstrates one way to differentiate noisy data. For this project, we were trying
to implement a model based estimation scheme to a 1000 liter fed-batch chemical reactor. For
our model, we needed to know the feedrate of the material being fed to the reactor. However
we did not have a flow meter on the feed line, but instead we could measured the weight of the
reactor using load cells. As the 3 hour batch progressed, feed was admitted to the reactor, and
the weight increased from about 800 kg to 1200 kg. By differentiating this weight, we could at
any time approximate the instantaneous feed rate. This is not an optimal arrangement, since we
expect considerable error, but in this case it was all that we could do.

The weight measurement of the reactor is quite noisy. The entire reactor is mounted on load cells,
and all the lines to and from the reactor (such as cooling/heating etc) are connected with flexible
couplings. Since the reactor was very exothermic, the feed rate was very small (30 g/s) and this is
almost negligible to the combined weight of the vessel ( 3000 kg) and contents ( 1000 kg). The
raw weight signal (dotted) of the vessel over the batch is shown in the upper figures in Fig. 5.28.
An enlargement is given in the upper right plot. The sample time for this application was T = 6
seconds.

A CRUDE METHOD OF DIFFERENTIATING

The easiest (and crudest) method to establish the feedrate is to difference the weight signal. Sup-
pose the sampled weight signal is W (t), then the feedrate, W = F , is approximated by
Wi Wi1
F = W
T
But due to signal discretisation and noise, this results in an unusable mess4 . The differencing
can be achieved elegantly in Matlab using the diff command. Clearly we must first smooth the
signal before differencing.

U SING A DISCRETE FILTER

This application is not just smoothing, but also differentiating. We can do this in one step by
combining a low pass filter (Glp ) with a differentiater, s, as

Gd (s) = Glp (s) s (5.36)

We can design any low pass filter, (say a 3rd order Butterworth filter using the butter command),
then multiply the filter polynomials by the differentiater. In the continuous domain, a differenti-
ater is simply s, but in the discrete domain we have a number of choices how to implement this.
Ogata, [152, pp 308] gives these alternatives;

1z 1

T Backward difference
s 1z 1 (5.37)
z 1 T Forward difference

2 1z1 Center difference
T 1+z 1
4I do not even try to plot this, as it comes out a black rectangle
5.5. NUMERICALLY DIFFERENTIATING INDUSTRIAL DATA 233

For a causal filter, we need to choose the backward or center difference variants. I will use the
backward difference Thus Eqn 5.36 using the backward difference in the discrete domain is

1 z 1
Gd (z) = Glp (z) (5.38)
T
with sample time T . Thus the denominator is multiplied by T , and the numerator gets convolved
with the factor (1 z 1 ). We need to exercise some care in doing the convolution, because the
polynomials given by butter are written in decreasing powers of z (not z 1 ). Note however that
fliplr(conv(fliplr(B),[-1,1])); is equivalent to conv(B,[1,-1]);. Listing 5.10 tests
this smoothing differentiator algorithm.

Listing 5.10: Smoothing and differentiating a noisy signal


dt = 0.05; t = [0:dt:20]'; % Sample time T = 0.05
U = square(t/1.5); [num,den]=ord2(2,0.45); % Input u(t) & plant dynamics
3 y = lsim(num,den,U,t)+randn(size(y))*0.005;%simulate it & add noise

dyn = gradient(y,dt); % Do crude approx of differentiation

[B,A] = butter(3,0.1); % Design smoother, = 0.1


8 Gd_lp = tf(B,A,dt,'variable','z-1'); % Convert to low-pass filter, Glp (z 1 )
Gdd = series(Gd_lp,tf([1 -1],dt,dt,'variable','z-1')) % Eqn. 5.38.
yns = filter(B,A,y); % Smooth y(t)
dyns = lsim(Gdd,y); % Smooth & differentiate y(t), See Fig. 5.27.

The results of the smoothing and the differentiating of the noisy signal are given in Fig. 5.27.
The upper plot compares the noisy measurement with the the smoothed version exhibiting the
inevitable phase lag. The lower plot in Fig. 5.27 shows the crude approximation of the deriva-
tive obtained by differencing (with little phase lag, but substantial noise), and the smooth and
differentiated version.

0.4

0.2
Measurement

0.2

0.4
1

0.5
Derivative

0.5
Figure 5.27: Smoothing, (upper)
1
0 5 10 15 20
and differentiating (lower) a noisy
time [s] measurement.

Note that even though the noise on the raw measurement is insignificant, the finite differencing
amplifies the noise considerably. While the degree of smoothing can be increased by decreasing
the low pass cut-off frequency, , we pay the price in increasing phase lag.

Fig. 5.28 shows actual industrial results using this algorithm. The right hand plots show an en-
larged portion of the signal. The upper left plot shows the raw (dotted) and smoothed (solid)
weight signal. The smoothed signal was obtained from the normal unity gain Butterworth filter
with a c = 0.02 1
2T = 600 s
1
. There is a short transient at the beginning, when the filter is first
invoked, since I made no attempt to correct for the non-zero initial conditions. The lower 2 plots
234 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

show the differentiated signal, in this case feed rate using the digital filter and differentiater. Note
that since the filter is causal, there will be a phase lag, and this is evident in the filtered signal
given in the top right plot. Naturally the differentiated signal will also lag behind the true fee-
drate. In this construction, the feed rate is occasionally negative corresponding to a decrease in
weight or filter transient. This is physically impossible, and the derived feedrate be chopped at
zero if it were to be used further in a model based control scheme for example.

1200 1140
Vessel weight [kg]

Vessel weight [kg]


1100 1120

1000 1100

900 1080

800 1060
8 10 12 10.5 11 11.5 12
time [hr] time [hr]

0.1 0.1
Feed rate [kg/s]

Feed rate [kg/s]

0.05 0.05

0 0

-0.05 -0.05
8 10 12 10.5 11 11.5 12
time [hr] time [hr]

Figure 5.28: Industrial data showing (upper) the raw & filtered weight of a fed-batch reactor
vessel [kg] during a production run, and (lower) the feedrate [kg/s]

5.6 S UMMARY

Filtering or better still, smoothing, is one way to remove unwanted noise from your data. Smooth-
ing is best done offline if possible. The easiest way to smooth data is to decide what frequencies
you wish to retain as the signal, and which frequencies you wish to discard. One then constructs
a filter function, H(f ) which is multiplied with the signal in the frequency domain to create the
frequency of the smoothed signal. To get the filtered time signal, you transform back to the time
domain. One uses the Fourier transform to convert between the time and frequency domains.

If you cannot afford the luxury of offline filtering, then one must use a causal filter. IIR filters
perform better than FIRs, but both have phase lags at high frequencies and rounded edges that
are deleterious to the filtered signal. The Butterworth filter is better than simple cascaded first
order filters, it has only one parameter (cut-off frequency), relatively simple to design and suits
most applications. Other more complicated filters such as Chebyshev or elliptic filters, with corre-
spondingly more design parameters are easily implemented using the signal processing toolbox
in M ATLAB if desired. Usually for process engineering applications, we are most interested in
low pass filters, but in some applications where we may wish to remove slow long term trends
(adjustment for seasons which is common in reporting unemployment figures for example), or
differentiate, or remove a specific frequency such as the 50Hz mains hum, then high-pass or notch
5.6. SUMMARY 235

filters are to be used. These can be designed in exactly the same way as a low pass filter, but with
a variable transformation.
236 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING
C HAPTER 6

I DENTIFICATION OF DYNAMIC
MODELS

Fiedlers forecasting rules:


Forecasting is very difficult, especially if its about the future. Consequently when presenting a forecast:
Give them a number or give them a date, but never both.

6.1 T HE IMPORTANCE OF SYSTEM IDENTIFICATION

Most advanced control schemes require a good dynamic process model to be effective. One option
is to set the control law to be simply the inverse of the plant so when the inevitable disturbances
occur, the controller applies an equal but opposite input to maintain a constant output. In practice
this desire must be relaxed somewhat to be physically realisable, but the general idea of inserting
a dynamic element (controller) into the feedback loop to obtain suitable closed loop dynamics
necessitates a process model.

When we talk about dynamic models, we mean some quantitative way to predict the system
outputs knowing the input. Fig. 6.1 shows the three components of interest; the input, the plant,
and the output. If we know any two components, we can in principle, calculate the third.
Does this match with reality?

Input u Model output, y

Try to establish this model

Figure 6.1: The prediction problem: Given a model and the input, can we predict the output?

As introduced in section 1.1, when convenient in this chapter, I will use the convention that inputs
and associated parameters such as the B polynomial are coloured blue, and the outputs and the A
polynomial are coloured red. Disturbances will be coloured green (because sometimes they come
from nature). These colour conventions may help make some of the equations in this chapter
easier to follow.

The prediction problem is given the plant and input, one must compute the output, the reconstruc-
237
238 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

tion problem is given the output and plant, one must propose a plausible input to explain the
observed results, and the identification problem, the one we are concerned with in this chapter, is
to propose a suitable model for the unknown plant, given input and output data.

Our aim for automatic control problems is to produce a usable model, that given typical plant
input data, will predict within a reasonable tolerance the consequences. We should however
remember the advice by the early English theologian and philosopher, William of Occam (1285
1349) who said: What can be accounted for by fewer assumptions is explained in vain by more.
This is now known as the principle of Occams razor, where with a razor we are encouraged to
cut all that is not strictly necessary away from our model. This motivates us to seek simple, low
order, low complexity, succinct, elegant models.

For the purposes of controller design we are interested in dynamic models as opposed to simply
curve fitting algebraic models. In most applications, we collect discrete data, and employ digital
controllers, so we are only really interested in discrete dynamic models.

Sections 3.3 and 3.3.3 revises traditional curve and model fitting such as what you would use in
algebraic model fitting. Section 6.7 extends this to fit models to dynamic data.

Good texts for for system identification include the classic [34] for time series analysis, [192] and
[126] accompanied with the closely related M ATLAB toolbox for system identification, [127]. An
early survey is [104]. For identification applications in the processing industries, [130] contains the
basics for modelling and estimation suitable for those with a chemical engineering background.
A good collection is found in [151] and a now somewhat dated detailed survey of applications,
including some interesting practical hints and observations with over 100 references is given in
[79].

6.1.1 B ASIC DEFINITIONS

When we formulate and use engineering models, we typically have in mind a real physical pro-
cess or plant, that is perhaps so complex, large, or just downright inconvenient, that we want to
use a model instead. We must be careful at all times to distinguish between the plant, model, data
and predictions. The following terms are adapted from [192].

System, S This is the system that we are trying to identify. It is the physical process or plant that
generates the experimental data. In reality this would be the actual chemical reactor, plane,
submarine or whatever. In simulation studies, we will refer to this as the truth model.

Model, M This is the model that we wish to find such that the model predictions generated by
M are sufficiently close to the actual observed system S output. Sometimes the model may
be non-parametric where it is characterised by a curve or a Bode plot for example, or the
model might be characterised by a finite vector of numbers or parameters, M(), such as
say the coefficients of a polynomial transfer function or weights in a neural network model.
In either case, we can think of this as a black-box model where if we feed in the inputs, turn
the handle, we will get output predictions that hopefully match the true system outputs.

Parameters, . These are the unknowns in the model M that we are trying to establish such
that our output predictions y are as close as possible to the actual output y. In a transfer
function model, the parameters are the coefficients of the polynomials; in a neural network,
they would be the weights of the neurons.

This then brings us to the fundamental aim of system identification: Given sufficient input/out-
put data, and a tentative model structure M with free parameters , we want to find reasonable
6.1. THE IMPORTANCE OF SYSTEM IDENTIFICATION 239

y
Plant, S

input, u error,

(hopefully small)
Model, M()
y

Figure 6.2: A good model with appropriate parameters, M(), duplicates the behaviour of the
true plant, S.

values for such that our predictions y closely follow the behaviour of our plant under test, y, as
shown in Fig. 6.2.

Models can be classified as linear or nonlinear. In the context of parameter identification, which
is the theme of this chapter, it is assumed that the linearity (or lack of) is with respect to the
parameters, and not to the shape of the actual model output. A linear model does not mean just a
straight line, but includes all models where the model parameters , enter linearly. If we have the
general model
y = f (, x)
with parameters and independent variables x, then if y/ is a constant vector independent
of , then the model is linear in the parameters, otherwise the model is termed nonlinear. Linear
models can be written in vector form as

y = f (x)T

Just as linear models are easier to use than nonlinear models, so too are discrete models compared
with continuous models.

6.1.2 B LACK , WHITE AND GREY BOX MODELS

The two extreme approaches to modelling are sometimes termed black and white box models.
The black box, or data-based model is built exclusively from experimental data, and a deliberate
boundary is cast around the model, the underlying contents inside about which we neither know
nor care. The Wood-Berry column model described in Eqn. 3.24 is a black box model and the
parameters were obtained using the technique of system identification. The white box, theoretical
or first-principles model, preferred by purists, contains already well established physical and
chemical relations ideally so fundamental, no experiments are needed to characterise any free
parameters.

However most industrial processes are such that the models are often partly empirical and partly
fundamental. This intermediate stage lying between black and white is known as a grey-box
model. Ljung in [129] somewhat whimsically distinguishes between palettes of grey. Smoke-
grey models are those where we may try to find a suitable nonlinear transformation such that the
model is better described by a linear model. Steel-grey models are those where we may locally
linearise about an operating point while slate-grey models are those where a hybrid of models
is suitable. Further details on the distinction between these model types are given in [188].

An example of this type of model is the Van de Waals gas law or correlations for the friction factor
in turbulent flow. Finally if the process is so complex that no fundamental model is possible, then
a black box type model may be fitted. Particularly hopeless models, based mostly on whim and
240 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

what functions happened to be lying around at the time, can be found in [77]. These are the types
of the models that should have never been regressed.

Of course many people have argued for many years about the relative merits of these two extreme
approaches. As expected, the practising engineer, tends to have the most success somewhere in
the middle, which, not unexpectedly, are termed grey box models. The survey in [188] describes
the advantages and disadvantages of these colour coded models in further details.

6.1.3 T ECHNIQUES FOR IDENTIFICATION

There are two main ways to identify process models: offline or online in real-time. In both cases
however, it is unavoidable to collect response data from the system of interest as shown in Fig. 6.3.
Most of the more sophisticated model identification is performed in the offline mode where all
the data is collected first and only analysed later to produce a valid model. This is typically
convenient since before any regressing is performed we know all the data, and we can do this in
comfort in front of our own computer workstation. The offline developed model (hopefully) fits
the data at the time that it was collected, but there is no guarantee that it will be still applicable for
data collected sometime in the future. Process or environmental conditions may change which
may force the process to behave unexpectedly.

Data logging computer

input output

bc
bc bc bc

signal generator PI transducer

Plant, S

control valve
(manipulated variable)

Figure 6.3: An experimental setup for input/output identification. We log both the input and the
response data to a computer for further processing.

The second way to identify models is to start the model regression analysis while you are col-
lecting the data in real-time. Clearly this is not a preferred method when compared with offline
identification, but in some cases we will study later, such as in adaptive control, or where we
have little input/output data at our immediate disposal, but are gradually collecting more, we
must use on-line identification. As you collect more data, you continually update and correct the
model parameters that were established previously. This is necessary when we require a model
during the experiment, but that the process conditions are likely to change during the period of
interest. We will see in chapter 7 that the online least squares recursive parameter estimation is
the central core of adaptive control schemes.
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 241

6.2 G RAPHICAL AND NON - PARAMETRIC MODEL IDENTIFICATION

Dynamic model identification is essentially curve fitting as described in section 3.3 but for dy-
namic systems. For model based control, (our final aim in this section), it is more useful to have
parametric dynamic models. These are models which have distinct parameters to be estimated in
a specified structure. Conversely non-parametric models are ones where the model is described
by a continuous curve for example, such as say a Bode or Nyquist plots, or collections of models
where the model structure has not been determined beforehand, but is established from the data
in addition to establishing the parameters.

In both parametric and non-parametric cases we can


make predictions, which is a necessary requirement of
any model, but it is far more convenient in a digital com-
puter to use models based on a finite number of param-
eters rather than resort to some sort of table lookup or
digitise a non-parametric curve.

How one identifies the parameters in proposed models


from experimental data is mostly a matter of preference.
There are two main environments in which to identify
models:
Perturb the process, and watch what
happens
1. Time domain analysis described in 6.2.1, and

2. frequency domain analysis described in 6.2.2.

A good tutorial overview of estimating models suitable for process control from step or relay
based experiments is given in [125].

6.2.1 T IME DOMAIN IDENTIFICATION USING GRAPHICAL TECHNIQUES

Many engineers tend to be more comfortable in the time domain, and with the introduction of
computers, identification in the time domain, while tedious, is now reliable and quite straight
forward. The underlying idea is to perturb the process somehow, and watch what happens.
Knowing both the input and output signals, one can in principle, compute the process transfer
function. The principle experimental design decision is the type of exciting input signal.

S TEP AND IMPULSE INPUTS

Historically, the most popular perturbation test for visualisation and manual identification was
the step test. The input step test itself is very simple, and the major response types such as first
order, underdamped second order, integrators and so forth are all easily recognisable from their
step tests by the plant personnel as illustrated in Fig. 6.4.

Probably the most common model structure for industrial plants is the first-order plus deadtime
model, (FOPDT),
K
Gp (s) = es (6.1)
s + 1
where we are to extract reasonable values for the three parameters: plant gain K, time constant
, and deadtime . There are a number of strategies to identify these parameters, but one of the
more robust is called the Method of Areas, [121, pp3233].
242 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Simple first order 1st order with deadtime 2nd order underdamped

1.5
1 1

0.8 0.8
Amplitude

Amplitude

Amplitude
1
0.6 0.6
1 1e 4s
0.4 s+1 0.4 s+1 1
0.5 s 2 +0.8s+1
0.2 0.2

0 0 0
0 2 4 6 8 0 5 10 15 0 5 10 15
Time (seconds) Time (seconds) Time (seconds)

Pure integrator Higher (3rd) order Inverse response

50 2
1
40 1.5
0.8
Amplitude

Amplitude

Amplitude
1
30 0.6
1 6s+2
0.5
20
s 0.4 1 3s 2 +4s+1
6s 3 +11s 2 +6s+1 0
0.2
10
0.5
0
0 1
0 10 20 30 40 0 5 10 15 20 25 0 5 10 15 20 25
Time (seconds) Time (seconds) Time (seconds)

Figure 6.4: Typical open loop step tests for a variety of different standard plants.

Algorithm 6.1 FOPDT identification from a step response using the Method of Areas.

Collect some step response data (u, y) from the plant, then

1. Identify the plant gain,


y() y(0)
K=
u() u(0)
and normalise the experimental data to obtain a unit step

y y(0)
yus =
K

2. Compute the area


Z
A0 = (K yus (t)) dt (6.2)
0

3. Compute the time t1 = A0 /K, and then compute a second area


Z t1
A1 = yus (t) dt (6.3)
0

as illustrated in Fig. 6.5.

4. Now the two remaining parameters of the proposed plant model in Eqn. 6.1 are

eA1 A0 eA1
= and = (6.4)
K K
where e = exp(1). Note that we must constrain that the deadtime is positive for causal
plants.
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 243

1.2

0.8
A0
Output

0.6

0.4

0.2
A1

0
t0 Figure 6.5: Estimating a first-order plus
-0.2
0 5 10 15 20 25 30 35 40 45
time delay model from step response data
time using the Method of Areas.

Listing 6.1 gives a simple M ATLAB routine to do this calculation and Fig. 6.5 illustrate the integrals
to be computed. Examples of the identification is given in Fig. 6.6 for a variety of simulated plants
(1st and 2nd order, and an integrator) with different levels of measurement noise superimposed.
Fig. 6.7 shows the experimental results from a step test applied to the blackbox filter with the
identified model.

Listing 6.1: Identification of a first-order plant with deadtime from an openloop step response
using the Areas method from Algorithm 6.1.
Gp = tf(3, [7 2 1],'iodelay',2); % True plant: Gp (s) = 3e2s /(7s2 + 2s + 1)
[y,t] = step(Gp); % Do step experiment to generate data, see Fig. 6.5.

4 % Calculate areas A0 & A1 as shown in Fig. 6.5.


K = y(end); % plant gain, K
A0 = trapz(t,K-y);
t0 = A0/K;
idx = find(t<t0);
9 t1 = t(idx); y1 = y(idx);
A1 = trapz(t1,y1);

% Create process model Gp (s) = Kes /( s + 1)


tau = exp(1)*A1/K; % time constant,
14 Dest = max(0,(A0-exp(1)*A1)/K); % deadtime, , must be 0
Gm = tf(K,[tau 1],'iodelay',Dest);

step(Gp,Gm) % Compare model with plant, see Fig. 6.5.

Of course not all plants are adequately modelled using a first-order plus deadtime model. For
example, Fig. 6.8 shows the experimental results from a step test applied to the electromagnetic
balance arm which is clearly second order with poles very close to the imaginary axis.

However many plant operators do not appreciate the control engineer attempting to step test
the plant under open loop conditions since it necessarily causes periods of off-specification prod-
uct at best and an unstable plant at worst. An alternative to the step test is the impulse test where
the plant will also produce off-spec product for a limited period,1 but at least it returns to the same
operating point even using limited precision equipment. This is important if the plant exhibits
severe nonlinearities and the choice of operating point is crucial. The impulse response is ideal
for frequency domain testing since it contains equal amounts of stimulation at all frequencies, but
is difficult to approximate in practice. Given that we then must accept a finite height pulse with
finite slope, we may be limited as to the amount of energy we can inject into the system without
hitting a limiting nonlinearity. The impulse test is more difficult to analyse using manual graphi-
1 Provided the plant does not contain an integrator
244 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

8 8

6 6

Output

Output
4 4

2 2

0 Plant 0 Plant
Model Model
2 2
0 20 40 60 0 50 100 150
time time

8 6
Plant
6 4 Model

Output
Output
4
2
2
Figure 6.6: The Areas method for 0
0 Plant
plant identification applied to a va- Model
riety of different plants with random 2
0 5 10 15 20
2
0 0.5 1 1.5 2
disturbances. time time

600

400
Output

200

Blackbox
0 Model

1000

Figure 6.7: Identification of the experimental 500


Blackbox using the Method of Areas. Note
that the identification strategy is reasonably 0
robust to the noise and outlier in the experi- 0 5 10 15 20 25
mental data. time

cal techniques (pencil and graph paper), as opposed to computerised regression techniques, than
the step test.

R ANDOM INPUTS

An alternative to the step or impulse tests is actually not giving any deliberate input to the plant
at all, but just relying on the ever-present natural input disturbances. This is a popular approach
when one wants still to perform an identification, but cannot, or is unable to influence the system
themselves. Economists, astronomers and historians fall into this group of armchair-sitting, non-
interventionist identificationalists. This is sometimes referred to as time series analysis (TSA), see
[34] and [56].

Typically for this scheme to work in practice, one must additionally consciously perturb the input
(control valve, heating coils etc) in some random manner, since the natural disturbance may not
normally be sufficient in either magnitude, or even frequency content by itself. In addition, it is
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 245

3000

2000
Output

1000

1000
2000
Input

1000 Figure 6.8: Two step tests subjected to the bal-


ance arm. Note the slow decay of the oscilla-
0 tions indicate that the plant has at least two
0 5 10 15 complex conjugate poles very close to the sta-
time ble half of the imaginary axis.

more complex to check the quality of model solution from comparison with the raw time series
analysis. The plant model analysed could be anything, and from a visual inspection of the raw
data it is difficult to verify that the solution is a good one. The computation required for this type
of analysis is more complex than graphically curve fitting and the numerical algorithms used for
time series analysis are much more sensitive to the initial parameters. However using random
inputs has the advantage that they continually excite or stimulate the process and this excitation,
provided the random sequence is properly chosen, stimulates the process over a wide frequency
range. For complicated models, or for low signal to noise ratios, lengthy experiments may be
necessary, which is infeasible using one-off step or impulse tests. This property of continual
stimulation is referred to as persistent excitation and is a property of the input signal and should
be considered in conjunction with the proposed model. See [19, 2.4] for further details.

Two obvious choices for random sequences are selecting the variable from a finite width uniform
distribution, or from a normal distribution. In practice, when using a normal distribution, one
must be careful not to inject values far from the desired mean input signal, since this could cause
additional unwanted process upsets.

R ANDOM BINARY SIGNALS

Electrical engineers have traditionally used random signals for testing various components where
the signal is restricted to one of two discrete signal levels. Such a random binary signal is called an
RBS. A binary signal is sometimes preferred to the normal random signal, since one avoids possi-
ble outliers, and dead zone nonlinearities when using signals with very small amplitudes. We can
generate an RBS in M ATLAB using the random number generator and the signum function, sign,
as suggested in [126, p373]. Using the signum function could be dangerous since technically it
is possible to have three values, 1, 1 and zero. To get different power spectra of our binary test
signal, we could filter the white noise before we sign it.

x = randn(100,1); % white noise


rbsx = 0.0 x; % binary signal
3

y = filter(0.2,[1 -0.8],x); % coloured noise


rbsy = 0.0 y; % binary value
246 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Compare in Fig. 6.9 the white noise (left plots) with the filtered white noise through a low-pass
filter (solid). The filter used in Fig. 6.9 is 0.2/(1 0.8q 1 ).

White noise Coloured noise


4 1

2 0.5

0 0

2 0.5

4 1

Random binary noise Random binary coloured noise


1 1

0.5 0.5

0 0
0 50 100 0 50 100

Figure 6.9: Comparing white and filtered (coloured) random signals. Upper: Normal signals,
lower: Two-level or binary signals.

If you fabricate in hardware one of these random signal generators using standard components
such as flip-flops or XOR gates, rather than use a process control computer, then to minimise
component cost, you want to reduce the number of these physical components. This is done in
hardware using a linear feedback shift register (LFSR), which is an efficient approximation to our
(almost random) random number generator. Fig. 6.10 shows the design of one version which uses
a 5 element binary shift register. Initially the register is filled with all ones, and at each clock cycle,
the elements all shift one position to the right, and the new element is formed by adding the 3rd
and 5th elements modulus 2.

Since we have only 5 positions in the shift register in Fig. 6.10, the maximum length the sequence
can possibly be before it starts repeating is 25 1 or 31 elements. Fig. 6.11 shows a S IMULINK
implementation and the resulting binary sequence which repeats every 31 samples.

binary operation
+

1 1 1 1 1 output

| {z }
5-element shift register

Figure 6.10: A 5-element binary shift register to generate a pseudo-random binary sequence.
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 247

Math
Function Sum

mod

Constant
2

1 1 1 1 1
z z z z z
Unit Delay1 Unit Delay2 Unit Delay3 Unit Delay4 Unit Delay5 Scope

(a) S IMULINK block diagram

Pseudo random binary sequence of length 31


1.2

0.8

0.6

0.4

0.2

0.2
0 10 20 30 40 50 60 70
sample #

(b) S IMULINK simulation output

Figure 6.11: S IMULINK simulation of a 5-element binary shift register (upper) and resulting output
(lower). Note the period of 31 samples. Refer also to Fig. 6.10.

Random number generators such as Fig. 6.10 are completely deterministic so are called pseudo
random binary generators since in principal anyone can regenerate the exact sequence given the
starting point since the construction in Fig. 6.10. RBS can be used as alternatives to our random
sequences for identification. These and other commonly used test and excitation signals are com-
pared and discussed in [205, p143].

6.2.2 E XPERIMENTAL FREQUENCY RESPONSE ANALYSIS

When first faced with an unknown plant, it is often a good idea to perform a non-parametric model
identification such as a frequency response analysis. Such information, quantified perhaps as a
Bode diagram, can then be used for subsequent controller design, although is not suitable for
direct simulation. For that we would require a transfer function or equivalent, but the frequency
response analysis will aid us in the choice of appropriate model structure if we were to regress
further a parametric model.

The frequency response of an unknown plant G(s) can be calculated by substituting s = i and
following the definition of the Fourier transform, one gets
R
Y (i)
def y(t)eit dt
G(i) = = R0 (6.5)
X(i) 0
x(t)eit dt

For every value of frequency, 0 < < , we must approximate the integrals in Eqn. 6.5. If we
choose the input to be a perfect Dirac pulse, then the denominator in Eqn. 6.5 will always equal
1. This simplifies the computation, but complicates the practical aspects of the experiment. If we
use arbitrary inputs, we must compute the integrals numerically, although they are in fact simply
the Fourier transform of y(t) and x(t) and can be computed efficiently using the Fast Fourier
Transform or FFT.
248 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

We will compare three alternative techniques to compute the frequency response of an actual
experimental laboratory plant, namely:

1. subjecting the plant to a series of sinusoids of different frequencies,

2. subjecting the plant to a single sinusoid with a time-varying frequency component (chirp
signal),

3. and subjecting a plant to a pseudo-random white noise input.

In the three alternatives, we must record (or compute) the steady-state amplitude ratio of the
output over the input and the phase lag of the output compared with the input for different input
frequencies.

Finally, to extract estimates of the model parameters from the frequency data, you could use the
M ATLAB functions invfreqs or the equivalent for the discrete domain, invfreqz. I have no
idea how robust this fitting works, and I would treat the results very cautiously at first. (Also
refer to problem 6.1.)

A SERIES OF INPUT SINE WAVES

The simplest way to experimentally characterise the frequency response of a plant is to input a
series of pure sine waves at differing frequencies as shown in in the S IMULINK diagram Fig. 6.12
which uses the real-time toolbox to communicate with the experimental blackbox.

Terminator uy
Simulink
Execution To Workspace
Control Sine Wave

DAC ADC

Saturation Scope
MCC DAQ DAC OUT MCC DAQ ADC IN

Maximum real time sampling rate is around 0.1s

Figure 6.12: Using the real-time toolbox in M ATLAB to subject sine waves to the blackbox for an
experimental frequency analysis.

Fig. 6.13 shows the response of the laboratory blackbox when subjected to 12 sine waves of in-
creasing frequency spanning from = 0.2 to 6 radians/second. The output clearly shows a re-
duction in amplitude at the higher frequencies, but also a small amplification at modest frequences.
By carefully reading the output amplitude and phase lag, we can establish 12 experimental points
given in table 6.1 suitable say for a Bode diagram.

However while conceptually simple, the sine wave approach is not efficient nor practical. Lees
and Hougen, [119], while attempting a frequency response characterisation of a steam/water heat
exchanger noted that it was difficult to produce a good quality sine wave input to the plant when
using the control valve. In addition, since the ranges of frequencies spanned over 3.5 orders of
magnitude (from 0.001 rad/s to 3 rad/s), the testing time was extensive.

Additional problems of sine-wave testing are that the gain and phase lag is very sensitive to bias,
drift and output saturation. In addition it is often difficult in practice to measure accurately phase
lag. [145, pp117-118] give further potential problems and some modifications to lessen the effects
of nonlinearities. The most important conclusion from the Lees and Hougen study was that they
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 249

= 0.2 = 0.3 = 0.5


2

= 0.8 = 1.0 = 1.3


2

= 1.5 = 2.0 = 2.5


2

2
0 5 10 15 20 25

= 3.0 = 4.0 = 6.0


2

2
0 5 10 15 20 25 0 5 10 15 20 25 0 5 10 15 20 25

Figure 6.13: The black box frequency response analysis using a series of sine waves of different
frequencies. This data was subsequently processed and is tabulated in Table 6.1 and plotted in
Fig. 6.20.

Table 6.1: Experimentally determined frequency response of the blackbox derived from the labo-
ratory data shown in Fig. 6.13. For a Bode plot of this data, see Fig. 6.20.

Input frequency Amplitude ratio Phase lag


(rad/s) (degrees)
0.2 1.19 -4.8
0.3 1.21 -5.1
0.5 1.27 -11.1
0.8 1.42 -20.8
1.0 1.59 -27.6
1.3 1.92 -46.7
1.5 2.08 -64.4
2.0 1.61 -119.2
2.5 0.88 -144.3
3.0 0.54 -156.8
4.0 0.27 -172.6
6.0 0.11 -356.1
250 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

obtained better frequency information, with much less work using pulse testing and subsequently
analysing the input/output data using Fourier transforms.

A CHIRP SIGNAL

If we input a signal with a time varying frequency component such as a chirp signal, u(t) =
sin(t2 ), to an unknown plant, we can obtain an idea of the frequency response with a single,
albeit long drawn-out experiment unlike the series of experiments required in the pure sinusoidal
case.

Fig. 6.14 shows some actual input/output data collected from the blackbox laboratory plant
collected at T = 0.1 seconds. As the input frequency is increased, the output amplitude decreases
as expected for a plant with low-pass filtering characteristics. However the phase lag information
is difficult to determine at this scale from the plot given in Fig. 6.14. Our aim is to quantify this
amplitude reduction and phase lag as a function of frequency for this experimental plant.

Experimental Blackbox
1

0.8

0.6
output

0.4

0.2

0
0 50 100 150 200 250 300 350

0.2

0.4
input

0.6

0.8

1
0 50 100 150 200 250 300 350
time (s)

Figure 6.14: Input (lower) /output (upper) data collected from the black box where the input is a
chirp signal.

Since the phase information is hard to visualise in Fig. 6.14, we can plot the input and output
signals as a phase plot similar to a Lissajou figure. The resulting curve must be an ellipse (since the
input and output frequencies are the same), but the ratio of the height to width is the amplitude
ratio, and the eccentricity and orientation is a function of the phase lag.

If we plot the start of the phase plot using the same input/output data as Fig. 6.14, we should see
the trajectory of an ellipse gradually rotating around, and growing thinner and thinner. Fig. 6.15
shows both the two-dimensional phase plot and a three dimensional plot with the time axis
added.

Fig. 6.16 shows the same experiment performed on the flapper. This experiment indicates that the
flapper has minimal dynamics since there is little discernable amplitude attenuation and phase
lag but substantial noise. The saturation of the output is also seen in the squared edges of ellipse
in the phase plot in addition to the flattened sinusoids of the flapper response.
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 251

Phase plot of the experimental Blackbox


1 Phase plot of the experimental Blackbox

Start
1
0.9

0.9
0.8
0.8
0.7
0.7

0.6 0.6

output (V)
output (V)

0.5 0.5

0.4
0.4

0.3
0.3
0.2

0.2
0.1
0
0.1
0.5
350 400
250 300
150 200
0 1 50 100
0
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 input (V)
input (V) time (s)

(a) 2D input/output phase plot (b) 2D input/output phase plot with time in the third di-
mension

Figure 6.15: Phase plot of the experimental black box when the input is a chirp signal. See also
Fig. 6.14.

Experimental Flapper Phase plot of the experimental Flapper


0.5 0.5

0.4

0.3 0.4
0.2
output

0.1
0.3
0

0.1

0.2 0.2 Start


output (V)

0 50 100 150 200 250 300 350 400

0 0.1

0.2
0
0.4
input

0.6
0.1

0.8

1 0.2
0 50 100 150 200 250 300 350 400 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
time (s) input (V)

(a) Time response (b) Phase plot of the experimental flapper

Figure 6.16: Flapper response to a chirp signal. There is little evidence of any appreciable dynam-
ics over the range of frequencies investigated.

A PSEUDO RANDOM INPUT

A more efficient experimental technique is to subject the plant to an input signal that contains
a wide spectrum of input frequencies, and then compute the frequency response directly. This
has the advantage that the experimentation is considerably shorter, potentially processes the data
more efficiently and removes some of the human bias.

The experimental setup in Fig. 6.17 shows an unknown plant subjected to a random input. If we
collect enough input/output data pairs, we should be able to estimate the frequency response of
the unknown plant.

Fig. 6.18 compares the magnitude and phase of the ratio of the Fourier transforms of the input and
output to the analytically computed Bode plot given that the unknown plant is in fact G(s) =
e2s /(5s2 + s + 6).

The actual numerical computations to produce the Bode plot in Fig. 6.18 are given in Listing 6.2.
252 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

G
u y
Band-Limited Unknown plant
White Noise
Figure 6.17: Experimental setup to
subject a random input into an un-
known plant. The input/output data
was collected, processed through List-
ing 6.2 to give the frequency response 1
shown in Fig. 6.18. UY

0
10

1
10

2
|G(i )| 10

3
10

4
10

5
10

0
Phase angle [degrees]

200

400

Figure 6.18: The experimental frequency 600

response compared to the true analytical 10


3
10
2
10
1
10
0 1
10
2
10
fN
Bode diagram. See the routine in Listing 6.2. frequency [rad/s]

While the listing looks complex, the key calculation is the computation of the fast Fourier trans-
form, and the subsequent division of the two vectors of complex numbers. It is important to note
that S IMULINK must deliver equally spaced samples necessary for the subsequent FFT transfor-
mation, and that the frequency vector is equally spaced from 0 to half the Nyquist frequency.

Listing 6.2: Frequency response identification of an unknown plant directly from input/output
data
G = tf(1,[5 1 6],'iodelay',2); % Unknown plant under test
Ts = 0.2; timespan = [0 2000]; % sample time, Ts , & period of interest
sl_optns = simset('RelTol',1e-5,'AbsTol',1e-5, ...
'MaxStep',1e-2,'MinStep',1e-3);
5 [t,x,UY] = sim('splant_noiseIO',timespan,sl_optns); % Refer Fig. 6.17.

n = ceil((pow2(nextpow2((length(t)))-1)+1)/2);
t = t(1:n); % discard down to a power of 2 for the FFT
u = UY(1:n,1); y = UY(1:n,2);
10

Fs = 1/Ts; Fn = Fs/2; % sampling & Nyquist frequency [Hz]


Giw = fft(y)./fft(u); % G(i) = FFT (Y )/FFT (U )
w=2*pi*(0:n-1)*2*Fn/n; % frequency in [rad/s]

15 [Mag,Phase,wa] = bode(G);% Analytical Bode plot for comparison


Mag = Mag(:); Phase = Phase(:);

subplot(2,1,1); % Refer Fig. 6.18.


loglog(w,abs(Giw),wa,Mag,'r-'); ylabel('|G(i \omega)| ')
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 253

20 subplot(2,1,2);
semilogx(w,phase(Giw)*180/pi, wa,Phase,'r-');
ylim([-720,10])
ylabel('Phase angle [degrees]'); xlabel('frequency \omega [rad/s]');

We can try the same approach to find the frequency response of the blackbox experimentally.
Fig. 6.19 shows part of the 213 = 8192 samples collected at a sampling rate of 10Hz or T = 0.1 to
be used to construct the Bode diagram.

4
output

0
4

3
input

0 50 100 150
time [s]

Figure 6.19: Part of the experimental black box response data series given a pseudo-random input
sequence. This data was processed to produce the frequency response given in Fig. 6.20.

The crude frequency response obtained simply by dividing the two Fourier transforms of the
input/output data from Fig. 6.19 is given in Fig. 6.20 which, for comparison, also shows the
frequency results obtained from a series of distinct sine wave inputs and also from using the
M ATLAB toolbox command etfe which will be subsequently explained in section 6.2.3.

Since we have not employed any windows in the FFT analysis, the spectral plot is unduly noisy,
especially at the high frequencies. The 12 amplitude ratio and phase lag data points manually
read from Fig. 6.13 (repeated in Table 6.1) show good agreement across the various methods.

6.2.3 A N ALTERNATIVE EMPIRICAL TRANSFER FUNCTION ESTIMATE

The frequency response obtained from experimental input/output data shown in Fig. 6.18 and
particularly Fig. 6.20 illustrate that the simple approach of simply numerically dividing the Fourier
transforms is suspect to excessive noise. A better approach is to take more care in the data pre-
processing stage by judicious choice of windows. The empirical transfer function estimate rou-
tine, etfe, from the System Identification toolbox does this and delivers more reliable and con-
sistent results. Compare the result given in Fig. 6.21 from Listing 6.3 with the simpler strategy
used to generate Fig. 6.18.

Listing 6.3: Non-parametric frequency response identification using etfe.


G = tf(1,[5 1 6],'iodelay',2); % Plant under test: G(s) = e2s /(5s2 + s + 6)
2 Ts = 0.2; t = [0:Ts:2000]'; % sample time & span of interest

u = randn(size(t)); y = lsim(G,u,t);

Dat = iddata(y,u,Ts); % collect data from simulation experiment


254 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

0
10
|G(i )|

1
10

50

0
Phase angle [degrees]

50

100

150

200
FFT
250 etfe
sine waves
300
2 1 0 1 2
10 10 10 10 10
frequency [rad/s]

Figure 6.20: The frequency response of the black box computed using FFTs (cyan line), the Sys-
tem ID toolbox function etfe, (blue line) and the discrete sine wave input data, (), read from
Fig. 6.13. The FFT analysis technique is known to give poor results at high frequencies.

7 Gmodel = etfe(Dat); % estimate a frequency response model


bode(Gmodel) % Refer Fig. 6.21.

Problem 6.1 1. Plot the frequency response of an unknown process on both a Bode diagram
and the Nyquist diagram by subjecting the process to a series of sinusoidal inputs. Each
input will contribute one point on the Bode or Nyquist diagrams. The unknown transfer
function is
0.0521s2 + 0.4896s + 1
G(s) =
0.0047s + 0.0697s3 + 0.5129s2 + 1.4496s + 1
4

Compare your plot with one generated using bode. Note that in practice we do not know
the transfer function in question, and we construct a Bode or Nyquist plot to obtain it. This
is the opposite of typical textbook student assignments.
Hint: Do the following for a dozen or so different frequencies:
Gcn=[0.0521 0.4896 1];
(a) First specify the unknown continuous time Gcd=[0.0047 0.0697 0.5129 1.4496 1];
transfer function. w=2; % rad/s
(b) Choose a test frequency, say = 2 rad/s, and t=[0:0.1:20]; % seconds
generate a sine wave at this frequency. u=sin(w*t); % input signal
y=lsim(Gcn,Gcd,u,t); plot(t,[u y])
(c) Simulate the output of the process given this in-
put. Plot input/output together and read off the
phase lag and amplitude ratio.
6.3. CONTINUOUS MODEL IDENTIFICATION 255

0
10
|G(i )|

1
10

Estimated
Actual
2
10
0

200
[deg]

Figure 6.21: An empirical transfer function


400 estimate from experimental input/output
data computed by etfe. The true fre-
600
quency response is overlayed for compari-
0.3 0.5 1 2 3
son. This is preferred over the simpler strat-
[rad/s] egy used to generate Fig. 6.18.

Note that the perfect frequency response can be obtained using the freqs command.
2. The opposite calculation, that is fitting a transfer function to an experimentally obtained fre-
quency response is trivial with the invfreqs command. Use this command to reconstruct
the polynomial coefficients of G(s) using only your experimentally obtained frequency re-
sponse.

Problem 6.2 Write a M ATLAB function file to return Gp (i) given the input/output x(t), y(t) us-
ing Eqn6.5 for some specified frequency. Test your subroutine using the transfer function from
Problem 6.1 and some suitable input signal, and compare the results with the exact frequency
response.
Hint: To speed up the calculations, you should vectorise the function as much as possible. (Of
course for the purposes of this educational example, do not use an FFT)

6.3 C ONTINUOUS MODEL IDENTIFICATION

Historically graphical methods, especially for continuous time domain systems, used to be very
popular partly because one needed little more than pencil and graph paper. For first-order or for
some second-order models common in process control, we have a number of graphical recipes
which if followed, enable us to extract reasonable parameters. See [183, Chpt 7] for further details.

While we could in principle invert the transfer function given a step input to the time domain
and then directly compare the model and actual outputs, this has the drawback that the solutions
are often convoluted nonlinear functions of the parameters. For example, the solutions to a step
response for first and second order overdamped systems are:
K  
y(t) = K 1 et/
s + 1
 
K 1 et/1 2 et/2
y(t) = K 1
(1 s + 1)(2 s + 1) 1 2
and the problem is that while the gain appears linearly in the time domain expressions, the time
constants appear nonlinearly. This makes subsequent parameter estimation difficult, and in gen-
256 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

eral we would need to use a tool such as the O PTIMISATION toolbox to find reasonable values for
the time constants. The following section describes this approach.

6.3.1 F ITTING TRANSFER FUNCTIONS USING NONLINEAR LEAST- SQUARES

Fig. 6.22 shows some actual experimental data collected from a simple black box plant subjected
to a low-frequency square wave. The data was sampled at 10Hz or Ts = 0.1. You can notice that
in this case the noise characteristics seems to be clearly dependent on the magnitude of y. By
collecting this input/output data we will try to identify a continuous transfer function.

3.5

3
output

2.5

1.5
3

2.5

2
0 10 20 30 40 50 60 70 80 90 100
time (s)

Figure 6.22: Experimental data from a continuous plant subjected to a square wave.

We can see from a glance at the response in Fig. 6.22 that a suitable model for the plant would be
an under-damped second order transfer function

Keas
G(s) = (6.6)
2 s2 + 2 s + 1
def  T
where we are interested in estimating the four model parameters of Eqn. 6.6 as = K a .
This is a nonlinear least-squares regression optimisation problem we can solve using lsqcurvefit.
Listing 6.4 will be called by the optimisation routine in order to generate the model predictions
for a given trial set of parameters.

Listing 6.4: Function to generate output predictions given a trial model and input data.
function ypred = fmodelsim(theta, t,u)
2 % Compute the predicted output, y(t), given a model & input data, u(t).
K = theta(1); tau = theta(2); zeta = theta(3); deadt = theta(4);

Keas
G = tf(K,[tau2 2*tau*zeta, 1],'iodelay',deadt); % G(s) = 2 s2 +2 s+1
ypred = lsim(G,u,t);
7 return

The optimisation routine in Listing 6.5 calls the function given in Listing 6.4 repeatedly in order
to search for good values for the parameters such that the predicted outputs match the actual
outputs. In parameter fitting cases such as these, it is prudent to constrain the parameters to be
non-negative. This is particularly important in the case of the deadtime where we want to ensure
that we do not inadvertently try models with acausal behaviour.

Listing 6.5: Optimising the model parameters.


6.3. CONTINUOUS MODEL IDENTIFICATION 257

K0 = 1.26; tau0 = 1.5; zeta0 = 0.5; deadt0 = 0.1; % Initial guesses for 0
theta0 = [K0,tau0,zeta0,deadt0]';
3 LowerBound = zeros(size(theta0)); % Avoid negative deadtime
theta_opt = lsqcurvefit(@(x,t) fmodelsim(x,t,u), theta0,t,y,LowerBound)

Once we have some optimised values, we should plot the predicted model output on top of the
actual output as shown in Fig. 6.23 which, in this case, is not too bad.

Listing 6.6: Validating the fitted model.


5 K = theta_opt(1); tau = theta_opt(2);
zeta = theta_opt(3); deadt = theta_opt(4);
Gopt = tf(K,[tau2 2*tau*zeta, 1],'iodelay',deadt);

ypred = lsim(Gopt,u,t);
10 plot(t,y,t,ypred,'r-') % See results of the comparison in Fig. 6.23.

0.5

0.5
output

1.5
Actual 1.166
fitted
G(s) = 0.5712 s2 +0.317s+1
2
0 10 20 30 40 50 60 70 80 90 100
time (s)

Figure 6.23: A continuous-time model fitted to input/output data from Fig. 6.22.

The high degree of similarity between the model and the actual plant shown in Fig. 6.23 suggests
that, with some care, this method does work in practice using real experimental data complete
with substantial noise. However on the downside, we did use a powerful nonlinear optimiser,
we were careful to eliminate the possibility any of the parameters could be negative, we chose
sensible starting estimates, and we used nearly 1000 data points.

6.3.2 I DENTIFICATION OF CONTINUOUS MODELS USING SUCCESSIVE DIF -


FERENTIATION

While most modern system identification and model based control is done in the discrete do-
main, it is possible, as shown above, to identify the parameters in continuous models, G(s), di-
rectly. This means that we can establish the continuous time poles and zeros of a transfer function
without first identifying a discrete time model, and then converting back to the continuous time
domain.

However there are two main reasons why continuous time model identification is far less useful
than its discrete time equivalent. The first reason is that since discrete time models are more use-
ful for digital control, it makes sense to directly estimate them rather than go through a temporary
258 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

continuous model. The second drawback is purely practical. Either we must resort to the delicate
task of applying nonlinear optimisation techniques such as illustrated above, or if we wish to use
the robust linear least-squares approach, we need reliably to estimate high order derivatives of
both the input and output data. In the presence of measurement noise and possible discontinuity
in gradient for the input such as step changes, it is almost impossible to construct anything higher
than a second-order derivative, thus restricting our models to second order or less. Notwithstand-
ing these important implementation issues, [31] describes a recursive-least squares approach for
continuous system identification.

As continuous time model identification requires one to measure not only the input and output,
but also the high order derivatives of y(t) and u(t), it turns out that this construction is delicate
even for well behaved simulated data, so we would anticipate major practical problems for actual
experimental industrial data.

If we start with the continuous transfer function form,

B(s)
Y (s) = U (s)
A(s)

then by rationalising and expanding out the polynomials in s we get


 
an sn + an1 sn1 + + a1 s + 1 Y (s) = bm sm + bm1 sm1 + + b1 s + b0 U (s)

Inverting to the time domain and assuming zero initial conditions, we get a differential equation

an y (n) + an1 y (n1) + + a1 y (1) + y = bm um + bm1 u(m1) + + b1 u(1) + b0 u

where the notation y (n) means the n-th time derivative of y(t). This can be written with some
rearrangement as a vector dot product

an

an1

..

.

a1
i
h
..
y= u (6.7)

y (n) y (n1) y (1) . u(m) u(m1)
bm

bm1

.
..
b0

which is now in a form suitable for least-squares regression of the vector of unknown coefficients
of the A and B polynomials. Note that if we desire na poles, we must differentiate the output na
times. Similarly, we must differentiate the input once for each zero we want to estimate. A block
diagram of this method is given in Fig. 6.24.

A simple simulation is given in Listing 6.7. I first generate a truth model with four poles and
three zeros. Because I want to differentiate the input, I need to make it sufficiently smooth by
passing a random input through a very-low-pass filter. To differentiate the logged data I will use
the built-in gradient command which approximates the derivative using finite differences. To
obtain the higher-order differentials, I just repeatedly call gradient. With the input/output data
and its derivatives, the normal equations are formed, and solved for the unknown polynomial
coefficients.

Listing 6.7: Continuous model identification of a non-minimum phase system


Gplant = tf(3*poly([3,-3,-0.5]), ...
6.3. CONTINUOUS MODEL IDENTIFICATION 259

parameters,

differentiator chain
s s
s s
Regress
s s
s s

Input u(t) b
Plant b Output y(t)

Plant under investigation

Figure 6.24: Continuous model identification strategy

3(s3)(s+3)(s+0.5)
poly([-2 -4 -7 -0.4])); % G(s) = (s+7)(s+4)(s+2)(s+0.4)

4 % Design smooth, differentiable input


dt = 5e-2; t = [0:dt:10]'; % Keep sample time T small
Ur = randn(size(t)); % white noise
[Bf,Af]= butter(4,0.2); u= filter(Bf,Af,Ur); % Input now smooth

9 y = lsim(Gplant,u,t); plot(t,[u,y]); % do input/output experiment

% Now do the identification . . .


na = 2; nb = 1; % # of poles & zeros to be estimated. (Keep small).
dy = y; % Zeroth derivative, y (0) = y
14 for i=1:na % build up higher derivatives recursively, y (1) , y (2) , . . .
dy(:,i+1) = gradient(dy(:,i),t); % should be correctly scaled
end % for
du = u; % do again for input signal
for i=1:nb % but only for nb times
19 du(:,i+1) = gradient(du(:,i),t); % could use diff()
end % for

b = y; % RHS
X = [-dy(:,2:na+1), du]; % data matrix
24 theta = X\b; % solve for parameters,

% re-construct linear model polynomials


Gmodel = tf(fliplr(theta(na+1:length(theta))'), ...
fliplr([1,theta(1:na)']))
29

yp = lsim(Gmodel,u,t); plot(t,[y],t,yp,'--'); % test simulation

Even though we know the truth model possessed four zeros and three poles, our fitted model
with just two poles and one zero gave a close match as shown in Fig. 6.25. In fact the plant output
(heavy line) is almost indistinguishable from the predicted output (light line) and this is due to
the lack of noise and smoothed input trajectory. However if you get a little over ambitious in
your identification, the algorithm will go unstable owing to the difficulty in constructing reliable
derivatives.
260 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

0.6
Plant
0.4 Model

output 0.2

0.2

0.4
1

0
input

2
0 1 2 3 4 5 6 7 8 9 10
time

Figure 6.25: A simulated example of a continuous model identification comparing the actual plant
with the model prediction showing almost no discernable error. Compare these results with an
actual experiment in Fig. 6.26.

6.3.3 P RACTICAL CONTINUOUS MODEL IDENTIFICATION

We may be tempted given the results presented in Fig. 6.25 to suppose that identifying a contin-
uous transfer function from logged data is relatively straight forward. Unfortunately however, if
you actually try this scheme on a real plant, even a well behaved plant such as the black-box, you
will find practical problems of such magnitude that it renders the scheme is almost worthless. We
should also note that if the deadtime is unknown, then the identification problem is non-linear in
the parameters which makes the subsequent identification problematic. This is partly the reason
why many common textbooks on identification and control pay only scant attention to identifica-
tion in the continuous domain, but instead concentrate on identifying discrete models. (Discrete
model identification is covered next in 6.4.)

The upper plot in Fig. 6.26 shows my attempt at identifying a continuous model of the blackbox,
which, compared with Fig. 6.25, is somewhat less convincing. Part of the problem lies in the diffi-
culty of constructing the derivatives of the logged input/output data. The lower plot in Fig. 6.26
shows the logged output combined with the first and second derivatives computed crudely using
finite differences. One can clearly see the compounding effects on the higher derivatives due to
measurement noise, discretisation errors, and so forth.

For specific cases of continuous transfer functions, there are a number of optimised algorithms
to directly estimates the continuous coefficients. Himmelblau, [86, p358360] gives once such
procedure for which reduces to a linear system which is easy to solve in M ATLAB. However
again this algorithm essentially suffers the same drawbacks as the sequential differential schemes
given previously.

I MPROVING IDENTIFICATION RELIABILITY BY USING L AGUERRE FUNCTIONS

One scheme that has found to be practical is one based on approximating the continuous response
with a series of orthogonal functions known as Laguerre functions. The methodology was devel-
oped at the University of British Columbia and is available commercially from BrainWave, now
part of Andritz Automation. Further details about the identification strategy are given in [202].
6.4. POPULAR DISCRETE-TIME MODELS STRUCTURES 261

Continuous model identification


0.1

output & prediction


0.05

0.05
Blackbox
Model
0.1
0 2 4 6 8 10 12 14 16 18 20
Sample time T=0.05 s

Measured output & derivatives


0.15

0.1

0.05
2
y, dy/dt, d y/dt

0
2

0.05

0.1

0.15

0.2
0 2 4 6 8 10 12 14 16 18 20
time (s)

Figure 6.26: A practical continuous model identification of the blackbox is not as successful as the
simulated example given in Fig. 6.25. Part of the problem is the reliable construction of the higher
derivatives. Upper: The plant (solid) and model prediction (dashed). Lower: The output y, and
derivatives y, y.

Compared to the standard curve fitting techniques, this strategy using orthogonal functions is
more numerically robust, and only involves numerical integration of the raw data, (as opposed to
the troublesome differentiation). On the downside, manipulating the Laguerre polynomial func-
tions is cumbersome, and the user must select an appropriate scaling factor to obtain reasonable
results. While [202] suggest some rules of thumb, and some algorithms for choosing the scaling
factor, it is still delicate and sensitive calculation and therefore tricky in practice.

Suppose we wish to use Laguerre functions to identify a complex high-order transfer function

(45s + 1)2 (4s + 1)(2s + 1)


G(s) = (6.8)
(20s + 1)3 (18s + 1)3 (5s + 1)3 (10s + 1)2 (16s + 1)(14s + 1)(12s + 1)

which was used as a challenging test example in [202, p32]. To perform the identification, the user
must decide on an appropriate order, (8 in this case), and a suitable scaling factor, p.

Fig. 6.27 compares the collected noisy step data with the identified step and impulse responses,
compares the identified and gives the error in the impulse response.

6.4 P OPULAR DISCRETE - TIME MODELS STRUCTURES

The previous section showed that while it was possible in theory to identify continuous models
by differentiating the raw data, in practice this scheme lacked robustness. Furthermore it makes
sense for evenly spaced sampled data systems not to estimate continuous time models, but to
estimate discrete models directly since we have sampled the plant to obtain the input/output
262 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

T =493.1, p =0.05
ss opt
1.5

Step test 1

0.5

T
ss
0.5

Step test: Order = 8, scale factor p=0.050


0.04
Inpulse comparison

0.03

0.02

0.01

0.01

0.05
error

0.05
0 100 200 300 400 500 600 700 800
time [s]

Figure 6.27: Identification of the 14th order system given in Eqn. 6.8 using an 8th-order Laguerre
series and a near-optimal scaling parameter.

data anyway. This section describes the various common forms of discrete models that we will
find useful in modelling and control. In most practical cases a linear model will suffice, but for
times when that is not adequate, section 6.4.4 describes simple nonlinear extensions.

If we collect the input/output data at regular sampling intervals, t = kT , for most applications it
suffices to use linear difference models such as

y(k) + a1 y(k 1) + a2 y(k 2) + + an y(k n) = b0 u(k) + b1 u(k 1) + + bm u(k m) (6.9)

where in the above model, the current output y(k) is dependent on m + 1 past and present inputs
u and the n immediate past outputs y. The ai s and bi s are the model parameters we seek.

A shorthand description for Eqn. 6.9 is

A(q 1 ) y(k) = B(q 1 )u(k) + e(k) (6.10)

where we have added a noise term, e(k), and where q 1 is defined as the backward shift-operator.
A signal flow diagram of this model is given in Fig. 6.28. Note that the grey dotted box completely
encloses the noise term e since in most practical cases, we cannot directly observe this term.
6.4. POPULAR DISCRETE-TIME MODELS STRUCTURES 263

noise, e


1
known input, u B output, y
A

Figure 6.28: A signal flow diagram of an auto-regressive model with exogenous input or ARX
model. Compare this structure with the similar output-error model in Fig. 6.31.

The polynomials A(q 1 ), B(q 1 ) in Eqn. 6.10 are known as shift polynomials

A(q 1 ) = 1 + a1 q 1 + a2 q 2 + + an q n
B(q 1 ) = b0 + b1 q 1 + b2 q 2 + + bm q m

where the A(q 1 ) polynomial is defined by convention to be monic, that is, the leading coeffi-
cient is 1. In the following development, we will typically drop the argument of the A and B
polynomials.

Even with the unmeasured noise term, we can still estimate the current y(k) given old values of y
and old and present values of u using

y(k|k 1) = a1 y(k 1) an y(k n) + b0 u(k) + b1 u(k 1) + + bm u(k m) (6.11)


| {z } | {z }
past outputs present & past inputs

= (1 A)y(k) + Bu(k) (6.12)

Our estimate or prediction of the current y(k) based on information up to, but not including k is
y(k|k 1). If we are lucky, our estimate is close to the true value, or y(k|k 1) y(k). Note
that the prediction model of Eqn. 6.11 involves just known terms, the model parameters and past
measured data. It does not involve the unmeasured noise terms.

The model described by Eqn. 6.10 is very common and is called an Auto-Regressive with Exoge-
nous, (externally generated) input, or an ARX model. The auto-regressive refers to the fact that
the output y is dependent on old ys, (i.e. it is regressed on itself), in addition to an external input,
u.

There is no reason why we could not write our models such as Eqn. 6.10 in the forward shift
operator, q as opposed to the backward shift q 1 . Programmers and those working in the digital
signal processing areas tended to favour the backward shift because it is easier and more natural
to write computational algorithms such as Eqn. 6.11, whereas some control engineers favoured
the forward notation because the time delay is given naturally by the difference in order of the A
and B polynomials.

6.4.1 E XTENDING THE LINEAR MODEL

Since the white noise term in Eqn. 6.10 is also filtered by the plant denominator polynomial, A(q),
this form is known as an equation error model structure. However in many practical cases we will
need more flexibility when describing the noise term. One obvious extension is to filter the noise
264 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

term with a moving average filter


y(k) + a1 y(k 1) + + an y(k n) = b0 u(k) + b1 u(k 1) + + bm u(k m)
| {z }
ARX model
+ e(k) white noise
+ c1 e(k 1) + + cn e(k n)
| {z }
coloured noise

This is now known as coloured noise, the colouring filter being the C polynomial. The A UTO -
R EGRESSIVE M OVING -AVERAGE with E X OGENOUS input, ARMAX, model is written in compact
polynomial form as
A(q)y(k) = B(q)u(k) + C(q)e(k) (6.13)
where we have dropped the notation showing the explicit dependance on sample interval k. The
ARMAX signal flow is given in Fig. 6.29. This model now has two inputs, one deterministic u,
and one disturbing noise, e, and one output, y. If the input u is zero, then the system is known
simply as an ARMA process.

noise, e


1
known input, u B output, y
A

Figure 6.29: A signal flow diagram of a ARMAX model. Note that the only difference between
this, and the ARX model in Fig. 6.28, is the inclusion of the C polynomial filtering the noise term.

While the ARMAX model offers more flexibility than the ARX model, the regression of the pa-
rameters is now nonlinear optimisation problem. The nonlinear estimation of the parameters in
the A, B and C polynomials is demonstrated on page 305. Also note that the noise term is still
filtered by the A polynomial meaning that both the ARX and ARMAX models are suitable if the
dominating noise enters the system early in the process such as a disturbance in the input variable.

S IMULATING AN ARMAX MODEL

We can easily simulate discrete time models in raw M ATLAB, although the handling the indices
and initialisation of the filters requires some care. Being able to simulate various discrete time
models is useful before we start regressing the model parameters in section 6.5.

Suppose we have an ARMAX model of the form Eqn. 6.13 with coefficient matrices
A(q) = q 2 q + 0.5, B(q) = 2q + 0.5, C(q) = q 2 0.2q + 0.05
Note that these polynomials are written in the forward shift operator, and the time delay given
by the difference in degrees, na nb , is 1. A M ATLAB script to construct the model is:

A = [1 -1 0.5]; % Model A(q) = q 2 q + 0.5


B = [2 0.5]; % No leading zeros B(q) = 2q + 0.5
C = [1 -0.2 0.05]; % Noise model C(q) = q 2 0.2q + 0.05
ntot = 10; % # of simulated data points
5 U = randn(ntot,1); E = randn(ntot,1);
6.4. POPULAR DISCRETE-TIME MODELS STRUCTURES 265

Now to compute y given the inputs u and e, we use a for-loop and we write out the equation
explicitly as

y(k + 2) = y(k + 1) 0.5y(k) + 2u(k + 1) + 0.5u(k) + e(k + 2) 0.2e(k + 1) + 0.05e(k)


| {z } | {z } | {z }
old outputs old inputs noise

It is more convenient to wind the indices back two steps so we calculate the current y(k) as op-
posed to the future y(k + 2) as above. We will assume zero initial conditions to get started.

n = length(A)-1; % Order na
z = zeros(n,1); % Padding: Assume ICs = zero
Uz = [z;U]; Ez = [z;E]; y = [z;NaN*U]; % Initialise
d = length(A)-length(B); % deadtime = na nb
5 zB = [zeros(1,d),B]; % pad B(q) with leading zeros

for i=n+1:length(Uz)
y(i) = -A(2:end)*y(i-1:-1:i-n) + zB*Uz(i:-1:i-n) + C*Ez(i:-1:i-n);
end
10 y(1:n) = []; % strip off initial conditions

This strategy of writing out the difference equations explicitly is rather messy and error-prone,
so it is cleaner, but less transparent, to use the idpoly routine to create an armax model and the
idmodel/sim command in the identification toolbox to simulate it. Again we must pad with
zeros the deadtime onto the front of the B polynomial.

>>G = idpoly(A,[0,B],C,1,1) % Create polynomial model A(q)y(t) = B(q)u(t) + C(q)e(t)


Discrete-time IDPOLY model: A(q)y(t) = B(q)u(t) + C(q)e(t)
A(q) = 1 - q-1 + 0.5 q-2

5 B(q) = 2 q-1 + 0.5 q-2

C(q) = 1 - 0.2 q-1 + 0.05 q-2

This model was not estimated from data.


10 Sampling interval: 1

>>y = sim(G,[U E]); % test simulation

Both methods (writing the equations out explicitly, or using idpoly) will give identical results.

We can replicate this simulation in S IMULINK by using discrete filters as shown in Fig. 6.30 which
directly follows Fig. 6.29. For the B and C blocks I could either use a FIR filter block, or the more
general IIR discrete filter with simply a denominator of 1. Note that we must ensure that the B
polynomial is padded with one zero to take account of the one unit of deadtime.

6.4.2 O UTPUT ERROR MODEL STRUCTURES

Both the linear ARX model of Eqn. 6.10, and the version with the filtered noise, Eqn. 6.13, as-
sume that both the noise and the input are filtered by the plant denominator, A. A more flexible
approach is to treat the input and noise sequences separately as illustrated in Fig. 6.31. This is
known as an output error model. This model is suitable when the dominating noise term enters
late in the process, such as a disturbance in the measuring transducer for example.

Again drawback of this model structure compared to the ARX model is that the optimisation
266 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Plant
Noise
[t, E]
e(k)

C(z)
Discrete FIR Filter1
1

Bz -1 1
[t, U] ysl
u 1 A(z) y

Input Discrete FIR Filter Discrete Filter To Workspace


u(k)

Figure 6.30: A discrete time ARMAX model implemented in S IMULINK. (See also Fig. 6.29.)

noise, e

B
known input, u
F y
output, y

Figure 6.31: A signal flow diagram of an output-error model. Compare this structure with the
similar ARX model in Fig. 6.28.

function is a nonlinear function of the unknown parameters. This is because the estimate y not
directly observable, but is itself a function of the parameters in polynomials B and F . This makes
the solution procedure more delicate and iterative. The S YSTEM I DENTIFICATION toolbox can
identify output error models with the oe command.

6.4.3 G ENERAL INPUT / OUTPUT MODELS

The most general linear input/output model with one noise input, and one known input is de-
picted in Fig. 6.32. The relation is described by

B(q) C(q)
A(q)y(k) = u(k) + e(k) (6.14)
F (q) D(q)

although it is rare that any one particular application will require all the polynomials. For example
[91, p76] point out that only the process dynamics B and F are usually identified, and D is a
design parameter. Estimating the noise C polynomial is difficult in practice because the noise
sequence is unknown and so must be approximated by the residuals.

Following the block diagram in Fig. 6.32 or Eqn. 6.14, we have the following common reduced
special cases:

1. ARX: Autoregressive with eXogenous input; C = D = F = 1


2. AR: Autoregressive with no external input; C = D = 1, and B = 0
6.4. POPULAR DISCRETE-TIME MODELS STRUCTURES 267

noise, e C
D

B 1
known input, u
F A
output, y

Figure 6.32: A general input/output model structure for linear dynamic systems involving five
polynomials.

3. FIR (Finite Input Response): C = D = F = A = 1

4. ARMAX: Autoregressive, moving average with eXogenous input; D = F = 1

5. OE (Output error): C = D = A = 1

6. BJ (Box-Jenkins): A = 1

In all these above models, our aim is to estimate the model parameters which are the coefficients of
the various polynomials given the observable input/output experimental plant data. As always,
we want the smallest, most efficient model that we can get away with. A good summary of linear
black box models is given in [188, section 3.1].

The next section will describe how we can find suitable parameters to these models. Whether we
have the correct model structure is another story, and that tricky and delicate point is described
in section 6.6. Nonlinear models are considered in section 6.4.4.

6.4.4 C OMMON NONLINEAR MODEL STRUCTURES

Up until now in this section, we have only discussed model structures that are linear (or pseudo
linear) in their parameters. However in many cases nonlinear models might be more appropriate,
perhaps for physical considerations. Nonlinear models, or more precisely, dynamic models that
are nonlinear in the parameters are considerably more complex to identify than linear models
such as ARX, or even pseudo-linear models such as ARMAX as outlined in the unified survey
presented in [188]. For this reason fully general nonlinear models are far less commonly used
for control and identification than linear models. However there are cases where we will need to
account for some process nonlinearities, and one popular way is to only consider static nonlin-
earities that can be separated from the linear dynamics.

Fig. 6.33 shows one way how these block-orientated static nonlinear models can be structured.
These are known as Hammerstein-Wiener models, or block-orientated static nonlinear models.
The only difference between the two structures is the order of the nonlinear block element.
In the Hammerstein model, it precedes the linear dynamics such as in the case for when we
are modelling input nonlinearities such as modelling equal percentage control valves, and for
the Wiener models, the nonlinearity follows the linear dynamics such as in the case where we
are modelling nonlinear thermocouples. Note of course that unlike linear systems, the order of
nonlinear blocks does make a difference since in general nonlinear systems do not commute.

The S YSTEM I DENTIFICATION TOOLBOX can identify a selection of commonly used nonlinear
models in Hammerstein and Wiener structures using the nlhw routine.
268 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Hammerstein Model

vt B(q 1 )
ut yt
A(q 1 )

Static nonlinearity Linear dynamics


vt = N (ut )

Wiener Model

B(q 1 ) vt
ut yt
A(q 1 )

Linear dynamics Static nonlinearity


yt = N (vt )

Figure 6.33: Hammerstein (upper) and Wiener (lower) model structures

6.5 R EGRESSING DISCRETE MODEL PARAMETERS

The previous section listed the various members of the discrete-time models, of which the ARX
and ARMAX are the most useful for control applications. Now we come to the identification
stage. Our aim is given a set of input, uk , and output, yk , data pairs, we wish to regress suitable
coefficients of the various polynomials in Fig. 6.32.

The simple auto-regressive with exogenous input (ARX) model only involves two polynomials,
A(q 1 ) and B(q 1 ),
A(q 1 ) y(k) = B(q 1 ) u(k). (6.15)
This can be arranged to give the current output, y(k), in terms of the past outputs and current and
past inputs as,

y(k) = 1 A(q 1 ) y(k) + B(q 1 ) u(k) (6.16)

or written out explicitly as

y(k) = a1 y(k 1) an y(k n) + b0 u(k) + b1 u(k 1) + + bm u(k m) (6.17)


| {z } | {z }
past outputs current & past inputs

Our job is to estimate reasonable values for the unknown n + m + 1 parameters a1 , . . . , an and
b0 , . . . , bm such that our model is a reasonable approximation to data collected from the true plant.
Collecting all the unknown parameters into a column vector, , defined as
h iT
def
= .. (6.18)
a1 a2 an . b0 b1 bm

and the immediate past data vector as


h iT
def .
= y(k 1) y(k 2) y(k n) .. u(k) u(k 1) u(k m) (6.19)
6.5. REGRESSING DISCRETE MODEL PARAMETERS 269

then we could write Eqn. 6.17 compactly as


y(k) = T (6.20)
The data vector is also known as the regression vector since we are using it to regress the values
of the parameter vector . Note that we place the negative sign associated with the outputs in
Eqn. 6.19 although there is nothing to stop us placing the negative sign in the corresponding
places associated with the ai coefficients in Eqn. 6.18.

While Eqn. 6.20 contains (n + m + 1) unknown parameters we have only one equation. Clearly
we need more information in the form of additional equations to reduce the degree of freedom
which in turn means collecting further input/output data pairs.

Suppose we collect a total of N data pairs. The first n are used to make the first prediction, and
the next N n are used to estimate the parameters. Stacking all the N n equations, of which
Eqn. 6.17 is the first, in a matrix form gives

yn1 yn2 y0 un un1 unm
yn
yn yn1 y1
yn+1 un+1 un unm+1


.. =
. .. .. .. .. .. .. .. ..

. . . . . . . .
yN
yN 1 yN 2 yN n1 uN uN 1 uN m
(6.21)
or in a compact form
yN = XN (6.22)
where XN is known as the data matrix.

Now the optimisation problem is, as always, to choose the parameter vector such that the model
M is an optimal estimate to the true system S. We will choose to minimise the sum of the squared
errors, so our cost function, J , becomes
N
X 2
J = yk Tk (6.23)
k=n

which we wish to minimise. In practice, we are rarely interested in the numerical value of J but
in the values of the elements in the parameter vector, , that minimises it,
( N )

X 2
T
= argmin yk k (6.24)
k=n

Eqn. 6.24 reads as the optimum parameter vector, , is given by the argument that minimises
the sum of squared errors. Eqn. 6.24 is a standard linear regression problem which has the same
solution as in Eqn. 3.42, namely
 1
= X N XN XN yN (6.25)

For the parameters to be unique, the inverse of the square matrix X N XN must exist. In order
to ensure this, the rectangular N (n + m + 1) data matrix, XN , must at least have more rows
than columns, or in other words it should be taller than it is wide. (Admittedly this criteria does
not appear to be the case visually at first glance when looking at Eqn. 6.21.) Secondly the square
matrix XN XN , which has dimensions of the number of unknowns, must have full rank. We can
ensure this second requirement by constructing an appropriate input sequence to the unknown
plant which is part of our experimental design.

Finally we should take care in numerically computing the unknown vector by using the improved
strategies given in section 3.3.2 or at least using Matlabs backslash as opposed to attempting to
follow Eqn. 6.25 verbatim.
270 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Figure 6.34: I have always wondered what the regression gun does.

E XAMPLE OF OFFLINE DISCRETE MODEL IDENTIFICATION .

Suppose we have collected some input/output data from a plant:

100

50
k
y

time, k input output 0


0 1 1 50
1 4 10 5
2 3 29
k

0
3 2 64
u

5
0 1 2 3 4
time, k

to which we would like to fit an ARX model of the form A(q)yk = B(q)uk . We propose a model
structure of
A(q) = 1 + a1 q 1 , and B(q) = b0
def  T
which has two unknown parameters, = a1 b0 which we are to estimate such that our
predictions adequately match the measured output above. We can write our model as a difference
equation at time k,
yk = a1 yk1 + b0 uk
which if we write out in full, at time k = 3 using the 4 previously collected input/output data
pairs, gives

y1 y0 u1  
y2 = y1 a1
u2
b0
y3 y2 u3

or by inserting the numerical values from the table above, we get



10 1 4  
29 = 10 3 a1
b0
64 29 2

The solution for the parameter vector is over-constrained as we have 3 equations and only 2
unknowns, so we will use Eqn. 6.25 to solve for the optimum .
 1
N = X N XN XN yN
6.5. REGRESSING DISCRETE MODEL PARAMETERS 271

and substituting N = 3 gives


1
  1 4   10
1 10 29 1 10 29
3 = 10 3 29
4 3 2 4 3 2
29 2 64
  
1 29 84 2136
=
20262 84 942 255
 
2
=
3
This means the parameters are a1 = 2 and b0 = 3 giving a model
3
M() =
1 + 2q 1
This then should reproduce the measured output data, and if we are lucky, be able to predict
future outputs too. It is always a good idea to compare our predictions with what was actually
measured.

time input output prediction


0 1 1
1 4 10 10
2 3 29 29
3 2 64 64

In this over-simplified simulated case our predictions are perfect, but we would not expect such
a fortuitous outcome in reality. This problem is continued on page 289 by demonstrating how by
using a recursive technique we can efficiently process new incoming data pairs.

The following sections generalise this model regression strategy. Section 6.5.1 shows how we can
automate the above parameter identification procedure for a general ARX model. Section 6.5.2
illustrates that if we neglect the noise model, then the estimated parameters will exhibit a bias.
This is the motivation to estimate the noise model as well and we will use the System Identifi-
cation toolbox routines described in sections 6.5.3 and 6.5.4 to estimate the more complex model
families given in Fig. 6.32.

6.5.1 S IMPLE OFFLINE ARX MODEL IDENTIFICATION ROUTINES

The following routines automate the offline identification procedure given in the previous section.
However it should be noted that if you are serious about system identification, then a better option
is to use the more robust routines from the the S YSTEM I DENTIFICATION T OOLBOX. These routines
will be described in section 6.5.3.

In Listing 6.8 below, we will use the true ARX process with polynomials
A(q) = 1 1.9q 1 + 1.5q 2 0.5q 3 , and B(q) = 1 + 0.2q 1 (6.26)
to generate 50 input/output data pairs.

Listing 6.8: Generate some input/output data for model identification


1+0.2q 1
G = tf([1,0.2,0,0],[1 -1.9 1.5 -0.5],1); % Unknown plant 11.9q 1 +1.5q 2 0.5q 3
G.variable = 'z-1';
3 u = randn(50,1); % generate random input, uk
y = lsim(G,u); % Generate output with no noise, yk = Guk
272 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Now that we have some trial input/output data, we can write some simple routines to calculate
using least-squares the coefficients of the underlying model by solving Eqn. 6.21. Once we have
performed the regression, we can in this simulated example, compare our estimated coefficients
with the true coefficients given in Eqn. 6.26.

Listing 6.9: Estimate an ARX model from an input/output data series using least-squares
1 na = 3; nb = 2; % # of unknown parameters in the denominator, na , & numerator, nb .

N = length(y); % length of data series


nmax = max(na,nb)+1; % max order

6 X = y(nmax-1:N-1); % Construct Eqn. 6.21


for i=2:na
X = [X, y(nmax-i:N-i)];
end
for i=0:nb-1
11 X = [X, u(nmax-i:N-i)];
end
y_lhs = y(nmax:N); % left hand side of y = X
theta = X\y_lhs % = X+ y

16 G_est = tf(theta(na+1:end)',[1 -theta(1:na)'],-1,'Variable','z-1')

If you run the estimation procedure in Listing 6.9, your estimated model, G_est, should be iden-
tical to the model used to generate the data, G, in Listing 6.8. This is because in this simulation
example we have no model/plant mismatch, no noise and we use a rich, persistently exciting
identification signal.

As an aside, it is possible to construct the Vandemonde matrices (data matrix X, in Eqn. 6.22)
using Toeplitz (or the closely related Hankel matrices) as shown in Listing 6.10.

Listing 6.10: An alternative way to construct the data matrix for ARX estimation using Toeplitz
matrices. See also Listing 6.9.
na = length(A)-1; nb = length(B); % # of parameters to be estimated
nmax = max(na,nb);

4 Y = toeplitz(y(nmax:end-1),y(nmax:-1:nmax-na+1));
U = toeplitz(u(nmax+1:end),u(nmax+1:-1:nmax-nb+2));
X = [Y,U]; % Form data matrix, X.

theta = X\y(nmax+1:end) % = X+ y

Note that in this implementation, na is both the order of the A(q 1 ) polynomial and the number
of unknown coefficients to be estimated given that A is monic, while nb is the order of B plus one.

6.5.2 B IAS IN THE PARAMETER ESTIMATES WITH NOISE MODELS

The strategy to identify the A and B polynomials in ARX systems has the advantage that the
regression problem is linear, and the estimated parameters will be consistent, that is, they can
be shown to converge to the true parameters as the number of samples increases provided the
disturbing noise is white. If however, the disturbances are non-white as in the case where we have
a non-trivial C(q) polynomial and significant noise, then the estimated A(q) and B(q) parameters
will exhibit a bias.
6.5. REGRESSING DISCRETE MODEL PARAMETERS 273

To illustrate this problem, suppose we try to estimate the A and B polynomial coefficients in the
the ARMAX process
(1 + 0.4q 1 0.5q 2 )yk = q 1 (1.2 q 1 )uk + (1 + 0.7q 1 + 0.3q 2 ) ek (6.27)
| {z }
noise filter

where we have a non-trivial colouring C polynomial disturbing the process.

If we use 200 data points (which is far more than we should strictly need), and undertake a stan-
dard least squares identification then instead of obtaining a1 something close to 0.4, we actually
get 0.252 which is quite in error. If we take 300 points, again far more than we should reasonably
expect to get an accurate answer, we get a1 0.222 which is hardly any better. Table 6.2 shows
the parameter values we obtain after using 200, 300 and half a million points. Fig. 6.35 shows this
failure to converge rapidly as a function of the number of data points used.

Table 6.2: Bias in the estimation of ARX models with noise


Exact 200 pts 300 pts 500,000 pts
a1 0.4 0.252 0.222 0.402
a2 0.5 0.635 0.662 0.498
b0 1.2 1.19 1.16 1.1994
b1 1 1.26 1.32 0.997

The results shown in Table 6.2 illustrate the failure for the ARX estimation routine to properly
identify the A(q) and B(q) parameters. Note that the estimate of a1 after 300 iterations is around
0.2 which is some distance for the true result of 0.4. Similar errors are evident for a2 and b1 and
it does not look like they will converge quickly even if we collect more data. This inconsistency
of the estimated parameters is a direct result of the C(q) polynomial which filters the white noise,
ek . As the disturbing signal is no longer white, the ARX estimation routine is now no longer
consistent.

To obtain the correct parameters without the bias, we have no option other than to estimate the
C(q) polynomial as well. This ARMAX estimation strategy is unfortunately a nonlinear estima-
tion problem and appropriate identification strategies are available in the S YSTEM I DENTIFICA -
TION T OOLBOX described next in section 6.5.3.

6.5.3 U SING THE S YSTEM I DENTIFICATION TOOLBOX

As an alternative to using the rather simple and somewhat home grown identification routines
developed in section 6.5.1 above, we could use the collection in the S YSTEM I DENTIFICATION
T OOLBOX. This comprehensive set of routines for dynamic model identification that closely fol-
low [126] and is far more reliable and robust than the purely illustrative routines presented so
far.

The simplest model considered by the S YS ID toolbox is the auto-regressive with exogeneous in-
put, or ARX model, described in Eqn. 6.10. However for computational reasons, the toolbox
separates the deadtime from the B polynomial so that the model is now written
A(q 1 )y = q d B(q 1 )u + e (6.28)
where the integer d is the assumed number of delays. The arx routine will, given input/output
data, try to establish reasonable values for the coefficients of the polynomials A and B using
least-squares functionally similar to the procedure given in Listing 6.9.

Of course in practical identification cases we do not know the structure of the model (i.e. the
order of B, A and the actual sample delay), so we must guess a trial structure. The arx command
274 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

input
10 output

Input/output
0

-10

1.5
b0
1
Paramaters,

0.5 a1

-0.5 a2

Figure 6.35: In this example, the ARX


-1 b1
estimation routine fails to converge
to the true A(q) and B(q) parameters.
This bias is due to the presence of the -1.5
C(q) polynomial which colours the 0 50 100 150 200 250 300
noise. Sample No. [k]

returns both the estimated parameters and associated uncertainties in an object which we will
call testmodel and we can view with present. Once we fitted out model, we can compare
the model predictions with the actual output data using the compare command. The script in
Listing 6.11 demonstrates the load, fitting and comparing, but in this case we deliberately use a
structurally deficient model.

Listing 6.11: Offline system identification using arx from the System Identification Toolbox
>>plantio = iddata(y,u,1); % Use the data generated from Listing 6.8.
2

>>testmodel = arx(plantio,[2 2 0]) % estimate model coefficients


Discrete-time IDPOLY model: A(q)y(t) = B(q)u(t) + e(t)
A(q) = 1 - 1.368 q-1 + 0.5856 q-2

7 B(q) = 1.108 + 0.8209 q-1

Estimated using ARX from data set plantio


Loss function 0.182791 and FPE 0.214581
Sampling interval: 1
12

>>compare(plantio,testmodel) % See Fig. 6.36.

Note however that with no structural mismatch, the arx routine in Listing 6.12 should manage
6.5. REGRESSING DISCRETE MODEL PARAMETERS 275

Measured Output and Simulated Model Output


10
Measured Output
8 testmodel Fit: 71.34%

2
y1

4
Figure 6.36: Offline system
6
identification using the System
8
Identification Toolbox and a
10
0 10 20 30 40 50 structurally deficient model.

to find the exact values for the model coefficients,

Listing 6.12: Offline system identification with no model/plant mismatch


>> testmodel = arx(plantio,[3 2 0]) % Data from Listing 6.11 with correct model structure.
2 Discrete-time IDPOLY model: A(q)y(t) = B(q)u(t) + e(t)
A(q) = 1 - 1.9 q-1 + 1.5 q-2 - 0.5 q-3

B(q) = 1 + 0.2 q-1

7 Estimated using ARX from data set plantio


Loss function 1.99961e-029 and FPE 2.44396e-029
Sampling interval: 1

which it does.

Note that in this case the identification routine perfectly identified our plant and that the loss
function given in the result summary is practically zero ( 1030 ). This indicates an extraordinary
goodness of fit. This is not unexpected, since we have used a perfect linear process, which we
knew perfectly in structure although naturally we could not expect this in practice.

The arx command essentially solves a linear regression problem using the special M ATLAB com-
mand \ (backslash), as shown in the simple version given in Listing 6.9. Look at the source of arx
for further details.

P URELY AUTO - REGRESSIVE MODELS

As the name suggests, purely auto-regressive models have no measured input,

A(q 1 ) yk = ek (6.29)

and are used when we have a single output time series of data which we suspect is simply dis-
turbed by random noise. Fig. 6.37 shows the block diagram of such an AR model.

In Listing 6.13 we attempt to fit a 3rd order AR model and we get reasonable results with the
coefficients within 1% of the true values. (We should of course compare the pole-zero plots of
the model and plant because the coefficients of the polynomials are typically numerically ill-
conditioned.) Note however that we used 1000 data points for this 3 parameter model, far more
than we would need for an ARX model.
276 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

noise, e

1
output, y
A

Figure 6.37: An AR only model has no external input and whose output is driven by an unknown
noise term. Compare this structure with the ARX model given in Fig. 6.28.

Listing 6.13: Demonstrate the fitting of an AR model.


1 >> N = 1e3; e = randn(N,1); % Take 1000 data points
>> B=1; A = [1 0.4 0.1 0.2]; % True AR plant is: A(q 1 ) = 1 0.4q 1 + 0.1q 2 + 0.2q 3
>> y = filter(B,A,e);

>> Gm = ar(iddata(y),3); % Fit AR model with no model/plant mismatch


6 Discrete-time IDPOLY model: A(q)y(t) = e(t)
A(q) = 1 + 0.419 q-1 + 0.1008 q-2 + 0.1959 q-3

Estimated using AR ('fb'/'now') from data set Dat


Loss function 1.00981 and FPE 1.01587
11 Sampling interval: 1

6.5.4 E STIMATING NOISE MODELS

Estimating noise models is complicated since some of the regressors are now functions themselves
of the unknown parameters. This means the optimisation problem is now non-linear compared
to the relatively straight forward linear arx case. However, as illustrated in section 6.5.2, we need
to estimate the noise models to eliminate the bias in the parameter estimation.

In Listing 6.14 we generate some input/output data from an output-error process,

B
y(k) = u(k d) + e(k)
F
1 + 0.5q 1
= u(k 1) + e(k)
1 0.5q 1 + 0.2q 2

whose parameters are, of course, in reality unknown. The easiest way to simulate an output-error
process is to use the idmodel/sim command from the System ID toolbox.

Listing 6.14: Create an input/output sequence from an output-error plant.


>> Ts = 1.0; % Sample time T
>> NoiseVariance = 0.1;

4 >> F = [1 -0.5 0.2]; % F (q 1 ) = 1 0.5q 1 + 0.2q 2


>> B = [0 1 0.5]; % Note one unit of delay: B(q 1 ) = q 1 + 0.5q 2
>> A = 1;C=1;D=1; % OE model: y = (B/F )u + e

>> G = idpoly(A,B,C,D,F,NoiseVariance,Ts)
9 Discrete-time IDPOLY model: y(t) = [B(q)/F(q)]u(t) + e(t)
B(q) = q-1 + 0.5 q-2
6.5. REGRESSING DISCRETE MODEL PARAMETERS 277

F(q) = 1 - 0.5 q-1 + 0.2 q-2

14 This model was not estimated from data.


Sampling interval: 1

>> N = 10000; % # of samples


>> U = randn(N,1); E = 0.1*randn(N,1); % Input & noise
19 >> Y = sim(G,[U E]);

Using the data generated in Listing 6.14, we can attempt to fit both an output-error and an arx
model. Listing 6.15 shows that in fact the known incorrect arx model is almost as good as the
structurally correct output-error model.

Listing 6.15: Parameter identification of an output error process using oe and arx.
1 >> Gest_oe = oe(Z,'nb',2,'nf',2,'nk',1,'trace','on');
>> present(Gest_oe);

Discrete-time IDPOLY model: y(t) = [B(q)/F(q)]u(t) + e(t)


B(q) = 1.002 (+-0.0009309) q-1 + 0.5009 (+-0.002302) q-2
6

F(q) = 1 - 0.4987 (+-0.001718) q-1 + 0.1993 (+-0.00107) q-2

Estimated using OE from data set Z


Loss function 0.00889772 and FPE 0.00890484
11 Sampling interval: 1
Created: 02-Jul-2007 11:35:15
Last modified: 02-Jul-2007 11:35:16

>> Gest_arx = arx(Z,'na',2,'nb',2,'nk',1);


16 >> present(Gest_arx);
Discrete-time IDPOLY model: A(q)y(t) = B(q)u(t) + e(t)
A(q) = 1 - 0.4816 (+-0.001879) q-1 + 0.1881 (+-0.001366) q-2

B(q) = 1.001 (+-0.001061) q-1 + 0.518 (+-0.002154) q-2


21

Estimated using ARX from data set Z


Loss function 0.0111952 and FPE 0.0112041
Sampling interval: 1
Created: 02-Jul-2007 11:35:16
26 Last modified: 02-Jul-2007 11:35:16

The System Identification toolbox has optimal parameters for the fitting of the nonlinear models.
One of the more useful options is to be able to fix certain parameters to known values. This can
be achieved using the FixParameter option.

6.5.5 F ITTING PARAMETERS TO STATE SPACE MODELS

So far we have concentrated on finding parameters for transfer function models, which means
we are searching for the coefficients of just two polynomials. However in the multivariable case
we need to identify state-space models which means estimating the elements in the , and C
matrices. If we are able to reliably measure the states, then we can use the same least-squares ap-
proach (using the pseudo inverse) that we used in the transfer function identification. If however,
and more likely, we cannot directly measure the states, x, but only the output variables, y then
we need to use a more complex subspace approach described on page 279.
278 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

The state-space identification problem starts with a multivariable discrete model

xk+1 = xk + uk , x n , u m (6.30)

where we want to establish elements in the model matrices , given the input/state sequences
uk and xk . To ensure we have enough degrees of freedom, we would expect to need to collect at
least (n + m) m data pairs, although typically we would collect many more if we could.

Transposing Eqn. 6.30 gives xTk+1 = xTk T + uTk T , or expanded out



1,1 2,1 n,1
  1,2 2,2 n,2
xTk+1 = x1 x2 xn k .

.. ..
.. . .
1,n 2,n n,n
| {z }
nn

1,1 2,1 n,1
  . .. ..
+ u1 u2 um k
.. . .
1,m 2,n m,n
| {z }
mn

we can see that it is simple to rearrange this equation into



1,1 2,1 n,1


1,2 2,2 n,2

. .. ..
. .
. .

xTk+1 = [ x1 x2 xn u1 u2 u m ]k
1,n 2,n n,n
(6.31)

1,1 2,1 n,1


.
..

1,m 2,n m,n
| {z }
matrix (n+m)n
where we have stacked the unknown model matrices and together into one large parameter
matrix . If we can estimate this matrix, we have achieved our model fitting.

Currently Eqn. 6.31 comprises of n equations, but far more, (n + m)n, unknowns; the elements
in the and matrices. To address this degree of freedom problem, we need to collect more
input/state data. If we collect say N more input/states and stack them underneath Eqn. 6.31, we
get
 
xTk+1  x1 x2 xn | u1 um k
xTk+2 x1 x2 xn | u1 um
k+1
.. = .. (6.32)
.
 . 

xTk+N +1 x1 x2 xn | u1 um k+N
| {z } | {z }
N n N (n+m)

or Y = X. Inverting this, using the pseudo inverse of the data matrix X, gives the least-squares
parameters  T 

= = X+ Y (6.33)
T
and hence the model coefficient and matrices as desired.
6.5. REGRESSING DISCRETE MODEL PARAMETERS 279

The computation of the model matrices and is a linear least-squares regression which is easy
to program, and robust to execute. However we are still constrained to the normal restrictions
on the invertibility and conditioning of the data matrix X apply in order to obtain meaningful
solutions. The largest drawback of this method is that we must measure states x, as opposed to
only outputs. (If we only have outputs then we need to use the more sophisticated technique
described on 279). If we had measured the outputs in addition to the states, it is easy to calculate
the C matrix subsequently using another least-squares regression.

In M ATLAB we could use the fragment below to compute and given a sequence of x(k), and
u(k).

[y,t,X] = lsim(G,U,[],x0); % Perform expriment to generate the I/O data

Xdata = [X,U]; Xdata(end,:) = []; % data matrix


4 y = X; y(1,:) = []; % delete first row

theta = Xdata\y; % solve for parameters = X+ Y


Phiest = theta(1:n,:)'; Delest = theta(n+1:end,:)'; % Reconstruct & .

If any of the parameters are known a priori, they can be removed from the parameter matrix,
and the constants substituted. If any dependencies between the parameters are known (such as
a = 3b), then this may be incorporated as a constraint, and a nonlinear optimiser used to search
for the resulting best-fit parameter vector. The S YSTEM I DENTIFICATION T OOLBOX incorporates
both these extensions.

Estimating the confidence limits for parameters in dynamic systems uses the same procedure
as outlined in 3.3.4. The difficult calculation is to extract the data matrix, X, (as defined by
Eqn. 3.61) accurately. [25, pp226228] describes an efficient method which develops and subse-
quently integrates sensitivity equations, which is in general, the preferred technique to that using
finite differences.

I DENTIFICATION OF STATE - SPACE MODELS WITHOUT STATE MEASUREMENTS

The previous method required state measurements, but it would be advantageous to be able still
to estimate the model matrices, but only be required to measure input, u, and output y data. This
is addressed in a method known as a subspace identification described in [72, p536] and in more
detail by the developers in [157]. The state-space subspace algorithm for identification, known as
n4sid, is available in the System Identification toolbox.

S UMMARY OF IDENTIFICATION ALTERNATIVES IN STATE - SPACE

Given the underlying system

xk+1 = xk + uk
yk = Cxk

then Table 6.3 summarises the identification possibilities.


280 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Table 6.3: Identification alternatives in state-space

We measure Wish to calculate Algorithm


page #
uk , yk , , , C xk Kalman filter 443
uk , yk , xk , , C Normal linear regression 277
uk , y k n, , , C, xk Subspace identification 279

6.6 M ODEL STRUCTURE DETERMINATION AND VALIDATION

The basic linear ARX model using in Eqn. 6.28 has three structural parameters: the number of
poles, na , the number of zeros nb and the number of samples of delay, d. ARMAX models have
four structural parameters. Before regressing suitable parameters given a tentative model struc-
ture, we need some method to establish what a suitable model structure is. Typically this involves
some iteration in the model fitting procedure.

If we inadvertently choose too high an order for the model, we are in danger of overfitting the
data, which will probably mean that our model will perform poorly on new data. Not only that,
but the estimation procedure itself could well be ill-conditioned given that we are searching for a
large number of fitted parameters. Furthermore, for reasons of efficiency and elegance, we would
like the simplest model, with the smallest number of parameters that still adequately predicts
the output. This is known as the principle of parsimony which aims to reward simpler, rather
than complex models. This is the rational for dividing the collected experimental data into two
sets, one estimation set used for model fitting, and one validation set used to distinguish between
models structures.

One way to penalise overly complex models is to use the Akaike Information Criteria (AIC)
which for normally distributed errors is
2p
AIC = + ln (V ) (6.34)
N
where p is the number of parameters, N is the number of observations, and V is the loss function
or sum of squared errors. If we increase the number of parameters to be estimated, the sum of
squared errors decreases, but this is partially offset by the 2p term which increases. Hence by
using the AIC as a performance criteria, we try to find a balance between models that fit the
data, but are parsimonious in parameters. The following example illustrates using the AIC for
determining the appropriate order of an interpolating polynomial to some (x, y) data. Since this
is a linear optimisation problem, it is mathematically equivalent to the fitting of an ARX model.

D ETERMINING THE ORDER OF AN INTERPOLATING POLYNOMIAL

Suppose we have collected some experimental data as shown in Fig. 6.38 and we wish to fit a
polynomial through the data cloud. The key question is: what is an appropriate polynomial
order to use? Fig. 6.38 shows three alternatives, a quadratic with 3 parameters, a cubic (with 4)
and an extreme 10th order polynomial. Which polynomial is best?

Now clearly our fit is going to improve as we increase the interpolation order, but that is probably
not what we want since we end up fitting overly complex models that tend to essentially just fit
the noise. Instead we should by plot the AIC from Eqn. 6.34 as a function of order as shown in
Fig. 6.39.

When using the AIC criteria we are looking for a distinct minimum to tell us the appropriate
polynomial order. However looking at Fig. 6.39 we can see that the AIC improves rapidly to
6.6. MODEL STRUCTURE DETERMINATION AND VALIDATION 281

10
y

15

20 data
n=3 Figure 6.38: Fitting various polynomi-
n=4
25
n=11
als to some noisy experimental data.
30 Our aim is to establish an appropriate
2 1 0 1 2
x order.

6.5

5.5

5
AIC

4.5

3.5

3 Figure 6.39: Using the AIC to establish


0 5 10 15 20
# of parameters model structure based on fitted data.

n = 5 meaning that a quartic polynomial is appropriate, but then the curve flattens out. There
is a distinct knee in the curve, but the AIC does not convincingly increase for the high order
polynomials.

The oversight that we have made in Fig. 6.39 is that we have used the same data for the validation
as that used for the fitting. In situations where we have scant experimental data, we may have
to resort to this, but it is far preferable to collect some separate data that we put aside and only
use for validation. Fig. 6.40 shows the improvement this separation of fitted and validation data
makes. Now we can see a distinct minimum in the AIC plot which clearly shows the penalty of
overfitting.

As it turns out, the experimental data using in this example was synthesised from a simple
quartic polynomial with some added noise. Consequently in this artificial case we know the true
number of parameters is 5. Note however that the AIC criteria suggests that optimum number of
parameters is 4, which in my opinion is acceptably close.

The S YSTEM I DENTIFICATION toolbox computes the AIC with the aic command or when using
the automated structure selection, selstruc. Alternatively for a given model estimated say with
armax, we can extract the following parameters

p = nparams(Model,'free') % # of free parameters


N = Model.es.DataLength % # of data pairs, N
3 V = Model.es.LossFcn % Sum of squared errors

and then compute the information criteria with

AIC = 2*p/N + log(V) % Akaike Information Criteria from Eqn. 6.34

Given a collection of potential models, we should pick the one that minimises the AIC.
282 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

20

20

40

y
60

80 Data for fitting


Data for validation
100
3 2 1 0 1 2
x

30

25

20

AIC
Figure 6.40: Using the AIC to estab-
15
lish model structure based on validation
data. Compare the AIC plot in this case 10
when using the validation data with
that where we used only the fitted data 5
0 5 10 15 20
in Fig. 6.39. # of parameters

6.6.1 E STIMATING MODEL ORDER AND DEADTIME

If we restrict our attention to linear models, the most crucial decision to make is that of model
order. We can estimate the model order from a spectral analysis, perhaps by looking at the high-
frequency roll-off of the experimentally derived Bode diagram.

An alternative strategy is to monitor the rank of the matrix XT X. If we over-estimate the order,
then the matrix XT X will become singular. Of course with additive noise, the matrix may not
be exactly singular, but it will certainly become illconditioned. Further details are given in [128,
pp496497].

I DENTIFICATION OF DEADTIME

The number of delay samples, d in Eqn. 6.28 is a key structural parameter to be established prior
to any model fitting. In some cases, an approximate value of the deadtime is known from the
physics of the system, but this is not always a reliable indicator. For example, the blackbox from
Fig. 1.5 should not possess any deadtime since it is composed only of passive resistive/capacitor
network. However the long time constant, the high system order, or the enforced one-sample
delay in the A/D sampler may make it appear if there is some deadtime.

If we suspect some deadtime, then we have the following options:

1. Do a step test and look for a period of inactivity.

2. Use the S YSTEM I DENTIFICATION T OOLBOX (SITB) and optimise the fit for deadtime. This
will probably require a trial & error search. The delayest routine may help us with this
identification.

3. Use some statistical tests (such as correlation) to estimate the deadtime.


6.6. MODEL STRUCTURE DETERMINATION AND VALIDATION 283

Note: Obviously the deadtime will be an integer number of the sampling time. So if you want
an accurate deadtime, you will need a small sample time T . However overly fast sampling will
cause problems in the numerical coefficients of the discrete model.

Method #1 Apply a step response & look for the first sign of life.

1. First we do an open loop step test using a sample time T = 0.5 seconds.

Blackbox step response


0.5 zoomed region input
0.06 output
0.04
0.4
0.02

0
0.3
input/output

0.02
19 20 21 22 23
zoomed region
0.2
0.28
0.26
0.1 0.24
0.22
0.2
0 0.18
40 41 42 43 44

15 20 25 30 35 40 45 50
time (s)

Figure 6.41: Identification of deadtime from the step response of the blackbox. Note that the
deadtime is approximately 2T 1 second.

2. We should repeat the experiment at a different sampling rate to verify the deadtime. Fig. 6.42
compares both T = 1 and T = 0.1 seconds.

0.2

0.15
i/o

0.1

0.05

0
20 21 22 23 24 25
Figure 6.42: Deadtime estimation at both T =
time [s] 1 and T = 0.1s.

Method #2: Use trial & error with the S YSTEM IDENTIFICATION T OOLBOX. We can collect some
input/output data and identify a model M() with different deadtimes, namely d = 0 to 3.
Clearly the middle case in Fig. 6.43 where d = 2 has the least error.

This shows, at least for the model M() with na = nb = 2, then the optimum number of delays is
d = 2.

M ODELLING THE BLACKBOX

A model of the experimental blackbox is useful to test control schemes before you get to the lab.
Based on the resistor/capcitor internals of the black-box, we expect an overdamped plant with
time constants in the range of 2 to 5 seconds.
284 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

0.5

Deadtime=0
0

0.5

0.5

Deadtime=1
0

0.5

0.5

Deadtime=2 0

0.5

actual
0.5
predicted
Deadtime=3

0
Figure 6.43: Model prediction and ac-
tual using a variety of tentative dead- 0.5
20 40 60 80 100 120 140 160
times, d = 0, 1, 2, 3. time (s)

Identification tests on my blackbox gave the following continuous transfer function model
0.002129s2 + 0.02813s + 0.5788 0.5s
Gbb (s) = e
s2 + 2.25s + 0.5362

or in factored form

0.002129(s2 + 13.21s + 271.9) 0.5s


= e
(s + 1.9791)(s + 0.2709)
Fig. 6.44 illustrates the S IMULINK experimental setup and compares the model predictions to the
actual measured data.

6.6.2 R OBUST MODEL FITTING

So far we have chosen to fit models by selecting parameters that minimise the sum of squared
residuals. This is known as the principle of the least squares prediction error and is equivalent
to the maximum likelihood method when the uncertainty distribution is Gaussian. This strategy
has some very nice properties first identified by Gauss in 1809 such as the fact that the estimates
are consistent, that is they eventually converge to the true parameters, the estimate is efficient in
that no other unbiased estimator has a smaller variance, and the numerical strategies are simple
and robust as described in [11].

However by simply squaring the error, we run into problems in the presence of outliers in the
data. In these cases which are clearly non-gaussian, but actually quite common, we must modify
the form of the loss function. One strategy due to [90] is to use a quadratic form when the errors
are small, but a linear form for large errors, another suggested in [11] is to use something like

2
f () = (6.35)
1 + a2
and there are many others. Numerical implementations of these robust fitting routines are given
in [206].
6.7. ONLINE MODEL IDENTIFICATION 285

uy
To Workspace2
0
In1 Out1
initialise
Manual Switch Saturation Scope1
bb_pcl711
Mux1

BandLimited
White Noise

0.002129s2 +0.02813s+0.5788
s2+2.25s+0.5362
BB Delay
Blackbox dyn

(a) Blackbox experimental setup in S IMULINK

0.6

0.4

0.2

0.2

0.4
u
0.6 y
y predict
0.8
25 30 35 40 45 50 55 60 65 70 75
time (seconds)
(b) Blackbox model predictions compared to actual data

Figure 6.44: A dynamic model for the blackbox compared to actual output data.

A different approach which still retains the advantages of linear regression, but helps mitigate
against the poor estimation performance when faced with ill-conditioned problems is to augment
the weighted least-squares cost function with a penalty term. This, as mentioned on page 109,
is known as Tikhonov regulisation or ridge regression. The common S YSTEM ID toolbox estimation
routines can employ regulisation as described in [131].

6.7 O NLINE MODEL IDENTIFICATION

In many practical cases such as adaptive control, or where the system may change from day to
day, we will estimate the parameters of our models online. Other motivations for online identifi-
cation given by [126, p303] include optimal control with model following, using matched filters,
failure prediction etc. Because the identification is online, it must also be reasonably automatic.
The algorithm must pick a suitable model form (number of dead times, order of the difference
equation etc.), guess initial parameters and then calculate residuals and measures of fit.

As time passes, we should expect that owing to the increase in number of data points that we are
continually collecting, we are able to build better and better models. The problem with using the
offline identification scheme as outlined in 6.4 is that the data matrix, X in Eqn. 6.22 will grow
and grow as we add more input/output rows to it. Eventually this matrix will grow too large to
store or manipulate in our controller. There are two obvious solutions to this problem:

1. Use a sliding window where we retain say only the last say 50 input/output data pairs, or
286 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

2. Use a recursive scheme where we achieve the same result but without the waste of throwing
old data away.

The following section develops this second, and more attractive alternative known as recursive
least-squares or RLS.

6.7.1 R ECURSIVE LEAST SQUARES ESTIMATION

As more data is collected, we can update the current estimate of the model parameters. If we
make the update at every sample time, our estimation procedure is now no longer off-line, but
now online. One approach to take advantage of the new data pair would be to add another row
to the X matrix in Eqn. 6.22 as each new data pair is collected. Then a new estimate can be
obtained using Eqn. 6.25 with the now augmented X matrix. This equation would be solved
every sample time.

However this scheme has the rather big disadvantage that the data X matrix grows downwards as
each new data point is collected so consequently one will eventually run out of storage memory,
to say nothing of the growing impracticality of the matrix inversion required. The solution is to
use a recursive formula for the estimation of the new parameter vector at time N + 1 in terms of
the old parameter vector and some sort of error term,
N +1 = N + K (6.36)
The structure of this update scheme for the new parameter scheme is both intuitive, and has the
attraction that we use constant-sized matrices without the need to sacrifice old data.

M EAN AND VARIANCE CALCULATED RECURSIVELY

Digressing somewhat, let us briefly look at the power of a recursive calculation scheme, and the
possible advantages that has for online applications. Consider the problem of calculating the
mean or average, x, of a collection of N samples, xi . The well known formula for the mean is
N
1 X
x = xi
N i=1

Now what happens, (as it often does when I collect and grade student assignments) is that after
I have laboriously summed and averaged the class assignments, I receive a late assignment or
value xN +1 . Do I need to start my calculations all over again, or can I can use a recursive formula
making use of the old mean xN based on N samples to calculate the new mean based on N + 1
samples? Starting with the standard formula for the new mean, we can develop an equivalent
recursive equation
N +1 N
!
1 X 1 X
xN +1 = xi = xN +1 + xi
N + 1 i=1 N +1 i=1
1
= (xN +1 + N xN )
N +1
1
= xN + (x x )
|{z} N + 1 | N +1{z N }
old | {z } error
gain

which is in the form of new estimate = old estimate + gain error. Note that now the computation
of the new mean, assuming we already have the old mean, is much faster and more numerically
sound.
6.7. ONLINE MODEL IDENTIFICATION 287

In a similar manner, the variance can also be calculated in a recursive manner. The variance at
sample number N + 1 based on the mean and variance using samples 1 to N is
 
2 2 1 1 2 2
N +1 = N + (xN +1 xN ) N (6.37)
N +1 N +1
which once again is the linear combination of the previous estimate and a residual. For further
details, see [122, p29]. This recursive way to calculate the variance published in by B.P. Welford
goes back to 1962 and has been shown to have good numerical properties.

A RECURSIVE ALGORITHM FOR LEAST- SQUARES

Suppose we have an estimate at time t = N of the model parameters, N , perhaps calculated


offline, and we subsequently collect a new data pair (uN +1 , yN +1 ). To find the new improved
parameters, N +1 , Eqn. 6.22 is augmented by adding a new row, where the row to be added is
h i
TN +1 = yN yN 1 yN n+1 ... uN +1 uN uN m+1 (6.38)

to the old data matrix XN to obtain the new, now slightly taller, system

yN XN
= N +1 (6.39)
yN +1 TN +1

or yN +1 = XN +1 N +1 . This we can solve in exactly the same way as in Eqn. 6.25,


 1
N +1 = XTN +1 XN +1 XTN +1 yN +1 (6.40)
1
h i XN h i yN
= X ...
XTN
..
. N +1
(6.41)
N N +1
TN +1 yN +1

except that now our problem has grown in the vertical dimension. Note however, the number of
unknowns in the parameter vector remains the same.

However we can simplify the matrix to invert by noting


1
h i XN  1
T
.. = XTN XN + N +1 TN +1 (6.42)
XN . N +1
TN +1

The trick is to realise that we have already computed the inverse of XTN XN previously in Eqn. 6.25
and we would like to exploit this in the calculation of Eqn. 6.42. We can exploit this using the
Matrix inversion lemma which states
1 1
(A + BDC) = A1 A1 B D1 + CA1 B CA1 (6.43)

which is also known as the Woodbury matrix identity. In the special case where B = b and
C = cT are both n 1 vectors, (and hence bcT is an outer product), then Eqn. 6.43 simplifies to
1 A1 bcT A1
A + bcT = A1 (6.44)
1 + cT A1 b
and this special case is sometimes known as the Sherman-Morrison formula. If we compare
Eqn. 6.44 with Eqn. 6.42, then with
def def
A = X X, b = c = N +1
288 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

we can rewrite Eqn. 6.42 as


 1  1
 1  1 XTN XN N +1 TN +1 XTN XN
XTN XN + N +1 TN +1 = XTN XN  1 (6.45)
1 + TN +1 XTN XN N +1

and we can also rewrite Eqn. 6.41 as


 1  1

 1 XTN XN N +1 TN +1 XTN XN
 
N +1 = XTN XN  1 XTN yN + TN +1 yN +1 (6.46)
1 + TN +1 XTN XN

N +1

and recalling that the old parameter value, N , was given by Eqn. 6.40, we start to develop the
new parameter vector in terms of the old,
 1
XTN XN TN +1 N +1  1
N +1 = N  1 N + XTN XN TN +1 yN +1
T T
1 + N +1 XN XN N +1
 1  1
XTN XN TN +1 N +1 XTN XN
 1 TN +1 yN +1 (6.47)
T
1 + N +1 XN XN TN +1
 1
XTN XN TN +1 
= N +  1 yN +1 N +1 N (6.48)
1 + N +1 XTN XN TN +1

If we define P as the covariance matrix,


 1
def
PN = X
N XN (6.49)

then the parameter update equation, Eqn. 6.48 is more concisely written as

N +1 = N + K yN +1 TN +1 N (6.50)

with the gain K in Eqn. 6.50 and new covariance given by Eqn. 6.45 as
PN N +1
K= (6.51)
1 + TN +1 PN N +1
" #
N +1 TN +1 PN h
T
i
PN +1 = PN I = PN I N +1 K (6.52)
1 + TN +1 PN N +1

Note that the form of the parameter update equation, Eqn. 6.50, is such that the new value of N +1
is the old value N with an added correction term which is a recursive form as anticipated in
Eqn. 6.36. Also as expected, the correction term is proportional to the observed error. A summary
of the recursive least-square (RLS) estimation scheme is given in Algorithm 6.2.

Algorithm 6.2 Recursive least-squares estimation


Initialise parameter vector, 0 , to something sensible (say random values), and set co-variance to
a large value, P0 = 106 I, to reflect the initial uncertainty in the trial guesses. Then at each sample
time, do the following:

1. At sample N , collect a new input/output data pair (yN +1 and uN +1 )


6.7. ONLINE MODEL IDENTIFICATION 289

2. Form the new N +1 vector by inserting uN +1 and yN into Eqn. 6.38.


3. Evaluate the new gain matrix K from Eqn. 6.51
4. Update the parameter vector N +1 , Eqn. 6.50
5. Update the covariance matrix PN +1 , Eqn. 6.52 which is required for the next iteration.
6. Wait out the remainder of one sample time T , increment sample counter, N N + 1, then
go back to step 1.

The routine in listing 6.16 implements the basic recursive least-squares (RLS) algorithm 6.2 in
M ATLAB. This routine will be used in subsequent identification applications, although later in
section 6.8.1 we will improve this basic algorithm to incorporate a forgetting factor.

Listing 6.16: A basic recursive least-squares (RLS) update (without forgetting factor)
function [thetaest,P] = rls0(y,phi,thetaest,P)
% Basic Recursive Least-Squares (RLS) parameter update
% Inputs: yk ,k : current output & past input/outputs column vector
4 % Input/Outputs: , P: parameter estimates & covariance matrix

K = P*phi/(1 + phi'*P*phi); % Gain K, Eqn. 6.51.


P = P-(P*phi*phi'*P)/(1 + phi'*P*phi); % Covariance update, P, Eqn. 6.52.
thetaest = thetaest + K*(y - phi'*thetaest);% N+1 = N + KN+1 yN+1 TN+1 N


9 return % end rls.m

The clever part of this recursive estimation algorithm is to note that we needed to efficiently
compute the matrix (A + bcT )1 when we already knew A1 . It is clear that the matrix formed
by the outer product bcT must only have a rank of 1 (since it is comprised of only two vectors),
so the update is known as a rank 1 update or perturbation. That means it is more efficient to
compute the inverse of the perturbation as opposed to computing the inverse from scratch.

A RECURSIVE ESTIMATION EXAMPLE

On page 270 we solved for the parameters of an unknown model using an offline technique by
collecting four data pairs. Now we will try the recursive scheme given that we have just collected
a new fifth input/output data pair at sample time k = 4,

time, k uk yk T P
..
.
1 4 10
2 3 29
3 2 64 2 3 0.0014 0.0041 0.0465
4 6 10 ? ? ? ? ?

and we wish to update our parameter estimates.

Previously we had computed that 3 = [2, 3]T , but now with the new data we can possibly
improve on that estimate using the update

4 = 3 + K4 y4 T4 3

where T4 = [y3 , u4 ] = [64, 6].


290 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Now the residual  


  2
y4 T4 3 = 10 64 6 = 120
3
is clearly non-zero indicating that something has changed, so we should continue to update the
parameter estimates. The covariance matrix at sample time k = 3 has previously been computed
and (repeated here) is
 1    
def 1 29 84 0.0014 0.0041
P3 = X
3 X3 = =
20262 84 942 0.0041 0.0465

so the new gain matrix K4 from Eqn. 6.51 is now


 
P3 4 0.0099
K4 = = ,
1 + T4 P3 4 0.0464

the new parameter estimate, 4 , is corrected to


     
2 0.0099 0.8074
4 = + 120 = ,
3 0.0464 8.5727

and the updated covariance matrix P4 from Eqn. 6.52 is also updated to
 
P3 4 T4 P3 0.0003 0.0013
P4 = P3 = .
1 + T4 P3 4 0.0013 0.0212

Note that P4 is still symmetric, and less immediately apparent, is that it is also still positive
definite which we could verify by computing the eigenvalues (which are incidentally 0.0002 and
0.0213; both positive). When we subsequently obtain another input/output data pair (u5 , y5 ), we
can just continue with this recursive estimation scheme.

We can compare the new estimates 4 , with the values obtained if we used to nonrecursive
method of Eqn. 6.25 on the full data set. At this level, both methods show much the same com-
plexity in terms of computation count and numerical round-off problems, but as N tends to ,
the non-recursive scheme is disadvantaged.

S TARTING THE RECURSIVE SCHEME

In practice there is a small difficulty in starting these recursive style estimation schemes, since
initially we do not know the covariance PN . However we have two alternatives: we could calcu-
late the first parameter vector using a non-recursive scheme, and the switch to a recursive scheme
only later once we know PN , or we could just assume that at the beginning, since we have no
estimates, our expected error will be large, say PN = 104 I. The latter method is more common in
practice, although it takes a few iterations to converge to the correct estimates if they exist.

6.7.2 R ECURSIVE LEAST- SQUARES IN M ATLAB

M ATLAB can automate the estimation procedure so that it is suitable for online applications. To
test this scheme we will try to estimate the same system S from Eqn. 6.26 used previously on
page 271 repeated here

1 + 0.2q 1
S: G(q 1 ) =
11.9q 1 + 1.5q 2 0.5q 3
6.7. ONLINE MODEL IDENTIFICATION 291

where the unknown vector of A and B polynomial coefficients consist of the five values
h iT
.
= 1.9 1.5 0.5 .. 1 0.2

which are to be estimated. Note of course we do not need to estimate the leading 1 in the monic
A polynomial.

The estimation is performed in open loop using a white noise input simply to ensure persistence
excitability. We will select a model structure the same as our true system. Our initial parameter
estimate is a random vector, and the initial covariance matrix is set to the relatively large value of
106 I as shown in Fig. 6.45. Under these conditions, we expect near perfect estimation.

Parameter estimates,

initial covariance, P0 = 106 I


0 = random RLS

1+1.2q1 output, y
white noise, u b

11.9q1 +1.5q2 0.5q3


b

Plant under test

Figure 6.45: Ideal RLS parameter estimation. (See also Fig. 6.47(a).)

Listing 6.17 calls the rls routine from listing 6.16 to do the updating of estimated parameters
and associated statistics using Eqns 6.51 and 6.52.

Listing 6.17: Tests the RLS identification scheme using Listing 6.16.
1 G = tf([1,0.2,0,0], ...
1+0.2q 1
[1 -1.9 1.5 -0.5],1); % unknown plant G = 11.9q 1 +1.5q 2 0.5q 3
G.variable = 'z-1';

N = 12; % # of samples
6 U = randn(N,1); % design random input u(k) = N (0, 1)
Y = lsim(G,U); % Compute output

n = 5; P = 1e6*eye(n); % Identification initialisation, P0 = 106 I


thetaest = randn(n,1); Param = [];
11

for i=4:N % start estimation loop


phi = [Y(i-1:-1:i-3); U(i:-1:i-1)]; % shift X register (column)
[thetaest, P] = rls0(Y(i),phi,thetaest,P); % RLS update of
Param = [Param; thetaest']; % collect data
16 end

If you run the script in Listing 6.17 above you should expect something like Fig. 6.46 giving
the online estimated parameters (lower) which quickly converge to the true values after about 5
iterations. The upper plot in Fig. 6.46 shows the input/output sequence I used for this open loop
identification scheme. Remember for this identification application, we are not trying to control
the process, and that provided the input is sufficiently exciting for the identification, we do not
really care precisely what particular values we are using.

The near perfect estimation in Fig. 6.46 is to be expected, and is due to the fact that we have no
292 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

input
5
output

Input/output
0

5
2

1.5 a2

1 b
0

0.5
Paramaters, b
1
0

0.5 a
3

Figure 6.46: Recursive least squares param- 1


eter estimation.
Upper: The input/output trend for an un- 1.5
known process. a
1
2
Lower: The online estimated parameters
quickly converge to the true parameters.
2.5
Note that no estimation is performed until 0 2 4 6 8 10 12
after we have collected 4 data pairs. Sample No.

systematic error or random noise and we have sufficient excitation in the input to stimulate a re-
sponse in the output. This means that after 4 iterations (we have five parameters), we are solving
an over constrained system of linear equations for the exact parameter vector. Of course if we
superimpose some random noise to y, which is more realistic, then our estimation performance
will suffer.

6.7.3 R ECURSIVE LEAST- SQUARES IN S IMULINK

The easiest way to perform recursive estimation in S IMULINK is to use the the identification rou-
tines from the S YSTEM I DENTIFICATION T OOLBOX. Fig. 6.47 shows both the S IMULINK block
diagram for the example given in Fig. 6.45, and the resultant trend of the prediction error.

The ARX block also stores the resultant models in a cell array which can be accessed from the
workspace once the simulation has finished. The structure of the model to be estimated, and the
frequency at which it is updated is specified by double clicking on the ARX block.

B UILDING YOUR OWN RLS IN S IMULINK

If we dont have the System Identification toolbox, we can still reproduce the functionality of the
ARX block by writing out the three recursive equations for the update in raw S IMULINK as shown
in Fig. 6.48. To create the vector of current and past input/output data, I have used a tapped
delay line which is the vertical chain of delays shown in Fig. 6.48(a).
6.7. ONLINE MODEL IDENTIFICATION 293

[1x7]
theta
1x7

Parameters
Recursive Polynomial Model Estimator

Output
Inputs
1+1.2z -1
u 1-1.9z -1+1.5z -2
-0.5z -3 y
Band-Limited Terminator
Plant under test
White Noise

(a) Identifying a simple plant using the RLS block in S IMULINK .

Actual Output (Red Line) vs. The Simulated Predicted Model output (Blue Line)
5

5
40 45 50 55 60 65 70 75 80 85
Time (secs)
Error In Simulated Model

0.5

40 45 50 55 60 65 70 75 80 85
Time (secs)
(b) The output of the RLS block in S IMULINK

Figure 6.47: RLS under S IMULINK. In this case, with no model/plant mismatch, and no distur-
bances, we rapidly obtain perfect estimation.

The bulk of the RLS update computation is contained in the RLS system block shown in Fig. 6.48(b)
which simply implements the thee equations given in Algorithm 6.2 on page 288. Note that the
initial condition of the P matrix must be set in the unit delay block to something like 106 I, and
not left to the default which is 0.

In reality, implementing something as complicated as a RLS scheme in a block diagram language


as shown in Fig. 6.48(b) is unwieldy and very mistake prone. While in early versions of S IMULINK
it was discouraged to embed m-file scripts for performance reasons, that has now changed, so
it would be better to implement the RLS update inside a M ATLAB function block. This is the
approach that will be taken in the adaptive pole-placement example in S IMULINK presented later
in section 7.3.4.
294 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

1+1.2z -1
y
u 1-1.9z -1+1.5z -2
-0.5z -3 y(k)
Band-Limited output
Plant under test Unit Delay2 1/z
White Noise
theta
y(k-1) 5 theta 5
Unit Delay3 1/z y tt parameters

[5x5]
y(k-2) phi P
5 P
Unit Delay4 1/z phi [5x5]
5 RLS system
C ariance
y(k-3)

1/z
u(k-1)
Unit Delay1

(a) Implementing RLS in raw S IMULINK using a tapped delay line.

theta(k)
5
1 1
y(k) Unit Delay
y z
5 5
5 5 5
yhat error 1
5 theta(k+1)
Dot Produ 5
5 5
theta(k+1)
Produ

[5x5] M 5
5 5
2 My 5
phi 5 K
phi
p Dide

5 u+1
5
1 !"#$%phi
p
phi Bias

P(k)

 Produ
[5x5] p
P(k) 1 [5x5] P(k+1) M 5 Transpose
My
[5x5] [5x5] M
[5x5] z [5x5] [1x5] K# 5
[5x5] [5x5] My uT
[5x5] [1x5]
[5x5]

eye(5) I

2
[5x5]
P(k+1)

(b) The internals of the RLS block in S IMULINK . The inputs are the current plant output, yk , and , and the outputs
are the parameter estimate, , and the covariance matrix, P.

Figure 6.48: Implementing RLS under S IMULINK without using the System Identification toolbox.
6.7. ONLINE MODEL IDENTIFICATION 295

S HIFT REGISTERS AS DISCRETE STATE - SPACE MODELS

An elegant way to generate the past values of input and output vector required for the vector
as opposed to the tapped delay line used in Fig. 6.48(a), is to implement a shift register in discrete
state-space. If we have a stream of data, and we want to keep a rolling collection of the most
recent three values, then we could run a discrete state-space system as

yk 0 0 0 yk1 1
yk1 = 1 0 0 yk2 + 0 yk (6.53)
yk2 0 1 0 yk3 0

where the state transition matrix has ones on the first sub-diagonal.

In general, if we want to construct a shift register of multiple old values of input and output, we
can construct a discrete state-space system as


yk 0 0 0 y k1 1

yk1 1 0 0 yk2 0

0 0
.
. 0 1 .. 0 .
..
...

. .

ykna 0 0 1 ykna1 0 
yk
= + (6.54)
uk
0 0 0 uk1
uk
1

uk1 uk2



1 0 0


0
. 0 .. 0 .
. .. .
. 0 1 . 0 . .

uknb 0 0 1 uknb1 0

where we are collecting na shifted values of y, and nb values of u.

6.7.4 T RACKING THE PRECISION OF THE ESTIMATES

One must be careful not to place too much confidence on the computed or inherited covari-
ance matrix P. Ideally this matrix gives an indication of the errors surrounding the parameters,
but since we started with an arbitrary value for P, we should not expect to believe these values.
Even if we did start with the non-recursive scheme, and then later switched, the now supposedly
correct covariance matrix will be affected by the ever present nonlinearities and the underly-
ing assumptions of the least square regression algorithm (such as perfect independent variable
knowledge) which are rarely satisfied in practice.

Suppose we attempt to estimate the two parameters of the plant


b0
G= (6.55)
1 + a1 q 1
where the exact values for the parameters are b0 = 1 and a1 = 0.5. Fig. 6.49 shows the parameter
estimates slowly converging to the true values, (starting with an unreasonably low P0 = 2I to
better illustrate the trend) and the elements of the covariance matrix P decreasing for the first 10
samples.

Now even though in this simulated example we have the luxury of knowing the true parameter
values, the RLS scheme also delivers an uncertainty estimate of the estimated parameters via
the covariance P matrix. Fig. 6.50, which uses the same data from Fig. 6.49, superimposes the
296 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Recursive leastsquares parameter estimation


2

Input/output
0

1 Output
Input
2
4

parameters,
1 b
0
a
1
0

Figure 6.49: Estimation of a two parameter


trace(P)

plant. Note that the parameter estimates 10


0

converge to their correct values, and that


the trace of the covariance matrix decreases
with time. See also Fig. 6.50 for the associ- 0 2 4 6 8 10
ated confidence ellipses. sample No.

confidence ellipses in parameter space around the the current best-estimate. (The darker points
and associated ellipses correspond to more recent data.) It is interesting to note that after the 10 or
so samples, the estimates in Fig. 6.49 have nearly converged to the true values, but the confidence
ellipses according to Fig. 6.50, show surprisingly large uncertainties.

4
b0

1
b*0

*
0 a1
Figure 6.50: The trend of the 95% con-
fidence regions for the first 10 estimates 6 5 4 3 2 1 0 1
shown in Fig. 6.49. a
1

A MORE REALISTIC ESTIMATION EXAMPLE

The impressive results of the fully deterministic estimation example shown in Fig. 6.46 is some-
what misleading since we have a no process noise, good input excitation, no model plant/mis-
6.8. MODIFICATIONS TO THE BASIC RLS SCHEME 297

match in structure and we only attempt to identify the model once. All of these issues are crucial
for the practical implementation of an online estimator. The following simulation examples will
investigate some of these problems.

Suppose we want to identify a plant where there is an abrupt change in dynamics at sample time
k = 50,
1.2 q 1

1 0.4 q 2
, for k < 50,
S() = 1 0.2 q 1
1.5 q

, for k 50
1 0.3 q 1 0.7 q 2
A real example of a such an abrupt change in dynamics might arise in the case of a gantry crane
when model 1 is when the crane is unloaded, and model 2 is when the crane has picked up a
heavy shipping container for example.

In this case shown, as in Fig. 6.46, the parameters quickly converge to the correct values as shown
in Fig. 6.51 for the first plant, but after the abrupt model change, they only reluctantly converge
to the new values.

Identifying abrupt plant changes


40
Input/output

20

20
1
error

1
1.6
b0
1.4

1.2

0.8
a2

0.6

0.4
Figure 6.51: Identifying an abrupt
a1 plant change at k = 50. The estima-
0.2
tors parameters initially converges
0 quickly to the first plant (lower
0.2
0 50 100 150 200 250 300
trend), but find struggle to converge
sample No. to the second plant.

This problem of an increasing tendency to resist further updating is addressed by incorporating


a forgetting factor into the standard RLS algorithm and is described next in 6.8.1.

6.8 M ODIFICATIONS TO THE BASIC RLS SCHEME

The basic RLS online identification algorithm of Algorithm 6.2 works reasonably well in practice,
at least for short intervals. However some problems start to be come evident, especially if the
algorithm is implemented on embedded hardware with limited precision over long periods of
time. The following sections describe some of the practical issues that arise in RLS and ways to
solve them.
298 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

6.8.1 T HE FORGETTING FACTOR

Sometimes it is a good idea to forget about the past. This is especially true when the process pa-
rameters have changed in the interim. As we saw in Fig. 6.51, if we simply implement Eqns 6.51
and 6.50, or alternatively run the routine in Listing 6.17, we will find the estimation works well
initially since the estimates converge to the true values, and the trace of the covariance matrix de-
creases as our confidence improves. However if the process subsequently changes again, we find
that the parameters do not converge as quickly to the new parameters. This may be surprising
since the convergence worked fine at first, so why didnt it work a second time?

The reason for this failure to converge a second time is due to the fact that the now minuscule
covariance matrix, reflecting our large, but misplaced confidence, is inhibiting any further large
changes in .

One solution to this problem, is to reset the covariance matrix to some large value, say 103 I for
example when the system change occurs. While this works well, it is not generally feasible, since
we do not know when the change is likely to take place. If we did, then gain scheduling may be
more appropriate.

An alternative scheme is to introduce a forgetting factor, , to decrease the influence of old sam-
ples. With the inclusion of the forgetting factor, the objective function of the fitting exercise is
modified from Eqn. 6.23 to
N
X 2
J = N k yk Tk , <1 (6.56)
k=1

where data n samples past is weighted by n . The smaller is, the quicker the identification
scheme discounts old data as shown in Fig. 6.52. Incidentally it would make more sense if the for-
getting factor were called the remembering factor, since a higher value means more memory!

Incorporating the forgetting factor into the recursive least squares scheme modifies the gain and
covariance matrix updates very slightly to
PN N +1
KN +1 = (6.57)
+ TN +1 PN N +1
!
1 PN N +1 TN +1 PN
PN +1 = PN
+ TN +1 PN N +1
PN  
= I N +1 KTN +1 (6.58)

To choose an appropriate forgetting factor, we should note that the information dies away with a
time constant of N sample units where N is given by
1
N= (6.59)
1
As evident from Fig. 6.52, a value of is 0.99 (which is typical) gives a time constant of 100
samples.

Consequently most textbooks recommend a forgetting factor of between 0.95 and 0.999, but lower
values may be suitable if you expect to track fast plant changes. However if we drop too much,
we will run into covariance windup problems which are further described later in section 6.8.3.

We can demonstrate the usefulness of the forgetting factor by creating and identifying a time
varying model, but first we need to make some small changes to the recursive least-squares up-
date function rls given previously in Listing 6.16, to incorporate the forgetting factor. as shown
6.8. MODIFICATIONS TO THE BASIC RLS SCHEME 299

=1 200
1

0.8 150
weighting

0.6

N
100
=0.995
0.4
=0.99
=0.98
50
0.2 =0.95

0 0
0 50 100 150 200 250 300 0.6 0.7 0.8 0.9 1
Historic time
(a) The weighting as a function of the forgetting factor, (b) The memory, N when using a forgetting factor
. from Eqn. 6.59. Typically = 0.99 which corresponds
to a time constant of 100 samples.

Figure 6.52: As the forgetting factor is reduced, older samples are deemed less important.

in Listing 6.18 which we should use from now on. Note that if is not specified explicitly, a value
of 0.995 is used by default.

Listing 6.18: A recursive least-squares (RLS) update with a forgetting factor. (See also List-
ing 6.16.)
function [thetaest,P] = rls(y,phi,thetaest,P,lambda)
% Basic Recursive Least-Squares (RLS) parameter update
% Inputs: yk ,k , : current output & past input/outputs column vector & forgetting factor
4 % Input/Outputs: , P: parameter estimates & covariance matrix

if nargin < 5
lambda = 0.995; % Set default forgetting factor,
end % if
9

K=P*phi/(lambda + phi'*P*phi); % Gain K, Eqn. 6.57.


P=(P-(P*phi*phi'*P)/(lambda+phi'*P*phi))/lambda;% Covariance update, P, Eqn. 6.58. 
thetaest = thetaest + K*(y - phi'*thetaest);% N+1 = N + KN+1 yN+1 TN+1 N
return % end rls.m

6.8.2 T HE INFLUENCE OF THE FORGETTING FACTOR

Figure 6.53 shows how varying the forgetting factor alters the estimation performance when
faced with a time varying system. The true time varying parameters S() are the dotted lines,
and the estimated parameters M() for different choices of from 0.75 to 1.1 are the solid lines.

When = 1, (top trend in Fig. 6.53), effectively no forgetting factor is used and the simulation
shows the quick convergence of the estimated parameters to the true parameters initially, but
when the true plant S() abruptly changes at t = 200, the estimates, M(), follows only very
sluggishly, and indeed never really converges convincingly to the new S(). However by incor-
porating a forgetting factor of = 0.95, (second trend in Fig. 6.53), better convergence is obtained
for the second and subsequent plant changes.

Reducing the forgetting factor still further to = 0.75, (third trend in Fig. 6.53), increases the con-
300 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

1 b1

=1
a2
0 a3
1

1 b1

= 0.95
a2
0 a3
1

2
= 0.75
1 b
1
a2
0 a3
1

2
Figure 6.53: Comparing the estimation per-
formance of an abruptly time-varying un- 1 b
1
= 1.1

known plant with different forgetting factors, a2


. 0 a3
Estimated parameters (solid) and true param-
1
eters (dashed) for (a) = 1, (b) = 0.95, (c)
= 0.75, (d) = 1.1. Note that normally one 0 200 400 600
would use 0.95, and never > 1. sample time [k]

vergence speed, although this scheme will exhibit less robustness to noisy data. We also should
note that there are larger errors in the parameter estimates (S() M()) during the transients
when = 0.75 than in the case where = 1. This is a trade-off that should be considered in the
design of these estimators. We can further decrease the forgetting factor, and we would expect
faster convergence, although with larger errors in the transients. In this simplified simulation
we have a no model/plant mismatch, no noise and abrupt true process changes. Consequently
in this unrealistic environment, a very small forgetting factor is optimal. Clearly in practice the
above conditions are not met, and values just shy of 1.0, say 0.995 are recommended.

One may speculate what would happen if the forgetting factor were set greater than unity ( > 1).
Here the estimator is heavily influenced by past data, the more distant pastthe more influence.
In fact, it essentially disregards all recent data. How recent is recent? Well actually it will disre-
gard all data except perhaps the first few data points collected. This will not make an effective
estimator. A simulation where = 1.1 is shown in the bottom trend of Fig. 6.53), and demon-
strates that the estimated parameters converge to the correct parameters initially, but fail to budge
from then on. The conclusion is not to let assume values > 1.0.

6.8.3 C OVARIANCE WIND - UP

Online estimation in practice tends to work well at first, but if the estimator is left running for
extended periods of time, such as a few days or even weeks, the algorithm often eventually fails.
6.8. MODIFICATIONS TO THE BASIC RLS SCHEME 301

One of the reasons leading to a breakdown is that during periods of little interest such as when
the system is a steady-state, there is no information arriving into the estimator to reduce further
the uncertainty of the estimated parameters. In fact the reverse occurs, the uncertainty grows,
and this is seen as a slow but sure exponential growth in the elements of the covariance matrix
P. Soon the matrix will overflow, and this is referred to as covariance wind-up. Any practical
algorithm must take steps to monitor for the problem and take steps to avoid it.

During periods of little interest, the covariance update, Eqn. 6.58, at sample time k, reduces to
effectively

Pk T
Pk+1 = I k+1 Kk+1
| {z }
almost zero
Pk
(6.60)

which means that the P matrix increases each sample by a factor of 1/ which under normal
circumstances is say around 2%. It will continue this exponential increase until either there is
sufficient excitation in input and output to start to drop the covariance again or it overflows.

Consequently it is prudent to watch the size of P by monitoring the size of the individual ele-
ments. A suitable scalar size measure is given by the trace of the P matrix, which is simply the
sum of the diagonal elements
n
X Xn
def
tr(P) = pi,i = i
i=1 i=1

The trace of a matrix has the nice property in that it also equals the sum of the n eigenvalues, i .
Since P is positive definite, the trace will always be positive. If the trace gets too large, then some
corrective action is required.

Suppose we try to identify the same system as used in section 6.7.2,

1.2q 1
S=
1 0.25q 1 0.50q 2

using a recursive least-squares algorithm with a forgetting factor of 0.98. Rather than using the
optimal white noise as an input excitation signal, we will use a poor input excitation, such as a
square wave with an excessively long period. While the parameters do converge to the correct
values, Fig. 6.54 shows also that the trace of P increases during the periods of no input excitation
to astronomically large values such as 1015 . Clearly the covariance matrix would grows without
bound and will soon exceed the numerical limits of our control computer. If instead we had used
the ideal white noise input signal, the continual excitation keeps the trace of P small. As evident
from Eqn. 6.60, the smaller our forgetting factor is, the faster the covariance matrix will windup.

M ODIFICATIONS ON THE FORGETTING THEME

Introducing a forgetting factor is just one attempt to control the behaviour of the covariance ma-
trix. As you might anticipate, there are many more modifications to the RLS scheme along these
lines such as variable forgetting factors, directional forgetting factors, constant trace algorithms,
adaptive forgetting factors, startup forgetting factors etc. See [205] for further details.

The variable forgetting factor introduced in [68] is particularly easy to implement. A similar
algorithm was developed at around the same time in [204] and the two algorithms compared in
[178].
302 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Input/output
0

5
2


0
20
10

=0.98
Figure 6.54: Covariance wind-up. In
tr(P)

this case with = 0.98, the param-


eters converge to their correct val-
ues, (middle trend), but the trace of
0
the covariance matrix grows during 10
the periods of little input excitation, 0 1000 2000 3000 4000 5000
(lower plot). sample No.

The time-varying forgetting factor is now no longer a constant value, but adjusted according to

(1 T K)2
(t) = 1 (6.61)
0

where is the residual error, and 0 is chosen as N 2 and where 2 is the anticipated measure-
ment noise variance of the system, and N adjusts the rate of adaption. One way to interpret N is
from Eqn. 6.59 thus giving
2
0 = N 2 =
1
using a nominal value of = 0.995. It may be also prudent to place some safety limits on (t)
such as
0.3 (t) 0.9999
Fig. 6.55 shows the result of using this simple improvement to the estimation algorithm when we
have multiple abrupt plant changes. Note that in this case, (as opposed to the situation given in
Fig. 6.51), we do manage to both re-identify the second and subsequent changes, and that the trace
of the covariance matrix does not ever get too large. During the changes however, the varying
forgetting factor does drop significantly at times reaching the lower limit of 0.3.

6.8.4 M AINTAINING A SYMMETRIC POSITIVE DEFINITE COVARIANCE MATRIX

A key concern when implementing a recursive identification algorithm that is of particular con-
cern in limited precision is to always ensure that the covariance matrix is symmetric positive
definite or SPD. Due to the arithmetic operations carried out with limited precision, the P matrix
can lose its positive definite properties, causing the algorithm to fail. The question then is what
to do about that. Ideally we would like to first check if the matrix is indefinite,2 and then if so,
adjust the matrix to the nearest positive definite matrix. Unfortunately this is not a simple or
2 perhaps by attempting a Cholesky factorisation
6.8. MODIFICATIONS TO THE BASIC RLS SCHEME 303

output & setpoint


4
2
0
2

0.01
input
0

0.01

2 a
2
1
b1
0

a
3
1

2 a
1
tr(P)

4
10

0.8

0.6
(t)

0.4

0 100 200 300 400


sample time

Figure 6.55: Using a variable forgetting factor to improve the identification of abrupt plant
changes. Note the improvement over the problems exhibited in Fig. 6.51.

computationally cheap correction, but the first thing we could do is to enforce symmetry with a
simple correction like

if issymmetric(P) % Check if matrix P is symmetric, and if not . . .



2 P = (P+P')/2; % Enforce symmetry with: P P+P 2
end

and then to enforce positive definiteness, we can take an eigenvalue decomposition, simply reset
any negative eigenvalues to a small positive number, and then rebuild the original matrix.

[V,D] = eig(P);
2 P = V*max(D,0+eps)*V';
304 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Strictly speaking, the final line should have been VDV1 , but since V is orthogonal, VT = V1 .
In fact this correction strategy is also suspectable to small numerical errors, so the similar routine
nearestSPD by John DErrico available from the Mathworks website should be used. Note that
we could use the M ATLAB command issymmetric to simply establish if the matrix is symmetric
or not.

6.9 O NLINE ESTIMATING OF NOISE MODELS

So far we have assumed that the noise acting on our system has been random white noise. How-
ever in practice, coloured noise is far more common, and blindly applying the RLS scheme to a
process disturbed by coloured noise will in general result in biased estimates. The text [205, p102]
contains further details. Recall that the ARMAX model of Fig. 6.29 is described by Eqn. 6.13
repeated here,
A(q)yk = B(q)ukd + C(q)ek (6.62)
| {z }
coloured noise
is comprised of three polynomials, A, B and C,
A = 1 + a1 q 1 + + ana q na , B = b0 + b1 q 1 + + bnb q nb
C = 1 + c1 q 1 + + cnc q nc
where the A and C polynomials are assumed monic.

Because we know so little about it, the noise model polynomial, C, is typically never larger than
first or second order. We wish to estimate the na + nb + nc + 1 unknown coefficients of A, B and
C. The problem we face when trying to estimate the coefficients of C is that we can never directly
measure the noise term ek . So the best we can do is replace this unknown error term with an
estimate such as the prediction error,

k = yk k k1 (6.63)
and augment the data vector with the estimates of the C polynomial. This scheme is referred to as
recursive extended least-squares, or RELS. Note carefully the difference in nomenclature between
the actual, but unmeasured error, e and the estimate of that error, .

The recursive extended least-squares algorithm given in Algorithm 6.3 is an extension of the stan-
dard recursive least-squares algorithm.

Algorithm 6.3 Recursive extended least-squares


We aim to recursively estimate the parameters of the A, B and C polynomials in the dynamic
process
A(q 1 )yk = B(q 1 )ukd + C(q 1 )ek
by measuring uk and yk . We note that A and C are assumed monic, so the parameter vector is
h iT
= a1 an ... b0 bn ... c1 cn
def
a b c

where na , nb and nc are the orders of the A, B and C polynomials respectively.

1. Construct the augmented data row vector using past input/output data and past prediction
error data,

.. ..
k = |yk1 , . .{z. , ykna}, . |ukd , . . . ,{z
ukd1nb , . k1 , . . . , knc
} | {z }
past outputs inputs predicted errors
6.9. ONLINE ESTIMATING OF NOISE MODELS 305

2. Collect the current input/output data, yk , uk , and calculate the present prediction error us-
ing the previously obtained model parameter estimates

k = yk k k1

3. Update the parameter covariance matrix, gain, and parameter estimate vector using the
standard RLS update equations, (Eqns 6.50, 6.51, and 6.52), from Algorithm 6.2.

4. Wait out the remaining of the sample time, and then return to step (1).

Of course we could modify these update equations to incorporate additional features such as
forgetting factors in the same manner as in the normal recursive least squares case.

An alternative algorithm is the approximate maximum likelihood, where instead of using the
prediction error, one uses the residual error,

k = yk k k (6.64)

and minor changes to the covariance and parameter update equations. (Note the difference be-
tween this and Eqn. 6.63.) Further implementation details are given in [205, p105].

6.9.1 A RECURSIVE EXTENDED LEAST- SQUARES EXAMPLE

For this simulated example, we will compare the parameter estimates of an unknown plant dis-
turbed by coloured noise using both the normal recursive least-squares (where we anticipate bi-
ased estimates), and the recursive extended least-squares algorithm to additionally estimate the
C polynomial and to remove the bias estimates of the A and B polynomials.

Our true plant is

(1 0.4q 1 0.5q 2 ) yk = (1.2q 1 + 0.3q 2 ) uk + (1 0.8q 1 0.1q 1 ) ek (6.65)


| {z } | {z } | {z }
A(q1 ) B(q1 ) C(q1 )

where ek N (0, 2.5), and the input uk N (0, 1). Note that we have coloured noise as C 6= 1 and
that this noise variance is relatively large.

If we just ignore the noise model, and attempt to estimate just the four unknown A and B polyno-
mial coefficients using the normal RLS algorithm, then we get large biases especially as evident
in the first half of the Fig. 6.56.

If however, we swap over to use the extended recursive least-squares algorithm at time k = 2000,
then the estimation performance is much improved, exhibits much less bias, and even manages
(eventually) to estimate the noise polynomial correctly as shown in the latter section of Fig. 6.56.
The downside of course is that now we must estimate the noise polynomial coefficients as well
increasing the number of unknown parameters from 4 to 6.

Example Estimating parameters in an ARMAX model.

For the dynamic system described by Eqn. 6.65, we will create an ARMAX model using the
idpoly object, create some suitable random input, and use the model to generate and output
as shown in Listing 6.19. Note how I must pad the B polynomial with a leading zero to take into
account the deadtime.
306 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

=0.9990
2

RLS RELS
1.5

b
1
Parameter estimates

1
c
2

0.5 a
3
b2

0
c
3

a
2
0.5

1
0 1000 2000 3000 4000 5000 6000
sample time [k]

Figure 6.56: Estimating model parameters using recursive least squares in the presence of
coloured noise. The first period shows the A and B estimates (solid) and true values (dashed)
using normal RLS. The second half of the trend shows the A, B and C estimates (solid) and true
values (dashed) using extended RLS. Note the eventual lack of bias in the latter RELS estimates.

Listing 6.19: Create an ARMAX process and generate some input/output data suitable for subse-
quent identification.
nt= 100; u= randn(nt,1); e= randn(nt,1); % # of samples, input & disturbance

3 A = [1 -0.4 0.5]; B = [0, 1.2, 0.3]; C = [1, 0.8 -0.1]; % True coefficients
G = idpoly(A,B,C) % Create true ARMAX plant
y = sim(G,[u e]); % Simulate Ay = Bu + Ce
Zdata = iddata(y,u,1); % collect output/input data, sample time T = 1.

To estimate the parameters we could either use the general pem or the more specific armax com-
mands. We will estimate a structurally perfect model, that is 2 parameters in each of the A, B and
C polynomials and 1 unit of dead time.

Listing 6.20: Identify an ARMAX process from the data generated in Listing 6.19.
>>G_est= pem(Zdata,'na',2,'nb',2,'nc',2,'nk',1);% Estimate A(q 1 ), B(q 1 ) & C(q 1 )
>>G_est= armax(Zdata,'na',2,'nb',2,'nc',2,'nk',1)% Alternative to using pem.m

4 >>present(G_est) % present results


G_est =
Discrete-time ARMAX model: A(z)y(t) = B(z)u(t) + C(z)e(t)
A(z) = 1 - 1.416 z-1 + 0.514 z-2

9 B(z) = 1.209 z-1 + 0.2935 z-2

C(z) = 1 + 0.7703 z-1 - 0.1181 z-2

>>yp = sim(G_est,[u e]); % form prediction


14 >>plot([y yp]) % check prediction performance in Fig. 6.57.

Fig. 6.57 compares the true and predicted output data. In the left-hand case, the fit is not perfect
even though we used the correct model structure which is partly due to the iterative nature of the
nonlinear regression, but mostly the poor fit is due to the fact that we used only 100 input/output
6.9. ONLINE ESTIMATING OF NOISE MODELS 307

samples.

On the other hand, if we used 10,000 samples for the fitting as in the right-hand case, we will
estimate a far superior model. It is interesting to note just how many input/output samples we
need to estimate a relatively modest sized system of only 6 parameters.

Using only 100 points for estimation Using 10000 points for estimation
15 2

4
10
6
Output & estimated output

8
5
10

0 12

14
5
16

18
10
Actual
Estimated 20
Estimated
15 22
0 20 40 60 80 100 9900 9920 9940 9960 9980 10000
sample time sample time

Figure 6.57: A validation plot showing the actual plant output compared to the model predictions
of an ARMAX model using series of different lengths. Note that to obtain good results, we need
a long data series.

Listing 6.21 repeats this estimation, but this time does it recursively in M ATLAB.

Listing 6.21: Recursively identify an ARMAX process.


1 A = [1 -0.4 0.5]; B = [1.2, 0.3]; C = [1, 0.8 -0.1]; % True plant to be estimated

na = length(A); nb = length(B); nc = length(C); % Order of the polynomials+1


d = na-nb; % deadtime

6 Npts = 5e4;
u = randn(Npts,1); a = randn(Npts,1);
y = zeros(Npts,1); err = y; thetaAll = []; % dummy

np = na+nb+nc - 2; % # of parameters to be estimated


11 theta = randn(np,1); P = 1e6*eye(np); % initial guess of
lambda = 1; % forgetting factor, 1

for i=na:Npts;
y(i)= -A(2:end)*y(i-1:-1:i-na+1)+ B*u(i-d:-1:i-nb+1-d)+ C*a(i:-1:i-nc+1);
16

phi = [-y(i-1:-1:i-na+1)', u(i-d:-1:i-nb+1-d)', err(i-1:-1:i-nc+1)']';


err(i,1) = y(i) - phi'*theta; % estimated noise sequence

K = P*phi/(lambda + phi'*P*phi); % update gain


21 P = (P-(P*phi*phi'*P)/(lambda + phi'*P*phi))/lambda; % covariance update
theta = theta + K*err(i,1); % update
thetaAll(i,:) = theta';
end

26 Aest = [1, thetaAll(end, 1:na-1)]; Best = thetaAll(end, na:na+nb-1);


Cest = [1, thetaAll(end,na+nb:end)]; % See Fig. 6.58.

As shown in Fig. 6.58, the estimates do eventually converge to the correct values without bias,
308 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

and furthermore, the step response of the final estimate compares favourably with the true step
response.

RELS: =0.9990 Model at k=4e4


2 5

0
1.5
Estimated
Actual
5
0 5 10 15 20 25
1 (seconds)
c2
b2
Parameter estimates

0.5
a3
b3
0 b
c31

0.5

Model at k=2e4

1 3
a
2
2

1.5 1
Estimated
Actual
0
0 5 10 15 20 25
2 (seconds)
0 1 2 3 4 5
sample time [k] 4
x 10

Figure 6.58: Recursive extended least-squares estimation where there is an abrupt model change
at time k = 2 104. The inserted figures compare a step response of the true plant with the current
estimate.

6.9.2 R ECURSIVE IDENTIFICATION USING THE SI TOOLBOX

The System Identification toolbox can also calculate the parameter estimates recursively. This
essentially duplicates the material presented in 6.7.1. However in this case there are many more
options to choose from regarding type of method, numerical control, starting point etc.

The toolbox has a number of recursive algorithms (See [127, p164]) but they all work in the same
way. For most demonstration purposes, we will recursively estimate the data pseudo online. This
means that we will first generate all the data, and then estimate the parameters. However we
will make sure that when we are processing element i, the data ahead in time, elements i + 1 . . .,
are not available. This pseudo online approach is quite common in simulation applications. The
Toolbox command that duplicates the recursive least squares estimation of a ARMAX process
with a forgetting factor is rarmax(z,nn,ff,0.95).

6.9.3 S IMPLIFIED RLS ALGORITHMS

The standard recursive least squares (RLS) algorithm such as presented in 6.7.1 requires two
update equations; one for the update gain, and one for the covariance matrix. But for imple-
mentation in a small computer, there is some motivation to simplify the algorithm, even at the
expense of the quality of parameter estimates. Astrom and Wittenmark, [18, p69-70] describe one
such method known as Kaczmarzs projection algorithm. The input/output relation is assumed

yk = k k
6.9. ONLINE ESTIMATING OF NOISE MODELS 309

where is the parameter vector and is the row data vector, and the correction to the estimated
parameter vector is
k = k1 +
k

where is chosen such that yk = k k which gives the full updating formula in our now familiar
form as
T  
k = k1 + y k k k1 (6.66)
k T
Note that in [18], there is some nomenclature change, and that the data vector is defined as a
column vector, where here I have assumed it is a row vector. This update scheme (Eqn 6.66) is
sometimes modified to avoid potential problems when the parameter vector equals zero.

The script in listing 6.22 first generates data from an ARX plant, and then the data is processed
pseudo-online to obtain the estimated parameters.

Listing 6.22: Kaczmarzs algorithm for identification


u = randn(50,1); % random input

3 A = [1 -1.25 0.8]; B = 0.3; % G = 0.3/(1 1.25q 1 + 0.8q 2 )


G = idpoly(A,B,1); y = sim(G,u);

theta = randn(3,1);% Initial parameter estimate, 0 , is random


Param = []; % estimated parameters so far
8 for i=4:length(u);
phi= [-y(i-1:-1:i-2)', u(i)]'; % regressor vector, k 


theta=theta+phi*(y(i)-phi'*theta)/(phi'*phi); % k k1 + Tk yt Tk k1
k
Param= [Param; theta']; % collect parameters
end %
13

Param_x = ones(size(Param(:,1)))*([A(2:3) ,B(1)]); % true parameters


plot([Param_x, Param]) % have they converged ?

The input/output data (top) and parameters from this data are shown in Fig. 6.59. The estimated
parameters converge relatively quickly to the true parameters, although not as quickly as in the
full recursive least-squares algorithm.

5
Input & output

5
1
a
2
Estimated & true parameters

0.5
b
0

0.5 Figure 6.59: The performance of a simpli-


fied recursive least squares algorithm, Kacz-
marzs algorithm.
1
Upper: The input/output data.
a
1 Lower: The estimated parameters (solid)
1.5 eventually converge to the true parameters
0 10 20 30 40 50
sample time (dashed).
310 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Problem 6.3 The following questions use input/output data from the collection of classic time
series data available from the time series data library maintained by Rob Hyndman at:
www-personal.buseco.monash.edu.au/hyndman/TSDL/ or alternatively from the DAISY:
D ATABASE FOR THE I DENTIFICATION OF S YSTEMS collection maintained by De Moor, Department
of Electrical Engineering, ESAT/SISTA, K.U.Leuven, Belgium at
www.esat.kuleuven.ac.be/sista/daisy/.

These collections are useful to test and verify new identification algorithms. Original sources for
this data include [34] amongst others.

1. The data given in /industry/sfbw2.dat shows the relationship in a paper making ma-
chine between the input stock flow (gal/min) and basis weight (lb/3300 ft2 ), a quality control
parameter of the finished paper. Construct a suitable model of this time series.
Ref: [159, pp500501].

2. This problem is detailed and suitable for a semester group project.


An alternative suite of programs to aid in model development and parameter identification is
described in [34, Part V]. The collection of seven programmes closely follows the method-
ology in the text, and are outlined in pseudo-code. Construct a T IME S ERIES A NALYSIS
toolbox with graphics based around these programs.
Test your toolbox on one of the time series tabulated in the text. A particularly relevant series
is the furnace data that shows the percentage of carbon dioxide output from gas furnace at
9 seconds intervals as a function of methane input, (ft3 /min), [34, Series J, pp532533].
This input/output data is reproduced in Fig. 6.60, although the input methane flowrate is
occasionally given as negative indicating that the series is probably scaled. The original
reference footnoted that the unscaled data lay between 0.6 and 0.04 ft3 /min.

65

60
% CO2

55

50

45
4

2
Methane flow

0
Figure 6.60: The percent CO2 in the
2
exhaust gas from a furnace as a func-
tion of the methane input (scaled). 4
0 500 1000 1500 2000 2500
Data from [34, p532533]. time [s]

6.10 I DENTIFICATION BY PARAMETER OPTIMISATION

Model identification can also be achieved using local parametric optimisation theory. This has
the advantage that it is conceptually simple, allows for general, possibly nonlinear functions,
and closely follows classical optimisation techniques. The basic idea is to update the parameter
vector so that a performance index is minimised. In this manner, the problem is reduced to one
of a normal minimisation problem where many computer algorithms already exist. However
one typically assumes that we have good starting guesses for the parameters, and that one must
live with the resulting slow rate of convergence. Also these simple optimisation schemes can
6.10. IDENTIFICATION BY PARAMETER OPTIMISATION 311

become unstable, and the resulting stability analysis is typically quite involved. This section
closely follows [114, pp66-73].

Suppose we are given a system with known denominator dynamics,


b0
y(t) = u(t) (6.67)
1 + a1 s + a2 s2
but where we wish to estimate the steady-state gain, b0 . Therefore our model M of the process is

M : (1 + a1 s + a2 s2 )y(t) = b0 u(t) (6.68)

where we wish b0 b0 . We will take the squared error as our performance index,
Z
1
J = 2 dt (6.69)
2
where the error between the actual and predicted is = y y.

The gradient optimisation technique is an intuitive method that tells us how to adjust our un-
known parameter vector to search for the minimum. We adjust our parameters in such a manner
as to decrease the residual at a maximum rate. We will use the steepest descent method,
J
b0 = k (6.70)
b0
where k > 0 is the adaption gain which can be adjusted to vary the speed of convergence. The
rate of change of our parameter is
 
db0 J
= k (6.71)
dt t b0
Now we can reverse the order of differentiation in Eqn. 6.71 provided the system does not change
too fast,
 
db0 j k 2
= k =
dt b0 t 2 b0

= k (6.72)
b0
Eqn. 6.72 is referred to as the MIT rule. To use the MIT rule in an adaptive scheme, we must
evaluate the sensitivity function,
y y
= (6.73)
b0 b0 b0
|{z}
=0

so therefore the adaption rule is


db0 y
= k (6.74)
dt b0
The sensitivity function of the predicted output y with respect to the parameter is obtained by
differentiating the original model, Eqn. 6.68. Note that

(1 + a1 s + a2 s2 )y = b0 u
y + a1 y + a2 y = b0 u

so that the partial derivative of model M with respect to b0 is

y y y
+ a1 + a2 =u (6.75)
b0 b0 b0
312 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

Again we can change the order of differentiation so


   
y y 2 y
+ a1 + a2 2 =u (6.76)
b0 t b0 t b0
y
(1 + a1 s + a2 s2 ) =u (6.77)
b0
Dividing Eqn. 6.77 by Eqn. 6.67 gives
y y
= (6.78)
b0 b0
thus giving the adaption law (from Eqn. 6.74)

y y
b0 = k = k (6.79)
b0 b0

which would all work very nicely except that we do not know b0 ! (It is after all, what we are
trying to estimate.) So we will substitute the estimate in the adaption law giving,

y
b0 = k (6.80)
b0

Algorithm 6.4 Searching for the plant gain Given an unknown system S producing measurable
outputs y(t) and a model M with an initial parameter estimate b0 producing outputs y(t) sub-
jected to known inputs u(t),

1. Using the model M with the current best estimate of b0 predict y and compare with the
actual system output, (t) = y(t) y(t).
2. Update the parameter estimate using the adaption law, Eqn. 6.80
3. Go back to step 1.

E XAMPLE OF PARAMETRIC ADAPTION

We will demonstrate this parametric adaption where our true, but only partially known, system
and corresponding model are

0.45 b0
S= , and model M=
s2 + 4.2s + 4.4 s2 + 4.2s + 4.4

where the initial estimate of the unknown gain b0 has the wrong sign at b0 = 0.3.

We will do most of the calculations in the discrete domain, so we will approximate the time
derivative in the adaption law, db0 /dt, with an Euler approximation which when discretised at
sample time T gives,  
y
b0 (t) = b0 (t 1) + T k
b0
The upper plot in Fig. 6.61 calculated from Listing 6.23 compares the result of using the model
M with no adaption (dash-dot) with one where the model is adapted according to the MIT rule
(dotted) line. Note that eventually the model with adaption closely follows the true process (thick
solid line). The lower plot shows that the parameter estimate b0 slowly converging to the true
value of 0.45. Note that in this case the adaptation gain k = 8 and that the adapted model has a
much reduced integrated error than the model with no adaption.
6.10. IDENTIFICATION BY PARAMETER OPTIMISATION 313

Adaption gain = 8
True
0.1
Open loop
Adapted
Plant & Model output

0.05

0.05

0.1
Figure 6.61: Parameter adaption us-
0.5 ing the steepest descent method. The
Gain estimate, b0

b
0
upper plot compares the true system
0
(solid) with the openloop model (dot-
ted) and the adapted model (dashed).
The lower plot shows the parame-
0.5
0 10 20 30 40 50 60 ter adapting gradually to the correct
Time T =0.175 [s] value of b0 = 0.45.
s

Listing 6.23: Adaption of the plant gain using steepest descent


S = tf(0.45,[1 4.2 4.4]); % True plant S = 0.45/(s2 + 4.2s + 4.4)

b0_est = -0.3; % Estimated of parameter b0 = 0.3, while actual b0 = 0.45


G = tf(b0_est,[1 4.2 4.4]); % Estimated model b0 /(s2 + 4.2s + 4.4)
5

dt = 0.175; t = [0:dt:60]'; % Simulate truth & openloop with a random input


U = 0.6*randn(size(t))+0.5*sin(t/4);
Yx = lsim(S,U,t); % truth output, S
Yol = lsim(G,U,t); % open loop estimate, M
10

Y = Yx; B=NaN*Y; k = 8; % Adaption gain, k = 8

for i=3:length(t)-1 % Now do the estimation . . .


G.num = b0_est; Gd = c2d(G,dt);
15 [num,den] = tfdata(Gd,'v');
Y(i) = -den(2:3)*Y(i-1:-1:i-2) + num*U(i:-1:i-2);

e = Yx(i)-Y(i) ; % model/plant error


b0_est = b0_est + dt*k*e*Y(i)/b0_est; % Update b0 b0 + T ky/b0 , Eqn. 6.80
20 B(i,:) = b0_est;
end % for

d = [Y-Yx, Yx-Yol]; jise = sum(abs(d)); % Performance analysis, See Fig. 6.61.

Choosing an appropriate adaptation gain is somewhat of an art. Too small a value and the adap-
tation will be too slow. Too large, and the adaptation may become unstable. Fig. 6.62 trends the
integrated absolute error (ISE) for the adaption scheme as a function of adaptation gain which can
be compared with the openloop scheme over the time horizon used in Fig. 6.61. In this applica-
tion, the best setting for k is around 50, but this is very dependent on starting conditions, problem
type etc. Above this value the performance rapidly deteriorates, and one would often be better
off using only the uncorrected openloop model. As k 0, then there effectively no adaption, and
this is equivalent to the openloop model.
314 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

3
10

2
10

ISE
1
10

0
10

Figure 6.62: Optimising the adaptation


optimal gain
gain. The IAE for the for different adap- 1
10
tation gains, k. The optimum gain, k , is 1
10
0
10 10
1
10
2
10
3

around 50. Adpation gain, k

6.11 C LOSED LOOP IDENTIFICATION

The identification techniques described this far assume that the input signal can be chosen freely
and that this signal has persistent excitation. However if you are trying to identify while con-
trolling, you have lost the freedom to choose an arbitrary input since naturally, the input is now
constrained to give good control rather than good estimation. Under good control, the input will
probably be no longer exciting nor inject enough energy into the process so that there is sufficient
information available for the identification. Closed loop identification occurs when the process is
so important, or hazardous, that one does not have the luxury of opening the control loop simply
for identification. Closed loop identification is also necessary, (by definition), in many adaptive
control algorithms.

If we use a low order controller, then the columns of the data matrix X in Eqn. 6.22 become
linearly dependent. This is easy to see if we use a proportional controller, ut = kyt , then this in
turn creates columns in X that only differ by the constant controller gain k. In reality, any noise
will destroy this exact dependency, but the numerical inversion may still be a computationally
difficult task.

One way to ensure that u(t) is still persistently exciting the process is to add a small dither or
perturbation signal on to the input u. This noisy signal is usually of such a small amplitude,
that the control is not seriously effected, but the identification can still proceed. In some cases
there may be enough natural plant disturbances to make it unnecessary to add noise to u. With
this added noise, added either deliberately or otherwise, the estimation procedure is the same as
described in 6.7.

The above arguments seems to suggest that closed loop identification is suboptimal, and that if
possible, one should always try to do the identification in open loop, design a suitable controller,
then close the loop. This avoids the possible ill-conditioning, and the biasing in the estimated
parameters. However Landau in a series of publications culminating in [113] argues that in fact
closed loop control, with suitable algorithms, actually develops better models since the identifica-
tion is constrained to the frequencies of interest for good closed loop control. It all depends in the
final analysis on whether we want a good model of the plant for say design in which case we do
the identification in open loop, or if we want good control in which case closed loop identification
may be better.

A very readable summary of closed loop identification is given in [93, pp517518], [18, p82] and
especially [205]. Identification in the closed loop is further discussed in chapter 7.
6.12. SUMMARY 315

6.11.1 C LOSED LOOP RLS IN S IMULINK

Implementing recursive identification using S IMULINK is not as straight forward as writing raw
M ATLAB code because of the difficulty in updating the parameters inside a transfer function
block. Under normal operation these are considered constant parameters within the filter block,
so it is not a problem. Fig. 6.63 shows the closed loop estimation using the same RLS blocks as
those used in Fig. 6.48 on page 294. In this example the identified parameters are not used in the
controller in any sort of adaptive manner, for that we need an adaptive controller which is the
subject of the following chapter.

5
5
phi theta theta
5 theta 5
[5x5] Parameters

phi
uy Plant i/o data phi generator y(k) P P
[5x5]
Covariance

y
3 RLS
3

PID Gd
r e u y
Spt PID Controller Terminator
Unknown plant

Figure 6.63: RLS estimation under closed loop conditions in S IMULINK. See also Fig. 6.48.

6.12 S UMMARY

System identification is where we try to regress or fit parameters of a given model skeleton or
structure to fit input/output data from a dynamic process. Ideally we would obtain not only
the values of the parameters in question, but also an indication of the goodness of fit, and the
appropriateness of the model structure to which the parameters are fitted. Identification method-
ologies usually rely on a trial & error approach, where different model structures are fitted, and
the results compared. The S YSTEM I DENTIFICATION T OOLBOX within M ATLAB, [127], is a good
source of tools for experimentation in this area, because the model fitting and analysis can be
done so quickly and competently.

Identification can also be attempted online, where the current parameter estimate is improved or
updated as new process data becomes available in an efficient way without the need to store all
the previous historical data. We do this by re-writing the least-squares update in a recursive algo-
rithm. Updating the parameters in this way enables one to follow time-varying models or even
nonlinear models more accurately. This philosophy of online estimation of the process model is a
central component of an adaptive control scheme which is introduced in chapter 7.

However there are two key problems with the vanilla recursive least-squares algorithm which
become especially apparent when we start to combine identification with adaptive control. The
first is the need to prevent the covariance matrix P from being illconditioned. This can, in part be
solved by ensuring persistent excitability in input. The second problem is that given non-white
noise or coloured noise, the RLS scheme will deliver biased estimates. However, there are many
extensions to the standard schemes that address these and other identification issues. Succinct
summaries of system identification with some good examples are given in [23, pp422431] and
[152, p856], while dedicated texts include [126, 192, 205].
316 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

P ROBLEMS

Problem 6.4 An identification benchmark problem from [109].

1. Simulate the second-order model with external disturbance vt ,


2
X 2
X 2
X
yt = ai yti + bi uti + di vti + e(t), e(t) N (0, 2 ) (6.81)
i=1 i=0 i=1

where the parameter values are:

a1 a2 b0 b1 b2 d1 d2
0.98 0.9 0.5 0.25 0.1 0.8 0.2 0.1

The input ut is normally distributed discrete white noise, and the external disturbance is to
be simulated as a rectangular signal alternating periodically between the values +1 and 1
at t = 100, 200, . . . Run the simulation for 600 time steps, and at t = 300, change a1 to 0.98.
2. This is a challenging identification problem, because the rarely varying external disturbance
signal gives little information about the parameters d. Identify the parameters in Eqn. 6.81
perhaps starting with 1|0 = 0, P1|0 = 50I and using an exponential forgetting factor, = 0.8.

Problem 6.5 1. Run through the demos contained in the S YSTEM I DENTIFICATION toolbox for
M ATLAB.
2. Investigate the different ways to simulate a discrete plant with a disturbing input. Use the
model structure
A(q 1 )yt = B(q 1 )ut + C(q 1 )et
Choose a representative stable system, say something like

A(q 1 ) = 1 0.8q 1 , B(q 1 ) = 0.5, C(q 1 ) = 1 + 0.2q 1

and choose some sensible inputs for ut and et . Simulate the following and explain any
differences you find.
(a) Write your own finite difference equation in M ATLAB, and compare this using filter.
(b) Use the transfer function object, tf, in MISO (multiple input/single output) mode.
Hint: Use cell arrays as
>>G = tf( {B ,C} , {A,A},T); % discrete version
>>G.variable = q;
(c) Use idpoly and idmodel/sim from the S YSTEM I DENTIFICATION toolbox.
(d) Build a discrete S IMULINK model
Hint: You may find problems due to the differing lengths of the polynomials. Either ensure
all polynomials are the same length, or swap from z 1 to z, or introduce delays.
3. Repeat problem 2 with an unstable model (A is unstable.)
4. What is the difference in S IMULINK when you place the A polynomial together with the B
and C polynomials? Particularly pay attention to the case where A is unstable.
5. Identification of plants that include coloured noise inputs is considerably more complex than
just using arx. The following illustrates this.

(a) Create a truth plant with polynomials, A, B, C. (Use idpoly).


6.12. SUMMARY 317

(b) Simulate this plant using idmodel/sim for the following two cases:
i. A noise-free version: i.e. random u(t) and no e(t)
ii. A version with coloured noise: i.e. random u(t), e(t).
(c) Estimate an ARX model (i.e. only A, B polynomials) using both data sets. Present
the model with uncertainty bounds on the estimated parameters using present. What
was the problem with the coloured noise version and how can we avoid it?
(d) Estimate the A, B and C polynomials using an extended-LS scheme such as armax
or pem. Has the identification improved significantly? How did the computational time
change from using to arx to armax?

6. The D AISY (Data Base for the Identification of Systems), [139], located at www.esat.kuleuven.ac.be/si
and contains a number of simulated and experimental multivariable data sets. Down load
one of these data sets, preferably one of the real industrial data trends, and make an identi-
fication.
7. RLS with modified forgetting factors
Construct a simulation to estimate the parameters in a dynamic model (as in tident.m) to
test various extensions to the forgetting factor idea. Create a simulation with output noise,
and step change the model parameters part way through the run. (For more information on
these various extensions, see [205, pp140160].)
(a) Use a start-up forgetting factor of the form
  
t
1 (t) = 0 + (1 0 ) 1 exp (6.82)
f
where 0 is the initial forgetting factor, (say 0.90.95), and f is the time constant that
determines how fast (t) approaches 1.0.
Note that Eqn. 6.82 can be re-written in recursive form as
1 (t) = 1 (t 1) + (1 )
where = e1/f and 1 (0) = 0 .
(b) Combine with the start-up forgetting factor, an adaptive forgetting factor of the form
 
f 2 (t)
2 (t) = 1 (6.83)
f 1 f sf (t)
where sf (t) is a weighted average of past values of the squared error 2 .
[205, p159] suggest using the filter
f 1 2
sf (t) = sf (t 1) +
f f

Combine both the start-up and the adaptive forgetting factor to get a varying forgetting
factor (t) = 1 (t)2 (t).
(c) A directional forgetting factor tries to update in only those directions in parameter space
where there is information available. The covariance update with directional forgetting
is " #
xt x
t Pt1
Pt = Pt1 I 1 (6.84)
rt1 + x t Pt1 xt

where the directional forgetting factor r(t) is


1
rt = (6.85)
x
t+1 Pt xt+1

and the scalar is like the original fixed forgetting factor.


318 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

What ever happened to elegant solutions?

8. Compare the operation count of the RLS scheme with square root updates with the nor-
mal update. Construct a sufficiently ill-conditioned example where you can demonstrate a
significant difference.
Summary of the RLS square root update scheme for y = T .
Define the square root of P as
def
P = SST
Update St by following

f t = STt1 t
t = 1 + f T f
1
t =
t + t
Lt = St1 f t
St = St1 t Lt f Tt

Update the parameter estimates using

Lt 
t+1 = t + yt T t
t

9. Modify the square root RLS scheme to incorporate a forgetting factor.

10. Construct an RLS update algorithm in S IMULINK, perhaps using the matrix/vector blocks in
the DSP blockset. Use the following update equations:
" #
Pk k+1 k+1 Tk+1 Pk
Kk+1 = , Pk+1 = Pk I
1 + Tk+1 Pk k+1 1 + Tk+1 Pk k+1

k+1 = k + Kk+1 yk+1 Tk+1 k

Note that as of Matlab R12, there is considerable support for vector/matrix manipulations in
raw S IMULINK.
6.12. SUMMARY 319

11. Construct a square root update algorithm (from Problem 8) in S IMULINK, perhaps using the
matrix/vector blocks in the DSP blockset.
12. Wellstead & Zarrop, [205, p103], develop an expression for the bias as a function of the
model parameters for a simple 1-parameter model
yt = ayt1 + et + cet1 , |a| < 1, |c| < 1
where et is white noise with variance e2 , as
c(1 a2 )
a a
1 + c2 + 2ac
(a) Validate with simulation the expression for the size of the bias, a a.
(b) The size of the bias should not be a function of the size of the variance of et . Is this a
surprising result? Verify this by simulation. What happens if you drop the variance of
the noise, e2 , to zero? Do you still observe any bias? Should you?
13. Starting with the S IMULINK demo rlsest which demonstrates an adaptive pole-placement
of a discrete plant do the following tasks.

(a) Modify the rlsest simulation to use a continuous plant of your choice and, for the mo-
ment, a fixed (non-adaptive) PID controller. (I.e. disconnect the closed-loop adaption
part of the controller.) Use Fig. 6.64 as a guide to what you should construct.

theta
To Workspace1

RLS
Estimator

parameters Recursive least squares


Parameter Estimator
State
phi Generator
t
Clock1 time
yr
To Workspace
ZOH ZOH1

Mux

spt/plant
+ u[n] 3(s+0.3) y[n]
PID +
(s+2)(s+1)(s+0.4) +
Setpoint
PID Controller Sum1
plant

distrurbance

Figure 6.64: A modified version of rlsest in Simulink.

Explain in detail how the state generator works and how the RLS estimator works.
Modify the blocks so that you can identify a different number of A terms from the B
terms, (such as say two B(q 1 ) and three A(q 1 ) polynomials terms).
(b) Run the estimation routine under closed-loop conditions. What do you observe espe-
cially as you change the noise level? (To be realistic, you should add at least one more
noise source, where ?)
320 CHAPTER 6. IDENTIFICATION OF DYNAMIC MODELS

(c) Modify the RLS estimator s-function to export the covariance matrix as well as the
updated parameters. Do these confirm your findings from part 13b?
(d) Modify your RLS scheme to better handle noise and the closed loop conditions. You
may like to use some simple heuristics based on the trace of the covariance matrix, or
use a directional and/or variable forgetting factor.

14. Read the M ATLAB Technical support note #1811 How Do I Create Time Varying Matrix Gain
and State-space Blocks as C MEX S-functions?. Download the relevant files and implement
an adaptive controller.
15. Model identification in state-space with state information.
The English mathematician Richardson has proposed the following simple model for an
arms race between two countries:

xk+1 = axk + byk + f (6.86)


yk+1 = cxk + dyk + g (6.87)

where xk and yk are the expenditures on arms of the two nations and a, b, c, d, f and g
are constants. The following data have been compiled by the Swedish International Peace
Research Institute, (SIPRI yearbook 1982).

Millions of U.S. dollars at 1979 prices and 1979 exchange rates


year Iran Iraq NATO WTO
1972 2891 909 216478 112893
1973 3982 1123 211146 115020
1974 8801 2210 212267 117169
1975 11230 2247 210525 119612
1976 12178 2204 205717 121461
1977 9867 2303 212009 123561
1978 9165 2179 215988 125498
1979 5080 2675 218561 127185
1980 4040 225411 129000
1981 233957 131595

(a) Determine the parameters of the model and investigate the stability of this model.
(b) Determine the estimate of the parameters based on 3 consecutive years. Look at the
variability of the estimates.
(c) Create a recursive estimate of the parameters. Start with 1975 with the initial values
determined from 197274.
C HAPTER 7

A DAPTIVE C ONTROL

Adopt, adapt and improve . . . Motto of the round table.


John Cleese whilst robbing a lingerie boutique.

7.1 W HY ADAPT ?

Since no one yet has found the holy grail of controllers that will work for all conceivable oc-
casions, we must not only select an appropriate class of controller, but also be faced with the
unpleasant task of tuning it to operate satisfactorily in our particular environment. This job is
typically done only sporadically in the life of any given control loop, but in many cases owing to
equipment or operating modifications, the dynamics of the plant have changed so as to render to
controller sub-optimal. Ideally the necessary re-tuning could be automated, and this is the idea
behind an adaptive controller.

In a control context, adaption is where we continually adjust the controller parameters and pos-
sibly structure in order to deliver a consistent closed loop performance despite changing process
conditions.

The problem is that for tight control we essentially want to have a high feedback gain, giving
a rapid response and good correction with minimal offset. However if the process dynamics
subsequently change due to environmental changes for example, the high feedback gain may
create an unstable controller. To prevent this problem, we adapt the controller design as the
process conditions change, to achieve a consistent performance.

Adaptive control was first used in the aerospace industry around the late 1950s where automatic
pilots adapted the control scheme depending on external factors such as altitude, speed, payload
and so forth. One classic example was the adaptive flight control system (AFCS) for the X-15 in
the late 1960s described in [8]. The adaptive control system was developed by Honeywell and
was fundamentally a gain scheduler that adapted the control loop gains during the flights that
ranged from sea level to space at 108 km. Unfortunately the crash of X-15-3 and loss of pilot
Michael Adams in 1967 somewhat terminated the enthusiasm of adaptive flight control, and the
research program was terminated shortly afterwards.

In the past couple of decades however, adaptive control has spread to the processing industries
with commercial industrial adaptive controllers now available. Some applications where adap-
tive control is now used are:
321
322 CHAPTER 7. ADAPTIVE CONTROL

Aeronautical control As the behaviour of high performance fighter aircraft depends on altitude,
payload, speed etc, an adaptive control is vital for the flight to be feasible.

Tanker steering The dynamics of oil supertankers change drastically depending on the depth of
water, especially in relatively shallow and constricted water-ways.

Chemical reactions Chemical processes that involve catalytic reactions will vary as the catalyst
denatures over time. Often coupled to this, is that the heat transfer will drop as the tubes in
the heat exchanger gradually foul.

Medical The anesthetist must control the drug delivery to the patient undergoing an operation,
avoiding the onsets of either death or consciousness.1

Some further examples of adaptive control are given in a survey paper in [184].

Adaptive control is suitable: Adaptive control is not suitable if:


if the process has slowly varying parameters PID control is already satisfactory
if clean measurements are available The system output is essentially constant
for batch processes or startup/shut down of The system is essentially linear and time invari-
continuous processes ant
for widely changing operating conditions (such If the system is nonlinear, but easy to model
as in pH controlled processes)

7.1.1 T HE ADAPTION SCHEME

When faced with tuning a new control loop, the human operator would probably do the following
steps:

1. Disturb the plant (slightly) and collect the resulting input/output data.

2. Use this data to estimate the plant dynamics.

3. Using the fitted model, an appropriate controller can be designed.

4. Update the controller with the new parameters and/or structure and test it.

If the scheme outlined above is automated, (or you find an operator who is cheap and does not
easily get bored), we can return to step 1, and repeat the entire process at regular intervals as
shown in Fig. 7.1. This is particularly advantageous in situations where process conditions can
change over time. The process and feedback controller are as normal, but we have added a new
module in software (dashed box in Fig. 7.1) consisting of an on-line identification stage to extract a
process model M(), and given this model information, a controller design stage to find suitable
controller tuning constants.

Of course there will be some constraints if we were to regularly tune our controllers. For example,
it is advisable to do the the estimation continuously in a recursive manner, and not perturb the
plant very much. We could adapt the three tuning constants of a standard PID controller, or we
could choose a controller structure that is more flexible such as pole-placement. The adaption
module could operate continuously or it could be configured to adapt only on demand. In the
latter case, once the tuning parameters have converged, the adaption component of the controller
turns itself off automatically, and the process is now controlled by a constant-tuning parameter
controller such as a standard PID controller.
1 One particularly ambitious application is described in [54, p429].
7.1. WHY ADAPT? 323

Adaptive Module

identification
control Control design Identify M() parameters
specifications
na , nb , ,

Measure
input & output

setpoint, r(t)
Controller b
Plant, S b
output, y(t)
u(t) y(t)

feedback loop

Figure 7.1: The structure of an indirect adaptive controller. (See also Fig. 7.5.)

7.1.2 C LASSIFICATION OF ADAPTIVE CONTROLLERS

As one may expect, there are many different types of adaptive control, all possessing the same
general components, and an almost equal number of classification schemes. To get an overview
of all these similar, but subtly different controllers, we can classify them according to what we
adapt;

Parametric adaption Here we continually update the parameters of the process model, or even
controller, or
Signal adaption where we vary a supplementary signal to be added directly to the output of the
static controller.

Alternatively we can classify them as to how we choose a performance measure,

Static A static performance measure is one such as overall efficiency, or maximising production.
Dynamic A dynamic performance measure is to optimise the shape of a response to a step test.
Functional of states and inputs This more general performance index is discussed more gener-
ally in chapter 9.

7.1.3 A SIMPLE EXAMPLE OF ADAPTION : G AIN SCHEDULING

The simplest type of adaptive control is gain scheduling. In this scheme, one tries to maintain
a constant overall loop gain, K by changing the controller gain, Kc in response to a changing
process gain Kp . Thus the product Kc Kp is always constant.

An example where gain scheduling is used is in the control of a liquid level tank that has a varying
cross sectional area. Suppose we are trying to control the level in a spherical tank by adjusting a
324 CHAPTER 7. ADAPTIVE CONTROL

valve on the outlet. The exiting flow is proportional to the square root of height of liquid above
the valve, which in turn is a nonlinear function of the volume in the tank. Fig. 7.2 shows how
this gain varies as a function of material in the tank, illustrating why this is a challenging control
problem. No single constant controller setting is optimal for both operating in the middle and the
extremes of the tank.

1.5

Plant gain K h1/2


1

p
0.5

V
max
0
8

c
Controller gain K 6

4
Figure 7.2: The plant gain, Kp is pro-
portional to the square toot of the hy-
2
draulic head which in turn is a non-
linear function of volume in the tank.
0
Consequently the controller gain, Kc , 0 1 2 3 4 5
should be the inverse of the plant gain. Volume Vmax

A way around this problem is to make the controller parameter settings a function of the level
(the measured variable, y). Here the controller first measures the level, then it decides what
controller constants (Kc , i , d ) to use based on the level, and then finally takes corrective action
if necessary. Because the controller constants change with position, the system is nonlinear and
theoretical analysis of the control system is more complicated.

To do this in practice one must:

1. Determine the open loop gain at the desired operating point and several points either side
of the desired operating point.

2. Establish good control settings at each of these three operating points. (This can be done by
any of the usual tuning methods such as Ziegler-Nichols, Cohen-Coon etc)

3. Generate a simple function (usually piecewise linear) that relates the controller settings to
the measured variable (level).

The gain scheduling described above is a form of open-loop or feedforward adaption. The pa-
rameters are changed according to a pre-set programme. If the tank is in fact different from the
tank that was used to calculate the parameter adaption functions such as having a cross section,
then the controller will not function properly. This is because the controller is not comparing its
own system model with the true system model at any time. However despite this drawback, gain
scheduling is an appropriate way to control things like tanks of varying cross sectional area. It is
unlikely that the tanks cross sectional area will change significantly with time, so the instrument
engineer can be quite confident that the process once tuned, will remain in tune.
7.1. WHY ADAPT? 325

7.1.4 I DENTIFY AND THEN CONTROL

The adaptive controller (with feedback adaption) must be aware of the process, or more accurately
it must be aware of changes in process behaviour hence the importance of system identification
covered in chapter 6. Central to the concept of controllers that adapt, is an identification algorithm
that constantly monitors the process trying to establish if any significant changes have occurred.
In the gain schedular given in section 7.1.3, the identification component was trivial, it was simply
the level reading. The purists will argue that the gain schedular is not a true adaptive controller,
because if the dynamics of the system change (such as the cross section of the tank suddenly
varies), the controller is unaware of this change, and cannot take appropriate corrective action.

There are two popular adaptive control techniques used currently in industry. The first approach
is to use a controller with auto-tuning. Normally these controllers are operating in just the same
manner as a traditional PID controller, but if the operator requests, the controller can be tuned
automatically, perhaps using the techniques described in section 4.6.2. This is sometimes called
tuning on demand. Once the auto-tuning part is complete, the tuning procedure switches itself
off and the controller reverts back to the traditional PID mode. These types of adaptive con-
trollers are used where adaptive control is required, but the loop may be too sensitive or critical
to have constant adaptation.

The second approach to adaptive control is called true adaptive control and is the focus of this
chapter. This is where the controller will adapt itself to the changing process without operator
demand. Unlike the auto-tuners, the true adaptive control remains continuously identifying
the process (using variants of the least-squares identification that was described in section 6.7),
and then subsequently updating the controller parameters using techniques that will be described
in section 7.3.

Industrially auto-tuners have been successful and are now accepted in the processing industries.
Most of the major distributed computer control companies provide an adaptive control in the
form of a self-tuner as part of their product (Satt Control, Bailey, ABB, Leeds-Northrup, Foxboro).
Astrom and Hagglund describe some of these auto-tuning commercial products in chapter 5 of
[16, 105132].

However the true adaptive control has not been accepted as much as the self-tuner. This is due to
the fact that true adaptive control requires a higher level of expertise than self-tuning regulators.
In addition, because the adaptive control is continuous, it requires a much larger safety network
to guarantee correct operation. Even though the adaptive controller does not require tuning,
the correct operation still requires estimates of values before the estimator can correctly function.
These types of parameters are things such as expected model order, sampling period and expected
dead time. If these values are significantly different from the true values, then poor adaption and
hence control will result. Section 7.2 will consider self-tuning regulators.

P OLYNOMIAL MANIPULATIONS

The controller design strategies in this section will use the so-called polynomial approach. In this
design technique, we will need to be able to add and multiply polynomials, of possibly differing
lengths. The two routines given in Appendix B on page 509 add and multiply polynomials and
are used in the following controller design scripts in this chapter. An alternative programming
methodology is to create a polynomial object and then overload the plus and times operators.
326 CHAPTER 7. ADAPTIVE CONTROL

7.2 S ELF TUNING REGULATORS (STR S )

A better adaptive controller than a gain schedular is one where the controller parameters are
continually updated as the process characteristics change with time in an undetermined manner.
These are called self tuning regulators and an early application was in the pulp and paper indus-
try controlling a paper machine cited in [10]. Further development culminating in the self tuning
regulator is described in the classic reference, [13]. Self tuning regulators are more flexible than
gain scheduling controllers, which to be effective, the relation between the input and output must
be known beforehand and is assumed constant. A textbook covering many aspects of the design
and use of self tuning regulators is [205].

The basic elements of a self tuning regulator are:

1. Identify the model using input-output data. Usually the model is estimated online by using
a recursive least squares parameter estimation.

2. A relation between the model parameters estimated in step 1 and the controller parameters.

3. The control algorithm. Many different types of control algorithms exist, but many are func-
tions that try to adjust the controller parameters, given the continually changing process
parameters in such a way that the closed loop response is constant. Because the controller
is being continuously updated to the new varying conditions, these control algorithms can
be quite vigorous.

Step 1, the identification phase, was covered in chapter 6. Steps 2 and 3 are the controller tuning
and the controller structure sections and are discussed further in this chapter.

Suppose we have previously identified our plant under test using an ARX model of the form

A(q)yk = B(q)uk + ek (7.1)

then it is natural to ask what should the control law be? The simplest control law is a direct model
inversion. We choose u such that y = ek which is the best possible result given the unknown noise.
This results in the control law
A(q)
uk = yk (7.2)
B(q)

which is called minimum variance control, [10, 13, 15]. Expanding this control law gives the
manipulated input at sample time k as

1
uk = (y + a1 yk1 + a2 yk2 + + an ykn b1 uk1 bm ukm ) (7.3)
b0 k

where yk is the setpoint.

Clearly for this type of controller to be realisable, the b0 term in Eqn. 7.3 must be non-zero, or
in other words, the plant must have no dead time. With that constraint, this means that we can
calculate the present control action u that will drive y to the desired y .

Even without deadtime however, this type of control law will exhibit extremely vigorous action in
an attempt to bring the process back to the setpoint in a single sample time. While the simulated
results look very impressive, (see for example Fig. 7.3), in practice we will need to alter the struc-
ture of the control law, relax it somewhat, to account for imperfect modelling and manipulated
variable constraints. These problems are partly circumvented in 7.4.4 and summarised in 7.5.
7.2. SELF TUNING REGULATORS (STRS) 327

7.2.1 S IMPLE MINIMUM VARIANCE CONTROL

The algorithm to construct an adaptive minimum variance controller consists of two parts: the
identification of the model parameters, , in this instance using recursive least-squares, and the
implementation of the subsequent minimum variance control law.

Algorithm 7.1 Simple minimum variance control for plants with no delay

1. Measure the current output yt .

2. Form the new input/output data matrix X.

3. Calculate the prediction yt from and t1 .

4. Apply the control law, Eqn. 7.3, to obtain the current input, ut , using the estimated parame-
ters.

5. Update the estimated parameters k using a recursive least-squares algorithm such as Al-
gorithm 6.2 or equivalent.

A simple minimum variance controller is demonstrated in Listing 7.1 To test the simulation, we
will construct a siumulation where we abruptly swap between two different plants:

0.2 + 0.05q 1
G1 (q 1 ) = , for 0 t < 100 (7.4)
1 0.78q 1 0.65q 2 + 0.2q 3
0.3 + 0.15q 1
G2 (q 1 ) = , for t > 100 (7.5)
1 0.68q 1 0.55q 2 + 0.1q 3

For this simulation we will assume no model/plant structural mismatch and we will use the
recursive estimation with the forgetting factor option.

Listing 7.1: Simple minimum variance control where the plant has no time delay
t = [0:200]'; yspt = square(t/18); % Create time & setpoint
2 y=zeros(size(yspt)); u=zeros(size(y));
nr=0.02; % noise or dither sigma

n1 = [0.2 0.05]; d1 = [1 -0.78 -0.65 0.2]; % system #1, Eqn. 7.4


n2 = [0.3 0.15]; d2 = [1 -0.68 -0.55 0.1]; % system #2, Eqn. 7.5, for t > 100
7 thetatrue = [ones(t(1:100))*[d1 n1]; ones(t(101:length(t)))*[d2 n2]];

na = 4;nb = 2; dead=0; nt=na+nb; % Estimated model parameter orders, na , nb & nk .


lam = 0.95; P=1000*eye(nt); % Identification parameters: , P0
thetaest=rand(nt,1); % random start guess for 0
12 Param=rand(na,nt);Pv = ones(na,1)*diag(P)';

for i=na+1:length(yspt)
theta = thetatrue(i,:)'; % Actual plant, G
thetar=[thetaest(1:na); thetaest(na+2:nt)]; % Estimated plant, G
17 b0 = thetaest(na+1);
yv=y(i-1:-1:i-na); uv=u(i-1:-1:i-nb+1); % old output & input data
u(i)=(yspt(i)-[yv; uv]'*thetar)/b0; % Min. Var. control law

c = [yv', u(i:-1:i-nb+1)']; % Now do identification


22 y(i) = c*theta(2:end) + 0.01*randn(1)*nr; % true process with noise
328 CHAPTER 7. ADAPTIVE CONTROL

[thetaest,P] = rls(y(i),c',thetaest,P,lam); % RLS with forgetting


Param = [Param; thetaest']; Pv = [Pv;diag(P)'];
end % for loop

Initially we will set the forgetting factor to 0.99, and we will introduce a small amount of noise
so the estimation remains reasonable even under closed loop conditions, nr=0.02. We will aim
to follow the trajectory given as the dashed line in the upper figure of Fig. 7.3.

Our minimum variance controller will both try to estimate the true plant and control the process
to the setpoint, y . The lower trend in Fig. 7.3 shows the parameter values for the estimated
model.

Minimum variance control


4
output & setpoint

G1 (q 1 ) G2 (q 1 )
2

2
20

10
input

10

20
= 0.99
1

1
0 50 100 150 200 250 300
time

Figure 7.3: Simple minimum variance control with an abrupt plant change at t = 100. (See also
Fig. 7.4.)

In this unrealistically ideal situation with no model/plant mismatch, the controlled response
looks practically perfect, despite the plant change at t = 100 (after of course the converging pe-
riod). Fig. 7.4 shows an enlarged portion of Fig. 7.3 around the time of a setpoint change. Note
how the response reaches the setpoint in one sample time, and stays there. This is as expected
when using minimum variance control since we have calculated the input to force y exactly equal
to the desired output y in one shot. In practice this is only possible for delay-free plants with no
model-plant mismatch.

Of course under industrial conditions (as opposed to simulation), we can only view the controlled
result, and perhaps view the trends of the estimated parameters, we dont know if they have
converged to the true plant parameters. However if the estimates converge at all to steady values,
and the diagonal elements of the covariance matrix are at reasonable values, (these are application
dependent), then one may be confident, but never totally sure, that a reasonable model has been
identified.
7.3. ADAPTIVE POLE-PLACEMENT 329

Minimum variance control

1
output & setpoint

0.5

0.5

10

0
Figure 7.4: A zoomed portion of Fig. 7.3
input

showing the near-perfect response around


10
a setpoint change when using simple min-
20
35 40 45
imum variance control on a plant with no
time delay.

7.3 A DAPTIVE POLE - PLACEMENT

Clearly the problem with the simple minimum variance control is the excessive manipulated
variable demands imposed by the controller in an attempt to reach the setpoint in one sample
time. While our aim is to increase the speed of the open-loop response, we also need to reduce
the sensitivity to external disturbances. Both these aims should be addressed despite changes in
the process.

Fig. 7.5 shows a generalised control structure given in shift polynomial form known as a two
degree of freedom polynomial controller. In this section we are going to modify our colour scheme
slightly. As shown in the figure, the controller polynomials will be coloured blue hues, and the
plant to be controlled, red hues. The B polynomial, because it operates on the blue input is
coloured purple, i.e. somewhere between blue and red.

Controller

pre-compensator Plant

1 u(t) B(q 1 )
r(t) H(q 1 ) + b
y(t)
F (q 1 ) A(q 1 )

setpoint

G(q 1 )

Figure 7.5: Two degree of freedom pole-placement control structure

If the controller polynomials G = H = 1, then we have the classic controlled feedback loop, other-
wise we have a slightly more general configuration known as a two-degree of freedom configura-
tion since the controller (dashed box in Fig. 7.5) has two inputs. Our aim, as controller designers,
is to select polynomials F , G, H given process polynomials B, A such that our performance aims
330 CHAPTER 7. ADAPTIVE CONTROL

are met. The closed loop transfer function of Fig. 7.5 is


HB
y(t) = r(t)
F A + GB
where clearly the stability is given by the denominator, F A + GB. Note that this polynomial
equation consists of two terms, each with a blue term multiplied by a red term. This fact will be
important later when we derive an algorithm to solve for the unknown blue polynomials.

Rather than demand the one-step transient response used in the minimum variance case, we can
relax this criterion, and ask only that the controlled response follow some pre-defined trajectory.

Suppose that we restrict the class of dynamic systems to the form,


Ay(t) = Bu(t 1) (7.6)
where we have assumed a one sample time delay between u and y. This is to leave us enough
time to perform any controller calculations necessary. We will use the controller structure given
in Fig. 7.5,
F u(t) = Hr(t) Gy(t) (7.7)
where the polynomials F, G, H are of the form:
F = 1 + f1 q 1 + f2 q 2 + + fnf q nf
G = g0 + g1 q 1 + g2 q 2 + + gng q ng
H = h0 + h1 q 1 + h2 q 2 + + hnh q nh
With the computational delay, the closed-loop is
(F A + q 1 BG)y(t) = q 1 BHr(t) (7.8)
and we desire the closed-loop poles to be assigned to specified locations specified in the polyno-
mial T ,
T = 1 + t1 q 1 + t2 q 2 + + tnt q nt (7.9)
where we have coloured the T polynomial a reddish pink because being a setpoint, it is associated
with the red output. Assigning the closed-loop poles to given locations is called pole-placement
or pole assignment. In practice, the closed loop polynomial T will be a design requirement given
to us perhaps by the customer or client.

Note that some authors such as [22] use a slightly different standard nomenclature convention for
the controller polynomials. Rather than use the variable names F, G, H in the control law Eqn. 7.7,
the variable names R, S, T are used giving a control law of Ru(t) = T r(t) Sy(t).

7.3.1 T HE D IOPHANTINE EQUATION AND THE CLOSED LOOP

To assign our poles specified in T , we must select our controller design polynomials F, G such
that the denominator of the actual closed-loop is what we desire,
F A + q 1 BG = T (7.10)
The polynomial equation Eqn. 7.10 is called the Diophantine equation, and may have many solu-
tions. To obtain a unique solution, the polynomials A, B must be coprime (no common factors),
and we should select the order of the polynomials such that
nf = nb
ng = na 1, (na 6= 0)
nt na + nb nc
7.3. ADAPTIVE POLE-PLACEMENT 331

One way to establish the coefficients of the design polynomials F, G and thus solve the Diophan-
tine equation is by expanding Eqn. 7.10 and equating coefficients in q i giving a system of linear
equations.
1 0 0 b0 0 0

.. f1

.. ..
a1
1 . b1 b0 . .

t1 a 1
.. .. .. .. f2 t2 a 2
a
2 a1 . . . b1 .
.

..


. .
.. . .. .. .
a2 . .

.
bnb . .
fnf

tnt ant

.. . .
= ant+1
(7.11)
a
na . . 1 0 bnb 0
g0 ..



. . .
ana . . a1 ..

0 0 b0 g1 ana

. .. .. 0
.. . a2 .
0 b1 ..


.
..
.
. ..
.. .
. . . gng | {z }
b
0 ana 0 bnb | {z }
| {z } c
A
The matrix A in Eqn. 7.11 has a special structure, and is termed a Sylvester matrix and will be
invertible if polynomials A, B are co-prime. Solving Eqn. 7.11

c = A1 b

gives us the coefficients of our controller polynomials, F , G. Now the dynamics of the closed-loop
requirement are satisfied,
BH
y(t) = r(t 1)
T
but not the steady-state gain, which ideally should be 1. We could either set H to be the scalar,

T
H= (7.12)
B q=1

which forces the steady-state gain to 1, or preferably cancel the zeros


1
H= (7.13)
B
which improves the transient response in addition to keeping the steady-state gain of unity. Can-
celling the zeros should only be attempted if all the zeros of B are well inside the unit circle.
Section 7.4.1 describes what to do if the polynomial B does has poorly damped or unstable zeros.

This scheme has now achieved our aim of obtaining a consistent dynamic response irrespective
of the actual process under consideration. Since we cannot expect to know the exact model B, A
we will use an estimated model, B, A in the controller design.

7.3.2 S OLVING THE D IOPHANTINE EQUATION IN M ATLAB

Since solving the Diophantine equation is an important part of the adaptive pole-placement reg-
ulator, the following functions will do just that. Listing 7.2 solves for the polynomials F and G
such that F A + BG = T . If a single time delay is introduced into the B polynomial such as in
Eqn. 7.10, then one needs simply to pad the B polynomial with a single zero. Note that the ex-
pression AF + BG is optionally computed using the polyadd routine given in Listing B.1, and
this polynomial should be close to the T polynomial specified.
332 CHAPTER 7. ADAPTIVE CONTROL

Listing 7.2: A Diophantine routine to solve F A + BG = T for the polynomials F and G.


function [F,G,Tcheck] = dioph(A,B,T);
% [F,G] = dioph(A,B,T);
% Solves the Diophantine equation by setting up the Sylvester matrix,
% AF + BG = T and subsequently solves for polynomials F & G.
5 % See [19, p175] (follows nomenclature of except R=F, S=G).

if abs(A(1)-1) > 1.0e-8;


error('Error: Polynomial A not monic A(1) = %g\n',A(1));
end % if
10

na = length(A)-1; nb = length(B)-1; % Order of polynomials A(q 1 ), B(q 1 )


nf = nb; ng = na-1; n = nf+ng+1; n = nf+ng+1; % Required orders for F & G
D = zeros(n,n); % construct Sylvester matrix
for i=1:nf
15 D(i:i+na,i) = A'; % fill in columns
end % for i
for i=1:ng+1
D(i:i+nb,nf+i) = B'; % fill in columns
end % for i
20 rhs = [A(2:na+1), zeros(size(na+1:n))]'; % right-hand side
rhs = [T(2:length(T)), zeros(size(length(T):n))]' - rhs;

FG= D\rhs; % Find solution for polynomials F & G


F= FG(1:nf)'; G= FG(nf+1:n)';% Monic polynomial F has leading 1 dropped.
25

if nargout >2 % then check that AF + BG = T as required


Tcheck = polyadd(conv(A,[1,F]), conv(B,G));
end
return

A N ALTERNATIVE D IOPHANTINE SCHEME

An alternative, and somewhat more elegant, scheme for solving the Diophantine equation is
given in Listing 7.3 which uses matrix convolution to construct the Sylvester matrix.

Listing 7.3: Alternative Diophantine routine to solve F A + BG = T for the polynomials F and G.
Compare with Listing 7.2.
1 function [F,G,Tcheck] = dioph_mtx(A,B,T);
% Alternative version for polynomial Diophantine equation solving
% Solves AF + BG = T for F & G.

da = length(A)-1; db=length(B)-1; % da = deg(A) etc.


6 T = [T, zeros(1,da+db-length(T)+1)]; % pad with zeros
dt=length(T)-1; % Convert to full length
dg =da-1; df=dt-da ; % assuming for RAB to be square
B = [ zeros(1, df-db+1), B]; % pad with leading zeros

11 Rac = [convmtx(A,df+1);convmtx(B, dg+1)]; % Construct RAB = [RA ; RB ]

FG = T/Rac; % [F, G] = T R1
AB
F = FG(1:df+1); G = FG(df+2:df+dg+2);% Note F & G are row vectors.

16 Tcheck = polyadd(conv(A,F), conv(B,G)); % Verify solution


return
7.3. ADAPTIVE POLE-PLACEMENT 333

The alternative scheme in Listing 7.3 is slightly different from the one presented earlier in that
this time the F polynomial contains the leading unity coefficient. A third method to construct
the Sylvester matrix is to use the lower triangle of a Toeplitz matrix in M ATLAB say with the
command tril(toeplitz(1:5)).

E XAMPLE OF A D IOPHANTINE PROBLEM

Suppose we only want to place one pole, T = 1 + t1 q 1 , given a na = 3, nb = 2 model. The


recommended orders of the controller polynomials are

nf = nb = 2
ng = na 1 = 2
nt = 1 na + nb nc = 5

and expanding the closed-loop equation Eqn. 7.10,

(1 + f1 q 1 + f2 q 2 )(1 + a1 q 1 + a2 q 2 + a3 q 3 )+

q 1 (b0 + b1 q 1 + b2 q 2 )(g0 + g1 q 1 + g2 q 2 ) = 1 + t1 q 1
which by equating the coefficients in q i , i = 1, . . . , 5 gives the linear system following Eqn. 7.11

1 0 b0 0 0 f1 t1 a 1
a1 1 b1 b0 0 f2 a2

a2 a1 b2 b1 b0 g0 = a3

a3 a2 0 b 2 b 1 g 1 0
0 a3 0 0 b 2 g2 0

A N EXAMPLE OF SOLVING THE D IOPHANTINE EQUATION

In this example we start with some known polynomials, and then try to reconstruct them using
the Diophantine routines presented previously.

Suppose we start with some given plant polynomials, say

A(q 1 ) = 1 + 2q 1 + 3q 2 + 3q 3 , B(q 1 ) = 5 + 6q 1 + 7q 2

where the orders are na = 3 and nb = 2. This means we should choose orders nf = nb = 2, and
ng = na 1 = 2, so let us invent some controller polynomials

F (q 1 ) = 1 + 8q 1 + 9q 2 , G(q 1 ) = 10 + 11q 1 + 12q 2

noting the A and F are monic. We can now compute the closed loop, AF + BG using convolution
and polynomial addition as shown in Listing 7.4

Listing 7.4: Constructing polynomials for the Diophantine equation example


>> A = [1 2 3 4]; B = [5 6 7];
>> F = [1 8 9]; G = [10 11 12]; % Note nf = nb & ng = na 1.
3

>> T = polyadd(conv(A,F),conv(B,G)) % Compute T (q 1 ) = A(q 1 )F (q 1 ) + B(q 1 )G(q 1 )


T =
1 60 143 242 208 120
334 CHAPTER 7. ADAPTIVE CONTROL

which gives T (q 1 ) = 1 + 60q 1 + 143q 2 + 242q 3 + 208q 4 + 120q 5 .

We are now ready to essentially do the reverse. Given the polynomials A(q 1 ), B(q 1 ), and
T (q 1 ), we wish to reconstruct F (q 1 ) and G(q 1 ) using the Diophantine routines given in List-
ings 7.2 and 7.3.

Listing 7.5: Solving the Diophantine equation using polynomials generated from Listing 7.4.
>> [F1,G1,Tcheck1] = dioph(A,B,T) % Use Listing 7.2
F1 =
8.0000 9.0000
4 G1 =
10.0000 11.0000 12.0000
Tcheck1 =
1 60 143 242 208 120

9 >> [F2,G2,Tcheck2] = dioph_mtx(A,B,T) % Use Listing 7.3


F2 =
1 8 9
G2 =
10.0000 11.0000 12.0000
14 Tcheck2 =
1 60 143 242 208 120

Note that in both cases we manage to compute the polynomials F (q 1 ) and G(q 1 ) which recon-
struct back to the correct T (q 1 ), although the routine in Listing 7.2 drops the leading 1 in the
monic F polynomial.

7.3.3 A DAPTIVE POLE - PLACEMENT WITH IDENTIFICATION

We have now all the components required for an adaptive controller. We will use an RLS mod-
ule for the plant identification, and then subsequently solve the Diophantine equation to obtain a
required closed loop response. Algorithm 7.2 describes the algorithm behind this adaptive con-
troller.

Algorithm 7.2 Adapting pole-placement algorithm


We will follow Fig. 7.6 which is a specific case of the general adaption scheme given in Fig. 7.1.

1. At each sample time t, collect the new system output, y(t).


2. Update the polynomial model estimates A, B using recursive least-squares on
Ay(t) = Bu(t 1) + e(t)

3. Synthesize the controller polynomials, F, G by solving the identity,


F A + q 1 BG = T
where T is the desired closed-loop response. (You could use the dioph routine given in
Listing 7.2.)
4. Construct H using either Eqn. 7.12 or Eqn. 7.13.
5. Apply control step
F u(t) = Gy(t) + Hr(t)
6. Wait out the remainder of the sample time & return to step 1.
7.3. ADAPTIVE POLE-PLACEMENT 335

Pole placement controller design


Identify Plant
Performance requirements A
T Diophantine RLS structure
B

H F G Plant

1 B(q 1 )
r(t) H(q 1 ) + b
y(t)
F (q 1 ) u(t) A(q 1 )

setpoint

G(q 1 )

2 DOF Controller

Figure 7.6: Adaptive pole-placement control structure with RLS identification. See also Fig. 7.5.

A N ADAPTIVE CONTROLLER WITH RLS IDENTIFICATION EXAMPLE

In this example will attempt to control three different plants in the configuration as shown in
Fig. 7.7. The open loop step tests of the plants are compared in Fig. 7.8. Of the three plants, G1 is
stable and well behaved, G2 has open-loop unstable poles, and G3 has unstable zeros.

Plant alternatives
z }| {

current active plant


G1
setpoint switch

r Gc G2 y


Adaptive G3
controller

current inactive plant models

Figure 7.7: Control of multiple plants with an adapting controller. We desire the same closed loop
response irrespective of the choice of plant.

In practice we must estimate the process model and if the plant changes, we redesign our con-
troller such that we obtain the same closed loop response. The customer for this application
demands a slight overshoot for the controlled response as shown in Fig. 7.9. We can translate this
336 CHAPTER 7. ADAPTIVE CONTROL

Openloop plant step responses


6000 PoleZero Map

1.5
5000 G1
G
1 2

4000
G2 (q1 ) G3

Imaginary Axis
G1 (q1 ) 0.5
output

3000
0

2000
0.5

1000
G3 (q 1
)
1

0 1.5
0 20 40 60 80 100 120 1 0.5 0 0.5 1 1.5
time Real Axis

Figure 7.8: Step responses of the 3 discrete open-loop plants (one of which is unstable), and the
corresponding pole-zero maps. Note that G3 has zeros outside the unit circle.

requirement for the denominator of the closed-loop transfer function to be

T = 1 1.4685q 1 + 0.6703q 2

Desired closed loop response


1.4

1.2

0.8
output

1
T(q 1 )
0.6

0.4

0.2

Figure 7.9: The desired closed loop response, 0


0 5 10 15 20 25 30 35
1/T . time

Listing 7.6 and Fig. 7.10 show the impressive results for this combined identification and adaptive
control simulation when using the three plants from Fig. 7.8. Note that the controller design
computation is done in the dioph.m routine given in Listing 7.2 while the identification is done
using the rls routine.

Here I assume a model structure where na = 5, nb = 3, and I add a small amount of noise,
= 0.005, to keep the estimation quality good. A forgetting factor of = 0.95 was used since the
model changes were known a priori to be rapid.

Listing 7.6: Adaptive pole-placement control with 3 different plants


Gx=struct('b',[3 0.2 0.5],'a',poly([0.95 0.9 0.85])); % 3 different plants
Gx(2) = struct('b',[2,2,1],'a',poly([1.1 0.7 0.65]));
Gx(3) = struct('b',[0.5 0.2 1],'a',poly([0.6 0.95 0.75]));

5 na = 3; nb = 3; % order of polynomials, na , nb
a = randn(1,na); b = randn(1,nb); % model estimates
7.3. ADAPTIVE POLE-PLACEMENT 337

np = length([a,b]); % # of parameters to be estimated


P = 1.0e4*eye(np); % Large initial co-variance, P0 = 104 I
lambda = 0.95; % Forgetting factor, 0 < < 1
10 param_est = [a,b]';

% Design a reasonable CLTF response with = 2 and = 0.4


tau=2; zeta = 0.4; Grc = tf(1,[tau2 2*tau*zeta 1]);
Grd = c2d(Grc,1); T = cell2mat(Grd.den);
15

dt = 1; t = dt*[0:700]'; r = square(t/10); % sample time & setpoint


y = zeros(size(t)); u = y; noise = 0.005*randn(size(t));
Param = zeros(length(t),np); trP = zeros(size(t));% plot holder
tG2 = 150; tG3 = tG2+200; k=1; % Swap over times for the different plants
20

for i=10:length(t)
if t(i) == tG2, k=2; end % Swap over to model #2
if t(i) == tG3, k=3; end % Swap over to model #3

25 % Plant response
y(i) = Gx(k).b*u(i-1:-1:i-length(Gx(k).b)) - ...
Gx(k).a(2:end)*y(i-1:-1:i-length(Gx(k).a)+1);

% Model identification
30 x = [-y(i-1:-1:i-na)', u(i-1:-1:i-length(b))']; % shift register
[param_est,P] = rls(y(i),x',param_est,P,lambda); % update parameters, Listing 6.18.
a=[1 param_est(1:na)']; b=param_est(na+1:na+nb)';% extract estimates: A, B
Param(i,:) = param_est'; trP(i) = trace(P);

35 % Controller design
[F,G] = dioph(a,b,T); % Solve Diophantine polynms, Listing 7.2.
%[F,G] = dioph_mtx(a,b,T); F(1)=[]; % Alternative Diophantine, Listing 7.3.
H = sum(T)/sum(b); % Get steady-state correct, H = T /B|q=1 .

40 % Control law, F u = Gy + Hr
u(i) = H*r(i) - G*y(i:-1:i-length(G)+1) - F*u(i-1:-1:i-length(F));
end % for i

%% Check the results


45 G = tf(sum(T),T,1); % desired closed loop
yref = lsim(G,r);

Fig. 7.10 shows the impressive results of this adaptive controller. When the plant changes, un-
beknown to the controller, the system identification updates the new model, and the controller
consequently updates to deliver a consistent response.

If the pole-placement controller is doing its job, then the desired closed loop response should
match the actual output of the plant. These two trajectories are compared in Fig. 7.11 which
shows an enlarged portion of Fig. 7.10. Here it can be seen that apart from the short transients
when the plant does change, and the identification routine is struggling to keep up, the adaptive
pole-placement controller is satisfying the requirement to follow the reference trajectory.

7.3.4 A DAPTIVE POLE - PLACEMENT IN S IMULINK

The time-varying filter blocks make it relatively easy to implement an adaptive pole-placement
controller in S IMULINK as showed in Fig. 7.12. The controller design is executed in the orange
RLS & APP sub-block which is shown in Fig. 7.13. This sub-block contains two m-file scripts
338 CHAPTER 7. ADAPTIVE CONTROL

output & setpoint 2


1
0
1
2
3
4

0.5
input

0.5

1 b
2
b
a0
3
0 b
a1

2
1 a
1
2
trace(P)

0 50 100 150 200 250 300 350 400 450


sample time

Figure 7.10: Adaptive pole-placement with RLS identification. After a period for the estimated
parameters to converge (lower trends), the controlled output response (upper trend) is again rea-
sonably consistent despite model changes. Results from Listing 7.6.

1.8 y

1.6 Reference, y*

1.4

1.2

0.8

0.6
0 50 100 150 200 250 300 350 400 450 500
time

Figure 7.11: An enlarged portion of Fig. 7.10 showing the difference between the actual plant
output y(t), and the reference output, y (t).
7.4. PRACTICAL ADAPTIVE POLE-PLACEMENT 339

that first do the RLS model identification update, and then do the pole-placement design based
on the currently identified model. The various memory blocks and the initial condition blocks are
needed to ensure that we do not have any algebraic loops when we start the simulation.

[1x3]
R
()*ory2 [1x3] [1x3]
()*ory G H

F
RLS
[1x3] &
Saturation [1x3] A''
f@>8)99678 faE9@>

T
u

ff
y
[1 0.1 0] 3
+, QQ FJLN
[0.05]
+,= 3
[1x3]
3
3

spt Y u
++. u y
: Den u y
Spt Set sam45678 96*e [1x3]
a;a496<) 8a67 -02 Saturation1
Va>?678 '5a79 9@ B) E@79>@lled
/
u
++. [1x3]
y +,- ()*ory1
Num
[1 0.1 0.2] [1x3] [1x3]

[1x3] [1x3]

num(z)
3(z) YO
3erminator
Desired response

Figure 7.12: Adaptive pole-placement with identification in S IMULINK. See also Fig. 7.13.

y F
[1x3]
2 y b eg 1
u theta1 theta [XZ3] W
theta b [1x3]
1 u h U
ss [XZ3]
[bZbc S
4 ff
Po d i 3
P 3
vgwxyz v{|zgw
r\]^_`date 3
`j\k`^lk]mno
q 3
[bZbc
qerminator

Figure 7.13: Internals of the Adaptive pole-placement design routine from Fig. 7.12.

7.4 P RACTICAL ADAPTIVE POLE - PLACEMENT

Some problems still remain before the adaptive pole-placement algorithm is a useful and practical
control alternative. These problems include:

Poor estimation performance while the system is in good control, and vice-versa, (poor
control during the periods suitable for good estimation). (This is known as the dichotomy
of adaptive control, or dual control).

Non-minimum phase systems possessing unstable zeros in B which must not be cancelled
by the controller.

Dead time where at least b0 = 0.

We must try to solve all these practical problems before we can recommend this type of controller.
340 CHAPTER 7. ADAPTIVE CONTROL

A N EXAMPLE OF BURSTING

If our controller does its job perfectly well, then the output remains at the setpoint. Under these
circumstances we do not have any identification potential left, and the estimated model starts to
drift. The model will continue to drift until the controller deteriorates so much that there is a
period of good excitation (poor control), that the model re-converges again. This

Fig. 7.14 shows the experimental results from using an adaptive controller, (in this case an adap-
tive LQR regulator) on the blackbox over a substantial period of time. The controller sampling at
T = 1 second was left to run over night for 10 hours (36,000 data points), and for a 3 hour period
in the middle we had no setpoint changes, and presumably no significant disturbances. During
this time, part of which is shown in Fig. 7.14, the adaptive controller oscillated into, and out of,
good control. This is sometimes known as bursting.

Adaptive LQR of the blackbox t=1.00 (sec)


0.4

0.3
output & setpoint

0.2

0.1

0.1

0.2

0.3
2.5 3 3.5 4

RLS: =0.995
1

0.5
Input

0.5

1
2.5 3 3.5 4
time (hours)

Figure 7.14: Bursting phenomena exhibited by the black box when using an adaptive controller.
With no setpoint changes or external disturbances, the adaptive controller tends to oscillate be-
tween periods of good control and bad estimation and the reverse.

7.4.1 D EALING WITH NON - MINIMUM PHASE SYSTEMS

If we use the steady-state version for H,



def T
H =
B q=1
as proposed in Eqn. 7.12, then we do not cancel any zeros, and we avoid the problem of inadver-
tently inverting unstable model zeros. However we can achieve a better transient response if we
cancel the zeros with the pre-compensator H,
1
H=
B
7.4. PRACTICAL ADAPTIVE POLE-PLACEMENT 341

giving the closed loop transfer function,

HB 1
y(t) = r(t 1) = r(t 1)
T T
which looks fine until we investigate closer what is likely to happen if B has zeros outside the unit
circle. Remember that the polynomial B is a function of the plant that we are trying to control
and hence an output of the estimation scheme. We have no influence on the properties of B; we
get what we are given.

In fact, we will have problems not only if the zeros are unstable, but even if they are stable but
poorly damped. Fig. 7.15 shows the adaptive pole-placement response of a plant with stable, but

poorly damped zeros when using H = 1/B. In fact the zeros are 0.95ej140 indicating that they
are close to, but just inside the unit circle. While the output response looks fine, it is the ringing
behaviour of the input that should cause concern.

3
yref
2 y*
output & setpoint

y
1

2
2

1.5

0.5
input

0.5

1.5

2
0 50 100 150 200 250 300 350 400 450 500
sample time

Figure 7.15: Adaptive pole-placement with a stable, but poorly damped, B polynomial. The
poorly damped zeros cause the ringing behaviour in the input.

The results in Fig. 7.15 were obtained by using the modified controller given in Listing 7.7 in
Listing 7.6.

Listing 7.7: The pole-placement control law when H = 1/B


if any(abs(roots(b))>1) % Use H = T /B|q=1 if any zeros of B are unstable.

H = polyval(T,1)/polyval(b,1); % Simple scalar pre-filter, H = T /B|q=1 .


4 u(i) = H*r(i) - G*y(i:-1:i-ng) - F*u(i-1:-1:i-nf);
else % Alternatively, use dynamic compensation, H = 1/B.
342 CHAPTER 7. ADAPTIVE CONTROL

v(i) = 1/b(1)*(-b(2:end)*v(i-1:-1:i-nb+1)+sum(T)*r(i));
u(i) = v(i) - G*y(i:-1:i-ng) - F*u(i-1:-1:i-nf); % control action
end

Suppose we have estimated a B polynomial possessing an unstable zero at say q 1 = +2/3, or


B = 1 1.5q 1
and we desire T = 1 0.7q 1 . Then the closed-loop transfer function is
BH q 1 (1 1.5q 1 )
y(t) = r(t 1) = Hr(t)
F A + q 1 BH 1 0.7q 1
Now suppose we use a dynamic pre-compensator, H = 1/B,
1
H=
q(1 1.5q 1 )
which has a time-advance term, q +1 , meaning that the control law is a non-causal function of the
setpoint. In other words, we need to know r(t + 1) at time t. In fact since we need to know only
future values of setpoint, and not output, this type of control law is easily implemented.

The main drawback stems, however, not from the acausality, but from the ill-advised cancellation.
In all practical cases, we will experience some small numerical round-off error such as where the
pole is represented in a finite-word length computer as 1.50001 rather than exactly 1.5. Conse-
quently the pole will not completely cancel the zero,
q 1 (1 1.5q 1 ) q
y(t) = r(t)
1 0.7q 1 (1 1.50001q 1)
| {z }
small error

leaving an unstable controller.

The remedy is that if we discover unstable modes in B, we simply do not cancel them, although
we continue to cancel the stable modes. We achieve this by separating the B polynomial into two
groups of factors: the stable roots, B + , (which we will colour green for good) and the unstable
roots, B , (which we will colour red for bad)
def
B = B+B (7.14)
and then set H = 1/B + only. A simple M ATLAB routine to do this factorisation is given in
Listing 7.8 in section 7.4.2.

So given the model and control law,


Ay(t) = Bu(t 1)
F u(t) = Gy(t) + Hr(t)
we modify the pole assignment to
AF + q 1 BG = T B + (7.15)
so closed-loop transfer function is
HB HB
y(t) = r(t 1) = r(t 1) (7.16)
T B+ T

We solve the modified pole-assignment problem, Eqn. 7.15, by noting that B + must be a factor of
F,
F = B + F1
7.4. PRACTICAL ADAPTIVE POLE-PLACEMENT 343

so we solve the now reduced Diophantine equation

AF1 + q 1 B G = T

in the normal manner.

We should always check for unstable zeros in the B polynomial when using online estimation
because we have no control over what the RLS routine will deliver. Even in a situation where we
know, or reasonably suspect a stable B, we may stumble across an unstable B while getting to the
correct values.

As an example, we can apply the adaptive pole-placement controller where we cancel the stable B
polynomial zeros, but not the unstable ones, using the three plants from page 335. Fig. 7.16 shows
the result of of this controller where we note that the performance of plant #3 is also acceptable
exhibiting no ringing.

For diagnostic purposes, we can also plot the instantaneous poles and zeros of the identified
model, and perhaps display this information as a movie, or perhaps as given in Fig. 7.16, shown
with decreasing intensity of colour for the older data. This will show how the poles and zeros
migrate, and when zeros, or even poles, leap outside the unit circle. Fig. 7.16 also shows the
magnitude of the estimated poles and zeros which clearly indicate when they exceed 1 and jump
outside the unit circle.

7.4.2 S EPARATING STABLE AND UNSTABLE FACTORS

When applying pole-placement in the case with unstable zeros, we need to be able to factor B(q)
into stable and unstable factors. Listing 7.8 does this factorisation, and additionally it also rejects
the stable, but poorly damped factors given a user-specified damping ratio, . (If you want simply
stable/unstable, use = 0.) Fig. 7.18 illustrates the region for discrete well-damped poles where
well-damped is taken to mean damping ratios > 0.15. This algorithm is adapted from [152,
p238].

Listing 7.8: Factorising an arbitrary polynomial B(q) into stable, B + (q), and unstable and poorly
damped, B (q), factors such that B = B + B and B + is defined as monic.
function [Bp,Bm] = factorBz(B,zeta);
2 % Factorise B(q) = B + (q)B (q) where B + are factors inside the unit circle and B (q) are outside.
% The optional parameter, defines the cutoff. Use = 0 for inside/outside circle.
% Note B + (q) or Bp is defined as monic.

if (exist('zeta') == 1)
7 zeta = 0.15; % default reasonable cut-off is = 0.15.
end
zeta = max(min(zeta,1),0); % Ensure [0, 1].
tol = 1e-6; r = roots(B);

12 % Now separate into poorly damped & good poles


mag_r = abs(r); rw = angle(r)/2/pi; % See equations in [152, p238].
mz = exp(-2*pi*zeta/sqrt(1-zeta2)*rw); % magnitude of z
idx = mag_r < mz;

17 Bp = poly(r(idx)); % Set B + (q) to contain all the zeros inside unit circle
Bm = poly(r(idx))*B(1); % All the others

if norm(conv(Bp,Bm)-B) > tol % Check B = B + B


warning('Problem with factorisation')
344 CHAPTER 7. ADAPTIVE CONTROL

2 G1 (q 1 ) G2 (q 1 ) Unstable zeros: G3 (q 1 )
output & setpoint

2
0.2

0.1
input

0.1

0.2
4
=0.99
2 a
2
b
b 2
0
0 b

1
a
3

2 a
1

4
5
10
tr(P)

0
10
0 500 1000 1500
sample time

def
Figure 7.16: Adaptive pole-placement control with H = 1/B and an unstable B polynomial in
Plant #3. Note that the controller does not exhibit any ringing, even with a plant with unstable
zeros.
7.4. PRACTICAL ADAPTIVE POLE-PLACEMENT 345

PoleZero Map
1.5
1.5 z1, z2

1
0.5 p1

|poles & zeros|


Imaginary Axis

p2
0

p3

0.5
0.5

1.5 0
1 0.5 0 0.5 1 1.5 0 500 1000 1500
Real Axis sample time

Figure 7.17: The migration of poles and zeros in the complex plane (left hand plot) and the time
trend of the magnitude of the poles and zeros (right-hand plot) from the simulation presented in
Fig. 7.16.

22 end % if
return

Figure 7.18: The boundary of the shaded area


is the loci of constant damping ratio with
= 0.15. Inside this region, the poles are con-
sidered well damped. (Figure plotted with
zgrid.)

7.4.3 E XPERIMENTAL ADAPTIVE POLE - PLACEMENT

Actually running an adaptive pole-placement scheme on a real plant, even a well behaved one
in the laboratory is considerably more difficult than the simulation examples presented so far.
Our desire is to test an adaptive pole-placement controller on the black-box to demonstrate the
following:

1. To follow a specified desired trajectory (as quantified by the T polynomial),

2. under both the NORMAL and SLOW positions on the black box, and
346 CHAPTER 7. ADAPTIVE CONTROL

3. to run stably for at least 10 minutes following both disturbances and servo changes.

The first design decision is to choose a desired response (closed loop poles) and suitable sampling
interval. We shall select a second-order prototype response with a time constant of = 2.1 sec-
onds and some overshoot, = 0.6. With a sampling time of T = 0.6 seconds, the desired discrete
closed loop transfer function is

1 0.03628 + 0.03236q 1
Gcl (s) = Gcl (q 1 ) =
4.41s2 + 2.52s + 1 1 1.641q 1 + 0.7097q 2
from which follows that T = 1 1.641q 1 + 0.7097q 2.

Fig. 7.19 shows the results from one experiment where the characteristics of the plant are changed
at t = 128 by manually switching over from 7 cascaded low-pass filters to 9. This increases the
effective time constant of the plant.

It is important to verify that the output from the blackbox did in fact follow the intended trajec-
tory once the model parameters have converged. An enlarged portion of Fig. 7.19 is repeated in
Fig. 7.20 comparing the intended trajectory, y , (dashed line) with the actual trajectory, y. This
shows that the response converged to the desired trajectory even after the plant subsequently
changed. Fig. 7.19 shows clearly that the shape of the input must vary considerably to force the
same output if the plant itself changes.

7.4.4 M INIMUM VARIANCE CONTROL WITH DEAD TIME

The minimum variance control formulation of 7.2.1 is not very useful not the least due to its in-
ability to handle any dead time and unstable process zeros. Even without the dead time, practical
experience has shown that this control is too vigorous and tends to be over-demanding causing
excessive manipulator variable fatigue. Moreover when using discrete time models, delay be-
tween input and output is much more common, and typically the deadtime is at least one sample
period. The best we can do in these circumstances is to make the best prediction of the output
we can for d samples times ahead, and use this prediction in our control law as done above. This
introduces the general concept of minimum variance control which combines prediction with
control. Further details are given in [23, pp367379].

Suppose we start with the ARMAX canonical model written in the forward shift operator q,

A(q)yk = B(q)uk + C(q)k (7.17)

and we assume that A and C are monic. We can always divide out the coefficients such that A
is monic, and we can choose the variance of the white noise term so that C is monic. For this
simplified development we will also need to assume that all the zeros of both C and B are inside
the unit circle. The pole excess of the system d, is the difference in the order of the A and B
polynomials and is equivalent to the system time delay. The output d steps ahead in the future
is dependent on the current and past control actions, u, which are known, and the current, past
and future disturbances, the latter which are not known. To avoid using the unknown disturbance
values we need to make an optimal prediction.

O PTIMAL PREDICTION

If we have a process driven by white noise ,


C(q)
zk = k (7.18)
A(q)
7.4. PRACTICAL ADAPTIVE POLE-PLACEMENT 347

Adaptive poleplacement with RLS

0.4
output & setpoint

0.3 Normal Slow


0.2
0.1
0
0.1
0.2

t=0.6,
1

0.5
input

0.5

=0.99, na=4, nb=3


2

1
parameters

5
10
tr(P)

4
10

3
10
0 50 100 150 200 250 300 350 400
time (sec)

Figure 7.19: Adaptive pole-placement of the black-box showing consistent output response be-
haviour despite the plant change when the switch was manually activated at t = 128 seconds.
Plot (A): black-box output (solid), intended trajectory (dashed) and setpoint point (dotted). (Refer
also to Fig. 7.20 for an enlarged version.)
Plot (B): Controller input to the plant, Plot (C): Plant parameter estimates, and Plot (D): Trace of
the covariance matrix.
348 CHAPTER 7. ADAPTIVE CONTROL

0.28

actual y
setpoint
0.26 Normal Slow intended y*

0.24

output & setpoint


0.22

Figure 7.20: A comparison of the in- 0.2


tended trajectory, y (dashed line)
with the actual trajectory y (solid
line) for the black box from Fig. 7.19. 0.18

Even after the plant change, the ac-


tual response gradually converges to 0.16
60 80 100 120 140 160 180 200 220 240 260 280
the desired trajectory. time (sec)

then the minimum variance predictor over d steps is

qG(q)
zk+d|k = zk (7.19)
C(q)

where G(q) is defined from


q d1 C(q) = A(q)F (q) + G(q) (7.20)
We can calculate the quotient F and remainder G polynomials in Eqn. 7.20 from polynomial
division in M ATLAB using the deconv function, or alternatively, one could calculate them by
equating the coefficients in Eqn. 7.20.

M INIMUM VARIANCE CONTROL LAW

Once we can make an optimal prediction using the formulation of Eqn. 7.20, we can predict the
output d samples times into the future and then calculate the control action required to equate
this with the setpoint. This will give the desired minimum variance control law as,

G(q)
uk = yk (7.21)
B(q)F (q)

The closed loop formulation by combining Eqn. 7.17 and Eqn. 7.21 is

G
Ayk = B yk + C k
BF
AF yk = Gyk + CF k
CF
yk = k = F k (7.22)
AF + G
where the last equation is simplified using Eqn. 7.20. Eqn. 7.22 indicates that the closed loop
response is a moving average process of order d 1. This provides us with a useful check in that
the covariance of the error will vanish for delays larger than d 1.

Listing 7.9 designs the F (q) and G(q) polynomials required in the control law, Eqn. 7.21, by using
deconvolution. Note this routine uses stripleadzeros to strip any leading zeros from the B(q)
polynomial from Listing B.3 given in Appendix B.
7.4. PRACTICAL ADAPTIVE POLE-PLACEMENT 349

Listing 7.9: Minimum variance control design


function [F,G,BF] = minvar_design(A,B,C)
2 % Design a minimum variance controller for plant B(q)/A(q) with noise polynomial C(q).
% Control law, Eqn. 7.21, is uk = G(q)/(B(q)F (q))yk
B = stripleadzeros(B); % Remove any leading zeros from the B polynomial
n = length(A); % order of plant

7 if length(C) 6= n
C = [C,zeros(1,1:n-length(C))]; % Force C(q) the same length as A(q)
end % if

d0 = length(A)- length(B); % Time delay is na nb = d0


12

qd0 = [1,zeros(1,d0-1)]; % Construct q d0 1


qd0C = conv(qd0,C); % Construct q d0 1 C(q)
[F,G] = deconv(qd0C,A);

17 BF = conv(B,F); % Construct B(q)F (q)


return

U NSTABLE PROCESS ZEROS

As in the adaptive pole-placement case outlined in section 7.4.1, if we have a non-minimum phase
plant with unstable zeros, then the resulting controller given by Eqn. 7.21 will be unstable since
essentially we are attempting to cancel right-hand plane zeros with right-hand plane poles. Again
the solution is to factor B into two factors, one containing all the stable modes, B + , and one
containing all the unstable modes, B , and using only B + in the control law. (See Listing 7.8 to
do this.) The F and G polynomials are obtained from solving the Diophantine equation which is
a generalisation of Eqn. 7.20. Refer to [23, pp380-385] for further details.

Moving average example

Consider the following plant system with a noise term,

(q 0.9)(q 0.8)(q 0.2)yk = 2(q 1.25)(q 0.4)uk + (q 3 0.7q 2 + 0.14q 0.2) ek

noting the unstable plant zero at q = 1.25.

Plant = zpk([1.25 0.4],[0.9 0.8 0.2],-2,1); % Plant poles & zeros


2 Ptf = tf(Plant);
B = stripleadzeros(Ptf.num{:});
A = Ptf.den{:};
C = [1 -0.7 0.14 -0.2]; % Noise polynomial
var_e = 1; % Variance of noise

The step response and pole-zero plot in Fig. 7.21 illustrates the potential difficulties with this
plant.

We can design a moving average controller for this NMP plant as follows.

B = stripleadzeros(B); % Remove the leading zeros, (if any)


[Bp,Bm] = factorBz(B); % Factorise B(q) into well-damped and unstable zeros

4 n = length(A); % order of plant


350 CHAPTER 7. ADAPTIVE CONTROL

PoleZero Map Step Response

20
1

15

0.5

Imaginary Axis

Amplitude
10

5
Figure 7.21: A non- 0.5
minimum phase plant 0
with an unstable zero 1
which causes an inverse 5
1 0.5 0 0.5 1 1.5 0 20 40 60 80
step response. Real Axis Time (seconds)

C = [C,zeros(1,n-length(C))]; % Back-pad with zeros to get same length as A(q) (if neces-
sary)

d = length(A)- length(Bp); % # of process zeros to cancel

9 % Now solve Diophantine eqn AR + BS = q d1 CB + to compute control law


qd = [1,zeros(1,d-1)]; % Construct q d1
qdCBp = mconv(qd,C,Bp); % Evaluate q d1 CB + using (multiple) convolution
[R,S] = dioph_mtx(A,B,qdCBp) % Listing 7.3.

14 [R1, R1rem] = deconv(R,Bp) % Check that R/B + has no remainder


G = S; BF = R; F=R1; % relabel
B = [zeros(1,length(A)-length(B)), B]; % Make B(q) same length as A to prevent AE loops

sim('sminvarctrl',[0 1e5]) % Now run the Simulink simulation


19 variance = var(ey);
varR1 = var_e*sum(R1.*R1);
disp([variance, varR1]) % Note that var{y} var{(F e)}

Note that this controller does not attempt to cancel all two process zeros since that would result
in an unstable controller giving rise to unbounded internal signals. However instead we cancel
the one stable process zero giving rise to a moving average controller.

In this example shown in Fig. 7.22, F (q) = q + 5.0567 which indicates that the expected covariance
of the output is y2 = (12 + 5.062 ) = 26.6 which, provided we take enough samples as shown in
Fig. 7.22(b), it is.

7.5 S UMMARY OF ADAPTIVE CONTROL

Adaptive control is where the controller adjusts in response to changing external conditions. This
is suitable for plants that are time varying, severely nonlinear or perhaps just unknown at the
controller design stage. The general adaptive controller has two parts; an identification part and
a controller design part although in some implementations these two components are joined to-
gether. The main styles of adaptive control are:

Where the controller parameters are some static function of the output. The simplest exam-
ple is gain scheduling. This has the advantage that it is easy to implement, but it is really
only open loop adaption, and may not perform as expected if the conditions change in an
unpredictable manner.
7.5. SUMMARY OF ADAPTIVE CONTROL 351

Random
Number

F(z)
ey
1
F To Workspace

C(z)
C
1

B(z) 1
1 A(z)
Add
B A Scope

Gain
G(z)
1
den(z)
Min var ctrlr
(a) A minimum variance moving average controller implementation in S IMULINK .

30

25

20 variance of output
Variance

15

10
variance of noise

0
3 4 5 6
10 10 10 10
Experiment duration

(b) The variance of the output should converge to


y2 = 26.6.

Figure 7.22: Minimum variance moving average control. If the controller is designed correctly,
then the output variance should be approximately equal to the variance of the noise signal filtered
through F , which in this case it is, provided we take enough samples.

Where the classical PID controller is augmented with a self-tuning option. This can be
based on the Ziegler-Nichols design, using a relay feedback to extract the ultimate frequency
and gain.
Where the parameters of the plant transfer function model are estimated online using the
techniques described in chapter 6, and the controller is designed to control this model. If
the controller is approximately the inverse of the process model, this leads to minimum
variance control, but is usually too severe, and even tends to instability for many actual
applications. A better alternative is adaptive pole-placement, where reasonable closed loop
poles are specified by the control engineer, and the controller is varied to provide this per-
formance as the identified model changes.

As with all high performance control schemes, there are some open issues with adaptive con-
trollers. For the simple schemes presented here we should be aware of the following issues:

1. The structure (order and amount of deadtime) of the plant must be known beforehand, and
352 CHAPTER 7. ADAPTIVE CONTROL

typically is assumed constant.


2. Minimum variance control, while simple to design, attempts to force the process back to the
desired operating point in one sample time. This results in overly severe control action.

3. There are potential numerical problems when implementing recursive estimators, particu-
larly in fixed point or with short word lengths.
4. If dead time is present, and usually dead time of at least one sample time is always
present, then the controller will be unrealisable since future values of y are needed to calcu-
late current values of u. In this case, a sub-optimal controller is used where the dead time is
simply ignored, or the future output is predicted.

However the real problem of adaptive control is that when the plant is in control, there is lit-
tle or no information going to the estimator. Hence the controller parameters begin to oscillate
wildly. This is not normally noticed by the operator, because the process is in control. However as
soon as a disturbance enters the plant, the controller will take extra action attempting to correct
for the upset, but now based on the wildly inaccurate plant model. During this time of major
transients however, the widely changing process conditions provide good information to the es-
timator which soon converges to good process parameters, which in turn subsequently generate
appropriate controller parameters. The adaptive controller then brings the process back under
control and the loop starts all over again. This results in alternating periods of very good control
and short periods of terrible control. One solution to this problem is to turn off the estimator
whenever the system is at the setpoint.

In summary when applying adaptive control, one wants:

A constant output for good control, but


a varying input and output for good estimation (which is required for good control)!

This conflict of interest is sometimes called the dichotomy of adaptive control.


7.5. SUMMARY OF ADAPTIVE CONTROL 353

P ROBLEMS

Problem 7.1 1. Construct a S IMULINK diagram to simulate the general linear controller with
2-degrees of freedom controller. That is, with the R, S and T controller polynomials. Follow
[19, Fig3.2, p93].
Hint: Use the discrete filter blocks in z 1 , since these are the only blocks that you can
construct S(z )/1 in.

2. Simulate the controlled response in problem 1 using the polynomials from [19, Ex3.1], re-
peated here
B(q) 0.1065q + 0.0902
= 2
A(q) q 1.6065q + 0.6065
and
R(q) = q + 0.8467, S(q) = 2.6850q 1.0319, T (q) = 1.6530q

3. Verify that the closed loop is the same as the expected model,

BT Bm
=
AR + BS Am

in the example 3.1, [19, Ex3.1, p9798].


Hint: You may like to use the M ATLAB symbolic toolbox.

4. Construct an m-file to do the minimum degree pole-placement design.

(a) Do an All zeros cancelled version


(b) Do an No zeros cancelled version
(c) Do an general version, perhaps with some automated decider which zeros to cancel.

5. For the minimum variance controller we need to be able to compute the variance of the
output of a FIR filter, refer [19, chapter 4].

(a) What is the theoretical output variance of a FIR filter with input variance of e2 ?
i. In S IMULINK you have the choice of the DSP format z 1 or the transfer function
version z. What would be the natural choice for q?
ii. Test your answer to part (1) by constructing a simulation in S IMULINK.
(b) Approximately how many samples do you need to obtain 2, 3 and 4 decimals accuracy
in your simulation?
(c) Using the hist function look at the distribution of both the input, e, and output, y. Is
the output Gaussian and how could you validate that statistically?

6. For stochastic control and for regulatory control topics we want to be able to elegantly sim-
ulate models of the form
A(q)y(t) = B(q)u(t) + C(q)e(t) (7.23)
where A, B, C are polynomials in the forward shift operator q. Note that the deadtime is
defined as deg(A)-deg(B).

(a) For the standard model in Eqn. 7.23 what are the constraints on the polynomials
A, B, C?
i. Choose sensible polynomials for A, B, C with at least one sample of deadtime.
Create a model object from the System Identification toolbox using the idpoly
command.
354 CHAPTER 7. ADAPTIVE CONTROL

ii. Simulate this model either using idmodel/sim or the idsim command. What is
the difference between these two commands? Choose a square wave for u(t) and
say random noise for e(t).
(b) Create a S IMULINK model and compare the simulated results with above. Use the
same random input.
(c) Write a raw M ATLAB script file to run the simulation and compare with the two alterna-
tives above. (They should give identical results.)
C HAPTER 8

M ULTIVARIABLE CONTROLLER
DESIGN

As our engineering endeavors became more sophisticated, so did our control problems. The
oil crises of the mid 1970s gave impetus for process engineers to find ways to recycle energy
and materials. Consequently the plants became much harder to control because they were now
tightly coupled to other plants. As the plants became multivariable, it made sense to consider
multivariable controllers.

If we have a dynamic system with say 3 states and 3 inputs, then assuming that our system is
controllable, we can feedback the three states directly to the three inputs, state 1 to input 1, state 2
to input 2 etc. This is termed de-centralised control, and our first dilemma is which state should
be paired with which input. We then have the problem of tuning the three loops, especially if
there are some interactions.

Alternatively we could feedback each state to all the inputs, which is termed centralised control.
In matrix form, the two alternative control laws look like

0 0
u = 0 0 x or u = x
0 0
| {z } | {z }
centralised de-centralised

Exploiting the off-diagonals of the gain matrix tends to be more efficient and is often what is
meant by multivariable control. While centralised or multivariable control structures typically
give better results for similar complexity, they are harder to design and tune since we have more
controller parameters (9 as opposed to 3) to select. Not surprisingly, decentralised control remains
popular in industry, none the least for the following reasons given by [89]:

1. Decentralised control is easier to design and implement than a full multivariable controller,
and also requires fewer tuning parameters.
2. They are easier for the operators to understand and possibly retune.
3. Decentralised controllers are more tolerant of manipulated or measurement variable failure.
4. Startup and shutdown operating policies are easier with decentralised controllers, since the
control loops can be gradually brought into service one by one.

Despite these application style advantages of decentralised controllers, it is useful to study what
is possible with a full multivariable controller. This is the purpose of this chapter.
355
356 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Finally a one-step optimal controller that is applicable for nonlinear processes called GMC (Generic
Model Control) is presented in 8.5. Exact feedback nonlinearisation is a generalisation of the
GMC controller follows in 8.6.

8.1 C ONTROLLABILITY AND OBSERVABILITY

This section will introduce in a descriptive manner the concept of controllability, observability
and other abilities. For a more formal treatment see for example [154, 94, p699-713] or [152,
p627 62]. The criteria of controllability is given first in 8.1.1 followed by the related property
of observability (which is the dual of controllability) in 8.1.2. These conditions are necessary for
some of the multi-variable controller and estimator design that follow. Methods to compute the
controllability matrix, and establishing the condition are given in 8.1.3.

8.1.1 C ONTROLLABILITY

When designing control systems, we implicitly assume that we can, at least in principle influence
the system to respond in some desired manner. However, it is conceivable, perhaps due to some
structural deficiency, that we cannot manipulated independently all the states of the system. The
controllability criteria tells us if this is the case.

Formally defined, a system is controllable if it is possible to transfer from any arbitrary initial state
x0 to any other desired state x in a finite time by using only an uncompensated control signal u.

Note that while the transformation is allowed to take place over more than one sample time,
it must be achieved in a finite time. In addition, even if the above controllability requirement is
satisfied, you may not be able to keep the state x at the desired position, the guarantee is only that x
could pass through x at some time in the future. Nevertheless, it is instructive to know if a given
system is controllable or not. Some care is needed with these terms. Astrom and Wittenmark, [23,
p127], define reachability as I have defined controllability above, and then define controllability as
a restricted version where x = 0. These two definitions are equivalent if A or is invertible.

First consider a continuous dynamic system that has a diagonal A matrix and an arbitrary B ma-
trix,

0 0
0 0

x = . x + Bu
0 0 . . 0
0 0

where an indicates the presence of a non-zero value. Now suppose if u(t) = 0, then the steady
state is xss = 0. Since A is a diagonal matrix, there is no interaction between the states. This
means that if one particular state xi , is 0 at some time t, then it will always remain at zero since
u= 0. Now consider the full system (u 6= 0), but suppose that the ith row of B are all zeros such
as the second row in the following control matrix,

0 0
0 0 0 0
B=
0 0
0

Then if xi happens to be zero, it will remain at zero for all time irrespective of the value of the
manipulated variable u. Quantitatively this is due to the following two points:
8.1. CONTROLLABILITY AND OBSERVABILITY 357

1. No manipulated variable, uj , can modify the value of xi because all the coefficients in the B
matrix for that particular state are zero.
2. No other states can influence xi because in this special case the system matrix A is diagonal.

Hence if xi is zero, it will stay at zero. The fact that at least one particular state is independent of
the manipulated vector means that the system is uncontrollable. We cannot influence (and hence
control) all the states by only changing the manipulated variables.

In summary, a system will be uncontrollable if the system matrix A is diagonal, and any one or
more rows of B are all zero. This is sometimes known as Gilberts criteria for controllability.

If the matrix A is not diagonal, then it may be transformed to a diagonal form (or Jordon block
form), and the above criteria applied. We can easily transform the system to a block diagonal
form using the canon function in M ATLAB. However a more general way to establish the con-
trollability of the n n system is to first construct the n (m n) controllability matrix C,
h i
C = B ... AB ... A2 B ... ... An1 B
def
(8.1)

and then compute the rank of this matrix. If the rank of C is n, then all n system modes are
controllable. This is proved in problem [152, A-6-6 p753]. The discrete controllability matrix is
defined in a similar manner;
h i
C = ... ... 2 ... ... n1
def
(8.2)

and also must be of rank n for the discrete system to be controllable.

An often less restrictive controllability requirement is output controllability. This is where we


wish only to control the outputs y, rather than the full state vector x. The output controllability
matrix C y is defined as
h i
C y = CB ... CAB ... CA2 B ... ... CAn1 B
def
(8.3)

Output controllability is given in Example A-9-11 Ogata, [154, p756], and [152, p636].

The rank of a matrix is defined as the number of independent rows (or columns) in a matrix, and
is a delicate numerical computation using a finite precision computer. The presence of a non-zero
determinant is not a good indication of rank because the value of the determinant scale so poorly.
By the same argument, a very small determinant such as say 106 is also not a reliable indication
of dependence. Numerically the rank is computed reliably by counting the number of non-zero
singular values, or the number of singular values above some reasonable threshold value. See
also 8.1.3.

The controllability matrix for the submarine system in 2.8.2 is



h i 0 0.095 0.0192
C = B ... AB ... A2 B = 0.095 0.0192 0.0048
0.072 0.0283 0.0098
which has a rank of 3. Hence the submarine system is controllable.

8.1.2 O BSERVABILITY

Observability is a related property to controllability. A system is observable if one can evaluate


the state position at some initial point, x0 , by recording only the output yt over a finite time
interval. In other words, the system is observable if all the states eventually effect the outputs.
358 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Note that the observation is allowed to take place over more than one sample time. Once we have
back-calculated x0 , we can use the history of the manipulated variable ut to forward calculate
what the current state vector xt is. It is clear that this condition of observability is important in
the field of estimation.

As a trivial example, suppose we had a 3 state system and we have three outputs which are related
to the states by
1 0 0
y = Cx = 0 1 0 x
0 0 1
Here the outputs (measurements) are identically equal to the states (C= I). It is clear that we can
evaluate all the states simply by measuring the outputs and inverting the measurement matrix
x = C1 y = Iy = y since C1 = I in this trivial example. This system is therefore observable.
Suppose however we measure only two outputs where C is now the non-square matrix
 
1 0 0
y = Cx = x
0 0 1

Now, obviously, if we recorded only one output vector (2 measurements), we could not hope to
reconstruct the full 3 state vector. But the observability property does not require us to estimate x
in one sample time, only in a finite number of sample times. Hence it may be possible for a system
with fewer outputs than states to be still observable.

To get a feel for the observability of a system, we will use a similar line of argument to that used
for the controllability. Suppose we have a diagonal system and an arbitrary output matrix

0 0
0 0

x = x (8.4)
0 0 ... 0
0 0
y = Cx (8.5)

Note that any system can be converted into the above form, and for observability criteria we are
not interested in the manipulated variable effects. Now if any of the columns of the C matrix are
all zeros, then this is equivalent to saying that that a particular state will never influence any of
the outputs, and since the system matrix is diagonal, that same state is never influenced by any
of the other states. Therefore none of the outputs will ever influence that state variable. Note the
symmetry of this criterion to the controllability criteria.

Formally for a linear continuous time system described by Eqn. 2.41, the n-state system is observ-
able if the np n observability matrix O is of rank n where

C C
CA C
2
2

def CA def
O = C

O= , or in the discrete case (8.6)
.. ..
. .
CAn1 Cn1

Some authors, perhaps to save space, define O as a row concatenation of matrices rather than
stacking them in a column as done above.

Problem 8.1 1. What are the dimensions of the observability and controllability matrices? What
is the maximum rank they can achieve?
2. Establish C and O for the Newell & Lee evaporator.
8.1. CONTROLLABILITY AND OBSERVABILITY 359

3. Examples B-9-13, B-9-10, B-9-11, [154, p771]


4. Examples B-6-6, B-6-7, [152, p805]
5. [158] gives some alternative mathematical definitions for controllability and observability:
A system is controllable if and only if

rank(B, A i I) = n, for i = 1, , n (8.7)

and observable if and only if

rank(C , A i I) = n, for i = 1, , n (8.8)

where are the eigenvalues of A.


Construct a M ATLAB m-file implementation using these algorithms. Test your routines on
various systems, perhaps using the rmodel command.

8.1.3 C OMPUTING CONTROLLABILITY AND OBSERVABILITY

The M ATLAB functions ctrb and obsv create the controllability and observability matrices re-
spectively, and the rank function can be used to establish the rank of them. While the concept
of observability and controllability applies to all types of dynamic systems, Eqns 8.1 and 8.6 are
restricted to linear time invariant systems. Read problem A-6-1 [152, p745] for a discussion on the
equivalent term reachable.

The algorithms given above for computing controllability or observability, (equations 8.1 and 8.6
or the discrete equivalents), seem quite straight forward, but are in fact often beset by subtle
numerical errors. There exist other equivalent algorithms, but many of these are also susceptible
to similar numerical problems. Issues in reliable numerical techniques relevant to control are
discussed in a series of articles collected by [160].

An example quoted in [158] illustrates common problems to which these simple algorithms can
fall prey. Part of the problem lies in the reliable computation of the rank of a matrix which is best
done by computing singular values.

Consider computing the controllability of the dynamic system


   
1 1
A= , B=
0 1

where is a small number. We can do this symbolically with

>> syms mu positive


>> A = [1 mu; 0 1]; B = [1; mu];
>> C = [B, A*B] % Compute controllability matrix, C
4 C =
[ 1, mu2 + 1]
[ mu, mu]
>> det(C)
ans =
9 -mu3

which tells us that the controllability matrix is


 
1 1 + 2
C=

360 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

and has a nonzero determinant of |C| = 3 indicating that the system is technically controllable
given a nonzero .

this example we are particularly interested in small, (but not minute) values of , say
However in
where || < , where is the machine precision. Note that this effectively means that 1 + 2 = 1
whereas 1 + > 1 when executed on the computer. Consequently the controllability matrix is
now  
1 1
C= , for || <

which indicates the system is uncontrollable.

We can try this numerically for values near the expected limit, = ,


1 >> mu = sqrt(eps(1)) %
mu =
1.4901e-08
>> A = [1 mu; 0 1]; B = [1; mu];
>> det(ctrb(A,B))
6 ans =
-3.3087e-24

where fortuitously in this case we get a non-zero determinant of equal to 3 as hoped. However
for (slightly) smaller values of ,


>> mu = sqrt(eps(1))/10 % <
mu =
3 1.4901e-09
>> A = [1 mu; 0 1]; B = [1; mu];
>> det(ctrb(A,B))
ans =
0

due to the roundoff, this tells us the system is uncontrollable.

Problem 8.2 Write a M ATLAB m-file to compute the exponential of a matrix using the Taylor series
formula,

X 1 k
eAt = A
k!
k=0
Test your algorithm on  
49 24
A=
64 31
and comment on the expected accuracy of your result.

Problem 8.3 1. When using good (in the numerical reliability sense) software, we tend to forget
one fundamental folklore of numerical computation. This folklore, repeated in [160, p2],
states
If an algorithm is amenable to easy hand calculation, then it is probably a poor
method if implemented in the finite floating point arithmetic of a digital computer.
With that warning in mind, calculate the singular values of

1 1
A= 0
0
8.1. CONTROLLABILITY AND OBSERVABILITY 361


where is a real number such that || < where is the machine precision. Calculate the
singular values both analytically, and using the relationship between singular values and the
eigenvalues of A A.

8.1.4 S TATE RECONSTRUCTION

If a system is observable, we should in principle, be able to reconstruct the states based only
on the current, and perhaps historical outputs. So, supposing we have collected the following
input/output data,

time (k) u(k) y(k) x1 (k) x2 (k)


0 4 4 ? ?
1 5 4 ? ?
2 36 ? ?

we should be able to reconstruct the states assuming a model of the form


   
0 1 0
xk+1 = xk + uk
3 2 1
 
yk = 4 0 x k

Note that we cannot simply invert the measurements back to the states, partly because the mea-
surement matrix is not square.

However we can see that y(k) = 4x1 (k) from the measurement relation, so we can immediately fill
in some of the blank spaces in the above table, in the x1 (k) column; x1 (0) = 1, x1 (1) = 1, x1 (2) =
9. From the model relation, we can see that x1 (k + 1) = x2 (k), so we can set x2 (0) = x1 (1) = 1.
Now we have established the initial condition
 
x(0) = 1 1

and with the subsequent manipulated variable history u(t), we can calculate x(t) simply using
the recursive formula, Eqn. 2.42. This technique of estimation is sometimes called state recon-
struction and assumes a perfect model and no measurement noise. Furthermore, the estimation
reconstruction takes, at most, n input-output sample pairs.

As a check, we can construct the observability matrix,


 
  4 0  
C  4 0
O= =   0 1 =
C 4 0 0 4
3 2

which clearly has a rank of 2 indicating that the state reconstruction is in fact possible.

We can generalise this state reconstruction scheme outlined above, following [20, p315], for any
observable discrete plant

xk+1 = xk + uk
yk = Cxk

where we wish to reconstruct the current state, xk , given past values of input and output, uk , uk1 , . . .
and yk , yk1 , . . .

Our basic approach will be to use the outputs from the immediate past n samples, to develop
linear equations for the n unknown states at the point n samples in the past. We will then use the
362 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

knowledge of the past inputs to roll this state forwards to the current sample k. For the present
development, we will just consider SISO systems.

Listing the immediate previous n values finishing at the current sample k gives

ykn+1 = Cxkn+1
ykn+2 = Cxkn+1 + Cukn+1
ykn+3 = C2 xkn+1 + Cukn+1 + Cukn+2
..
.
yk = Cn1 xkn+1 + Cn2 ukn+1 + + Cuk1 (8.9)

To make things clearer, we can stack the collected input and output values in a vector,

ykn+1
ykn+2 ukn+1
ukn+2
Yk =
..
, U =

(8.10)
. k1 ..

yk1
.
uk1
yk

and we note that the input vector collection Uk1 is one element shorter than the output vector
collection Y k . Now we can write Eqn. 8.9 more succinctly as

Y k = Oxkn+1 + Wu Uk1 (8.11)

with matrices defined as



C 0 0 0

C


C 0 0


O= C2
,

Wu = C C 0
(8.12)
.. .. .. .. ..
. . . . .
Cn1 Cn2 Cn3 C

and we may recognise O is the observability matrix defined in Eqn. 8.6.

Solving Eqn. 8.11 for the states (in the past at sample time k n + 1) gives

xkn+1 = O 1 Yk O 1 Wu Uk1 (8.13)

provided O is non-singular. This is why we require the observability condition to be met for state
reconstruction.

The minor problem with Eqn. 8.13 is that it delivers the state n sample times in the past, rather
than the current state, xk , which is of more immediate interest. However, it is trivial to use
the plant model and knowledge of the past inputs to roll the states onwards to current time
k. Repeated forward use of the model gives

xk = n1 xkn+1 + n2 ukn+1 + n3 ukn+2 + + uk1

The entire procedure for state reconstruction is summarised in the following algorithm.

Algorithm 8.1 State reconstruction assuming a perfect observable model and no noise. Given

xk+1 = xk + uk , yk = Cxk

then the current state, xk , is given by the linear equation

xk = Ay Y k + Bu Uk1
8.1. CONTROLLABILITY AND OBSERVABILITY 363

where the matrices are


Ay = n1 O +
 
Bu = n2 n3 n1 O + Wu
and Y k , Uk1 are defined in Eqn. 8.10 and O, Wu are defined in Eqn. 8.12. We should note:

To reconstruct the state (at the current time k), we need to log past input and output data,
and the minimum number of data pairs required is n, where n is the number of states. Any
more will gives a least-squares fit.
Since we invert O, this matrix must not be singular. (Alternatively we require that the
system must be observable.) For the multivariable case we should use the pseudo-inverse,
O + , as written above.
We use in the algorithm, but do not invert, the controllability matrix.

A simple state reconstructor following Algorithm 8.1 is given in Listing 8.1.

Listing 8.1: A simple state reconstructor following Algorithm 8.1.


function x_est = state_extr(G,Yk,Uk)
% Back extract past state from outputs & inputs (assuming observable)
3

Phi = G.a; Del = G.b; C = G.c;


error(ABCDCHK(Phi,Del,C)) % bullet proofing
[p,n] = size(C); [n,m] = size(Del);

8 Wo = obsv(G); % same as: O = obsv(Phi,C)


Ay = Phi(n-1)/Wo; % similar to n1 O 1

S = C*Del;
for i=1:n-2
13 S = [S; C*Phii*Del]; % Building Wu
end % for

zC = zeros(p,m); % or zeros(size(C*Del));
Wu1 = [zC;S]; Wu = Wu1;
18 for i=n-3:-1:0
Wu1 = [zC; Wu1]; Wu1(end-p+1:end,:) = [];
Wu = [Wu, Wu1];
end % for

23 Co = Del; % Reversed controllability matrix


for i=1:n-2
Co = [Phii*Del,Co];
end % for

28 Bu = Co - Ay*Wu; x_est = Ay*Yk + Bu*Uk;


return

Fig. 8.1 illustrates the results of the state reconstruction algorithm in the case where there is no
model/plant mismatch, but there is noise on both the input and output. Consequently, given that
the algorithm has the wrong u(t) and y(t) trends, then the reconstructed states ( in Fig. 8.1) are
slightly different from the true states.

In summary, this reconstruction algorithm takes current and historical outputs and inputs, and,
provided the system is observable, then we can reconstruct the current states. This strategy is
364 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

0.3

0.2

Output
0.1

0 actual y
measured y
0.1
4
actual u
measured u
2

u
0

2
x
1
est
1
1
x

1
2
x
2

Figure 8.1: Reconstructing states. est


1
Since the state reconstructor is given
2
x

an incorrect input and a noisy out-


0
put, then it is no surprise that the es-
timated states, , are slightly different
from the true states (solid lines in the 1
0 1 2 3 4 5 6 7
lower two trends). time [s]

important because it then allows us to use state feedback in the cases where we only measure the
outputs. In the presence of process or measurement noise, and/or model/plant mismatch this
reconstruction will be in error, although in many cases this may not be significant. An improved
version of this algorithm that takes into account the statistics of the expected noise and plant
errors is known as state estimation and is described in section 9.5.

8.2 S TATE SPACE POLE - PLACEMENT CONTROLLER DESIGN

This section describes how to design a controller to control a given process, using state space
techniques in the discrete domain.

When a process engineer asks the control engineer to design a control system, the first question
that the control engineer should ask back, is what sort of performance is required? In this way,
we start with some sort of closed loop specification, which, for example, may be that we demand
that the system return to setpoint after a disturbance with no offset, or rise to a new setpoint in a
reasonable time, perhaps one open-loop time constant. If the loop is a critical one, then we may
wish to tighten this requirement by specifying a smaller time to reach control, or conversely if
we have a non-critical loop we could be satisfied with a more relaxed settling time requirement.
It is our job as control engineers to translate these vague verbal requirements into mathematical
constraints, and if possible, then design a controller that meets these constraints.

When we talk about a response, we are essentially selecting the poles (or eigenvalues) of the closed
8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN 365

loop transfer function, and to some lesser degree, also the zeros. Specifying these eigenvalues by
changing the controller parameters is called controller design by pole placement. In order to
design a controller by pole placement, we must know two things:

1. A dynamic model of the process, and

2. what type of response we want (i.e. what the closed loop eigenvalues should be).

However choosing appropriate closed loop poles is non-trivial. Morari in [140] reminds us that
poles are a highly inadequate measure of closed loop performance, and he continues to assert
that because of the lack of guidance to the control designer, pole placement techniques have been
abandoned by almost everybody today. Notwithstanding this opinion, it is still instructive for
us to understand how our closed loop response is affected by different choices of eigenvalues, and
how this forms the basis for more acceptable tuning methods such as linear quadratic controllers
discussed in the following chapter.

Suppose we have a single inputmultiple output (SIMO) system such as

xk+1 = xk + uk (8.14)

where u is a scalar input and we wish to control all n states by using a proportional feedback
control law such as
u = Kx (8.15)
where K is called the state feedback gain matrix. (Actually in this SIMO case, the gain matrix
collapses to a row vector). Substituting Eqn 8.15 into Eqn 8.14 gives the closed loop response as

xk+1 = ( K) xk (8.16)

Now we desire (K) to have the n eigenvalues at arbitrary positions 1 , 2 , , n To achieve


this, we must check that arbitrary pole placement is achievable. In other words, the system must
be controllable. This means that the controllability matrix Co , must be of rank n. If this condition
is satisfied, we can design the feedback gain matrix K, by using Ackermanns formula
h i1
 . .. .
K= 0 0 0 1 .. . .. n1 () (8.17)

where is the matrix characteristic equation of the desired closed loop transfer function. Note
that the matrix to be inverted in Eqn 8.17 is the discrete controllability matrix Co , hence the need
for full rank.

A quick way to calculate () is to use poly and polyvalm. For instance we can verify the
Cayley-Hamilton theorem, (any matrix satisfies its own characteristic equation), elegantly using

1 >> A = rand(3) % take any test matrix


A =
0.0535 0.0077 0.4175
0.5297 0.3834 0.6868
0.6711 0.0668 0.5890
6

>> polyvalm(poly(A),A) % Cayley-Hamilton theorem: (A) should = 0


ans =
1.0e-016 *
0.1388 0.0119 0.1407
11 0.1325 0 0.7412
-0.0496 0.0073 0.5551
366 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Algorithm 8.2 SIMO state-space pole-placement

Given a continuous system model, x = Ax + Bu, or discrete version, xk+1 = xk + uk , and


design requirements for the closed loop poles, , then

1. Check the controllability of the system, ctrb(A, B), or ctrb(, ). If not all the states are
controllable, then terminate the algorithm, otherwise carry on.
2. Compute the characteristic polynomial, , and substitute either matrix A or .
3. Compute the row-vector state feedback gain using Eqn. 8.17.

Example Pole placement of the double integrator system.

In this example, we wish to design a controller for the discrete double integrator system sampled
at T = 1 that has closed loop eigenvalues at = 0.5 0.5i.

In this case we will assume a zeroth-order hold on


the discretisation. giving matrices
   
2 0.5 1
Gc = tf(1,[1 0 0]); % Continuous system = , =
2 0 0
G = ss(c2d(Gc,1)); % Sampled at Ts = 1

To use Ackermann formula, Eqn. 8.17, we need to both compute the controllability matrix and the
matrix characteristic equation .
def
Note that as (z) = (z 1 )(z 2 ) = z 2 z + 0.5, then
 
1 0.5
() = 2 + 0.5I =
2 0.5
and that the controllability matrix  
1 2
C=
0 2
has full rank.

>> mu = [0.5+0.5i; 0.5-0.5i]; % Design requirements for


>> alpha = polyvalm(poly(mu),G.a)
3 >> Cb = ctrb(G.a,G.b)

Finally the feedback gain is computed by

 
>> K = [0 1]/Cb*alpha % giving K = 1 0.25

We can easily check that the closed loop poles are as required with

>> eig(G.a-G.b*K) % Check that = eig( K)


ans =
0.5000 + 0.5000i
4 0.5000 - 0.5000i

Rather than constructing Ackermanns equation by hand, M ATLAB can design a pole placement
controller using the acker routine. At heart this routine simply computes Eqn. 8.17 as
8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN 367

1 k = ctrb(a,b)\polyvalm(real(poly(p)),a); % Form gains using Ackermans formula


k = k(n,:);

or, as noted in the next section, acker is very ill-conditioned, so we could use the preferred place
function. The latter function gives an estimate of the precision of the closed loop eigenvalues, and
an optional error message if necessary. (See also Fig. 8.3.)

[K,prec] = place(G.a,G.b,mu)
K =
3 1.0000 -0.2500
prec =
15

Eqn 8.15 assumes that the setpoint is at x=0. (We can always transform our state variables by
subtracting the setpoint so that this condition is satisfied, but this becomes inefficient if we are
likely to change the setpoint regularly.) In these cases, it is better to modify our control law from
Eqn 8.15 to
u = K(r x) (8.18)
where r is a vector of the state setpoints or reference states, now assumed non-zero. Consequently
the closed loop response is now modified from Eqn 8.16 to

xk+1 = ( K) xk + Krk (8.19)

8.2.1 P OLES AND WHERE TO PLACE THEM

While pole-placement seems an attractive controller design strategy, the problem lies in deciding
where the closed loop poles out to be.

Listing 8.2 designs a pole-placement controller for the blackbox, G(s) = 0.98/(2s + 1)(s + 1), with
discrete closed loop poles specified, somewhat arbitrarily, at T = [0.9, 0.8].

Listing 8.2: Pole-placement control of a well-behaved system


>> Gc = tf(0.98,conv([2 1],[1,1])) % Blackbox model
>> G = ss(c2d(Gc,0.1)) % Convert to discrete state-space with sample time Ts = 0.1
>> [Phi,Delta,C,D] = ssdata(G);

5 >> lambda = [0.9 0.8]'; % Desired closed loop discrete poles T = [0.9, 0.8]
>> K = place(Phi,Delta,lambda)% Compute controller gain K
K =
2.4971 -2.2513

Now we can check the closed loop performance from a non-zero initial condition with

Gcl = ss(Phi-Delta*K,zeros(size(Delta)),eye(size(Phi)),0,Ts);
2

x0 = [1; -2]; % start position


initial(Gcl,x0) % See Fig. 8.2

The results of a controlled disturbance recovery is given in the left hand plot of Fig. 8.2. However
an obvious question at this point is where exactly should I place the poles. The three trends in
368 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN


T T T j80
= [0.9, 0.8] = [0.9, 0.8] = 0.6e

6 6 6
States x1 & x2

4 4 4

2 2 2

0 0 0

2 2 2
100 100 100
input

0 0 0

100 100 100


0 2 4 0 2 4 0 2 4

1 1.7 1.2
1 1.7 1.2
1 1.7 1.2
2.2 0.2 2.2 0.2 2.2 0.2
0.5 0.4 0.7 0.5 0.4 0.7 0.5 0.4 0.7
2.7 0.6 2.7 0.6 2.7 0.6
0.8 0.8 0.8
0.2 0.2 0.2
0 0 0

0.2 0.2 0.2


2.7 2.7 2.7
0.5 0.7 0.5 0.7 0.5 0.7
2.2 2.2 2.2
1.7 1.2 1.7 1.2 1.7 1.2
1 1 1
1 0 1 1 0 1 1 0 1

Figure 8.2: Pole-placement of the black box for three different pole locations. From left to right:

(a) T = [0.9, 0.8], (b) T = [0.9, 0.8], and (c) = 0.6ej80 .

Fig. 8.2 illustrate the difference. A conservative choice is as given above is where we place the
poles in right-hand side of the origin, on on the real axis close to the +1 point. In this case I chose
T = [0.9, 0.8]. The second choice is the mirror image of those poles T = [0.9, 0.8], and the
third choice is a pair of complex conjugates at 0.66 80 .

Clearly there is a trade off between the speed of the disturbance rejection, the amount of input
energy, and the degree of oscillation. If we can quantify these various competing effects, then
we can design controllers systematically. This is the subject of optimal control given in the next
chapter.

T HE POLE - PLACEMENT DESIGN PROBLEM IS ILL - CONDITIONED

The design of controllers using pole-placement is inherently numerically ill-conditioned, espe-


cially for large systems. Ackermanns algorithm requires double precision for systems larger than
about 10th order, and outright fails for systems larger than about 15th order. Fig. 8.3 illustrates
this degradation in accuracy by plotting an estimate of the number of accurate decimal places in
the solution using place for 1500 randomly generated discrete models from orders 3 to 15. Note
that once the order is above 9 or so, we should only expect about 6 decimal places of accuracy.
8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN 369

20

15

10
precision

5
poleplace
Acker
0

5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
System order

Figure 8.3: The loss of accuracy using pole-placement algorithms place and acker as a function
of system order for 500 randomly chosen models. It is clear that both routines struggle for systems
of order above about 6.

8.2.2 D EADBEAT CONTROL

If the system is controllable such that any arbitrary poles are allowable, one is naturally inclined to
extrapolate the selection to poles that will give a near perfect response? Clearly a perfect response
is one when we are always at the setpoint. If a step change in reference value occurs, then we
instantaneously apply a corrective measure to instantaneously push the system to the new desired
point. A system that rises immediately to the new setpoint has a closed loop time constant of = 0
or alternatively a continuous pole at s = . This corresponds to a discrete pole at eT or the
origin. If we decide to design this admittedly rather drastic controller, we would select all the
desired closed loop poles at = 0. Such a controller is feasible in the discrete domain and is
called a deadbeat controller.

The design of a deadbeat controller is the same as for an arbitrary pole placement controller.
However it is better to use the acker function rather than the place function, because the place
function will not allow you to specify more multiple eigenvalues than inputs. This restriction does
not apply for the Ackermann algorithm.

Suppose we design a deadbeat controller for the continuous plant



0.3 0 0 3
x = 0 0.5 2 x + 2 u (8.20)
0 2 0.5 5

To design a deadbeat controller, we specify the desired discrete closed loop eigenvalues as = 0
and select a sample time of T = 4, we obtain
 
K = 3.23 102 3.69 104 1.98 102

The closed loop transition matrix is



0.075 0.003 0.139
cl = K = 0.063 0.019 0.095
0.060 0.133 0.056

and we could also compute that all the elements in 3cl are in the order of 1019 and all the
eigenvalues of eig( K) are in the order of 107 as required.

Such a matrix where An1 6= 0, but An = 0 is called a Nilpotent matrix.


370 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Fig. 8.4 shows the impressive controlled response of the plant (Eqn. 8.20) under deadbeat control.
The top half of the diagram shows the states which converge from the initial condition, ( x0 =
 
2 3 1 ), to the setpoint x=0 in only three sample times (3T = 12 s). It takes three
sample times for the process to be at the setpoint, because it is a third order system.

Deadbeat controller
4

states
0

4
Figure 8.4: The states (top) and the
manipulated variable (bottom) for 0.04
input

the deadbeat controlled response. 0.02


The states after the third sample time, 0
t 12, remain exactly at the setpoint 0 5 10 15 20
at the sample times. time (s), sample time T=4

The deadbeat controlled response looks almost perfect in Fig. 8.4, at least after the 3 samples
required to get top the setpoint. We can investigate this by repeating the simulation at a smaller
time interval (say T = 0.05 s) but with the same input as before, to see what happens between
the sample intervals. (This is how I evaluated the smooth state curve in Fig 8.4 between the states
at the controller sample time denoted with a in Listing 8.3.) Note that the states are oscillating
between the samples, but quickly die out after 10 seconds.

Listing 8.3: A deadbeat controller simulation


1 x0= [-2 -3 1]'; dt=4; tf=dt*5; t=[0:dt:tf]'; % initial condition & time vector

A= [-0.3,0,0;0,-0.5,2;0,-2,-0.5];B= [3;-2;5]; % Continuous plant, Eqn. 8.20


Gc= ss(A,B,eye(3),0); Gd= c2d(Gc,dt,'zoh'); % Convert to a discrete model

6 mu= [0;0;0]; % Desired closed loop eigenvalues, = 0


K= acker(Gd.a,Gd.b,mu); % construct deadbeat controller

Gd_cl = Gd; Gd_cl.a = Gd.a-Gd.b*K; % closed loop


[y,t,x]= lsim(Gd_cl,zeros(size(t)),[],x0);
11 U = [-K*x']'; % back extract input

% Reduce sample time at same conditions to see inter-sample ripple


dt = dt/100; t2 = [0:dt:tf]'; % new time scale, Ts = Ts /100
[ts,Us] = stairs(t,U); % create new input
16 ts = ts + [1:length(ts)]'/1e10; ts(1) = 0;
U2 = interp1(ts,Us,t2); % approx new input
[y2,t2,x2]= lsim(Gc,U2,t2,x0);% continuous plant simulation

subplot(2,1,1);plot(t,x,'o',t2,x2); % Refer Fig. 8.4


21 subplot(2,1,2);plot(ts,Us);
8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN 371

Naturally under practical industrial conditions, the plant will not be brought under control so
rapidly owing to either modelplant mismatch, or to manipulated variable constraints. This type
of controller can be made more robust by specifying a larger sample time, which is in fact the
only tuning constant possible in a deadbeat controller. Ogata gives some comments and cautions
about deadbeat control in [152, pp670-71].

Problem 8.4 Create a system with eigenvalues of your choice; say


 T
v= 0.2 0.3 0.1

and design a deadbeat controller. Simulate this using the dlsim function. (Try a sample time
of 7.) Investigate the same controller, but at a higher sampling rate and plot the intersample
behaviour of the states.

Investigate the effect on the gain when using different sample rates.

8.2.3 P OLE - PLACEMENT FOR TRACKING CONTROL

The pole-placement controller developed in Algorithm 8.2 is only a proportional controller so


will suffer from the problem of offset given setpoint changes as graphically demonstrated in the
upper plot in Fig. 8.6. Such controllers are suitable for disturbance rejection because they return
to a setpoint of zero after a disturbance, but they are not much good when we wish to track an
arbitrary setpoint.

To include tracking characteristics in our pole-placement controller, we will need to introduce


extra states, z defined as the integral of the normal states
Z
def
z= x dt (8.21)

which can be augmented to the original system as


      
x A 0 x B
= + u. (8.22)
z I 0 z 0

The feedback control law is now


 
 x
u= Kp Ki (8.23)
z

where Kp are the gains for the proportional controller and Ki are the gains for the integral con-
troller. Now the pole-placement controller is designed for the new system with n additional
integral states in the same manner as for the proportional state only case. In M ATLAB we can
formulate the new augmented system as

Ai = [A zeros(size(A)); eye(size(A)) zeros(size(A))]


Bi = [B; zeros(size(B))]

This modification improves the system type, (i.e. the number of poles at the origin), enabling
our controller to track inputs. [111, 1014,p802] gives further details. We shall see later in sec-
tion 9.4.6 how we can design optimal pole-placement servo controllers known as linear quadratic
regulators.
372 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

SISO SERVO INTEGRAL OUTPUT CONTROL

If we want to track setpoint changes (servo-control) for a SISO system, we need only to add one
extra integral state. The augmented system is

x = Ax + Bu
z = r y
y = Cx

where x is the vector of original states, u, y and r are the scalar input, output and setpoint respec-
tively. We simply concatenate the new integral state to the existing,
h i
x =
def ..
x1 x2 xn . z

so that z is relabelled as xn+1 . The feedback gain row-vector in the control law u = Kx now has
an additional element h i
K = k1 k2 kn ... kn+1

and the augmented closed-loop system is now


 
dx  0
= A BK x + r
dt 1

where the augmented matrices are:


  
A 0 B
A = , B =
C 0 0
| {z } | {z }
(n+1)(n+1) (n+1)1

To implement, the augmented system must be controllable, which is a tougher requirement than
just ctrb(A, B). A block diagram of a pole-placement controller with a single integral state is
given in Fig. 8.5, after [111, p802]. The wide lines in the diagram are vector paths.

integral state
xn+1 x x
+ xn+1 kn+1 +
u
B + C y
R R
r


K
full state feedback

Figure 8.5: State feedback control system with an integral output state.

We can compare the effect of adding an integral state to remove the offset given the SISO system
   
4 3 2
x = x+ u
1 6 1
 
y= 1 0 x
8.3. ESTIMATING THE UNMEASURED STATES 373

where we want to control x1 to follow reference r(t) using pole-placement. I will place the closed-
loop eigenvalues for the original system at
 
6 + 2i
=
6 2i

and with the extra integral state, I will add another pole at 3 = 10. Fig. 8.6 compares the
response of both cases. Clearly the version with the integral state that tracks the setpoint with no
steady-state error is preferable.

2
Integral state
output & setpoint

2 setpoint Figure 8.6: Adding an integral


standard
3 Proportional only integral states
state removes the offset for
4 an LQR (or pole-placement)
0 1 2 3 4 5 6 7 8 9 10
time [s] controller.

8.3 E STIMATING THE UNMEASURED STATES

The pole-placement control design strategies as developed in 8.2 utilises all the states in the
control law u = Kx so implicitly assumes that all the states are measurable. In many cases this
is not practically possible, so the only alternative then is to estimate the states.

The states can be reconstructed using the outputs, the inputs, and the model equations. In fact we
need not restrict ourselves to just estimating states, we could also estimate derived quality control
variables, or even estimate the outputs themselves. The quantities that we estimate are known as
observers, and offer cost and reliability advantages over actual sensors in many applications.

The simplest state estimator is where we use a model and subject it to the same input u that is
applied to the true process. The resultant x is the estimate for the true state x.

xk+1 = xk + uk (8.24)

If we know the true model, ,, and the manipulated variable history uk , then this estimator will
work provided we select the correct initial condition x0 . Since we have no feedback information
as to how well we are doing, this estimator is called an open loop estimator. If the conditions
stated above are not met, and they will rarely be satisfied in practice, then the estimator will
provided estimates that will probably get worse and worse. The error of the estimate is

k = xk xk (8.25)

Ideally, we would like to force this error vector to zero. This is actually still a control problem, al-
though while we are not controlling the plant, we are controlling the error of the estimate scheme.
We introduce a feed back error scheme such as

xk+1 = xk + uk + L (yk Cxk ) (8.26)

which is called a prediction error observer since the prediction is one sample period ahead of the
observation. In the single output case, L is a (n 1) column vector and is called the observer gain
374 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

matrix and is a design parameter. The L stems from the fact that it is a Luenberger observer and
we need to carefully distinguish it from the controller gain, K. Following our colour convention,
we will sometimes colour the estimator, L red because it involves states, and the feedback gain,
K blue because it is part of the controller.

Now the error model is


k+1 = ( LC) k (8.27)

We must select the matrix L such that the observer system LC is stable and has eigenvalues
(observer poles) that are suitably fast. This means that even if we were to get the initial condition
wrong for x0 , we can still use the estimator since any initial error will eventually be forced to
zero. Typically we would choose the poles of the observer system (Eqn. 8.27) much faster than
the openloop system. To be able to solve for L, the system must be observable.

For many chemical process applications, the sampling rate is relatively slow, since the processes
are generally quite slow, so the basic estimator of Eqn 8.26 can be modified so that the prediction
of xk+1 uses the current measurement yk+1 rather than the previous measurement yk as is the case
in Eqn 8.26. This is only possible if the computational time is short, compared with the sample
time, but if this is the case, then the sample period is better utilised. This type of formulation is
termed a current estimator and both are described in [71, pp1478] and in terms of the Kalman
filter in [152, pp87375]. The current estimator is

xk+1 = xk + uk (8.28)

xk+1 = xk+1 + L yk+1 Cxk+1 (8.29)

In practice for most systems, the two forms (current and prediction estimators) are almost equiv-
alent.

The design of the observer gain L by pole-placement is similar to the controller gain K given in
section 8.2, but in the observer case we have the closed loop expression

eig( LC) rather than eig( K)

However we note that


eig( LC) = eig(T CT LT )

since eig(A) = eig(AT ), so we can use the M ATLAB functions place or acker for the observer
problem by using the dual property

L = place(T , CT , )T

Table 8.1 shows the equivalent relationships between the matrices used for observation and reg-
ulation and make the relevant substitution.

Table 8.1: The relationship between regulation and estimation

Control Estimation

B C
K L
Co Ob
eig( K) eig( LC)
8.4. COMBINING ESTIMATION AND STATE FEEDBACK 375

8.4 C OMBINING ESTIMATION AND STATE FEEDBACK

We can combine the state feedback control from section 8.2 with the state estimation from sec-
tion 8.3 into a single algorithm. This is useful because now we can use the estimator to provide
the states for our state feedback controller. For linear plants, the design of the state feedback
controller is independent from the design of the observer. The block diagram shown in Fig. 8.7,
which is adapted from [20, p438], shows the configuration of the estimator, L, and state-feedback
controller, K.

+ q 1 C + output, y


Plant


input, u + q 1 x C +


Estimator

K
State feedback

Figure 8.7: Simultaneous control and state estimation

Remember that in this example, we have taken the perhaps unreasonable assumption that we
have model plant mis-match and we assume that the plant is linear, but we are faced with the
problem that we do not know the initial conditions of the states x0 .

The equations are summarised below. The true plant dynamics are

xk+1 = xk + uk (8.30)

and the measurement is


yk = Cxk (8.31)
but we do not know this true state vector x, so we estimate it with

xk+1 = xk + uk + L (yk Cxk ) (8.32)

so consequently the scalar control law

uk = Kxk (8.33)

can only use what we know, that is the estimate, x, not the actual state, x. Combining Eqns 8.33
and 8.30 gives
xk+1 = xk Kxk (8.34)
376 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

and combining Eqns 8.32 to 8.33 gives the closed loop


xk+1 = ( LC K) xk + LCxk (8.35)
Concatenating Eqns 8.30 and 8.35 together gives the homogeneous equation
    
xk+1 K xk
zk+1 = = (8.36)
xk+1 LC K LC xk
| {z }
cl-ob
I will call the transition matrix for Eqn 8.36 the closed-loop/observer transition matrix (cl-ob ).
Ogata gives a slightly different formulation of the combined observerstate feedback control sys-
tem as Eqn 6-161 in [152, p714]. Ogata also recommends choosing the observer poles 45 times
faster than the system response.

E XAMPLE OF A COMBINED STATE - FEEDBACK AND OBSERVER

Suppose we are given a continuous plant


0.98
G(s) = , T = 0.1
(3s + 1)(2s + 1)
which we will sample at T = 0.1. The design specifications are that the estimator poles are to be
relatively fast compared to the controller poles
   
def 0.8 def 0.92
est = and ctrl =
0.85 0.93
We can design the controller and estimator gains as follows

Listing 8.4: Pole placement for controllers and estimators


G = tf(0.98,conv([2 1],[3 1])); % Blackbox plant
Gss = ss(G); % convert to state-space
3

Gss1 = Gss; % Now create a special version with full outputs


Gss1.c = eye(2); Gss1.d = [0;0];

T = 0.1; Gssz = c2d(Gss,T); % Convert to discrete state-space at Ts = 0.1


8

p = [0.92 0.93]'; % Design controller gain, K, such that . . .


K = place(Gssz.a, Gssz.b, p); % ctrl = eig( K)

pe = [0.80 0.85]'; % Design estimator gain, L, such that . . .


13 L = place(Gssz.a', Gssz.c', pe)'; % est = eig( LC)

sort(eig(Gssz.a-L*Gssz.c)) - sort(pe) % Test results should be zero

which gives controller, K, and observer, L, gains as


 
  0.2146
K= 1.3848 1.6683 and L=
0.4109
Fig. 8.8(a) gives the S IMULINK configuration for the controller which closely resembles the block
diagram given in Fig. 8.7. To test the estimation and control, we will simulate a trend where the
actual and estimated starting conditions are different, namely
   
1 1.6
x0 = , which is not the same as x0 =
1 2
8.5. GENERIC MODEL CONTROL 377

The trends shown in Fig. 8.8(b) show the (discrete) estimated states converging rapidly to the true
continuous states, and that both converge, although slightly less rapidly, to the setpoint.

8.5 G ENERIC MODEL CONTROL

Control of multivariable nonlinear processes such as

x = f (x, u, d, , t) (8.37)

is in general, much harder than that for linear processes. Previously we discussed the simple state
feedback control law; u = K(x x) where x is the desired state vector or setpoint. We showed
that we could augment this simple proportional only law to incorporate integral action to ensure
zero offset to get a control law such as 8.23. Now suppose that we wish to control not x, but the
derivative of x, dx/dt, to follow some desired derivative trajectory (x) . We now wish to find u(t)
such that x = r(t) where r(t) is some reference function that we can specify and where dx /dt is
given by Eqn 8.37.

For any controller, when the process is away from the desired state, x , we want to force the rate
of change of the states, x to be such that it is moving in the direction of the desired setpoint, and
in addition, we would like the closed loop to have zero offset. The control law
Z t
(x) = K1 (x x) + K2 (x x) d (8.38)
0

combines these two weighted ideas. Ignoring any model-plant mistmatch, (x) = x, so substitut-
ing in the plant dynamics Eqn. 8.37 to Eqn. 8.38, we get the control law
Z t

f (x, u, t) K1 (x x) K2 (x x) d = 0 (8.39)
0

Solving Eqn. 8.39 for u is the control law and this scheme is called Generic Model Control or
GMC and was developed by Lee and Sullivan in [117]. Note that solving Eqn 8.39 for u is in
effect solving a system of m nonlinear algebraic equations. This must be done at every time step.
Actually, the solution procedure is not so difficult since the previous solution ut1 can be used as
the initial estimate for ut .

8.5.1 T HE TUNING PARAMETERS

The GMC control law (Eqn 8.39) imbeds a model of the process, and two matrix tuning parame-
ters, K1 and K2 . Selecting different tuning parameters changes the type of response. Assuming
we have a perfect model, the control law (Eqn 8.39) is
Z t
x = K1 (x x) + K2 (x x) d (8.40)
0

If we differentiate once the control law Eqn 8.40, we get

x = K1 (x x) + K2 (x x)
x + K1 x + K2 x = K1 x + K2 x (8.41)
378 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

states yv To Workspace

4 4

4
Mux1

2 2

2
states
K*u input
2 output
ctrl
Gain plant BB output
C model
2
K*u

K* u
estimator Gain1

2 2 1 2
K*u
z
2
Delta
Unit Delay
Phi
K*u

(a) S IMULINK configuration

1 x1 (t)

0.5

0
x1

0.5

1.5 x1 (t)

2 actual
estimated
2 x2 (t)

1.5
2

1 x2 (t)
x

0.5

0
0 1 2 3 4 5 6 7
time
(b) The estimated states converge to the true states which in turn converge to the
setpoint.

Figure 8.8: Simultaneous control and estimation


8.5. GENERIC MODEL CONTROL 379

1.6

increasing
1.4
2
1.2

1.5
1
response

response
0.8 1

0.6 0.5

0.4 =0.2 0
=0.5 0 10
0.2 =1 0.5
=3 5
1
0
0 2 4 6 8 10 1.5 0 normalised time
Normalised time, =1 shape factor

(a) Various choices for shape factor. = 0.2, 0.5, 1 and (b) A 3-dimensional plot of the same relation.
= 3.

Figure 8.9: The normalised ( = 1) response curve for a perfect GMC controller for various choices
of shape factor .

and taking Laplace transforms of 8.41 gives

s2 x + sK1 x + K2 x = sK1 x + K2 x
x(s) K1 s + K2
= 2
x (s) s + K1 s + K2
2 s + 1
= 2 2 (8.42)
s + 2 s + 1

where
1 k1
= , = (8.43)
k2 2 k2
or alternatively
2 1
k1 =
, k2 = 2 (8.44)

Note that Eqn 8.42 is a modified second-order transfer function. The time domain solution to Eqn
8.42 for a step change in setpoint, X (s), is given by
    
t  1 2 t p 1 2 t
2 2 2
1 e 1 + cos + 1 sin
x(t) = 2
, 6= 1 (8.45)
1

For the limiting case where = 1, the solution to Eqn. 8.42 simplifies to just
 
t t
x(t) = 1 + e 1 (8.46)

Fig. 8.9 shows the response for a perfectly GMC controlled process for different . Note that as
, the output response approaches a step response at t = 0, and as 0, the output tends to
x(t) = sin(2t) + 1. This improvement comes, naturally, at the expense of decreased robustness.

Fig. 8.10 shows the comparison between the overshoot, decay ratio and the tuning value. These
values are easily obtained from Fig. 8.9 or analytically starting with Eqn. 8.45. Note that for > 1,
there is no oscillation in the response, and that a 0.22 gives a quarter decay ratio.
380 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

overshoot
1
decay ratio

overshoot & decay ratio


0.8

0.6

0.4

0.2

Figure 8.10: The relation between the shape


factor , the maximum overshoot (solid) 0
and decay ratio (dashed) for the perfectly 0 0.2 0.4 0.6 0.8 1 1.2
controlled GMC response when = 1 shape factor

Algorithm 8.3 Designing the GMC controller

To choose appropriate values for K1 and K2 , do the following:

1. For each state to be controlled, choose an appropriate shape from Fig. 8.9 for the response. In
other words, choose a value. For tight control with little overshoot, choose a shape corre-
sponding to 3 or higher , or if the loop is not so important, choose a more conservative
response such as 1.
2. Specify the time scale by choosing an appropriate value for . Often it is convenient to
specify the rise time. For distillation column control, you may choose a cross over time of
30 minutes, or for a flow controller you may wish to cross the setpoint after only 5 seconds.
The time scale is application dependent.
3. From the specified and , evaluate the scalars k1 and k2 using Eqn. 8.44.
4. The diagonal matrices K1 and K2 are formed by collecting the k scalars into diagonal n n
matrices.

8.5.2 GMC CONTROL OF A LINEAR MODEL

Since GMC can be applied to a nonlinear process, one would expect it to work especially well given
a linear process x = Ax + Bu. Now the GMC control law simplifies from the general Eqn. 8.39 to
 Z t 
u = B1 K1 x (K1 + A)x + K2 (x x) d (8.47)
0
1
noting that B must exist and hence at least should be square. This means that we must have
the same number of manipulated variables as state variables.

It is easy to test such a controller using a stable random model using rmodel. Hopefully our
randomly generated control matrix B is inevitable. (If it is not, just re-run the script file.) To tune
the response, I will select a slightly different shape factor and time constant for each state, namely
1 = 0.7, 1 = 1.5
2 = 0.8, 2 = 2.6
8.5. GENERIC MODEL CONTROL 381

Running the script file in Listing 8.5 gives results much like Fig. 8.11.

Listing 8.5: GMC on a Linear Process

dt = 0.5; n = 2;% sample time & # of states


m = n; p = 1; % ensure square system & 1 output

[A,B,C,D]=rmodel(n,p,m); % generate random continuous model


5 D = zeros(size(D)); C = eye(size(C)); % not interesting
[Phi,Del] = c2dm(A,B,C,D,dt,'zoh'); % discretise

x0 = round(3*randn(n,1)); % nice starting position


t = [0:dt:80]'; % simulation lifetime
10 Xspt = [square(t/8), 0.75*square(t/8*0.7)+3]; % setpoint

% GMC tuning rules


zeta=[0.7 0.8]'; tau=[1.5 2.6]'; % Shape factor , & time constant , for each state.
K1 = 2*diag(zeta./tau); K2 = diag(1.0./tau./tau); % Diagonal K1 , K2 .
15 Ie = zeros(size(x0)); % reset integrator

x= x0; X = []; Uc = [];


for i=1:length(t) % start simulation
err = (Xspt(i,:)-x); % current error (x x).
20 Ie = Ie + err*dt; % update integral Rt
rhs = K1*err + K2*Ie - A*x; % K1 x (K1 + A)x + K2 0 (x x) d
u = B\rhs; % Solve (linear) control law, Eqn. 8.47.

x = Phi*x + Del*u; % Do 1-step integration


25

X = [X; x']; Uc = [Uc;u']; % collect input for plotting


end % for

plot(t,[Xspt, X],t,Uc,'--') % Refer Fig. 8.11.

We would expect the GMC controller to perform well, but what should really convince us is if the
actual response y(t) follows the analytical & desired trajectory, y (t) since GMC is really a model
reference controller. These two curves are compared in Fig. 8.12 which shows an enlarged portion
of Fig. 8.11. The differences between the actual response and the specified trajectory are due to
sampling discretisation.

This demonstration of the GMC controlling a linear process does not do the algorithm justice,
since the controller really is intended for nonlinear processes. This is demonstrated in the follow-
ing section.

8.5.3 GMC APPLIED TO A NONLINEAR PLANT

Section 8.5.2 did not show the true power of the GMC algorithm since the plant was linear and
any linear controller such as an LMR would work. However this section introduces a nonlinear
382 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Linear GMC
5

4 x2

Outputs & setpoints


2

1
x
1
2

3
20

0 u2

20
inputs

40

Figure 8.11: The response of 60


a randomly generated linear u
1
80
process controlled by GMC. 0 10 20 30 40 50 60 70 80
See also Fig. 8.12. time

1.5
output & reference output

0.5

Figure 8.12: Comparing the actual re-


0.5 Actual y
sponse, y, to a setpoint change with the *
Reference y
expected response, y . This response is 1 setpoint
a zoomed part of the response given in
48 50 52 54 56 58 60 62
Fig. 8.11. time

model of a continuously stirred tank reactor, CSTR, from [124]. The CSTR model equations are

Fa Ta Fb Tb (Fa + Fb )T H Uar
T = + r (T Tc ) (8.48)
V V V Cp V Cp
Fa Cai (Fa + Fb )Ca
Ca = r (8.49)
V V
Fc Uar
Tc = (Tci Tc ) + (T Tc ) (8.50)
Vc Vc Cp

where the reaction kinetics are second order

E
r = Ke RT Ca2 (8.51)
8.5. GENERIC MODEL CONTROL 383

Table 8.2: The state, manipulated and disturbance variables and parameter values for the Litch-
field nonlinear CSTR, [124].
(a) State, input and disturbance variables
variable description units
T reactor temperature K
Ca concentration of A kgmol m3
Tc temperature of jacket K
Tci temperature of cooling inlet K
Cai inlet concentration of A kgmol m3
Fb inlet flow of stream B m3 s1
Ta inlet temp. of stream A 50 C (333K)
Fa inlet flow of stream A 3.335 106 m3 s1
Tb inlet temp. of stream B 50 C (333K)
Fc cooling flow 3 104 m3 s1
(b) Model parameters
parameter value units parameter value units
V 0.0045 m3 K 16.6 106 m3 /kmol/s
Vc 0.0074 m3 E/R 5.6452 103 K
1000 kg/m3 H 74.106 103 kJ/kg
Cp 4.187 kJ/kg/K Uar 0.175 kW/K

The state, manipulated and disturbance variables as explained in Table 8.2 are defined as1
def  T def  T def  T
x= T Ca Tc , u= Tci Cai Fb , d= Ta Fa Tb Fc (8.52)

The model parameters are also given in Table 8.2.

The plant dynamics defined by Eqns 8.48 to 8.50 are nonlinear owing to the reaction rate r, being
an exponential function of the temperature T . This type of nonlinearity is generally considered
significant.

D ESIGNING THE GMC CONTROLLER

First we note that have n states and n manipulated variables. Thus the B matrix, or at least
a linearised version of the nonlinear control function will be square. We also note that the GMC
algorithm assumes that the process states (x) are measured, hence the measurement relation must
be C = I. The GMC control law, Eqn. 8.39, for this case is
Fa H Uar

0 0 TbVT Tci V (Ta T ) Cp r C  p
(T Tc )
0 Fa Ca Cai + FaVCa + r


V V
Fc
Vc 0 0 Fb FVc Tc c + VUcarCp (T Tc )
Z t
K1 (x x) K2 (x x) d = 0
0

Now you will notice that the control law is a system of algebraic equations unknown in u. For this
case, the control law is linear in u because I had the freedom of choosing the manipulated variables
carefully. If, for example, I had chosen Fa rather than Fb as one of the manipulated variables, then
1 The original reference had only one manipulated variable T . This was changed for this simulation so the GMC is
ci
now a true multivariable controller.
384 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

the system would be control nonlinear, and hence more difficult to solve. But in this case, the
system is control linear, the GMC control law is a linear system of algebraic equations, and that
the 33 matrix to be inverted is in a lower diagonal form already making the computation straight
forward.

S IMULATING GMC WITH A NONLINEAR PROCESS

We will use a GMC controller to drive the states to follow a setpoint as follows:
 
 300 0.03 290 
0 t < 5 min.

x = 310 0.03 290 5 t < 10 min.

 
310 0.025 295 t > 10 min.
 
starting from an initial condition of x0 = 290 0.035 280 and with a relatively coarse sam-
ple time of T = 10 seconds. We set the shape factor = 0.5, and the time constant for the response
1 1
to = 20 seconds. This gives k1 = 20 and k2 = 400 . The disturbance variables (d) are assumed
constant for this simulation and are taken from Table 8.2.

The state derivative x as a function of x, u, d, , t is evaluated using Eqns 8.48 to 8.50 and is given
in listing 8.7 which is called by the integrator ode45 in listing 8.6.

Listing 8.6: GMC for a batch reactor


1 % GMC CSTR simulation
% Ref: Litchfield, Campbell & Locke Trans I Chem E Vol 57 1979

x0 = [290, 0.035, 280]'; % initial condition


dt = 10; t = [0:dt:15*60]'; % sample time (seconds)
6 Xspt = [300+10*square(t/150,70), 0.03+0.05*square(t/150), ...
295-5*square(t/150,30)]; % setpoint
d = [333 3.335e-6 333 3e-4]'; % disturbance vector constant
Ta = d(1); Fa = d(2); Tb = d(3); Fc = d(4);

11 k1 = 1/20; k2=1/400; % GMC tuning constants


K1 = eye(3)*k1; K2 = eye(3)*k2;

%Parameters for the CSTR


V = 0.0045; Vc = 0.0074; rho = 1000.0;
16 K = 16.6e6; ER = 5.6452e3; dH = -74.106e3;
Cp = 4.187; %kJ/Kg.K
Uar = 0.175; % kW/K
rCp = rho*Cp ;

21 x=x0;Ie=zeros(size(x)); X=[]; U=[];


for i=1:length(t) % start simulation
xspt = Xspt(i,:)'; % current setpoint
T = x(1); Ca = x(2); Tc = x(3); % state variables
Ie = Ie+(xspt-x)*dt; % update integral error
26 r = K*exp(-ER/T)*Ca*Ca; % reaction rate
R = [Fa/V*(Ta-T)-dH*r/rCp-Uar*(T-Tc)/rCp;-(Fa*Ca/V+r); ...
-Fc*Tc/Vc+Uar/rCp/Vc*(T-Tc)];
A = [0 0 (Tb-T)/V; 0 Fa/V -Ca/V; Fc/Vc 0 0];
u = A\(K1*(xspt-x)+K2*Ie - R); % solve for u GMC control law
31 % u = max(zeros(size(u)),u); % ensure feasible

[ts,xv] = ode45('fcstr',[0,dt/2,dt],x,[],u, ...


8.5. GENERIC MODEL CONTROL 385

V, Vc, K, ER, dH, rCp, Uar, Ta, Fa, Tb ,Fc); % integrate nl ODEs
x = xv(3,:)'; % keep only last state
36

X(i,:) = x'; U(i,:) = u';


end %for
plot(t,[X,U]); % will need to rescale

Listing 8.7: The dynamic equations of a batch reactor


1 function x_dot = fcstr(t,x,flag,u, ...
V, Vc, K, ER, dH, rCp, Uar, Ta, Fa, Tb ,Fc);
% CSTR dynamic model, called from gmccstr.m

T = x(1); Ca = x(2); Tc = x(3); % states


6 Tci = u(1); % Inputs, Coolant temp in Kelvin
Cai = u(2); Fb = u(3);

r = K*exp(-ER/T)*Ca2; % reaction rate kmol/m3/s


x_dot(1,1)=(Fa*Ta+Fb*Tb-(Fa+Fb)*T)/V - dH*r/rCp - Uar/V/rCp*(T-Tc);
11 x_dot(2,1)=(Fa*Cai - (Fa+Fb)*Ca)/V - r ;
x_dot(3,1)=Fc*(Tci-Tc)/Vc+Uar*(T-Tc)/rCp/Vc;
return % fcstr.m

Physically the manipulated variables are constrained to be positive, and unless these constraints
are added explicitly to the GMC optimisation control law, negative values for some of the inputs
will occur. In the above example I simply clamp the input when these violations occur. If you
removed the u=max(zeros(size(u)),u); line from the control law, you will have the uncon-
strained version that ignores the physically realistic constraints which is the case shown in the
simulations given in Fig. 8.13.

Fig 8.13 shows the response of the CSTR controlled by the GMC. The upper plots show the state
trajectories and the lower plots give the manipulated variables used. Note that the controlled
response is good, but not perfect as we can still see some interaction between the states. The re-
action concentration Ca particularly seems strongly coupled to the other states, and this coupling
is not completely removed by the GMC controller. We could improve on this response both by
decreasing the sample time, or tightening the controller design specifications.

Just for fun, we can plot a three dimensional plot of the trajectory of the state variables from
the CSTR example under GMC control. Fig. 8.14 shows the actual state trajectory (solid line)
converging to the 4 distinct setpoints, (), in state space. Note that the independent variable time
is removed in this illustration.

A LTERNATIVE MANIPULATED VARIABLES CHOICES

Manipulating the inlet concentration of A or the inlet temperature of the cooling fluid is not very
practical. It is much easier to manipulate flow rates rather than temperatures of concentrations.
Therefore in this situation, a better manipulated variable vector would be u = [Fa , Fb , Fc ] rather
than that used above. Note however that if we had chosen Fa , Fc and Tci as u, we would have
a control problem that is not only nonlinear, but one that is also degenerate. If you construct the
GMC control law of Eqn. 8.39, you will get a structure such as

0 0 Fa Z
0 0 Fc = g(x) + K1 (x x) + K2 (x x) (8.53)
0 Tci
386 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

40

Temp T,T (deg C)


T
30
T
c

C
20

10
0.4

0.35

A
0.3

Conc C
0.25

0.2 C
A

40
Temp T (deg C)

30
Tci
Ci

20

10
30
Conc C , Flow F * 5e4

Figure 8.13: The nonlinear CSTR controlled


by a GMC controller where = 0.5 and 20
b

= 20 seconds and sample time T = 10 10 C


seconds. ai
Ai

Upper: The state trajectories (temperatures 0 F


b
T and Tc , and reaction concentration Ca ).
Lower: The manipulated variable trajecto- 10
0 5 10 15
ries Tci and Cai and 5 104 Fb . time (min)

35

30
Temperature Tc

25

20

15

10
0.5

0.4
40
0.3 35
Figure 8.14: A phase plot of T, Tc and
30
Ca for the GMC controlled CSTR ex- 0.2 25
ample. The state trajectory is the Conc CA 20
solid line, and the setpoints are . 0.1 15 Temperature T
8.6. EXACT FEEDBACK LINEARISATION 387

where we know everything on the right hand side of Eqn. 8.53, (it is a constant for the purposes
of this calculation) and wish to solve for u. Unfortunately, however, we can see from the structure
matrix on the left hand-side of Eqn. 8.53, that Fa appears alone in two equations, and the remain-
ing two manipulated variables appear together in one equation. Thus this nonlinear algebraic
system is degenerate since in general we cannot expect a single unknown Fa to satisfy two inde-
pendent conditions. Note that even though we do not have an overall degree of freedom problem,
(we have three unknowns and three equations), we do have a sub-structure that has a degree of
freedom problem.

Problem 8.5 1. Re-run this simulation and test the disturbance rejection properties of the GMC
controller. You will need to choose some suitable disturbance profile.

2. Re-run the simulation for different choices of GMC tuning constants. For each simulation,
verify that the response is the linear response that you actually specified in the GMC design.
(You may need to decrease the sampling interval to T 2 seconds to get good results.) Try
for example;

(a) = 20, = 1.5


(b) = 5, = 0.5
(c) any other suitable choice.

8.6 E XACT FEEDBACK LINEARISATION

For certain nonlinear systems, we can derive controllers such that the closed loop input/output
behaviour is exactly linear. This is quite different from our traditional approach of approximating
the linearities with a Taylor series approximation and then designing a linear controller based
on the approximate model. Designing exact feedback linearisation uses nonlinear results from
the field of differential geometry, which, to say the least, is somewhat abstract to most engineers.
Lie algebra for nonlinear systems will replace matrix algebra for linear systems, and Lie brackets
and Lie derivatives will be extensions to normal matrix operations. A good tutorial of this topic
with applications of interest to the process industry, and one from where most of this material
was taken, is [105] and the follow up, [106]. A review of nonlinear control for chemical processes
which includes a section on exact feedback linearisation is [30] and the classic text book for this
and other topics in nonlinear control is [191]. Extensions for when the process is not affine in
input, and for systems where the relative degree is greater than unity are discussed in [82].

8.6.1 T HE NONLINEAR SYSTEM

We will only deal with SISO nonlinear systems of the form:

x = f (x) + g(x)u (8.54)


y = h(x) (8.55)

where f , g are vector functions, or vector fields, h(x) is a scalar field and the input u, and output
y are scalars. Note that the system described by Eqn 8.54 is written in affine form, that is the
manipulated variable enters linearly into the system. It is always possible to transform an arbitray
system into affine form by introducing a new variable. For example, given the non-affine system

x = f (x, u)
388 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

we can define a new input, , which is the derivative of our original input, u = , so now our
augmented system becomes      
x f (x, u) 0
= + (8.56)
u 0 1
which is now in affine form. Whether creating a new variable which requires differentiating the
input is practical, or even feasible under industrial conditions is a different matter.

8.6.2 T HE INPUT / OUTPUT FEEDBACK LINEARISATION CONTROL LAW

Given the nonlinear system in Eqns 8.54 and 8.55, we will try to construct a control law of the
form
u = p(x) + q(x) y (8.57)
where y is the desired setpoint. The closed loop system obtained by substituting Eqn 8.57 into
Eqns 8.54 and 8.55 is
x = f (x) + g(x)p(x) + g(x)q(x) y (8.58)
y = h(x) (8.59)
and is linear from reference variable y to actual output y. The dashed box in Fig. 8.15 contains
the nonlinearity, but as far as we are concerned, viewing the system from the outside, it may as
well be a linear black box.

Reference input x x(t)


u(t) output

1
bc bc h()
y(t)
y (t) s

f ()

q(x) p(x)
g()

Nonlinear plant

Figure 8.15: The configuration of an input/output feedback linearisation control law

Note that for a controller in this configuration, the internal states will still be nonlinear. Notwith-
standing, once we have a linear dynamic relation from reference, y (t), to outputs, y(t), it is much
easier to design output-based controllers to control this now linear system, rather than the origi-
nal nonlinear system. It is also clear from Fig. 8.15, that the controller requires state information
to be implemented.

The control design problem is to select a desired linear response, perhaps by specifying the time
constants of the desired closed loop, and then to construct the p(x) and q(x) functions in the
control law, Eqn. 8.57. These are given by the following relations:
(Lrf h(x) + 1 Lr1
f h(x) + + r1 Lf h(x) + r h(x))
p(x) = (8.60)
Lg Lr1
f h(x)
1
q(x) = (8.61)
Lg Lr1
f h(x)
8.6. EXACT FEEDBACK LINEARISATION 389

where the Lie derivative, Lf h(x), is simply the directional derivative of the scalar function h(x)
in the direction of the vector f (x). Here, the Lie derivative is calculated by
n
h(x) X h(x)
Lf h(x) = f (x) = fi (x) (8.62)
x i=1
xi

One can apply the Lie derivative operator repeatedly, L2f = Lf Lf either to the same function, or to
different functions as in Lg L2f .

The relative order of a system such as Eqns 8.548.55 is the smallest integer r such that
Lg Lr1
f h(x) 6= 0 (8.63)
An alternative interpretation of relative order is that r is the smallest order of derivatives of y that
depends explicitly on u. For linear systems, the relative order is simply the difference between
the number of poles and the number of zeros of the system, i.e. na nb .

Finally, the i in Eqn 8.60 are the user-chosen tuning constants that determine how our output
will respond to the command y . They are defined by
dr y dr1 y dy
+ 1 + + r1 + r y = y (8.64)
dtr dtr1 dt
and typically it is convenient to choose Eqn. 8.64 as some sort of stable low-pass filter, such as a
Butterworth filter.

Algorithm 8.4 Exact feedback linearisation design procedure:

1. We are given a single-input/single-output nonlinear model (which may need to be trans-


formed into affine form), Eqn 8.548.55.
2. Find the relative order, r, of the system using Eqn. 8.63. This will govern the order of the
desired response differential equation chosen in the next step. (If r = 1, perhaps a GMC
controller is easier to develop.)
3. We choose a desired linear response, which we consider reasonable, in the form of Eqn 8.64,
that is, we need to choose r time constants or equivalent parameters.
4. We create our exact linearising nonlinear controller, Eqn 8.57 using the relations Eqns 8.60
and 8.61 by constructing Lie derivatives. (A symbolic manipulator such as M APLE may be
useful here.)
5. Closing the loop creates a linear input/output system, which can be controlled by a fur-
ther controller sitting around this system, designed by any standard linear controller design
technique such as LMR or pole placement.(However this is hardly necessary since we have
the freedom to choose any reasonable response in step 3.)

C AUTIONS AND RESTRICTIONS

This technique is not applicable if the zero dynamics are unstable. This is analogous to the
linear case where we are not allowed to cancel right-hand plane zeros with unstable poles. Tech-
nically, it works, but in practice, the slightest model mis-match will cause an unstable pole which
in turn will create an unstable controller. This then excludes process systems with dead time.
Clearly also owing to the nature of the cancellation of the process nonlinearities, exact feedback
linearisation is very sensitive to modelling errors. The technique demands state feedback, but
only delivers output linearisation. Full state linearisation techniques do exist, but exist only un-
der very restrictive conditions.
390 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

8.6.3 E XACT FEEDBACK EXAMPLE

Building an exact feedback controller is relatively easy for well behaved smooth nonlinear sys-
tems. The only calculus operation required is differentiation, and this poses few problems for
automated computer tools. This example shows how to build such a controller using a simple
program written for a symbolic manipulator. With this program, we can subsequently change
the system, and regenerate the new nonlinear control law. This is a departure from all the other
simulations presented so far, since here we can change the entire structure of the process in our
program, not just the order or the values of the parameters, and subsequently re-run the simula-
tion.

Suppose we have a simple 3-state nonlinear SISO system,


x1 sin(x2 ) + (x2 + 1)x3 0
x2 = x51 + x3 + 0 u (8.65)
x3 x21 1
| {z } | {z }
f (x ) g (x )
 
y= 1 0 0 x (8.66)
| {z }
h(x)

which is already in the form of Eqn. 8.54Eqn. 8.55 or affine form, and we wish to construct an
exact feedback linearised system following Eqn. 8.57. To do this we will follow Algorithm 8.4.

Since we will need to compute some Lie derivatives analytically,

h(x)
Lf h(x) = f (x)
x

to establish the relative order r, it may help to write a small procedure to do this automatically.
Listing 8.8 gives an example using the Symbolic toolbox.

Listing 8.8: Find the Lie derivative for a symbolic system


function L = lie_d(F,h,x)
2 % Find the Lie derivative Lf h(x)
% Note jacobian returns a row vector
h(x)
L = F'*jacobian(h,x)'; % f (x) x
return

First we define the actual nonlinear functions f (x), h(x), and g(x) for our particular dynamic
problem.

syms x1 x2 x3 ystar real % Specify symbolic variables as real or will have conjugate problems
x = [x1 x2 x3]';

F = [sin(x2)+(x2+1)*x3; x15+x3; x12]; % Eqn. 8.65


5 h = [1 0 0]*x
g = [0 0 1]'
8.6. EXACT FEEDBACK LINEARISATION 391

The Lie derivatives are easy to construct,

h(x)
Lf h(x) = f (x)
x
  1
= sin(x2 ) + (x2 + 1)x3 , x51 + x3 , x21 0
0
= sin(x2 ) + (x2 + 1)x3

and check in M ATLAB using the routine giving in Listing 8.8

>> lie_d(F,h,x)
ans =
sin(x2)+(x2+1)*x3

To find the relative degree, we simply repeatedly call the function in Listing 8.8 starting at r = 1
and incrementing r until Lg Lr1
f h(x) 6= 0 at which point we have established the relative order.

Listing 8.9: Establish relative degree, r (ignore degree 0 possibility)


Lfn = h % Dummy start
2 LgLfnh = lie_d(g,h,x) % Lg L1f h(x) case.
r = 1; % # relative degree counter

while LgLfnh == 0
r =r+1; % increment relative degree counter, r
7 Lfn = lie_d(F,Lfn,x) % need to build-up
LgLfnh = lie_d(g,Lfn,x) % stop when non-zero
end % while

Now that we know the relative degree r = 2, we can design our desired response filter

Listing 8.10: Design Butterworth filter of order r.


1 [z,p,K] = buttap(r) % Low-pass Butterworth filter analogue prototype
beta = real(poly(p));
%[B,beta] = butter(r,2,'s') % Alternative: but must scale with B(end).

Now we can construct the controller components p(x) and q(x) to form the closed loop.

Listing 8.11: Symbolically create the closed loop expression


qx = 1/LgLfnh; % Controller components q(x), p(x) , Eqn. 8.60, Eqn. 8.61.
2 px = beta(r+1)*h;
Lfn = h;
for i=r-1:-1:0
Lfn = lie_deriv(f,Lfn,x);
px = px+ beta(i+1)*Lfn;
7 end
px = -px*qx;

xdot = f+g*(px+qx*ystar) % Closed loop, Eqn. 8.58, hopefully linear.

In practice, we are only practically interested in the control law, Eqn. 8.57,
392 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

3
y
y 3
y* u 3
x Scope
Signal multiply states 3
Generator Nonlinear Plant 1
Out1

y
1./(u(2)+1) q(x) p(x)
px

u
3 3

butter

yref

Analog
Filter Design

(a) S IMULINK configuration. Compare with Fig. 8.15.

1.5

0.5 actual y
y=x1

0.5
desired trajectory
1

1.5
10
x 10
2
error between trajectory and actual
1

1
0 10 20 30 40 50 60 70 80 90 100
time

(b) The closed loop response of the controlled nonlinear system compared to the desired linear
response

Figure 8.16: This example of exact feedback linearisation shows that there is no practical differ-
ence between the output of the nonlinear system and the desired linear filter.

>> (px+qx*ystar) % Control law, u = p(x) + q(x)y


ans =
(-4*x1-2*2(1/2)*(sin(x2)+(x2+1)*x3)-(x15+x3)
*(cos(x2)+x3)-x12*(x2+1))/(x2+1)+1/(x2+1)*ystar

which could, in principle, now be implemented on the nonlinear plant.

A good test for this controller is to see if the closed loop response is indeed indistinguishable
from the specified linear filter. As demonstrated in Fig. 8.16, there is no discernable difference
indicating that the nonlinear controller design worked as intended in this instance.

Problem 8.6 Create an exact nonlinear feedback controller for a penicillin fermentation process.
8.6. EXACT FEEDBACK LINEARISATION 393

The system dynamics are:

dX
= (S, X)X DX (8.67)
dt
dS
= (S, X)X + D(Sf S) (8.68)
dt
y=X (8.69)

where X, S are the cell mass and glucose concentration, D is the dilution rate and manipulated
variable, and Sf is the feed glucose concentration, considered constant in this example. The
empirical growth rate expressions are:

x S
(S, X) = (8.70)
Kx X + S
(S, X) = (S, X)/Yxs + (S)/Yps + m (8.71)
p S
(S) = (8.72)
Kp S(1 + S/Ki )

and for simulation purposes, use parameters values as

x 1.5 Kx 2.34
p 3.45 Kp 1.56
Yxs 0.45 Yps 0.78
Ki 0.89 m 0.34

and manipulated, disturbance and initial conditions as

manipulated D 0<D<1
disturbance Sf 1.25 could also be varied
initial conditions X0 0.05
S0 0.6

Choose an appropriate and design a linearising controller using Eqn 8.57. Test the closed loop
response by simulation and verify that your input/output response is actually linear.

Problem 8.7 1. What is the relative order of the following nasty system from 3.4.1?

x1 = x2
x2 = 6x1 7x2 + 24x3 + 10x22 + u
x3 = x3 x2 x52
y = x1

2. Implement and simulate an exact linearising controller for this system.

3. What problems do you face when you try to construct an input/output GMC controller for this
system? What requirement therefore must be satisfied to apply GMC?

4. Choose a suitable alternative measurement function such that GMC will work, and imple-
ment this as a M ATLAB simulation. Compare the performance of this controller with one
developed using geometric methods.
394 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

8.7 S UMMARY

This chapter introduced the theory and technique of the design of state space control systems.
Writing the dynamic equations in a discrete matrix form is both convenient for complex systems,
and intuitive since the design is done in the time domain. The major disadvantage is the fact that
the computation is more complex, and is the reason why the design is typically carried out on a
computer. The important conclusions of this chapter are:

Controller design by pole placement means that we choose a gain matrix K, such that the
closed loop system ( K) has a desired dynamic response or alternatively has the de-
sired eigenvalues. To do this, the system must be controllable.
We can design observers L in much the same way as controllers.

However, in this chapter we only covered the design of single inputmultiple output controllers
or single measurement - multiple state observers. The true multivariable case is under-constrained,
and is covered in chapter 9.

Control of a nonlinear process may require a different approach and typically more computation.
Generic model control or GMC is a simple technique that can be used to control a nonlinear plant
in an optimal manner. Many of the so-called difficult processes in chemical process control are
due to nonlinearities and these are well suited to techniques such as GMC. An alternative, more
general nonlinear controller, particularly suitable for SISO nonlinear systems in control affine
form is one where a controller is sought that transforms the nonlinear system into a linear open-
loop system that can subsequently be controlled using the far more extensive library of linear
controllers.
C HAPTER 9

C LASSICAL OPTIMAL CONTROL

God does not throw dice ? Nor is it our business to prescribe to God how He should run the
world.
Attributed to Niels Bohr replying to Einstein.

9.1 I NTRODUCTION

This, and the following chapter, is concerned with designing optimal controllers. In practice opti-
mal controllers are complicated owing to issues such as plant nonlinearities, constraints on state
and manipulated variables for example, but for some applications the improvement in controlled
response is well worth the implementation effort. Following a small aside, demonstrating opti-
mal tuning of a known controller, we will begin with the general optimal control problem in 9.3,
develop a solution strategy and test it on a batch reactor example in 9.2.4. Building on the gen-
eral nonlinear optimisation material, we will make some simplifying assumptions, and develop
an optimal linear controller in 9.4

Chapter 8 described the pole placement method of controller design for single input/single out-
put (SIMO) systems. It was also briefly mentioned that the evaluation of the gain matrix K for
MIMO systems is now under-constrained and an infinite number of possible controller configura-
tions satisfy the original design specifications. However, by using these extra degrees of freedom,
optimal controllers are possible. Section 9.4 will describe an optimal multivariable controller
termed a Linear Quadratic Regulator (LQR). Section 9.5 will present the dual problem of an opti-
mal estimator, the Kalman filter.

Model predictive control, which is the subject of chapter 10, lies somewhere between the full
generic optimal control problem, and the linear quadratic control. This chapter introduces the
concept of a moving horizon for the optimisation and compares a computationally intensive but
very general nonlinear optimal scheme with a much faster linear optimal controller based on a
FIR model.

9.2 PARAMETRIC OPTIMISATION

The simplest type of optimisation problem is known as parametric optimisation where we at-
tempt to find the best values of a finite number of parameters. Finding the best three PID tuning
395
396 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

constants to minimise say the integral of square error is an example of parametric optimisation
described in 9.2.1.

Alternatively we could try to find the best shape of a trajectory for a missile to hit a target whilst say
minimising energy. In principle the shape may require an infinite number of discrete parameters
to completely describe it, perhaps using a Taylor series. This then becomes an infinite dimensional
parametric optimisation problem, so we must tackle it with other techniques such as variational
calculus.

9.2.1 C HOOSING A PERFORMANCE INDICATOR

Optimal control is where you try to extremetise something to get the best controlled result. To
optimise, we must at least be able to quantify the controlled performance, that is to put numbers
to the quality of control. Typically however, this assessment is, in practice, a tricky function of
many variables such as process economics, safety customer satisfaction etc, which are all very
difficult to quantify.

We call the value that quantifies the performance the objective function, and typically use the
symbol J . Some authors use the term performance index or perhaps profit function, and natu-
rally we would like to maximise this quantity. However by convention most numerical software
packages are designed to minimise the objective function, and we certainly do not want to min-
imise the profit function, but rather minimise the cost function. Consequently the term cost
function is often used in optimisation where it is assumed that we are interested in minimising
the objective function. For our purposes, where we are typically interested in minimising the
error residuals, we will be using cost function for our objective function.

A simple example of optimal control is the optimal tuning of a PID controller for a given process
responding to a setpoint change. First we must choose an objective function that we wish to opti-
mise, often some sort of cost term (which we want to minimise), and then we search for the tuning
variables that minimise this cost. Since it is sometimes difficult to correlate cost to performance
in a simple manner, we often choose a performance objective more easily quantifiable such as the
integral of the difference between the actual process and the setpoint. This is termed the integral
of the absolute error or IAE performance index,
Z
JIAE = |t | dt (9.1)
0

where t is defined as the difference between setpoint and actual, rt yt at time t. Classically con-
trol engineers have tried to design closed-loop control systems that behave similarly to a second-
order, slightly under-damped process. From experience this is found to be a reasonable trade-off
between speed of response, overshoot and ease of design. We can plot both the process response,
and visualise the area given by Eqn. 9.1 we wish to minimise using the floodfill command fill.

1 t = linspace(0,10)';
[num,den] = ord2(1,0.54); % Choose = 0.54
y = step(num,den,t); % do simulation
fill([t;0],[y;1],'c') % floodfill area as shown in Fig. 9.1.

A damping factor of = 0.54 is near the recommended value.

Other common performance index definitions are given in Table 9.1. Classically the least-squares
index, ISE, was popular because for simple problems the solution for the optimum is analytical.
However the so-called robust measures such as IAE or ITAE are now popular, although a nonlinear
optimiser is required to find the optimum tuning parameters.
9.2. PARAMETRIC OPTIMISATION 397

1.4

1.2

controlled response 1

0.8

0.6

0.4

0.2 Figure 9.1: The area of the shaded fig-


ure is the approximate value of the per-
0 formance integral. You could try other
0 2 4 6 8 10
time values for , say 0.1.

Fig. 9.2 shows the steps to numerically compute the ITAE performance index for a given con-
trolled response. In this case the ITAE is 3.85 cost units at time t = 10. Ideally we need to wait
until the final plot of the intergral stops increasing, but in this case we can see that while we have
stopped slightly prematurely, there is little to be gained from increasing simulating much longer.
Notice how the time weighting emphasizes deviations late in the response.

ITAE performance
1.5
y and setpoint

0.5

0
0.5
error=(yr)

0.5

1
1
|(yr)|

0.5

0
1
|(yr)|*t

0.5
Figure 9.2: A breakdown of the ITAE
0 performance index. The top plot
ITAE=3.8
4 shows the controlled response, and
the second plot the error. The third
|(yr)|*t

2 and fourth plots show the difference


between the IAE and the ITAE. The
cumulative integral under the curve
0
0 2 4 6 8 10 in the fourth plot (shaded portion)
time gives the overall performance.

We can compare the performance of a second order process with different damping ratios to find
the optimum, , for a given criteria.

1 t = linspace(0,10)'; dt = t(2)-t(1);
398 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

zeta = linspace(0.01,1.5,100)';% trial values


Perf = [];
for i=1:length(zeta); % For each . . .
[num,den] = ord2(1,zeta(i)); % 2nd order
6 y = step(num,den,t); % do simulation

err = 1-y; % error


Perf(i,:) = trapz(t,[abs(err) err.2 t.*abs(err) t.*err.2]);
end % for
11 plot(zeta,Perf) % Find optimum for each index as shown in Fig. 9.3.

These curves are given in Fig. 9.3 from which we can extract the optimum damping ratios which
are summarised in Table 9.1. Note that the optimum are all in the 0.5 to 0.75 region. Ogata, [154,
pp295-303], and especially Fig 438 on page 298 compares these alternatives in a similar manner.

Optimum tuning of a 2nd order response


1.4

1.2

1
1 IAE
10
Cost factor

0.8
output

ITAE
ITSE 0.6

ISE 0.4 ISE


0 ITSE
10 0.2 IAE
ITAE
0
0 0.5 1 1.5 2 0 2 4 6 8 10
Damping factor, time

Figure 9.3: Various performance criteria as a function of damping ratio for a second order plant.
The minima of the various choices are listed in Table 9.1.

Table 9.1: Common integral performance indices and the optimum shape factor, , assuming a
second order prototype response.

Name J comment
Z
Integral Squared Error, ISE 2t dt 0.507 Analytical, Gauss noise
0
Z
Integral Time Squared Error, ITSE t2t dt 0.597 Encourages ISE to stabilise rapidly
0
Z
Integral Absolute Error, IAE |t | dt 0.657 Recommended index
0
Z
Integral Time Absolute Error, ITAE t|t | dt 0.748 Similar to ITSE
0
9.2. PARAMETRIC OPTIMISATION 399

9.2.2 O PTIMAL TUNING OF A PID REGULATOR

For example, say we wish to tune a PID controller to minimise the integral of the absolute error
times time of a response to a step change in setpoint. For this example I will choose the ITAE
performance index, which is a slight modification on the IAE performance index given previously
in Eqn. 9.1, Z
J = t |t | dt (9.2)
0

The parameters, , we can adjust in this example are the PID tuning constants Kc , i and d ,
while the controller structure is fixed as a three term PID controller. The optimisation problem is
to choose suitable parameters such that the scalar J is minimised subject to the constraint that

Y (s) Gc Gp
= (9.3)
R(s) 1 + Gc Gp

where R(s) = 1/s. Here Gc is assumed a PID controller, and we will also assume that the manipu-
lated variable is unbounded. Since we chose the ITAE performance index, (Eqn 9.2), no analytical
solution will exist for the optimisation. The ITAE is a good performance index, but difficult to
analytically manipulate hence the solution is best achieved numerically.

First the performance index, J , must be written in terms of the parameter vector . To do this,
we must insert our particular process model into Eqn. 9.3 and also substitute the PID controller
transfer function with the unknown parameter vector . One can then algebraically invert Eqn 9.3
to obtain a time domain solution yt if the model is sufficiently simple, or we could simulate the
closed loop system for a given . The error t is yt yt where yt is the reference setpoint, in this
case 1. One then substitutes this equation into Eqn 9.2 and numerically integrates to calculate the
performance index J . Finally any numerical optimisation routine can be used to locate the mini-
mum J as a function of the PID tuning parameters. Many such general purpose optimisers exist
such as Powells conjugate directions, Nelder-Mead, simplex etc. (See also [206] for optimisation
routines in M ATLAB.)

The hard part in attempting to optimally tuned PID controllers is locating the optimal solution.
In the above description, it was done via simulation using an iterative numerical technique. This
solution technique is completely general and can be applied to systems with nonlinear process
models, bounds on the manipulated variables and complex performance indices. It is however
often slow and computationally expensive, and prone to fall into local minima etc, as any numer-
ical technique.

Example: Optimal PID tuning Suppose we wish to optimally tune a PID controller given the non-
minimum phase plant,
3s + 1
Gp (s) = 2 (9.4)
(s + 0.8s + 0.52)(s + 4)(s + 0.4)
We can see the non-minimum phase from the right-hand zero s = +1/3 from the numerator term.
Gp (s) has two complex and two real poles.

Simulating the openloop model gives us some information about the severity of the inverse re-
sponse and the dominant time constants.

>>Gp = zpk(1/3,[-0.4, -4, -0.4+0.6i, -0.4-0.6i],-3)


Gp =

4 -3 (s-0.3333)
---------------------------------
(s+0.4) (s+4) (s2 + 0.8s + 0.52)
400 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Step Response

1.5

Amplitude
0.5

0.5
>>step(Gp) % 0 5 10
Time (seconds)
15 20

We can use fminsearch (which uses Nelder-Meads simplex method), or one of the optimisation
routines from the O PTIMISATION toolbox to do the searching for parameters. Listing 9.1 returns
the IAE performance for a given plant and PID controller tuning constants subjected to a step-test.

Listing 9.1: Returns the IAE performance for a given tuning.


function [j,y] = foptpidt(x,Gp);
2 % Computes the IAE for a given PID tuning
% Called via a numerical optimiser via optpidt.m
Kc = x(1); taui=1/x(2); taud=x(3); % PID controller parameters
Gc = tf(Kc*[taui*taud, taui, 1],[taui, 0]); % PID controller
Gcl = feedback(Gc*Gp,1); % closed loop Gc Gp /(1 + Gc Gp )
7

tsim = 50; dt=0.1; t = [0:dt:tsim]';


y = step(Gcl,t); % Now do simulation
R 50
j = sum(abs(y-1))*dt; % J (x) = 0 || dt
return

The main script file that calls the optimiser to establish the best PID tuning constants is given in
Listing 9.2. Here we specify the plant of interest which we would like to tune, Eqn. 9.4, some
trial PID constants as a first estimate, and then we call the numerical search routine to refine these
estimates.

Listing 9.2: Optimal tuning of a PID controller for a non-minimum phase plant. This script file
uses the objective function given in Listing 9.1.
Kc = 0.5; taui=0.2; taud=1.; % trial start PID constants
x0 = [Kc,taui,taud]'; % trial tuning vector
3s1
Gp = zpk(1/3,[-0.4,-4,-0.4+0.6i,-0.4-0.6i],3);% Gp (s) = (s2 +0.8s+0.52)(s+4)(s+0.4)
4
R 50
Perf = @(x) foptpidt(x,Gp); % J (x) = 0
|| dt

optns = optimset('TolFun',1e-2,'MaxIter',200); % optimisation tolerances


x = fminsearch(Perf,x0,optns) % Return optimal Kc , 1/I , D
9

[jbest,y] = foptpidt(x,Gp,jtype); % best-shot


t = [0:0.1:50]';
plot(t,y,t,ones(size(y)),'--'); % Refer Fig. 9.4.

In the above example we used the IAE performance objective, but we could just as well have
chosen other alternatives such as ISE or ITAE, and we would see slightly different optimal tuning
parameters. The following table gives the values to which I converged for the three commonly
used performance indices.

Performance Kc reset , 1/i d


IAE 0.407 0.382 1.43
ISE 0.359 0.347 1.93
ITAE 0.388 0.317 1.17
9.2. PARAMETRIC OPTIMISATION 401

A comparison of the closed-loop response with a PID controller using these tuning constants is
given in Fig. 9.4. You can see that all responses are similar, although I personally favour the ITAE
alternative.

1.2
IAE
1

0.8
ITAE
Output & setpoint

0.6
ISE
0.4

0.2

0 Figure 9.4: A comparison of the op-


2 = 7.100
timal PID tuning for a step response
0.2 || = 7.267
t|| = 7.352 of a non-minimum phase system us-
0.4 ing IAE, ISE and ITAE performance
0 5 10 15 20 25 30 35 40
Time objectives.

I MPLEMENTATION DETAILS

To be honest, I have cheated somewhat in the above example. Optimisation, even for small di-
mensional problems such as this is always difficult and typically needs some expertise to get
working properly. For the above case, even though I chose good starting guesses for the tuning
constants, the optimiser still had some numerical conditioning problems while searching for the
solution. We can avoid these problems partly by encouraging the optimiser to search in locations
that we think the solution will be, and also by telling the optimiser explicitly parameter spaces
to avoid such as negative values. If the optimiser tentatively tries a negative controller gain, the
closed loop system will be unstable, and the objective function meaningless. However the com-
puter program does not know this, and will still try to extract an improved estimate from this
information. Typically in these types of examples, the objective function loses any real depen-
dance on the parameters, and the response surface becomes flat, or worse, vertical! This is the
principle cause behind the poor conditioning warnings M ATLAB gives sometimes.

The final point to watch out for in an optimisation problem such as this is that the optimum does
not lie far outside any reasonable tuning values. For first or second order systems, the optimum
controller gain is infinity, and the optimiser will naturally tend in this direction. In this case either
constraints are needed, or the objective function can be modified to take into account the vigorous
controller action, or a new more complex model can be developed.

Optimal PID tuning #2 Finding optimal PID tuning constants for actual plant equipment is a good
example of practical optimisation. In this example, we wish to find the best Kc and i values for a
step response of the black box,

0.002129s2 + 0.02813s + 0.5788 0.5s


Gbb (s) = e ,
s2 + 2.25s + 0.5362

presented in Fig. 1.5 using an ISE performance criteria.

A contour of the performance surface is given in Fig. 9.5 from which we can directly identify the
minimum corresponding to the tightest control is obtained when using Kc 4.4 and i 9.8. For
comparison, you can see from the insert plots of the error, that for small gains and large integral
times, the response is too sluggish, while for large gains and small integral times (large integral
actions), the response is overly oscillatory.
402 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

30

0.2
J=1.41

0.4
0.5

0.2

0.3
0.5

0.6
25

0.3
J=5.63
20 0.4
Integral time, i

0.1
0.2

0.1
0.3
15
0.4
0.5
0.6

0.2

0.3
0.3

0.1
0.2
10 J=1.14
0.1
0.2

J=1.98 0.1
5
0.4 0.5
0.1 0.2 0.3 0.6
0.4
0.6

J=2.09
0.5

0.2 0.3 0.4 0.5


0.3 0.4 0.5 0.6
0
0 1 2 3 4 5 6 7 8
Controller gain, K
c

Figure 9.5: Optimum PI tuning of the blackbox plant. The contour plot shows the ISE performance
for a range of gains and integral times while the inserts show the actual error given a step change
in reference at those specific Kc , i combinations. The optimum tuning constants are Kc = 4.4
and i = 9.8.

Naturally the optimum is only for a step test using our specific model. However the results for
other similar input scenarios should be similar. To construct the contour plot of Fig. 9.5 we used
an exhaustive grid search which requires lots of trials, hence the use of a model. An alternative,
and more efficient strategy is to use a nonlinear numeric optimiser which is the subject of the next
section.

9.2.3 C ALLING S IMULINK FROM INSIDE AN OPTIMISER

In cases where we have a complex model with nonlinearities or time delays (such as the black
box model of Fig. 9.5), it may be more convenient to use S IMULINK to compute the model outputs
inside the objective function of an optimiser. In this case we use the sim command to execute a
S IMULINK simulation from within the objective function, but we must take care to tell S IMULINK
to obtain its parameters from the current workspace as opposed to the global or base workspace
which it will do by default. Further implementation details for these types of optimisation prob-
lems using S IMULINK are given in [60, 8.6].

Example: Find the optimum PI tuning constants for controlling the plant

2
G(s) =
0.25s2 + 0.6s + 1
9.2. PARAMETRIC OPTIMISATION 403

The S IMULINK model of the closed loop (named in this example sopt_pi.mdl) is given in Fig. 9.6
and uses the variables K and taui as the controller constants. The objective function is called via

1
error

K*[taui 1](s) 2
taui.s 0.25s2 +0.6s+1
Step PI ctrl Scope
Plant

Figure 9.6: S IMULINK model where the PI tuning constants are to be optimised. Note that we
export the error in an output block at the highest level and that the PI controller uses parameters
K and taui which are to be optimised.

the optimiser, (fminunc in this case), which in turn executes the S IMULINK model from Fig. 9.6
and returns the time-weighted integral of the square error as a performance index is given in
listing 9.3.

Listing 9.3: Returns the ITSE using a S IMULINK model.


function j = foptslink_pi(x)
% Optimisation objective function that will call a S IMULINK model
3 K = x(1); taui = x(2); % parameters to be optimised, x
t0 = 0; tf = 20; h = 0.1; % start & finish times

% Key line: Instruct S IMULINK to obtain its parameters from the local, function workspace
opts = simset('SrcWorkspace','current','DstWorkspace','current');
8

[t,states,error] = sim('sopt_pi',[t0:h:tf],opts); % run simulink model


j = trapz(t,t.*error.2); % compute ITSE
return

From the M ATLAB command line we call a general unconstrained optimiser to compute the two
optimum PI tuning constants.

>> x0 = [0.2, 0.1]'; % Start guess for Kc and i .


>> x = fminunc(@foptslink_pi,x0) % See function in Listing 9.3.

4 Optimization terminated successfully:


Current search direction is a descent direction, and magnitude of
directional derivative in search direction less than 2*options.TolFun
x =
66.5402
9 25.6549

The optimisation routine found that a gain of Kc = 66.5 and an integral time of i = 25.7 min-
imised our integral-time squared error performance index.
404 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

9.2.4 AN OPTIMAL BATCH REACTOR TEMPERATURE POLICY

This example illustrates how to establish an optimal input profile u(t) for a given dynamic system
and initial conditions. In this section we are going to approximate the smooth profile with a small
number of constant variables and we are going to use parametric optimisation to obtain a slightly
sub-optimal solution. Later in section 9.3.4 we will repeat this optimisation problem, but solve
for the continuous profile using functional optimisation.

Suppose we have a batch reactor as shown in Fig. 9.7 in which two chemical reactions are carried
out in series,
k1 k2
A B C (9.5)

The first reaction produces the desired product, B, which is what we wish to maximise after
the two hour reaction duration, but the second reaction consumes our valuable product, and
produces a poisonous byproduct C, which naturally we wish to minimise.

A Stop reaction after 2 hours

product (at end), x(tf )

steam heating, u(t) condensate (cold)


heating coil

Figure 9.7: Production of a valuable chemical in a batch reactor.

In this case, the only manipulated variable is the reaction temperature, u(t), which we can ad-
just over time by varying the heating input. The concentrations of A and B are the two states:
def
x = [ca , cb ]T . As the temperature varies, so does the reaction rates since they are governed by the
Arrhenius rate constants (k1 , k2 ), which in turn changes the ratio of desired product to by-product.
The plant dynamics are given by the reaction kinetics and initial conditions are

dx1
= 2e6/u x1 , x1 (t = 0) = 0.9
dt
dx2
= 2e6/u x1 5e11/u x2 , x2 (t = 0) = 0.1
dt
Our objective for the operation of this reactor is to maximise the amount of B after 2 hours, by
adjusting the temperature over time as the reaction progresses, or
Z tf =2
J (u) = x2 (tf = 2) + 0 dt (9.6)
0

The reason for the seemingly superfluous greyed out integral in the performance objective above
will become evident when we look at the standard optimal control objective function in the next
section.

If we decide to operate at a constant temperature for the entire 2 hour batch, then Fig. 9.8(a)
shows us which temperature is the one to maximise the amount of B at the end of the reaction.
9.2. PARAMETRIC OPTIMISATION 405

Alternatively, if we decide to run the batch reactor for 1 hour at one temperature, and the second
hour for another then we would expect the optimum to improve. Fig. 9.8(b) shows the three
dimensional response surface for this case.

(a) Optimisation of a single constant temperature (b) The response surface of the objective function when
using 2 temperatures

(c) The response volume of the objective function when using 3 tempera-
tures

Figure 9.8: Visualising the objective function using one, two, and three different temperatures
throughout the 2 hour batch reaction. Compare with Fig. 9.9.

We could in principle continue this discretisation of the input profile. Fig. 9.8(c) shows the sit-
uation when we decide to use three temperatures equally spaced in time (40 mins each) across
the 2 hour duration of the batch. In this case the objective function becomes a response volume so
we must use volume visualisation techniques to see where the optimum lies. Fig. 9.9 compares
the optimum trajectories for the case where we select 1,2,3,5 and 7 equally spaced temperatures
divisions. Also given is the limiting case where the number of temperature divisions tend to .
The analytical computation for this latter case is given in section 9.3.4.
406 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

7
analytical T* profile
6
5 T constant
4

7
6
5 2 divisions
4

7
6
5 3 divisions
4

7
6
5 5 divisions
4

7
Figure 9.9: The optimum temperature pro- 6
Temperature

files for cases where we have different num-


5 7 divisions
bers of temperature divisions. Note how
as the number of allowable temperature di- 4
visions increase, the discretised trajectory
converges to the smooth analytical opti- 0 0.5 1 1.5 2
mum profile. Time [hrs]

9.3 T HE GENERAL OPTIMAL CONTROL PROBLEM

The optimal control problem has the following characteristics:

1. A scalar performance index or cost function, J (u) that is to be minimised (or maximised)
by adjusting the manipulated variables u(t).

2. Constraints that must be either:

(a) continuously satisfied, that is the constraint must be satisfied for all time, or
(b) satisfied only at the end of the optimisation problem, a terminal constraint.

These constraints can either be hard (they must never be violated under any conditions, or
soft (they can be violated under special conditions).
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 407

3. The optimisation horizon is the length of time over which the optimisation is allowed to
take place. Sometimes the horizon is infinite, but in practical cases the horizon has some
upper limit over which an optimal solution is sought to minimise the objective function.

The optimal control problems can be written as a standard mathematical optimisation problem,
thus many algorithms and therefore computer programmes exist to give a solution. Two common
solution procedures are Pontryagins maximum principle and dynamic programming. [64, pp16
20] or [53] covers these aspects in more detail and a simple introduction to Pontryagins maximum
principle is given in [175, Chpt 13] and [171, pp 84105].

The optimal control problem as described above is quite general, and the solution will be different
for each case. Section 9.3.2 develops the equations to solve the general optimal control problem
using variational calculus and Section 9.2.4 demonstrates this general approach for a simple non-
linear batch reactor application. The accompanying M ATLAB simulation clearly shows that the
computation required is involved, and motivates the development of less general linear optimal
control designs. This results in the Linear Quadratic Regulator (LQR) described in subsequently
Section 9.4.

There are many other texts that describe the development of optimal controllers. Ogata has al-
ready been mentioned, and you should also consult [100, p226] for general linear optimal control,
[166, 170, 171] for chemical engineering applications and [40] is considered a classic text in the
optimisation field. A short summary available on the web is [47]. A survey really highlighting
the paucity of applications of linear quadratic control, (which is a simplified optimal controller
for linear systems), in the process industries is [97].

9.3.1 A REVISION OF MULTIVARIABLE CALCULUS

This section reviews some basic theory of the derivatives of scalar and vector functions with
respect to a vector. We used this information when developing the least-squares solution to poly-
nomial fits in section 3.3.1.

In the following, we will follow the convention that the derivative of a scalar function V with
respect to a vector x,  
dV dV dV dV
= (9.7)
dx dx1 dx2 dxn
is a row vector. (I have deliberately chosen the symbol V for this scalar since one can immediately
recognise this as a voltage-like variable, since voltage is a scalar quantity.)

The Jacobian, J, is a matrix of derivatives of an m vector function f with respect to an n vector x,


is and (m n) matrix

df1 df1 df1

dx1 dx2 dxn

df2 df2 df2

dx
J= 1 dx2 dxn
.
.
.

df dfm dfm
m

dx1 dx2 dxn
which can also be thought of as a series of individual gradients stacked on top of each other.

Using these definitions, we can derive some common derivatives. The derivative of a (constant)
matrix times a vector is
d(Ax)
=A (9.8)
dx
408 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

and makes intuitive sense.

The derivative of a quadratic form is


d(xT Ax)  
= xT AT + A (9.9)
dx
= 2xT A if A is symmetric (9.10)
and can be proved by using the chain rule.

Note that some authors define the gradient as a column vector hence their corresponding deriva-
tives are transposed from those given above.1

9.3.2 T HE OPTIMAL CONTROL FORMULATION

To develop our optimal controller, we need to define, as listed in section 9.3, an objective func-
tion, a feasible search space, and list our constraints. The Optimal Control Problem, or OCP, is to
minimise the scalar performance criteria
Z tf
J (u(t)) = (x(tf )) + L(x, u, t) dt (9.11)
0

over a time horizon from 0 to a final time tf , where our system dynamics follow the general
nonlinear dynamic model,
x = f (x, u, t), x(t = 0) = x0 (9.12)
by choosing an appropriate input trajectory u(t).

This performance criteria J (u) to be minimised is quite general and is usually related to process
economic considerations and is often called a cost function. Technically Eqn. 9.11 is termed a
functional since it maps the function u(t) to the scalar J .

The objective function, repeated here,


Z tf
J (u(t)) = (x(tf )) + L(x, u, t) dt (9.13)
| {z }
Mayer problem |0 {z }
Lagrange problem
| {z }
Bolza problem

is quite general and consists of two terms. The first term (or Mayer problem) is called the ter-
mination criteria, and is only concerned about the penalty of the final state. The second integral
term is concerned with the cost incurred getting to the final state and is typically some sort of ISE
or IAE type term. The scalar integrand, L(x, u, t), is called the Lagrangian. For example if
L(x, u, t) = x Qx
where the matrix Q is a positive definite matrix, then this term computes the squared deviations
in state variables and is used when we are concerned with the cost of deviations in state variables.
Alternatively if
L(x, u, t) = 1
1 Note that Ogata, [152, Appendix A] has a very good summary of matrix operations including differentiation, although

he defines the gradient as a column vector. Hence he uses the following:


Ax
= A
x
x Ax
= 2Ax if A is symmetric.
x
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 409

then we are concerned sorely with the total time for the optimisation (which we want to min-
imise). If only the terminal criterion, , is present, we have a Mayer problem, if we are only
concerned with the integral term, we have a Lagrange problem, and if we have both terms, then
we have what is known as a Bolza problem. We can convert between all three types of problems
by introducing extra state variables, see [53, p71] for more hints how to do this.

T HE AUGMENTED COST FUNCTION

We solve this constrained optimisation problem by converting it to an unconstrained optimisation


problem using generalised Lagrange multipliers. These are a vector of introduced variables ,
that are pre-multiplied to each of the dynamic constraint equations and then added to the original
objective function. Converting Eqns 9.11 and 9.12 to an unconstrained optimisation problem gives
Z tf  
J =+ L(x, u, t) + (f (x, u) x) dt (9.14)
0

where the dimension of the introduced variables, dim() = dim(x) = n. Since we have intro-
duced one for each state, these Lagrange multipliers are sometimes known as co-state variables,
or adjoint states.

It is convenient to simplify the notation by introducing the scalar Hamiltonian


def
H = L + f (x, u) (9.15)

so now our augmented objective function, Eqn. 9.14 is more concisely written as
Z tf  
J = + H T x dt (9.16)
0

We can eliminate state derivative, x, in the term T x in Eqn. 9.16 by integrating by parts. Recall
that the scalar integration by parts is
Z Z
uv = uv u v

so if we set u = T and v = x so consequently v = x(t), then


Z tf tf Z tf
T x = T (t) x(t) T (t)x(t) dt

0 0 0
  Z tf
= (tf ) x(tf ) T (0) x(0)
T
T (t)x(t) dt (9.17)
0

Substituting the right-hand expression in Eqn. 9.17 for the T x term in Eqn. 9.16 gives
0 Z tf  

J = + x + H + x dt (9.18)
tf 0

Using variational calculus, we get the Euler-Lagrange equations which define the dynamics of
the co-state variables
 T  T
H (x(tf ))
= with final conditions (tf ) = (9.19)
x x

where we emphasise that the co-state dynamics are specified by terminal boundary conditions at
final time t = tf , as opposed to the initial conditions at t = 0 given for the state dynamics in
Eqn. 9.12.
410 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Finally, the optimum control input profile is given by extremetising the Hamiltonian, or equiva-
lently solving for u such that
H
=0 (9.20)
u
Solving these two equations, and applying the manipulated variable to our dynamic system
(Eqn. 9.12) will result in optimal operation.

In many practical cases, the manipulated variable may be constrained between an upper and
lower limit. In this case, we would not blindly use the stationary point of the Hamiltonian
(Eqn. 9.20) to extract the manipulated variable, but we would simply minimise H, now conscious
of the constrained manipulated variable.

In summary, the Optimal Control Problem, or OCP, is given in Algorithm 9.1.

Algorithm 9.1 The Optimal Control Problem

The solution procedure of the optimum control problem (OCP) is as follows:

1. Given a dynamic model with initial conditions

x = f (x, u, t), x(t = 0) = x0 (9.21)

2. and a scalar performance index which we wish to extremetise over a finite time from t = 0
to final time tf ,
Z tf
J (u) = (x(tf )) + L(x, u, t) dt (9.22)
0

3. then we can form the Hamiltonian


def
H(x, u, ) = L + T f (9.23)

4. where the n introduced co-state dynamics are given by


 T  T
H (x(tf ))
= with final conditions (tf ) = (9.24)
x x

5. The optimum input, u (t), is given by extremetising H, or in the unconstrained case, by


solving
H
=0 (9.25)
u
for u(t).

6. We can now solve the two coupled ODEs (plant dynamics from step 1, and co-state dynam-
ics from step 4 with the substituted control law from step 5), as a boundary value problem.

7. If desired, we can back-calculate the optimum control input trajectory u(t) as a function of
the now known x(t) and (t).

Section 9.3.4 solves the optimal control problem for some simple examples, including the optimal
temperature profile of a batch reactor from section 9.2.4. For the batch reactor we have only a
termination criteria in our objective function, and no state or manipulated variable energy cost.
This makes the calculation somewhat easier.
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 411

9.3.3 T HE TWO - POINT BOUNDARY PROBLEM

Equations 9.12, 9.199.20 need to be solved in order to find the desired u(t), the optimal control
policy at every sampling instance. This, however, is not a trivial task. Normally we would expect
to know the initial conditions for the states so in principle we could integrate Eqn. 9.12 from the
known x(0) for one sample interval, excepting that we do not know the correct control variable
(u) to use. However, we can solve for the instantaneous optimum u by solving Eqn. 9.20. The
only problem now is that we do not know the correct co-state () variables to use in Eqn. 9.20.
So far the only things we know about the co-state variables are the dynamics (Eqn 9.19), and the
final conditions, but unfortunately not the initial conditions.

These types of ODE problems are known as Two Point Boundary Value problems. If we have n state
dynamic equations, then the number of state and co-state dynamic equations is 2n. We know n
initial state conditions, but that still leaves us n initial conditions short for the costates. (To solve
the system of ODEs, we must know as many initial conditions as DEs). However we do know the
final values of the costates, and this information supplies the missing n conditions.

Methods to solve two point boundary value problems are described in [163, Chp16] and [200,
p176] and are in general much harder to solve than initial value ODE problems, and often require
an iterative approach. The most intuitive way to solve two point boundary value ODEs is by
the shooting method or as it is sometimes termed, the boundary condition iteration, or BCI [170,
p237]. This technique is analogous to the artillery gunner who knows both his own and the
enemys position, but does not know the correct angle to point the gun so that the shell lands on
the enemy. Typically an iterative approach is used where the gun is pointed at different angles
with an observer who watches where the shells land, and then communicates with the gunner
what corrections to make. However, the iteration loop outside the integration is not the only
complication. In many physical cases, numerically integrating the state equations backwards or
the co-state equations forwards is unstable, where integrating in the opposite direction causes no
problems.

An alternative method known as Control Vector Iteration (CVI) avoids some of the failings of the
BCI method. A comparison of these two techniques is given in [95]. Perhaps the most robust
scheme is to use collocation which is the strategy used by the M ATLAB routine bvp4c designed
for boundary value problems.

9.3.4 O PTIMAL CONTROL EXAMPLES

The following examples illustrate some simple optimal control problems. In all cases we follow
Algorithm 9.1 and optionally use the S YMBOLIC toolbox to assist in developing the necessary
equations.

S IMPLE CART OPTIMAL CONTROL PROBLEM

Imagine that you are to maximise the distance travelled by a pushing a shopping trolley for a
certain length of time. The position and speed of the cart are the states, and you are supplying
the acceleration, u(t).

The dynamics of the cart are given by

x = x + u, x(0) = 0, x = 0

where we assume we start from rest at the origin. We will re-write this second order dynamic
412 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

system in Cauchy form (using the same variable name as a slight abuse of notation) as

x1 = x2 , x1 (0) = 0 (9.26)
x2 = x2 + u, x2 (0) = 0 (9.27)

Our objective is to maximise the distance travelled at some specified time, T , which is simply
then J = x1 (T ), but clearly this is an uninteresting optimal control problem since the optimal
solution would simply be use infinite acceleration, u(t) = . Instead, to make the problem
more interesting and realistic, we will penalise excessive acceleration, because we humans will
get tired. Now our objective function is to find the appropriate acceleration scheme u(t) such that
we minimise Z
1 T 2
min J = x1 (T ) + u (t) dt (9.28)
2 0
In this case, we will set the final time to T = 1.

We will solve this optimal control problem following the steps presented in Algorithm 9.1.
def
From the objective function, Eqn. 9.28, we can see that the Mayer part is (x(T )) = x1 (T ), and
the Lagrange part is 1/2u2 (t). We construct the Hamiltonian as

H(x, u, ) = L + T f
1
= u2 + 1 x2 2 x2 + 2 u (9.29)
2
T
The state dynamics are given by = H
x , or

H
1 = =0 (9.30)
x1
H
2 = = 1 + 2 (9.31)
x2
with terminal conditions  
1
(t = T ) = = (9.32)
x T 0
Finally we must solve H/u = 0 to compute the control law,
H
= 2 + u(t) = 0
u
meaning that our control law is u(t) = 2 (t).

In summary then, by substituting the control law into the original dynamic system, we must solve
the following 2 state, and 2 costate boundary value differential equation system

x1 = x2 , x1 (0) = 0
x2 = x2 2 , x2 (0) = 0
1 = 0, 1 (T ) = 1
2 = 1 + 2 , 2 (T ) = 0

While we could directly solve this 4-variable boundary value problem, in this somewhat simple
case we can make some important simplifications. First we can see that the dynamic expression
for 1 is particularly simple, 1 = 0 means that 1 is constant, and the terminal condition tells us
what that constant is. Hence 1 (t) = 1 for all t. This removes one variable from our problem.

Now the dynamic expression for 2 is equally simple, and now only involves 2 , namely 2 =
1 + 2 . We can solve this analytically, perhaps using the symbolic toolbox with
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 413

1 >> syms lam2(t) T % define symbolic variables, 2 (t) & final time T

>> dsolve(diff(lam2) == 1+lam2, lam2(T)==0) % Solve 2 = 1 + 2 with 2 (T ) = 0


ans =
exp(-T)*exp(t) - 1

to get an explicit expression 2 (t) = etT 1 and hence the control law u(t) = 2 (t) = 1 etT .
This removes a second variable from our original 4-state variable BVP. Now we need only to solve
the initial value ODE problem

x1 = x2 , x1 (0) = 0
tT
x2 = x2 + 1 e , x2 (0) = 0

to solve the original OCP which is considerably easier to solve than the original 4 state boundary
value problem. This is shown in Listing 9.4.

Listing 9.4: Solving the optimal control problem for a simple cart
T = 1; % fixed end time
x0 = [0;0]; % initial position & velocity of cart

% State dynamics with optimal control law


5 xdot = @(t,x) ...
[ x(2); ...
-x(2) + 1-exp(t-T)];

[t,x] = ode45(xdot,[0,T],x0); % Solve the ODE (not BVP)


10 plot(t,x) % See Fig. 9.10
legend('x(t)','v(t)')

Fig. 9.10 shows the result of the optimal input trajectory, u(t), over a 1 second period which pushes
the cart a distance of 0.17m in that time.
0.25
distance, x (t)
1

0.2 velocity, x2 (t)

0.17
x(t) & v(t)

0.15

0.1

0.05

0
0.8

0.6
u(t)

0.4

0.2 Figure 9.10: Optimal control for a simple cart.


Note that after 1 second, the cart has trav-
0
0 0.2 0.4 0.6 0.8 1 elled 0.17m distance and the force, u(t), has
time [s] dropped to zero.
414 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

While the trajectory calculated by solving an initial value ODE in Fig. 9.10 is optimal, we were for-
tunate in this simple example in that we could decouple the co-state variables and their associated
terminal conditions and therefore solve them separately. Of course for general optimal control
problems we probably could not expect to do this, so instead we will need to solve the original
boundary value problem. The next example, the Rayleigh optimal control problem, shows how
we can solve for the optimal trajectory using a boundary value ODE approach.

R AYLEIGH S OPTIMAL CONTROL PROBLEM

In Rayleighs optimal control problem, we wish to minimise


Z 2.5 
J (u) = x21 + u2 dt (9.33)
0

subject to the state dynamics and state initial conditions

x1 = x2 , x1 (t = 0) = 5

x2 = x1 + 2 0.1x22 x2 + 4u, x2 (t = 0) = 5

From the objective function, Eqn. 9.33, we can see that L = x21 + u2 and = 0.

For this problem it is convenient to use the Symbolic toolbox to develop the necessary equations.
First we define the state dynamics, the objective function, and form the Hamiltonian.

>> syms u lam1 lam2 x1 x2 real % define symbolic variables


def def
>> x = [x1 x2]'; lambda = [lam1 lam2]'; % x = [x1 , x2 ]T , = [1 , 2 ]T
>> f = [x(2); ...
4 -x(1)+(2-0.1*x(2)2)*x(2) + 4*u];% State dynamics x = f (x, u) R
>> phi = 0; L = x(1)2 + u2; % Terms in the objective function, + L dt

>> H = L + lambda'*f % Hamiltonian, H = L + T f


H =
9 lam1*x2 + u2 + x12 - lam2*(x1 - 4*u + x2*(x22/10 - 2))

The above symbolic manipulation shows us that the Hamiltonian is

H = L + T f
  2 
x2
= x21 + u2 + 1 x2 2 x1 4u + x2 2
10

Given the Hamiltonian, we can construct the co-state dynamics, and the terminal conditions from
Eqn. 9.24.

1 >> lam_dot = -jacobian(H,x)' % Eqn. 9.24. Note: Jacobian is a row vector


lam_dot =
lam2 - 2*x1
lam2*((3*x22)/10 - 2) - lam1
>> lam_final = jacobian(phi*x(1),x)'
6 lam_final =
0
0
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 415

T
Or from = (H/x) and (tf ) = (/x)T we get
  " #    
1  22 2x1 1 (tf ) 0
= 3x , =
2 1 102 2 1 2 (tf ) 0

The optimal control input is given by solving Eqn. 9.25 for u(t), or

>> uopt = solve(diff(H,u),u) % Eqn. 9.25


2 uopt =
(-2)*lam2

showing us that
u (t) = 22 (t)

Now all that is left to do is to form and solve our two-point boundary value problem. We can
substitute the expression for the optimum u(t) found above in the state dynamics, and append
the co-state dynamics

>> x_dot = subs(f,u,uopt);


2 >> [x_dot; lam_dot] % State & co-state dynamics
ans =
x2
- 8*lam2 - x1 - x2*(x22/10 - 2)
lam2 - 2*x1
7 lam2*((3*x22)/10 - 2) - lam1

So in summary, to solve the optimal control problem, we need to solve the following boundary
value ODE problem,
x2 

x1 x1 (0) 5
x2 x1 + 2 0.1x22 x2 82 x2 (0) 5
=
, 1 (tf ) = 0

1
 22 2x
1
3x
2 1 102 2 1 2 (tf ) 0

using the two-point boundary value ODE routine bvp4c. We need additionally need to specify
a suitable grid for the collocation points to give an initial estimate trajectory for the state and co-
state profiles over the integration interval. In this case I will simply supply constant trajectories.

zdot = @(t,z) ...


[z(2); ...
3 -8*z(4) - z(1) - z(2)*(z(2)2/10 - 2); ...
z(4) - 2*z(1) ;...
z(4)*(3/10*z(2)2-2) - z(3)]; % State & co-state dynamics

% Set up boundary conditions


8 x0 = [-5;-5]; lam_final = [0;0]; % Initial state conditions & terminal co-state conditions
BCres = @(a,b) [a(1:2)- x0; ...
b(3:4)- double(lam_final)]; % Boundary residuals
solinit = bvpinit(linspace(0,2.5,10),[-1 -1 -0.2 -0.5])

13 sol = bvp4c(zdot,BCres,solinit); % Now solve TPBVP

The optimum profile is given in Fig. 9.11 where we can quickly check that the boundary condi-
tions are satisfied. We can also visually check that the optimum input u (t) = 22 (t) as required.
416 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

The circles plotted show the position of the collocation points and we should check that they cap-
ture the general trend of the profile.

6
x (t)
2
4

x(t)
0

4 x (t)
1

6
10

5
u(t)

5
5

0
(t)

Figure 9.11: The optimum control 2(0)


policy for the Rayleigh problem. The 5 1
upper two trends are the states, x(t), 1(0) 2
the middle is the optimal control pol-
10
icy u (t), and the lower two trends 0 0.5 1 1.5 2 2.5
are the corresponding co-states, (t). time

A N OPTIMAL BATCH REACTOR POLICY REVISITED

In the optimum profile determination for the batch reactor example in section 9.2.4, we approx-
imated the continuous temperature profile with a finite series of constant zeroth-order holds. In
this example, we will compute the exact continuous curve.

Recall that the plant dynamics are given by the reaction kinetics and initial conditions are
dx1
= 2e6/u x1 , starting from x1 (t = 0) = 0.9
dt (9.34)
dx2 6/u 11/u
= 2e x1 5e x2 , starting from x2 (t = 0) = 0.1
dt
and the objective function to be maximised after a batch reaction of total time tf = 2 hours is
Z t=2
J = x2 (t = 2) + 0 dt
0

which indicates that the Lagrangian L(x, u, t) = 0 and (x(tf )) = x2 (tf ). The Hamiltonian func-
tion, H = L + T f , in this instance is
 
  2e6/u x1
H = 0 + 1 2
2e6/u x1 5e11/u x2
 
= 21 e6/u x1 + 2 2e6/u x1 5e11/u x2
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 417

The costates vary with time following = (H/x)T and finish with

d1
= 2e6/u (1 2 ), ending at 1 (t = 2) = 0
dt (9.35)
d2
= 5e11/u 2 , ending at 2 (t = 2) = 1
dt
since = x2 . Now the optimum temperature trajectory is the one that maximises H or the
solution to dH/du = 0, which in this case can solved analytically to give

5
u (t) =   (9.36)
552 x2
ln 12x1 (2 1 )

We can replicate this development of constructing of the costate dynamics and optimal input with
the help of a symbolic manipulator as shown in Listing 9.5 below.

Listing 9.5: Analytically computing the co-state dynamics and optimum input trajectory as a
function of states and co-states
>> syms u lam1 lam2 x1 x2 real % define symbolic variables
def def
2 >> x = [x1 x2]'; lambda = [lam1 lam2]'; % x = [x1 , x2 ]T , = [1 , 2 ]T

>> f = [-2*exp(-6/u)*x(1); ...


2*exp(-6/u)*x(1) - 5*exp(-11/u)*x(2)];
>> phi = x(2); L = 0;
7 >> H = L + lambda'*f; % H = L + T f
>> lam_dot = -jacobian(H,x)'; % H/x
lam_dot =
(2*lam1)/exp(6/u) - (2*lam2)/exp(6/u)
(5*lam2)/exp(11/u)
12 >> lam_final = jacobian(phi,x)'% tfinal = /x
lam_final =
0
1
>> uopt = solve(diff(H,u),u)
17 uopt =
5/log(-(55*lam2*x2)/(12*lam1*x1 - 12*lam2*x1))

Once again we can solve the two-point boundary value problem succinctly using bvp4c as demon-
strated in Listing 9.6. We need to supply the state and co-state dynamic equations, Eqn. 9.34 and
Eqn. 9.35, the four required boundary conditions, the initial x1 (0), x2 (0) and the final 1 (2), 2 (2),
and the optimum input trajectory as a function of states and co-states.

Listing 9.6: Solving the reaction profile boundary value problem using the boundary value prob-
lem solver, bvp4c.m.
u = @(x) 5/log(55*x(4)*x(2)/12/x(1)/(x(4)-x(3))) % Optimum input trajectory, Eqn. 9.36
2

% State and co-state dynamics, Eqn. 9.34 & Eqn. 9.35.


zdot = @(t,x) ...
[-2*exp(-6/u(x))*x(1);
2*exp(-6/u(x))*x(1) - 5*exp(-11/u(x))*x(2);
7 2*exp(-6/u(x))*(x(3)-x(4));
5*exp(-11/u(x))*x(4) ];

% Boundary residuals x1 (0) = 0.9, x2 (0) = 0.1 and 1 (2) = 0, 2 (2) = 1


BCres = @(a,b) [a(1:2)-[0.9; 0.1]; ...
418 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

12 b(3:4)-[0;1]];

tf = 2; % Final time tfinal = 2


solinit = bvpinit(linspace(0,tf,10),[0.6,0.2,0.4 0.6])
sol = bvp4c(zdot,BCres,solinit); % Now solve TPBVP
17

xint = linspace(0,tf,1e3)'; Sxint = deval(sol,xint); % interpolate solution


plot(sol.x, sol.y,'ro',xint,[Sxint(1,:)',Sxint(2,:)'],'-') % See Fig. 9.12

The optimum input temperature policy is shown in Fig. 9.12 with the resultant state and costate
trajectories. The amount of desired product B that is formed after 2 hours using this policy is
0.438 and we can be confident that no other temperature policy will improve on that result.

Optimal batch reactor operation


1

0.8
Ca
Conc; Ca, Cb

0.6 C (2)=0.438
b

0.4
Cb
0.2

0
8

6
Temp

2
1.5

1 2
Costates,

0.5
Figure 9.12: The concentra-
tions, the optimum tempera- 0 1
ture trajectory, and co-states
for the batch reactor as evalu- 0.5
ated using an optimum control 0 0.5 1 1.5 2
policy. time (hours)

In both the Rayleigh and the batch reactor optimal control examples we could extract analytical
expressions for the optimal control policy as an explicit function of states and co-states, u (t) =
g(x, , t) which made the subsequent boundary-value problem easier to solve. However in many
practical cases, the solution of Eqn. 9.25 to give u (t) may not be possible to find in closed form. In
these cases, we may need to embed an algebraic equation solver such as fsolve, or an optimiser
such as fminunc.m inside the BVP solution routine. However the problem to be solved now is a
differential algebraic equation or DAE which introduces considerable computational problems.
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 419

9.3.5 P ROBLEMS WITH A SPECIFIED TARGET SET

In some optimal control problems we wish not only to establish the optimum profile for u(t)
starting from a given initial condition, x(0), but we might also require some (or all) of our states
to satisfy some arbitrary condition at the final time,

(xf , tf ) = 0 (9.37)

These problems are known as OCP problems with a target or manifold set. Further details and
solution strategies for these sorts of problems are given in [47, 166]. The main difference here is
that we need to introduce a new vector of Lagrange multipliers, , the same dimension as the
number of target state constraints in Eqn. 9.37, and these get added to the modified objective
functional.

A simplified, but very common situation is where we wish some of our states to hit a specified
target, and perhaps leave the remainder of our states free. In this case the boundary conditions
simplify to:

1. if xi (tf ) is fixed, then we use that as a boundary constraint, i.e. xi (tf ) = xif ,

2. if xi (tf ) is free, then we use i (tf ) = /xi (tf ) as usual for the boundary constraint.

Note that together, these two conditions supply the additional n boundary conditions we need to
solve the two-point boundary value problem of dimension 2n.

Suppose we wish to find the optimum profile u(t) over the interval t [0, 1] to minimise
Z 1
J = u2 dt
0

subject to the dynamics


x = x2 sin(x) + u, x(0) = 0
but where we also require the state x to hit a target at the final time

x(1) = 0.5

In this case we have one state variable and one associated co-state variable, so we need two
boundary conditions. One is the initial state condition, and the other is the specified state target.
This means that for this simple problem we do not need to specify any boundary conditions for
the costate. The rest of the OCP problem is much the same as that given in Algorithm 9.1.

1 syms x u lam real % define symbolic variables, u(t), x(t), (t)

tf = 1; % final time
xtarget = 0.5; % target constraint

6 f = [x2.*sin(x) + u]; % State dynamics x = f (x, u)


phi = 0; L = u2; % Terms in the objective function, , L

H = L + lam'*f % Hamiltonian, H = L + f

11 lam_dot = -jacobian(H,x)'; % ensure column vector


uopt = solve(diff(H,u),u)
420 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Running the above code gives the optimum input as u = /2 and the the two-point boundary
problem to solve as
   
x x2 sin(x) 2
=  (9.38)
x2 cos(x) + 2x sin(x)

in two variables. The two boundary conditions are

x(0) = 0, and x(1) = 0.5

% Define state & co-state dynamics, Eqn. 9.38.


ztmp = @(t,z1,z2) ...
3 [z12*sin(z1) - z2/2;
-z2*(z12*cos(z1) + 2*z1*sin(z1))];

zdot = @(t,z) ztmp(t,z(1),z(2));


x0 = [0];
8 BCres = @(a,b) [a(1)- x0; ...
b(1)- xtarget]; % Boundary residuals: Initial & terminal state conditions

xlam_init = @(t) [0.5*t; -1];

13 solinit = bvpinit(linspace(0,tf,10),xlam_init)
sol = bvp4c(zdot,BCres,solinit); % Now solve TPBVP

t = sol.x; z = sol.y'; % Extract solution


x = z(:,1); lam = z(:,2);
18 u = -lam(:,1)/2; % Optimum input: u = /2

Fig. 9.13 shows the result of this optimal control problem. We should note that the state, x, did
indeed manage to hit the specified target at t = 1. Finally we can establish the optimum profile
for u(t).

Problem 9.1 (These problems were taken in part from [53, p76].)

1. Show that to determine the optimal input trajectory u(t) given the one dimensional system,

x = x + u, x(0) = 1

such that Z 1
1 
J (u) = x2 + u2 dt
2 0

is minimised reduces to solving the two point boundary value problem,


     
x 1 1 x x(0) = 1
= ,
1 1 (1) =0

Can you find an analytical solution for this problem?


2. Construct the two point boundary problem as above, (but do not solve it) for the two state,
one input system      
0 1 0 1
x = x+ u, x(0) =
0 0 1 0
R 1 
with the cost functional J (u) = 12 0 x x + u2 dt
9.4. LINEAR QUADRATIC CONTROL 421

0.8

0.6
target
States

0.4

0.2

0.5

0.45
*
u

0.4

0.35
0.6

0.7
costates,

0.8

0.9

1.1
Figure 9.13: Optimal control
0 0.2 0.4 0.6 0.8 1 with a target where x(1) must
time equal 0.5.

9.4 L INEAR QUADRATIC CONTROL

The general optimal controller from section 9.3 has some nice properties, not the least that the
trajectory is optimal. However there are some obvious disadvantages, some of which are sub-
stantial. Quite apart from the sheer complexity of the problem formulation and the subsequent
numerical solution of the two-point boundary value problem, there is the fact that the final re-
sult is a manipulated variable trajectory which is essentially an open loop controller. This is fine
provided we always start from the same conditions, and follow down the prescribed path, but
what happens if due to some unexpected disturbance we fall off this prescribed path? Do we
still follow the pre-computed trajectory for u(t), or must we re-compute a new optimal profile for
u(t)?

Ideally we would like our optimal controller to have some element of feedback, which after all is a
characteristic of all our other practical controllers. Furthermore if we restrict our attention to just
linear plant models, and use just a quadratic performance objective, then the subsequent prob-
lem setup, and implementation, is made considerably easier. These types of optimal controllers
designed to control linear plants with quadratic performance are known as Linear Quadratic Reg-
ulators, or LQR.
422 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

9.4.1 C ONTINUOUS LINEAR QUADRATIC REGULATORS

The linear quadratic control problem is to choose a manipulated variable trajectory u(t), that min-
imises a combination of termination error and the integral of the squared state and manipulated
variable error over a time horizon from t = 0 to some specified (although possibly infinite) final
time t = T , Z
1 1 T T 
J = x(T )Sx(T ) + x Qx + uT Ru dt (9.39)
2 2 0
given the constraint of linear plant dynamics

x = Ax + Bu, x(t = 0) = x0 (9.40)

As we assume linear plant dynamics, and the performance objective, Eqn. 9.39, is of a quadratic
form, this type of controller is known as a continuous Linear Quadratic Regulator or LQR.

The weighting matrices Q and S are positive semi-definite (and typically diagonal) matrices,
while R must be a strictly positive definite matrix. Note that any real positive diagonal ma-
trix is also positive definite. The expressions xT Qx and uT Ru are called quadratic forms and
2 2
are sometimes compactly written as ||x||Q and ||u||R respectively.

Using the terminology introduced in the general optimal control problem given in section 9.3.2
Eqn. 9.11 we have the following in the specific case of an LQR

f = Ax + Bu, x(t = 0) = x0 (9.41)


1 1
L(x, u, t) = x Q x + u R u (9.42)
2 2
1
= x (T ) S x(T ) (9.43)
2
for the state dynamics constraint, the Lagrangian or getting there cost, and the final cost term
respectively.

For many applications, there is no explicit termination cost, since all the cost of state deviations
is accounted for in the energy term, thus often simply the matrix S = 0. In addition, the upper
limit of the time horizon is often set to T = , in which case we are obviously now no longer
concerned with the termination cost term.

The Hamiltonian in the LQR case (following the definition from Eqn. 9.15) is
1 T 1
H = L + T f = x Qx + uT Ru + T (Ax + Bu) (9.44)
2 2
Now applying the Euler-Lagrange equations (Eqns 9.19 and 9.20) using the differentiation rules
from section 9.3.1 gives
  T
d

H
T 12 x Qx + Ax
= =
dt x x
 T
= x T Q + T A
= Qx A (9.45)

since Q is symmetric and we have made use of the matrix identity that (AB)T = BT AT .

Similarly using Eqn. 9.20 gives


H
= 0 = uT R + T B (9.46)
u
9.4. LINEAR QUADRATIC CONTROL 423

or RT u = BT (again using the above matrix identity) which implies that the optimal control
law is
u = R1 B (9.47)
Eqn. 9.47 defines the optimal u as a function of the co-states, . Note that now our problem has
2n unknown states: the original n x states, and the n introduced co-states, . We can write this
in a compact form using Eqns 9.47, 9.41 and 9.45
 " #
BR1 B
 
x A x
= (9.48)
Q A

with mixed boundary conditions


x(0) = x0 , and (T ) = Sx(T )
From Lyapunovs theorem, we know that the costates are related to the states via
(t) = P(t)x(t) (9.49)
where P(t) is a time varying matrix. Differentiating Eqn. 9.49 using the product rule gives

= Px + Px (9.50)
Now we can substitute the control law based on the costates (Eqn 9.47) into the original lin-
ear dynamic model (Eqn 9.41) and further substitute the relation for the states from the costates
(Eqn. 9.49) giving
x = Ax + Bu (9.51)
 
= Ax + B R1 B (9.52)
1
= Ax BR B Px (9.53)
Now substituting Eqn. 9.53 into Eqn. 9.50 and equating with the Euler-Lagrange equation Eqn. 9.45
gives  
= Qx A = Px + P Ax BR1 B Px (9.54)
Rearranging, and cancelling x from each term gives the matrix differential equation

P = PA A P + PBR1 B P Q (9.55)
which is called the differential matrix Riccati equation. The boundary requirement or termination
value, from Eqn. 9.19 is
P(t = T ) = S (9.56)
and which can be solved backwards in time using the original dynamic model matrices A, B,
and the controller design matrices Q, R. While we need to know the model and design matrices
for this solution, we need not know the initial condition for the states. This is fortunate, since
it means that we can solve the matrix Riccati equation offline once, store the series of P(t), and
apply this solution to our problem irrespective of initial condition. We only need to repeat this
computation if the plant model or design criteria change.

In summary, we now have developed an optimal state feedback controller of the proportional
form
u = K(t)x (9.57)
where the time varying matrix feedback gain is (from Eqn 9.47) defined as

K(t) = R1 B P(t) (9.58)


where the time-varying P matrix is given by Eqn. 9.55 ending with Eqn. 9.56. Since we have
assumed that the setpoint is at the origin in Eqn. 9.57, we have a regulatory problem, rather than
a servo or tracking problem. Servo controllers are discussed in 8.2.3.
424 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

9.4.2 A NALYTICAL SOLUTION TO THE LQR PROBLEM

The problem with implementing the full time-varying optimal controller is that we must solve the
nonlinear differential matrix Riccati equation, Eqn. 9.55. This is a complicated computational task
and furthermore we must store offline the resultant gain matrices at various times throughout the
interval of interest. However there is an analytical solution to this problem which is described
in [62, 2.4] and [166, 5.2]. The downside is that it requires us to compute matrix exponentials,
and if we want to know P(t) as opposed to just the costates, we must do this computation every
sample time.

We start by solving for both states and costates in Eqn. 9.48, repeated here
    
x A BR1 B x x(0) = known
= , (9.59)
Q A (T ) = known
| {z }
H
where the matrix H is known as the Hamiltonian matrix. This approach has the attraction that
it is a linear, homogeneous differential equation with an analytical solution. Starting from any
known point at t = 0, the states and co-states at any future time are given by
   
x(t) x(0)
= eHt (9.60)
(t) (0)

However the outstanding problem is that we do not know (0) so we cannot get started!
def
If we define eHt = (t), then we can re-write Eqn. 9.60 as
   
x(t) x(0)
= (t) (9.61)
(t) (0)

and remember that with M ATLAB it is straight forward to compute using matrix exponentials
since it is a function of known, constant matrices. We then partition the matrix into four equal
sized n n blocks as  
11 (t) 12 (t)
(t) =
21 (t) 22 (t)
which means we can express Eqn. 9.61 as

x(t) = 11 (t)x(0) + 12 (t)(0) (9.62)


(t) = 21 (t)x(0) + 22 (t)(0) (9.63)

Employing the Riccati transformation, (t) = P(t)x(t), changes Eqn. 9.63 to

P(t)x(t) = 21 (t)x(0) + 22 (t)(0)

and multiplying Eqn. 9.62 by P(t) and equating to the above gives

P(t)11 (t)x(0) + P(t)12 (t)(0) = 21 (t)x(0) + 22 (t)(0)

Solving for the mystery (0) gives


1
(0) = [22 (t) P(t)12 (t)] [P(t)11 (t) 21 (t)] x(0)

Now at the final point, t = T , we know that P(T ) = S from the transversality conditions, so (0)
is given by
1
(0) = [22 (T ) S12 (T )] [S11 (T ) 21 (T )] x(0) (9.64)
which is something that we can compute since all the terms are known.
9.4. LINEAR QUADRATIC CONTROL 425

Now having established (0), we can use Eqn. 9.60 to compute analytically both states, x(t), and
costates, (t), at any time t. Consequently, we can compute P(t) at any intermediate time between
0 and T using
1
P(t) = [22 (T t) S12 (T t)] [S11 (T t) 21 (T t)] (9.65)
In many applications, it suffices to ignore the costates and just consider P(0) which is given di-
rectly by
1
P(0) = [22 (T ) S12 (T )] [S11 (T ) 21 (T )] (9.66)
We will investigate further this important simplification in section 9.4.3.

A N LQR TIME - VARYING EXAMPLE

Suppose we wish to control the continuous plant


     
2 4 4 2
x = x+ u, x(0) =
3 1 6 1
using an optimal controller of the form u = K(t)x to drive the process from some specified
initial condition to zero. For this application, we decide that the manipulated variable deviations
are more costly than the state deviations,
   
1 0 0 0
Q= , R = 2, S=
0 1 0 0
Listing 9.7 first defines the plant, then solves for the initial costates, (0), using Eqn. 9.64, and
then finally simulates the augmented state and costate system given by Eqn. 9.59.

Listing 9.7: Computes the full time-evolving LQR solution


A = [-2 -4; 3 -1]; B=[4,6]'; x0 = [-2,1]'; % Plant: x = Ax+Bu & initial state, x(t = 0)
2 n=length(A); % Problem dimension
Q = eye(2); R=2; S=zeros(size(A)); % Design criteria: Q, R and S
T = 0.5; % final time, T

H = [A, -B/R*B'; -Q, -A']; % Create Hamiltonian using Eqn. 9.59


% (t) = eHt
def
7 Phi = expm(H*T);

Phi11 = Phi(1:n,1:n); Phi12 = Phi(1:n,n+1:end); % Construct 11 (t), 12 (t), . . .


Phi21 = Phi(n+1:end,1:n); Phi22 = Phi(n+1:end,n+1:end);
Lam_0 = (Phi22 - S*Phi12)\(S*Phi11 - Phi21)*x0; % costate initial condition, Eqn. 9.64
12

z0 = [x0;Lam_0]; % augment states & costates


G = ss(H,zeros(2*n,1),eye(2*n),0);
initial(G,z0,T) % simulate it

The results of this simulation are given in Fig. 9.14 which also compares the near identical results
from a slightly sub-optimal controller which uses simply the steady state solution to the Riccati
equation discussed next in section 9.4.3. Note that the performance drop when using the consid-
erably simpler steady-state controller is small, and is only noticeable when the total time horizon
is relatively short.

An alternative solution procedure to the analytical solution given in Listing 9.7 is to numerically
integrate the continuous time Riccati equation, Eqn. 9.55, backwards using say the workhorse
ode45 with initial condition Eqn. 9.56. The trick that the function in Listing 9.8 does is to collapse
the matrix Eqn. 9.55 to a vector form that the ODE function can then integrate. The negative sign
on the penultimate line is because we are in fact integrating backwards.
426 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Long horizon: Tfinal= 2 Short horizon: Tfinal=0.5


2 2
x
2
0 0
states

2 x 2
1

4 4

0.5 0.5
u (t)

u(t)
0 0 u(t)
u

u(t)

0.5 0.5

1 1

K
2
0.5 0.5
K

K
1
0 0

0.5 0.5
P22
P
11
0 0
P

P12

0.5 0.5

0.5 0.5

2

0 0

1
0.5 0.5
0 0.5 1 1.5 2 0 0.1 0.2 0.3 0.4 0.5
time [s] time [s]

Figure 9.14: Comparing the full time-varying LQR control with a steady-state sub-optimal con-
troller for a long horizon (left) and a short horizon (right). The upper two plots trend the two
states and one input for the steady-state case and for the time varying case. The lower three plots
trend the time-varying elements of the matrices K(t), P(t), and the co-state vector (t) for the full
time-varying optimal case.
9.4. LINEAR QUADRATIC CONTROL 427

Listing 9.8: The continuous time differential Riccati equation. This routine is called from List-
ing 9.9.
function dPdt = mRiccati(t, P, A, B, Q)
% Solve the continuous matrix differential Riccati equation
% Solves dP/dt = AT P PA + PBBT P Q

5 P = reshape(P, size(A)); % Convert from (n2 1) vector to n n matrix


dPdt = -A.'*P - P*A + P*B*B.'*P - Q; % Eqn. 9.55
dPdt = -dPdt(:); % Convert back from n n to (n2 1)
return

We call the function containing the matrix differential equation in Listing 9.8 via the ode45 in-
tegrator as shown in Listing 9.9. Note that we can express BR1 B efficiently as ZZT where
Z = B(chol(R)1 ).

Listing 9.9: Solves the continuous time differential Riccati equation using a numerical ODE inte-
grator.
A = [-2 -4; 3 -1]; B=[4,6]'; % Plant dynamics of interest
2 Q = eye(2); R=2; S=zeros(size(A)); % Weighting matrices, Q, R
T = 1; % Final time for the finite time horizon

P0 = S; % Initial condition for P


[t,P] = ode45(@(t,P) mRiccati(t,P,A,B/(chol(R)),Q),[0 T],P0(:)); % Call listing 9.8
7 P = flipud(P); % reverse time order of P(t)

Now we have generated a trajectory for the four elements of the P(t) matrix as a function of time.
We could then pre-compute the time varying gains, K(t), and then store them ready to be used in
our closed loop simulation.

Compared to the analytical solution, this numerical integration scheme requires that we integrate
n2 equations as opposed to 2n when we combine the co-states. Furthermore we must store the
trajectory of pre-computed time-varying gains. In practice of course, this is storage requirement
rapidly becomes unwieldy and, as we shall see in the next section, rarely worth the effort.

9.4.3 T HE STEADY- STATE SOLUTION TO THE MATRIX R ICCATI EQUATION

In the optimal controller scheme presented above, the gain K(t), is a continuously time varying
matrix, as opposed to a more convenient constant matrix. A time varying controller gain requires
excessive computer resources even if we did discretise, since we need to store an (m n) gain
matrix at every sample time in the control computer over the horizon of interest.

We can make an approximation to our already approximate optimal controller by using only the
steady-state gain derived from the steady-state solution of the matrix differential Riccati equation
when P = 0. With this approximation, the differential Eqn. 9.55 is simplified to the algebraic
matrix equation,
0 = A P + P A P BR1 B P + Q (9.67)
where the steady-state solution, P , is now a constant gain matrix which is considerably more
convenient to implement. Eqn. 9.67 is known as the continuous-time algebraic Riccati equation,
or CARE.

The now slightly sub-optimal steady-state controller gain is still obtained from Eqn. 9.58,

K = R1 B P (9.68)
428 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

and is also a constant gain matrix.

Whether the performance deterioration due to this approximation is significant depends on the
plant, and on the time horizon of interest. However as evident in Fig. 9.14, the differences in
performance are negligible for all but the shortest time horizons. Even for the case where T = 2
which is still a relatively brief optimisation horizon, the gains are essentially constant for more
than two thirds of the horizon, and this feature is generally true for all plants, and optimisation
scenarios provided the horizon is reasonably long.

We can solve for the elements of the matrix P in Eqn. 9.67 using one of several strategies. One
direct way is to by equating the individual elements in each of the matrices and thus reducing the
single matrix equation to a set of algebraic equations. However unlike the Lyapunov equation,
Eqn. 2.104, which we could collapse to a system of linear equations using vectorisation and Kro-
necker products as described in section 2.9.5, this time the steady-state Riccati equation includes
a quadratic term of the form P BP which means our resultant system of equations will not be
linear. However an iterative scheme based on this idea is given on page 431.

Another strategy is to compute the analytical expression for P(T ), Eqn. 9.66, with a large time
horizon T . Unfortunately Eqn. 9.66 is numerically very ill-conditioned when T is large, let alone
, and it is difficult to know a priori exactly what time horizon is sufficiently large. A final strat-
egy given later in Listing 9.13 is only applicable for the discrete version of the optimal controller.

The control toolbox in M ATLAB provides a reliable algebraic Riccati equation routine, care, that
solves Eqn. 9.67 for P in a numerically reliable manner2 , and the ric routine verifies the com-
puted P from care (or equivalent) by calculating the residual of Eqn. 9.67. The linear quadratic
regulator lqr routine finds the controller gain directly which is slightly more convenient than
the two-step procedure solving the algebraic Riccati equation explicitly. All three routines are
demonstrated in Listing 9.10.

A STEADY- STATE CONTINUOUS LQR EXAMPLE

Simulating a steady-state optimal state-feedback controller is trivial in M ATLAB. We can try a


steady-state version of the optimal controller for the example given previously on page 425. A
script to solve the controller design problem using the toolbox functions are, care or lqr, and
to simulate the results in M ATLAB is given in Listings 9.10 and 9.11.

Listing 9.10: Calculate the continuous optimal steady-state controller gain.


>>A = [-2, -4; 3 -1]; B = [4; 6]; % Plant under consideration
>>Q = eye(2); R = 2; % Controller weighting matrices, Q, R.
3

>>P = are(A, B/R*B', Q) % Solve for X such that AT X + XA XBX + Q = 0


P =
0.1627 -0.0469
-0.0469 0.2514
8 >>K = R\B'*P % Now compute gain from Eqn. 9.68, K = R1 B P .
K =
0.1847 0.6603

>>[Kerr,Perr] = ric(A,B,Q,R,K,P) % Check results


13 Kerr =
0 0
Perr =
1.0e-014 *
2 An older toolbox routine, are.m, essentially also does this calculation.
9.4. LINEAR QUADRATIC CONTROL 429

0.0999 0.2720
18 0.1471 0.1887 % Residuals should 0 which they do

>> [P2,L,G]=care(A,B/chol(R),Q);% Solve for X such that AT X + XA XBBT X + Q = 0

>> K2 = chol(R)\G; % Compute controller gain given that G = BT X


23

>>[K2,P2,E] = lqr(A,B,Q,R) % Gain from lqr should gain from care.


>> K2
K2 =
0.1847 0.6603

This completes the controller design problem, the rest is just a simple closed-loop simulation as
shown in Listing 9.11.

Listing 9.11: Closed loop simulation using an optimal steady-state controller gain.
C = eye(size(A)); D = zeros(2,1);
Gcl = ss(A-B*K,zeros(size(B)),C,D); % Hopefully stable optimal closed loop
3

T = 1; x0 = [-2,1]'; % final time & initial state, x0


[Y,t,X]=initial(Gcl,x0,T);
U = -X*K'; % back-calculate input, u(t), for plotting

8 subplot(2,1,1);plot(t,X); ylabel('states'); % Refer Fig. 9.15.


subplot(2,1,2);plot(t,U); ylabel('input'); xlabel('time (s)')

The state and input trajectories of this optimal controller given a non-zero initial condition are
given in the Fig. 9.15.

1
x (t)
2

0
states

2 x1(t)

3
0.4

0.2
input

0.2

0.4
0 0.2 0.4 0.6 0.8 1
Figure 9.15: A steady-state continuous LQR
time (s) controller.

The difference between the fully optimal time varying case and the simplification when using
the steady-state solution to the Riccati equation is given in Fig. 9.14 which uses the same ex-
ample from page 428. For short optimisation intervals there is a small difference between the
steady-state solution and the full evolving optimal solution as shown in the right-hand figure of
Fig. 9.14. However as the final time increases, there is no discernible difference in the trajectories
of the states nor in the input in the left-hand figure of Fig. 9.14. This goes along way to explain
why control engineers hardly ever bother with the full optimal solution. The lower two plots in
Fig. 9.14 trend the time varying P(t) and K(t) matrices.
430 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

M ULTIPLE SOLUTIONS TO THE ALGEBRAIC R ICCATI EQUATION

As mentioned in section 9.4.3, being a quadratic, there are multiple solutions to the continuous
algebraic Riccati equation, Eqn. 9.67. We can investigate all these solutions using the symbolic
toolbox for small sized problems by equating the coefficients and solving the resultant system
of quadratic equations. Note that there are in fact only (n2 + n)/2 unknown elements in P as
opposed to n2 due to the symmetry.

Suppose we wish to design an LQR for the simple SISO system G(s) = 1/(s(s + 1)), with equal
state and input weighting which is a problem initially discussed in [5].

1 G = tf(1,[1 1 0]) % Plant


[A,B,C,D] = ssdata(ss(G)) % actually transposed from [5], but no matter . . .
Q = eye(2); R = 1;

Matlab choses a state-state representation as


   
1 0 1
A= , B=
1 0 0
Now inserting the plant and tuning matrices into the continuous algebraic Riccati equation, Eqn. 9.67,
gives the nonlinear matrix system
0 = A P + PA PBR1 B P + Q
           
1 1 p11 p12 p11 p12 1 1 p11 p12 1 0 p11 p12 1 0
= + +
0 0 p12 p22 p12 p22 0 0 p12 p22 0 0 p12 p22 0 1
noting of course that since P is symmetric, p12 = p21 . From this point onwards, the algebraic
manipulation gets messy, so we will use the Symbolic toolbox to help us.

>>n = 2; % Dimension of problem


2 >>P = sym('P',[n,n]); % Create symbolic P matrix
>>P = triu(P) + triu(P,1).' % Force P to be symmetric
>>Res = A'*P+P*A - P*B/R*B'*P+Q % Continuous algebraic Riccati equation, Eqn. 9.67.
Res =
[ - P1_12 - 2*P1_1 + 2*P1_2 + 1, P2_2 - P1_2 - P1_1*P1_2]
7 [ P2_2 - P1_2 - P1_1*P1_2, 1 - P1_22]

This final line shows us that we have 4 conditions to be satisfied, (although two of them are the
same),
0 = p11 2 2p11 + 2p12 + 1
0 = p22 p12 p11 p12
0 = 1 p12 2
giving us enough equations to solve for the three unknown elements in P. We can see immedi-
ately that p212 = 1 giving us two solutions: p12 = 1. We could continue this substitution by hand,
but it might be faster at least to use the analytical solve command:

>> Res = Res(find(triu(ones(size(Res))))) % Drop redundant equations . . .


>> S = solve(Res) % and solve
3 S =
P1_1: [3x1 sym]
P1_2: [3x1 sym]
P2_2: [3x1 sym]
9.4. LINEAR QUADRATIC CONTROL 431

The analytical solve routine returned three solutions for the three unknowns, namely:

>> [S.P1_1 S.P1_2 S.P2_2]


ans =
[ 1, 1, 2]
4 [ -1, -1, 0]
[ -3, 1, -2]

which tells us that we have three distinct P matrix solutions,


     
1 1 1 1 3 1
Pa = , Pb = , Pc =
1 2 1 0 1 2

but only one of which is positive definite


     
0.38 1.6 3.6
eig(Pa ) = , eig(Pb ) = , eig(Pc ) =
2.62 1.6 1.3
and therefore stabilising. Practically we can extend this approach to solving 3rd (with 8 solutions)
and 4th order (with 15 solutions) systems, but anything larger seems to stall.

Of course, we could have used the standard numerical M ATLAB command and avoided all the
spurious non-stabilising solutions:

[P,E,K] = care(A,B,Q,R) % standard way

S OLVING THE ALGEBRAIC R ICCATI EQUATION USING K RONECKER PRODUCTS

Notwithstanding that the algebraic Riccati equation of Eqn. 9.67 is nonlinear, we can devise an
iterative scheme to solve for P in a manner similar to that for the Lyapunov equation using
vectorisation and Kronecker products described in section 2.9.5.

Collapsing Eqn. 9.67 to vectorised form gives


 
I AT + AT I I P BR1 BT vec(P ) = vec(Q) (9.69)

which we now must solve iteratively due to the unknown P in the coefficient matrix in Eqn. 9.69.
Listing 9.12 uses this solution strategy which works (most of the time) for modest sized problems,
but is not numerically competitive with, nor as reliable as, the ARE scheme in M ATLAB.

Listing 9.12: Solving the algebraic Riccati equation for P using Kronecker products and vectori-
sation given matrices A, B, Q and R.
n = size(A,1); I = eye(size(A)); % Identity, I
BRB = B/R*B'; % Constant matrix, BR1 BT
D = kron(I,A') + kron(A',I); % vec(AT P + PA) = (I AT + AT I)vec(P)
4

P = I; % Start with P = I
tol = 1e-8; dP = tol*1e2; % Suggested tolerance, 105
while norm(dP) > tol
Pnew = reshape(-(D-kron(I,P*BRB))\Q(:),n,n); % Solve Eqn. 9.69 iteratively for P
9 dP = Pnew - P;
P = Pnew;
end
432 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

In a similar manner to solving the Lyapunov equation using fsolve on page 84, we could also
construct the algebraic Riccati matrix equation to be solved and submit that to the general non-
linear algebraic equation solver fsolve. This works in so far as it delivers a solution, but the
solution may not be a stabilising solution, or in other words while the matrix P does satisfy the
original matrix equation, Eqn. 9.67, it may not be symmetric positive definite, and hence lead to a
stable closed loop system, as shown in the example given on page 430. This is due to the fact that
the matrix Riccati equation has multiple solutions and general purpose numerical schemes such
as fsolve will return the first solution it comes across as oppose to the stabilising one that we
are interested in. For this reason it is better to use the dedicated Riccati solving routines than the
general purpose fsolve.

9.4.4 T HE DISCRETE LQR

In the discrete domain, the optimisation problem analogous to Eqn. 9.39 is to search for the N
future optimal control moves, u0 , u1 , , uN 1 to minimise
N 1
1 1 X T 
J = xN SxN + xk Qxk + uTk Ruk (9.70)
2 2
k=0

where the continuous plant dynamics, Eqn. 2.41, are now replaced by the discrete equivalent
xk+1 = xk + uk (9.71)
and the state feedback control law, Eqn. 9.57, is as before. Now the discrete matrix difference
Riccati equation analogous to Eqn. 9.55 is
 1
Pk = Q + Pk+1 Pk+1 R + Pk+1 Pk+1 (9.72)

with final boundary condition


PN = S (9.73)
and the feedback gain K at time t = kT is
 1
Kk = R + Pk+1 Pk+1 (9.74)

An alternative form to Eqn. 9.74 is


Kk = R1 T T (Pk Q) (9.75)
which uses the current Pk rather than the future version, but has the disadvantage that must
be invertible. Note that the optimal feedback gain K, is now an (m n) matrix rather than a row
vector as in the SIMO pole placement case described in chapter 8.

It is possible to compute the value of the performance index J , of Eqn. 9.70 in terms of the initial
states and the initial P0 matrix,
1
J = xT0 P0 x0 (9.76)
2
although this precise value of J has probably little practical worth in most industrial control
applications.

U SING THE SLIGHTLY SUB - OPTIMAL DISCRETE STEADY- STATE GAIN

Clearly the discrete optimal gain given by Eqn. 9.74 is again time varying as it was in Eqn. 9.58 for
the continuous case, but this can be computed offline since it is only dependent on the constant
plant matrices and , the weighting matrices Q and R, and the time varying Pk .
9.4. LINEAR QUADRATIC CONTROL 433

u(t)
+ b
z 1 C y(t)

Plant

K
Exported states, x(t)
LQR gain

Figure 9.16: A block diagram of LQR control.

However, once again as in the continuous case given in 9.4.3, we can use the considerably sim-
pler steady-state approximation to Eqn. 9.72 with little reduction in optimality. We solve for the
steady state covariance,
 1
P = Q + P P R + P P (9.77)

by starting with P = 0 and stepping, (rather than integrating), backwards in time until Pk
approximately equals Pk+1 . This is known as the discrete-time algebraic Riccati equation, or
DARE.

It follows that the resultant steady-state (and slightly sub-optimal) gain K given by Eqn. 9.74
 1
K = R + P P

will now also be constant.

S OLVING THE DISCRETE ALGEBRAIC R ICCATI EQUATION

A simple way to solve the discrete algebraic Riccati equation implemented in the listing 9.13 is to
iterate around Eqn. 9.77 starting from a zero initial condition.

Listing 9.13: Calculate the discrete optimal steady-state gain by iterating until exhaustion. Note
it is preferable for numerical reasons to use lqr for this computation.
function [K,P] = dlqr_ss(Phi,Delta,Q,R)
% Steady-state discrete LQR by unwinding the Riccati difference eqn, Eqn. 9.72.
% Routine not as reliable as dlqr.m
4 P = zeros(size(Phi)); % Start with P = 0 at t =
tol = 1e-5; % default tolerance

P0 = P+tol*1e3;
while norm(P0-P,'fro') > tol
9 P0=P;
P = Q + Phi'*P*Phi - Phi'*P*Delta*((Delta'*P*Delta+R)\Delta')*P*Phi;
end % while
1 T
K = (Delta'*P*Delta + R)\Delta'*P*Phi; % K = R + T P P
end % end function dlqr_ss
434 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

However, the backward iteration calculation technique given in dlqr_ss in listing 9.13 is both
somewhat crude and computationally dangerous. For this reason it is advisable to use the func-
tionally equivalent, but more robust dlqr (or lqr) provided in the C ONTROL T OOLBOX. We
could also use dare if we only wanted to solve for the P matrix.

C OMPARING THE CONTINUOUS AND DISCRETE STEADY- STATE LQR

Listing 9.14 compares a continuous LQR regulator for the system given on page 425 with a discrete
LQR sampled at a relatively coarse rate of Ts = 0.25s. Note that in both the discrete and continuous
cases we design the optimal gain using the same lqr function. M ATLAB can tell which algorithm
to use from the supplied model characteristics.

Listing 9.14: Comparing the continuous and discrete LQR controllers.


A = [-2, -4; 3 -1]; B = [4; 6]; % Continuous plant, x = Ax + Bu
2 C = eye(size(A)); D = zeros(2,1); G = ss(A,B,C,D)

Q = eye(2); R = 2; % Controller weighting matrices Q = I2 , R = 2


K = lqr(G,Q,R); % Optimal gain, K = [k1 , k2 ]
Gcl = ss(A-B*K,zeros(size(B)),C,D); % Continuous closed loop
7

T = 1; x0 = [-2,1]'; % final time & initial condition


[Y,t,X]=initial(Gcl,x0,T); U = -X*K'; % Simulate closed loop & back-calculate input

% Discrete version of the LQR controller


12 Ts = 0.25; % Rather coarse sampling time, Ts , to exaggerate differences
Gd = c2d(ss(A,B,C,D),Ts); % Discrete plant, xk+1 = xk + Buk , with Ts = 0.25

Kd = lqr(Gd,Q,R); % Design discrete LQR gain


Gcl_d = ss(Gd.A-Gd.b*Kd,zeros(size(B)),C,D,Ts);
17

[Yd,td,Xd]=initial(Gcl_d,x0,T); Ud = -Xd*Kd'; % Simulate & back-calculate input

subplot(2,1,1); plot(t,X,td,Xd,'o'); % Refer Fig. 9.17


subplot(2,1,2); plot(t,U,td,Ud);

Even though the heavily discretised LQR controller has quite a different control input trajectory
as evident in the lower figure of Fig. 9.17, the difference between the discrete output, , and the
continuous LQR controller, (solid line), is noticeable in that the solid line does not pass exactly
through the circles, but the difference is very marginal.

Problem 9.2 1. Load in the Newell & Lee evaporator matrices (refer Appendix D.1) and design
an LQR with equal weighting on the Q and R matrices by typing Q = eye(A); R = eye(B);
and using the dlmr function with a sample time of 1.0. This feedback matrix should be the
same as the gain matrix given in [149, p55]3 . Comment on any differences in solution you
obtain.
2. Repeat the LQR design using the supplied dlqr function rather than the dlqr_ss func-
tion. Using dlqr however, the gain elements are now slightly different. Is this difference
important and how would you justify your answer?
3. Use either dlqr_ss or dlqr to generate feedback gains for the limiting cases when either
(or both) Q or R are zero. What difficulties do you encounter, and how do you solve them?
3 Newell & Lee define the control law in a slightly different way; u = Kx omitting the negative sign. This means that

the K on p55 should be the negative of that evaluated by M ATLAB.


9.4. LINEAR QUADRATIC CONTROL 435

1
x2(t)

0
states

2 x1(t)

3
0.4
continuous input
0.2
Figure 9.17: Comparing discrete, ,
input

0 and continuous, , LQR controllers.


discrete input, T =0.25
s Note that even with a relatively
0.2 coarse sample time, the controlled re-
sponse in both cases is very simi-
0.4
0 0.2 0.4 0.6 0.8 1 lar, (but not identical), at the sampled
time (s) points.

C OMPUTING THE PERFORMANCE OF A STEADY- STATE DISCRETE LQR CONTROLLER

As mentioned previously, in most cases we are uninterested in the exact value of the objective
function since we are far more interested in the control law in order to minimise it. However
it is interesting to numerically validate the actual performance with the analytical expectations.
In the steady-state case, we are uninterested in the termination part of the performance index of
Eqn. 9.70 leaving just

1X T 
J = xk Qxk + uTk Ruk (9.78)
2
k=0
1 T
= x P x0 (9.79)
2 0
reinforcing the fact that while the controller gain is not dependent on the initial condition, the
value of the performance index is.

Suppose we had a plant


     
0 0 1 2
xk+1 = xk + uk , starting from x0 =
0.5 1 0 2

with controller design matrices Q = diag[1, 0.5], and R = 1, and we wanted to compute the per-
formance index J comparing the numerical summation given by Eqn. 9.78 against the analytical
expression given by Eqn. 9.79.

>> Phi = [0 0; -0.5, 1]; Del = [1 0]'; % Plant matrices , and controller design Q, R
>> Q = diag([1 0.5]); r = 1;
>> [K,P,E] = dlqr(Phi,Del,Q,r) % Solve the algebraic Riccati equation for K & P .
4 K =
0.2207 -0.4414
P =
1.5664 -1.1328
-1.1328 2.7656
436 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Now that we have the steady-state Riccati equation solution we can compute the performance
directly , J .

>> x0 = [2,2]'; % Initial condition x0


2 >> J = 0.5*x0'*P*x0 % Performance J = 12 xT0 P x0 , Eqn. 9.79.
J =
4.1328

To obtain an independent assessment of this, we could simulate until exhaustion and numeri-
cally compute Eqn. 9.78. Since we have simulated the closed loop, we need to back-extract the
input by using the control law and the states.

1 Gcl = ss(Phi-Del*K,0*Del,eye(2),0, 1); % Form closed loop ( K)


[y,t,x] = initial(Gcl,x0); % Let M ATLAB establish a suitable on final time, k
u = -x*K'; % Back extract control input from closed loop simulation, u = Kx

Now M ATLAB has returned a vector of input and states, and we need toP compute the summation
given in Eqn. 9.78. One elegant way to calculate expressions such as xT Qx given the states
stacked in rows is to use the M ATLAB construction sum(sum(x.*x*Q)). This has the advantage
that it can be expressed succinctly in one line of code and it avoids an explicit loop.

T
P 
>> uRu = u'*u*r; % k uk Ruk , but only works for scalar r
1
xTk Qxk + uTk Ruk
P 
2 >> J_numerical = 0.5*(sum(sum(x.*x*Q'))+uRu) % Performance 2 k
J_numerical =
4.1328

In this case the numerical value of J is very close to the analytical value.

VARYING THE Q AND R WEIGHTING MATRICES

The Q and R weighting matrices in the objective function Eqn. 9.39 reflect the relative importance
of the cost of state deviations with respect to manipulated variable deviations and are essentially
the tuning parameters for this control algorithm. If we increase the Q matrix, then the optimum
controller tries hard to minimise state deviations and is prepared to demand very energetic con-
trol moves to do it. Likewise, if we increase R, the weighting on the deviations from u, then
excessive manipulated variables moves are discouraged, so consequently the quality of the state
control deteriorates. If we are only really interested in output deviations, yT y, as opposed to
state deviations, then Q is often set equal to CT C. This follows simply from the output relation
y = Cx.

Fig. 9.18 and the listing below demonstrate the design and implementation of a discrete LQR
for a stable, interacting arbitrary plant. We will start from a non-zero initial condition and we
subject the system to a small external disturbance at sample time k = 20. In this listing, the state
deviations are considered more important than manipulated deviations.

1 % Create an simple, stable, interacting, 3*2 plant


Phi = [0.8 0.1 0.1; -0.1 0.7 0.2; 0 0.6 0.1];
Del = [1,2; -2,1; 0,-3]; C = [1,0,0]; D=[0,0];

t=[0:40]'; dist = 20; % sample vector & disturbance point


6 Q = eye(size(Phi)); [n,nb] = size(Del); R=eye(nb,nb); % tuning constants
9.4. LINEAR QUADRATIC CONTROL 437

State deviations are deemed costly: q=100,r=1 Input movements are deemed costly: q=1,r=100

2 2 x2(t)

1.5 1.5

1 1
State Q=100

State Q=1
0.5 0.5
x3(t)

0 0

0.5 0.5

1 xQx=12.3 1 x1(t) xQx=27.4

1.5 1.5

2 2

1 1

0.5 0.5

input R=100
input R=1

0 0
uRu=1.571 uRu=0.059
0.5 0.5

1 1
0 10 20 30 40 0 10 20 30 40
Sample number Sample number

Figure 9.18: Varying the state and manipulated weights of a discrete linear quadratic regulator
where there is a disturbance introduced at sample time k = 20. When state deviations are costly (q
is large compared to r), the states rapidly return to setpoint at the expense of a large manipulated
movement.

K = dlqr(Phi,Del,100*Q,R); % Design LQR controller with Q = 100I, R = I


Gd = ss(Phi-Del*K,Del,C,D,1); % closed loop system
U = zeros(size([t,t]));U(dist,:) = [1,0.2]; % not much + disturbance
11 x0 = [-1,2,0]'; % given initial condition
[Y,T,X3] = lsim(Gd,U,t,x0); % do closed loop simulation
U3 = X3*K'; % extract control vector from x
stairs(t,[X3,U3]);
P T P T
16 % Compute performance indices: x Qx, & u Ru
perfx3 = sum(sum(X3.*(Q*X3')')), perfu3 = sum(sum(U3.*(R*U3')'))

Fig. 9.18 compares two responses. The left controlled response is when we consider state devia-
tions more important than manipulated variations, i.e. the weighting factor for the states is 100
times that for the manipulated variables. In the right plot, the opposite scenario is simulated.

Clearly in Fig. 9.18Pthe left controlled response returns a relatively low ISE (integral squared er-
ror)
P T on the states, xT Qx = 12.3, but at the cost of a high ISE on the manipulated variables,
u Ru = 1.57. Conversely the right controlled response shows the situation, where perhaps
owing to manipulated variable constraints, we achieve a much lower ISE on the manipulated
variables, but at a cost of doubling the state deviation penalty.

As Fig. 9.18 shows, the design of an optimal LQR requires the control designer to weigh the rel-
ative costs of state and manipulated deviations. By changing the ratio of these weights, different
controlled responses can be achieved.

As we did in 8.2, we can modify the simulation given above to account for non-zero setpoints,
438 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

x , by using the control law, u = K(x x) rather than Eqn 9.57, so simulating the closed loop

xk+1 = ( K)xk + Kxk

thus enabling response to setpoint changes.

9.4.5 A NUMERICAL VALIDATION OF THE OPTIMALITY OF LQR

If the LQR is indeed an optimal controller, its performance must be superior than any arbitrarily
designed pole-placement controller such as those tested in section 8.2.1. Listing 9.15 designs an
optimal controller for the blackbox plant with equal state and input weights.

Listing 9.15: An LQR controller for the blackbox


>> G = ss(tf(0.98,conv([2 1],[1,1])),0.1) % Blackbox plant with sample time Ts = 0.1

3 >> [n,m] = size(G.b); % system dimensions



1 0
>> Q = eye(n); % state weight, Q =
0 1
>> R = eye(m); % Input weight, R = r = 1

>> K = dlqr(G.a,G.b,Q,R) % Compute optimal controller gain K.


8 K =
4.7373 -3.5942

Now it would be interesting to compare this supposedly optimal controller with a pole-placement
controller with poles arbitrarily placed at say T = [0.5 0.5i].

Listing 9.16: Comparing an LQR controller from Listing 9.15 with a pole-placement controller
1 >> pc = [0.5-0.5i; 0.5+0.5i]; % Desired closed loop poles, = [0.5 0.5i]
>> Kp = acker(G.a,G.b,pc); % Pole-placement controller gain
Kp =
13.6971 -5.7713

6 >> Gcl = G; Gcl.b = 0*G.b; Gcl.c = eye(n);


>> Gcl.a = G.a-G.b*K; % Closed loop system with LQR, ( K)
>> Gclpp = Gcl; Gclpp.a = G.a-G.b*Kp; % Closed loop system with pole-placement

>> x0 = [-1,2]'; % Simulate disturbance rejection


11 >> initial(Gcl,Gclpp,x0) % Refer Fig. 9.19.

The trajectory of the states for the two schemes are given in Fig. 9.19. However what is surprising
is that the pole-placement response looks superficially considerably better than the somewhat
sluggish, though supposedly optimal, LQR response. How could this be? Surely the optimally
designed controller should be better than any ad-hoc designed controller?

What Fig. 9.19 has not shown us is the input trajectory. This is also important when compar-
ing controlled responses, although typically perhaps not quite as important as the state variable
trajectories.

We need to consider the input trajectory for at least two reasons. Excessive deviations in input
will almost certainly reach the input saturation constraints that are present in any real control
system. Secondly, excessive movement in the input can cause unreasonable wear in the actuators.
Finally in the case where the input magnitude is related to cost, say fuel cost in an engine, then
9.4. LINEAR QUADRATIC CONTROL 439

Comparing LQR & Poleplacement


2
poleplacement
0
x1

4 LQR

6
2

0 Figure 9.19: Comparing the state tra-


jectories for both pole-placement and
x2

2
LQR. Note that the optimal LQR
4 dlqr is considerably more sluggish to re-
Poleplacement =0.50.5i turn to the setpoint than the pole-
6 placement controller.
0 0.5 1 1.5 2 2.5 3

the larger the magnitude, the more expensive the operating cost of the corrective action will be.
Fig. 9.20 reproduces the same problem, but this time includes the input trajectory. In this instance,
the cost of the pole-placement controller is J = 1288 units, while for the LQR it is under half at
553.

Performance: LQR = 553, Poleplace = 1288


2
poleplacement
0
states

4 LQR

6
10

0 Figure 9.20: Comparing pole-


placement and LQR, but this time
input

10 including the input trajectories.


20 dlqr Compared to Fig. 9.19, the input
Poleplacement =0.50.5i trajectory for the optimal LQR
30 controller is considerably less active.
0 0.5 1 1.5 2 2.5 3

Obviously different pole locations will give different performances, and it is up to us as control
designers to choose the most appropriate pole location for the particular control application at
hand. Using M ATLAB, it is easy to do an exhaustive search over a reasonable domain to find the
best pole location.

For real systems (that is systems with real coefficients), the desired poles must appear in conjugate
pairs. That means that we only need to consider one half of the unit circle. Furthermore, we
should only consider the 1st quadrant, because in the 2nd quadrant, the response will be overly
oscillatory.

Naturally it goes without saying that in the discrete domain we must be inside the unit circle.

I have chosen a set of trial poles lying on a polar grid from 0 degrees round to 90 , and from a
magnitude of about 0.1 to 0.9. These (unequally spaced) trial pole locations are shown in Fig. 9.21.

For each of the pole locations specified in Fig. 9.21, I will:


440 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

1
0.5/T
0.6/T 0.4/T
0.8
0.7/T 0.1
0.3/T
0.2
0.6 0.3
0.8/T 0.4 0.2/T
0.5
0.4 0.6
0.7
0.9/T 0.8 0.1/T
0.2
0.9
1/T

{}
0
1/T

0.2
0.9/T 0.1/T
0.4

0.8/T 0.2/T
0.6

0.8
0.7/T 0.3/T
Figure 9.21: We will test a pole-placement
0.6/T 0.4/T
controller at each of the pole locations spec- 1
0.5/T
1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1
ified by a +. (See also Fig. 9.22 following.) {}

1. Design a pole placement controller, then

2. simulate the response for a reasonably long time which I assume will approximate infinity,
and then
P
3. compute the performance criteria, J = i=0 xT Qx + uT Ru.

As you can see from the resultant contour plot in Fig. 9.22, the performance of our previously
arbitrarily chosen pole location of T = [0.5 0.5i] is reasonable, but not great, and clearly not
optimal. From the plot, the best performance is located at a pole location of approximately
[0.75 0.15i]T marked as a in Fig. 9.22.

The question is: can we find this location without doing an exhaustive search, or indeed any type
of numerical optimisation search strategy?

When we look at the closed loop poles of the LQR controller,

Listing 9.17: Computing the closed loop poles from the optimal LQR controller from Listing 9.15.
>> [K,S,E] = dlqr(G.a,G.b,Q,R)
K =
4.7373 -3.5942 % Controller gain (as before).
4 S =
90.4087 -65.2391
-65.2391 50.4963
E =
0.7800 + 0.1664i % Optimum closed loop poles, .
9 0.7800 - 0.1664i

then we see that in fact the computed closed loop poles are intriguingly close to the optimum
found numerically in Fig. 9.22. It turns out that the LQR routine did indeed find the optimum
controller without going to the computational trouble of an exhaustive search. Just remember
of course, that this is not a proof of the optimality of the LQR, it is just a numerical validation.
Nonetheless, you should find it a convincing argument.

Note of course, if we change the dynamic system, or the controller design weights, the of course
the shape of the performance surface in Fig. 9.22, and the corresponding optimum will also
change.
9.4. LINEAR QUADRATIC CONTROL 441

Figure 9.22: The performance of the closed loop for all the possible pole locations in the first
quadrant. When T = [0.5 0.5i], the weighted performance is approximately 1288, while the
optimum performance, , is considerably better at around 550 at pole location [0.78
0.17i]T .

Problem 9.3 1. Simulate the controlled response of the evaporator for about 500 time steps (T
= 0.5 time units) with the initial conditions
 
x0 = 1 4.3 2.2

for the 4 cases:


(a) Q = R = I
(b) Q = 103 R = I
(c) 103 Q = R = I
(d) any other combination of your choice.
In each case, sketch the results of the input u, and the states x with time and calculate the
ISE for the states, manipulated variables and the weighted sum. What conclusions can you
deduce from the relative importance of Q and R?
2. Compare the performance of the full multivariable LQR controller with that of a controller
comprised of just the diagonal elements of the controller gain. What does this say about the
process interactions?

9.4.6 T RACKING LQR CONTROL

Since the LQR is essentially just an optimal version of pole-placement, for the tracking (or servo)
control problem we can simply follow the approach taken in section 8.2.3 where we introduced
442 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

auxiliary states which were defined as the integral of the outputs. The M ATLAB routine lqi
designs LQR controllers for servo applications.

As an example, let us design a continuous servo LQR controller for the SISO system
20(s + 2)
G(s) =
(s + 3)(s + 5)(s + 7)
Clearly this system has m = 1 input, p = 1 output and n = 3 internal states. We desire that the
output follows without offset some reference command signal. To achieve this we are going to
design an optimal LQR controller with weighting matrices Qa (of dimension of the augmented
states, n + p,) and R of dimension m as shown in Listing 9.18.

Listing 9.18: Tracking LQR controller with integral states


1 >> Gc = ss(zpk([-2],[-3 -5 -7],20)); % Plant to be controlled
>> [A,B,C,D] = ssdata(G);
>> [n,m] = size(B); p = size(C,1); % Dimensions state-space model

>> Qa = eye(n+p); R = eye(m); % Choose equal weighting for now


6 >> Ka = lqi(G,Qa,R) % Design augmented gain [K, Ki ]
Ka =
-0.7832 0.3879 0.4898 -1.0000

This final augmented gain, is comprised of two parts: the first n elements are associated with
the original plant states, and the final p elements are associated with the outputs we wish to
control with zero offset. An implementation in S IMULINK for this application showing the signal
dimensions is given in Fig. 9.23.

3 Cont 3 Cont 3 1 3 Cont Cont


1 Cont 3 -Ka* u B* u C* u
4 Cont 4 u
Cont 3 s x 3 y
D1 Cont s 3
Add Terminator
Add2 y Integrator Ka feedback B Integrator C
Band-Limited
White Noise
A

3
A* u
3 Cont

Figure 9.23: An LQR servo controller for a SISO plant with 3 states implemented in S IMULINK.
See also Fig. 9.24 for the results of this simulation.

If we are really only interested in controlling the outputs to their setpoint as opposed to all the
states to zero, then we should indicate that desire by the appropriate design of the (augmented)
controller weighting matrix Qa . In this case if I wish to have fast control at the expense of input
movement, I would choose something like

0 0 0 0
Qa = 0 0 0 0 , R = 1

0 0 0 0
0 0 0 10
remembering that Qa need only be semidefinite, but R must be strictly positive definite. Fig. 9.24
shows the result for this servo controlled example for three different tuning ratios. Note that as
Q gets large relative to R, then the output trends faster to the setpoint at the expense of a more
vigorous input.
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 443

0.6
Q=1,R=1
Q=5,R=1
0.4 Q=1,R=20

0.2
y

0.2 Q=1,R=1
Q=5,R=1
1
Q=1,R=20

0.5
u

0.5 Figure 9.24: LQI servo control for 3


0 20 40 60 80 100 120 140 160 180 200
time (s) different Q/R tuning ratios.

In experimental, rather than simulated applications, since we do not measure the states directly,
we can use a plant model to produce state estimates based on the measured outputs using the
simple state reconstructor from section 8.1.4, or we could use a state estimator which is the subject
of the following section, section 9.5.

In the experimental optimal regulator example shown in Fig. 9.25, at each sample time we:

1. update an estimate of a simple 3 state model online using recursive least-squares,


2. which we use to reconstruct the states given past input and output data,
3. and in parallel recompute the optimal steady state LQR gains given that the model has been
updated.
4. Using the new LQR gains and the reconstructed states, we apply a state feedback control
law.

In this application the sample time was relatively long, T = 1.5 seconds, and the RLS forgetting
factor was = 0.995. No feedback control was used for the first 20 sample times in order to given
sufficient time for the estimation algorithm to deliver a reasonable model. From the trend of the
controller gains in the bottom plot in Fig. 9.25, you can see that although the model parameters
look reasonably constant, the values of the individual controller gains are quite excited.

9.5 E STIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS

Chapter 6 looked at various alternatives to identify the parameters of dynamic systems, given
possibly noisy input/output measured data. This section looks at ways to estimate not the pa-
rameters, (they are assumed known by now), but the states themselves, x, by only measuring the
inputs u, and the outputs y. In all cases, we are assuming that the dynamic system is subjected to
uncertainty. This state reconstruction is a very powerful technique since if this estimation works,
we can use state feedback control schemes without actually going to the trouble of measuring the
states, but instead using only their estimates.

In section 8.1.4 we built a simple state reconstructor which works well in the case of no measure-
ment noise or model/plant mismatch. The problem is that in practical cases there will always be
disturbing signals which will degrade our state estimates derived from such a straight forward
444 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

LQR of the blackbox t=1.5


output & setpoint 1

0.5

0.5
1

2 2
0.5 dlqr servo: = 1, =1000
q r
Input

0.5

1
4

2 RLS: =0.995,

a
2 b
0 abb
012
3
a
1
2
20

10
2
K &K

0
1

10

20
0 50 100 150 200 250
time (s)

Figure 9.25: An experimental application of LQR servo control of the black box with 103 q = r
with online identification and state reconstruction. Plots: (a) output and setpoint; (b) input; (c)
model parameters ; (d) controller gains. The control started after 20 seconds.

algorithm. The question is: can we devise a better scheme that mitigates against this degradation?
The answer is given in the form of an optimal state estimator that takes explicitly into account the
expected uncertainty.

These estimators work by using a model to predict the output, y, which can be compared to the
measured output, y. Based on this difference we can adjust, if necessary, the predicted states.
The estimation of states in this way is known, for historical reasons, as a Kalman filter. A block
scheme of how the estimator works is given in Fig. 9.26. Here we have a control scheme that while
demanding state-feedback, actually uses only state estimates rather than the actual hard-to-obtain
states.

There are many texts describing the development (eg: [52, 96]) and applications (eg: [39, 193]) of
the Kalman filter.
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 445

Noise

State Measurement

Plant actual outputs, y


Input, u

y
Model g(x) +
Estimator

predicted states
u = Kx corrected states, x
K

State feedback Software

Figure 9.26: A state-based estimation and state feedback control scheme.

9.5.1 R ANDOM PROCESSES

The only reason that our estimation scheme will not work perfectly, at least in theory, is due to the
corrupting noise, and to a lesser degree, incorrect initial conditions. Now in practice, noise is the
garbage we get superimposed when we sample a real process, but when we run pure simulations,
we can approximate the noise using some sort of random number generator such as described pre-
viously in 6.2. These are correctly termed pseudo random number generators, and most of them
are of the very simple linear congruent type, and are therefore deterministic and not strictly ran-
dom. Whole books have been written on this subject and many of them quote John von Neumann
saying in 1951 Anyone who considers arithmetical methods of producing random digits is, of
course, in a state of sin.

G ENERATING R ANDOM SEQUENCES

M ATLAB can generate two types of random numbers: namely uniform or rectangular, and normal
or Gaussian. Using the uniform scheme, we can generate any other arbitrary distribution we may
desire including Gaussian. However in most practical cases, it is the Gaussian distributed random
number we are more interested in. To visualise the difference, try:

150

100

50

x = rand(1000,1); hist(x) % uniformly distributed, 0


0 0.5 1

300

200

100

2 y = randn(1000,1); hist(y) % normally distributed, 0


5 0 5

Sometimes it is interesting to compare the actual probability density function or PDF obtained
using the hist command with the theoretical PDF. To directly compare these, we need to scale
the output of the histogram such that the integral of the PDF is 1. Fig. 9.27 compares the actual
(scaled) histogram from 104 samples of a random variable, N ( = 12, = 2) with the theoretical
446 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

probability density function. The PDF of a normal variate is, [194, p60],
 2 !
1 1 x
f (x) = exp (9.80)
2 2 2

0.25

0.2

mean = 12.0116
frequency

0.15
2
= 4.03369
0.1

0.05

0
2 4 6 8 10 12 14 16 18 20
x

Figure 9.27: The probability density function of a normally distributed random variable x
N ( = 12, = 2); theoretical (solid) and experimental (histogram).

Listing 9.19: Comparing the actual normally distributed random numbers with the theoretical
probability density function.
xsig = 2; xmean = 12; % Variance, , & mean, x
x = xsig*randn(10000,1)+xmean; % sample (realisation) of r.v. x.
3 xv = linspace(min(x),max(x),100)'; % approx range of x
pdf= @(xv) 1/sqrt(2*pi)/xsig*exp(-0.5*(((xv-xmean)/xsig).2));% expected PDF, Eqn. 9.80.
area = trapz(xv,pdf(xv)); % Note integral of PDF should=1.0

[numx,xn] = hist(x,30); % Generate histogram [incidence #, x-value]


8 dx = mean(diff(xn)); % assume evenly spaced
scale = 1/dx/sum(numx); % scale factor so total area = 1
freq = numx*scale; % normalise so total area =1
bar(xn,freq); hold on; plot(xv,pdf(xv));hold off % See Fig. 9.27

M ULTIVARIABLE RANDOM NUMBERS

For a single time series xk of n elements, the variance is calculated from the familiar formula
n
1X
Var(x) = x2 = (xi x)2 (9.81)
n i=1

where x is the mean of the vector of data. In the case of two time series of data (xk , yk ) the
variances of x and y are calculated in the normal fashion using Eqn. 9.81, but the cross-variance
or co-variance between x and y is calculated by
n
1X
Cov(x, y) = (xi x) (yi y) (9.82)
n i=1

Note that while the variance must be positive or zero, the co-variance can be both positive and
negative and that Cov(x, y) = Cov(y, x). For this two vector system, we could assemble all the
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 447

variances and cross-variances in a matrix P known as the variance-covariance matrix


 
x2 Cov(x, y)
P=
Cov(y, x) y2

which is of course symmetric with positive diagonals for time series of real numbers. In normal
usage, this name of this matrix is shortened to just the covariance matrix.

For an n sequences, the (n n) covariance matrix P will be a positive (semi) definite symmetric
matrix, P  0. M ATLAB can calculate the standard deviation, , of a vector with the std com-
mand and it can calculate the co-variance matrix of a multiple series using the cov command.

One characteristic about two independent random time series of data is that they should have no
cross-correlation, or in other words, the co-variance between x and y, should be approximately
zero. For this reason, most people assume that the co-variance matrices are simply positive diago-
nal matrices, and that the off-diagonal or cross-covariance terms are all zero. We can numerically
check this by creating 3 random vectors, x, y, z, and then calculating the covariance matrix.

x=randn(1000,1) +3; % generate 3 independent vectors


y=randn(1000,1)*6+1; % standard deviation, = 6
z=randn(1000,1)*2-2; % variance 2 = 22 = 4
4 a=[x y z];
cov(a) % find co-variance

The covariance matrix of the three time series is in my case, (yours will be slightly different),

1.02 0.11 0.10 1 0 0
Cov(x, y, z) = 0.11 37.74 0.30 0 36 0
0.10 0.30 4.22 0 0 4

which shows the dominant diagonal terms which are approximately the true variances (1, 36, 4)
and the small, almost zero, off-diagonal terms.

G ENERATING RANDOM NOISE SEQUENCES WITH SPECIFIED STATISTICS

For simulation and testing of our stochastic estimation algorithms later, it is useful to be able to
generate noise sequences that have a specified covariance matrix, Q. We may want to do this to
test our algorithm with deliberately correlated noise. As shown above, simply generating n noise
sequences using randn delivers a covariance matrix of I.

One procedure to generate correlated noise is as follows:

1. Generate uncorrelated Gaussian random variables, a, with mean zero and unit variance
using the randn command.
448 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

2. If the desired covariance matrix is diagonal, just multiply each column by the corresponding
square root of the variance.
3. If the desired covariance is not diagonal, then we must transform the variables with identity
covariance, I, to one with the specified statistics, Q, using a lower triangular linear transfor-
mation, w = H a.
cov(a) = I cov(w) = Q
a N (0, 1) H w

linear transformation
To find the transformation matrix H, we note that since the output sequence w must have
the required covariance Q, then

E{wwT } = E{(Ha)(Ha)T } = Q

and since E{aaT } = I from the way we generated sequence a, this means that

HHT = Q

which is a Cholesky factorisation. Note M ATLAB returns an upper triangular matrix H from
chol).
4. If you want to do the reverse, i.e. go from variables with covariance Q, to unity, then use
the transformation H1 QHT .

To illustrate this way to generate correlated noise, we will generate three sequences of 10,000
random deviates with covariance matrix equal to

5 2 1
Q= 2 4 0
1 0 1

and we will confirm it numerically with cov.

Listing 9.20: Generating correlated noise sequences with a known covariance, Q.


>> Q = [5 2 -1; 2 4 0; -1 0 1] % required covariance, Q

>> H = chol(Q)'; % Q must be +ve def, H is lower triangular

5 >> a = randn(1e4,3); % generate 3 independent vectors of mean zero, = 1


>> cov(a) % covariance should be I
ans =
0.9750 0.0121 0.0177
0.0121 0.9955 0.0101
10 0.0177 0.0101 0.9969

>> w = a*H'; % equivalent to w = Ha


>> cov(w) % covariance should be Q
ans =
15 4.8752 1.9984 -0.9346
1.9984 4.0042 0.0303
-0.9346 0.0303 0.9802

Note that the generated sequences do not have the exact specified covariances, but it is within 1
or 2%.
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 449

9.5.2 C OVARIANCE MATRICES AND CONFIDENCE ELLIPSES OF RANDOM SE -


QUENCES

In the many cases where the random data does exhibit cross-correlation (such as in the case above
where Q had non-zero off-diagonal elements), we will see this clearly if the x, y data pairs fall in
a pattern enclosed by a rotated ellipse. We can compute the 95% percent confidence limits of this
ellipse using
T p(n 1)
(x ) P1 (x ) F,p,(np)
np
where p is the number of parameters (usually 2), and n is the number of data points (usually
> 100). F,p,(np) is the F -distribution at a confidence of , in this case 95%. To compute values
from the F -distribution we could either use the functions given in Listing 9.21,

Listing 9.21: Probability and inverse probability distributions for the F -distribution.
pf = @(x,v1,v2) betainc(v1*x/(v1*x+v2),v1/2,v2/2);
qf = @(P,v1,v2) fsolve(@(x) pf(x,v1,v2)-P,max(v1-1,1))

or use equivalent routines from S TIX B OX toolbox from Holtsberg mentioned on page 4 or alter-
natively use finv(P,v1,v2) from the Statistical toolbox.

The easiest way to construct this ellipse is to first plot a circle parametrically, scale the x and y
axes, rotate these axes, then finally translate the ellipse so that the center is at the mean of the
data. The axes of the ellipse are given by the eigenvectors of P, and the lengths are given by the
square root of the eigenvalues of P.

E XAMPLE OF CONFIDENCE LIMIT PLOTTING

Here we will generate 1000 random (x, y) samples with deliberate correlation and we will com-
pute the ellipse at a 95% confidence limit. Furthermore we can check this limit by using the
inpolygon routine to calculate the actual proportion of data points that lie inside the ellipse. We
expect around 95%.

1. First we generate some correlated random data,


x N (0.3, 1) and y N (2, 1.1) + x
These data pairs are correlated because y is explicitly related to x. (Alternatively, we could
have generated similarly correlated sequences using the strategy presented on page 447.)

Listing 9.22: Generate some correlated random data.


N = 1e3; % # of samples in Monte-Carlo
x = randn(N,1)-0.3; % N (x = 0.3, = 1)
3 y = 1.1*randn(N,1)+x + 2; % Deliberately introduce some cross-covariance
plot(x,y,'.') % See Fig. 9.28(a).

We can plot the (x, y) points and we can see a that the buckshot pattern falls in a rotated
ellipse indicating correlation as expected as shown in Fig. 9.28(a).
2. The distribution of the data in Fig. 9.28(a) is more clearly shown in a density plot or a 3D
histogram. Unfortunately M ATLAB does not have a 3D histogram routine, but we can use
the normal 2D histogram in slices along the y-axis as demonstrated in the following code
snippet.
450 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Listing 9.23: Plot a 3D histogram of the random data from Listing 9.22.
1 Nb = 30; % # of bins in a quick & dirty 3D histogram
xmin = min(x); ymin = min(y); xmax = max(x); ymax = max(y);
xv = linspace(xmin,xmax,Nb); yv = linspace(ymin,ymax,Nb);
[X,Y] = meshgrid(xv,yv); Z= 0*X;

6 for i=1:length(xv)-1
idx = find(x>xv(i)&x<xv(i+1));
Z(:,i) = hist(y(idx),yv)';
end % for
bar3(Z) % crude 3D histogram

This gives the density and 3D histogram plots given in Fig. 9.28(b) and (c).

xy data plot Density plot


8 8 50

45
3D histogram
6 6
40
50
35
4 4
30

2 2 25
y

0
20
0 0
15

10
2 2
x y 5

4 4
5 0 5 5 0 5
x x

Figure 9.28: Correlated noisy x, y data. (a) Individual (x, y) data. (b) A density plot. (c) A his-
togram in 2 independent variables.

3. To plot an ellipse, we plot first plot x = a cos(t), y = b sin(t) with t [0, 2], rotate, and
finally then shift the origin to . The directions of the of the semi-major and semi-minor
axes of the ellipse are given by the eigenvectors of P and the lengths are given by the square
roots of the eigenvalues.
If V is the collection of eigenvectors vi of the (p p) matrix P, then in the usual case of
p = 2,
def  
V = v1 v2
and the transformation to the un-rotated ellipse is

u = VT (x )

This is shown in Listing 9.24 which calls the F -distribution using the anonymous functions
given previously in Listing 9.21.

Listing 9.24: Compute the uncertainty regions from the random data from Listing 9.23.
X = cov(x,y); % sampled covariance matrix
[V,D] = eig(X); e = diag(D);

n = length(x); p = 2; % # of data points & # of parameters


5 alpha = 0.95; % confidence limit, = 0.95
scale= p*(n-1)/(n-p) * qf(alpha,p,n-p); % Inverse F -distribution, See Listing 9.21.
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 451

t = linspace(0,2*pi,5e2)'; % Generate (scaled) parametric circle


uxc = sqrt(e(1)*scale)*cos(t); uyc = sqrt(e(2)*scale)*sin(t);
10

% Now rotate VT (x x)
c = (inv(V')*[uxc, uyc]')'; % basically V1 x
xc = c(:,1); yc = c(:,2);
xc = xc + mean(x); yc = yc + mean(y); % shift for mean

4. As a final check, we could use the inpolygon routine to see if the actual proportion of
points inside the ellipse matches the expected %. The final data points and the 95% ellipse
is given in Fig. 9.29.

Listing 9.25: Validating the uncertainty regions computed theoretically from Listing 9.24.
1 in = inpolygon(x,y,[xc;xc(1)],[yc;yc(1)]);
prop_inside = sum(in)/n;

plot(x(in),y(in),'r.','MarkerSize',1); hold on
plot(x(in),y(in),'b.','MarkerSize',1); plot(xc,yc,'k-');
6 hold off axis('equal'); grid % See Fig. 9.29.

For my data, 94.6% of the 1000 data points lie inside the confidence limits which is in good
agreement to the expected 95%.

95% confidence limit


6

2
y

1
Figure 9.29: Randomly generated data with
2 a 95% confidence limit superimposed. In
3 actual fact, 94.6% of the data points lie in-
6 4 2 0 2 4 side the confidence limits, which is a good
x agreement.

This strategy is expandable higher dimensions. Fig. 9.30 shows a case where we generate 3 corre-
lated random streams using

X = randn(1000,3);
X(:,3) = X(:,3) + 0.7*X(:,1);

where the z component is strongly correlated to the x variable. This means that our ellipsoid that
encloses the bulk of the data points is slanted along the x z direction.
452 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Figure 9.30: Randomly generated data in 3


dimensions with a 95% confidence surface
limit superimposed.

9.5.3 C OMBINING DETERMINISTIC AND STOCHASTIC PROCESSES

If we have a perfect model of the process, no noise and we know the initial conditions x0 exactly,
then a perfect estimator will be simply the open loop predictions of x, given the system model
and present and past inputs u. However in practice there is always noise present both in the
measurements and in the state dynamics, so this open loop prediction scheme will soon fail even
if the initial estimates and model are perfect. A more realistic description of an industrial process
is
xk+1 = xk + uk + wk , wk N (0, Q) (9.83)
yk = Cxk + vk , vk N (0, R) (9.84)
where the terms v, w are zero mean, white noise sequences with covariance matrices Q and R
respectively. Note how in Fig. 9.31, the two unobservable noise signals, v and w are wholly inside
the dashed box, as are the true states, xk coloured grey in this figure to highlight that they are
essentially hidden. We, as external observers, can only access those signals outside the dashed
box, namely the manipulated inputs uk and the observations, yk .

Plant noise, w

Measurement noise, v

xk+1 xk
uk + q 1 C + yk

Figure 9.31: A block diagram of a stochastic state-space system

This model is still a discrete linear dynamic model, which may or may not be adequate model,
but the final noise terms are inserted to account for all the non-deterministic components of the
model.
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 453

M EAN AND VARIANCE PROPAGATION IN A DYNAMIC SYSTEM

Predicting the behaviour of the state mean and variance given a stochastic dynamic system sounds
quite complicated, but actually in the discrete domain it is simple. The mean of the system,
Eqn. 9.83 is the expected value,

xk+1 = E {xk + uk + wk } (9.85)


= xk + uk + wk (9.86)

but since the input is deterministic, (u u), and with no loss of generality the mean of the noise
is zero, (E(w) = 0), Eqn. 9.86 collapses to

xk+1 = xk + uk (9.87)

which is simply the deterministic part of the full stochastic dynamic process, and also which
should be intuitive.

The propagation of the state variance, P is slightly more complex. The state variance at the next
sample time is, by definition,
def 
Pk+1 = E (xk+1 xk+1 )(xk+1 xk+1 ) (9.88)

and if we substitute the new mean from Eqn. 9.87, the process, Eqn. 9.83 we get

Pk+1 = E ((xk xk ) + wk )((xk xk ) + wk ) (9.89)

which if we expand out, and note that the variance of w is Q, then we get the following matrix
equation

Pk+1 = Pk + Pxk wk + Pwk xk + Q (9.90)


| {z }
very small, 0

= Pk + Q (9.91)

We can ignore the two inner cross-product terms in Eqn. 9.90) because of the uncorrelatedness of
states and the white noise w. In a similar manner to the above application of definition, expan-
sion and eliminating cross terms, one can find expressions for the propagation of the mean and
variance of the output variables, y. Further details are given in [122, pp6264].

Eqn. 9.91 is called the discrete Algebraic Lyapunov difference Equation, or ALE. If the system is
stable, then the steady-state covariance, P , will be given at sample k = , or by solving the
following matrix equation,
P = P + Q (9.92)
in exactly the same way we used in section 2.9.4 when establishing the stability of a linear discrete-
time system either using Kronecker products, or using the M ATLAB dlyap command. Note that
the final state covariance does not depend on the initial uncertainty, P0 .

The propagation of the mean and variance of a linear dynamic system, given stochastic distur-
bances, is fundamental to what will be developed in the following section which describes the
Kalman filter estimation scheme. The Kalman filter tries to reduce the variance over time which
in effect is to reduce the magnitude of the matrix P using a clever feedback scheme, thus obtaining
improved state estimates.

Problem 9.4 Verify by M ATLAB Monte-Carlo simulation that the steady-state covariance for a
stable linear system subjected to white noise input with known variance is given by the solution
to the ALE, Eqn. 9.92. Compare your simulated actual covariance (perhaps using cov) with the
predicted covariance (perhaps using dlyap).
454 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

9.5.4 T HE K ALMAN FILTER ESTIMATION SCHEME

The estimation scheme consists of two parts: a prediction and a correction. For the prediction
step, we will predict using the openloop model, the new state vector and the state covariance
matrix. The correction part will adjust the state prediction based on the current measurement,
and in so doing, hopefully reduce the estimated state covariance matrix.

We will denote the state vector as x and an estimate of this as x. The prediction of the state using
just the model is called the a priori estimate and denoted, x . Once we have assimilated the
measurement information, we have the a posteriori estimate which is denoted simply as x.

At sample time k, we have:

The a priori estimate x


k , which is the estimate of the true, but unknown, state vector xk
before we assimilate the current measurement, yk .
We also assume we know the a priori covariance of x
k,
T
P
k = E{(xk xk )(xk xk ) }

Now we want to improve the estimate of xk by using the current measurement. We will do this
by the linear blending equation

xk = x
k + Lk yk Cxk (9.93)
where xk , is the corrected state estimate or a posteriori estimate, and Lk , the yet to be determined
blending factor or Kalman gain. We use the symbol L for the Kalman gain instead of the com-
monly used K because we have already used that symbol for the similar feedback gain for the
linear quadratic regulator.

The best value of L will be one that minimises the mean-square error of the a posteriori covariance,
Pk = E{(xk xk )(xk xk )T } (9.94)
and substituting in Eqn. 9.93, gives
 
Pk =E{ (xk x k ) Lk (Cxk + vk Cxk )
 
(xk x
k ) Lk (Cxk + vk Cxk ) }

Now noting that (xk x


k ) is uncorrelated with vk , then
T
Pk = (I Lk C) P T
k (I Lk C) + Lk RLk (9.95)
is an expression for the updated covariance Pk in terms of the old covariance P
k.

We want to find the particular Lk such that we minimise the terms along the diagonal of Pk . Note
that
d trace(AB)
= BT
dA
d trace(ACAT )
= 2AC
dA

So now expanding Pk from Eqn. 9.95 to explicitly show the terms in L gives
 
Pk = Pk L k CP
k P T T
C L k + Lk CP T
C + R k Lk
T
| {z k } | k
{z }
linear in L
quadratic in L
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 455

We wish to minimise the trace of P since that is the sum of the diagonal elements which we want
to minimise. (Minimising the sum is the same as minimising each individually when positive.)

d trace(Pk )
=0
d Lk
T  T
= 2 CP
k + 2Lk CP
k C T
+ Rk

Solving for the optimum gain gives


 1
Lk = P
kC
T
CP T
k C + Rk (9.96)

where the matrix Lk is called the Kalman gain.

Substituting Eqn. 9.96 into Eqn. 9.95 we have:


 1
Pk = P
k Pk C T
CP T
k C + R k CP
k

or  1
Pk = P
k L k CP T
k C + R k Lk
or even
Pk = (I Lk C) P
k (9.97)
All expressions for computing the updated Pk from the prior P
k are equivalent, but

1. All but Eqn. 9.95 are valid only for the optimum gain. (Eqn. 9.95 is valid for any gain.)
2. The last and simplest expression is the most used, but is suspectable to numerical ill-conditioning.

Prediction updates

The updated estimate x


k is obtained by using the discrete model

x
k+1 = xk

Note that we can ignore wk since it has mean zero and is uncorrelated with any previous ws.

The a priori error is



e
k+1 = xk+1 xk+1
= (xk + wk ) xk
= ek + wk

and the error covariance associated with x


k+1 is

P T T
k+1 = E{ek ek } = E{(ek + wk )(ek + wk ) }
= Pk T + Q

T HE PREDICTION

The best prediction we can make for the new state vector is by using the deterministic part of
Eqn. 9.83. Remember that Eqn 9.83 is our true process S(), and the deterministic part excludes
the noise term since we cannot predict it4 . We may also note that the expected value of the noise
4 If we could predict it, then it would be part of the deterministic part!
456 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

term is zero, so if we were to add anything, it would be zero. Thus the prediction of the new state
estimate is at sample time kT
x
k = xk1|k1 + uk (9.98)
which is precisely Eqn. 9.87. In addition to predicting the states one sample time in the future,
we can also predict the statistics of those states. We can predict the co-variance matrix of the state
predictions Pk|k1 , the a priori co-variance as

Pk|k1 = Pk1|k1 + Q (9.99)

which again is Eqn. 9.91. Without any measurement information, the state estimates obtained
from Eqn 9.98 with variances from Eqn 9.99 are the best one can achieve. If our original dynamic
system is stable, ( is stable), then over time, the estimate uncertainty P will converge to a steady-
state value, and will not further deteriorate, despite the continual presence of process noise. This
is termed statistical steady-state. Conversely however, if our process is unstable, then in most
cases, the uncertainty of the states will increase without bound, and P will not converge.

Stable system or unstable, we can improve these estimates if we take measurements. This intro-
duces the correction step of the Kalman filter.

T HE CORRECTION

We first take a measurement from the real process to obtain yk . We can compare this actual
measurement with a predicted measurement to form an error vector k known as the predicted
measurement residual error
k = yk yk = yk Cxk|k1 (9.100)
It would seem natural to correct our predicted state vector by a magnitude proportional to this
error 
xk|k = xk|k1 + Lk yk Cxk|k1 (9.101)
where Lk is called the Kalman gain matrix at time t = kT and is similar to the gain matrix in
the observers presented in chapter 8. The Kalman gain matrix Lk and co-variance of the state
variables after the correction Pk|k , the a posteriori co-variance, are
 1
Lk = Pk|k1 C CPk|k1 C + Rk (9.102)
Pk|k = (I Lk C) Pk|k1 (9.103)

and note the similarity of these equations to Eqns 9.72 and 9.74. This predictor/corrector scheme
comprises the Kalman filter update.

I NITIALISING THE ALGORITHM

To start the estimation algorithm, we need estimates of the initial state, x0 , and covariance, P0|0 .
In practice, both these variables quickly converge to near optimal values, so the long-term per-
formance of the estimator is not overly sensitive to these parameters. As in the recursive least
squares estimator algorithms, it normally suffices to set P0|0 106 I. If more outputs than states
are available (m > n), then it is possible to solve for the initial states (in a least squares sense) and
for the covariance given an initial measurement y0 ,
 1
P0|0 = C R1 C (9.104)
x0|0 = P0|0 C R1 y0 (9.105)
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 457

The above two formulae can be derived (assuming no input) from Eqn 3.46 where the weight-
ing matrix W = R. However [75, p1539] argues that using the above equations for the initial
estimates is probably not worth the extra effort.

Algorithm 9.2 Discrete Kalman filter algorithm

What we must know before we can implement an evolving Kalman filter:

Model matrices , , C

The covariance matrices Q, R of the noise sequences.

Initial prior state estimate, x


0 and its error covariance, P0 .

At the kth iteration, do the following:

1. Compute Kalman gain, Lk , from a priori information,


 1
Lk = P
k C T
CP T
k C + Rk

2. Correction of the a priori state estimate and its error covariance using the current measure-
ment

xk = x
k + Lk yk Cxk
Pk = (I Lk C) P
k

3. Project ahead or prediction step using the model

x
k+1 = xk

P T
k+1 = Pk + Q

4. Wait out the remainder of the sample time, increment counter, k k + 1, and then go back
to step 1.

Kalman filter example

Simulated results presented in Fig. 9.32 show the improvement of the estimated state when using
a Kalman filter with a randomly generated 2-state discrete model, and plant and measurement
noises with covariances
Q = 0.2I, R=I

Fig. 9.32 compares (a) an open loop model with (b) states computed by directly inverting the
measurement matrix, and (c) states obtained using the Kalman filter. The best practical scheme is
the one where the estimated states best approximates the true states.

To compare the performance quantitatively, the sum of the square errors, (sse), is computed for
each of the three alternatives. The Kalman filter, not surprisingly, delivers the best performance.
458 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Open loop model: sse(ol) = 281 Measurements only: sse(y) = 201 State estimator: sse(KF) = 75
4 4 4
estimated states
2 2 2
1

0 0 0
x

2 2 2
true states
4 4 4

4 4 4

2 2 2
2

0 0 0
x

2 2 2

4 4 4
0 50 100 0 50 100 0 50 100

Figure 9.32: Comparing an openloop model prediction (left column), with direct measurement
inversion (middle column) and the Kalman filter (right column). The Kalman filter has the lowest
state error. See also the performance of the Kalman filter for a range of conditions in Fig. 9.36.

9.5.5 T HE STEADY- STATE FORM OF THE K ALMAN FILTER

Analogous to the optimal regulator gain given in Eqn. 9.58, the Kalman gain matrix Lk in Eqn. 9.101
is also time varying. However in practical instances, this matrix is essentially constant for most
of the duration of the control interval, so a constant matrix can be used with very little drop in
performance. This constant matrix is called the steady-state form of the Kalman gain matrix and
is equivalent to the steady-state LQR gain matrix used in 9.4.

We can solve for the steady-state Kalman gain matrix by finding P such that

 1
P = Q + P PC R + CPC CP (9.106)

is satisfied. To solve Eqn. 9.106 for P, we could use an exhaustive iteration strategy where we
iterate until P converges, or we could notice that this equation is in the form of a discrete alge-
braic Riccati equation for which we can use the function dare. We should note that routine
dare solves a slightly different Riccati form than Eqn. 9.106, so follow the procedure given in
Listing 9.26.

Listing 9.26: Solving the discrete time Riccati equation using exhaustive iteration around
Eqn. 9.106 or alternatively using the dare routine.
P = eye(n); Pnew = 0; % Initial condition for P should be positive definite.
tol = 1e-6; i=0;
3 while norm(P-Pnew)>tol % Repeat until Pk+1 Pk .
P = Pnew; i=i+1;
Pnew = Q + A*P*A' - A*P*C'/(R+C*P*C')*C*P*A'; % Refer Eqn. 9.106.
end

8 % Now try using dare.m


1
% Solve P = Q + P PC R + CPC CP
[Pdare,L,G] = dare(A',C',Q,R); % We are assuming E = I, S = 0
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 459

Once we have the steady-state matrix P, then the steady-state Kalman gain is given by Eqn. 9.102,
 1
L = PC CPCT + R (9.107)

which is known as the prediction-form of the Kalman filter; see also Eqn. 9.110. Note also that I
have dropped the time subscripts on P and L since now they are both time invariant.

The M ATLAB routine dlqe (discrete linear quadratic estimator) uses an equivalent, but numer-
ically robust scheme to compute the steady-state discrete optimum Kalman gain matrix L and
steady-state covariance P.

9.5.6 C URRENT AND FUTURE PREDICTION FORMS

In fact there are two common ways to implement a discrete Kalman filter depending on what com-
putational order is convenient in the embedded system. Ogata refers to them as the Prediction-
type and the Current Estimation type in [152, p880-881], while the Control toolbox in M ATLAB
refers to them as delayed and current in the help file for kalman.

The main difference between the current estimate and the prediction type is that the current esti-
mate uses the most current measurements yk to correct the state xk , whereas the prediction type
uses the old measurements yk1 to correct state xk .

For both cases, we assume our true plant is of the form we used in Eqns 9.83 and Eqn. 9.84,
repeated here with = I

xk+1 = xk + uk + wk
yk = Cxk + vk

and we want to estimate x in spite of the unknown noise terms.

Prediction-type
The prediction-type state estimate is given by

xk+1 = xk + uk + L (yk Cxk ) (9.108)

where the constant covariance and Kalman gain are


 1
P = Q + P PC R + CPC CP (9.109)
 1
L = PCT R + CPCT (9.110)

A block diagram of the prediction type Kalman filter implementation is given in Fig. 9.33.

Current estimation-type
The current-estimation state estimate is given by

xk+1 = zk+1 + L yk+1 Czk+1 (9.111)
where zk+1 = xk + uk (9.112)

and where the covariance is the same as above in Eqn. 9.109, but the Kalman gain is now
 1
P = Q + P PC R + CPC CP (9.113)
 1
L = PCT R + CPCT (9.114)
460 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Process noise, wk Measurement noise, vk

xk Outputs, yk
+ 1 +
q C


Inputs, uk True plant

Kalman gain

xk
+ q 1 C +

Plant model & estimator


State estimates available for feedback, x

Figure 9.33: A block diagram of a steady-state prediction-type Kalman filter applied to a linear
discrete plant. Compare this with the alternative form in Fig. 9.34.

A block diagram of the current estimator type Kalman filter implementation is given in Fig. 9.34.

Note that the prediction-type Kalman filter is obtained simply by multiplying the current estimate
gain with as evident by comparing equations 9.110 and 9.114. Also note that the M ATLAB
routine dlqe solves for the current estimator type.

Listing 9.27 computes the Kalman gain for a discrete plant with matrices
 
1 2  
= , C = 2 3 , and with covariances Q = 2I, R=1
3 4

Listing 9.27: Alternative ways to compute the Kalman gain


>> A = [1,2; 3 4]; C = [2,-3]; % Discrete plant, here A used as .
>> Q = 2*eye(2); R = 1; Gam = eye(2);

>> P = dare(A',C',Q,R) % Solve Eqn. 9.109 for P.


5 P =
27.0038 60.4148
60.4148 148.0538

>> Ke_ce = P*C'/(R+C*P*C') % Current estimator-type Kalman gain, Eqn. 9.114


10 Ke_ce =
-0.1776
-0.4513
>> Ke_pt = A*Ke_ce; % Prediction-type Kalman gain, Eqn. 9.110
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 461

Process noise, wk Measurement noise, vk

xk Outputs, yk
+ q 1 C +


True plant
Inputs, uk

uk1
1
q

xk1 zk
xk q 1 + C +
error,
xk
+ L
State estimates available for feedback
Kalman gain
Plant model & estimator

Figure 9.34: A block diagram of a steady-state current estimator-type Kalman filter applied to a
linear discrete plant. Compare with the alternative prediction form in Fig. 9.33.

Ke_pt =
15 -1.0801
-2.3377

>> Ke = dlqe(A,Gam,C,Q,R) % Note dlqe delivers the current estimator-type Kalman gain
Ke =
20 -0.1776
-0.4513

Both implementations of the steady-state Kalman filter are very similar to the estimator part of
Fig. 8.7. The main difference between this optimal estimator and the estimator designed using
pole-placement in section 8.3 and 8.4 is that we now take explicit notice of the process and mea-
surement noise signals in the design of the estimator.

Of course knowing the states is not much use if you are not going to do anything with that infor-
mation, so of course we could add a feedback controller to this strategy in the same was as we did
for the pole-placement designed simultaneous state feedback controller and estimator in Fig. 8.7.

I N CLOSED FORM SUITABLE FOR SIMULATION

If we combine the plant dynamics with the Kalman filter, we get the following system which is
convenient for simulation.

For the predictor form, by combining the plant dynamics in Eqns 9.83 and Eqn. 9.84 with the
462 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Kalman filter update Eqn. 9.108, we get



       uk
xk+1 0 xk 0
= + wk (9.115)
xk+1 LC LC xk 0 L
vk

where x is the true but unknown state vector, x are the corrected best estimates of x, u are the
manipulated variable inputs, and w and v are the measurement and process noise. We colour the
true states grey, because they are unobservable and hence hard to see.

For the current estimator form, by combining the plant dynamics with Eqns 9.111 and 9.112 we
get

       uk
xk+1 0 xk 0
= + wk (9.116)
xk+1 LC LC xk LC 0 L
vk

For simplicity, let us suppose that we want equal weightings on the state and measurements.
With this simplification we can write

Q = qI, R = rI

where q and r are positive scalars. As q/r 0, L 0 and the estimated dynamic equation
becomes xk+1|k = xk|k1 + uk which is the pure open-loop model prediction. Conversely as
q/r , we are essentially discarding all process information, and only relying on measurement
back-calculation.

Problem 9.5 Design a Kalman gain for the Newell & Lee evaporator. In this example, we are only
measuring level and pressure, so the measurement matrix is now
 
1 0 0
C=
0 0 1

The covariance matrices and optimum discrete gain are

Q = 0.1*eye(8); R = diag([0.1, 5.0]); L = dlqe(Phi,Delta,C,Q,R)

Compare this gain with that on [149, p63]. Is the difference significant?

9.5.7 AN APPLICATION OF THE K ALMAN FILTER

Let us suppose the composition meter on the evaporator failed, but we are required to estimate it
for a state feed-back controller. Since we have an industrial evaporator, we have noise superim-
posed on both the measurements and the process. We can simulate a scenario such as this. We will
use Eqn. 9.116 as a starting point, but we will also add another equation, that of the uncorrected
model predictions. Now we can compare 4 things:

1. The raw measurement unavoidably corrupted by noise, and possibly incomplete;

yk = Cxk + v (9.117)

2. The uncorrected model state predictions

xk+1 = xk + uk (9.118)
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 463

3. The corrected state estimates using both model and the measurements processed by the
Kalman Filter;
xk+1 = f (xk , yk ) (9.119)

4. and of course the true state variable5

xk+1 = xk + uk + w (9.120)

The simulation given in listing 9.28 uses a randomly generated discrete plant model. The actual
manipulated variable trajectories are unimportant for this example, and the measurement noise
variance is one fifth of the process noise variance. To further demonstrate the advantage of the
estimator, I will assume that we have an error in the initial condition of the states.

Listing 9.28: State estimation of a randomly generated discrete model using a Kalman filter.
n = 3;nc = 2;m = 2; % # of states, outputs, inputs
[Phi,Del,C,D]=drmodel(n,nc,m); % generate random discrete model
Gam = eye(n); D = zeros(size(D)); % process noise model & ignore any direct pass-through
4

t = [0:100]'; N = length(t); % discrete sample time


U = [square(t/10), square(t/11)];
x0_true = zeros(n,1); x0_est = [-10 5 -7]' % True & estimated initial conditions

2
9 v=0.5;w=0.1; Q=w*eye(n); R=v*eye(nc); % Noise covariances, Q = w I, R = v2 I
W=sqrt(w)*randn(N,n);V=sqrt(v)*randn(N,nc);% State & measurement noise sequences

[yp,xp] = dlsim(Phi,Del,C,D,U,x0_est); % True deterministic model

14 L = dlqe(Phi,Gam,C,Q,R); % Steady-state optimal Kalman gain, Eqn. 9.107.

Phia = [Phi,zeros(n);L*C, Phi-L*C*Phi]; % Augmented system for simulation, Eqn. 9.116.


Dela = [Del, Gam,zeros(size(L)); Del-L*C*Del, zeros(size(Gam)),L];
Ca=eye(2*n);Da=zeros(2*n,m+n+nc);
19 [y,x] = dlsim(Phia,Dela,Ca,Da,[U,W,V],[x0_true;x0_est]);

x_true = x(:,1:n); % true states, x


x_est = x(:,n+1:2*n); % Kalman estimated/corrected states, x
y_meas = (C*x_true')'+V; % Actual noisy measurement, y
24 y_est = (C*x_est')'; % Predicted measurement, y = Cx

subplot(3,1,1); plot(t,x_true(:,1),t,x_est(:,1),'--',t,xp(:,1));
subplot(3,1,2); plot(t,x_true(:,2),t,x_est(:,2),'--',t,xp(:,2));
subplot(3,1,3); plot(t,x_true(:,3),t,x_est(:,3),'--',t,xp(:,3));

The most obvious scheme to reconstruct an estimate of the true states, is to use only the measure-
ments y. However using this scheme we have two difficulties, first we have noise superimposed
on the measurements, and second, we cannot reconstruct all the states since C is not inevitable,
not even square. Without the estimator, we can only use the incomplete raw output measure-
ments, rather than either the corrected states or filtered measurements. In addition, the poor state
estimate at t = 0 is compensated.

We can improve on the measurements by using the model information and the Kalman filter.
Fig. 9.35 clearly shows that the Kalman estimates, ( ), are better than using simply an open-loop
model estimator.
5 once again only available in simulation land
464 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

10

1
5

x
0

8
6
4
2
0

2
x
true
2 KFcorr
4 No corr

4
2
0
2
3
x

Figure 9.35: Estimation using the Kalman 4


filter. Each of the three trends shows the 6
true state ( ), the corrected estimate ( ), 8
and for comparison, the uncorrected open 0 50 100 150
loop model trajectory, ( ). sample time

9.5.8 T HE ROLE OF THE Q AND R NOISE COVARIANCE MATRICES

The Kalman filter is an optimum model based filter, and is closely related to the optimum LQR
controller described in section 9.4. If properly designed, the Kalman filter should give better
results than a filter using only measurement or model information. We can rank different design
alternatives by comparing the integral of the squared error between the true states x and our
predictions x. The only design alternative for the Kalman filter is the selection of the Q and R
matrices. The theory leads us to believe that if we select these matrices exactly to match the actual
noise conditions, then we will minimise the error between our prediction and the true states.

Normally the covariance matrices Q and R, are positive diagonal matrices, although strictly they
need only to be positive definite. The elements on the diagonal must be positive since they are
variances which are squared terms. If Q is large relative to R, then the model uncertainty is high
and the estimate values will closely follow the measured values. Conversely if R is large relative
to Q, then the measurement uncertainty is large, and the estimate will closely follow the model.

Suppose we have a very simple system discrete dynamic system

xk+1 = 0.5xk + wk (9.121)


yk = xk + vk (9.122)

if we were to design a Kalman filter with model and measurement covariances as Q = 1.5, R =
0.5, we could use dlqe as shown in Listing 9.29.

Listing 9.29: Computing the Kalman gain using dlqe.


>> Phi = -0.5; Gam = 1; C = 1; % System xk+1 = 0.5xk + wk with yk = xk + vk
2 >> Q = 1.5; R = 0.5; % Plant & measurement covariance matrices
>> L = dlqe(Phi,Gam,C,Q,R)
L =
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 465

0.7614

Now if we were lose all confidence in our measurements, effectively that means Q = 0 (or R
becomes very large), and we essentially discard any update. In this case the Kalman gain should
be L = 0.

>> L = dlqe(Phi,Gam,C,0,R) % Excellent measurements, bad plant noise, Q = 0


L =
0

Alternatively if we lose all confidence in our model, then R becomes small, and the Kalman gain
in this case tends to 1 in the case of the current estimator. (Note that R 6= 0 because R must be
strictly positive definite.)

>> L = dlqe(Phi,Gam,C,Q,1e-6) % Bad measurements, R 0


2 L =
1.0000

Now the update expression using the current measurement estimator is

xk = x
k + L(yk xk )

so obviously if L = 1, then xk = x
k + yk xk = yk which means our state estimate is derived
completely from the measurement as anticipated given that we have a very poor model.

In the case of the prediction type Kalman filter, we get

>> L = Phi*dlqe(Phi,Gam,C,Q,1e-6) % Prediction-type KF, Eqn. 9.110.


2 L =
-0.5

which also makes sense intuitively. In this case the update is xk+1 = xk + L(yk xk ) or in this
case xk+1 = 0.5xk 0.5yk + 0.5xk = 0.5yk .

T HE OPTIMALITY OF THE K ALMAN FILTER

The optimality of the Kalman filter is best demonstrated using a Monte-Carlo simulation. The
script given in Listing 9.30 consists of two parts: first the inner loop simulates estimation of a
simple third order plant and calculates the ISE performance,
Z tf
J = (x x) (x x) dt (9.123)
0

using a steady-state Kalman filter, while the outer loop varies the Kalman design q/r ratios. The
inner loop is repeated many times (since slightly different results are possible owing to the differ-
ent noise signals) in order to obtain a statistically valid result.

In Listing 9.30, the true process has q/r = 20. We will then compare the performance of a Kalman
filter using different q/r values as the design parameters. If all goes well, we should detect a
minimum (optimum) near the known true value of q/r = 20.
466 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Listing 9.30: Demonstrating the optimality of the Kalman filter.


qr_v = logspace(-4,3,20)'; % # of points to plot. Span 104 to 103
2 nsum = 10; % # of inner iterations in Monte-Carlo simulation

s = tf([1 0],1); tau = 2; zeta = 0.5;


G = tf((3*s-1)/(7*s+1)/(tau2*s2+2*zeta*s+1)); % G(s) = (7s+1)(3s1
2 s2 +2 s+1)

Ts = 0.2; % sample time, Ts


7 Gdss = ss(c2d(G,Ts,'zoh')); % Discretise model
Phi = Gdss.a; Del = Gdss.b; C= Gdss.c; D = Gdss.d;
[n,m] = size(Del); p = length(C*C'); % system dimensions

t = Ts*[0:1000]'; N = length(t); % time vector


12 U = 0.2*square(t/80).*square(t/15) + square(t/50); % any input
x0 = [-1 1 0]'; % Initial condition for states, x0

Gam = eye(n);R = eye(p); % constant KF design, Q = R = 1


ves = 1e-6; wes = 20*ves; % measurement & process noise
17 x0_est = x0; % [0;0;0]; % x0 is the estimate of x0

Jstats=[];
for i=1:length(qr_v) % outerloop: Design tests
L = dlqe(Phi,Gam,C, qr_v(i)*eye(n),R); % Design Kalman gain, L
22

LC=L*C; % Formulate the extended system


Phib = [Phi, zeros(n); LC, Phi-LC]; % Current predictor
Delb = [Del,Gam,zeros(size(L));Del+L*D,zeros(size(Gam)),L];
Ga = ss(Phib,Delb,eye(size(Phib)),0,Ts);
27

jx=[];
for j=1:nsum % Repeat this simulation many times
W = sqrt(wes)*randn(N,n); V = sqrt(ves)*randn(N,p); % Add noise
x = lsim(Ga,[U,W,V],t,[x0;x0_est]);
32 dx = x(:,[1:3])-x(:,[4:6]); % x x
jx(j) = sum(sum(dx.*dx)); % ISE performance, J , Eqn. 9.123
end % j
Jstats(i,:)=[mean(jx),std(jx),min(jx),max(jx)]; % collect statistics
end % i
37 Jm = Jstats(:,1);
h=loglog(qr_v,Jm,'-',qr_v,Jstats(:,[3 4]),':',...
qr_v,Jm*[1 1]+Jstats(:,2)*[1 -1],'--' );
set(line(qr_v,Jm),'MarkerSize',18,'LineStyle','.');
y = axis; y = y(3:4); h = line(wes/ves*[1,1],y);
42 set(h,'LineStyle','-.','color','k');
xlabel(sprintf('Q/R ratio (# of loops=%3g)',nsum)); ylabel('error')

The plot you obtain should be similar to Fig. 9.36 which was produced using 103 inner loops.
This curve shows that the actual minimum obtained by simulation agrees quite well with the
theoretically expected optimum expected at q/r = 20. Actually the curvature of this function near
the optimum is quite flat which means that one can design the q/r ratio an order of magnitude
in error, and it really does not degrade the system performance much. This is fortunate since it is
almost impossible to choose these design values accurately for any practical industrial process.

A DAPTING THE KF TUNING MATRICES

In any actual implementation of the Kalman filter, one does not usually know a priori appropri-
ate values for the Q and R covariance matrices, and therefore these can be effectively viewed as
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 467

5
x 10
2

1.8

1.6 maximum

1.4

1.2
2
error

0.8 (q/r)*

0.6
Figure 9.36: The performance of a
0.4 Kalman filter for different q/r ratios.
The shaded areas give the approxi-
0.2 mate 1 and 2 uncertainty regions
of the performance. The optimum
0
10
4
10
2 0
10 10
2 performance should be at q/r = 20,
Q/R ratio (# of loops=1000) vertical dot-dashed line.

tuning parameters. Tuning the KF is a cumbersome trial-and-error process that normally requires
expensive state verification. Some techniques make use of the fact that the innovations (or residu-
als, see Eqn 9.100), of an optimal KF should be zero-mean white-noise. One can easily store these
innovations, and test this assumption once a sufficient number have been collected. [27] describes
one technique due to Mehra that updates the Kalman gain and the R matrix if the whiteness con-
dition of the innovations is not satisfied. There are many variations of adaptive KF tuning based
on this theme, but for a variety of reasons, they have found little acceptance in practice.

9.5.9 E XTENSIONS TO THE BASIC K ALMAN FILTER ALGORITHM

There are many extensions to the basic Kalman filter algorithm, and the next few sections briefly
describes some of the more important extensions. (A full treatment is given in [96] and in the later
chapters of [52] to name just a few.) The next few sections describe modifications to the update
equations that are more robust to numerical round-off, using the filter for nonlinear problems,
tricks to save computing time and estimating parameters along with states. These alternatives
are generally preferred to the standard algorithms.

N UMERICALLY STABLE K ALMAN FILTER DESIGN EQUATIONS

The Kalman filter design equations presented in 9.5, whilst correct, do have a tendency to fail on
actual industrial examples. This is due, in part to:

The covariance matrices becoming indefinite, (i.e., losing their positive definiteness), owing
to the limited precision of the machine, and
the poor numerical properties of the covariance update equation.

This section describes an alternative scheme to design a Kalman filter, that tries to avoid these
two problems. Some solution strategies are very similar to the covariance concerns expressed
in section 6.8.4 when considering parameter identification, and if we inadvertently lose positive
468 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

definiteness in the middle of our control calculation, we can always reset to the nearest symmetric
positive definite matrix via nearestSPD as described in section 6.8.4. See [101] or [6, p147] for
further details.

Recall that the measurement update is


xk = xk + K (yk Cxk ) (9.124)
Pk = (I Kk C) Mk (9.125)

The troublesome equation is the measurement covariance update, Eqn 9.125. There are many vari-
ations attempting to improve the numerics of the recursive update, and some of the more simple
rearrangements are given in [122, p69-70]. Two of the most popular are the Upper-Diagonal (U-
D) factorisation due to Bierman, and Potters square root algorithm. The key idea for the U-D
algorithm is that the covariance matrix is factored into

P = UDU
where U and D are upper triangular and diagonal matrices respectively, and these are the matri-
ces that are propagated rather than P. The algorithm for the U-D update is slightly complicated
and difficult to vectorise in a form elegant for M ATLAB, but many programs exist and one version
is given in [205, p167-168].

T HE C HOLESKY MATRIX SQUARE ROOT

Any positive semi-definite symmetric matrix can be factored into the product of a lower triangular
matrix and its transpose. This is analogous to a square root in the scalar sense, and is called the
Cholesky matrix square root. If the Cholesky square root of matrix P is S, then by definition
def
P = SS (9.126)

For example
1 2 3 1 0 0 1 2 3
2 8 2 = 2 2 0 0 2 3 (9.127)
3 2 14 3 2 1 0 0 1
| {z } | {z }
P S
M ATLAB can extract the Cholesky square root from a matrix using the chol command, although
this function actually returns the transpose ST .

U PDATING THE MATRIX SQUARE ROOT

The basic idea of the square root filter is to replace the covariance matrices with their square roots.
The covariance matrix can then never be indefinite, since it is calculated as the product of its two
square roots. Using the square roots also has the advantage that we can compute with double
accuracy. We can define the following square roots;
def def def def
M = SS , P = S + S
+, Q = UU , R = VV (9.128)
Now the measurement covariance update equation can be written as

F = S C (9.129)
 1/2
G = R + F F (9.130)
1
S+ = S SFG (G + V) F (9.131)
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 469

and the correction is


x = x + SFG G1 (y Cx) (9.132)

In Eqn. 9.130, one finds the square root of a matrix. M ATLAB can do this with the sqrtm (square
root matrix) command. The expression A is a shorthand way of writing (A1 ) which is also
equivalent to (A )1 . We can verify Eqn 9.131 by multiplying it with its transpose and comparing
the result with Eqn 9.125. This algorithm is often called Potters algorithm.

Listing 9.31 calculates the steady state Kalman gain using the square root method of Potter. The
functionality and arguments are the same as the built-in Matlab function dlqe.

Listing 9.31: Potters algorithm.


function [K,M,P] = potter(Phi,Gamma,C,Q,R);
2 % [K,M,P] = potter(Phi,Gamma,C,Q,R);
% Potter's square root filter algorithm to design a Kalman filter
% Ref: Algorithm from IEEE trans AC vol12 1971
% See dlqe for argument list & description

7 % Starting estimates & constants


GQG = Gamma*Q*Gamma'; K = zeros(C'); Kprev = -0.5*eye(C'); tol =
1.0e-6; P = zeros(Phi);

V = chol(R)'; % extract roots to start


12

while norm(K-Kprev,'fro') > tol


Kprev = K;
M = Phi*P*Phi' + GQG; % Time update

17 % Square root form for the measmnt update


S = chol(M)'; % Note transpose required from chol
F = S'*C'; G = sqrtm(R+F'*F);
Sp = S - (S*F/(G'))*((G+V)\F');
P = Sp*Sp'; % Will always be positive definite
22 K = S*F/(G')/G;
end % while
return % end function potter.m

Modify the potter algorithm to make it more efficient. (For example the calculation of K is not
strictly necessary within the loop.)

S EQUENTIAL PROCESSING

The Kalman filter was developed and first implemented in the 1960s when computational power
was considered expensive. For this reason, there has been much optimisation done on the algo-
rithm. One of these improvements was to sequentially process the measurements one at a time
rather than as a vector. [6, p142] gives two reasons for this advantage. First if the measurement
noise covariance matrix R is diagonal, there is a reduction in processing time, and second, if for
some reason there is not enough time between the samples to process the entire data, (owing to
an interrupt for example), then there is only a partial loss of update information to the optimised
states, rather than a full loss of improvement.

In the sequential processing formulations of the Kalman filter equations, the Kalman gain matrix
L is in fact a single column vector, and the measurement matrix C is a row vector.
470 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

D IAGONALISING THE R MATRIX

If the measurement noises are correlated, then R will not be diagonal. In this rare case, we can
transform the system to where the measurements are uncorrelated, and work with these. The
transformation uses a Cholesky factorisation. R is factored into

R = SS (9.133)

Now the transformed measurements are

y = S 1 y (9.134)

9.5.10 T HE E XTENDED K ALMAN F ILTER

The Extended Kalman Filter or EKF is used when the process or measurement models are non-
linear. Once we have a nonlinear process, all the guarantees that the Kalman filter will converge
and that the filter will deliver the optimal state estimates no longer apply. Notwithstanding, it is
still worth a try, since in most cases even sub-optimal estimates are better than none. Almost all
the published accounts of using a Kalman filter in the chemical processing industries are actually
an extended Kalman filter applied to a nonlinear model.

The EKF is a logical extension from the normal linear Kalman filter. The estimation scheme is
still divided up into a prediction step and an estimation step. However for the model and mea-
surement prediction steps, the full nonlinear relations are used. Integrating the now nonlinear
process differential equation may require a simple numerical integrator such as the Runge-Kutta
algorithm. Thus the linear model update and measurement equations are now replaced by

xk|k1 = f (xk1|k1 , uk , d, , t) (9.135)


y = g (x) (9.136)

Given the nonlinear model and measurement relations, updating the covariance estimates is no
longer a simple, or even tractable task. Here we assume that over our small prediction step of
interest, the system is sufficiently linear that we can approximate the change in error statistics,
which are only an approximate anyway, with a linear model. Now all the other equations of
the Kalman filter algorithm are the same, except that the discrete state transition matrix , is
obtained from a linearised version of the nonlinear model evaluated at the best current estimate,
xk|k1 following the techniques described in 3.5. The creation of the linear approximation to the
nonlinear model can be done at every time step, or less frequently, depending on the application
and the availability of cheap computing. Linearising the measurement equation, Eqn 9.136 can
be done in a similar manner.

E STIMATING PARAMETERS ONLINE USING THE K ALMAN FILTER

Chapter 6 discussed estimating model parameters online which could then be subsequently used
in an adaptive control scheme. In addition, there are many text books explaining how to offline
regress parameters from experimental data. While there is some debate about the nomenclature,
parameters are generally considered to be constant, or at least change much more slowly than
the states. In a heat exchanger for example, the heat exchange surface area is constant, thus
a parameter, while the flow may vary depending on demand, thus a state. Some variables lie
somewhere in between, they are essentially constant, but may vary over a long time period. The
heat transfer coefficient may be one such variable. It will gradually get less as the exchanger fouls
(owing to rust for example), but this occurs over time spans much larger than the expected state
changes.
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 471

In many engineering applications, however, there is some, (although possibly not complete)
knowledge about the plant, and this partial knowledge should be exploited. This was not the
case for the identification given in chapter 6, where only an arbitrary ARMA model was identi-
fied. The Kalman filter can be extended to estimate both states as given in 9.5 and also param-
eters. Of course once parameters are to be estimated, the extended system generally becomes
nonlinear, (or if already nonlinear, now more nonlinear) so the Extended Kalman filter or EKF must
be used. In this framework, the parameters to be estimated are treated just like states. I typically
simply append the parameter vector () to the bottom of the state vector x. Normally we would
like to assume that the parameter is roughly constant, so it is reasonable to model the parameter
variation as a random walk model,
d
=0+v (9.137)
dt
where v is a random vector of zero mean and known variance. A plot comparing normally dis-
tributed white noise with a random walk is shown in Fig. 9.37 and can be reproduced using:

1 v = randn(500,1); % white noise: mean zero, variance 1.0, Normal distbn.


x = cumsum(v); % generate random walk
plot([v x])

Eqn 9.137 is the simplest dynamic equation for a parameter. In some specialised examples, it
may be worth while to have other simple dynamic models for the parameters, such as random
ramps to model instrument drift for example, but in general this extension is not justified. Of
course, with the extra parameters, the dimension of the estimation problem has increased, and
the observability criteria must still hold.

25

20
Random walk
15

10

5 Figure 9.37: An example of a random


10 walk process: i+1 = i + . The solid
White noise line, , is white noise with a mean of
15 zero, and a variance of 1. The dashed
0 100 200 300 400 500
Sample # line is .

N ON -G AUSSIAN NOISE

The assumption that the measurement and process noise sequences are Gaussian, un-correlated,
and mean zero are usually not satisfied in practice. Coupled with the fact that the true system is
probably nonlinear, it is surprising that an optimal linear estimator, such as the Kalman filter, or
even the EKF works at all.

Most textbooks on regression analysis approximate the residual between actual experimental data
and a model as made up of two components; a systematic error and a random error. The Kalman
filter is based on the assumption that the only error is due to a random component. This in general,
is a very strong assumption, and not satisfied in practice. However coloured noise is treated
theoretically in [52, Chpt 5].
472 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

One easy test to make is how the Kalman filter performs when the noise is correlated. We can
generate correlated or coloured noise by passing uncorrelated or white noise through a low pass
filter.

Problem 9.6 Chapter 7 discussed auto-tuning by relay feedback. One of the key steps in con-
structing an automated tuning procedure is to identify the period and amplitude of the oscillation
in the measurement. Often, however, this is difficult owing to the noise on the measurement,
but we can use an Extended Kalman filter to estimate these quantities in the presence of noise.
Further details and alternative schemes are discussed in (author?) [12]. A signal that is the sum
of a sinusoid and a constant bias can be represented by the following equations;
x2
x1 = 2
x3
x1
x2 = 2
x3
x3 = 0
x4 = 0
y = x1 + x4

and the period and amplitude of the oscillation given by


q
P = x3 , = x21 + x22

The idea is to estimate the states of the process given the output y, and then to calculate the
period and amplitude online.

1. Linearise the dynamic equations to the standard state space form.

2. Design a steady-state Kalman filter.

3. Simulate the estimation process. Plot the estimates of the period and amplitude for a noisy
signal, and for a time varying signal.

9.5.11 C OMBINING STATE ESTIMATION AND STATE FEEDBACK

One of the key aims of the Kalman filter is to provide estimates of unmeasured states that can then
be used for feedback. When we combine state estimation with state feedback we have a Linear
Quadratic Gaussian or LQG controller.

Fig. 9.38 shows how we could implement such a combined estimator and controller in S IMULINK
which essentially follows the scheme given in Fig. 8.7. The actual plant is in grey which is dis-
turbed by green state and measurement noise. The estimator (Kalman) gain is in dark red, and
the state feedback controller is in blue.

The design of the discrete regulator, perhaps using dlqr, and the design of the estimator, perhaps
using dlqe are completely independent. In this implementation we are using the 5 state aircraft
model from [62] listed in Appendix D.2. In the listing below, we will assume the discrete time
model sampled at Ts = 0.1s is Gd. The variance for both the state and measurement noise is set to
0.01.

% Assume Gd is the discrete time model from Appendix D.2.


2 n = size(Gd.a,1); m = size(Gd.b,2); p = size(Gd.c,1); % Plant dimensions, n, m, p
9.5. ESTIMATION OF STATE VARIABLES IN STOCHASTIC SYSTEMS 473

measurement
plant noise
noise
5 3

5 3
5 5 5 5 1 3 3
K*u K*u
3 z 5 x 5 3 y
5
Add Add1
Delta Unit Delay C

Phi

K*u
5 5

Ke
3
K*u
5

5 3
5 5 1 5 3 3
K*u K*u Add3
3 5 5 z xe 5 ye 3
5
Add2
Delta est Unit Delay2 C est

Phi est
5 5
K*u

Kc
u 3 5
K*u

Figure 9.38: An example of a discrete LQG optimal estimator/controller implemented in


S IMULINK. This is simply the estimator from Fig. 9.33 where the estimated states are fed back
through a static gain. (See also Fig. 8.7.)

Q = eye(n,n); R = eye(m); % Weighting for the LQR controller


[Kc,S,E] = dlqr(Md.a, Md.b,Q,R); % Optimal controller gain, u = Kc x

7 Qe = eye(n,n); Re = 100*eye(p); % Weighting for the LQG estimator


[Ke,Se,Ee] = dlqe(Md.a,eye(n),Md.c, Qe,Re) % Kalman estimator gain, Ke

The controlled results are given in Fig. 9.39 which also shows the (typically invisible) true states
in grey.

9.5.12 O PTIMAL CONTROL USING ONLY MEASURED OUTPUTS

The optimal controllers of 9.4 had the major drawback that they required state feedback since
the control law was a function of the states, namely Eqn. 9.57 or u = Kx. We addressed this
problem in 9.5 by first designing an observer to produce some estimated states which we could
use in place of the true, but unknown, states in the controller. However it would be valuable to
be able to design an optimal controller directly that uses output, rather than state feedback. Such
an optimal output feedback controller in fact exists, but it is difficult to compute, and may only
deliver a local optimum. The derivation of the optimum output controller can be found in [123,
chapter 8], and improvements to the design algorithm are given in [168].
474 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

100

80 x2

60

40
x & xe

20

x4
0 x5

x
x 31
20

40
20

0 u1
u3
u

20
u2
40
0 1 2 3 4 5 6 7 8 9 10
time (s)

Figure 9.39: LQG regulatory control of an aircraft. The true, but hidden, states are given in grey.
(See also Fig. 9.38.)

Once again we assume a linear plant

x = Ax + Bu, x(0) = x0
y = Cx

but this time we desire an output controller of the form

u = Ky

which minimises the familiar quadratic performance index


Z
1 T 
J (K) = x Qx + uT Ru dt
2 0
where A, Q nn , B np , C rn , R pp , K pr . Also we assume, Q > 0 (positive
semi-definite) and R > 0.

The optimum gain Kis given by the simultaneous solution to the following design equations:

0 = ATc P + PAc + CT KT RKC + Q (9.138)


0 = Ac S ++X SATc (9.139)
 1
K = R1 BT PSCT CSCT (9.140)

where the closed loop Ac and the covariance of the initial state are defined as
def 
Ac = A BKC, X = E x(0)xT (0)

and the performance index is given by

J = trace (PX) (9.141)


9.6. SUMMARY 475

The coupled nature of the multiple Lyapunov equations mean that we will need to use a numeri-
cal technique to solve for the K that satisfies all three design equations. Our alternatives are:

1. To use a general-purpose unconstrained optimiser to search for the values of K that min-
imise Eqn. 9.141 directly where P is given by the solution to Eqn. 9.138.
2. To use an unconstrained optimiser and exploit the gradient information since the gradient
of J with respect to K is

J  
= 2 RKCSCT BT PSCT
K
where we have computed the auxiliary matrices P, S by solving the two Lyapunov equa-
tions Eqns 9.1389.139 perhaps using the M ATLAB function lyap.
3. To follow a simple fixed-position algorithm where we guess a value for K and then iterate
around Eqns 9.138 to 9.140.

Numerical experiments show that the direct optimisation is prone to infinite looping, and the
fixed-point algorithm can converge to local optima.

9.6 S UMMARY

This chapter dealt with optimal control using classical methods. Optimal control is where we try
to control a process in some best manner. The term best in the engineering field usually means
most economic (either most profit, or least cost). While it is naive to expect our controller to
perform optimally in the field, in practice they have been found to out-perform more conventional
controllers designed via methods such as loop shaping or frequency response for example. I think
of optimal controllers as simply another set of design rules to construct a workable, satisfactory
controller, but would not expect them to be the best in any strict and absolute sense.

A convenient objective function is the quadratic objective function that tries to minimise the
weighted sum of state and manipulated deviations squared over time. The optimal controller
design problem given nonlinear process dynamics and constraints is, in general, too hard to solve
online, although offline open-loop optimal profiles are possible using numerical techniques. The
optimisation problem given linear dynamics is tractable and the solution is termed the linear mul-
tivariable regulator (LMR or LQG). The LMR in its simple guise is a multivariable proportional
controller. The fact that estimation is closely related to regulation was shown in chapter 8 and the
optimal estimator is the Kalman filter.
476 CHAPTER 9. CLASSICAL OPTIMAL CONTROL
C HAPTER 10

P REDICTIVE CONTROL

To be computationally practical, the optimal controllers of chapter 9 required that the system
satisfy certain mathematical restrictions. To use variational calculus, the dynamic models must
be differentiable; to use LQG, we require linear models with no saturation constraints. In contrast
the controllers described in this chapter take into account constraints in the manipulated and
control variables, something which is crucial for practical control applications.

10.1 M ODEL PREDICTIVE CONTROL

Model predictive control, or MPC, first started appearing in a variety of flavours in the late 1970s
and has since then emerged as one of the most popular and potentially valuable control method-
ologies, particularly popular in the chemical processing industries. A good comprehensive sur-
vey of theoretical and practical applications is given in [54, 73, 164, 169].

The central theme of all these variants of predictive control is a controller that uses a process
model to predict responses in a moving or receding horizon. Unlike the optimal control of sec-
tion 9.3.2 or the linear cases following, this control strategy uses a finite optimisation horizon,
(refer Fig. 10.1). At every sample time, the optimiser built into the controller selects the best finite
number of future control moves over a horizon of 10 or so samples, of which the controller will
subsequently implement just the first. At the next sample time, the whole optimisation procedure
would be repeated over the new horizon, which has now been pushed back one step. This is also
known as receding horizon control. Since an optimisation problem is to be solved each sampling
time step, one hopes that the calculations are not too involved!

This long-term predictive strategy coupled with the seeking of multiple optimal future control
moves turns out to be rather a flexible control scheme where constraints in manipulated and
state variables can be easily incorporated, and some degree of tuning achieved by lengthening or
shortening the horizon. By predicting over the dead time and two or three time constants into the
future, and by allowing the controller to approach the setpoint in a relaxed manner, this scheme
avoids some of the pitfalls associated with the traditionally troublesome processes such as those
with inverse response or with excessive dead-time.

However relying predictions from models are not without their problems. I pity the Reserve Bank
of New Zealand which has the unenviable task of predicting the future time-varying interest rates
at quarterly intervals as shown in Fig. 10.2. The dark solid line shows the actual 90-day interest
rates from March 1999 until March 2003 compared to the predictions made at the originating
point. This data was obtained from [196, p18] where is it cautioned: While we consider our
477
478 CHAPTER 10. PREDICTIVE CONTROL

Predicted outputs






Known future

setpoints, r

Past values b


b
b

b b
Prediction horizon, Np

time, t
now

Ts
Past Future
Planned future
manipulated inputs
current input

Control horizon, Nc

current
time

Figure 10.1: The basic idea behind predictive control showing the control and prediction horizons.
The controller tries to select future input values such that the output tends to the setpoint in some
optimal manner.

approach generally useful, there are risks. For example, if the interest rate projections are taken as
a resolute commitment to a particular course of policy action, then those that interpret it as such
will be misled.

Figure 10.2: Successive future 90-day interest


predictions by the Reserve Bank at quarterly
intervals.

The idea behind MPC is that we have a dynamic, possibly nonlinear, plant described as

yi+1 = f (yi , ui ) (10.1)

where we wish the plant output y(t) to follow as closely as practical a possibly varying setpoint
or reference r(t).
10.1. MODEL PREDICTIVE CONTROL 479

To establish the control move, we find a suitable u(t) solve that minimises the quantity

Np Nc
X 2
X
Jt = (rt+i yt+i ) + wu2t+i1 (10.2)
i=1 i=1
| {z } | {z }
prediction horizon control horizon

subject to some input constraints


ui U (10.3)

The set U in the constraint equation, Eqn. 10.3 is the set of allowable values for the input and
typically takes into account upper and lower saturation limits in the actuator. For example in the
case of a control valve, the allowable set would be from 0 (shut) to 100% (fully open).

The optimisation of Eqn. 10.2 involves finding not just the one next control move, but a series of
Nc future moves, one for each sample time over the future control horizon. In practice the con-
trol horizon is often about 515 samples long. The quantity optimised in Eqn. 10.2 is comprised
of two parts. The first expression quantifies the expected error between the future model pre-
dictions y and the future setpoints, r. Typically the future setpoint is a constant, but if changes
in reference are known in advanced, they can be incorporated. The comparison between error
and expected output is considered over the prediction horizon, Hp , which must be longer than the
control horizon.

The second term in Eqn. 10.2 quantifies the change in control moves. By making the weighting
factor, w, large, one can discourage the optimal controller from making overly excited manipu-
lated variable demands.

At each sample time, a new constrained optimisation problem is solved producing an updated
vector of future control. At the next sample instance, the horizons are shifted forward one sample
time T , and the optimisation repeated. Since the horizons are continually moving, they are often
termed receding horizons.

The chief drawback with this scheme is the computational load of the optimisation every sample
time. If the model, Eqn. 10.1, is linear, and we have no constraints, then we are able to analyti-
cally extract the optimum of Eqn. 10.2 without resorting to a time consuming numerical search.
Conversely if we meet active constraints or have a nonlinear plant, then the computation of the
optimum solution becomes a crucial step. Further details for nonlinear predictive control are
given in the survey of [83].

Algorithm 10.1 Model Predictive Control (MPC) algorithm

The basic general predictive control algorithm consists of a design phase and a implementation
phase. The design phase is:

1. Identify a (discrete-time) model of the plant to be controlled.

2. Select a suitable sampling time and choose the lengths of the control and prediction hori-
zons, Hc , Hp , where Hc Hp and the control move weightings, w. Typically Hp is chosen
sufficiently long in order to capture the dominant dynamics of the plant step response.

3. For further flexibility, one can exponentially weight future control or output errors, or not
include the output errors in the performance index for the first few samples. Making the
prediction horizon longer than the control horizon effectively adds robustness to the algo-
rithm, without excessively increasing the optimisation search space.
480 CHAPTER 10. PREDICTIVE CONTROL

4. Incorporate constraints in the manipulated variable if appropriate. In the absence of con-


straints, and as the prediction horizons tend to , the MPC will lead to the standard linear
quadratic regulator as described in 9.4.

The implementation phase is carried out each sampling time.

1. Use an optimiser to search for the next future Nc optimum control moves ut starting from
the current position.

2. Implement only the first optimum control move found from step 1.

3. Wait one sample period, then go back to step 1. Use the previous optimum ut vector, (shifted
forward one place) for the initial estimate in the optimiser.

The following sections demonstrate various predictive controllers. Section 10.1.1 applies a predic-
tive controller to a linear plant, but with input constraints. Because this is a nonlinear optimisation
problem, a generic nonlinear optimiser is used to solve for the control moves. As you will see if
you run the example in Listing 10.1, this incurs a significant computational burden. If we ignore
the constraints, then we can use a fast analytical solution as shown in section 10.1.2, but this loses
one of the big advantages of MPC in the first place, namely the ability to handle constraints. The
M ODEL P REDICTIVE C ONTROL T OOLBOX for M ATLAB described in [28] is another alternative to
use for experimenting with MPC.

10.1.1 C ONSTRAINED PREDICTIVE CONTROL

This section demonstrates a constrained predictive control example using the following linear
non-minimum phase plant,
1.5(1 2.5s)
Gp = 2 (10.4)
4s + 1.8s + 1
but where the manipulated variable is constrained to lie between the values 3.5 < u < 3.

Sampling Eqn. 10.4 at T = 0.75 with a zeroth-order hold gives the discrete system

0.005z 1 0.4419z 2
Gp (z) =
1 1.596z 1 + 0.7136z 2
whose open loop response to a step change is illustrated in Fig. 10.3.

To make the control problem more interesting, we will assume that

Since we have a constraint in the manipulated variable, I will use the general nonlinear con-
strained optimiser fmincon from the O PTIMISATION T OOLBOX. The function to be minimised is
fmodstpc in Listing 10.2 which returns the performance index.

We can see from the discrete step response that a sample time of T = 0.75 is a reasonable choice.
However the response drops before it climbs to the final steady-state, exhibiting an inverse re-
sponse or non-minimum phase behaviour owing to the right hand plane zero at s = +2.5. With
this sort of inverse behaviour, we must take care that the prediction horizon is sufficiently long
to span the initial inverse response which again we can estimate approximately from the discrete
step response. In this case I will set Hp = 20 and optimise for 10 inputs, Hc = 10. The control
weightings are set to w = 1.

The script file to run the simulation and to obtain results similar to Fig. 10.4 is given in Listing 10.1.
10.1. MODEL PREDICTIVE CONTROL 481

Step Response

1.5

1
Amplitude

0.5

1.5(12.5s)
0.5
Gp = 4s2 +1.8s+1 , Ts = 0.75

1
0 5 10 15 20 25 Figure 10.3: An inverse plant model sam-
Time (sec) pled at Ts = 0.75s.

Listing 10.1: Predictive control with input saturation constraints using a generic nonlinear opti-
miser
ntot=150; dt=0.75; t=dt*[0:ntot-1]'; % time vector
2

r = [2,3,-2,-7,2]; % Interesting setpoint


Yspt = ones(size(t(1:fix(length(t)/length(r))+1)))*r(:)';
Yspt = Yspt(1:length(t))'; % Cut off excess data points

7 tau = 2.0; zeta = 0.45; Kp=1.5;taup=2.5; % Create non-minimum phase plant


Gss = ss(c2d(tf(Kp*[-taup 1],[tau2 2*tau*zeta 1]),dt)); % discrete plant

Hc = 10; % control horizon, Hc


Hp = Hc+10; % prediction horizon, Hp > Hc
12 Umin = randn(Hc+1,1); % start guess
Ulb = -3.5*ones(size(Umin)); Uub = 3*ones(size(Ulb)); % bounds
Uopt=zeros(size(t)); Y=zeros(size(t)); % initialise u(t)
x = [0,0]'; % initialise starting point
optns = optimset('Display','off','TolFun',1e-3,'TolX',1e-3); % optimiser options
17

for i=1:ntot-Hp-1; % for each sample time ...


Umin = [Umin(2:end);Umin(end)]; % roll & repeat
Umin = fmincon(@(u)fmodstpc(u,Gss,x,Yspt(i:i+Hp-1),Hp),Umin, ...
[],[],[],[],Ulb,Uub,[],optns);
22

x = Gss.a*x + Gss.b*Umin(1,:)'; % do 1-step prediction


Y(i,:) = Gss.c*x; Uopt(i,:) = Umin(1)'; % collect
end % for i
Yspt(1) = []; Yspt=[Yspt;1]; % shift due to fence-post error
27

subplot(2,1,1); % Now plot results


plot(t,Y,t,Yspt,'r--') % plot results
ylabel('output & setpoint')
subplot(2,1,2);
32 [ts,Us] = stairs(t,Uopt);
plot(ts,Us,t,[ones(size(t))*mean([Ulb,Uub])],'k--')
xlabel('time [s]'); ylabel('Input')

The objective function to be minimised called by the optimiser is given in Listing 10.2. Note that
482 CHAPTER 10. PREDICTIVE CONTROL

auxiliary variables, such as the model, and horizon lengths are passed as anonymous variables
through to the function to be optimised.

Listing 10.2: Objective function to be minimised for the predictive control algorithm with input
saturation constraints
1 function Perf = fmodstpc(U,G,x0,Yspt,Hp)
% Short-term-prediction error called via fmincon via TGPC.M

if Hp > length(U), % normally the case


U = [U;U(end)*ones(Hp-length(U),1)]; % pad extra
6 end % if
% Control weightings N 2
P c
ssdu = sum(diff(U).2); i=1 ut+i1
[Y,t,X] = lsim(G,U,[],x0); % do linear model prediction

Perf = sum((Y-Yspt).2) + ssdu; % Performance index J


11 return %

The controlled performance as shown in Fig. 10.4 is very good, as indeed it should be.

MPC: T =0.75, H =20, H =10


s p c
5
output & setpoint

10
4
umax
2
Input

2
umin
4
0 20 40 60 80 100 120
time [s]

Figure 10.4: Predictive control of a non-minimum phase system with constraints on u with a
control horizon of Hc = 10 and a prediction horizon of Hp = 20. Refer also to Fig. 10.5 for a
zoomed portion of this plot.

Especially note that since we are making use of a vector of future setpoints, the control algorithm
starts preparing for the setpoint change before the actual setpoint changes. This acausal behaviour,
illustrated more clearly in Fig. 10.5, is allowed because we assume we know future setpoints,
improves the response compared with a traditional feedback only PID controller. Naturally the
control becomes looser as the manipulated variable hits the upper and lower bounds.

You will soon realise, if you run this example, that the computational load for the optimal in-
put trajectory is considerable. With such a performance load, one needs to investigate ways to
increase the speed of the simulation. Reasonable starting estimates for the new set of future con-
trol moves are simply obtained from the previous optimisation problem. Good starting guesses
for the nonlinear optimiser are essential to ensure that the controller manages to find a reason-
able output in the time available. Even though our model, ignoring the manipulated variable
constraints, is linear, we have used a general nonlinear optimiser to solve the control problem.
This means that the same algorithm will work in principle for any plant model, whether linear
10.1. MODEL PREDICTIVE CONTROL 483

MPC: T =0.75, H =20, H =10


s p c
output & setpoint 5

Hp Hc Hp
10
4

2
Input

0
Figure 10.5: A zoomed portion of Fig. 10.4
2 centered about the setpoint change around
t = 46 showing clearly the acausal response
4
30 35 40 45 50
possible if we take advantage of the vector of
time [s] future setpoints.

or nonlinear, and any constraint formulation. Naturally there are many extensions to the basic
algorithm that address this issue such as only optimising every n-th sample time, or lowering Nc .

However substantial simplifications can be made if we restrict our attention to just linear plants
and bounded output, input and input rate constraints. Under these restrictions, the optimal MPC
formulation can be re-written as a standard quadratic program, or QP for which there are efficient
solution algorithms. Further details of this are given in [26, 167], and a commercial M ATLAB
implementation is described in [29].

VARYING THE CONTROL AND PREDICTION HORIZONS

Investigating different controller design decisions is computationally expensive for this applica-
tion, but we can relax the termination requirements for the optimiser with little noticeable change
in results. From a computational complexity viewpoint, we want to reduce the search space,
and hence reduce the horizons. However, reducing the horizons excessively, will result in a poor
controlled response.

Fig. 10.6 demonstrates what is likely to happen as the control and prediction horizons grow
shorter. For a short control horizon, Nc = 2, and a reasonably short prediction horizon, Np = 8,
but one that still covers inverse response, the controlled response is still quite reasonable as shown
in Fig. 10.6(a). However decreasing the prediction horizon any further, even just slightly such as
to Np = 7, gives problems as exhibited in Fig. 10.6(b). The inverse response horizon is 5.

MPC ON THE LABORATORY BLACKBOX

Fig. 10.7 shows the result of using the constrained MPC on the black box assuming a constant
model of the blackbox as
0.98
M: G(s) =
(3s + 1)(5s + 1)
sampling at T = 0.75 seconds with a prediction horizon of Hp = 20, and control horizon of
Hc = 10 samples. Clearly we should also estimate the controller model online perhaps using a
recursive least-squares type algorithm.
484 CHAPTER 10. PREDICTIVE CONTROL

GPC: Ts=0.75, Hp= 8, Hc= 2 GPC: Ts=0.75, Hp= 7, Hc= 2


5 5
output & setpoint

output & setpoint


0 0

5 5

10
4 10
4

2 2
Input

Input
0 0

2 2

4 4
0 20 40 60 80 100 120 0 20 40 60 80 100 120
time [s] time [s]

(a) Short control & reasonable prediction horizons, (b) Short control & prediction horizons,Hc = 2, Hp = 7.
Hc = 2, Hp = 8.

Figure 10.6: Varying the horizons of predictive control of a non-minimum phase system with
constraints on u.

BlackBox: MPC t=0.75, H =20, H =10


p c
0.5
output & setpoint

0.5

1
1

0.5
input

0.5

1
0 20 40 60 80 100 120 140 160 180 200
time (s)

Figure 10.7: Constrained MPC applied to the laboratory blackbox with sampling at T = 0.75 sec
and prediction & control horizons Hp = 20, Hc = 10.

10.1.2 D YNAMIC MATRIX CONTROL

If you actually ran the simulation of the general optimal predictive controller from 10.1.1, and
waited for it to finish, you would have stumbled across the one big drawback computational
load. As previously anticipated, one way to quicken the search is to assume a linear system,
ignore any constraints, and make use of the analytical optimisation solution. To do this, we must
express our model as functions of input only, that is, a finite impulse response model, rather than
the more parsimonious recursive infinite impulse response model. There are again many different
flavours of this scheme, but the version here is based on the step response model,


X
y(t) = gi u(t i) (10.5)
i=1
10.1. MODEL PREDICTIVE CONTROL 485

where the gi s are the step response coefficients, and u(t) is the change in input, or ut ut1 , refer
Fig. 10.8. Suppose we start from rest, and make a unit step change in u at time t = 0. Using
Eqn. 10.5, the first few output predictions are
y1 = g1 u0 + g2 u1 + (10.6)
|{z} | {z }
=1 =0
y2 = g1 u1 + g2 u0 + g3 u1 + (10.7)
|{z} |{z} | {z }
=0 =1 =0
y3 = g1 u2 + g2 u1 + g3 u0 + (10.8)

Since all the control moves changes are zero, except u0 = 1, then it follows that y1 = g1 , y2 =
g2 , y3 = g3 and so on. So the values of the output at each sample time yk in response to a unit step
at k = 0 are the step response coefficients, gk . These coefficients form an infinite series in general,
but for stable systems, converge to a limit as shown in Fig. 10.8.
Output, y

gNp
b Approximate
Impulse coeff. g6 b b b

b
b
steady state
h6 = g 6 g 5
b

g4
b

b b b
b
time, t
b
b g3 Ts
0 g1
g2 Step response
coefficients, gi

Input
Unit step
now

Past Future

Figure 10.8: Step response coefficients, gi , and impulse response coefficients, hi , for a stable sys-
tem.

We can write the response of the model in a compact matrix form as



g1 0 0 0
y1 u0
y2
g2 g 1 0 0
u1
.. = g3
g2 g1 0
..


. . .. .. .. .. .
.. . . . .
yN p uNc 1
gNp gNp 1 gNp 2 gNp Nc +1
| {z }
dynamic matrix

yforced = G u (10.9)
The Np Nc matrix G in Eqn. 10.9 is referred to as the dynamic matrix, hence the term dynamic
matrix control or DMC. It is also closely related to a Hankel matrix which provides us with a
convenient, but not particularly efficient, way to generate this matrix in M ATLAB. Eqn. 10.9 tells
us how our system will respond to changes in future moves in the manipulated variable. Specifi-
cally Eqn. 10.9 will predict over the prediction horizon, Np , the effect of Nc control moves. As in
486 CHAPTER 10. PREDICTIVE CONTROL

the general optimisation case, the input is assumed constant, u = 0, past the end of the control
horizon.

All we need to do for our optimal controller, is to find out what to set these future control moves
to, in order that the future outputs are close to the desired setpoints. Actually Eqn. 10.9 only
predicts how our system will behave given future control changes, (termed the forced response),
but it ignores the ongoing effect of any previous moves, (termed the free response). We must
add the effect of the free response, if any, to Eqn. 10.9 to obtain a valid model prediction. This
simple addition of two separate responses is a consequence of the linear superposition principle
applicable for linear systems. We can calculate the free response over the prediction horizon, yfree
simply by integrating the model from the current state with the current constant input for Np
steps. Therefore to ensure our future response is equal to the setpoint, r, at all sample times over
the prediction horizon, we require
r = yfree + yforced (10.10)

Again, since we will make the requirement that the control horizon is shorter than the prediction
horizon, our dynamic matrix, G, is non-square, so the solution of Eqn. 10.9 for the unknown
control moves involves a pseudo inverse. Typically we further weight the control moves to prevent
an over-excited input demands by a positive weight , so the optimal solution to the predictive
control problem is
 1
u = G G + I G (r yfree ) (10.11)

This analytical solution to the optimal control problem is far preferred above the numerical search
procedure required for the general nonlinear case. Note that the matrix inversion in Eqn. 10.11 is
only done once, offline and thus does not constitute a severe online numerical problem. The effect
of the control move weighting improves the numerical conditioning of the matrix inversion.

Fig. 10.9 shows how this control philosophy works for a plant with a prediction horizon of 10 and
a control horizon of 4. The output is intended to follow the setpoint, and in this case the setpoint
change at k = 15 is known in advance and that information is available to the controller. (There
is also a disturbance at k = 19 which is not known to the controller.) A small amount of noise is
added to the output to continually stress the controller.

The control law, Eqn. 10.11, generates 4 control moves in the future, but implements only the first.
The grey trends in both the output and the input show at each sample time what the controller is
thinking will happen in the future. Fig. 10.9(b) shows a zoomed portion of Fig. 10.9(a) just prior
to the step change. The four future predictions are plotted, getting progressively lighter in colour
further into the future.

A DYNAMIC MATRIX CONTROL EXAMPLE

Fig. 10.10 shows the performance of a DMC controller for the non-minimum phase plant of
Eqn. 10.4 from 10.1.1. I generated the step response coefficients elegantly from the transfer func-
tion model simply using the dstep command. For illustration purposes in this simulation I have
varied the control move weighting, in Eqn. 10.11 as = 100 for t < 50, then = 1 for the
interval 50 < t < 100, and then finally = 0.

The controller uses a prediction horizon of Hp = 25, a control horizon of Hc = 8 at a sample time
of T = 0.5. The manipulated variable constraints present in 10.1.1 are relaxed for this example.
As the control move weighting is decreased, the output performance improves at the expense of
vigorous manipulated variable movements. Note that a weighting = 0 is allowable, but will
result in an over-excited controller.

The m-file for this DMC example, but using a constant weight of = 1 is given in Listing 10.3.
10.1. MODEL PREDICTIVE CONTROL 487

Ts= 1, Hp=10, Hc= 4, =10.0

1
0.8
Y & spt

0.6
0.4
0.2
0 Disturbance here

0.2
1

0.5
U

0 5 10 15 20 25 30 35 40
sample interval
(a) Dynamic matrix control showing the predictions for a step change and a disturbance.

Ts= 1, Hp=10, Hc= 4, =10.0


0.025

0.02

0.015

0.01
yk & spt

0.005

0.005

0.01

0.015
1

0.8

0.6
uk

0.4

0.2

0.2
5 6 7 8 9 10 11 12
sample interval

(b) A zoomed portion of the DMC control showing the future predictions at each sample interval
getting lighter in colour as the prediction gets longer.

Figure 10.9: Dynamic matrix predictive control showing at each sample time the future control
moves and the resultant prediction of the plants output given those anticipated control moves.
For example at each sample, the controller generates the next 4 control moves, but actually only
implements the first.
488 CHAPTER 10. PREDICTIVE CONTROL

T =0.5, H =25, H = 8
s p c
1

0.5
Y & spt

0.5

1
1

0.5
U

0.5
= 100 = 20 = 0.1
1
0 50 100 150
sample time

Figure 10.10: DMC predictive control of a non-minimum phase system with Hc = 8, Hp = 25


showing the effect of varying the control move weighting .

Listing 10.3: Dynamic Matric Control (DMC) control


ntot=250; dt=0.5; t=dt*[0:ntot-1]';% time vector
r = [1,3,-1,-2.5,3.5,5.4,-4,-6,5,2.2]; % random square wave setpoint
Yspt = ones(size(t(1:fix(length(t)/length(r))+1)))*r(:)';
4 Yspt = Yspt(1:length(t))'; % Cut off excess data points

Kp=1.5;taup=2.5; tau = 2.0; zeta = 0.45; % process model


Gpc = tf(Kp*[-taup 1],[tau2 2*tau*zeta 1]); % Kp (p s + 1)/( 2 s2 + 2 s + 1)
Gp = c2d(Gpc,dt); % discrete plant
9 [Phi,Del,C,D] = ssdata(Gp); % state space for x0

Hc = 8; % Control horizon, Hc
Hp = 25; % Prediction horizon, Hp > Hc

14 [g,ts] = step(Gp,dt*Hp); % step response coefficients


g(1) = []; % only consider Hp points (ignore first zero)

% Formulate DMC G matrix & inverse once; (offline) Hankel version


G = flipud(hankel(flipud(g))); G(:,Hc+1:end) = [];
19 weight=1; % control move weighting w 0
GG = G'*G; % G G
1
invG = (GG + weight*eye(size(GG)))\G'; % G G + I G

x = [0,0]'; y=C*x; u=0; p=0; % cheat here but Ok


24 Uopt=zeros(size(t)); Y=zeros(size(t)); % initialise u(t)
for i=1:ntot-Hp-1;
du = invG*(Yspt(i:i+Hp-1)-p); % analytical optimisation
u = du(1)+u ; % only take first change
x = Phi*x + Del*u; y = C*x; % apply 1 control move
29

% Calculate future free response (assume all future du=0)


10.2. A MODEL PREDICTIVE CONTROL TOOLBOX 489

[p,px] = dlsim(Phi,Del,C,D,u*ones(Hp+1,1),x);
p(1)=[]; % kill repeat of x0

34 Y(i,:) = y; Uopt(i,:) = u;
end % for i

Yspt(1) = []; Yspt=[Yspt;1];


subplot(2,1,1); plot(t,Y,t,Yspt,':') % plot results
39 subplot(2,1,2); stairs(t,Uopt);

The high fidelity of the controlled response of both predictive control algorithms, (Figures 10.4
and 10.10) is helped by the lack of model/plant mismatch in the previous simulations. In practice,
faced with the inevitable mismatch, one can extend the basic DMC algorithm such as varying the
weights and/or horizon lengths or by including an adaptive model and therefore performing the
matrix inversion online. In some cases, particularly for unstable processes, or for cases where the
sampling rate is either very fast or very slow compare with the step response of the system, the
DMC algorithm will fail primarily owing to the poor conditioning of G. In these cases singular
value decomposition may help.

DMC OF THE BLACKBOX

A DMC controller is tried on the laboratory blackbox in Fig. 10.11. Unlike the GPC example on
484 however, here we will identify the model online using a recursive least-squares estimation
algorithm. The model-based controller assumes the structure

b0 + b1 q 1 + b2 q 2
M: G(q 1 ) =
1 + a1 q 1 + a2 q 2 + a3 q 3

for the blackbox with a sample time of T = 1.5 seconds and a forgetting factor of = 0.995. The
controller horizon is set to 8 samples, and the prediction horizon is set to 20 samples. The control
move weighting is set to 0.2.

10.2 A M ODEL P REDICTIVE C ONTROL T OOLBOX

A Model Predictive Control toolbox developed by Currie, [58, 59] allows one to rapidly build and
test various MPC configurations for a wide variety of linear, or linearisable nonlinear plants with
both soft and hard constraints on the output, the input and input rate of change. The following
sections describe this toolbox in more detail.

10.2.1 A MODEL PREDICTIVE CONTROL GUI

The easiest way to get started is to use the Model Predictive Control toolbox GUI as shown in
Fig. 10.12. In this M ATLAB GUI, one can load and experiment with different plants (and models of
those plants), adjust prediction horizons, operating constraints, and setpoints. What is interesting
to observe in the GUI application are animations of the future output and input predictions given
to the right of the solid line in the scrolling plot.
490 CHAPTER 10. PREDICTIVE CONTROL

0.8

0.6 DMC of the blackbox t=1.50, Hp=25, Hc= 8, =0.20

0.4
output & setpoint

0.2

0.2

0.4

0.6

0.8
1
RLS Model ID: na= 3, nb= 2
0.5
Input

0.5

1
0 50 100 150 200 250 300
time (s)

Figure 10.11: Adaptive dynamic matrix control (DMC) of the blackbox with control horizon Hc =
8 and prediction horizon Hp = 25.

10.2.2 MPC TOOLBOX IN M ATLAB

The MPC toolbox is easy to use in script mode. This section shows how to build a multivariable
MPC controller using the Wood-Berry column plant in Eqn. 3.24 from section 3.2.4. This plant
has 2 inputs (reflux and steam) and two outputs (distillate and bottoms concentration) and we
will use a sampling time of Ts = 2 seconds. For this application we will assume no model/plant
mismatch.

Listing 10.4: Setting up an MPC controller


1 G = tf({12.8, -18.9; 6.6 -19.4}, ...
{ [16.7 1],[21 1]; [10.9 1],[14.4 1]}, ...
'ioDelayMatrix',[1,3; 7,3]) % The Wood-Berry column plant from Eqn. 3.24

Ts = 2; Gd = c2d(G,Ts); % Discretise the plant at Ts = 2 seconds


6 Plant = jSS(Gd); % Convert the LTI model to a special form required for the MPC controller
Model = Plant; % Assume (for the moment) no model-plant mismatch

Now we are ready to specify the controller tuning parameters which include the length of the
control and prediction horizons, the values of the input and output saturation constraints, and
the relative input and output weights in the objective function.
10.2. A MODEL PREDICTIVE CONTROL TOOLBOX 491

Figure 10.12: An MPC graphical user interface

Np = 10; % Prediction Horizon


Nc = 5; % Control Horizon
3

con.u = [-1 1 0.1; ... % Input constraints: umin , umax , umax


-1 1 0.1]; % One row for each input
con.y = [-2 2; % Output constraints: ymin , ymax
-2 2] ;
8

uwt = [1 1]'; % Individual weighting on the inputs


ywt = [1 0.1]'; % We are less interested in controlling bottoms, y2

Kest = dlqe(Model); %Estimator Gain

Now we are ready to initialise the model (and plant). To that, we need to know how many inter-
nals states we have in our discrete model. That number depends on the discretisation method,
and the sampling rate, so the easiest way is to inspect the properties of the plant and model.

>> Plant
Plant =
3 -- jMPC State Space Object --
States: 10
Inputs: 2
Outputs: 2
Sampling Time: 2 sec
8

>> Plant.x0 = zeros(10,1); % Initial condition of the 10 states


>> Model.x0 = zeros(10,1);
492 CHAPTER 10. PREDICTIVE CONTROL

Finally we are ready to design a setpoint trajectory, assemble the MPC controller, and simulate
the results. In this case we are using the high-speed MEX version of the optimisation routines.

T = 200; % Desired simulation length


setp = 0.5*square(0.05*Ts*[0:T]',50)+0.7; % Design interesting setpoint
setp(:,2) = 0.5*square(0.1*Ts*[0:T]',50)-0.7;
setp(1:10,:) = 0;
5

MPC1 = jMPC(Model,Np,Nc,uwt,ywt,con,Kest); % Construct an MPC controller


simopts = jSIM(MPC1,Plant,T,setp); % Set some simulation options

simresult = sim(MPC1,simopts,'Mex'); % Simulate & plot results


10 plot(MPC1,simresult,'summary');
suptitle(sprintf('Horizons: N_p=%2i, N_c=%2i',Np,Nc))

The results for this multivariable MPC example are shown in Fig. 10.13. Note how the control of
y2 is considerably looser than that for y1 . This is a consequence of the output weights which are
ten times higher for y1 . Also note that the change in input u never exceeds the required 0.1.

Horizons: N =10, N = 5
p c
Outputs: yp(k) States: xp(k)
6

1
4
Amplitude

Amplitude

0 2

0
1

2
0 50 100 150 200 0 50 100 150 200

Input: u(k) Change in Input: u(k)


0.6 0.1

0.4 0.05
Amplitude

Amplitude

0.2 0

0 0.05

0.2 0.1
0 50 100 150 200 0 50 100 150 200
Sample Sample

Figure 10.13: Multivariable MPC of the Wood-Berry column.

10.2.3 U SING THE MPC TOOLBOX IN S IMULINK

It is also easy to implement MPC controllers in S IMULINK. Fig. 10.14 shows an MPC controller
from the jMPC toolbox connected to an unstable, non-minimum phase plant

s1
G(s) = (10.12)
(s 2)(s + 1)

which is difficult to control and has been used in various benchmark control studies, [102].

To use the MPC controller block in S IMULINK, we first must design an MPC controller in the same
way as we did in section 10.2.2. Once we have designed the controller, we can simply insert it
into the discrete MPC controller block and connect it to our plant. However in this case instead of
10.2. A MODEL PREDICTIVE CONTROL TOOLBOX 493

double (s1) double


u
mdist (s2)(s+1)
Plant Scope
delu
double
setp
ym
Signal
Generator yp xm

Discrete MPC Controller

Figure 10.14: The MPC block in the S IMULINK environment.

using the true plant inside the model-based MPC controller, we will inadvertently use the slightly
different model
0.95(s 1)
G = (10.13)
(s 2.1)(s + 1)
thereby introducing some model/plant mismatch.

Model = jSS(zpk([1],[2.1 -1],0.95));% Mismatch: Compare Eqn. 10.12 with Eqn. 10.13.
Ts = 0.1; Model = c2d(Model,Ts); % Discretize Model

4 Np = 40; Nc = 5; % Prediction & control Horizons


con.u = [-5 5 0.3]; % Input & output constraints
con.y = [-2 2];
uwt = 1; ywt = 0.5; % input & output weightings
Kest = dlqe(Model);
9

T = 800; setp = square(0.2*Ts*[0:T]',50); % Setpoint


setp(1:10,:) = 0;
Model.x0 = zeros(2,1); % Initial values

14 MPC1 = jMPC(Model,Np,Nc,uwt,ywt,con,Kest); % Build the MPC controller

The last line created an MPC controller object called MPC1 which we can place in the MPC con-
troller block in the S IMULINK diagram in Fig. 10.14. Note that the true plant, Eqn. 10.12 is never ac-
tually used in the MPC controller. The results of the subsequent S IMULINK simulation in Fig. 10.15
show that we hit, but do not exceed, the input rate constraint during the transients.

10.2.4 MPC CONTROL OF AN UNSTABLE 3 DEGREE OF FREEDOM HELICOPTER

The 3-DOF helicopter manufactured by Quanser1 shown in Fig. 10.16 is a popular test rig for
testing multivariable controllers, [57]. The helicopter can be modelled using 6 states,
def  
xT = p p

which are the angles (measured in radians) of the elevation, , the pitch, p, and the travel, , from
a nominal zero position, and their corresponding rates of change. The physical limitations of
the elevation angle are about 50 , while the pitch is limited to about 100 . The travel angle
1 www.quanser.com
494 CHAPTER 10. PREDICTIVE CONTROL

y
1

2
4

u
2

4
0.5

u
0
Figure 10.15: MPC of a SISO non-minimum
phase unstable plant with model/plant 0.5
0 10 20 30 40 50 60 70
mismatch from Fig. 10.14. time

is unlimited due to the slip rings on the central shaft. These state constraints provide a good
incentive to use MPC control.

The angles are directly measured via a rotary encoder, but the time derivatives, (, p, ) must be
estimated either using a state estimator, or some sort of carefully designed differentiator. The two
control inputs are the two independent voltages applied to the two motors driving the front and
def
back rotors, uT = [Vf , Vb ].

Figure 10.16: A 3 degree of freedom


helicopter manufactured by Quanser.

The nonlinear equations of motion for the helicopter are as follows:


 
K f la Tg
= (Vf + Vb ) cos(p) (10.14)
Je Je
 
K f lh
p = (Vf Vb ) (10.15)
Jp
 
K f la
= (Vf + Vb ) sin(p) (10.16)
Jt
with the model parameters either directly measurable from the equipment, or alternatively avail-
able from the Quanser documentation. Note that the sin() and cos() terms introduce both nonlin-
earities and cross-coupling to the model and that the double integrators for all three states mean
that this system is triply unstable.
10.2. A MODEL PREDICTIVE CONTROL TOOLBOX 495

Fig. 10.17 shows how we would control the 3DOF helicopter using an MPC controller. The plant
block in Fig. 10.17(a) expanded in Fig. 10.17(b) contains the the dynamic equations given above,
or we could directly control the actual helicopter. The former allows us to test the controller in
a simulation mode, the latter allows us to actually control some external hardware provided we
have suitable external connections.

(a) MPC control of a Helicopter in S IMULINK

(b) The internals of the nonlinear Helicopter plant above following Eqns 10.1410.16.

Figure 10.17: MPC control of a helicopter.

In order to build a MPC controller of helicopter, the model equations, Eqns (10.14)(10.16) are
entered into M ATLAB as a normal ODE function file as shown in Listing 10.5. We will use this
nonlinear model as the base for the MPC controller design.

Listing 10.5: Nonlinear dynamics of a helicopter


1 function xdot = heli_nlQ(t,x,u,param)
% Nonlinear helicopter dynamics following Eqns 10.1410.16

[Je,La,Kf,Fg,Tg,Jp,Lh,Jt,m_f,m_b,m_w,Lw,g] = param{:}; %Assign Parameters

6 xdot(1,1) = x(4); % elevation velocity


496 CHAPTER 10. PREDICTIVE CONTROL

xdot(2,1) = x(5); % pitch velocity


xdot(3,1) = x(6); % rotation (travel) velocity

xdot(4,1) = (Kf*La/Je)*(u(1)+u(2))*cos(x(2))-Tg/Je; % , Eqn. 10.14


11 xdot(5,1) = (Kf*Lh/Jp)*(u(1)-u(2)); % p, Eqn. 10.15
xdot(6,1) = -(Kf*La/Jt)*(u(1)+u(2))*sin(x(2)); % , Eqn. 10.16
return

We can either use this nonlinear model as the basis of the MPC design (as shown in the listing
below), or we can deliberately use a different model, perhaps to test some model/plant mismatch.

[Kf,m_h,m_w,m_f,m_b,Lh,La,Lw, ...
2 g,K_EC_T,K_EC_P,K_EC_E,Je,Jt,Jp,Fg,Tg] = setup_constants();

C = eye(3,6); % Output Matrix, C. We only measure the angles, not the rates of change
Vop = g*(Lw*m_w - La*m_f - La*m_b)/(2*La*Kf); % Operating Point

7 param = {Je,La,Kf,Fg,Tg,Jp,Lh,Jt,m_f,m_b,m_w,Lw,g}; %parameter cell array


Plant = jNL(@heli_nlQ,C,param); % See Listing 10.5 for the nonlinear ODEs
u0 = [Vop;Vop];

Now we need to linearise the plant about some operating point in order to build the linear internal
model that will be used in the MPC controller.

1 Model = linearize(Plant,u0,zeros(6,1)); % Automatically formulate a linearized model


UD = Model.UserData;

Ts = 0.04; % Discretize model at Ts & add 2 samples of deadtime


set(Model,'iodelay',2*Ts);
6 Model = c2d(Model,Ts);
Model.UserData = UD;

Model = jSS(Model); %Build jSS object

Now that we have a model, we need to specify the MPC tuning options such as control and
prediction horizons, the input and output constraints, the various control weightings, and the
design of the estimator.

1 Np = 45; Nc = 5; % Prediction, Np , & control, Nc , horizons

con=[]; pitch = 30*pi/180; %Constraints


con.u = [-20 20 10;
-20 20 10];
6 con.y = [-Inf Inf; % Or could limit elevation to 27.5
-pitch pitch; % Limit pitch to 30
-Inf Inf]; % Or could limit rotation to 180 degrees

con.slack = [Inf 5e3 Inf]; % Heavy penalty for soft constraints


11

uwt = [1 1]'; %Input & output Weighting


ywt = [10 5 15]';
%Estimator Gain
Q = 2*eye(10); R = 0.1*eye(3); Q(4,4) = 1; Q(5,5) = 1; Q(6,6) = 0.1;
16 Kest = dlqe(Model,Q,R);
10.2. A MODEL PREDICTIVE CONTROL TOOLBOX 497

Finally we are ready to construct the MPC controller. It might be useful to include descriptive
variable names and units.

opts=jMPCset('InitialU',u0,'QPMaxIter',20,...
'InputNames',{'Motor 1 Input','Motor 2 Input'},...
'InputUnits',{'V','V'},...
4 'OutputNames',{'Elevation Angle','Pitch Angle','Rotation Angle'},...
'OutputUnits',{'rad','rad','rad'});

% Build MPC controller object & place in the S IMULINK block in Fig. 10.17(a).
MPC1 = jMPC(Model,Np,Nc,uwt,ywt,con,Kest,opts);

The control object, MPC1 in the above listing, can now be inserted into the S IMULINK MPC control
block in Fig. 10.17(a). We are now ready to test out MPC controller.

The resultant MPC control given in Fig. 10.18 shows the results of an extremely large travel set-
point change of 400, over a full circle. Notice how the MPC controller does violate the (soft) state
constraints at times, but does eventually manage to bring the helicopter under control. For com-
parison, a linear controller such as LQR/LQG would not be able to maintain stability for such a
large setpoint change.

10
elevation [deg]

10

20

30
40

20
pitch [deg]

20

0
travel [deg]

200

400

600
20
Motor voltages

10

0
Figure 10.18: MPC control of a
10
helicopter for an extremely large
20 travel setpoint change of over 1 full
0 5 10 15 20 25 30 35 40
time [s] rotation.

10.2.5 F URTHER READINGS ON MPC

Model predictive control is now sufficiently mature that it has appeared in numerous texts, [136,
176, 201], all of which give an overview of the underlying theory, and insight into the industrial
practice. Industrial applications using commercial implementations such as Pavilion Technolo-
gies Pavilion8, Honeywells RMPCT or Perceptive Engineering have been surveyed in [164]. A
commercial MPC toolbox for M ATLAB is available in [29] while freeware versions also exist from
[58] and [4].
498 CHAPTER 10. PREDICTIVE CONTROL

10.3 O PTIMAL CONTROL USING LINEAR PROGRAMMING

Optimal control problems quickly demand excessive computation, which all but eliminates them
from practical real-time applications. If however our model is a linear discrete-time state-space
model, our performance objective is to minimise the sum of the absolute value, rather than the
more normal least-squares, and we have linear constraints on the manipulated variables, then
the general constrained optimisation problem becomes a linear program. Linear programs or LPs
are much more attractive to solve than a general constrained nonlinear optimisation problem of
comparable complexity because, in theory, they can be solved in a finite number of operations,
and in practice they are far more successful at solving problems with more than 50 variables and
100 constraints on modest hardware. The theory of linear programming is well known, and if the
problem is poorly conditioned, or not well-posed, then the software will quickly recognise this,
ensuring that we do not waste time on impossible, or poorly thought out problems.

There are many computer codes available to solve linear programs, including various routines in
O PTI or linprog from the O PTIMIZATION T OOLBOX. An introduction to linear programming
with emphasis on the numerical issues with M ATLAB is given in [206], and an experimental ap-
plication is given in [150] from which this section was adapted.

As in the predictive control development, we want a controlled response that minimises the error
between setpoint, r, and the actual state, say something like Eqn. 9.11
Z
J = f (x, r)
0

where we could either use the absolute value of the error, l1 norm, or perhaps minimise the maxi-
mum error, l norm. In both cases these problems can be converted to a convex linear program.

We will also consider only a finite time horizon, and since the model is discrete, we will use a
simple sum in place of the integral. The optimisation problem is then to choose the set of future
manipulated variables uk over the time horizon such that the performance index
N X
X n
J = |ri xi | (10.17)
k=0 i=1

is minimised subject to the process model. In Eqn. 10.17, n is the number of states, and N is the
number of samples in the time horizon, (typically around 1020). If desired, Eqn. 10.17 could be
further modified by adding state weighting if some states are more important than others.

10.3.1 D EVELOPMENT OF THE LP PROBLEM

The hardest part of this optimal controller is formulating the LP problem. Once that is done, it
can be exported to any number of standard numerical LP software packages to be solved.

We will use a discrete state model for the predictive part of the controller.

xk+1 = xk + uk (10.18)
Unwinding the recursion in Eqn. 10.18, we can calculate any subsequent state, xk given the ma-
nipulated variable history, and the stating condition, x0 , (refer Eqn. 2.89,

xk = k x0 + k1 u0 + k2 2 u1 + + uk1 (10.19)
If rk is the setpoint at sample time k, then the error is

rk xk = rk k x0 k1 u0 uk1 (10.20)
10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING 499

We desire to minimise the sum of the absolute values of these errors, as given in Eqn. 10.17, but
the absolute value sign is difficult to incorporate in a linear programming format. We can instead
introduce two new non-negative variables,

1i := ri xi
if ri xi (10.21)
2i := 0

otherwise

1i := 0
if xi ri (10.22)
2i := xi ri

Note that we could write the above four relations in M ATLAB without the IF statement as some-
thing like:

e1 = abs(r-x)*round(sign(r-x)/2 + 0.5)
2 e2 = abs(x-r)*round(sign(x-r)/2 + 0.5)

for scalar x and r.

Using definitions Eqns 10.2110.22, the absolute value is

|ri xi | = 1i + 2i (10.23)

and the error is


ri xi = 1i 2i (10.24)

Both relations are valid at all times because only one of either 1 or 2 is non-zero for any particular
state at any particular time.

Re-writing the objective function, Eqn. 10.17 in terms of the new variables using Eqn. 10.23 gives
the linear relation without the troublesome absolute values.
N X
X n
J = (1i + 2i )k (10.25)
k=0 i=1

The penalty paid is that we have doubled the number of decision variables, thereby doubling the
optimisation space.

Eqn. 10.25 is a linear objective function suitable for a linear program. The constraints arise from
the model dynamics which must be obeyed. Substituting Eqn. 10.24 into Eqn. 10.20 gives N
blocks of equality constraints, each with n equations. To make things clearer, the first few blocks
are written explicitly out below.

11 21 +u0 = r1 x0
| {z }
r1 x1
12 22 +2 u0 + u1 = r2 2 x0
| {z }
r2 x2
..
.
+k1 u0 + k2 u1 + + uk1 = rk k x0
| 1k {z 2k}
rk xk
500 CHAPTER 10. PREDICTIVE CONTROL

Naturally this is best written in a matrix form.


u0

u
1

.
.
.

u
N 1

0 0 I I
11 r1 x0
2
0 I I 2 x0
r

21 = ..

(10.26)

.. .. .. ..
.
. . . 0 . 12
N x0

r N
N 1 N 2

I I
22


..
.


1N

2N

The dimensions of the partitions in this large constraint matrix are given in Fig. 10.19 where
N is the number of future sample horizons, n is the number of states, and m is the number of
manipulated variables.

m n
z }| { z }| {

n 0 0 I I n

0 I I
Nn .. .. .. .. ..
. . . . .


I I
N 1 N 2
| {z }| {z }
Nm 2N n

Figure 10.19: LP constraint matrix dimensions

The constraint matrix in Fig. 10.19 is a N n (N m + 2N n) matrix, the decision variable vector has
N m manipulated variables and 2N n auxiliary variables comprising of (N m + 2N n) 1 vector,
and the right-hand side is a N n 1 vector.

In general applications it is necessary for the control variable, u, to be unrestricted in sign, which
is counter to the formulation required in most linear programming packages. To achieve this we,
must introduce two further new variables, both non-negative, for each manipulated variable.

u := s t, s, t 0 (10.27)

If the unrestricted u > 0, then t = 0, otherwise u is negative and s = 0. This substitution modifies
10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING 501

the A matrix in the constraint equation, Eqn. 10.26, to



0 0 0 0 I I


0 0 I I


.. .. .. .. .. .. ..

. . . 0 . . . 0 .
N 1 N 2 N 1 N 2
I I

and the decision variables are now


h i
.. ..
s0 s1 sN 1 . t0 t1 tN 1 . 11 2N

Using this ordering of decision variables, the objective function, Eqn. 10.25, is the sum of the s.
In linear programming, the objective function is written as J = c x, where c is termed the cost
vector, and x is the decision variable vector to be optimised. In our case the cost vector is given as
" #
0 0 0 1 1 1 1 1
c= | {z } | {z } (10.28)
2N m 2N n

Note that the manipulated variables do not explicitly enter the objective function. Naturally the
ones in Eqn. 10.28 could be replaced by different positive individual time and state weightings, if
so desired.

This completes the formal development of the LP as far as the unconstrained optimum controller
is concerned. However since we are already solving a constrained optimisation problem, there
is little overhead in adding extra constraints such as controller upper and lower bounds to the
problem. In practice the manipulated space is limited, (a control valve can only open so far, an
electric drive is limited in top speed), so we are interested in constraints of the form

ulower u uupper (10.29)

These constraints can be appended to the other constraint equations.

S IMPLIFICATIONS WHEN USING THE OPTIMISATION TOOLBOX

When using linprog.m from the O PTIMIZATION T OOLBOX, certain simplifications are possible
to the standard LP problem form. These simplifications reduce the problem size, and therefore
also reduce the solution time. Since the algorithm does not make the non-negative constraint
assumption, we do not need to introduce the variable transformation given by Eqn. 10.27, and
suffer the associated growth of the constraint matrix. This is not the case in general, if we were to
use other LP routines such as described in [206] or [78], then we would need to use non-negative
variables.

Most LP programs assume the constraints are given as Ax b, i.e. less-than inequalities rather
than equalities as in Eqn. 10.26. While the conversion is trivial, it again increases the number of
constraint equations. However the routine linprog conveniently allows the user to specify both
equality and inequality constraints.

The third simplification is to exploit how linprog handles constant upper and lower bounds
such as needed in Eqn. 10.29. These are not added to the general constraint matrix, Ax b, but
are supplied separately to the routine. All these modifications reduce the computation required
to solve the problem.
502 CHAPTER 10. PREDICTIVE CONTROL

O PTIMAL CONTROL USING LINEAR PROGRAMMING

Listing 10.6 demonstrates a finite time-horizon optimal controller with a randomly generated
stable multiple-input/multiple output (MIMO) model. The optimisation problem is solved using
the linear program linprog from the O PTIMISATION T OOLBOX.

Listing 10.6: Optimal control using linear programming


n = 2; m = 2; % # of states & manipulated variables
[Ac,Bc,C,D]=rmodel(n,1,m); % Some arbitrary random model for testing
3

Gc = ss(Ac,Bc,C,D);
dt = min(1. ./abs(eig(Ac)))/0.5; % sparse sample time reasonable guess
G = c2d(Gc,dt,'zoh');
G.d = zeros(size(G.d)); % clear this
8 if rank(ctrb(G)) 6= n
error('System not controllable')
end % if

N = 20; % future optimisation horizon


13 x0 = randn(n,1); % random start point
r = [ones(floor(N/2),1)*randn(1,n); ...
ones(ceil(N/2),1)*randn(1,n)]; % setpoint change

% start construction of the LP matrix


18 Phin = G.b; Aeq = G.b; % better for when m > 1
for i=2:N
Phin = G.a*Phin;
Aeq = [Aeq; Phin]; % rough method
end
23

Z = zeros(size(G.b)); % padding
for i=2:N
Aeq = [Aeq, [Z;Aeq(1:(N-1)*n,(i-2)*m+1:(i-1)*m) ]];
end % for i
28

% weighting: Could use blkdiag here (but messy)


W = eye(n); % change if want different state weighting
A2 = zeros(N*n,2*N*n); % initialise
for i=1:N
33 A2((i-1)*n+1:i*n,2*(i-1)*n+1:2*i*n) = [W, -W];
end % for
Aeq = [A2,Aeq]; % put +ve variables before general variables

% Now right-hand side .....


38 U = zeros(N+1,m); % unforced, therefore zero input (drop start term)
[yuf,t,xuf]=lsim(G,U,[],x0); % simulate unforced response
xuf(1,:) = []; % drop start term since dlsim repeats it.
b = r - xuf; % error
b = b'; beq = b(:); % columnerise
43

% Now objective function


[nr,nc] = size(Aeq); % current size
c = zeros(1,nc); % row vector for the moment
c(1:2*N*n) = 1; % unity weights for the moment !
48

% Set up A*x < b (or -Ax > 0)


b = zeros(2*N*n,1); % All #s are positive
A = zeros(2*N*n,nc)
10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING 503

A(:,1:2*N*n) = -eye(2*N*n);
53

tic; % now ready to solve LP ..................


ulow = -13; uhigh = 14; % upper/lower constraints
xlower = [zeros(2*N*n,1); ulow*ones(N*m,1)];
xupper = [1e6*ones(2*N*n,1); uhigh*ones(N*m,1)];
58 xsol=linprog(c',A,b,Aeq,beq,xlower,xupper); % first 2Nn are +ve
toc

% extract solution, if found


U = xsol(2*N*n+1:end); % must reshape if m=
6 1

63 U = reshape(U,m,length(U)/m)'; % MI system

[y,t,xopt] = lsim(G,U,[],x0); % Now test system response


%k = [0:length(xopt)-1]';
[ts,rs] = stairs(t,r); [ts,Us] = stairs(t,U);
68

subplot(2,1,2);
plot(ts,Us,t,ones(size(t))*[ulow, uhigh],'--');
h = line(ts,Us); set(h,'LineWidth',2);
%set(gca,'YLim',[-4,4.5])
73 xlabel('sample #'); ylabel('Manipulated')

% Sub-sample output response


nsub = 500; % # of sub-samples total (approx 500)
ts2 = ts(:,1) + [1:length(ts)]'/1e10; ts2(1) = 0; % destroy monotonicity
78 t2 = linspace(0,t(end),nsub)';
U2 = interp1(ts2,Us,t2);

[y,t2,xoptsc] = lsim(Gc,U2,t2,x0);
subplot(2,1,1);
83 plot(ts(:,1),rs,':',t2,xoptsc);
h = line(t,xopt); set(h,'Marker','.','MarkerSize',24,'LineStyle','none');
ylabel('state'); title('LP optimal control')

You should run the simulation a number of times with different models to see the behaviour of
the optimal controller. One ideal 2-input/2 state case is given in Fig. 10.20. Here the states reach
the setpoint in two sample times, and remain there. The manipulated variables remain inside the
allowable upper (4) and lower (3) bounds. It is clear that the trajectory is optimal, since the sum
of the absolute error at the sample points is zero. One cannot do better than that.

In another case, the manipulated bounds may be active and consequently the setpoint may not be
reached in the minimum time, and our objective function will not reach zero. In Fig. 10.21, both
steady-states are reachable, but owing to the upper and lower bounds on u, they cannot be met
in just two sample times. In other words we have hit the manipulated variable constraints during
the transients.

Our system does not need to be square, Fig. 10.22 shows the results for a 3-state, 2 input problem.
In this case, one state is left floating. Some small inter-sample ripple is also evident in this case.

Note that since this is a predictive controller, and it knows about future setpoint changes, it an-
ticipates the changes and starts moving the states before the setpoint change. Fig. 10.23 shows the
case for a 2 3 system where owing to the active upper and lower bounds on u, the controller
knows that it cannot reach the setpoint in one sample time, hence it chooses a trajectory that will
minimise the absolute error and in doing so, starts the states moving before the setpoint.
504 CHAPTER 10. PREDICTIVE CONTROL

LP optimal control
3

state 1 2

1
0

0.2

0.4
state 2

0.6

0.8

1
20
Manipulated

10

10

20
0 1 2 3 4 5 6
sample #

Figure 10.20: LP optimal control with no active manipulated variable bounds. Note the perfect
controlled performance. We hit the setpoint in one sample time, and stay there. One cannot
do much better than that within the limitations of discrete control. The plot was generated by
Listing 10.6.

LP optimal control
2

1
state

2
0 1 2 3 4 5 6 7 8 9

4
Figure 10.21: LP optimal control
2
with active manipulated variable
Manipulated

constraints. During transients, the 0


manipulated variables hit the up-
2
per constraints, so compromising the
controlled performance for a short 4
0 1 2 3 4 5 6 7 8 9
period. sample time
10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING 505

LP optimal control
2

0
state

3
0 1 2 3 4 5 6 7 8 9

2
Manipulated

2
Figure 10.22: Non-square LP optimal
4 control with 3 states and 2 inputs.
0 1 2 3 4 5 6 7 8 9
sample time Note the small inter-sample ripple.

LP optimal control
1

0.5

0
state

0.5

1.5

2
0 1 2 3 4 5 6 7 8 9

2
Manipulated

2
Figure 10.23: LP optimal
4 control showing acausal
0 1 2 3 4 5 6 7 8 9
sample time behaviour
506 CHAPTER 10. PREDICTIVE CONTROL
A PPENDIX A

L IST OF SYMBOLS

S YMBOL D ESCRIPTION U NITS m phase margin s


t time s , actual PID parameters -
T, t sample time s d, dead time samples, s
P period s
time constant s
K Gain signal power
shape factor - W weight kg
() Dirac delta function - M model -
f (t) function of time - S system -
f (T ) sampled function f (T ) - vector of parameters -
L() Laplace transform - error vector -
Z() z transform - t1/2 t-statistic -
s Laplace variable - cov() co-variance
z z transform variable - C past input/output data -
q 1 backward shift operator - K gain matrix -
radial velocity radians/s P co-variance matrix -
f frequency Hz I identity matrix -
V(x) scalar Lyapunov function - forgetting factor -
P, Q positive definite matrices - f factor 4
J Jacobian matrix - E{} expected value
num numerator - t sample time s
den denominator -
x state vector n1
M mass kg u input vector m1
F material flow kg/s y output vector r1
density kg/m3 d disturbance vector p1
h height m A system matrix nn
A cross sectional area m2 B control matrix nm
P pressure differential kPa D disturbance matri x np
T temperature K, C C measurement matrix rn
H Enthalpy kJ transition matrix nn
I current mA discrete control matrix nm
cp heat capacity kJ/kg.K discrete disturbance matrix n p
T temperature differential K time delay s
Q, q heat loss (gain) kJ/s # whole samples delay -
J moment of inertia - z augmented state vector -
angle radians Co controllability matrix -
relative gain matrix - Ob observability matrix -
u manipulated variable - K controller gain -
error - L observer gain -
i integral time s J,j performance index $
d derivative time s Q, R weighting matrices -
Kc proportional gain - x mean of x -
Ku ultimate gain - x standard deviation of x -
u ultimate frequency rad/s , Lagrange multipliers
Pu ultimate period s H Hamiltonian

507
508 APPENDIX A. LIST OF SYMBOLS

termination criteria

S UBSCRIPTS D ESCRIPTION
N Nyquist
u,l upper & lower
ss steady-state

S UPERSCRIPTS D ESCRIPTION
x estimate (of x)
x mean (of x)
x desired value (of x)
A PPENDIX B

U SEFUL UTILITY FUNCTIONS IN


M ATLAB

The routine in Listing B.1 adds two or more polynomials of possibly differing lengths. The resultant
polynomial order is the same as the maximum order of the input polynomials. Note that this routine
pads with zeros to the left so that all the polynomials are of the same size. It assumed (but not checked)
that the polynomials are row vectors in descending order.

Listing B.1: Polynomial addition.


function R = polyadd(A,varargin)
% Adds two or more row vectors of possibly differing lengths.
% R(x) = A(x) + B(x) + C(x) + .
np = length(varargin); nmax = length(A);
5 for i=1:np % find maximum order in the input argument list
nmax = max(nmax,length(varargin{i}));
end % if
R = [zeros(1,nmax-length(A)),A];
for i=1:np
10 varargin{i} = [zeros(1,nmax - length(varargin{i}),1), varargin{i}];
R = R+varargin{i};
end % if
return

This routine is useful for checking the results of a Diophantine solution, A(x)R(x) + B(x)S(x) = T (x).

Tc = polyadd(conv(A,R),conv(B,S))

The routine in Listing B.2 convolves or multiplies two or more polynomials. Unlike polynomial addi-
tion, we do not need to ensure that all the polynomials are of the same length.

Listing B.2: Convolution (polynomial multiplication) of two or more polynomials.


function y = mconv(a,varargin)
2 % Multiple convolution of polynomial vectors
% R(x) = A(x)B(x)C(x) .

% Would be faster to do divide & conquer, and lots of smaller first.


if nargin < 2
7 error('Need two or more arguments')
509
510 APPENDIX B. USEFUL UTILITY FUNCTIONS IN MATLAB

end % if

y = conv(a,varargin{1}); % do the first pair


for i=2:nargin-1 % now do any remaining
12 y = conv(y,varargin{i});
end return

Listing B.3 strips the leading zeros from a row polynomial, and optionally returns the number of zeros
stripped which may, depending on the application, be the deadtime.

Listing B.3: Strip leading zeros from a polynomial.


function [Br,i] = stripleadzeros(B)
2 % Strip the leading zeros from a (vector) polynomial & return deadtime (# of leading zeros)
% [Br,i] = stripleadzeros(B)

for i=0:length(B)-1
if B(i+1) % if Bi 6= 0
7 Br = B(i+1:end);
return % jump out of routine
end
end % for
return
A PPENDIX C

T RANSFORM PAIRS

time function Laplace transform z-transform


f (t) F (s) Z
1 z
unit step 1
s z1
impulse (t) 1
1 Tz
unit ramp t
s2 (z 1)2
2 T 2 z(z + 1)
t2
s3 2(z 1)3
n!
tn
sn+1
1 z
eat
s+a z eaT
1 T zeaT
teat
(s + a)2 (z eaT )2
s
(1 at)eat
(s + a)2
a z sin(aT )
sin(at)
s2 + a2 z 2 2z cos(aT ) + 1
s
cos(at)
s2 + a2
bt
e eat 1
ab (s + a)(s + b)
a
ebt sin(at)
(s + b)2 + a2
s+b
ebt cos(at)
(s + b)2 + a2
1 aebt beat 1
+
ab ab(b a) s(s + a)(s + b)

511
512 APPENDIX C. TRANSFORM PAIRS
A PPENDIX D

U SEFUL TEST MODELS

The following are a collection of multivariable models in state-space format which can be used to test
control and estimation algorithms.

D.1 A FORCED CIRCULATION EVAPORATOR

The linearised forced circulation evaporator developed in [149, chapter 2] and mentioned in 3.2.3 in
continuous state-space form, x = Ax + Bu, is

0 0.10455 0.37935
A= 0 0.1 0 (D.1)
2 2
0 1.034 10 5.4738 10
3.636 102

0.1 0.37266 0 0.36676 0.38605 0 0
B = 0.1 0 0 0 0.1 0.1 0 0
0 3.6914 102 7.5272 103 3.6302 102 3.2268 103 0 3.5972 103 1.7785 102
(D.2)

with state and input variables defined as in Table 3.3,


 T
x= L2 P2 x2 (D.3)
h iT
u= .. (D.4)
F2 P100 F200 . F3 F1 x1 T1 T200

In this case we assume that the concentration cannot be measured online, so the output equation is
 
1 0 0
y= x (D.5)
0 1 0

A=[0, 0.10455 , 0.37935


0, -0.1 , 0
0, -1.034e-2 , -5.4738e-2];
4 B=[-0.1, -0.37266 , 0 , 0.36676 , 0.38605 , 0 , -3.636e-2, 0
-0.1, 0 , 0 , 0 , 0.1 , 0.1, 0 , 0
0 , 3.6914e-2, -7.5272e-3, 3.6302e-2, 3.2268e-3, 0 , 3.5972e-3, 1.7785e-2];
C= eye(2,3); D = zeros(2,8);

9 G = ss(A,B,C,D);
G.StateName = {'L2','P2','x2'}
513
514 APPENDIX D. USEFUL TEST MODELS

G.InputName = {'F2','P100','F200','F3','F1','x1','T1','T200'}
G.Notes = {'Newell & Lee Evaporator'};

D.2 A IRCRAFT MODEL

The following model is of an aircraft reported in [62, p31]. The state model is

0 0 1.132 0 1 0 0 0
0 0.0538 0.1712 0 0.0705 0.12 1 0

A= 0 0 0 1 , B = 0
0 0 0 (D.6)


0 0.0485 0 0.8556 1.013 4.419 0 1.665
0 0.2909 0 1.0532 0.6859 1.575 0 0.0732

1 0 0 0 0
C= 0 1 0 0 0 (D.7)
0 0 1 0 0

where the aircraft states & control inputs are defined as:

state description units measured


x1 altitude m yes
x2 forward velocity m/s yes
x3 pitch angle degrees yes
x4 pitch rate, x3 deg/s no
x5 vertical speed m/s no
u1 spoiler angle deg/10
u2 forward acceleration m/s2
u3 elevator angle deg

Note that we only measure altitude, velocity and pitch angle, we do not measure the rate states.

A reasonable initial condition is


 
x0 = 10 100 15 1 25 .

A = [0,0,1.132 0 -1; 0 -0.0538 -0.1712 0 0.0705; 0 0 0 1 0; ...


0. 0.0485, 0 -0.8556 -1.013; 0 -0.2909 0 1.0532 -0.6859];
3 B = [0 0 0; -0.12 1 0; 0 0 0; 4.419 0 -1.665; 1.575 0 -0.0732];
C = eye(3,5);
D = zeros(3,3);

G = ss(A,B,C,D);
8 G.StateName={'altitude','forward vel','pitch angle','pitch rate','vertical speed'};
G.StateUnit={'m','m/s','degrees','deg/s','m/s'};
G.InputName={'spoiler angle','for acc''n','elevator angle'};
G.InputUnit={'deg/10','m/s2','deg'};
G.notes ={'Aircraft model from Dorato et al'};
B IBLIOGRAPHY

[1] J. Abate and P.P. Valko. Multi-precision Laplace transform inversion. International Journal for
Numerical Methods in Engineering, 60:979993, 2004. 32, 34

[2] James L. Adams. Flying Buttresses, Entropy, and O-rings: The World of an Engineer. Harvard Univer-
sity Press, 1991. 10

[3] Advantech Co. Ltd. PC-Multilab Card Users Manual, 1992. 14

[4] Johan Akesson. MPCtools: A toolbox for simulation of MPC controllers in Matlab. Technical
report, Lund Institute of Technology, Lund, Sweden, January 2006. 497

[5] Brian D. O. Anderson and John B. Moore. Optimal Control. Linear quadratic methods. PrenticeHall,
1971. 430

[6] Brian D.O. Anderson and John B. Moore. Optimal Filtering. Information and System Sciences.
Prentice-Hall, 1979. 468, 469

[7] Jim Anderson, Ton Backx, Joost Van Loon, and Myke King. Getting the most from Advanced
Process Control. Chemical Engineering, pages 7889, March 1994. 11, 12

[8] Anon. Experience with the X-15 Adaptive Flight Control System. Technical Report NASA TN
D-6208, NASA Flight Research Centre, Edwards, California, Washington, D.C., March 1971. 321

[9] K-E Arzen. Realisation of expert system based feedback control. PhD thesis, Department of Automatic
control, Lund Institute of Technology, Lund, Sweden, 1987. 153

[10] K. J. Astrom. Computer Control of a Paper Machine an Application of Linear Stochastic Control
Theory. IBM Journal of Research and Development, 11(4):389405, 1967. 326

[11] K. J. Astrom. Maximum likelihood and prediction error methods. Automatica, 16:551574, 1980.
284

[12] K. J. Astrom. ZieglerNichols Auto-tuners. Technical Report LUTFD2/(TFRT3167) /01025/,


Lund University, Lund, Sweden, 1982. 172, 176, 472

[13] K. J. Astrom and B. Wittenmark. On Self Tuning Regulators. Automatica, 9:185199, 1973. 326

[14] Karl J. Astrom and Tore Hagglund. Advanced PID Control. ISA, Research Triangle Park, NC, USA,
2006. 136, 184

[15] Karl-Johan Astrom. Introduction to stochastic control theory. Academic press, 1970. 326

[16] Karl-Johan Astrom and Tore Hagglund. Automatic tuning of PID controllers. Instrument Society of
America, Research triangle park, NC, 1988. 136, 145, 158, 169, 170, 325

[17] Karl-Johan Astrom and Tore Hagglund. Revisiting the Ziegler-Nichols step response method for
PID control. Journal of Process Control, 14:635650, 2004. 155, 156
515
516 BIBLIOGRAPHY

[18] Karl-Johan Astrom and Bjorn Wittenmark. Adaptive control. AddisionWesley, 1989. 12, 308, 309,
314
[19] Karl-Johan Astrom and Bjorn Wittenmark. Adaptive Control. AddisionWesley, 2 edition, 1995.
245, 332, 353
[20] Karl-Johan Astrom and Bjorn Wittenmark. Computer-Controlled Systems: Theory and Design.
PrenticeHall, 3 edition, 1997. 64, 145, 361, 375
[21] K.J. Astrom and R.D. Bell. Drum-boiler dynamics. Automatica, 36(3):363378, 2000. 184
[22] K.J. Astrom and B. Wittenmark. Adaptive Control. AddisonWesley, 1989. 330
[23] K.J. Astrom and B. Wittenmark. Computer Controlled Systems: Theory and Design. PrenticeHall, 2
edition, 1990. 315, 346, 349, 356
[24] Gregory L. Baker and J. P. Gollub. Chaotic dynamics: an introduction. Cambridge University Press,
1990. 125
[25] Yonathan Bard. Nonlinear parameter estimation. Academic Press, 1974. 279
[26] R.A. Bartlett, A. Wachter, and L. T. Biegler. Active set vs. interior point strategies for model predic-
tive control. In Proceedings of the American Control Conference, pages 42294233, Chicago, Illinois,
USA, 2000. 483
[27] B. Bellingham and F.P. Lees. The detection of malfunction using a process control computer: A
Kalman filtering technique for general control loops. Trans. IChemE, 55:253265, 1977. 467
[28] Alberto Bemporad, Manfred Morari, and N. Lawrence Ricker. Model Predictive Control Toolbox. The
MathWorks Inc., 2006. 480
[29] Alberto Bemporad, Manfred Morari, and N. Lawrence Ricker. Model predictive control toolbox 3.
Technical report, The Mathworks, 2009. 483, 497
[30] B. Wayne Bequette. Nonlinear control of chemical processes: A review. Ind. Eng. Chem. Res.,
30:13911413, 1991. 387
[31] A. Besharati Rad, Wai Lun Lo, and K.M. Tsang. Self-tuning pid controller using newton-raphson
search method. Industrial Electronics, IEEE Transactions on, 44(5):717725, Oct 1997. 158, 258
[32] Torsten Bohlin. A grey-box process identification tool: Theory and practice. Technical Report IR-
S3-REG-0103, Department of Signals Systems & Sensors, Royal Institute of technology, SE-100 44
Stockholm, Sweden, August 2001. 87
[33] C. Bohn and D.P. Atherton. An Analysis Package Comparing PID Anti-Windup Strategies. IEEE
Control Systems, 15:3440, 1995. 145
[34] G.E.P. Box and G.M. Jenkins. Time Series Analysis: Forecasting and Control. HoldenDay, 1970. 238,
244, 310
[35] M. Braun. Differential Equations and their Applications. SpringerVerlag, 1975. 88, 123
[36] John W. Brewer. Kronecker products and matrix calculus in system theory. IEEE Transactions on
Circuits and Systems, 25(9):772781, September 1978. 83, 84
[37] E.H. Bristol. On a new measure of interaction for multivariable process control. IEEE transactions
on Automatic Control, AC11(133), 1966. 105
[38] Jens Trample Broch. Principles of Experimental Frequency Analysis. Elsevier applied science, London
& New York, 1990. 222
[39] Robert Grover Brown and Patrick Y.C. Hwang. Introduction to Random Signals and Applied Kalman
Filtering. John Wiley & Sons, 2 edition, 1992. 444
BIBLIOGRAPHY 517

[40] Arthur E. Bryson and Yu-Chi Ho. Applied Optimal Control. Ginn & Co., Waltham, Mass., 1969. 407
[41] James L. Buchanan and Peter R. Turner. Numerical Methods and Analysis. McGrawHill, 1992. 123
[42] F. Buchholt and M. Kummel. A Multivariable Selftuning Regulator to Control a Double Effect
Evaporator. Automatica, 17(5):737743, 1981. 95
[43] Richard L. Burden and J. Douglas Faires. Numerical Analysis. PWS Publishing, 5 edition, 1993. 123
[44] C. Sidney Burrus, James H. McClellan, Alan V. Oppenheim, Thomas W. Parks, Ronald W. Schafer,
and Hans W. Schuessler. Computer-Based Exercises for Signal Processing using Matlab. PrenticeHall,
1994. 17, 205
[45] B. Carnahan and J.O. Wilkes. Numerical solution of differential equations - an overview. In R. Mah
and W.O. Seider, editors, FOCAPD 80, Henniker, New Hampshire, 1980. 88
[46] C.C. Hang and K. J. Astrom and Q.G. Wang. Relay feedback auto-tuning of process controllers
A tutorial overview. J. Process Control, 12:143162, 2002. 181
[47] B. Chachuat. Nonlinear and dynamic optimization: From theory to practice. Tech-
nical report, Automatic Control Laboratory, EPFL, Switzerland, 2007. Available from
infoscience.epfl.ch/record/111939/files/. 407, 419
[48] Edward R. Champion. Numerical Methods for Engineering Applications. Marcel Dekker, Inc., 1993.
123
[49] Cheng-Liang Chen. A simple method of on-line identification and controller tuning. 35(12):2037
2039, 1989. 162
[50] Cheng-Liang Chen. A closed loop reactioncurve method for controller tuning. Chemical Engi-
neering Communications, 104:87100, 1991. 162, 169
[51] Robert Chote. Why the Chancellor is always wrong. New Scientist, page 26, 31 October 1992. 93
[52] C.K. Chui and G. Chen. Kalman Filtering with Real-Time Applications. SpringerVerlag, 1987. 444,
467, 471
[53] C.K. Chui and G. Chen. Linear Systems and Optimal Control. SpringerVerlag, 1989. 407, 409, 420
[54] David Clarke. Advances in Model-Based Predictive Control. Oxford University Press, 1994. 188, 322,
477
[55] David W. Clarke. PID Algorithms and their Computer Implementation. Trans. Inst. Measurement
and Control, 6(6):305316, Oct-Dec 1984. 136
[56] J.D. Cryer. Time series analysis. PWS, 1986. 244
[57] Jonathan Currie, Arrian Prince-Pike, and David I. Wilson. A cost effective high-speed auto-coded
embedded model predictive controller. International Journal of Intelligent Systems Technologies and
Applications, 13(1/2):3755, 2014. 493
[58] Jonathan Currie and David I. Wilson. A Model Predictive Control toolbox intended for rapid
prototyping. In Tim Molteno, editor, 16th Electronics New Zealand Conference (ENZCon 2009), pages
712, Dunedin, New Zealand, 1820 November 2009. 489, 497
[59] Jonathan Currie and David I. Wilson. Lightweight Model Predictive Control intended for em-
bedded applications. In 9th International Symposium on Dynamics and Control of Process Systems
(DYCOPS), pages 264269, Leuven, Belgium, 57 July 2010. 489
[60] James B. Dabney and Thomas L. Harman. Mastering Simulink 4. PrenticeHall, 2001. 402
[61] J.J. DiStefano, A.R. Stubberud, and I.J. Williams. Feedback and Control Systems. Schaums Outline
Series, McGraw-Hill, 1990. 74
518 BIBLIOGRAPHY

[62] Peter Dorato, Chaouki Abdallah, and Vito Cerone. Linear-quadratic Control: An Introduction.
PrenticeHall, 1995. 424, 472, 514

[63] Richard Dorf. Modern control systems. AddisonWesley, 5 edition, 1989. 68

[64] M. Drouin, H. Abou-Kandil, and M. Mariton. Control of complex systems. Methods and technology.
Plenum Press, 223 Spring St, New York NY 10013, 1 edition, 1991. 12, 407

[65] D.B. Ender. Control analysis and optimisation. Technical report, Instrument Soc. America, Tech-
mation, Tempe, Arizona 85282, 1990. 153

[66] D. Grant Fisher and Dale E. Seborg. Multivariable Computer Control. North-Holland, 1976. 95

[67] Samuel C. Florman. The Existential Pleasures of Engineering. St. Martins Press, New York, 1976. 11

[68] T. R. Fortescue, L. S. Kershenbaum, and B. E. Ydstie. Implementation of self-tuning regulators


with variable forgetting factors. Automatica, 17(6):831835, 1981. 301

[69] Lloyd D. Fosdick, Elizabeth R. Jessup, and Carolyn J.C. Schauble. Elements of Matlab. High
Performance Scientific Computing, University of Colorado, January 1995. Available from
cs.colorado.edu/pub/HPSC. 4, 5

[70] A. S. Foss. Critique of chemical process control theory. American Inst. Chemical Engineers J,
19(2):209214, 1973. 46

[71] G.F. Franklin and J.D. Powell. Digital control of dynamic systems, pages 131183. AddisonWesley,
1980. 43, 71, 374

[72] G.F. Franklin, J.D. Powell, and M.L. Workman. Digital Control of Dynamic Systems. Addison
Wesley, 3 edition, 1998. 279

[73] C.E. Garcia, D.M. Prett, and M. Morari. Model predictive control: Theory and practice A survey.
Automatica, 25(3):335348, 1989. 477

[74] Paul Geladi and Bruce R. Kowalski. Partial Least-Squares Regression: A Tutorial. Analytica Chim-
ica Acta, 185:117, 1986. 113

[75] S.F. Goldmann and R.W.H. Sargent. Applications of linear estimation theory to chemical pro-
cesses: A feasibility study. Chemical Engineering Science, 26:15351553, 1971. 457

[76] Graham C. Goodwin, Stefan F. Graebe, and Mario E. Salgado. Control System Design. Prentice
Hall, 2001. 129

[77] Felix Gross, Dag Ravemark, Peter Terwiesch, and David Wilson. The Dynamics of Chocolate in
Beer: The Kinetic Behaviour of theobroma cacao Paste in a CH3 CH2 OHH2 OCO2 Solution. Journal
of Irreproducible Results, 37(4):24, 1992. 240

[78] Tore K. Gustafsson and Pertti M. Makila. L1 Identification Toolbox for Matlab. Abo Akademi, Fin-
land, August 1994. From ftp.abo.fi/pub/rt/l1idtools. 501

[79] I. Gustavsson. Survey of Applications of Identification in Chemical and Physical Processes. Auto-
matica, 11:324, 1975. 238

[80] Juergen Hahn, Thomas Edison, and Thomas F. Edgar. A note on stability analysis using bode
plots. Chemical Engineering Education, 35(3):208211, 2001. 75

[81] A. Helbig, W. Marquardt, and F. Allgower. Nonlinearity measures: definition, computation and
applications. Journal of Process Control, pages 113123, 2000. 133

[82] M.A. Henson and D.E. Seborg. Input-output Linearization of General Nonlinear Processes. Amer-
ican Inst. Chemical Engineers J, 36(11):17531757, 1990. 125, 387
BIBLIOGRAPHY 519

[83] Michael A. Henson. Nonlinear model predictive control: current status and future directions.
Computers in Chemical Engineering, 23(2):187202, 1998. 479

[84] Michael A. Henson and Dale E. Seborg. Nonlinear Process Control. Prentice Hall, Saddle River,
New Jersey, 1997. 93

[85] David R. Hill. Experiments in Computational Matrix Algebra. The Random House, 1988. 56

[86] David M. Himmelblau. Process Analysis by Statistical Methods. John Wiley & Sons, 1970. 118, 260

[87] Jesse B. Hoagg and Dennis S. Bernstein. Nonminimum-phase zeros: Much to do about nothing.
IEEE Control Systems Magazine, pages 4557, June 2007. 52, 60, 185

[88] Roger A. Horn and Charles R. Johnson. Topics in Matrix Analysis. Cambridge University Press,
1991. 83

[89] Morten Hovd and Sigurd Skogestad. Pairing Criteria for Unstable Plants. In AIChE Annual Meet-
ing, page Paper 149i, St. Louis, Nov 1993. 355

[90] P. J. Huber. Robust regression: Asymptotics, conjectures, and Monte Carlo. Ann. Math. Statist.,
1(5):799821, 1973. 284

[91] Enso Ikonen and Kaddour Najim. Advanced Process Identification and Control. Marcel Dekker, 2002.
266

[92] Vinay K. Ingle and John G. Proakis. Digital Signal Processing using Matlab V.4. PWS Publishing
Company, 1997. 206

[93] Rolf Isermann. Paramater adaptive control algorithmsA tutorial. Automatica, 18(5):513528,
1982. 314

[94] M.L. James, G. M. Smith, and J. C. Wolford. Applied Numerical Methods for Digital Computation with
Fortran and CSMP. Harper & Row, 2 edition, 1977. 123

[95] R.K. Jaspan and J. Coull. Trajectory Optimization Techniques in Chemical Reaction Engineering
II: Comparison of Methods. American Inst. Chemical Engineers J, 18(4):867869, July 1972. 411

[96] Andrew H. Jazwinski. Stochastic Processes and Filtering Theory. Academic Press, 111 Fifth Avenue,
New York, 1970. 444, 467

[97] A. Johnson. LQG Applications in the Process Industries. Chemical Engineering Science, 48(16):2829
2838, 1993. 407

[98] Michael A. Johnson and Mohammad H. Moradi. PID COntrol: New Identification and Design Meth-
ods. SpringerVerlag, London, UK, 2005. 136, 138

[99] Arthur Jutan and E. S. Rodriguez II. Extension of a New method for On-Line Controller Tuning.
The Canadian Journal of Chemical Engineering, 62:802807, December 1984. 162

[100] Thomas Kailath. Linear Systems. PrenticeHall, 1980. 407

[101] Paul G. Kaminski, Arthur E. Bryson, and Stanley F. Schmidt. Discrete Square Root Filtering: A
survey of current techniques. IEEE Trans. Automatic Control, AC-16:727735, 1971. 468
[102] L.H. Keel and S.P. Bhattacharyya. Robust, fragile, or optimal? Automatic Control, IEEE Transactions
on, 42(8):1098 1105, Aug 1997. 492

[103] David Kincaid and Ward Cheney. Numerical Analysis. Mathematics of Scientific Computing. Brook-
s/Cole, 1991. 123
[104] K.J. Astrom and P. Eykhoff. System identification A survey. Automatica, 7(2):123 162, 1971.
238
520 BIBLIOGRAPHY

[105] Costas Kravaris and Jeffery C. Kantor. Geometric Methods for Nonlinear Process Control. 1 Back-
ground. Ind. Eng. Chem. Res., 29:22952310, 1990. 387
[106] Costas Kravaris and Jeffery C. Kantor. Geometric Methods for Nonlinear Process Control. 2 Con-
troller Synthesis. Ind. Eng. Chem. Res., 29:23102323, 1990. 387
[107] Erwin Kreyszig. Advanced Engineering Mathematics. John Wiley & Sons, 7 edition, 1993. 222
[108] B. Kristiansson and B. Lennartson. Robust and optimal tuning of PI and PID controllers. IEE
Proc.-Control Theory Applications, 149(1):1725, January 2002. 193
[109] R. Kulhavy and M.B. Zarrop. On a general concept of forgetting. Int. J. Control, 3(58):905924,
1993. 316
[110] Benjamin C. Kuo. Automatic Control Systems. PrenticeHall, 6 edition, 1991. 41
[111] Benjamin C. Kuo. Automatic Control Systems. PrenticeHall, 7 edition, 1995. 371, 372
[112] A. Kuznetsov. On the convergence of the Gaver-Stehfest algorithm. SIAM J. Numerical Analysis,
51(6):29842998, May 2013. 33, 34
[113] I. D. Landau. Identification in closed loop: a powerful design tool (better design models, simpler
controllers). Control Engineering Practice, 9:5165, 2001. 314
[114] Yoan D. Landau. Adaptive Control: The model reference approach. Marcel Dekker, 1979. 311
[115] Leon Lapidus and John H. Seinfeld. Numerical Solution of Ordinary Differential Equations. Academic
Press, 1971. 88
[116] Alan J. Laub. Matrix Analysis for Scientists and Engineers. Society for Industrial and Applied Math-
ematics, 2004. 83
[117] P.L. Lee and G.R. Sullivan. Generic Model Control (GMC). Computers in Chemical Engineering,
12(6):573580, 1988. 377
[118] T.H. Lee, Q.G. Wang, and K.K. Tan. Knowledge-based process identification from relay feedback.
Journal of Process Control, 5(6):387397, 1995. 181, 182
[119] S. Lees and J.O. Hougen. Determination of pneumatic controller characteristics by frequency
response. Ind. Eng. Chem., 48:1064, 1956. 248
[120] J. R. Leigh. Control Theory: A guided tour. IEE Control Series 45. Peter Peregrinus Ltd., 1992. 76
[121] A. Leva, C. Cox, and A. Ruano. Hands-on pid autotuning: a guide to better utilisation.
In Marek Zaremba, editor, IFAC Professional Brief, chapter 3. IFAC, 2002. Available from
www.ifac-control.org/publications/pbriefs/PB_Final_LevaCoxRuano.pdf. 241
[122] Frank L. Lewis. Optimal Estimation. John Wiley & Sons, 1986. 287, 453, 468
[123] Frank L. Lewis and Vassilis L. Syrmos. Optimal Control. John Wiley & Sons, 2 edition, 1995. 473
[124] R.J. Litchfield, K.S. Campbell, and A. Locke. The application of several Kalman filters to the
control of a real chemical reactor. Trans. IChemE, 57(2):113120, April 1979. 382, 383
[125] Tao Liu, Qing-Guo Wang, and Hsiao-Ping Huang. A tutorial review on process identification from
step or relay feedback test. Journal of Process Control, 23(10):1597 1623, 2013. 170, 241
[126] Lennart Ljung. System Identification: Theory for the User. PrenticeHall, 1987. 238, 245, 273, 285, 315
[127] Lennart Ljung. System Identification Toolbox, for use with Matlab. The MathWorks Inc., May 1991.
238, 308, 315
[128] Lennart Ljung. System Identification: Theory for the User. PrenticeHall, 2 edition, 1999. 282
BIBLIOGRAPHY 521

[129] Lennart Ljung. Perspectives on system identification. Annual Reviews in Control, 34(1):112, 2010.
239

[130] Lennart Ljung and Torkel Glad. Modeling of Dynamic Systems. PrenticeHall, 1994. 238

[131] Lennart Ljung, Rajiv Singh, and Tianshi Chen. Regularization features in the system identification
toolbox. In 17th IFAC Symposium on System Identification, volume 48, pages 745 750, Beijing,
China, 1921 October 2015 2015. 285

[132] Charles F. Van Loan. The ubiquitous kronecker product. Journal of Computational and Applied
Mathematics, 123:85100, 2000. 83

[133] W. L. Luyben. Process Modeling, Simulation, and Control for Chemical Engineers. McGraw-Hill, 2
edition, 1990. 88

[134] W.L. Luyben. Process modelling simulation and control for chemical engineers. McGraw-Hill, 1973. 100

[135] Paul A. Lynn and Wolfgang Fuerst. Introductory Digital Singnal Processing with Computer Applica-
tions. John Wiley & Sons, 2 edition, 1994. 209, 222

[136] J. M. Maciejowski. Predictive Control with Constraints. PrenticeHall, 2002. 497

[137] Sven Erik Mattsson. On Modelling and Differential/Algebraic Systems. Simulation, pages 2432,
January 1989. 133

[138] Cleve Moler and John Little. Matlab Users Guide. The MathWorks Inc., December 1993. 1

[139] B. De Moor, P. De Gersem, B. De Schutter, and W. Favoreel. DAISY: A database for identification
of systems. Journal A, 38(3):45, 1997. 317

[140] Manfred Morari. Three critiques of process control revisited a decade later. In D.M. Prett, C.E.
Graca, and B.L. Ramaker, editors, The Shell Process control workshop, pages 309321. Butterworths,
1987. 46, 365

[141] Manfred Morari. Process control theory: Reflections on the past and goals for the next decade. In
D.M. Prett, C.E. Graca, and B.L. Ramaker, editors, The second Shell Process control workshop, pages
469488. Butterworths, December 1988. 97

[142] Manfred Morari. Some Control Problems in the Process Industries. In H.L. Trentelman and J.C.
Willems, editors, Essays on Control: Perspectives in the Theory and its applications. Birkhauser, 1993.
145

[143] Manfred Morari. Model Predictive Control: Multivariable Control Technique of Choice in the
1990s. In David Clarke, editor, Advances in Model-Based Predictive Control, pages 2237. Oxford
University Press, 1994. 46, 223

[144] Manfred Morari and Evanghelos Zafiriou. Robust Process Control. PrenticeHall, 1989. 103, 127

[145] Jerzy Moscinski and Zbigniew Ogonowski. Advanced Control with Matlab and Simulink. Ellis Hor-
wood, 1995. 248

[146] Frank Moss and Kurt Wiesenfeld. The benefits of Background Noise. Scientific American, pages
5053, August 1995. See also Amateur Scientist in the same issue. 197

[147] Ivan Nagy. Introduction to Chemical Process Instrumentation. Elsevier, 1992. 12

[148] Ioan Nascu and Robin De Keyser. A novel application of relay feedback for PID auto-tuning. In
IFAC CSD03 Conference on Control systems Design, Bratislava, Slocak Republic, 2003. 181

[149] R.B. Newell and P.L. Lee. Applied Process Control A Case Study. PrenticeHall, 1989. 95, 96, 434,
462, 513
522 BIBLIOGRAPHY

[150] R.E. Nieman and D.G. Fisher. Experimental Evaluation of Optimal Multivariable Servo-control
in Conjunction with Conventional Regulatory Control. Chemical Engineering Communications, 1,
1973. 498

[151] R.E. Nieman, D.G. Fisher, and D.E. Seborg. A review of process identification and parameter
estimation techniques. Int. J of control, 13(2):209264, 1971. 238

[152] Katsuhiko Ogata. Discrete Time Control Systems. PrenticeHall, 1987. 15, 23, 25, 27, 30, 37, 39, 51,
56, 57, 65, 76, 78, 80, 85, 110, 232, 315, 343, 356, 357, 359, 371, 374, 376, 408, 459

[153] Katsuhiko Ogata. Discrete-Time Control Systems. PrenticeHall, 1987. 26

[154] Katsuhiko Ogata. Modern Control Engineering. PrenticeHall, 2 edition, 1990. 74, 78, 80, 83, 111,
154, 171, 356, 357, 359, 398

[155] B. A. Ogunnaike, J. P. Lemaire, M. Morari, and W. H. Ray. Advanced multivariable control of a


pilot-plant distillation column. 29(4):632640, July 1983. 100

[156] Alan V. Oppenheim and Ronald W. Schafer. Digital Signal Processing. PrenticeHall, 1975. 208

[157] Overschee and De Moor. Subspace Identification. Kluwer Academic Publshers, 1996. 279

[158] Chris C. Paige. Properties of Numerical Algorithms Related to Computing Controllability. IEEE
transactions on Automatic Control, AC26(1):130138, 1981. 359

[159] Sudharkar Madhavrao Pandit and Shien-Ming Wu. Time Series and System Analysis with Applica-
tions. Wiley, 1983. 310

[160] Rajni V. Patel, Alan J. Laub, and Paul M. van Dooren. Numerical Linear Algebra Techniques for
Systems and Control. IEEE Press, 1994. A selected reprint volume. 65, 359, 360

[161] Linda Petzold. Differential/Algebraic Equations are not ODEs. SIAM J. Sci. Stat. Comput.,
3(3):367384, 1982. 127

[162] C.L. Phillips and H.T. Nagle. Digital Control System Analysis and Design. PrenticeHall, 2 edition,
1990. 202

[163] W.H. Press, B.P. Flannery, S.A Teukolsky, and W.T. Vetterling. Numerical Recipes: The Art of Scientific
Computing. Cambridge University Press, 1986. 112, 222, 223, 230, 411

[164] S. Joe Qin and Thomas A. Badgwell. A survey of industrial model predictive control technology.
Control Engineering Practice, 11(7):733764, July 2003. 477, 497

[165] R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical
Computing, Vienna, Austria, 2013. ISBN 3-900051-07-0. 4, 120

[166] W. Fred Ramirez. Process Control and Identification. Academic Press, 1994. 407, 419, 424

[167] C. V. Rao, S. J. Wright, and J. B. Rawlings. Application of interior-point methods to model predic-
tive control. Journal of Optimization Theory and Applications, 99(3):723757, 1998. 483

[168] T. Rautert and E.W. Sachs. Computational design of optimal output feedback controllers. Techni-
cal Report Nr. 95-12, Universitat Trier, FB IV, Germany, June 1995. 473

[169] James B. Rawlings. Tutorial overview of Model Predictive Control. IEEE Control Systems Magazine,
20(3):3852, June 2000. 477

[170] W. Harmon Ray and Julian Szekely. Process Optimization. John Wiley & Sons, 1973. 407, 411

[171] W.H. Ray. Advanced Process Control. McGrawHill, NY, 1 edition, 1981. 407

[172] John R. Rice. Numerical Methods, Software, and Analysis. McGrawHill, 1983. 123
BIBLIOGRAPHY 523

[173] C. Magnus Rimvall and Christopher P. Jobling. Computer Aided Control System Design. In W.S.
Levine, editor, The Control Handbook, chapter 23, pages 429442. CRC Press, 1995. 3

[174] William H. Roadstrum and Dan H. Wolaver. Electrical Engineering for all Engineers. John Wiley &
Sons, 2 edition, 1994. 13

[175] E.R. Robinson. Time Dependent Chemical Processes. Applied Science Publishers, 1975. 407

[176] J. A. Rossiter. Model-Based Predictive Control: A Practical Approach. CRC Press, 2003. 497

[177] Hadi Saadat. Computational Aids in Control Systems using Matlab. McGrawHill, 1993. 53

[178] S.P. Sanoff and P.E. Wellstead. Comments on: implementation of self-tuning regulators with vari-
able forgetting factors. Automatica, 19(3):345 346, 1983. 301

[179] Robert Schoenfeld. The Chemists English. VCH Verlagsgesellschaft mbH, D-6940 Weinheim, Ger-
many, 2 edition, 1986. 222

[180] J. Schoukens and R. Pintelon. Identification of Linear Systems. Pergamon, 1991. 112

[181] Tobias Schweickhardt and F. Allgower. Linear control of nonlinear systems based on nonlinearity
measures. Journal of Process Control, 17:273284, 2007. 133

[182] Dale E. Seborg, Thomas F. Edgar, and Duncan A. Mellichamp. Process Dynamics and Control. Wiley,
2 edition, 2005. 155, 158

[183] D.E. Seborg, T.F. Edgar, and D.A. Mellichamp. Process Dynamics and Control. Wiley, 1989. 15, 30,
106, 142, 154, 156, 255

[184] D.E. Seborg, T.F Edgar, and S.L. Shah. Adaptive Control Strategies for Process Control: A survey.
American Inst. Chemical Engineers J, 32(6):881913, 1986. 322

[185] Warren D. Seider, J.D. Seader, and Daniel R. Lewin. Process Design Principles: Synthesis, Analysis
and Evaluation. John Wiley & Sons, 1999. 107

[186] Bahram Shahian and Michael Hassul. Control System Design Using Matrixx . PrenticeHall, 1992.
70

[187] Kermit Sigmon. Matlab primer, 3rd edition. Dept. of Mathematics, University of Florida, 1993. 4,
5

[188] Jonas Sjoberg, Qinghua Zhang, Lennart Ljung, Albert Benveniste, Bernard Delyon, Pierre-Yves
Glorennec, Hakan Hjalmarsson, and Anatoli Juditsky. Nonlinear black-box modeling in system
identification: a unified overview. Automatica, 31(12):16911724, 1995. Trends in System Identifi-
cation. 239, 240, 267

[189] S. Skogestad. Dynamics and Control of Distillation Columns A Critical Survey. In DYCORD+,
pages 1135. Int. Federation of Automatic Control, 1992. 97

[190] Sigurd Skogestad. Simple analytic rules for model reduction and PID controller tuning. Journal of
Process Control, 13(4):291309, 2003. 138, 157

[191] Jean-Jacques E. Slotine and Weiping Li. Applied Nonlinear Control. PrenticeHall, 1991. 77, 78, 387

[192] T. Soderstrom and P. Stocia. System Identification. PrenticeHall, 1989. 238, 315

[193] Harold W. Sorenson, editor. Kalman filtering: Theory and Application. Selected reprint series. IEEE
Press, 1985. 444

[194] Henry Stark and John W. Woods. Probability, Random Processes, and Estimation Theory for Engineers.
PrenticeHall, 2 edition, 1994. 446
524 BIBLIOGRAPHY

[195] George Stephanopoulos. Chemical Process Control: An introduction to Theory and Practice. Prentice
Hall, 1984. 106, 149
[196] Dominick Stephens. The Reserve Banks Forecasting and Policy System. Technical report, Eco-
nomics Department, The Reserve Bank of New Zealand, Wellington, NZ, 2004. 477
[197] O. Taiwo. Comparison of four methods of on-line identification and controller tuning. IEE
Proceedings-D, 140(5):323327, 1993. 162, 169
[198] Les Thede. Analog and Digital Filter Design using C. PrenticeHall, 1996. 206
[199] J. Villadsen and M.L. Michelsen. Solution of differential equation models by polynomial approximation.
PrenticeHall, 1978. 88

[200] Stanley M. Walas. Modeling with Differential Equations in Chemical Engineering. Butterworth
Heinemann, 1991. 88, 127, 411
[201] Liuping Wang. Model Predictive Control System Design and Implementation using Matlab. Springer,
2009. 497
[202] Liuping Wang and William R. Cluett. From Plant Data to Process Control. Taylor and Francis, 11
New Fetter Lane, London, EC4P 4EE, 2000. 260, 261

[203] Ya-Gang Wang, Zhi-Gang Shi, and Wen-Jian Cai. PID autotuner and its application in HVAC
systems. In Proceedings of the American Control Conference, pages 21922196, Arlington, VA, 2527
June 2001. 181
[204] P.E. Wellstead and S. P. Sanoff. Extended self-tuning algorithm. International Journal of Control,
34(3):433455, 1981. 301
[205] P.E. Wellstead and M.B. Zarrop. Self-tuning Sytems: Control and Signal Processing. John Wiley &
Sons, 1991. 247, 301, 304, 305, 314, 315, 317, 319, 326, 468
[206] David I. Wilson. Introduction to Numerical Analysis with Matlab or whats a NaN and why do I care?
Auckland University of Technology, Auckland, New Zealand, July 2007. 465pp. 4, 123, 199, 206,
284, 399, 498, 501
[207] R.K. Wood and M.W. Berry. Terminal composition control of a binary distillation column. Chemical
Engineering Science, 28(9):17071717, 1973. 97
[208] Minta Yuwana and Dale E. Seborg. A New Method of On-line Controller Tuning. American Inst.
Chemical Engineers J, 28(3):434440, May 1982. 162
[209] J.G. Ziegler and N.B. Nichols. Optimum Settings for Automatic Controllers. Trans. ASME, 64:759
768, 1942. 154
I NDEX

, see Kronecker product Jordan, 51


observable, 50
A/D, 13 CARE, 427
Ackermann, 365, 369 Cauchy form, 49
adaptive control Cayley-Hamilton theorem, 365
applications, 321 ceiling function, 206
classification, 323 characteristic polynomial, 52
general scheme, 322 Chebyshev
suitable, 322 design, 207
adjugate, 52, 60 Chebyshev filter, see filter
affine, 387 Cholesky, 302, 448
AIC, see Akaike Information Criteria Cholesky decomposition, 79, 83
Akaike Information Criteria, 280 Cholesky matrix square root, 468
algebraic Riccati equation, 428 closed loop, 47
are, 428 co-states, 409
solving, 428, 431 coloured noise, 304
alias, 17 Computer design aids, 4
detecting, 17 condition number, 56
ARMAX model, 264 confidence ellipses, 121
arms race, 320 confidence intervals, 118
ARX model, 263 controllability, 356
ATV tuning method, 170 computing, 359
auto-tuners, 325
correlated noise, 447
autonomous systems, 47
correlation, 447
balance arm, 5, 143 covariance, 447, 449
Bernoullis equation, 91 wind-up, 300
bilinear transform, 39 covariance matrix, 119, 288, 295, 447
binomial coefficients, 33 reset, 298
black-box, 178 cover-up rule, 26
blackbox, 5
Bode D/A, 13
diagram, 41, 75 DAEs, 127
Bristol, 105 dare, 433, 458
Bromwich integral, 31 data acquisition, 4
bursting, 340 dead time compensation, 188
bvp4c, 411, 417 deadbeat control, 369
deadtime, 282
c2d.m, 64 difference equations, 20
canon, 57, 357 differentation
canonical form, 51, 57 vector, 407
controllable, 50 differentiation
diagonal, 50 noise, 197
525
526 INDEX

digital filter, 213 Euler, 20


aliasing, 17 finv, 449
diophantine first-order plus deadtime, 241
convmtx, 332 fixed-position, 475
Diophantine equation, 330 flapper, 7
Dirac delta function, 21 FOPDT, see first-order plus deadtime
direct method, see Lyapunov forgetting factor, 297, 298, 328
discrete algebraic Riccati equation, 458 influence, 299
distillation varying, 301
interaction, 104 Fourier transform, 221, 251
model, 100 definitions, 223
simulation, 103 fast, 227
dlqe, 459 slow, 224
DMC, see dynamic matrix control frequency pre-warping, 214
double integrator, 90 fsolve, 173
DSP, 210 function approximation, 222
dynamic matrix control, 484 fvtool, 220

eigenvalues gain scheduling, 323


generate, 56 Gauss, 109
ellipsoid, 451 Gaver-Stehfest algorithm, 33
estimation, 373 Gibbs phenomenon, 224
etfe, 253 Gilberts criteria, 357
Euler gradient, 258, 407
relations, 223
exact feedback linearisation, 387 Hamiltonian, 409
cautions, 389 Hammerstein models, 267
design procedure, 389 Hankel matrix, 272, 485
exhaustive search, 402 Hardamard product, , 106
extended Kalman filter, 470 helicopter, 7

F distribution, 449 identification, 237


FFT, see Fourier transform, 247 black and white, 239
fft, 230 closed loop, 314
filter, 195, 210, 258 continuous models, 255
bandstop, 220 differentiation, 257
bi-quad, 218 frequency response, 252
Butterworth, 203 frequency testing, 247
characteristics, 205 graphical, 241, 255
cascade, 200 model, 238
Chebyshev, 206 noise models, 304
DFI, 216 nonlinear, 267
DFII, 217, 220 offline, 241
digital, 210, 213 online, 285
elliptic, 203, 209, 220 parameters, 238
hardware, 216 PRBS, 245
ideal, 200 real-time, 240
low pass, 211 state-space, 277
minimum order, 206 subspace, 279
proto-type, 202 step response, 241
quantisation, 220 system, 238
SOS, 218 time, 241
visualisation, 220 two-dimensional, 116
final value theorem, 23 initial value theorem, 23
finite differences, 19 integral states, 371
INDEX 527

integral-only control, 139 loss function, see performance index


integration by parts, 409 lqi, 442
inter-sample ripple, 370 LQR, see linear quadratic regulator
interaction, 104 blackbox, 441
Internal Model Control, 156 servo, 441
inverse response, 184 tracking, 441
invfreqs, 248 LTI object, 37
Luenberger observer, 374
Jacobian, 407 Lyapunov, 77, 79, 475
Jordan form, 64, 65 continuous stability, 78
Jury test, 74 discrete, 453
discrete stability, 78
Kaczmarz, 308 function, 77
Kalman filter, 444, 454, 458 matrix equation, 80
algorithm, 457 solving, 83
current estimation type, 459
extended, 470 Maple, 4
prediction type, 459 mapping, 20
steady-state, 458 margin, 164, 174
Kaos, 270 Matlab, 3
Kronecker delta function, 24 Matlab alternatives
Kronecker product, 80, 83, 431, 453 Octave, 4
Kronecker tensor product, 80 SciLab, 4
matrix exponential, 61
Lagrangian, 408 matrix inversion lemma, 287
Laguerre functions, 260 maximum likelihood, 284
Laplace transform Maxwell, 1
inverting measurement noise, 119
Bromwich, 31 Method of Areas, 241
Gaver-Stehfest, 33 micro-controllers, 13
numerically, 31 minimum variance control, 327
least squares, 110 dead-time, 346
least-squares, 108 MIT rule, 311
improvements, 112 model
nonlinear, 114 CSTR, 93
partial, 113 distillation, 96
recursive, 286 high purity, 103
recursive estimation, 288 Wood-Berry, 97
solution, 111 double integrator, 90
level control, 91 evaporator, 95
Liapunov, see Lyapunov order, 282
Lie algebra, 387 pendulum, 89
limit oscillation, 171 stirred heated tank, 92
linear program, 498 validation, 280
linear quadratic regulator, 422 model predictive control, 477
continuous, 422 model reference control
discrete, 432 GMC, 381
steady-state, 427 modelling, 87, 88
time varying, 424 cautions, 93, 260
linearise, 128 evaporators, 95
finite difference, 131 extrapolation, 93
linmod, 131 models
tank model, 131 discrete, 261
Lissajou, 250 ARMAX, 263
load disturbance, 161 ARX, 262
528 INDEX

BJ, 267 Opti, 114


family, 266 parametric, 395
FIR, 267 orthogonal, 56
general, 266 output optimal control, 473
OE, 265 output variance, 11
linear, 239 output-error model, 265
nonlinear, 267 overfitting, 280
monotonicity index, 184
MPC, see model predictive control packed matrix notation, 49
algorithm, 479 packed notation, 67
horizons, 483 Pade approximation, 39
Toolbox, 480 parameter optimisation, 310
toolbox, 497 parametric adaption, 312
multivariable differentiation, 110 Parseval, 223
partial fractions, 25, 27
Nader, Ralph, 11 Matlab, 27
Nelder-Mead, 115 partial least-squares, 113
Newell-Lee evaporator, see modelling PDEs, 123
Newton-Rhapson, 160 pendulum, 89
Nilpotent matrix, 369 simulation, 124
noise penicillin, 392
coloured, 304 performance index, 109
industrial, 196 IAE, 396
models, 304 periodogram, 227
non-minimum phase, 185, 341 persistent excitation, 245, 314
nonlinear measure, 132 PI-D, 142
nonlinear models, 131, 267 PID, 135
nonlinearities cascade, 138
types, 129 classical, 136
Nyquist discrete, 145
diagram, 41, 42 drawbacks, 184
frequency, 16, 17, 206, 211, 213 parallel, 136, 138
dimensionless, 17 position form, 149
Nyquist diagram, 75 programming, 145
standard, 137
observability, 356 textbook, 136
OCP, 408, 410 tuning, 154
ODEs, 123 pidtool, 157
numerical solutions, 123 pidtune, 157
two-point boundary value problems, 127 closed loop, 157
OE model, see output-error model Ziegler-Nichols, 158
offset, 371 velocity form, 149
Opti toolbox, 4, 114, 116 pole excess, see time delay
optimal control, 395 pole-placement, 329, 369
generic problem, 406 acker, 366
Hamiltonian, 409 partial cancellation, 342
linear programming, 498 place, 367
PID tuning, 396 practical issues, 339
Pontryagin, 407 Simulink, 337
TPBVP, 411 unstable zeros, 340
optimal control problem, 408 poles, 52
optimal prediction, 346 polyfit, 112
optimisation, 115, 311, 395 polynomial
cost function, 396 addition, 325
nonlinear, 114 multiplication, 325
INDEX 529

polynomial controller design, 325 RLS, 286


poorly damped, 341 robust fitting, 284
positive definite, 302, 422 Routh table, 73
enforce, 303
lost, 467, 468 safety, 11
power spectral density, 227 sample and hold
prediction error, 304 zoh, 37
process control sample time selection, 14
economics, 10 sampling, 14
psd, see power spectral density sampling theorem, 15
pseudo-inverse, 113 alias, 16
SattControl, ECA400, 170
quadratic form, 422 self tuning controllers
derivative, 408 relay feedback, 169
quadratic program, 483 self tuning regulators, 326
quantisation error, 14 problems, 351
sensitivity function, 191
R statistical package, 4, 120 Sherman-Morrison formula, 287
random inputs, 244 shift register, 295
random numbers, 446 Simulink
reachable, 359 real-time, 8
receding horizon control, 477 single precision, 220
recursive singular value decomposition, 112
mean, 286 SISO, single input/single output, 44
variance, 287 Slyvesters criteria, 81, 83
recursive estimation Smith predictor, 188
extended, 304 Simulink, 190
SI toolbox, 308 smooth
simplified, 308 input, 258
Kaczmarz, 309 smoothing
starting, 290 fourier, 229
recursive extended least-squares, 305 SOS, 218
recursive least squares, see least squares, 443 SPD, 302
regression gun, 270 spectral analysis, 225
regression vector, 269 experimental, 247
relative gain array, 104 spectral plot, 227
dynamic, 107 spline, 198
relative order, 389 stability, 71
relay, 170, 178 BIBO, 71, 72
autotuning, 173 closed loop, 75
example, 173 continuous time, 72
relay feedback, 169 definition, 71
noise, 176 discrete time, 76
RELS, see recursive estimation Jury test, 74
remembering factor, see forgetting factor Lyapunov, 77
residue, 25 nonlinear, 77
RGA, see relative gain array Routh array, 73
Riccati equation unit circle, 76
algebraic, 428 stacking, 83
computing solution, 433 state, 47
differential, 423 state estimation, 361, 373, 443, 454
time varying, 429 observer, 374
ridge regression, 109 state reconstruction, 361
ringing, 341 state space, 47
ripple, 207 conversions, 49, 51
530 INDEX

pole-placement, 364 transmission zeros, 52, 60


statistic, t, 120 Tustins approximation, 39
Statistics toolbox, 4 Tustins method, see bilinear transform
steady-state models, 88
steepest descent, 311 unstable zeros, 341, 343, 349
step tests, 241
Stixbox, 4 vectorisation, 83
stochastic processers, 445
stochastic processes, 452 waypoints, 32
subspace identification, 279 well damped, 343
Sylvester equation, 80, 84 white noise, 291
Sylvester matrix, 331 Wiener models, 267
symbolic algebra, 4, 23, 24, 63 Wood-Berry column model, 97
Maple, 4 Woodbury identity, 287
MuPad, 4
symbols, 507 X-15, 321
symmetric, 302, 430
symmetric positive definite Yuwana-Seborg tuning, 162
nearest, 304
System Identification toolbox, 305 z-transform, 21
system type, 371 geometric series, 23
inversion, 23
tapped delay, 216, 292 computational, 29
target set, 419 long division, 27
Tikhonov regulisation, 109 partial fractions, 25
time delay symbolic, 24
discrete, 70 zero-pole-gain, 57
time series analysis zeros, 52, 60
project, 310 zeroth-order hold, 41
Toeplitz matrices, 272 Ziegler-Nichols
trace of a matrix, 301 closed loop, 158
transfer function, 57 example, 160
time constant, 57 openloop, 154
transition matrix, 61 zoh, see zeroth-order hold

Você também pode gostar