Lecture 27

Linear Quadratic Regulator (LQR) Design I

Dr. Radhakant Padhi

Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore

LQR Design: Problem Objective

To drive the state X of a linear (rather linearized) system X = A X + BU to the origin by minimizing the following quadratic performance index (cost function)

1 T 1 J = ( X f S f X f ) + ( X T Q X + U T RU ) dt 2 2 t0 where S f , Q 0 (psdf), R > 0 (pdf)

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 2

tf

LQR Design: Guideline for Selection of Weighting Matrices

S f 0 (psdf), Q 0 (psdf), R > 0 (pdf) These are usually chosen as diagonal matrices, with s fi = maximum expected/acceptable value of 1/x qi = maximum expected/acceptable value of (1/x ri = maximum expected/acceptable value of (1/ui2
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 3

2 if

2 i

) )

LQR Design: Some Facts to Remember

The pair { A, B} needs to be controllable and the pair A, Q needs to be detectable

S f 0 (psdf), Q 0 (psdf), R > 0 (pdf)

(these are usually chosen as diagonal matrices)

By default, it is assumed that t f Constrained problems (state and control inequality constraints) are not considered here. Those will be considered later.
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 4

LQR Design: Problem Statement

Performance Index (to minimize):
1 T 1 T J = ( X f S f X f ) + ( X Q X + U T RU ) dt 2 t0 2
( X f
tf

L ( X ,U )

Path Constraint:

X = A X + BU

Boundary Conditions: X ( 0 ) = X 0 : Specified

t f : Fixed, X ( t f ) : Free
5

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

LQR Design:

Necessary Conditions of Optimality

1 T Terminal penalty: ( X f ) = ( X f S f X f ) 2 1 T Hamiltonian: H = ( X Q X + U T RU ) + T ( AX + BU )
2

State Equation:

X = AX + BU

Costate Equation: = ( H / X ) = ( QX + AT ) Optimal Control Eq.: ( H / U ) = 0 U = R 1 BT Boundary Condition: f = ( / X f ) = S f X f

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 6

LQR Design: Derivation of Riccati Equation

Guess:

(t ) = P (t ) X (t )

Justification:
From functional analysis theory of normed linear space, ( t ) lies in the "dual space" of X ( t ) , which is the space consisting of all continuous linear functionals of X ( t ) . Reference: Optimization by Vector Space Methods D. G. Luenberger, John Wiley & Sons, 1969.
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

LQR Design: Derivation of Riccati Equation

Guess

(t ) = P (t ) X (t )
= PX + PX
= PX + P ( AX + BU ) = PX + P ( AX BR 1 BT )

( QX + AT PX ) = ( P + PA PBR 1 BT P ) X P + PA + AT P PBR 1 BT P + Q ) X = 0 (
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

= PX + P ( AX BR 1 BT PX )

LQR Design: Derivation of Riccati Equation

Riccati equation

P + PA + AT P PBR 1 BT P + Q = 0
Boundary condition
P (t f ) X f = S f X f
P (t f ) = S f

(X

is free )

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

LQR Design: Solution Procedure

Use the boundary condition P ( t ) = S and integrate the Riccati Equation backwards from t f to t0 Store the solution history for the Riccati matrix Compute the optimal control online
f f

U = ( R 1 BT P ) X = K X
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 10

LQR Design: Infinite Time Regulator Problem

Theorem (By Kalman)

As t f , for constant Q and R matrices, P 0 t

Algebraic Riccati Equation (ARE)

PA + AT P PBR 1 BT P + Q = 0
Note:
ARE is still a nonlinear equation for the Riccati matrix. It is not straightforward to solve. However, efficient numerical methods are now available. A positive definite solution for the Riccati matrix is needed to obtain a stabilizing controller.
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 11

Example 1:

Stabilization of Inverted Pendulum

Dr. Radhakant Padhi
Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore

A Motivating Example:

Stabilization of Inverted Pendulum

System dynamics:
L

( Linearized about vertical equilibrium point )

System dynamics (state space form): Define: x1 = , x2 = x1 0 x = 2 2 n
X A

2 = n2 u, n = g / L

mg

Performance Index (to minimize): 1 1 J = 2 + 2 u 2 dt 2 0 c 1 0 1 Q= , R= 2 0 0 c

13

1 x1 0 x + 1 u 0 2
X B

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

A Motivating Example:
ARE: PA + AT P PBR 1 BT P + Q = 0

Stabilization of Inverted Pendulum

u
L

mg

p1 p2 Let P = (a symmetric matrix) p2 p3 2 2 2 2 p2n p1 p2n p3n c 2 p2 c 2 p2 p3 1 0 0 0 + + = 2 2 p3n p2 p1 p2 c 2 p2 p3 c 2 p3 0 0 0 0 Equations:

2 2 2 p2n c 2 p2 + 1 = 0 2 p1 + p3n c 2 p2 p3 = 0 (repeated) 2 2 p2 c 2 p3 = 0

p2 =

1 2 n4 + c 2 2 n c 1 2 p2 c
14

p3 =

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

A Motivating Example:
Hence, p2 needs to be positive. Therefore 1 2 n + n4 + c 2 , c2 Moreover, p2 =
2 p1 + p3n c 2 p2 p3 = 0 2 p1 = c 2 p2 p3 p3n (not needed in this problem)

Stabilization of Inverted Pendulum

However, p3 is a diagonal term, which needs to be real and positive. p3 = 1 2 p2 c

mg

Gain Matrix: K = R 1 BT P = c 2 p2 Control: u = K X = c 2 p2 + p3 c 2 p3

)
15

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

A Motivating Example:
Analysis

Stabilization of Inverted Pendulum

Open-Loop System:

1 2 I A = = 2 n = 0 2 n = n

mg

( right half pole: unstable system )

4 Define: 2 = n + c 2

Closed-Loop System: 0 ACL = A BK = 2 2 n c p2 Closed-Loop Poles:

1 c 2 p3

p2 =

1 (n2 + 2 ) c2

I ACL = 0

1 2 2 2 1/ 2 p3 = 2 p2 = 2 (n + ) c c
16

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

A Motivating Example:
Analysis

Stabilization of Inverted Pendulum

Closed-Loop Poles:

+ 2 ( +
2 2 n

2 1/ 2

+ = 0
2

mg

1,2 =

( Note:

1/ 2 1/ 2 1 1 2 n + 2 ) j 2 n2 ) ( ( 2 2

2 = n4 + c 2 > n2

Both of the closed-loop poles are strictly in the left-half plane. Hence, the closed-loop is guaranteed to be asymptotically stable.

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

17

Example 2:

Finite-time Temperature Control

Dr. Radhakant Padhi
Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore

Example: Finite Time Temperature Control Problem

System dynamics :

= a ( n ) + bu
where a, b : Constants

: Temperature

n : Ambient temperature (Constant = 200C)

u : Heat input
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 19

Problem formulations
Case 1: Cost Function: Case 2:
Cost Function:
tf 2 1 2 J = s f ( f 30 ) + u dt 2 0 s f > 0 : Weightage

1 2 J = u dt 20

tf

( t f ) = f = 300 C
(Hard constraint)

i.e. ( t f ) 300 C (Soft Constraint)

20

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

Solution:
Solution:

( a ) , ( 0 ) = a x = ax + bu , x ( 0 ) = ( a a ) = 0
x 1 2 = u + ( ax + bu ) 2 = = a x = 0 u = b u

Necessary conditions x = ax + bu

= a u = b
21

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

Solution: Case 1 (Hard constraint) a (t t f ) a (t f t ) =e f = e f

u = be
a t f t

f
a t f t

x = ax b f e
2

Taking laplace transform: sX ( s ) x ( 0 ) = aX ( s ) b 2 f e at f 0

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

1 sa
22

Solution: Case 1 (Hard constraint)

1 X ( s ) = b f e 2 2 s a 1 1 at f 1 2 = b f e 2a s a s + a at f 1 2 e at e at ) Hence x(t ) = b f e ( 2a
2 at f

However, x (t f ) = ( f a ) = 10 C
0
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 23

Unknown

Solution: Case 1 (Hard constraint)

x(t f ) = 10 = b f e
2 at f

1 at f at f e e 2a

b 2 f 2 at f 10 = 1 e 2a 20a f = 2 2 at f b 1 e

)
10 ( e at e at ) e at f 1 ( e at e at ) = at f at f 2a e e

20 a 2 x(t ) = b b 2 1 e 2 at f

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

24

Solution: Case 1 (Hard constraint)

Note : x(t f ) = 10 e

at f

e e

at f

(e

at f

at f

= 10

(i.e. The boundary condition is "exactly met".) Controller : 20a a (t f t ) u (t ) = b e b 2 1 e 2 at f

20a e at = b e at f e at f

25

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

Solution: Case 2 (Soft constraint)

f 300 C
x f 100 C. Hence the cost function is
tf 2 1 2 J = s f ( x f 10 ) + u dt 2 0 f f = s f ( x f 10 ) x f = + 10 s f However, we have

b2 at f x ( t ) = f e ( eat e at ) 2a
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 26

Solution: Case 2 (Soft constraint)

f b2 2 at f At t = t f , x ( t f ) = f 1 e = + 10 sf 2a

1 b2 2 at f i.e. f + 1 e = 10 s f 2a 20 s f a i.e. f = 2a + s f b 2 1 e 2 at f 20 s f a a (t f t ) a (t f t ) Hence = e f = e 2a + s f b 2 1 e 2 at f

27

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

Solution: Case 2 (Soft constraint)

u ( t ) = b 20s f a a (t f t ) = be 2a + s f b 2 1 e 2 at f at 10s f abe a (t f t ) = be s f b 2 at f at f at f ae + 2 e e

28

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

Correlation between hard and soft constraint results

As s f ,
s f

lim u ( t ) S .C . = lim

10abe at at f b 2 at f at f e e ae + 2

s f

b e

1 s f 20ae at e

at f

at f

= u ( t ) H .C .

i.e. The "soft constraint" problem behaves like the "hard constraint" problem when s f .
ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore 29

ADVANCED CONTROL SYSTEM DESIGN Dr. Radhakant Padhi, AE Dept., IISc-Bangalore

30