Optimization Methods: Dynamic Programming Applications Reservoir Operation

Module 6 Lecture Notes 6 Reservoir Operation Introduction In the previous lectures, we discussed about the application of dynamic programming in water allocation and capacity expansion of infrastructure. Another major application is in the field of reservoir operation, which will be discussed in this lecture. Reservoir operation Steady state optimal policy Consider a single reservoir receiving inflow it and making releases rt for each time period t. The maximum capacity of the reservoir is K. The optimization problem is to find the sequence of releases to be made from the reservoir that maximizes the total net benefits. These benefits may be from hydropower generation, irrigation, recreation etc. Let St and S t +1 be the initial and final storages for time period t. Expressing net benefits as a function of St , S t +1 and rt , the net benefit for period t is NBt (S t , S t +1 , rt ) . If there are T periods in a year, then the objective function is to maximize the total net benefits from all periods.

Maximize

NB (S , S
t =1 t t

t +1

, rt )

This is subject to continuity and also capacity constraints. Neglecting all minor losses like evaporation, seepage etc and assuming that there is no overflow, the continuity relation can be written as, S t +1 = St + it rt for t = 1,2,...T

The capacity constraint can be expressed as, St K for t = 1,2,...T

D Nagesh Kumar, IISc, Bangalore

M6L6

Optimization Methods: Dynamic Programming Applications Reservoir Operation

The above formulated problem can be solved as a sequential process either using forward or backward approach. Here the stages are the time periods and the states are the storage volumes. Assume that there are T periods in a year. In order to find the steady state policy, select a period in a particular year in the near future (to get steady solution). Usually in almost all problems, the last period T is taken as the terminal period. At this stage, the optimal release rt will be independent of the inflow it and also the net benefit NBt . Now, consider the terminal period as T of a particular year after which reservoir is no longer useful. Solving this problem in a backward recursion method, let t represents the period in a year from T to 1 and n represents the periods remaining from t till end. Thus, t will take values starting from T, decreasing to 1 (which will complete one year) and then again taking a value of T and repeating the values. The value of n starts from 1 (while considering the Tth period of last year) and while moving backwards its value keeps on increasing i.e. at the beginning of the last year, the value of n = T and at the beginning of second last year its value will be equal to 2T and so on.

Starting from T of last year, which is at the far right, there is only one period remaining. Thus, in this case t=T and n=1. Let fT1 (ST ) be the maximum net benefit in the last period of the year considered. fT1 (ST ) can be expressed as f T1 (S T ) =

rT 0 rT ST + iT rT ST + iT K

max

[NBT {ST , (S T + iT rT ), rT }]

which should be solved for all ST values from 0 to K. Considering the last two stages together for which t=T-1 and n=2, the objective function can be written as

D Nagesh Kumar, IISc, Bangalore

M6L6

Optimization Methods: Dynamic Programming Applications Reservoir Operation f T21 (S T 1 ) = max

rT 1 0 rT 1 ST 1 + iT 1 rT 1 ST 1 + iT 1 K

[NB

T 1

{ST 1 , (S T 1 + iT 1 rT 1 ), rT 1 } + f T1 (S T 1 + iT 1 rT 1 )]

This also is solved for all ST 1 values from 0 to K. In general, for a period t of a particular year with n periods remaining, the function can be written as
n 1 f t n (S t ) = max NBt {S t , (S t + it rt ), rt } + f t + 1 (S t + it rt ) rt 0 rt S t + it rt S t + it K

where the index t decreases from T to 1 and then takes the value T again for the previous year and the cycle repeats while the index n starts from 1 and increases at each successive stage. This cycle can be repeated till the optimum values of rt for an initial storage St will be the same as the corresponding rt and St of previous year. Such a solution is called stationary solution. The maximum net benefit can be obtained as the difference of f t n +T (S t ) and f t n (S t ) for any St and t. Numerical example (Loucks et al., 1981) Consider a reservoir for which the desirable constant storage is 20 units and the constant release is 25 units. The capacity of the reservoir is 30 units and the inflows for three seasons are given as 10, 50 and 20 units. The problem is to find the optimum St and rt that minimizes the total squared deviation from the release and storage targets given. Hence, the objective function is (20 S t ) + (25 rt ) . Let St take the discrete values of 0, 10, 20, 30
2 2

and rt take the values of 10, 20, 30, 40.

Solution:

D Nagesh Kumar, IISc, Bangalore

M6L6

Optimization Methods: Dynamic Programming Applications Reservoir Operation

Consider a year after which the reservoir is no longer useful. The problem can be expressed as a sequential process as shown in the figure below

Here no. of seasons (periods), T = 3. Considering the last period for which t = 3 and n = 1, the optimization function is Minimize f 31 (S 3 ) = (20 S 3 ) + (25 r3 )
2

Inflow for 3rd season, I3 = 20 units and capacity of the reservoir, K = 30 units. The release constraints can be expressed as r3 S 3 + I 3 S 3 + 20 r3 S 3 + I 3 K S 3 + 20 30

and

The computation for the first subproblem (n = 1) is shown in the table below.

D Nagesh Kumar, IISc, Bangalore

M6L6

Optimization Methods: Dynamic Programming Applications Reservoir Operation Table 1 State variable, S3 0 Release, r3 10 20 10 10 20 30 10 20 20 30 40 10 30 20 30 40

(20 S3 )2 + (25 r3 )
625 425 325 125 125 225 25 25 225 325 125 125 325

f 31 (S 3 )

Optimal release, r3* 20

425

125

20, 30

25

20, 30

125

20, 30

Now considering the last two periods (n =2), the optimization function is Minimize f 22 (S 2 ) = (20 S 2 ) + (25 r2 ) + f 31 (S 2 + I 2 r2 )
2 2

Inflow for 2nd season, I2 = 50 units. The release constraints can be expressed as

r2 S 2 + 50

and

r2 S 2 + 50 30
The computation for the second subproblem (n = 2) is shown in the table below. For S2=30, r2 S 2 + 50 30 i.e. r2 50 i.e. r2 50 . Since r2 can take values only of 10, 20, 30 and 40 only, the release cannot be made for S2=30. Table 2

D Nagesh Kumar, IISc, Bangalore

M6L6

Optimization Methods: Dynamic Programming Applications Reservoir Operation State variable, S2 (20 S 2 )2 2 ( ) + 25 r 2 425 425 625 125 325 225 na S2+ I2 r2 30 20 10 30 20 30 na

6 Optimal

Release, r2 20

(S 2 + I 2 r2 )
125 25 125 125 25 125 na

f 31

(5)+(3)

f 22 (S 2 )

release, r2*

550 450 750 250 350 350 na 350 na 40 na 250 30 450 30

30 40

10 20 30

30 40 40 na

The same procedure is repeated for all stages till n = 7. The summarized solution for this problem is given in the table below. Table 3 Initial Storage, St 0 10 20 30 n=1
f 31 (S 3 )

n=2

n =3
r2

r3

f 22 (S 2 ) 450 250 350 -Table 4

f13 (S1 ) 1075 575 275 375

r1

425 125 25 125

20 20, 30 20, 30 20, 30

30 30 40 na

10 10, 20 20 30

Initial Storage, St 0 10 20 30

n=4
f 34 (S 3 )

n =5

n =6
r2

r3

f 25 (S 2 ) 725 525 625 --

f16 (S1 ) 1350 850 550 650

r1

1200 600 300 400

10 10 20 30

30 30 40 na

10 10, 20 20 30

D Nagesh Kumar, IISc, Bangalore

M6L6

Optimization Methods: Dynamic Programming Applications Reservoir Operation Table 5 Initial Storage, St 0 10 20 30

n=7
f 37 (S 3 )

r3

1475 875 575 675

10 10 20 30

At this stage, the value of r3 at n = 7 and n = 4 are exactly the same. Also the difference
f 37 (S 3 ) - f 34 (S 3 ) = 275 is same for all S t . This value is the minimum total squared deviations

from the target release and storage. Thus, the stationary policy obtained is given below. Table 6 Optimal Releases St
r1

r2

r3

0 10 20 30

10 10, 20 20 30

30 30 40 --

10 10 20 30

A main assumption made in dynamic programming is that the decisions made at one stage is dependent only on the state variable and is independent of the decisions taken in other stages. In cases where decisions made at one stage are dependent on the earlier decisions, then dynamic programming will not be an appropriate optimization technique.

D Nagesh Kumar, IISc, Bangalore

M6L6

Optimization Methods: Dynamic Programming Applications Reservoir Operation

References / Further Reading:

1. Bellman, R., Dynamic Programming, Princeton University Press, Princeton, N.J, 1957. 2. Loucks, D.P., J.R. Stedinger, and D.A. Haith, Water Resources Systems Planning and Analysis, Prentice Hall, N.J., 1981. 3. Mays, L.W. and K. Tung, Hydrosystems Engineering and Management, McGraw-Hill Inc., New York, 1992. 4. Vedula, S., and P.P. Mujumdar, Water Resources Systems: Modelling Techniques and Analysis, Tata McGraw Hill, New Delhi, 2005.

D Nagesh Kumar, IISc, Bangalore

M6L6