Iterative Dynamic Programming

Rein Luus1-58488-148-8, 9781584881483

Dynamic programming is a powerful method for solving optimization problems, but has a number of drawbacks that limit its use to solving problems of very low dimension. To overcome these limitations, author Rein Luus suggested using it in an iterative fashion. Although this method required vast computer resources, modifications to his original scheme have made the computational procedure feasible. With iteration, dynamic programming becomes an effective optimization procedure for very high-dimensional optimal control problems and has demonstrated applicability to singular control problems. Recently, iterative dynamic programming (IDP) has been refined to handle inequality state constraints and noncontinuous functions.Iterative Dynamic Programming offers a comprehensive presentation of this powerful tool. It brings together the results of work carried out by the author and others – previously available only in scattered journal articles – along with the insight that led to its development. The author provides the necessary background, examines the effects of the parameters involved, and clearly illustrates IDP’s advantages.

---

Table of contents :
ITERATIVE DYNAMIC PROGRAMMING……Page 3
Contents……Page 6
About the author……Page 11
Preface……Page 12
Notation……Page 14
1.1 Introduction……Page 16
1.2.1 Operator……Page 17
1.2.2 Vectors and matrices……Page 18
1.2.4 Taylor series expansion……Page 20
1.2.6 Sign definite……Page 21
1.4 Continuous-time system model……Page 22
1.5 Discrete-time system model……Page 23
1.6 The performance index……Page 25
1.8.1 Linear gas absorber……Page 26
1.8.2 Nonlinear continuous stirred tank reactor……Page 28
1.8.3 Photochemical reaction in CSTR……Page 30
1.8.4 Production of secreted protein in a fed-batch reactor……Page 31
1.9 Solving algebraic equations……Page 32
1.9.1 Separation of the equations into two groups……Page 33
Example 1……Page 34
Example 2……Page 36
Example 3……Page 38
Example 4……Page 40
Example 5……Page 42
Example 6……Page 43
1.9.3 Application to multicomponent distillation……Page 46
1.11 References……Page 47
2.2 Linear programming……Page 50
2.2.1 Example — diet problem with 5 foods……Page 53
2.2.2 Interpretation of shadow prices……Page 57
2.3 LJ optimization procedure……Page 59
2.3.1 Determination of region size……Page 61
2.3.3 Model reduction example……Page 63
Example 1……Page 69
Example 2……Page 71
2.3.5 Handling equality constraints……Page 73
Example:Geometric problem……Page 75
2.4 References……Page 79
3.2 Examples……Page 82
3.2.1 A simple optimal path problem……Page 83
3.2.2 Job allocation problem……Page 84
3.2.3 The stone problem……Page 87
3.2.4 Simple optimal control problem……Page 88
3.2.5 Linear optimal control problem……Page 90
3.2.6 Cross-current extraction system……Page 91
3.4 References……Page 95
4.1 Introduction……Page 96
4.5 First iteration……Page 97
4.5.3 Continuation in backward direction……Page 98
4.6 Iterations with systematic reduction in region size……Page 99
4.8 Use of accessible states as grid points……Page 100
4.9 Algorithm for IDP……Page 101
4.10 Early applications of IDP……Page 104
4.11 References……Page 105
5.1 Introduction……Page 106
5.2 Comparison of uniform distribution to random choice……Page 107
5.2.1 Uniform distribution……Page 108
5.2.2 Random choice……Page 109
5.3 References……Page 113
6.1 Introduction……Page 114
6.2.1 Bifunctional catalyst blend optimization problem……Page 115
6.2.2 Photochemical CSTR……Page 119
6.3 Multi-pass approach……Page 121
6.3.1 Nonlinear two-stage CSTR system……Page 122
6.4 Further example……Page 124
6.4.1 Effect of region restoration factor……Page 126
6.4.3 Effect of the number of time stages……Page 127
6.5 References……Page 132
7.2 Problem formulation……Page 134
7.3 Algorithm for IDP for piecewise linear control……Page 135
7.4.1 Nonlinear CSTR……Page 137
7.4.2 Nondifferentiable system……Page 139
7.4.3 Linear system with quadratic performance index……Page 141
7.4.4 Gas absorber with a large number of plates……Page 151
7.5 References……Page 153
8.1 Introduction……Page 154
8.3.1 Example 1……Page 155
8.3.2 Example 2……Page 157
8.3.3 Example 3 — Nonlinear two-stage CSTR system……Page 158
8.4 References……Page 163
9.1 Introduction……Page 164
9.2 Variable stage-lengths when final time is free……Page 170
9.2.1 IDP algorithm……Page 171
9.3 Problems where final time is not specified……Page 172
9.3.1 Oil shale pyrolysis problem……Page 173
9.3.2 Modified Denbigh reaction scheme……Page 176
9.4 Systems with specified final time……Page 180
9.4.1 Fed-batch reactor……Page 183
9.5 References……Page 190
10.1 Introduction……Page 192
10.2.1 Example 1……Page 193
10.2.2 Example 2……Page 196
10.2.3 Example 3……Page 202
10.2.4 Example 4……Page 203
10.3 Yeo’s singular control problem……Page 206
10.4 Nonlinear two-stage CSTR problem……Page 208
10.5 References……Page 212
11.2.1 Problem formulation……Page 214
11.2.2 Quadratic penalty function with shifting terms……Page 215
Example 1……Page 216
Example 2……Page 220
Example 3: Fed-batch reactor……Page 224
Example 4……Page 226
Example 5: Two-stage CSTR system……Page 229
11.2.3 Absolute value penalty function……Page 233
Example 6: Crane problem……Page 234
11.2.4 Remarks on the choice of penalty functions……Page 238
11.3.1 Problem formulation……Page 239
11.3.2 State constraint variables……Page 240
Example 7: Chemical reactor control……Page 241
Example 8: Fed-batch fermentor……Page 242
11.4 References……Page 249
12.2 Time optimal control problem……Page 251
12.3 Direct approach to time optimal control……Page 252
12.4.1 Example 1: Bridge crane system……Page 253
12.4.2 Example 2: Two-link robotic arm……Page 255
12.4.3 Example 3: Drug displacement problem……Page 257
12.4.4 Example 4: Two-stage CSTR system……Page 260
12.4.5 Example 5……Page 263
12.5 High dimensional systems……Page 265
12.6 References……Page 267
13.2 Problem formulation……Page 269
13.3.1 Example 1-Luus-Tassone problem……Page 270
13.4 References……Page 278
14.1 Introduction……Page 279
14.2 Example: Lee-Ramirez bioreactor……Page 281
14.2.1 Solution by IDP……Page 282
14.3 References……Page 286
15.1 Introduction……Page 287
15.2 Optimal control of oil shale pyrolysis……Page 288
15.3 Future directions……Page 292
15.4 References……Page 296
A.1 Program listing……Page 297
A.2 Output of the program……Page 299
B.1 Main program for the diet problem……Page 300
B.2 Input subroutine……Page 301
B.3 Subroutine for maximization……Page 302
B.4 Output subroutine……Page 303
C.1 Five food diet problem……Page 304
C.2 Model reduction problem……Page 306
C.3 Geometric problem……Page 308
D.1 CSTR with piecewise constant control……Page 310
D.2 IDP program for piecewise linear control……Page 316
D.3 IDP program for variable stage lengths……Page 321
E.1 DVERK……Page 327