Mathematical theory of adaptive control

Vladimir G. Sragovich, Jan Spalinski9812563717, 9789812563712, 9789812701039

The theory of adaptive control is concerned with construction of strategies so that the controlled system behaves in a desirable way, without assuming the complete knowledge of the system. The models considered in this comprehensive book are of Markovian type. Both partial observation and partial information cases are analyzed. While the book focuses on discrete time models, continuous time ones are considered in the final chapter. The book provides a novel perspective by summarizing results on adaptive control obtained in the Soviet Union, which are not well known in the West. Comments on the interplay between the Russian and Western methods are also included.

---

Table of contents :
CONTENTS……Page 14
Preface……Page 8
Editor’s Note……Page 12
1.1. Random Processes and Systems of Probability Distributions……Page 18
1.2. Controlled Random Processes……Page 22
1.3. Definition of Adaptive Control……Page 36
1.4. Learning Systems……Page 42
1.5. Bayesian Approach on a Finite Interval……Page 46
2.1. Formulation of the Problem……Page 50
2.2. Optimal Properties of Finite Automata……Page 53
2.3. Automata with Increasing Memory……Page 66
2.4. -Automata and Their Modifications……Page 72
2.5. Automata with Formed Structure……Page 85
2.6. Asymptotic Optimality of Automata with Variable Structure……Page 89
3.1. Formulation of the Problem……Page 94
3.2. Convergence Conditions of Stochastic Approximation Procedures……Page 98
3.3. Survey of Asymptotic Properties of Stochastic Approximation Methods for HPIV……Page 103
3.4. Calculation of the Conditional Extremum……Page 106
4.1. Games with Consistent Interests……Page 114
4.2. Some Remarks on Minimax Control of Vector HPIV……Page 119
4.3. Recurrent Procedure of Searching Equilibrium Strategies in a Multi-person Game……Page 121
4.4. Games of Automata……Page 125
5.1. Preliminary Remarks……Page 140
5.2. Structure of Finite Homogeneous Controlled Markov Chains……Page 142
5.3. Unconditional Optimal Adaptive Control for Finite Markov Chains……Page 150
5.4. The First Control Algorithm for a Class of Markov Chains (identificational)……Page 153
5.5. The Second Control Algorithm for a Class of Markov Chains (automata)……Page 156
5.6. The Third Control Algorithm for a Class of Markov Chains (stochastic approximation)……Page 159
5.7. Adaptive Optimization with Constraints on Markov Chains……Page 171
5.8. Minimax Adaptive Problems on Finite Markov Chains……Page 178
5.9. Controlled Graphs with Rewards……Page 184
6.1. Preliminary Remarks……Page 190
6.2. Control of Conditional Markov Chains……Page 191
6.3. Optimal Adaptive Control of Partially Observable Markov Chains and Graphs……Page 198
6.4. Control of Regenerative Processes……Page 201
6.5. Structure of -optimal Strategies for Controlled Regenerative Processes……Page 203
6.6. Adaptive Strategies for Controlled Regenerative Processes……Page 213
7.1. Preliminary Results……Page 220
7.2. Optimal Automaton Control for Markov Processes with A Compact State Space and A Finite Control Set……Page 228
7.3. Searching Optimal Strategies for Ergodic Markov Processes with Compact Spaces of States and Controls……Page 232
7.4. Control of Finite Semi-Markov Processes……Page 238
7.5. Control of Countably Valued Semi-Markov Processes……Page 242
7.6. Optimal Control of Special Classes of Markov Processes with Discrete Time……Page 254
8.1. Formulation of the Problem……Page 268
8.2. Some Properties of Stationary Processes……Page 269
8.3. Auxiliary Results for CSP……Page 271
8.4. Adaptive Strategies for CSP……Page 279
9.1. Formulation of the Problem……Page 284
9.2. Finite-converging Procedures of Solving A Countable System of Inequalities……Page 286
9.3. Sufficient Conditions for Existence of FCP……Page 290
9.4. Stabilization of Solutions of Linear Difference Equations: Part I……Page 293
9.5. Stabilization of Solutions of Linear Difference Equations: Part II……Page 298
10.1. Auxiliary Results……Page 304
10.2. Control of Homogeneous Equations xt+1 = Axt + But……Page 314
10.3. Optimal Tracking Problem for ARMAX……Page 319
10.4. Optimal Tracking and Consistency of Estimates for ARMAX……Page 327
10.5. Adaptive Modal Control……Page 337
10.6. On Strong Consistency of LSE and SGE of Parameters……Page 345
10.7. Linear-Quadratic Problem (LQP)……Page 356
10.8. LQP for ARMAX-type Equations……Page 369
11.1. Preliminary Results……Page 376
11.2. Control of Homogeneous Equations……Page 382
11.3. Control with A Model Reference……Page 387
11.4. Steepest Descent Method……Page 398
11.5. Stabilization of Solutions of Minimum Phase Equations……Page 404
11.6. Stabilization of Minimum Phase Equations with Nonlinearities……Page 412
11.7. Stabilization of Linear Minimum Phase Equations in Hilbert Space……Page 414
11.8. Control of Stabilizable Equations……Page 421
11.9. Two Special Problems of Adaptive Control……Page 430
12.1. Preliminary Results……Page 438
12.2. Stabilization of Solutions of Minimum Phase Ito Equations……Page 447
12.3. Identifcation Methods for Ito Equations……Page 456
12.4. LQP for Stochastic Ito Equations……Page 458
Comments and Supplements……Page 462
General References……Page 476
Special References……Page 478
Additional References……Page 486
Index……Page 488