Self-learning control of finite Markov chains

Poznyak, Najim, Gomez-Ramirez.0-8247-9249-X

This rigorously focused reference/text presents a number of new and potentially useful self-learning (adaptive) control algorithms and theoretical as well as practical results for both unconstrained and constrained finite Markov chains – efficiently processing new information by adjusting control strategies directly or indirectly. Offering new material and many descriptive simulations and concrete examples, Self-Learning Control of Finite Markov Chains supplies fundamental mathematical concepts of self-learning control of constrained and unconstrained finite Markov chains…states theorems related to the convergence, the speed of convergence, and the optimal selection of the design parameters of several efficient self-learning algorithms…analyzes the asymptotic properties (convergence with probability 1 as well as convergence in the mean squares) using the Lyapunov approach and martingales theory…discusses ways that adaptive algorithms form a new estimate, incorporating new information (realizations) from the old estimate using a fixed amount of computation and memory…introduces normalization procedures and regularized Lagrange and penalty functions…showcases the novel approach of a partially frozen control strategy…examines the asymptotic properties of different algorithms derived from schemes originally used in modeling animal learning patterns…and more. Featuring highly practical MATLAB programs for instruction and elaboration of key concepts, Self-Learning Control of Finite Markov Chains is a versatile reference for electrical, electronics, control, and software engineers; mathematicians; statisticians; and economists involved in stochastic games; and an invaluable text for upper-level undergraduate and graduate students in these disciplines.