Martingales and Markov chains: solved exercises and theory

Paolo Baldi, Laurent Mazliak, Pierre Priouret1584883294, 9781584883296

A thorough grounding in Markov chains and martingales is essential in dealing with many problems in applied probability, and is a gateway to the more complex situations encountered in the study of stochastic processes. Exercises are a fundamental and valuable training tool that deepen students’ understanding of theoretical principles and prepare them to tackle real problems.In addition to a quick but thorough exposition of the theory, Martingales and Markov Chains: Solved Exercises and Elements of Theory presents, more than 100 exercises related to martingales and Markov chains with a countable state space, each with a full and detailed solution. The authors begin with a review of the basic notions of conditional expectations and stochastic processes, then set the stage for each set of exercises by recalling the relevant elements of the theory. The exercises range in difficulty from the elementary, requiring use of the basic theory, to the more advanced, which challenge the reader’s initiative. Each section also contains a set of problems that open the door to specific applications.Designed for senior undergraduate- and graduate level students, this text goes well beyond merely offering hints for solving the exercises, but it is much more than just a solutions manual. Within its solutions, it provides frequent references to the relevant theory, proposes alternative ways of approaching the problem, and discusses and compares the arguments involved.

---

Table of contents :
Contents……Page 4
Preface……Page 6
Definition and First Properties……Page 7
Conditional Expectations and Conditional Laws……Page 10
Exercises……Page 11
Solutions……Page 13
General Facts……Page 20
Stopping Times……Page 22
Exercises……Page 23
Solutions……Page 24
First Definilions……Page 27
The Stopping Theorem……Page 28
Maximal Inequalities……Page 29
Square Integrablc Martingales……Page 31
Convergence Theorems……Page 32
Exercises……Page 34
Problems……Page 43
Solutions……Page 48
Tiansition Matrices, Markov Chains……Page 75
Construction and Existence……Page 78
Computations on the Canonical Chain……Page 79
Potential Operators……Page 80
Passage Problems……Page 81
Recurrence, Transience……Page 83
Recurrent Irreducible Chains……Page 86
Periodicity……Page 91
Exercises……Page 93
Problems……Page 109
Solutions……Page 123
References……Page 186
Index……Page 188