Introduction to stochastic processes

Paul G. Hoel9780395120767, 0395120764

An excellent introduction for electrical, electronics engineers and computer scientists who would like to have a good, basic understanding of the stochastic processes! This clearly written book responds to the increasing interest in the study of systems that vary in time in a random manner. It presents an introductory account of some of the important topics in the theory of the mathematical models of such systems. The selected topics are conceptually interesting and have fruitful application in various branches of science and technology.

---

Table of contents :
Front cover……Page 1
Series……Page 2
Title page……Page 3
Date-line……Page 4
General Preface……Page 5
Preface……Page 7
Table of Contents……Page 9
1 Markov Chains……Page 11
1.1 Markov chains having two states……Page 12
1.2 Transition function and initial distribution……Page 15
1.3 Examples……Page 16
1.4 Computations with transition functions……Page 22
1.4.1 Hitting times……Page 24
1.4.2 Transition matrix……Page 26
1.5 Transient and recurrent states……Page 27
1.6 Decomposition of the state space……Page 31
1.6.1 Absorption probabilities……Page 35
1.6.2 Martingales……Page 37
1.7 Birth and death chains……Page 39
1.8 Branching and queuing chains……Page 43
1.8.1 Branching chain……Page 44
1.9 Proof of results for the branching and queuing chains……Page 46
1.9.1 Branching chain……Page 48
1.9.2 Queuing chain……Page 49
2.1 Elementary properties of stationary distributions……Page 57
2.2 Examples……Page 59
2.2.1 Birth and death chain……Page 60
2.2.2 Particles in a box……Page 63
2.3 Average number of visits to a recurrent state……Page 66
2.4 Null recurrent and positive recurrent states……Page 70
2.5 Existence and uniqueness of stationary distributions……Page 73
2.5.1 Reducible chains……Page 77
2.6 Queuing chain……Page 79
2.6.1 Proof……Page 80
2.7 Convergence to the stationary distribution……Page 82
2.8 Proof of convergence……Page 85
2.8.1 Periodic case……Page 87
2.8.2 A result from number theory……Page 89
3.1 Construction of jump processes……Page 94
3.2 Birth and death processes……Page 99
3.2.1 Two-state birth and death process……Page 102
3.2.2 Poisson process……Page 104
3.2.3 Pure birth process……Page 108
3.2.4 Infinite server queue……Page 109
3.3 Properties of a Markov pure jump process……Page 112
3.3.1 Applications to birth and death processes……Page 114
4.1 Mean and covariance functions……Page 121
4.2 Gaussian processes……Page 129
4.3 The Wiener process……Page 132
5.1.1 Continuity of the mean and covariance functions……Page 138
5.1.2 Continuity of the sample functions……Page 140
5.2 Integration……Page 142
5.3 Differentiation……Page 145
5.4 White noise……Page 151
6 Stochastic Differential Equations, Estimation Theory, and Spectral Distributions……Page 162
6.1 First order differential equations……Page 164
6.2 Differential equations of order $n$……Page 169
6.2.1 The case $n=2$……Page 176
6.3 Estimation theory……Page 180
6.3.1 General principles of estimation……Page 183
6.3.2 Some examples of optimal prediction……Page 184
6.4 Spectral distribution……Page 187
Answers to Exercises……Page 200
Glossary of Notation……Page 209
Index……Page 211
Back cover……Page 214