## Markov Chains and Stochastic Stability

Authors : Sean Meyn, Dept. of Electrical and Computer Engineering, University of Illinois and Richard Tweedie, Division of Biostatistics, University of Minnesota

ISBN : 0387198326

Pages : 548

Publication Date : 1993, recompiled September 2005

Publisher : Springer-Verlag

The book is divided into three parts. Chains which are psi-irreducible are the focus of this book, and it is in Part I that this class of chains is initially developed. Numerous applications are described, and the foundations and basic definitions are developed directly, and illustrated through these applications. In the second part a description of the possible stable regimes for a Markov chain are described. The dichotomies which exist between transience and recurrence are developed here, and positive recurrence is described for general state space chains. Drift criteria are introduced to enable practical verification of the various forms of stability for specific models. In Part III existing and new ergodic theory is presented, and here the drift criterion approach makes its full impact. Total variation norm limit theorems, laws of large numbers, and functional central limit theorems are obtained through the combined use of drift criteria and splitting techniques.

Many of the theoretical results appear here for the first time, and much of the theory and the models which are used to illustrate the theory, and to provide extensions of the theory in special cases, have not previously been brought together in book form. This book thus provides a readable account of the development over the last two decades of a fundamental and applicable area of stochastic processes, and as such will be of value not only in probability theory but in the many discplines where these models form the basis of analysis.

### ENTIRE BOOK (568 pages in total):

postscript / gzipped postscript / pdf

### INDIVIDUAL CHAPTERS:

Front Matter (Preface, Contents, etc.): postscript / pdf

#### I. COMMUNICATION and REGENERATION

1. Heuristics (pages 3-22): postscript / pdf

2. Markov Models (pages 23-54): postscript / pdf

3. Transition Probabilities (pages 55-81): postscript / pdf

4. Irreducibility (pages 82-102): postscript / pdf

5. Pseudo-atoms (pages 103-129): postscript / pdf

6. Topology and Continuity (pages 130-152): postscript / pdf

7. The Nonlinear State Space Model (pages 153-173): postscript / pdf

#### II. STABILITY STRUCTURES

8. Transience and Recurrence (pages 177-203): postscript / pdf

9. Harris and Topological Recurrence (pages 204-233): postscript / pdf

10. The Existence of pi (pages 234-259): postscript / pdf

11. Drift and Regularity (pages 260-289): postscript / pdf

12. Invariance and Tightness (pages 290-310): postscript / pdf

#### III. CONVERGENCE

13. Ergodicity (pages 313-333): postscript / pdf

14. f-Ergodicity and f-Regularity (pages 334-357): postscript / pdf

15. Geometric Ergodicity (pages 358-386): postscript / pdf

16. V-Uniform Ergodicity (pages 387-414): postscript / pdf

17. Sample Paths and Limit Theorems (pages 415-450): postscript / pdf

18. Positivity (pages 451-469): postscript / pdf

19. Generalized Classification Criteria (pages 470-496): postscript / pdf

#### IV. APPENDICES

A. Mud Maps (pages 500-505): postscript / pdf

B. Testing for Stability (pages 506-510): postscript / pdf

C. A Glossary of Model Assumptions (pages 511-519): postscript / pdf

D. Some Mathematical Background (pages 520-532): postscript / pdf

References: postscript / pdf

Index: postscript / pdf

Symbol Index: postscript / pdf