1. Virtual Laboratories
  2. 0
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5
  8. 6
  9. 7
  10. 8
  11. 9
  12. 10
  13. 11
  14. 12
  15. 13
  16. 14
  17. 15
  18. 16

15. Markov Chains

Summary

A Markov process is a random process in which the future is independent of the past, given the present. Thus, Markov processes are the natural stochastic analogs of the deterministic processes described by differential and difference equations. They form one of the most important classes of random processes.

Basic Topics

  1. Introduction
  2. Recurrence and Transience
  3. Periodicity
  4. Invariant and Limiting Distributions
  5. Time Reversal

Special Models

  1. The Ehrenfest Chains
  2. The Bernoulli Laplace Chain
  3. Reliability Chains
  4. The Branching Chain
  5. Queuing Chains
  6. Birth Death Chains
  7. Random Walks on Graphs

Applets

Topics in PDF Format

External Resources

The study of Markov chains is a core topic in every textbook on stochastic processes.

Quote

  1. Virtual Laboratories
  2. 0
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5
  8. 6
  9. 7
  10. 8
  11. 9
  12. 10
  13. 11
  14. 12
  15. 13
  16. 14
  17. 15
  18. 16