1. Probability Review
1.1 Basic Concepts
1.2 Independence and Conditional Probability
1.3 Expectation
1.4 Random Vectors
1.5 Transforms of Probability Distributions
1.6 Transformations of Random Vectors
1.7 Conditonal Expectation of Discrete Variables
1.8 Strong Law of Large Numbers

2. Discrete-Time Markov Models
2.1 The Transition Matrix
2.2 Markov Recurrences
2.3 First-Step Analysis
2.4 Topology of the Transition Matrix
2.5 Steady State
2.6 Time Reversal
2.7 Regeneration

3. Recurrence and Ergodicity
3.1 Potential Matrix Criterion
3.2 Recurrence and Invariant Measures
3.3 Positive Recurrence
3.4 Empirical Averages
3.5 Midterm Review

4. The Structure of Hidden Markov Models
4.1 Hidden Markov Models
4.2 The Viterbi Algorithm
4.3 Parameter Estimation for Hidden Markov Models

5. Continuous-Time Markov Models
5.1 Poisson Processes
5.2 Distribution of Continuous-Time HMC
5.3 Kolmogorov's Differential Systems
5.4 The Regenerative Structure
5.5 Recurrence
5.6 Long-Run Behavior