6.262 Discrete Stochastic Processes

6.262 Discrete Stochastic Processes (Spring 2011, MIT OCW). Instructor: Professor Robert Gallager. Discrete stochastic processes are essentially probabilistic systems that evolve in time via random changes occurring at discrete fixed or random intervals. This course aims to help students acquire both the mathematical principles and the intuition necessary to create, analyze, and understand insightful models for a broad range of these processes. The range of areas for which discrete stochastic-process models are useful is constantly expanding, and includes many applications in engineering, physics, biology, operations research and finance. (from


Lecture 01 - Introduction and Probability Review
Lecture 02 - More Review; The Bernoulli Process
Lecture 03 - Law of Large Numbers, Convergence
Lecture 04 - Poisson (the Perfect Arrival Process)
Lecture 05 - Poisson Combining and Splitting
Lecture 06 - From Poisson to Markov
Lecture 07 - Finite-state Markov Chains; The Matrix Approach
Lecture 08 - Markov Eigenvalues and Eigenvectors
Lecture 09 - Markov Rewards and Dynamic Programming
Lecture 10 - Renewals and the Strong Law of Large Numbers
Lecture 11 - Renewals: Strong Law and Rewards
Lecture 12 - Renewal Rewards, Stopping Trials, and Wald's Inequality
Lecture 13 - Little, M/G/1, Ensemble Averages
Lecture 14 - Review
Lecture 15 - The Last Renewal
Lecture 16 - Renewals and Countable-state Markov
Lecture 17 - Countable-state Markov Chains
Lecture 18 - Countable-state Markov Chains and Processes
Lecture 19 - Countable-state Markov Processes
Lecture 20 - Markov Processes and Random Walks
Lecture 21 - Hypothesis Testing and Random Walks
Lecture 22 - Random Walks and Thresholds
Lecture 23 - Martingales (Plain, Sub, and Super)
Lecture 24 - Martingales: Stopping and Converging
Lecture 25 - Putting It All Together

6.262 - Discrete Stochastic Processes (Spring 2011)
Instructor: Professor Robert Gallager. Course Notes. Lecture Slides. Assignments and Solutions. Exams and Solutions. Download Course Materials.