Stochastic processes
BMETE95AM41
2024/25. 1st semester

Classes

Homeworks

• 1st homework assignment: due at 10.15 on 13th Sep
• 2nd homework assignment: due at 14.15 on 25th Sep
• 3rd homework assignment: due at 14.15 on 2nd Oct
• 4th homework assignment: due at 14.15 on 9th Oct
• 5th homework assignment: due at 14.15 on 16th Oct
• 6th homework assignment: due at 10.15 on 25th Oct
• 7th homework assignment: due at 14.15 on 6th Nov
• 8th homework assignment: due at 14.15 on 13th Nov
• 9th homework assignment: due at 10.15 on 22nd Nov
• 10th homework assignment: due at 10.15 on 6th Dec

Homework exercise tests (HWET)

• 13th Sep, Fri 10.15-10.45
• 27th Sep, Fri 10.15-10.45
• 11th Oct, Fri 10.15-10.45
• 25th Oct, Fri 10.15-10.45
• 8th Nov, Fri 10.15-10.45
• 22nd Nov, Thu 10.15-10.45
• 6th Dec, Fri 10.15-10.45

Exam

• 13th Dec, Fri 10.00 H405A
• 9th Jan, Thu 10.00 H507
• 23rd Jan, Thu 10.00 H507

Literature

• Lecture slides by Károly Simon
• R. Durrett: Essentials of Stochastic Processes, Second edition, Springer 2012. A legal copy is available here.
• G. F. Lawler: Introduction to Stochastic Processes, Second edition, Chapman & Hall/CRC 2006
• R. Durrett: Probability: Theory and Examples, 5th edition, Cambridge University Press 2017. A legal copy is available here.
• D. A. Levin, Y. Peres, E. L. Wilmer: Markov Chains and mixing times, American Mathematical Society, 2017. A legal copy is available here.

Lecture diary

• week 1, 4th Sep, Wed, 90 min: gambler's ruin, Markov property, Markov chain, evolution of distributions on the state space, Ehrenfest chain, absorbing, recurrent, transient states, stationary distribution
• week 1, 6th Sep, Fri, 180 min: computation of stationary distribution, social mobility chain, classification of states: transience, positive and null recurrence, convergence theorems, exit distributions from transient states, Perron-Frobenius theorem, periodic Markov chains
• week 2, 11th Sep, Wed, 90 min: all stationary distribution of non-irreducible chains, inventory chain, Wright-Fisher model without and with mutation, simple random walk on simple graphs, knight moves on the chessboard, mean first passage time matrix
• week 2, 13th Sep, Fri, HWET + 105 min: basketball chain, stationary distribution of the Ehrenfest chain with generating functions, bistochastic matrices, detailed balance, reversible Markov chains, time reversal
• week 3, 18th Sep, Wed, 90 min: time reversal and adjoint operators, birth and death chains, two year collage, exit distribution, one-step argument, harmonic characterization of the exit probability
• week 3, 20th Sep, Fri, 135 min: exit distribution in the Wright-Fisher model, gambler's ruin (fair and unfair), tennis at 3:3, exit distribution in terms of the reduced transition matrix, expected exit time with one-step argument, harmonic characterization, expected exit time in two year collage, tennis at 3:3, gambler's ruin (fair and unfair), general description of the exit distribution, expected exit time and expected number of visits in terms of the reduced transition matrix
• week 4, 25th Sep, Wed, 90 min: probability that a state is visited before the other, return happens before the visit of another state in irreducible Markov chain, properties of the generating function of probability distributions, branching processes
• week 4, 27th Sep, Fri, HWET + 150 min: extinction probability in branching processes, stopping times, strong Markov property, recurrent and transient states, characterization by powers of the transition matrix, null recurrence of simple symmetric random walk in one dimension
• week 5, 2nd Oct, Wed, 90 min: upper bound on the tail of hitting times, review of exponential and Poisson distributions, definition and basic properties of the Poisson process
• week 5, 4th Oct, Fri, no lecture
• week 6, 9th Oct, Wed, 90 min: interevent times in the Poisson process, conditioning the Poisson process to the number of events at a given time, distribution of points, inhomogeneous Poisson process, compound Poisson process, thinning and inhomogeneous thinning of Poisson process, M/G/infinity queue
• week 6, 11th Oct, Fri HWET + 105 min: superposition of Poisson processes, Poisson race, barbershop example, Markov chains in continuous time, Markov property, continuity assumption, Chapman-Kolmogorov equation, infinitesimal generator, Kolmogorov's forward differential equation
• week 7, 16th Oct, Wed, 90 min: Kolmogorov's backward equation, conditions for infinite state space, waiting times, routing matrix, irreducibility, stationary distribution, detailed balance
• week 7, 18th Oct, Fri, 135 min: construction of continuous time Markov chains in finite state space, spectrum of the infinitesimal generator, stationary distribution for two states, 3 frogs at the pond, birth and death chains, stationary distribution of the barbershop, M/M/s queue and M/M/s queue with balking, branching processes, pure birth processes, fast growing population models, explosion
• week 8, 23rd Oct, Wed, holiday
• week 8, 25th Oct, Fri, HWET + 150 min: exit distribution and expected exit time for continuous time Markov chains, when can the kindergarden teacher go home, stationary distribution, expected queue length, expected waiting time for the M/M/1 queue, M/M/1 queue with finite waiting room, Little's formula, queue length and expected waiting time for the barbershop, M/M/s queue
• week 9, 30th Oct, Wed, 90 min: conditional expectation for discrete and jointly continuous random variables, examples, sigma-algebra, measurability, general definition of conditional expectation with respect to sigma-algebras
• week 9, 1st Nov, Fri, holiday
• week 10, 6th Nov, Wed, 90 min: basic measure theory, measures, signed measures, Radon-Nikodym theorem, existence and uniqueness of the conditional expectation, properties
• week 10, 8th Nov, Fri, HWET + 105 min: definition of martingales, examples, martingales as functions of Markov chains, convex functions of martingales, martingales in L²
• week 11, 13th Nov, Wed, 90 min: further examples of martingales, orthogonality of martingale differences for L² martingales, Pythagoream formula, betting strategies, martingale transform, stopping times
• week 11, 15th Nov, Fri, 135 min: stopped martingale, examples with random walk, one part of the optional stopping theorem, Wald's equation, martingale convergence theorem, Doob's martingale inequality, Pólya's urn
• week 12, 20th Nov, Wed, 90 min: limiting distribution in Pólya's urn, Hoeffding's inequality and its application in problem 5.11
• week 12, 22nd Nov, Fri, HWET + 105 min: problem 5.11 and 5.10, review of normal distribution, properties, multivariate normal distribution, definition of Brownian motion
• week 13, 27th Nov, Wed, 90 min: Lévy's construction of Browian motion, properties of Brownian motion, multidimensional Brownian motion
• week 13, 29th Nov, Fri, no lecture (open day)
• week 14, 4th Dec, Wed, 90 min: continuity of the time inversion of Brownian motion, further properties of Brownian motion (reflection principle, recurrence, zero set), one-parameter family of Markov kernels
• week 14, 6th Dec, Fri, HWET + 105 min: Yule process, log-optimal portfolio