Probabilistic Machine Learning (Spring 2025)

Instructor: Seong-Hwan Jun.

Outline

Homeworks

• Assignment 1
• Assignment 2
  • Guide for Problem 1.
  • Guide for Problem 2.
• Assignment 3
  • Q2: To implement collapsed-Gibbs sampler, refer to Normal-Wishart (Section 8) on Kevin Murphy’s Conjugate Bayesian Analysis of the Gaussian distribution.
  • Q3: Refer to Chapter 8 of PML2 regarding the details of Kalman filter.

Course Schedule

Jan 21: Overview; condensed PDF slides; Python primer.

Jan 23: Probability and Bayes; condensed PDF slides; sampling.ipynb.

Jan 28 Part 1: Optimization and Sampling; condensed PDF slides; optimization.ipynb.

Jan 28 Part 2: Directed Graphical Models; condensed PDF slides.

Jan 30: Hidden Markov models and forward pass (PML2 9.1-9.2); forward.ipynb.

Feb 4: HMM backward pass and parameter estimation (PML2 9.1-9.2); Kalman filter (PML2 8.1-8.2). forward_backward.ipynb.

Feb 6: Intro to phylogenetics; Felsenstein.ipynb.

Feb 11: EM algorithm and Gaussian mixture model. GMM.ipynb.

Feb 13: Factor graph and sum-product algorithm.

Feb 18: Sequential Importance Sampling and Bootstrap Particle Filter. smc.ipynb.

Feb 20: Discussion of On-line inference for hidden Markov models via particle filters. Slides.

Feb 25: Markov chain Monte Carlo methods. Slides.

Feb 27: Discussion of Markov Chain Sampling Methods for Dirichlet Process Mixture Models. Slides.

Mar 4: Discussion of HW1 and HW2.

Mar 6: Hamiltonian Monte Carlo. Slides.

Mar 11-13: Spring break (no classes).

Mar 18: Variational Inference basics: KL divergence, mean-field VI, coordinate ascent VI with Gaussian mixture model example. Notes.

Mar 20: Class cancelled.

Mar 25: Stochastic VI: REINFORCE, Reparameterization trick, variational auto-encoder with MNIST example. Slides; VAE notebook.

Mar 27: SVI with Pyro. Slides; Bayesian linear regression example.

Apr 1: Discussion.