Bayesian Learning, 7.5 hp

AI generated image of a mixture distribution

Star

Aim

This is a course on the Master’s Program in Statistics and the Master’s Program in Data Science, Statistics and Decision Analysis at Stockholm University.

The course gives a gentle, but solid, introduction to Bayesian statistics, with special emphasis on models and methods in computational statistics and machine learning.

Contents

  • We will get off to a shocking start by introducing a very different probability concept than the one you are probably used to: subjective probability.
  • We will then move on to the mathematics of the prior-to-posterior updating in basic statistical models, such as the Bernoulli, normal and multinomial models. Bayesian prediction and decision making under uncertainty is carefully explained, and you will hopefully see why Bayesian methods are so useful in modern application where so much focuses on prediction and decision making.
  • The really interesting stuff starts to happen when we study regression and classification. You will learn how prior distributions can be seen as regularization that allows us to use flexible models with many parameters without overfitting the data and improving predictive performance.
  • A new world will open up when we learn how complex models can be analyzed with simulation methods like Markov Chain Monte Carlo (MCMC), Hamiltonian Monte Carlo (HMC) and approximate optimization methods like Variational Inference (VI).
  • You will also get a taste for probabilistic programming languages for Bayesian learning, in particular the popular Stan language in R.
  • Finally, we’ll consider the case with multiple possible models and introduce Bayesian model averaging and model selection.

Literature

  • Villani, M. (2025a). Bayesian Learning. This is the main book for the course.
  • Gelman, Carlin, Stern, Dunson, Vehtari, Rubin (2014). Bayesian Data Analysis (BDA). Chapman & Hall/CRC: Boca Raton, Florida. 3rd edition. Here is the book webpage and a free PDF version.
  • Additional material and handouts distributed during the course.

If you need to refresh some basic mathematics, like derivatives and integrals, you may find the first chapter of this Prequel book useful:

Structure

The course consists of lectures, mathematical exercises and computer labs where you work on a two-part home assignments.

Examination

The course is examined by a

Schedule

The course schedule can be found on TimeEdit. A tip is to select Subscribe in the upper right corner of TimeEdit and then paste the link into your phone’s calendar program.

Interactive material

The course makes use of interactive Observable notebooks in javascript that runs in your browser. The widgets will be linked below each relevant lecture. All widgets used in the course are available here.

Teachers


Mattias Villani
Course responsible and Lecturer
Professor


Oskar Gustafsson
Exercises and Computer labs
PhD in Statistics and Senior Lecturer


Akram Mahmoudi
Computer labs
PhD in Statistics and Lecturer


Valentin Zulj
Computer labs
PhD student in Statistics and Teaching assistant