Probabilistic Models and Inference

What you'll learn

• Apply Bayes' theorem with conjugate priors to compute posteriors, credible intervals, predictive distributions, Bayes factors and hierarchical shrinkage estimates
• Use standard distributions, maximum likelihood and MAP estimation, exponential-family properties, multivariate Gaussian conditioning and KL divergence
• Represent joint distributions as Bayesian networks and Markov random fields, read conditional independence from graphs, and perform exact inference by variable elimination and on hidden Markov models
• Learn graphical model parameters and structure, apply loopy belief propagation and conditional random fields, and estimate causal effects by back-door adjustment
• Fit Gaussian mixtures and other latent variable models with the EM algorithm, explain why EM increases the likelihood, and select the number of components
• Perform Gaussian process regression with suitable kernels, learn hyperparameters by marginal likelihood, and explain approximations for classification and large data
• Estimate expectations with Monte Carlo, importance sampling and MCMC (Metropolis-Hastings, Gibbs, HMC), and diagnose convergence with trace plots, R-hat and effective sample size
• Fit Bayesian linear regression and generalized linear models, apply the Laplace approximation to logistic regression, and build hierarchical regressions with partial pooling
• Derive and implement variational inference with the ELBO, mean-field updates and reparameterization gradients, and compare its behavior with MCMC
• Apply the Bayesian workflow with probabilistic programming, prior and posterior predictive checks, PSIS-LOO and WAIC comparison, and simulation-based calibration
• Implement Kalman filters, smoothers, extended and unscented filters and particle filters for state-space models, and diagnose filters from their innovations
• Assess and improve calibration, quantify uncertainty in deep networks, construct conformal prediction intervals, apply Bayesian optimization, and design a probabilistic decision system