Mathematics for Machine Learning

What you'll learn

• Use vectors, matrices, eigendecompositions and the SVD to compute similarities, solve systems, find low-rank approximations and perform PCA
• Solve least-squares problems as projections, use rank, leverage and condition numbers, and find minimum-norm solutions of underdetermined systems
• Explain distance concentration and near-orthogonality in high dimensions, search embeddings by cosine similarity, apply random projections and use kernels
• Compute gradients, Jacobians and Hessians of common machine learning functions, derive least squares and softmax cross-entropy gradients, and verify backpropagation numerically
• Explain forward- and reverse-mode automatic differentiation and their costs, choose floating-point formats, and write numerically stable computations
• Apply expectation, variance, Bayes' rule, maximum likelihood and MAP estimation, and explain loss functions and regularization as probabilistic models
• Work with multivariate Gaussians: Mahalanobis distance, Cholesky sampling, conditioning, Bayesian linear regression and Gaussian process regression
• Decompose error into bias, variance and noise, size test sets with concentration bounds, reason about generalization bounds and explain double descent
• Calculate entropy, cross-entropy, KL divergence, perplexity and mutual information, and relate them to training and evaluating models
• Express deep learning operations as tensor algebra: einsum, attention, convolution as matrices, parameter and FLOP counts, and low-rank adaptation
• Recognize convex problems, predict gradient descent behaviour from curvature and conditioning, and solve constrained problems with Lagrange multipliers
• Derive, implement, optimize and evaluate logistic regression and a two-layer network from first principles