Reinforcement Learning

What you'll learn

• Formulate decision problems as bandits and Markov decision processes, and apply exploration strategies, returns, value functions and Bellman equations
• Analyze bandit regret, handle non-stationary rewards, apply contextual bandits such as LinUCB, and evaluate new policies offline with IPS and doubly robust estimators
• Compute optimal policies for known MDPs with policy evaluation, policy iteration and value iteration, and explain generalized policy iteration
• Formulate RL problems with suitable states, rewards, potential-based shaping, discounting and constraints, and reason with belief states in POMDPs
• Learn values and policies from experience with Monte Carlo, TD(0), n-step methods, SARSA and Q-learning, and compare on-policy and off-policy behavior
• Combine learning and planning with Dyna, prioritized sweeping and Monte Carlo tree search, and explain AlphaZero and MuZero
• Apply linear and neural function approximation, explain the deadly triad, and describe DQN, model-based and offline RL
• Apply directed exploration methods and distributional RL, and evaluate deep RL agents with appropriate statistics
• Derive and implement policy gradient, REINFORCE with baseline, actor-critic and PPO methods, and explain continuous control algorithms and RLHF
• Use learned models with short rollouts and MPC, and learn safely from fixed data with conservative offline RL and off-policy evaluation
• Learn from demonstrations with behavior cloning, DAgger and inverse RL, and apply game-theoretic and centralized-training ideas to multi-agent RL
• Explain KL-regularized RLHF, reward over-optimization, DPO and GRPO with verifiable rewards, and plan a safe RL deployment for an industrial control problem