Large Language Models and Generative AI

What you'll learn

• Explain next-token pretraining, cross-entropy and perplexity, subword tokenization, decoder-only and mixture-of-experts transformers, and estimate parameters, training compute and memory
• Explain causal self-attention, rotary position embeddings, the KV cache and its reduction by grouped-query and sliding-window attention, efficient attention kernels and context extension
• Apply scaling laws to predict loss, allocate compute between model size and data, and explain over-training for inference, emergence, data limits and test-time compute
• Describe how pretraining data is sourced, extracted, filtered, deduplicated with MinHash and mixed, estimate the value of repeated and synthetic data, and detect benchmark contamination
• Describe supervised fine-tuning, RLHF with Bradley-Terry reward models, DPO, AI feedback, reinforcement learning with verifiable rewards and parameter-efficient fine-tuning, and recognize reward hacking
• Evaluate language models with benchmarks, pass@k, confidence intervals, paired comparisons, Bradley-Terry preference ratings and validated model judges, and build application evaluation sets
• Use sampling settings, prompting, retrieval-augmented generation, tool use and agents effectively and securely, estimate inference memory and cost, and describe the US regulatory context
• Explain prefill and decode bottlenecks, batching and KV memory management, quantization, speculative decoding, distillation and the cost and energy of inference
• Explain diffusion, classifier-free guidance, latent diffusion, diffusion transformers and multimodal language models, and the provenance, deepfake and copyright issues of generative media
• Train and use embeddings with contrastive learning, explain CLIP and text embedding models, and build approximate nearest-neighbor search with measured recall
• Apply layered safeguards, red-teaming, calibrated classifier thresholds, watermarking, privacy protection and fairness testing to deploy language models responsibly
• Specify, build, evaluate and operate a language model application, using evaluation-driven development, cost and latency models and staged launch