Natural Language Processing

What you'll learn

• Normalize and tokenize text, train and apply byte-pair encoding, build TF-IDF representations, and estimate and evaluate smoothed n-gram language models with perplexity
• Explain the distributional hypothesis and build word embeddings with PPMI and SVD and with skip-gram negative sampling, and assess their properties, limitations and biases
• Build and compare text classifiers from naive Bayes to fine-tuned encoders, and label sequences with BIO tagging, Viterbi decoding and CRF-style transition constraints
• Describe and analyze recurrent, LSTM, GRU, bidirectional and encoder-decoder models, diagnose vanishing gradients, and apply greedy, beam and sampling-based decoding
• Explain neural machine translation, apply beam search with length normalization, and evaluate translations with BLEU, chrF and learned metrics
• Derive and implement scaled dot-product and multi-head attention, positional encodings and transformer blocks, and estimate parameters, compute and KV-cache memory
• Compare encoder, decoder and encoder-decoder pretraining, estimate parameters and training compute, and apply scaling laws to choose model and data size
• Explain instruction tuning, RLHF and DPO, write and test effective prompts, use chain-of-thought and self-consistency, set decoding parameters, and mitigate hallucination
• Build dense and hybrid retrieval with contrastive training, approximate nearest-neighbor indexes and reranking, and design, chunk and evaluate retrieval-augmented generation
• Adapt and serve models efficiently with LoRA, quantization, distillation, KV-cache management and speculative decoding, and choose a model size for an application
• Design reliable information extraction and question answering systems, measure them with exact match and F1, and build tool-using agents with appropriate safeguards
• Evaluate NLP systems with classification, span, generation, retrieval and benchmark metrics and paired bootstrap tests, and assess applications, risks and rules for deployment