Evaluation and applications

Building an NLP system is only half the work; knowing whether it works, how well, for whom, and compared with what is the other half. Language is ambiguous and outputs are often open-ended, which makes evaluation harder than in many other areas of machine learning. This lesson covers how NLP systems are evaluated, from classification metrics through generation metrics, retrieval metrics, benchmarks and human judgment to statistical testing, and then surveys the major applications and the risks and rules that govern their deployment.

Evaluating classifiers and taggers

For classification tasks (sentiment, topic, spam, intent), accuracy is the share of correct predictions, but it misleads when classes are imbalanced: a spam filter that never flags anything is 98% accurate if 2% of messages are spam. Precision, recall and F1 focus on a class of interest:

Precision asks how many flagged items were right; recall asks how many true items were found. Which matters more depends on the application: a medical screening tool needs high recall, while an automatic email deletion rule needs high precision. With several classes, micro-averaging pools all decisions (so frequent classes dominate) and macro-averaging averages per-class scores (so every class counts equally, including rare ones).

For named entity recognition, the standard is exact-match span evaluation: a predicted entity counts as correct only if its boundaries and type both match the gold annotation. A prediction with the right type but wrong boundaries is both a false positive and a false negative.

# entity spans as (start, end, type); exact match of boundaries and type counts as correct
gold = [{(0, 2, "PER"), (5, 6, "ORG"), (9, 10, "LOC")},
        {(1, 3, "ORG"), (6, 7, "LOC")},
        {(0, 1, "PER"), (3, 5, "PER"), (8, 9, "LOC"), (12, 14, "ORG")}]
pred = [{(0, 2, "PER"), (5, 6, "LOC"), (9, 10, "LOC")},
        {(1, 2, "ORG"), (6, 7, "LOC")},
        {(0, 1, "PER"), (3, 5, "PER"), (8, 9, "LOC"), (10, 11, "MISC")}]

def prf(tp, fp, fn):
    p = tp / (tp + fp) if tp + fp else 0.0
    r = tp / (tp + fn) if tp + fn else 0.0
    f = 2 * p * r / (p + r) if p + r else 0.0
    return p, r, f

types = sorted({t for s in gold + pred for (_, _, t) in s})
totals = {t: [0, 0, 0] for t in types}
for g, p in zip(gold, pred):
    for t in types:
        gt = {e for e in g if e[2] == t}; pt = {e for e in p if e[2] == t}
        totals[t][0] += len(gt & pt); totals[t][1] += len(pt - gt); totals[t][2] += len(gt - pt)

for t in types:
    p, r, f = prf(*totals[t])
    print(f"{t:5s} tp={totals[t][0]} fp={totals[t][1]} fn={totals[t][2]}  P={p:.2f} R={r:.2f} F1={f:.2f}")
tp, fp, fn = (sum(v[i] for v in totals.values()) for i in range(3))
print(f"micro P/R/F1 = {prf(tp, fp, fn)[0]:.3f} / {prf(tp, fp, fn)[1]:.3f} / {prf(tp, fp, fn)[2]:.3f}")
scored = [t for t in types if totals[t][0] + totals[t][2] > 0]     # types present in the gold data
print(f"macro F1 = {sum(prf(*totals[t])[2] for t in scored) / len(scored):.3f}  over {scored}")

LOC   tp=3 fp=1 fn=0  P=0.75 R=1.00 F1=0.86
MISC  tp=0 fp=1 fn=0  P=0.00 R=0.00 F1=0.00
ORG   tp=0 fp=1 fn=3  P=0.00 R=0.00 F1=0.00
PER   tp=3 fp=0 fn=0  P=1.00 R=1.00 F1=1.00
micro P/R/F1 = 0.667 / 0.667 / 0.667
macro F1 = 0.619  over ['LOC', 'ORG', 'PER']

The tagger is perfect on people and finds every location, but it mislabels one organization as a location, gets one organization's boundaries wrong and misses another, and invents a MISC entity. Micro F1 (0.667) pools all entities; macro F1 (0.619) averages the three gold types equally and is pulled down by the zero score on organizations. Reporting per-type scores shows where the system fails, which a single number hides.

Evaluating generated text

When the output is text, as in translation and summarization, there are many correct answers, so exact match is useless. Automatic metrics compare outputs with one or more human references:

• BLEU (2002), the standard for machine translation for two decades, computes the precision of matching n-grams (n = 1 to 4) against the references, with counts clipped so repeated words are not rewarded, combined by geometric mean and multiplied by a brevity penalty for outputs shorter than the reference.
• ROUGE, used for summarization, measures recall of reference n-grams (ROUGE-1, ROUGE-2) or the longest common subsequence (ROUGE-L).
• chrF compares character n-grams, which suits languages with rich morphology.
• Learned metrics such as BERTScore and COMET compare contextual embeddings or are trained to predict human quality judgments, and correlate better with human ratings than n-gram overlap.

import math
from collections import Counter

def ngrams(t, n):
    return Counter(tuple(t[i:i + n]) for i in range(len(t) - n + 1))

def bleu(cand, refs, max_n=4):
    c = cand.split(); rs = [r.split() for r in refs]
    log_p = 0.0
    for n in range(1, max_n + 1):
        cn = ngrams(c, n)
        max_ref = Counter()
        for r in rs:
            for g, k in ngrams(r, n).items():
                max_ref[g] = max(max_ref[g], k)
        clipped = sum(min(k, max_ref[g]) for g, k in cn.items())
        total = max(sum(cn.values()), 1)
        p = (clipped if clipped else 0.1) / total        # floor smoothing for zero matches
        log_p += math.log(p) / max_n
    ref_len = min((abs(len(r) - len(c)), len(r)) for r in rs)[1]   # closest reference length
    bp = 1.0 if len(c) > ref_len else math.exp(1 - ref_len / len(c))
    return bp * math.exp(log_p)

refs = ["the cat is sitting on the mat", "a cat sits on the mat"]
for cand in ["the cat is sitting on the mat",
             "a cat is sitting on the mat",
             "the cat sat on the mat",
             "on the mat the cat is sitting",
             "the the the the the the the",
             "the cat"]:
    print(f"{bleu(cand, refs):.3f}  {cand}")

1.000  the cat is sitting on the mat
0.880  a cat is sitting on the mat
0.254  the cat sat on the mat
0.595  on the mat the cat is sitting
0.039  the the the the the the the
0.043  the cat

An exact copy of a reference scores 1.0, and a close paraphrase 0.88. But "the cat sat on the mat", a perfectly good translation, scores only 0.25 because it uses "sat" instead of "is sitting" and shares few 3-grams and 4-grams with the references, while the scrambled "on the mat the cat is sitting" scores 0.60. Clipping stops "the the the..." from scoring well, and the brevity penalty punishes "the cat". BLEU is useful for comparing systems over large test sets, where errors average out, but poor for judging individual sentences. Modern practice reports a learned metric such as COMET alongside BLEU or chrF, and uses human evaluation for important decisions.

Evaluating large language models

General-purpose language models are evaluated on suites of benchmarks covering knowledge (MMLU and its harder successors), reasoning and mathematics (GSM8K, MATH and competition problems), coding (HumanEval, SWE-bench, which asks models to fix real software issues), reading comprehension, multilingual ability, and safety behavior. Several problems complicate these numbers:

• Saturation: benchmarks that once separated models are now near their ceilings, so new, harder ones are constantly needed.
• Contamination: benchmark questions found on the web may have been in the training data, inflating scores. Held-out, recently created or private test sets reduce the risk.
• Prompt sensitivity: small changes in instructions, examples or answer formatting can change scores by several points.
• Validity: a high score on multiple-choice questions does not guarantee usefulness in real tasks, which are open-ended and long.

For open-ended responses, human preference comparisons remain the reference: raters see two outputs and choose the better one, and results are aggregated into ratings such as Elo scores. Using a strong language model as a judge (LLM-as-a-judge) scales this cheaply and agrees with humans reasonably often, but judges favor longer answers, their own style and the first option shown, so judge evaluations need position swapping, calibration against human labels and care. Increasingly, systems are evaluated on realistic tasks with automatic checks of the outcome, such as whether code passes tests or whether an agent completed a task correctly.

Is the difference real?

Test sets are samples, so a measured difference between two systems may be due to chance. With 500 test examples, an accuracy of 80% has a standard error of about \(\sqrt{0.8 \times 0.2/500} = 1.8\) percentage points. Comparing two systems on the same examples is much more sensitive than comparing them on different ones, because most examples are easy or hard for both. The paired bootstrap resamples test examples with replacement, recomputing the difference each time, to estimate its uncertainty.

import numpy as np

rng = np.random.default_rng(0)
n = 500                                           # test examples
difficulty = rng.random(n)                        # shared difficulty makes errors correlated
truth_a = difficulty < 0.80                       # system A correct on about 80%
truth_b = difficulty + rng.normal(0, 0.1, n) < 0.82   # B slightly better, mostly the same examples
print(f"accuracy A = {truth_a.mean():.3f}, B = {truth_b.mean():.3f}, difference = {truth_b.mean() - truth_a.mean():+.3f}")

diffs = []
for _ in range(10000):                            # paired bootstrap: resample the same examples for both
    i = rng.integers(0, n, n)
    diffs.append(truth_b[i].mean() - truth_a[i].mean())
diffs = np.array(diffs)
lo, hi = np.percentile(diffs, [2.5, 97.5])
print(f"95% interval for B - A: [{lo:+.3f}, {hi:+.3f}]")
print(f"share of resamples where B is not better: {(diffs <= 0).mean():.3f}")
disagree = (truth_a != truth_b).sum()
print("examples where the systems disagree:", int(disagree))

unpaired = [truth_b[rng.integers(0, n, n)].mean() - truth_a[rng.integers(0, n, n)].mean() for _ in range(10000)]
lo2, hi2 = np.percentile(unpaired, [2.5, 97.5])
print(f"unpaired (independent) resampling interval: [{lo2:+.3f}, {hi2:+.3f}]")

accuracy A = 0.766, B = 0.794, difference = +0.028
95% interval for B - A: [+0.004, +0.052]
share of resamples where B is not better: 0.014
examples where the systems disagree: 40
unpaired (independent) resampling interval: [-0.024, +0.080]

System B is 2.8 points more accurate. Because the two systems agree on all but 40 of the 500 examples, the paired interval is narrow, \([+0.4, +5.2]\) points, and B is better in about 99% of resamples. Resampling the systems independently ignores this pairing and gives an interval of \([-2.4, +8.0]\) that includes zero, wrongly suggesting the difference could be noise. With small test sets or many comparisons, report intervals and avoid declaring winners on differences of a fraction of a point.

Applications

NLP underlies a wide range of products and services:

• Classification and moderation: spam filtering, sentiment analysis, routing support tickets, detecting harmful content.
• Information extraction: named entities, relations and events from documents, contracts, medical records and news; turning text into structured data.
• Machine translation: neural translation since 2016, now largely by large multilingual models, approaching human quality for common language pairs but still weaker for low-resource languages and specialized domains.
• Summarization and question answering: condensing reports, answering questions over documents and knowledge bases.
• Speech: speech recognition and synthesis, now built on transformer models trained on hundreds of thousands of hours of audio.
• Conversational assistants and agents: general-purpose chat models and agents that use tools, write code and carry out multi-step tasks (AI-410).
• Search and retrieval: finding relevant documents, increasingly as the first stage of retrieval-augmented generation.

Search and retrieval-augmented generation

Search systems rank documents for a query. The classical method BM25 refines TF-IDF (Lesson 1) with term-frequency saturation (the tenth occurrence of a word adds much less than the first) and document-length normalization:

with typical \(k_1 \approx 1.2\) and \(b \approx 0.75\). Retrieval is evaluated with metrics such as recall@k (the share of relevant documents found in the top \(k\)) and mean reciprocal rank (MRR, the average of \(1/\text{rank}\) of the first relevant result).

import math
from collections import Counter

docs = ["how to reset a home wifi router to factory settings",
        "router lights blinking orange means no internet connection",
        "recipe for a quick tomato pasta sauce",
        "change the wifi password on your router admin page",
        "pasta cooking times for spaghetti and penne",
        "internet slow at night check router channel and interference",
        "tomato plants need full sun and regular watering",
        "update router firmware to fix security problems"]
queries = {"reset router": {0},
           "no internet router lights": {1, 5},
           "tomato sauce recipe": {2},
           "router security": {7, 3},
           "spaghetti dinner ideas": {2, 4},
           "fix slow connection": {5, 1}}

toks = [d.split() for d in docs]
N = len(docs); avgdl = sum(map(len, toks)) / N
df = Counter(w for t in toks for w in set(t))

def bm25(q, t, k1=1.2, b=0.75):
    tf = Counter(t); score = 0.0
    for w in q.split():
        if w in tf:
            idf = math.log(1 + (N - df[w] + 0.5) / (df[w] + 0.5))
            score += idf * tf[w] * (k1 + 1) / (tf[w] + k1 * (1 - b + b * len(t) / avgdl))
    return score

recall3, rr = [], []
for q, rel in queries.items():
    ranking = sorted(range(N), key=lambda i: -bm25(q, toks[i]))
    top3 = ranking[:3]
    recall3.append(len(rel & set(top3)) / len(rel))
    first = next(r for r, i in enumerate(ranking, 1) if i in rel)
    rr.append(1 / first)
    print(f"{q:24s} top 3 = {top3}  relevant = {sorted(rel)}  first relevant at rank {first}")
print(f"recall@3 = {sum(recall3) / len(recall3):.3f}   MRR = {sum(rr) / len(rr):.3f}")

reset router             top 3 = [0, 7, 1]  relevant = [0]  first relevant at rank 1
no internet router lights top 3 = [1, 5, 7]  relevant = [1, 5]  first relevant at rank 1
tomato sauce recipe      top 3 = [2, 6, 0]  relevant = [2]  first relevant at rank 1
router security          top 3 = [7, 1, 3]  relevant = [3, 7]  first relevant at rank 1
spaghetti dinner ideas   top 3 = [4, 0, 1]  relevant = [2, 4]  first relevant at rank 1
fix slow connection      top 3 = [7, 1, 5]  relevant = [1, 5]  first relevant at rank 2
recall@3 = 0.917   MRR = 0.917

BM25 handles queries that share words with relevant documents well, but fails where vocabulary differs: "spaghetti dinner ideas" finds the spaghetti cooking-times document but misses the pasta sauce recipe, which shares no query words, and "fix slow connection" ranks a firmware document first because it contains "fix". Dense retrieval with sentence embeddings (Lessons 2 and 9) matches meaning rather than words and handles such queries better, while BM25 is better at exact names, codes and rare terms. Production systems usually combine both (hybrid search) and rerank the top results with a cross-encoder, a transformer that reads the query and document together.

Retrieval-augmented generation (RAG) retrieves relevant passages for a question and gives them to a language model to answer from, with citations. RAG lets models use private or up-to-date information without retraining and reduces, but does not eliminate, fabricated answers. Its quality depends first on retrieval: if the right passage is not retrieved, the model cannot use it. RAG systems are therefore evaluated in stages, measuring retrieval recall, the faithfulness of answers to the retrieved passages, and end-to-end answer correctness.

Risks

NLP systems, especially generative ones, carry distinctive risks:

• Hallucination: fluent, confident statements that are false, including fabricated citations and legal cases, which have led to sanctions for lawyers who filed them in court.
• Bias and unequal performance: systems may treat dialects, languages and groups differently, for example flagging African American English as toxic more often, or performing worse for speakers of less-resourced languages.
• Privacy: models can memorize and reveal training data, and user inputs to hosted services may contain personal or confidential information.
• Security: prompt injection, in which instructions hidden in retrieved documents or web pages hijack a model's behavior, is a major risk for systems that read untrusted text or act on a user's behalf.
• Misuse: generating spam, disinformation and phishing at scale, and impersonation through synthetic voice and text.

Mitigations combine evaluation of these specific risks before deployment, grounding answers in retrieved sources with citations, human review for consequential decisions, input and output filtering, limiting the actions an agent can take, monitoring in production, and clear disclosure to users that they are interacting with an AI system.

Rules in the United States

The US has no comprehensive federal AI statute. Federal policy shifted in 2025 from the 2023 executive order on AI safety, revoked in January 2025, toward promoting AI development, and a December 2025 executive order directed federal agencies to challenge state AI laws considered burdensome. Existing laws still apply: the FTC acts against deceptive AI claims and unfair practices, civil rights and fair lending laws prohibit discriminatory outcomes from automated decisions, and sector rules govern health (HIPAA) and finance. States have moved independently: New York City requires bias audits of automated hiring tools, Colorado has enacted an AI act for high-risk systems whose start has been delayed and debated, California has adopted transparency requirements for frontier model developers and for AI-generated content, and Texas and other states have passed their own AI laws. Organizations must track rules in each state where they operate.

Whatever the jurisdiction, good practice is the same: document the intended use and limits of an NLP system, evaluate it on data representative of the people it will serve and on the specific risks above, keep humans responsible for consequential decisions, and monitor it after deployment. Evaluation is not a one-time step but a continuing part of operating a language system.

This lesson draws together the methods of Lessons 7 to 11: evaluation applies to every system built with them.