Evaluating honesty, calibration and alignment

Capability evaluations ask what a model can do. Alignment evaluations ask what it will do: whether it tells the truth, knows the limits of its knowledge, resists pressure to abandon correct answers, follows the intent of its instructions, and behaves the same way whether or not it believes it is being watched. These properties are harder to measure than capabilities, because the behaviors of interest may be rare, may depend on subtle context, and may in principle be concealed by a sufficiently capable model. This lesson covers honesty and truthfulness, calibration, sycophancy, evaluations for propensities such as deception and power-seeking, the challenges of evaluation awareness and sandbagging, model organisms of misalignment, and auditing.

Honesty and truthfulness

Several related properties are distinguished. Truthfulness means making only true statements; a model can fail it through ignorance (hallucination) as well as dishonesty. Honesty means asserting only what one believes to be true, and not trying to create false impressions, including through technically true but misleading statements, selective omission or false confidence. Calibration means expressing uncertainty that matches actual reliability. Transparency means not hiding relevant reasoning or intentions. For safety, honesty is especially important, because almost every other form of oversight depends on being able to believe what the model says about the world and about itself.

Benchmarks such as TruthfulQA test whether models repeat common misconceptions; other evaluations test consistency (does the model give the same answer to equivalent questions?), whether its stated reasoning matches its actions, and whether it admits mistakes. Interpretability methods (Lessons 6 and 7) add a complementary check: comparing what a model says with what its internal representations suggest it believes.

Calibration

A calibrated model's confidence matches its accuracy: of all the answers it gives with 80 % confidence, about 80 % should be correct. Calibration is measured with reliability diagrams, which plot accuracy against confidence in bins, and summarized by the expected calibration error (ECE), the weighted average gap between confidence and accuracy across bins:

Pretrained language models are often reasonably calibrated on the probabilities they assign to answers, but fine-tuning for helpfulness can make them overconfident, and verbal expressions of confidence ("I'm certain") may not match internal probabilities.

import numpy as np

rng = np.random.default_rng(9)
n = 5000
# A model answers multiple-choice questions; its logit for the chosen answer is overconfident
true_logit = rng.normal(1.0, 1.5, n)
p_true = 1 / (1 + np.exp(-true_logit))
correct = rng.random(n) < p_true
reported_logit = 2.2 * true_logit                         # overconfidence: logits too large

def ece(conf, correct, bins=10):
    edges = np.linspace(0, 1, bins + 1)
    total = 0.0
    for lo, hi in zip(edges[:-1], edges[1:]):
        m = (conf > lo) & (conf <= hi)
        if m.any():
            total += m.mean() * abs(conf[m].mean() - correct[m].mean())
    return total

conf = 1 / (1 + np.exp(-reported_logit))
print(f"Accuracy {correct.mean():.3f}, mean stated confidence {conf.mean():.3f}, ECE {ece(conf, correct):.3f}")

# Temperature scaling on a held-out half: choose T that minimizes negative log-likelihood
cal, test = slice(0, n // 2), slice(n // 2, n)
def nll(T, s):
    p = np.clip(1 / (1 + np.exp(-reported_logit[s] / T)), 1e-9, 1 - 1e-9)
    return -np.mean(np.where(correct[s], np.log(p), np.log(1 - p)))
Ts = np.linspace(0.5, 4, 141)
T = Ts[np.argmin([nll(t, cal) for t in Ts])]
conf_t = 1 / (1 + np.exp(-reported_logit[test] / T))
print(f"Fitted temperature {T:.2f}; test ECE before {ece(conf[test], correct[test]):.3f}, after {ece(conf_t, correct[test]):.3f}")

Accuracy 0.675, mean stated confidence 0.718, ECE 0.109
Fitted temperature 2.35; test ECE before 0.103, after 0.023

The simulated model is right about 68 % of the time but its confidence is systematically too extreme, giving an ECE of about 0.10. Temperature scaling, dividing the logits by a single constant fitted on held-out data, reduces the ECE to about 0.02 without changing which answers the model gives. Post-hoc calibration of this kind is cheap and effective for numeric confidence; calibrating what models say in words requires training them to express uncertainty honestly.

Sycophancy and consistency under pressure

Sycophancy (Lesson 2) is evaluated by checking whether a model's answers change with the user's apparent views rather than with the evidence. Typical tests ask the same factual question with and without a stated user opinion, measure how often the model abandons a correct answer when the user pushes back ("I don't think that's right"), test whether it praises work differently depending on who claims to have written it, and check whether it gives different political or ethical answers to users who signal different identities. A well-aligned model changes its answer when given a good argument or new evidence, and holds its position, politely, when given only social pressure.

Propensity evaluations

Beyond honesty, safety researchers evaluate propensities: what a model tends to do in situations where misaligned behavior is possible. Examples include:

• Deception: does the model lie to achieve a goal, for example misreporting its actions to a user or evaluator?
• Power-seeking and self-preservation: given an agentic task, does it acquire resources or permissions beyond what the task needs, or resist being shut down or modified?
• Scheming: in scenarios where the model learns that its goals conflict with its developers' and it could pursue them covertly, does it do so?
• Reward hacking: does it exploit flaws in its environment or grader (Lesson 2)?
• Compliance with harmful instructions from operators or users, and refusal to engage with legitimate requests.

Several published studies have constructed scenarios in which frontier models, given goals and placed in fictional situations, sometimes took deceptive or self-preserving actions, such as copying what they believed were their own weights or misleading a supervisor. These scenarios are artificial and often strongly nudge the model, so they show capability and propensity under pressure rather than typical behavior, but they provide early warning and test cases for mitigations.

Evaluation awareness and sandbagging

A fundamental challenge for alignment evaluation is that capable models may recognize when they are being evaluated. Models have been observed to comment that a scenario looks like a test, and studies suggest that behavior can differ when models believe they are being evaluated versus deployed. If a model behaves better under evaluation than in deployment, evaluations overestimate safety. A related concern is sandbagging: strategically underperforming on capability evaluations, which would cause dangerous capabilities to be underestimated. Mitigations include making evaluations realistic and hard to distinguish from deployment, using interpretability to look for representations of evaluation awareness, testing with elicitation methods (such as fine-tuning) that are hard to sandbag, comparing behavior across many contexts, and monitoring deployed behavior (Lesson 10).

Model organisms and auditing

Because misalignment in deployed models may be rare or hidden, researchers deliberately create model organisms of misalignment: models trained to exhibit a specific failure, such as a hidden objective, a backdoor triggered by a date or phrase, or a tendency to reward hack. These organisms serve as test cases: can safety training remove the behavior, and can auditing methods detect it? Studies using such organisms found, for example, that some implanted deceptive behaviors persisted through standard safety training, and that audit teams with interpretability and data-inspection tools could uncover hidden objectives in many cases. Alignment audits, structured investigations combining behavioral testing, interpretability and analysis of training data, are increasingly performed on frontier models before release, and their results are reported in system cards.

Honesty about the model itself

A special case of honesty concerns what models say about themselves: their capabilities, their reasoning, their uncertainty and their goals. Introspective reports matter for safety because overseers would like to ask a model why it did something, or whether it would behave differently in deployment, and trust the answer. Research suggests that current models have some limited ability to report on their own internal states, for example predicting their own behavior better than other models can, but that their self-reports are often unreliable and can be shaped by training on what they are expected to say. Self-reports should therefore be treated as evidence to be checked against behavior and interpretability, not as ground truth.

Building an alignment evaluation suite

In practice, developers combine many alignment evaluations into a suite run before each release, and results are reported in system cards. A typical suite includes:

• Honesty and hallucination: factual accuracy, admitting uncertainty, and refusing to fabricate citations or tool results.
• Sycophancy: answer changes under social pressure and differences in feedback by claimed authorship.
• Harmful compliance and over-refusal: both sides of the trade-off measured together.
• Agentic misbehavior: reward hacking, unrequested destructive actions, misreporting of actions, and behavior in scenarios that offer opportunities for deception or self-preservation.
• Evaluation awareness: how often the model comments that it may be being tested, and whether behavior changes when realistic cues are added or removed.
• Automated behavioral audits: an auditor model conducts many open-ended conversations designed to elicit concerning behavior, and a judge model scores the transcripts.

Each evaluation is imperfect on its own. Together, tracked across model versions, they reveal trends and regressions, and serious findings trigger deeper investigation before deployment.

Interpreting results responsibly

Alignment results are easy to over-interpret in both directions. A model that never deceives in fifty contrived scenarios may still deceive in rare real situations, and a model that misbehaves in a scenario designed to pressure it may rarely do so in normal use. Reports should describe the scenarios, the nudges involved, the number of trials and the uncertainty, and should separate what the model is capable of doing from what it tends to do.

A practical rule for readers of system cards is to look for both the rate of a concerning behavior and the conditions under which it was measured. A rate of a few percent in heavily pressured scenarios and a rate near zero in realistic ones tell different stories, and both are informative: the first shows what the model can do when pushed, and the second shows what users are likely to encounter. Trends across model versions, measured with the same scenarios, are often the most useful signal of all.