Bias mitigation techniques

Once a disparity has been measured and a fairness goal chosen, the next question is how to reduce it. Mitigation techniques are usually grouped by where they act in the pipeline. Pre-processing methods change the training data; in-processing methods change the learning algorithm; post-processing methods change the predictions of a trained model. Each family has strengths, limitations and legal considerations. This lesson surveys the main techniques, implements three of them on a simulated hiring data set with biased historical labels, and discusses how to choose and validate a mitigation in practice.

Start with the problem, not the algorithm

Technical mitigation should come after asking whether the problem can be fixed upstream. If the disparity comes from a bad target variable, change the target. If it comes from missing data, collect more. If it comes from a feature that should not be used, remove it and test for proxies. If it comes from how outputs are used, change the process. Algorithmic fixes are most useful once the data and problem framing are as good as they can be, and they work best when the cause of the disparity is understood. Applying a fairness algorithm without understanding why the disparity exists can hide symptoms while leaving the cause in place.

Pre-processing

Pre-processing transforms the data before any model is trained, so it works with any learning algorithm:

• Resampling: oversample under-represented groups or outcomes, or undersample over-represented ones, so the training set is more balanced.
• Reweighing: assign weights to training examples so that, after weighting, group membership and the label are statistically independent. Kamiran and Calders proposed setting the weight of each (group, label) cell to the expected frequency under independence divided by the observed frequency.
• Relabelling ("massaging"): change some labels near the decision boundary to reduce disparity, an aggressive technique that alters the ground truth.
• Fair representations: learn a transformed feature space that preserves information useful for prediction while removing information about the sensitive attribute, for example with adversarial training.
• Data collection and augmentation: gather or synthesize more data for poorly served groups, often the most effective remedy for quality-of-service disparities.

In-processing

In-processing techniques change the objective or constraints of training:

• Constrained optimization: minimize loss subject to a fairness constraint, for example that the difference in true positive rates is below a tolerance. The reductions approach of Agarwal and colleagues turns any classifier into a constrained one by repeatedly reweighting examples in a game between the learner and a constraint player; it is implemented in open-source libraries such as Fairlearn.
• Regularization: add a penalty proportional to a fairness metric, such as the covariance between predictions and the sensitive attribute.
• Adversarial debiasing: train a predictor alongside an adversary that tries to recover the sensitive attribute from the predictor's output; the predictor is penalized when the adversary succeeds.
• Distributionally robust optimization: minimize the worst-case loss across groups instead of the average loss, which protects groups that would otherwise be dominated by the majority.

In-processing methods can achieve better trade-offs than pre- or post-processing because they optimize directly, but they require access to the training procedure and can be harder to tune.

Post-processing

Post-processing adjusts the outputs of an existing model. The most common technique chooses separate decision thresholds for each group so that selection rates, true positive rates or both are equal, as in the equalized odds post-processing of Hardt and colleagues. Calibration can also be adjusted per group. Post-processing is attractive because it works with any model, including vendor models, and is easy to explain. Its main drawback is that it requires the sensitive attribute at decision time, and treating people differently based on a protected characteristic at the point of decision is restricted or prohibited in some domains and jurisdictions.

A worked comparison

The simulation creates job applicants with a skill score that determines true qualification identically in both groups. The historical labels, however, miss 30 % of qualified members of group B, a form of label bias. One feature is a noisy measure of skill; another is a proxy correlated with group membership. A logistic regression trained on the historical labels is compared with the same model trained with reweighing, and with the baseline model plus a lower threshold for group B chosen to equalize selection rates. Crucially, all metrics are computed against the true qualifications, which in reality we rarely know.

import numpy as np
# Mitigation: reweighing before training and group thresholds after training
rng = np.random.default_rng(5)
n = 20_000
a = rng.random(n) < 0.35                                   # True means group B
skill = rng.normal(0, 1, n)
y = (skill + rng.normal(0, 0.7, n)) > 0.3                  # true qualification, same rule for both groups
y_hist = y & ~(a & (rng.random(n) < 0.30))                 # historical labels miss 30 % of qualified B members
x = np.column_stack([skill + rng.normal(0, 0.5, n), a * 1.0 + rng.normal(0, 0.3, n), np.ones(n)])  # feature 2 is a proxy

def fit(x, t, w, steps=400, lr=0.5):
    b = np.zeros(x.shape[1])
    for _ in range(steps):
        p = 1 / (1 + np.exp(-x @ b))
        b -= lr * x.T @ (w * (p - t)) / w.sum()
    return 1 / (1 + np.exp(-x @ b))

def report(label, pred):
    sel_a, sel_b = pred[~a].mean(), pred[a].mean()
    tpr_a, tpr_b = pred[~a & y].mean(), pred[a & y].mean()
    acc = (pred == y).mean()
    print(f"{label:22s}  {sel_a:.2f}  {sel_b:.2f}  {sel_b / sel_a:5.2f}  {tpr_a:.2f}  {tpr_b:.2f}  {acc:.3f}")

print("method                  selA  selB  ratio  tprA  tprB  accuracy (vs true labels)")
w = np.ones(n)
p0 = fit(x, y_hist, w)
report("baseline", p0 > 0.5)
# reweighing: weight each (group, label) cell so group and label look independent
for g in (False, True):
    for l in (False, True):
        cell = (a == g) & (y_hist == l)
        w[cell] = (np.mean(a == g) * np.mean(y_hist == l)) / cell.mean()
p1 = fit(x, y_hist, w)
report("reweighing", p1 > 0.5)
# group specific thresholds chosen to equalize selection rate with the baseline rate of group A
t_b = np.quantile(p0[a], 1 - (p0[~a] > 0.5).mean())
report("group thresholds", np.where(a, p0 > t_b, p0 > 0.5))
print(f"threshold for group B: {t_b:.2f} instead of 0.50")

method                  selA  selB  ratio  tprA  tprB  accuracy (vs true labels)
baseline                0.34  0.22   0.64  0.64  0.44  0.758
reweighing              0.32  0.28   0.87  0.61  0.53  0.763
group thresholds        0.34  0.34   1.00  0.64  0.63  0.766
threshold for group B: 0.36 instead of 0.50

The baseline model learns the label bias: it selects 22 % of group B against 34 % of group A, a ratio of 0.64, and qualified members of group B are selected only 44 % of the time against 64 % for group A. Reweighing improves the ratio to 0.87 and the TPR for group B to 0.53 while slightly improving accuracy. Group thresholds equalize selection rates and nearly equalize true positive rates (0.64 against 0.63), and accuracy rises to 0.766. Because the historical labels were biased, the fairer models are also more accurate against the truth. In real projects the true labels are unknown, so this benefit cannot be measured directly; it is one reason why understanding the source of bias matters more than optimizing against the available labels.

Legal and ethical considerations

Mitigation choices interact with anti-discrimination law. Some laws prohibit disparate treatment, using a protected attribute directly in decisions, while also prohibiting unjustified disparate impact. Group-specific thresholds are a form of disparate treatment and may be unlawful in some contexts, such as US employment, where "race norming" of test scores is explicitly banned. Pre-processing and in-processing methods that use the attribute only during training occupy a less settled position. Organizations should seek legal advice on which techniques are permissible in their domain and jurisdiction, document the rationale, and prefer approaches that fix root causes. AI-440 discusses the legal landscape in more depth.

In Canada, human rights law permits special programs designed to relieve disadvantage for protected groups (for example under section 16 of the Canadian Human Rights Act and the Canadian Charter of Rights and Freedoms section 15(2)), which can provide a basis for some targeted measures; organizations should obtain legal advice for their province and sector.

Choosing and validating a mitigation

A practical procedure for mitigation is:

• Diagnose: identify the source of the disparity using the questions from Lesson 2.
• Define the target: choose the fairness metric and an acceptable tolerance with stakeholders.
• Try upstream fixes first: better labels, data or features.
• Compare techniques: train several mitigated models and plot them on a fairness and accuracy Pareto chart.
• Validate on held-out data with confidence intervals, for every relevant group and intersection, because mitigation for one attribute can worsen disparities for another.
• Check side effects: calibration, performance for the majority group, robustness and explainability.
• Document the choice and its rationale, and monitor after deployment, because data drift can undo mitigation (Lesson 12).

Tools

Open-source libraries make these techniques accessible. Fairlearn provides metrics, the reductions approach and threshold optimization for scikit-learn models. IBM's AI Fairness 360 includes reweighing, adversarial debiasing, reject option classification and many metrics. Google's What-If Tool and the Responsible AI Toolbox offer interactive exploration. Tools make the computations easy; the hard parts remain choosing what to measure and deciding which trade-offs are acceptable.

Mitigation for unstructured data

For images, audio and text, mitigation often focuses on data and evaluation. Collecting more examples from poorly served groups, for example voice recordings from speakers with underrepresented accents, is usually the most effective step. Data augmentation can help when collection is hard, such as varying lighting and skin tone rendering in images, but synthetic variation must be realistic or the model learns artefacts. Group-balanced sampling during training ensures that each mini-batch includes enough examples from smaller groups. Worst-group optimization methods such as group distributionally robust optimization explicitly minimize the loss of the worst-performing group, which is particularly useful when a model relies on spurious correlations that hold for the majority but not for minorities. Finally, evaluation sets must be large enough per group to detect the gaps that matter.

Common pitfalls in practice

Several pitfalls recur in real mitigation projects. Teams sometimes optimize a fairness metric on the same data they use to report it, overstating the improvement; a separate held-out set avoids this. They sometimes mitigate on a binary attribute and miss intersectional groups, which can end up worse off. They may apply a threshold adjustment chosen on last year's data without checking that base rates and score distributions are still the same. And they may report the fairness improvement without the side effects on calibration or on the majority group's accuracy, leaving decision-makers unable to judge the trade-off. A short mitigation report listing the method, the data used to choose its parameters, results on held-out data for all groups and intersections, and side effects addresses all of these.

Mitigation also needs maintenance. A reweighing scheme tuned to one data set must be recomputed when the data change, and thresholds need review when the population shifts. Building mitigation into the training pipeline, with automated fairness checks that fail the build if gaps exceed agreed tolerances, keeps it from silently decaying.