Fairness definitions in ML: demographic parity and equalized odds
Demographic parity requires that prediction outcomes be independent of a sensitive attribute, while equalized odds requires equal error rates across groups. These definitions can conflict, as shown by trade-off theorems. Practical tools such as Fairlearn and the What-If Tool help evaluate and mitigate disparities.
Why fairness definitions matter
Human biases can be unintentionally replicated by machine learning algorithms. The Google Machine Learning Crash Course notes that evaluating a model responsibly requires auditing training data and evaluating predictions for bias.
To address these concerns, researchers have developed formal definitions of fairness. The textbook Fairness and Machine Learning introduces non-discrimination criteria in a decision-theoretic setting.
Demographic parity
Demographic parity requires that the predicted outcome be independent of the sensitive attribute. This means that the proportion of positive predictions should be the same across groups defined by that attribute.
This definition focuses on the distribution of outcomes rather than on correctness. It does not consider whether the model's predictions are accurate for each group.
Equalized odds and equality of opportunity
Equalized odds requires that true positive and false positive rates be equal across groups. The paper "Equality of Opportunity in Supervised Learning" proposes this criterion and shows how to adjust a predictor to remove discrimination.
Equality of opportunity is a relaxation that requires only equal true positive rates. The paper describes both as oblivious measures that depend only on joint statistics of predictor, target, and sensitive attribute.
Trade-offs between fairness definitions
Fairness conditions can be mutually incompatible. The paper on inherent trade-offs proves that three fairness conditions cannot be satisfied simultaneously except in highly constrained special cases.
This result means that practitioners must choose which definition to prioritize, as no single model can meet all reasonable fairness criteria at once.
Tools for assessing fairness
Several open-source tools support fairness assessment and mitigation. The Fairlearn project provides metrics and algorithms, emphasizing that fairness is sociotechnical and requires considering societal context.
Visual tools allow users to explore model behavior. The What-If Tool enables users to test performance in hypothetical situations and visualize different fairness metrics.
| Definition | Condition | Group focus | Example |
|---|---|---|---|
| Demographic parity | Prediction independent of sensitive attribute | Proportions of outcomes across groups | Equal selection rates for loan approval |
| Equalized odds | Equal true positive and false positive rates across groups | Error rates for each group | Similar misclassification across genders |
| Equality of opportunity | Equal true positive rates across groups | Positive outcomes for qualified individuals | Same detection rate for qualified applicants |
Frequently asked questions
What is demographic parity?
Demographic parity requires that the predicted outcome be independent of a sensitive attribute, so the proportion of positive predictions is equal across groups.
What is equalized odds?
Equalized odds requires that true positive and false positive rates are equal across groups, ensuring similar error rates.
How do fairness definitions conflict?
Research shows that three fairness conditions cannot be satisfied simultaneously except in special cases, so trade-offs are inevitable.
What tools can help measure fairness?
Fairlearn provides metrics and algorithms, while the What-If Tool allows visual probing of model behavior for different fairness metrics.
What is the difference between equalized odds and equality of opportunity?
Equalized odds requires equal true positive and false positive rates, while equality of opportunity requires only equal true positive rates.