Fairness Metrics
Understand the mathematical foundation of fairness analysis with demographic parity, equal opportunity, and disparate impact.
Demographic Parity
Measures whether different demographic groups receive positive outcomes at the same rate.
P(Ŷ = 1 | A = a) = P(Ŷ = 1 | A = b)
Positive prediction rate should be equal across groups
Male Approval Rate
Female Approval Rate
30% difference indicates potential bias
- FairDifference ≤ 10%
- Warning10% - 20%
- Biased> 20%
Equal Opportunity
Requires equal true positive rates across groups. Qualified individuals should have equal chance of positive outcome.
P(Ŷ = 1 | Y = 1, A = a) = P(Ŷ = 1 | Y = 1, A = b)
True positive rate (TPR) should be equal across groups
TPR = True Positives / (True Positives + False Negatives) Group A: TPR = 80 / (80 + 20) = 0.80 Group B: TPR = 60 / (60 + 40) = 0.60 Difference: |0.80 - 0.60| = 0.20
A 20% TPR difference means qualified individuals in Group B are 20% less likely to receive approval than Group A, indicating systematic disadvantage.
Disparate Impact
Legal standard measuring if selection rate for one group is less than 80% of another group's rate (80% rule).
DI = P(Ŷ = 1 | A = unprivileged) / P(Ŷ = 1 | A = privileged)
Ratio of selection rates between groups
White approval rate: 75% Black approval rate: 50% DI = 50% / 75% = 0.67 Result: 0.67 < 0.80 → Fails 80% rule
- PassDI ≥ 0.80
- Warning0.70 - 0.80
- Fail< 0.70
- * Based on EEOC guidelines
Choosing Metrics
Demographic Parity
Use when you want equal representation in positive outcomes (hiring quotas, lending targets)
Equal Opportunity
Use when you want to ensure qualified individuals have equal chance (education admissions, promotions)
Disparate Impact
Use for legal compliance with EEOC and fair lending regulations
It's mathematically impossible to satisfy all fairness definitions simultaneously when base rates differ between groups. Choose metrics aligned with your use case and regulatory requirements.