Machine Learning Fairness
Your Fairness certificate
This document records the fairness evaluation for your model. Save or print it for stakeholders, audits, or compliance workflows.
What Fairness Means in Machine Learning
Fairness checks look beyond accuracy. They ask whether a model behaves differently for groups that should be treated carefully.
Compare outcomes across groups
A model can be accurate overall and still approve, reject, flag, or rank one group differently from another. Fairness metrics make those differences visible.
Choose a sensitive feature
Common examples include sex, age group, race, ethnicity, disability status, or another feature that needs special review in your domain.
Use fairness with model quality
Fairness does not replace accuracy, precision, recall, calibration, or business review. It adds another check before a model is trusted.
Document the result
A certificate is a simple summary for stakeholders. It should show the metric, achieved score, threshold, issue date, and issuer.
Which Fairness Metric Should You Use?
There is no single fairness metric for every problem. Pick the metric that matches the decision your model supports.
Demographic Parity Ratio
Use it when positive outcome rates should be similar across groups, for example approval, selection, or access decisions.
Equal Opportunity Ratio
Use it when qualified or truly positive cases should be found at similar rates across groups, such as detecting eligible applicants.
Predictive Parity Ratio
Use it when a positive prediction should mean a similar level of confidence or correctness across groups.
Thresholds, Rules, and Legal Context
A fairness score needs context. A threshold is a review rule, not proof that a model is lawful or harmless.
Start with a practical benchmark
A ratio threshold of 0.80 is often used as a simple first review point. Scores below the threshold should trigger investigation.
Remember the 80% rule is not universal
The four-fifths rule comes from employment-selection guidance. It is a practical signal for adverse impact review, not a universal legal definition.
Raise the bar for high-risk decisions
Healthcare, credit, hiring, education, insurance, and public-sector decisions often need stricter review, documentation, and human oversight.
Review laws and standards
AI governance frameworks emphasize risk management, bias testing, transparency, human oversight, monitoring, and accountability.
Rules and Laws to Keep in Mind
Fairness certificates are useful documentation, but they are not legal advice. Treat them as one part of a broader AI governance workflow.
In the European Union, the AI Act uses a risk-based approach. High-risk AI systems can require risk management, data governance, transparency, human oversight, and controls for accuracy, robustness, and cybersecurity. See the Council of the EU AI Act overview.
In the United States, the NIST AI Risk Management Framework is a voluntary framework for managing AI risks, including harmful bias. See the NIST AI Risk Management Framework.
For employment decisions, the EEOC has emphasized that AI and algorithmic tools must comply with federal civil rights laws. The four-fifths rule is described by the EEOC as a practical rule of thumb, not a legal definition. See the EEOC AI and Algorithmic Fairness Initiative and EEOC Uniform Guidelines Q&A.
Fairness Certificate FAQ
Short answers for teams that are new to AI fairness evaluation.
Does passing a fairness check mean the model is legally compliant?
No. Laws vary by jurisdiction, sector, and use case. A certificate is a documentation and review tool, not a legal opinion.
What if I cannot satisfy two fairness metrics simultaneously?
In most real-world settings it is mathematically impossible to satisfy all fairness definitions at once. Decide which metric best addresses the harm you want to prevent, document the trade-off, and involve domain and legal experts.
Can this certificate be used for high-risk EU AI Act applications?
It is a useful evidence artefact but is not sufficient alone. High-risk systems require a full risk management system, technical documentation, conformity assessment, and ongoing monitoring.
How often should fairness be re-evaluated?
Re-evaluate whenever the model is retrained, the data distribution shifts, or the deployment context changes. For high-stakes decisions, consider continuous monitoring rather than point-in-time checks.