Fairness Monitoring AI Prompt

Implement ongoing fairness monitoring for this production model.

Model: {{model_name}}
Sensitive attributes to monitor: {{sensitive_attributes}} (e.g. age_group, gender, region)
Fairness metric: {{fairness_metric}}

1. Fairness metrics — implement all of the following:

   a. Demographic parity (statistical parity):
   - Positive prediction rate should be equal across groups
   - Disparity = |P(ŷ=1 | group=A) - P(ŷ=1 | group=B)|
   - Alert threshold: disparity > {{dp_threshold}} (e.g. 0.05 = 5 percentage points)

   b. Equal opportunity:
   - True positive rate (recall) should be equal across groups
   - Requires ground truth labels
   - Disparity = |TPR_A - TPR_B|

   c. Predictive parity:
   - Precision (positive predictive value) should be equal across groups
   - Disparity = |Precision_A - Precision_B|

   d. Calibration by group:
   - Among predictions with score ~0.7, 70% should actually be positive, in every group
   - Plot calibration curves separately for each group

2. Monitoring implementation:
   - Compute all fairness metrics weekly on the last 4 weeks of labeled predictions
   - Track trends: is any metric getting worse over time?
   - Statistical significance: use bootstrap confidence intervals to determine if disparities are significant

3. Alerting:
   - Demographic parity disparity > {{dp_alert_threshold}}: Slack alert to model owner and legal/compliance team
   - Equal opportunity disparity > {{eo_alert_threshold}}: same alert
   - Fairness degradation trend: if any metric worsens for 3 consecutive weeks: escalate

4. Fairness-performance tradeoff:
   - Document the explicit tradeoff between overall performance and fairness
   - If improving fairness requires accepting a performance hit: this is a product and legal decision, not just a technical one

5. Regulatory context:
   - Flag which regulations apply to this model (ECOA, FCRA, EU AI Act, GDPR)
   - Document compliance status per regulation

Return: fairness metrics implementation, monitoring pipeline, alerting configuration, and regulatory compliance checklist.