Automated Retraining Pipeline AI Prompt

Build an automated model retraining pipeline triggered by monitoring signals.

Trigger conditions (any one sufficient):
1. Performance trigger: rolling 7-day AUC drops below {{performance_threshold}}
2. Drift trigger: PSI > 0.2 on any of the top-5 most important features
3. Data volume trigger: {{new_labeled_samples}} new labeled samples accumulated since last training
4. Schedule trigger: weekly retrain regardless of performance (for models in fast-changing domains)

Pipeline steps:

1. Trigger detection job (runs daily):
   - Query monitoring database for each trigger condition
   - If any condition is met: log which trigger fired, create a retraining job request
   - Deduplication: if multiple triggers fire simultaneously, create only one retraining job
   - Rate limiting: do not trigger more than {{max_retrains_per_week}} retrains per week (prevents trigger storms)

2. Data preparation:
   - Fetch training data from the feature store: last {{training_window}} days of labeled data
   - Apply the same preprocessing pipeline as the current production model
   - Validate: training set must have ≥ {{min_training_samples}} labeled samples
   - Log dataset statistics: row count, label distribution, date range, feature means

3. Training job:
   - Use the same hyperparameters as the current production model (only data is updated)
   - Allow for hyperparameter re-search if triggered by {{hp_retune_trigger}} (e.g. monthly)
   - Track the run in the experiment tracker: link to trigger event, dataset version, git commit

4. Evaluation and gate:
   - Run the performance gate against the challenger model
   - If gate passes: register in model registry as 'Staging'
   - If gate fails: alert team, keep current production model, investigate why new data did not improve the model

5. Deployment:
   - Auto-deploy to staging environment
   - Run integration tests in staging
   - If all tests pass: auto-promote to production (or require human approval for high-stakes models)

Return: trigger detection script, pipeline orchestration code (Airflow DAG or Prefect flow), and gate integration.