LLM Benchmark and Evaluation Suite AI Prompt

Design a comprehensive evaluation suite for this LLM application before production deployment.

Application: {{application}}
Key capabilities required: {{capabilities}}
Risk level: {{risk_level}}
Stakeholders: {{stakeholders}}

1. Evaluation dimensions:
   A production LLM evaluation must cover:
   - Capability: can the model perform the required tasks?
   - Accuracy / factuality: does the model produce correct outputs?
   - Safety: does the model avoid harmful outputs?
   - Robustness: does the model perform consistently across diverse inputs?
   - Latency and cost: does the model meet operational requirements?

2. Task-specific capability evaluation:
   - Create a golden test set: 200-500 examples with verified ground truth answers
   - Measure: exact match, F1, ROUGE, or human evaluation depending on the task type
   - Segment by difficulty: easy / medium / hard / adversarial

3. Standard benchmark references:
   - General reasoning: MMLU, HellaSwag, ARC, WinoGrande
   - Coding: HumanEval, MBPP, SWE-Bench
   - Math: GSM8K, MATH
   - Safety: TruthfulQA, BBQ (bias benchmark), WinoBias, ToxiGen
   - Long context: SCROLLS, LONGBENCH
   - Custom: build a domain-specific eval set from real user queries

4. Safety evaluation:
   - Refusal appropriateness: does the model correctly refuse harmful requests WITHOUT over-refusing legitimate ones?
   - Harmful content rate: % of responses containing harmful content across 1000+ adversarial prompts
   - Bias audit: test for demographic bias using equivalent prompts differing only in group identity
   - Consistency: does the model give the same answer to paraphrase of the same question?

5. LLM-as-judge meta-evaluation:
   - Use GPT-4 or Claude as an independent judge to score a sample of outputs
   - Validate the LLM judge's scores against human labels on 100 examples (inter-rater reliability)
   - LLM judges are biased toward verbose, confident-sounding responses — account for this

6. A/B evaluation protocol:
   - For each model version change: compare 500+ output pairs using LLM-as-judge
   - Report: win rate, tie rate, loss rate vs baseline
   - Minimum detectable difference: with 500 pairs at alpha = 0.05, can detect 5% difference in win rate

7. Pre-launch checklist:
   ☐ Capability eval: primary metric >= target on golden test set
   ☐ Safety eval: harmful content rate < 0.1% on adversarial prompts
   ☐ Latency: p99 < SLA on realistic load
   ☐ Regression: no capability drop vs baseline > 5%
   ☐ Bias audit: no demographic group has significantly worse outcomes
   ☐ Guardrail stack tested and validated

Return: evaluation suite design, benchmark selection, golden test set construction, safety test plan, and pre-launch checklist.