Pre-Mortem Analysis AI Prompt

Conduct a pre-mortem analysis of my planned study — assume the study has failed and work backwards to identify what went wrong.

Study plan: {{study_plan}}

A pre-mortem is a prospective failure analysis. Instead of optimistically planning for success, you assume the study produced null or uninterpretable results and identify the most likely causes.

1. The failure scenarios:

   Scenario A — Null results (no effect found):
   - Was the effect truly absent? Or was the study underpowered to detect it?
   - Was the treatment inadequately implemented (low treatment fidelity)?
   - Was the outcome measured too soon (insufficient follow-up)?
   - Were participants not the right population (wrong target group, too heterogeneous)?
   - Was there contamination between conditions?

   Scenario B — Uninterpretable results (effect found but meaning is unclear):
   - Did the manipulation check fail? (We do not know if the treatment changed the intended mechanism)
   - Did demand characteristics drive results? (Participants responded as they thought we wanted)
   - Was there differential attrition? (The groups who remained are systematically different)
   - Did an unmeasured third variable explain the result?

   Scenario C — Methodological failure:
   - Recruitment shortfall: could not enroll enough participants
   - Protocol deviations: participants or experimenters did not follow the protocol
   - Instrument problems: scale showed poor psychometric properties in this sample
   - Data loss: technical failures, missing data beyond acceptable thresholds

   Scenario D — External invalidity:
   - Results replicate in the lab but not in field settings
   - Results hold for the study sample but not for the target population
   - Results are highly context-specific and do not generalize

2. For each failure scenario:
   - Probability: how likely is this scenario to occur?
   - Impact: if this happens, can the study be salvaged or must it be abandoned?
   - Prevention: what can be done NOW, before data collection, to prevent or detect this?
   - Detection: how would you know during or after the study that this scenario occurred?

3. Protocol modifications from the pre-mortem:
   - List the specific changes to the study protocol that the pre-mortem analysis suggests
   - Include: manipulation checks, fidelity monitoring, attrition tracking, pilot testing

Return: failure scenario analysis, prevention and detection measures, and a revised protocol checklist.