Avoiding Common Analysis Mistakes AI Prompt

Review my analysis for common mistakes that could lead to wrong conclusions — even when the math is correct.

My analysis: {{analysis_description}}
My conclusion: {{conclusion}}

Check for each of these traps and tell me honestly whether I have fallen into any of them:

1. Cherry-picking (looking only at data that supports your conclusion):
   - Did I look at the full dataset, or did I filter down to a subset where the pattern is clearest?
   - Did I try multiple time periods or segments and only report the one that shows the pattern?
   - The test: would the same conclusion hold if I looked at a different time period, different region, or different product?

2. Overfitting the narrative to the data:
   - Did I find a pattern and then construct an explanation for it after the fact?
   - Patterns found by looking at data often do not replicate in new data — this is the overfitting trap
   - The test: was this pattern something I predicted before looking at the data, or did I discover it by exploring?

3. Ignoring the base rate:
   - Example: '80% of our churned customers were contacted by support in their last month' sounds alarming — but if 80% of ALL customers contact support each month, this tells us nothing
   - Did I compare my finding to the base rate of the broader population?

4. Simpson's Paradox:
   - This is when a trend appears in the overall data but reverses within each subgroup
   - Example: overall conversion rate improved, but it declined in every individual region — because the mix shifted toward regions with naturally higher rates
   - Did I check whether my overall trend holds within the individual subgroups?

5. Availability bias in data selection:
   - Did I use this data because it was available, not because it is the right data?
   - Is there important data that I do not have that could change the conclusion?

For each trap: tell me if I fell into it, how serious the problem is, and what I should do to correct or acknowledge it.