Sample Representativeness Audit AI Prompt

Evaluate the representativeness of my sample and the generalizability of my findings.

Study sample: {{sample_description}}
Target population: {{target_population}}
Recruitment method: {{recruitment_method}}

1. Define the population hierarchy:
   - Target population: the full population to which you want to generalize
   - Accessible population: the population you can realistically recruit from
   - Sampling frame: the list or mechanism from which you draw participants
   - Study sample: who actually participated
   - At each level: identify who is systematically excluded and why

2. WEIRD problem assessment:
   Assess whether your sample is WEIRD (Western, Educated, Industrialized, Rich, Democratic):
   - Western: is the sample from Western countries only? Many behavioral findings do not replicate cross-culturally.
   - Educated: what is the educational distribution? Is it higher than the target population?
   - Industrialized: is the sample from urban, industrialized settings? Rural or lower-resource populations may differ.
   - Rich: what is the income distribution? Is convenience sampling overrepresenting higher-income individuals?
   - Democratic: is the sample from stable democracies? Political context affects many research constructs.
   For each dimension: how different is your sample from your target population?

3. Volunteer bias:
   - People who volunteer for research differ systematically from those who do not
   - Volunteers tend to be more educated, more conscientious, more open to new experiences
   - Assess: in what ways might your volunteers differ from the target population on theoretically relevant variables?

4. Attrition bias:
   - Who dropped out? Compare completers vs dropouts on baseline characteristics.
   - Is dropout related to treatment condition or outcome severity?
   - What does differential attrition do to the representativeness of your final analytic sample?

5. Generalizability statement:
   - Write an honest, specific generalizability statement for the paper: 'Results are most likely to generalize to [specific population]. Caution is warranted in applying findings to [different groups] because [specific reason].'

Return: population hierarchy analysis, WEIRD assessment, volunteer and attrition bias evaluation, and generalizability statement.