Multiple Testing in Finance AI Prompt

Address the multiple testing problem in this quantitative research context.

Research context: {{context}} (number of strategies tested, signals screened, parameters optimized)
Number of tests performed: {{n_tests}}

1. The multiple testing problem in finance:
   - With 100 independent tests at α = 0.05, expect 5 false positives by chance alone
   - Finance researchers test thousands of factors, strategies, and parameter combinations
   - Harvey, Liu, and Zhu (2016): with 315 published factors by 2012, the minimum t-statistic needed for significance should be 3.0, not 2.0
   - Most published factor premiums may be false discoveries

2. Family-wise error rate (FWER) corrections:
   Controls the probability of ANY false positive:

   Bonferroni correction:
   - α_adjusted = α / n_tests
   - For 100 tests at α = 0.05: α_adjusted = 0.0005 (t-stat ≈ 3.5 required)
   - Conservative: assumes all tests are independent (they rarely are)

   Holm-Bonferroni (step-down):
   - Less conservative than Bonferroni while still controlling FWER
   - Sort p-values: p(1) ≤ p(2) ≤ ... ≤ p(m)
   - Reject H(i) if p(i) ≤ α / (m - i + 1)

3. False Discovery Rate (FDR) corrections:
   Controls the expected proportion of false positives among rejections:
   More powerful than FWER methods when many tests are truly non-null.

   Benjamini-Hochberg (BH) procedure:
   - Sort p-values: p(1) ≤ p(2) ≤ ... ≤ p(m)
   - Find largest k such that p(k) ≤ (k/m) × q (target FDR = q, e.g. q = 0.10)
   - Reject all H(1) through H(k)
   - Appropriate when you have many tests and can tolerate some false positives

   Storey's q-value:
   - Adaptive FDR: estimates the proportion of true null hypotheses π₀ and adjusts
   - More powerful than BH when many tests are truly non-null

4. Bootstrap-based multiple testing:
   - White's Reality Check: tests whether the best-performing strategy outperforms after accounting for selection
   - Romano-Wolf stepdown procedure: controls FWER while being less conservative than Bonferroni
   - Procedure: permute returns to create null distribution of max performance statistics

5. Adjusting t-statistics for multiple comparisons:
   Following Harvey et al. (2016):
   - Minimum t-statistic for significance given M prior tests: t_min ≈ sqrt(log(M/2) × 2 × (1 + log(1/q)))
   - For M = 100, q = 0.05: t_min ≈ 3.0
   - For M = 1000, q = 0.05: t_min ≈ 3.5

6. Practical recommendations:
   - Pre-specify tests before looking at data
   - Report all tests performed, not just significant ones
   - Apply BH or Romano-Wolf correction to all reported results
   - Require t > 3.0 as a baseline for any factor claiming to be new

Return: adjusted p-values under each correction method, required t-statistics for significance, and multiple testing corrections applied to my specific results.