Tag: hypothesis-testing
Interview Questions
- "No significant effect", but the interval says nothing
- Four strategies, six t-tests, and a hidden error problem
- Is this return series white noise?
- What fraction of your "significant" results are actually true?
- What a p-value actually is
- The permutation test, inference by reshuffling labels
- Does the signal predict the market? A chi-squared test of independence
- Are trade arrivals Poisson? Goodness of fit with an estimated rate
- Is the roulette wheel fair? Goodness of fit with unequal expected counts
- Is the order router really uniform? A chi-squared test that rejects
- Is this die fair? The chi-squared goodness-of-fit test
- Read the hypothesis test straight off the interval
- What a p-value actually means
- Does a 55% win rate prove the desk has an edge?
- How to actually prove two things are the same
- Testing whether two strategies are equally volatile
- Testing a 2x2 table when the counts are tiny
- Setting the threshold when the two errors cost differently
- Proving the cheaper system is "no worse" than the old one
- One-tailed tests: legitimate shortcut or cheating?
- What a p-value is, and is not
- The pairing trick that rescues a hopeless comparison
- The "observed power" trap after a null result
- How big a sample for 90% power?
- Power of a two-sided test
- Power to detect a winning strategy
- The smallest effect a test can catch
- Statistical power, definition and levers
- How long to tell a Sharpe 1.5 from a Sharpe 1.0?
- p < 0.001 on 10 million users, and still not worth shipping
- Why not just run many t-tests? ANOVA and multiple comparisons
- With only 8 data points, why not just use 1.96?
- Two execution algos on the same names: pair the test
- Is the strategy's mean return above zero? One-sample t and the Sharpe link
- Two desks with very different vols: Welch or pooled?
- Comparing two strategies, pooled, Welch, or paired t-test?
- Is Strategy A really beating Strategy B?
- What is statistical power, and what drives it?
- Type I and type II errors in an A/B test
- False positives and negatives in a fraud model
- A significant signal, but is it real?
- Type I and type II errors, in trading terms
- Two ways to be wrong: false alarms vs missed signals
- When the "pooled" t-test quietly lies to you
- Turning a z-score into a p-value in your head