Tag: ab-testing
Interview Questions
- Why you run an A/B test for whole weeks, not "until it's ready"
- How big does the A/B test need to be?
- One guardrail metric out of twenty went red
- The test isn't significant, can we just run it longer?
- A two-day test came back flat, is the feature useless?
- The best of fifty backtests looks amazing
- Why you can't stop an A/B test when it "hits significance"
- Confidence intervals you're allowed to peek at anytime
- Testing an A/B/C experiment the right way
- Did the new checkout button actually lift conversions?
- Cutting A/B variance in half using yesterday's data
- When treating some users secretly changes the others
- Ten metrics on the dashboard, one lit up green
- The feature that wins in week one and fades by week four
- One metric to decide, several to keep you honest
- Randomize by user or by session? It changes the math
- The standard error trap in "clicks per session"
- Your 50/50 split came out 50.2/49.3, is that a problem?
- Blocking on a strong predictor to sharpen your test
- The overall test failed, but it "works for left-handed users"
- A 40% lift appeared overnight. Be suspicious, not thrilled