Trainers & Games
Calibration Game
How well do you know what you don't know? Rate your confidence on quant and markets facts, and get scored on how honest your probabilities are.
Calibration Game
You'll see a series of true-or-false statements about markets and quant finance. For each one, set a slider to how likely you think it is TRUE. You're scored not just on being right, but on how honest your confidence was, the single most underrated skill in trading.
What is calibration? It means your confidence matches reality, if you say you're 90% sure, you should be right about nine times in ten. Your answers are graded with the Brier score, which rewards being confident when you're right and punishes being confident when you're wrong. Always guessing 50% scores 0.25, so beating that means you actually know something. In trading, knowing the limits of your own certainty is everything.
Learn how it works
Five worked examples. Read a couple before you dive in, try to answer first, then reveal the solution.
A die averages 3.5 (intuitive, TRUE)
Statement: "A fair six-sided die averages 3.5 per roll." True or false, and what probability would a well-calibrated player set?
Show solution
TRUE. The average is (1+2+3+4+5+6) ÷ 6 = 21 ÷ 6 = 3.5. This is basic and checkable, so you can be very confident, set something like 95%.
- Brier = (0.95 − 1)² = 0.0025 (much better than a coin-flip's 0.25).
When a fact is solid and verifiable, high confidence is rewarded.
The S&P 500 weighting (counter-intuitive, FALSE)
Statement: "The S&P 500 is a price-weighted index." True or false, and what confidence should you set?
Show solution
FALSE. The S&P 500 is market-cap weighted; it's the Dow Jones that is price-weighted. Many people mix these up, so it's counter-intuitive. If you know it, set a low probability that it's true, say 10%.
- Brier = (0.10 − 0)² = 0.01.
Being confidently correct on the right side pays, but note you were near 10%, not 0%, leaving room for doubt.
Correlation above +1 (intuitive, FALSE)
Statement: "A correlation coefficient can be greater than +1." True or false, and what probability would you set?
Show solution
FALSE. A (Pearson) correlation is always between −1 and +1 by construction. This is a firm rule, so set a low true-probability, around 5%.
- Brier = (0.05 − 0)² = 0.0025.
Firm mathematical bounds are a place to be confident, pushing toward the extreme (near 0% true) is justified here.
What a call option is (intuitive, TRUE)
Statement: "Buying a call option gives you the right, not the obligation, to buy the stock." True or false, and what confidence?
Show solution
TRUE. A long call is a right to buy at the strike, never an obligation, you only exercise if it's worthwhile. Set a high confidence like 90%.
- Brier = (0.90 − 1)² = 0.01.
Contrast: it's the option seller who takes on an obligation. Knowing which side carries the duty keeps you calibrated.
Monty Hall (counter-intuitive, TRUE)
Statement: "In the Monty Hall problem, switching doors wins about 2/3 of the time." True or false, and how confident should you be?
Show solution
TRUE. Switching wins 2/3, staying wins only 1/3, famously counter-intuitive, since it feels like 50/50 once a door is opened. If you've seen the proof, set around 85–90%; if you only half-remember it, being honest at 60% is smarter than overclaiming.
- At 90%: Brier = (0.90 − 1)² = 0.01.
- At 60%: Brier = (0.60 − 1)² = 0.16, still better than guessing wrong with confidence.
Calibration means matching your number to how sure you truly are, not to how sure you wish you were.
What you'll learn
Calibration, making your stated confidence match reality. Being 90% sure should mean right 9 times in 10; this trains that discipline with a Brier score.