Metacognitive Calibration Arena
Tetlock Brier Scoring: Assess subjective certainty (0%–100%) and classify failure modes before feedback.
Item #1 // Domain: probability1 of 18 items
“If you roll a fair 6-sided die 4 times, the probability of rolling AT LEAST ONE six is greater than 50%.”
Estimate Proposition Truth Probability (Credence):
Max Uncertainty: 50/50 Coin Toss (41%–60%)
50%TRUE
0% (Certain False)50% (Max Entropy)100% (Certain True)
If proposition is TRUE: Brier Penalty: 0.25
If proposition is FALSE: Brier Penalty: 0.25
Forced Pre-Feedback Error Hypothesis:
If your prediction turns out to be false, what is the most likely cognitive failure mechanism?