HUMANEVALPRO
Facility/Metacognitive Calibration

Metacognitive Calibration Arena

Tetlock Brier Scoring: Assess subjective certainty (0%–100%) and classify failure modes before feedback.

Item #1 // Domain: probability1 of 18 items

“If you roll a fair 6-sided die 4 times, the probability of rolling AT LEAST ONE six is greater than 50%.”

Estimate Proposition Truth Probability (Credence):
Max Uncertainty: 50/50 Coin Toss (41%–60%)
50%TRUE
0% (Certain False)50% (Max Entropy)100% (Certain True)
If proposition is TRUE: Brier Penalty: 0.25
If proposition is FALSE: Brier Penalty: 0.25
Forced Pre-Feedback Error Hypothesis:

If your prediction turns out to be false, what is the most likely cognitive failure mechanism?