Engine · Chapter 10, Aggregating Information

Proper scoring

Which scores make an honest forecast the best forecast?

The forecaster

Belief q (fortieths) 16 q = 2/5

Report p (fortieths) 16 p = 2/5

The book's scenario

Brier penalty $(p - X)^{2}$

$\frac{6}{25}$

expected under your belief; the honest report expects $\frac{6}{25}$ , the least possible. You are honest.

Absolute-error penalty $∣ p - X ∣$

$\frac{12}{25}$

the honest report expects $\frac{12}{25}$ ; the best report is 0, expecting $\frac{2}{5}$ . You are paid to round your belief to the nearer certainty.

D The book certifies, at q = 2/5: honest absolute-error penalty $\frac{12}{25}$ , report 0 expects $\frac{2}{5}$ ; both curves on the grid of step 1/40. Agreement, exactly.

Brier: a parabola with its bottom exactly at the belief, by the identity

E_{q} [(p - X)^{2}] = (p - q)^{2} + q (1 - q)

· absolute error: a straight line,

E_{q} ∣ p - X ∣ = q + p (1 - 2 q)

, minimised at 0 when

q < \frac{1}{2}

, at 1 when

q > \frac{1}{2}

, and by every report when

q = \frac{1}{2}

The identity, at your numbers

$q (p - 1)^{2} + (1 - q) p^{2} = \frac{6}{25} = (p - q)^{2} + q (1 - q) = \frac{6}{25}$

$q + p (1 - 2 q) = \frac{12}{25}$

The model it runs

Model F1 · p. 413

Which penalty functions make honesty optimal, and what do the others reward?

1Decision-makers: one forecaster; the scorer is a contract, not a player
2Time: report, then outcome
3Information: the forecaster's belief $q$ ; the scorer sees only $p$ and $X$
4Commitment: the penalty function is fixed before the report
5Payoffs: minus the penalty
6Strategies: reports $p \in [0, 1]$

Analysis contract

(i) concept: the report minimising expected penalty under $q$
(ii) deviations: every report on the rational grid of step $\frac{1}{40}$ , and the closed forms
(iii) beliefs: $q$ , a parameter
(iv) selection: none
(v) attainment: exact identities