Saddle Point

Chapter 10 · begins on page 385

Aggregating Information

Counters on the shelves back to it on the cover
Try it yourself Proper scoring Which scores make an honest forecast the best forecast? Open the engine →

The models

Each model in the book is stated on one card: who decides, when, with what information and commitment, what they get, what they may do, and the question it answers.

Model F1 · p. 413

Which penalty functions make honesty optimal, and what do the others reward?

Model F1 · Chapter 10 · page 413

Which penalty functions make honesty optimal, and what do the others reward?

Scoring a probability; Model F1


1Decision-makers
one forecaster; the scorer is a contract, not a player
2Time
report, then outcome
3Information
the forecaster's belief ; the scorer sees only and
4Commitment
the penalty function is fixed before the report
5Payoffs
minus the penalty
6Strategies
reports

The analysis contract

(i) concept
the report minimising expected penalty under
(ii) deviations
every report on the rational grid of step , and the closed forms
(iii) beliefs
, a parameter
(iv) selection
none
(v) attainment
exact identities
In the book: Chapter 10, Aggregating Information, page 413Run this model →

Model F2 · p. 414

What does the sponsor's money buy, and when does honesty survive strategic traders?

Model F2 · Chapter 10 · page 414

What does the sponsor's money buy, and when does honesty survive strategic traders?

Quadratic market scoring rule; Model F2


1Decision-makers
a sponsor (mechanism) and sequential traders
2Time
one pass of trades, then the outcome
3Information
each trader: the current public forecast and its own signal
4Commitment
the payment rule is fixed; the sponsor underwrites it
5Payoffs
traders: their payments; the sponsor: minus their sum
6Strategies
each trader's report as a function of what it knows

The analysis contract

(i) concept
myopic best reports; one two-move manipulation analysed exactly
(ii) deviations
every report on the rational grid of step and the closed forms
(iii) beliefs
Bayes' rule from stated signals; one behavioural follower as a stated deviation
(iv) selection
none
(v) attainment
exact; the manipulation optimum in closed form
In the book: Chapter 10, Aggregating Information, page 414

Model F3 · p. 416

What is lost by averaging probabilities instead of combining evidence?

Model F3 · Chapter 10 · page 416

What is lost by averaging probabilities instead of combining evidence?

Two experts and two pools; Model F3


1Decision-makers
two experts (honest by assumption) and an aggregator
2Time
signals, reports, aggregate, outcome
3Information
conditionally independent signals; the aggregator sees only the two posteriors
4Commitment
the aggregation rule is fixed
5Payoffs
the aggregate's expected Brier score under the generating model
6Strategies
none; this is a comparison of rules

The analysis contract

(i) concept
exact expected scores over the joint distribution
(ii) deviations
none
(iii) beliefs
Bayes' rule; the independence is stated, not assumed implicitly
(iv) selection
none
(v) attainment
exact
In the book: Chapter 10, Aggregating Information, page 416

Model F5 · p. 418

What does the subsidy buy, and how do insiders and outside stakes change what the price says?

Model F5 · Chapter 10 · page 418

What does the subsidy buy, and how do insiders and outside stakes change what the price says?

An insider, an analyst and a stake in the price; Model F5


1Decision-makers
a sponsor (mechanism); an insider; an analyst; a price-setter with an outside stake; readers of the price are not players
2Time
research decision; trades in sequence, one each; outcome
3Information
the insider knows ; the analyst holds a signal of accuracy if it bought one, and sees the price; the price-setter, its belief ; , , , are common knowledge
4Commitment
the payment rule and scale are fixed before trading
5Payoffs
trading payments; minus the research cost; plus if the closing price is at least
6Strategies
buy the signal or not; reports as functions of what each trader knows

The analysis contract

(i) concept
each trader's best response, one trade each; the analyst's research decision by expected net payment
(ii) deviations
every report on the grid of step (insider, analyst) and (price-setter); the closed forms
(iii) beliefs
Bayes' rule from the stated signals; the analyst infers the insider's absence from an unmoved price
(iv) selection
ties: research; honesty
(v) attainment
exact; every quantity at , , by enumeration of all states
In the book: Chapter 10, Aggregating Information, page 418

Model F6 · p. 421

What do quotes, the price path and an audit reveal when some orders know the statement?

Model F6 · Chapter 10 · page 421

What do quotes, the price path and an audit reveal when some orders know the statement?

A statement, a dealer and an audit; Model F6


1Decision-makers
a competitive dealer; informed and uninformed traders; an auditor (a sampling procedure); the speaker is not a player
2Time
orders in sequence, one unit each; the statement; the audit
3Information
informed traders know ; the dealer sees the order directions only; all parameters are common knowledge
4Commitment
quotes are conditional expectations (zero expected profit); the sampling rate and penalties are fixed before trading
5Payoffs
trading gains at the quotes, valued at ; minus penalties when identified
6Strategies
the informed trader's decision to trade (deterrence variants); everything else is mechanical

The analysis contract

(i) concept
the dealer's zero-profit quotes; in the fixed variant an equilibrium of the entry decision against the quotes it induces
(ii) deviations
trade or not, for the potential informed trader, at every sampling rate of step up to (fixed) and (proportional)
(iii) beliefs
Bayes' rule from the order directions
(iv) selection
ties: proportional variant, no trade; fixed variant, the mixing share
(v) attainment
exact; every path of statement, order types and directions enumerated for
In the book: Chapter 10, Aggregating Information, page 421

What is proved, in words

The book follows each result with one sentence that says what it means without symbols. The formal statement is a click away; the proofs are in the book.

Proposition 10.2 · Propriety is an identity; impropriety is a policy; Model F1 · p. 414

Under the quadratic score honesty is optimal by an identity; under absolute error a forecaster with a definite leaning is paid to round its belief to the nearer certainty.

Proposition 10.2 · Chapter 10 · page 414

Propriety is an identity; impropriety is a policy; Model F1

In words Under the quadratic score honesty is optimal by an identity; under absolute error a forecaster with a definite leaning is paid to round its belief to the nearer certainty.


For every , : the Brier penalty is strictly proper, the identity is proved by expanding the squares and checked in exact arithmetic on rational pairs. The absolute-error penalty satisfies , affine in : it is minimised at for , at for , and at every when — never uniquely at an interior belief. At the honest report expects and the report expects : a forecaster evaluated by absolute error is paid to round every uneven belief to the nearer certainty. The logarithmic penalty is also strictly proper (Good 1952); it is cited, not computed, since its values are not rational.

The proof is in the book: Chapter 10, Aggregating Information, page 414

Proposition 10.4 · Telescoping subsidy and myopic honesty; Model F2 · p. 415

In a market scoring rule each trader is paid for the improvement it makes, the sponsor's cost is bounded, and a trader who will not trade again does best by reporting its belief.

Proposition 10.4 · Chapter 10 · page 415

Telescoping subsidy and myopic honesty; Model F2

In words In a market scoring rule each trader is paid for the improvement it makes, the sponsor's cost is bounded, and a trader who will not trade again does best by reporting its belief.


The forecast path is and the realised payments telescope: the sponsor's total cost is — here if the event occurs and (a gain) if not — and never exceeds , which is at most for (and in general). By the identity of Proposition 10.2 (p. 414), a trader's expected payment under belief is , maximised exactly at the honest : a one-shot trader has nothing to gain from distortion, whatever the others do.

The proof is in the book: Chapter 10, Aggregating Information, page 415

Proposition 10.7 · The price of averaging; Model F3 · p. 416

Averaging the probabilities of two experts with conditionally independent evidence wastes three eighths of what their combined evidence is worth here: agreeing moderate opinions should combine into a stronger one, not average into the same.

Proposition 10.7 · Chapter 10 · page 416

The price of averaging; Model F3

In words Averaging the probabilities of two experts with conditionally independent evidence wastes three eighths of what their combined evidence is worth here: agreeing moderate opinions should combine into a stronger one, not average into the same.


On agreeing good signals the Bayes pool reports where the average reports ; on disagreement both report . The exact expected Brier scores are for one expert, for the average, and for the Bayes pool: averaging two experts with conditionally independent signals recovers five eighths of the improvement that their combined evidence allows and forgoes the other three eighths, because two agreeing moderate probabilities are stronger evidence than either alone, and an average cannot say so. The comparison assumes the stated independence; perfectly correlated experts would make the Bayes pool overconfident, and the rule, not the slogan, carries the assumption.

The proof is in the book: Chapter 10, Aggregating Information, page 416

Proposition 10.10 · Murphy's decomposition, exactly; Model F4 · p. 417

A forecaster can be perfectly calibrated and know nothing, and a slightly miscalibrated one can still have skill: calibration is neither sufficient nor necessary for a better score, and the decomposition shows where skill comes from — resolution, net of miscalibration.

Proposition 10.10 · Chapter 10 · page 417

Murphy's decomposition, exactly; Model F4

In words A forecaster can be perfectly calibrated and know nothing, and a slightly miscalibrated one can still have skill: calibration is neither sufficient nor necessary for a better score, and the decomposition shows where skill comes from — resolution, net of miscalibration.


The identity holds exactly on the table: forecaster A scores ; the climatology forecaster scores . Climatology is perfectly calibrated — in every group the issued probability equals the observed frequency — and has zero resolution: it never distinguishes one event from another. Forecaster A is slightly miscalibrated and much more resolving, and wins. Calibration is a virtue and an empty one on its own; skill lives in resolution, net of miscalibration. For the Brier score the three parts add with these signs; the statement is about this decomposition and this table, not about every proper score.

The proof is in the book: Chapter 10, Aggregating Information, page 417

Proposition 10.12 · The subsidy buys resolution; insiders can crowd it out; stakes bunch the price; Model F5 · p. 419

A subsidised market whose closing price is calibrated pays exactly for how sharply that price separates outcomes, and a miscalibrated price hands part of the payment back; an insider adds that sharpness and collects the subsidy, but by taking the analyst's reward it can make the price less informative while leaving it perfectly calibrated; and a trader with a stake in the price crossing a line rounds nearby beliefs onto the line, which only a payment scale growing like the inverse square of the distance prevents.

Proposition 10.12 · Chapter 10 · page 419

The subsidy buys resolution; insiders can crowd it out; stakes bunch the price; Model F5

In words A subsidised market whose closing price is calibrated pays exactly for how sharply that price separates outcomes, and a miscalibrated price hands part of the payment back; an insider adds that sharpness and collects the subsidy, but by taking the analyst's reward it can make the price less informative while leaving it perfectly calibrated; and a trader with a stake in the price crossing a line rounds nearby beliefs onto the line, which only a payment scale growing like the inverse square of the distance prevents.


(a) For any finite joint law of the closing price and , the sponsor's expected payment is

(times in general): the subsidy pays for resolution and is refunded by miscalibration.

(b) In the insider variant every report is honest, the insider moves the price to , and the analyst's signal is worth when the price is unmoved, where is its posterior. The analyst buys the signal if and only if , that is . At every the closing price is calibrated, ; its expected Brier score is

the sponsor's expected cost equals the resolution, which is minus that score; the insider's expected profit is ; the analyst's expected net profit is while it researches and after.

At the Brier score jumps from to the limit and returns to only at : on the price is less informative than at , although its Brier score still falls as rises.

(c) In the stake variant, a price-setter with reports if and otherwise. For , honesty at every belief at least below therefore holds if and only if , and at scale every belief in is reported as . In the base values the band is (a belief of gains by reporting ), and for ; since the sponsor's payment never exceeds (Proposition 10.4 (p. 415)), honesty at distance is bought with a worst-case liability of when .

The proof is in the book: Chapter 10, Aggregating Information, page 419

Proposition 10.15 · Quotes, spread and who pays; Model F6 · p. 422

When a quarter of the orders know what a statement will say, a dealer who only breaks even must quote a spread of a quarter, and every unit the informed gain is lost by someone who traded without knowing; the less the statement is believed, or the more predictable it is, the less there is to gain.

Proposition 10.15 · Chapter 10 · page 422

Quotes, spread and who pays; Model F6

In words When a quarter of the orders know what a statement will say, a dealer who only breaks even must quote a spread of a quarter, and every unit the informed gain is lost by someone who traded without knowing; the less the statement is believed, or the more predictable it is, the less there is to gain.


(a) At the first quotes are , so the spread is ; an informed order gains and an uninformed order loses in expectation; the dealer's expected profit is zero, so the informed traders' total gain equals the uninformed traders' total loss. At base values the quotes are and , the spread , the gain and the loss ; at the spread is , the gain and the loss .

(b) At and any prior , an informed order gains in expectation. At and the quotes are and and the gain is .

The proof is in the book: Chapter 10, Aggregating Information, page 422

Proposition 10.16 · The run-up is the resolution of the pre-statement price; Model F6 · p. 423

The part of a statement's move that happens before the statement is exactly the part of the uncertainty that the order flow has already resolved, so a quarter of informed orders can leave a run-up of a sixteenth; measured against the outcome the run-up shrinks as the statement is believed less, while measured against the statement's own move it does not; and a price that is perfectly calibrated before the statement says nothing about how much of the order flow knew it.

Proposition 10.16 · Chapter 10 · page 423

The run-up is the resolution of the pre-statement price; Model F6

In words The part of a statement's move that happens before the statement is exactly the part of the uncertainty that the order flow has already resolved, so a quarter of informed orders can leave a run-up of a sixteenth; measured against the outcome the run-up shrinks as the statement is believed less, while measured against the statement's own move it does not; and a price that is perfectly calibrated before the statement says nothing about how much of the order flow knew it.


For , every and every , the pre-statement price is calibrated, so that , and

At and the run-up share is ; at base values it is for and for . At , for every , the statement-conditioned run-up does not depend on and equals the share at , while the run-up share itself is and, for , falls as the statement is believed less: at , and , and , so while ; at the two coincide, and at , and is undefined. The informed traders' expected total gain over a window is for and for ; an informed second order gains , less than the first order's , because the price has already learned.

The proof is in the book: Chapter 10, Aggregating Information, page 423

Proposition 10.17 · What a sampled audit and a flag can detect; Model F6 · p. 424

An audit that samples one order in ten will usually miss informed trading in a short window, and an alert system that is accurate order by order, but flags some innocent orders, produces mostly false alarms when informed orders are rare enough — rarer than ; a system that never flags the innocent is the exception.

Proposition 10.17 · Chapter 10 · page 424

What a sampled audit and a flag can detect; Model F6

In words An audit that samples one order in ten will usually miss informed trading in a short window, and an alert system that is accurate order by order, but flags some innocent orders, produces mostly false alarms when informed orders are rare enough — rarer than ; a system that never flags the innocent is the exception.


(a) The probability that an audit identifies at least one informed order among orders is . At base values it is per order, for and for (truncated); on the grid of step the smallest sampling rate giving at least over twenty orders is , which gives .

(b) The probability that a flagged order is informed is : at base values, and at , where of all orders are flagged.

The proof is in the book: Chapter 10, Aggregating Information, page 424

Proposition 10.18 · Deterrence, and what the enforcement record shows; Model F6 · p. 424

A penalty that grows with the gain deters trading on any positive gain at a sampling rate that no market condition changes; a fixed penalty works through the spread, and the number of cases it produces need not grow with surveillance: at the base parameters it first rises and then falls as sampling tightens, and for a small enough penalty it keeps rising to full sampling — so the same count can mean twice as much informed trading under weaker surveillance.

Proposition 10.18 · Chapter 10 · page 424

Deterrence, and what the enforcement record shows; Model F6

In words A penalty that grows with the gain deters trading on any positive gain at a sampling rate that no market condition changes; a fixed penalty works through the spread, and the number of cases it produces need not grow with surveillance: at the base parameters it first rises and then falls as sampling tightens, and for a small enough penalty it keeps rising to full sampling — so the same count can mean twice as much informed trading under weaker surveillance.


(a) In the proportional variant the informed trader's expected net gain is : for every positive gain it trades if and only if , a threshold that no other parameter of the one-slot contract moves, while at the tie rule selects no trade at every sampling rate; at the threshold is .

(b) In the fixed variant the equilibrium informed share is if , if , and if ; the spread equals and an uninformed order loses . For and , expected identifications per order, , are largest at : when the inner expression exceeds the identifications still increase at and the maximum sits at the boundary; at no informed order exists and identifications vanish at every sampling rate. At base values the kink is at , where identifications peak at ; trading is fully deterred from ; at and identifications are both while is and (Table 10.3 (p. 409)).

The proof is in the book: Chapter 10, Aggregating Information, page 424

Worked cases

A parallax in history

1904–1924 · Chapter 10 · p. 390

Distant observations, one question

Then

1904–24: Walker builds regression forecasts of India's monsoon from distant predictors.

Now

The WMO's ENSO Update combines expert and model judgements; shared ancestry limits their evidence (Proposition 10.7 (p. 416)).

1904–1924 · Chapter 10 · page 390

Distant observations, one question

Then

1904–24: Walker builds regression forecasts of India's monsoon from distant predictors.

Now

The WMO's ENSO Update combines expert and model judgements; shared ancestry limits their evidence (Proposition 10.7 (p. 416)).


What happened

The India Meteorological Department's history of its science records that seasonal forecasting of the monsoon began after the famines of the 1860s and 1870s, with a first operational forecast in 1886 by Blanford from the inverse relation between Himalayan snow and the following monsoon; that Gilbert Walker became Director General in 1904 and began a systematic search for objective predictors, from a paper of 1908, issuing the first regression-based forecast in 1909; that his regression models were used for north-west and peninsular India from 1924 to 1987; and that in the course of that search he identified three large-scale seesaws in pressure — the Southern Oscillation, between the sea-level pressures of Tahiti and Darwin, the North Atlantic Oscillation and the North Pacific Oscillation. The same history records that the Southern Oscillation was linked to the warming of the eastern tropical Pacific by Bjerknes in the 1960s, and that the coupled phenomenon was then named ENSO D [India Meteorological Department n.d.]. Walker's memoir of 1923 speaks of “a swaying of pressure on a big scale backwards and forwards between the Pacific Ocean and the Indian Ocean”; the three names follow in 1924 D [Katz 2002, pp. 99–101, quoting the memoirs].

The model

Walker's predictors fill field 3 of Model F3's card — what each forecaster observes — with a fact the model takes as given: the information relevant to a local decision may lie thousands of kilometres away, and whether two observations are two pieces of evidence or one depends on a dependence structure that must be known before they are combined R.

What could be known

In Walker's time the information set was bounded by the stations that existed and by what could be brought to one desk; a predictor was a correlation found by hand in a few decades of tables. Today the information set is far broader: station and satellite observations can be combined with outputs from model ensembles within hours. What has not changed is the arithmetic of Proposition 10.7 (p. 416) and Exercise 10.1 (p. 411): two outputs trained on the same record are closer to two summaries of one witness than to two witnesses, and a long list of agreeing forecasts hides shared ancestry more easily than a short one. Broader access expands the information set; dependence determines how much information it contains, and the dependence assumption belongs on the model card.

Where the parallel stops

The account rests on the Department's history and a study quoting Walker's memoirs; nothing here claims that his forecasts used a model like F3 or that present ensembles share his limitations.

In the book: Chapter 10, page 390All sixteen parallaxes on the timeline →

The people

Chapter 10 · Aggregating Information

Glenn W. Brier

1913 – 1998

Statistician at the United States Weather Bureau in Washington, where for many years he headed meteorological statistics (in 1958 he was chief of the Statistics Section in its Office of Meteorological Research); later he was associated with Colorado State University. He worked on the verification and evaluation of forecasts. His 1950 note introduced the quadratic score for probability forecasts that now carries his name; it is summed over all forecast categories, so that the binary score in this chapter is half of his.

The life, in moments →

Main works

  • “Verification of forecasts expressed in terms of probability” (Monthly Weather Review 78, 1950)

Chapter 10 · Aggregating Information

Leonard J. Savage

1917 – 1971

Statistician whose The Foundations of Statistics built the modern theory of subjective probability and personal decision on axioms of coherent preference. He taught at the University of Chicago (1946–60), the University of Michigan (1960–64) and Yale (1964–71). His paper on elicitation, published posthumously in December 1971, characterised the scoring rules under which announcing one's true probability is optimal — the propriety theory this chapter instantiates.

The life, in moments →

Main works

  • The Foundations of Statistics (1954)
  • “Elicitation of personal probabilities and expectations” (Journal of the American Statistical Association 66, 1971)

Chapter 10 · Aggregating Information

Yuri I. Zhuravlev

1935 – 2022

Mathematician, a Moscow State University graduate of 1957, who worked in Novosibirsk (1959–69) and from 1969 at the Computing Centre of the Soviet and then Russian Academy of Sciences; corresponding member of the Academy from 1984 and full member from 1992; professor at Moscow State University, where he held the chair of mathematical methods of forecasting from 1997 to 2022; working in discrete mathematics and pattern recognition. His algebraic approach to recognition treated whole families of imperfect classification algorithms as objects on which operations are defined, proving that suitable closures of such families contain correct algorithms — an algebraic perspective, developed in the late 1970s, that invites comparison with modern ensemble methods.

The life, in moments →

Main works

  • “Algebras over sets of incorrect (heuristic) algorithms” (DAN SSSR 235(4), 1977)
  • “On an algebraic approach to problems of recognition or classification” (Problemy kibernetiki 33, 1978)