Engine · Chapter 8, Many Agents

Learning to play

Does the average of best replies become unexploitable, and what certifies it?

The game

weighted rock–paper–scissors (zero-sum) Shapley's example (not zero-sum)

R wins pay 1

P wins pay 2

S wins pay 3

Steps

Run length T 1000

Shown up to 1000

The book's runs

Exploitability of the empirical mixtures at step 1000

$\frac{2}{25}$ ≈ 0.0800

The row player’s best payoff against the column player’s average, minus what the row player’s own average guarantees against a best reply. In a finite zero-sum game Robinson’s theorem says this gap tends to zero.

D The book's checkpoint at T = 1000: $\frac{2}{25}$ . Agreement, exactly.

Average play so far

Row player	$(\frac{123}{250}, \frac{39}{250}, \frac{44}{125})$
Column player	$(\frac{123}{250}, \frac{39}{250}, \frac{44}{125})$
Last moves	R against R

Row player's average play, step 1 to 1000

Column player's average play

the gap, computed exactly at each dot · the book's certified checkpoints. Zero-sum: the gap tends to zero, as Robinson proved it must, though not necessarily at every step.

The model it runs

Model A3 · p. 333

Does the play's average become unexploitable, and what certifies the answer?

1Decision-makers: two players; the “learning algorithm” is their stated rule, not a third agent
2Time: $1 0^{4}$ steps computed, with checkpoints
3Information: each sees the full history of the opponent's actions
4Commitment: none; the rule is myopic by construction
5Payoffs: zero-sum in the first game; not in the second
6Strategies: fictitious play with the stated tie rule

Analysis contract

(i) concept: exploitability of the empirical mixtures (zero-sum); NashConv (general sum)
(ii) deviations: exact best responses; the measured gap is exact, not a lower bound
(iii) beliefs: the empirical counts, by definition of the dynamic
(iv) selection: lowest-index ties, stated
(v) attainment: exact at every checkpoint; limits cited (Robinson), non-convergence exhibited