we-solve (beta)

Econ-610 · economics · Economics

Learning to sustain cooperation

LLM status Open unverified

Statement

Original subject group: Game theory and strategic interaction

Audit classification: Published research agenda. Fudenberg–Levine pp.162–165 discusses off-path learning and cooperation. The experimental question is editorial; the original formula double-counted random continuation.

Definitions and precise research target

Model and research target. In the explicitly specified repeated prisoner’s dilemma, determine how experimentally provided information about off-path punishment and forgiveness changes the correctly normalized cooperation measure $C(1)-C(0)$. Estimate its dependence on continuation $\beta$, the full monitoring kernel and elicited beliefs; fit a prespecified structural learning model and evaluate its forecasts in new monitoring regimes. This is an empirical research specification, not a universal equilibrium-existence conjecture.

There are two players, each choosing $C$ or $D$ every surviving period. Stage payoffs are $(2,2)$ at $(C,C)$, $(1,1)$ at $(D,D)$, $(0,3)$ at $(C,D)$, and $(3,0)$ at $(D,C)$. Definitions and maintained assumptions (editorial unless a source definition is named). Let $N$ be the last played period, with $\Pr(N\ge t)=\beta^{t-1}$, and suppose termination draws are independent of actions and monitoring. Define $Y_t=\mathbf1\{a_{1t}=a_{2t}=C\}$ on surviving histories and use \[C(Z)=(1-\beta)\mathbb E\!\left[\sum_{t=1}^{N}Y_t\mid Z\right]=(1-\beta)\sum_{t\ge1}\beta^{t-1}\mathbb E[Y_t\mid N\ge t,Z].\] Do not multiply observed surviving outcomes by another $\beta^{t-1}$: survival already supplies that factor. Then $0\le C(Z)\le1$. The private monitoring kernel must specify each player’s signal conditional on the complete action profile; marginal error probability $\rho$ alone does not specify joint errors. For a concrete baseline take independent action-misclassification errors conditional on actions. Treatment changes only the declared feedback kernel and is assigned by pair. A learning model specifies beliefs over opponents’ contingent strategies and update and choice rules at every private history. Incentivized belief reports require a scoring rule and error model.

Variants and assumption changes

  1. Public versus private monitoring (editorial). Randomize a common public action signal versus independent private signals with matched marginal accuracy. Estimate the cooperation contrast and test whether a fitted off-path learning model predicts it.
  2. Pure time discount (editorial). Add a discount factor $\delta$ distinct from survival. Use $(1-\beta\delta)\sum_t(\beta\delta)^{t-1}\mathbb E[Y_t\mid N\ge t,Z]$, and specify the corresponding estimator for terminated histories.

References and source audit

  1. Off-Path Play pp.162–163; Cooperation pp.163–165. Supports the stated research area; exact displayed specification is editorial. https://economics.mit.edu/sites/default/files/2022-09/Whither%20Game%20Theory%20Towards%20a%20Theory%20of%20Learning%20in%20Games.pdf
  1. Drew Fudenberg; David K. Levine. 2016. Whither Game Theory? Towards a Theory of Learning in Games. Active Learning: Learning about Off-Path Play, pp.162–163; Cooperation in Infinitely Repeated Games, pp.163–165. https://economics.mit.edu/sites/default/files/2022-09/Whither%20Game%20Theory%20Towards%20a%20Theory%20of%20Learning%20in%20Games.pdf. DOI: https://doi.org/10.1257/jep.30.4.151.

Common conventions: Source mathematical conventions

These conventions apply to each entry unless explicitly replaced.

Models and identification. A primitive model $m$ specifies all probability laws, feasible choices, information, equilibrium and measurement rules used by its target. The admissible class $\mathcal M$ is fixed independently of outcomes. If $P_O(m)$ is its observable probability law and $\tau(m)$ the target, its identified set at observable law $P$ is \[ \mathcal I_\tau(P)=\{\tau(m):m\in\mathcal M,\ P_O(m)=P\}. \] Point identification means that this set is a singleton. An empty set signals incompatibility of data and assumptions. Coordinate bounds need not describe the joint set. Bounds are sharp only if every retained target is attainable or arbitrarily approachable by compatible models. Finite bounds require explicit restrictions on unobserved quantities; unrestricted confounding, hidden wealth, extrapolation or arbitrary equilibrium selection can yield uninformative sets.

Causal interventions. A potential outcome $Y_i(p)$ is the outcome under a complete policy regime $p$, including timing, financing, information and market spillovers. The target population and units are fixed before treatment. With interference, use the complete assignment vector or a justified exposure mapping; individual no-interference notation alone cannot identify an economy-wide intervention. A difference of mean potential outcomes does not require the cross-world joint distribution. Conditioning on post-treatment migration, survival, enrollment or employment changes the estimand and needs separate justification.

Statistical statements. An estimator, confidence set, forecast or test specifies a sampling law, model class and loss/coverage criterion. Uniform $(1-\alpha)$-coverage means $\inf_{m\in\mathcal M}\Pr_m\{\tau(m)\in C_n\}\geq1-\alpha$, or an explicitly asymptotic version. The whole parameter space trivially has coverage, so informativeness requires a stated width, risk or power objective. A forecasting improvement is relative to a named benchmark, information set and evaluation population.

Equilibrium and optimization. An equilibrium comprises feasible plans, optimality, beliefs where needed, budget constraints and all market/resource clearing. The primitives or a stated benchmark define each correspondence $\mathcal E(p;m)$. Nonemptiness is assumed or proved, never inferred just from compact policy sets. A selected-equilibrium target uses a declared selection rule; a robust target quantifies over all permitted equilibria. Use suprema or infima unless attainment is established. An $\varepsilon$-optimal policy is feasible and within $\varepsilon$ of the finite optimum value. Report an empty feasible set separately.

Welfare and accounting. Normative utility, interpersonal normalization, social weights, numeraire, discounting and the use of public receipts are primitives. Behavioral utility need not coincide with the welfare criterion. Household welfare includes ownership income and rents once; adding profits again to compensating variation generally double-counts them. A monetary equivalent is defined by an explicit transfer and price convention, with monotonicity and range conditions. Average effects, mechanism contributions, transfers and welfare losses are different targets. Infinite sums require convergence and dynamic feasible plans require terminal or no-Ponzi conditions.

Source versions. A working-paper date, revision date and journal publication year are distinguished. A DOI for a journal version is not automatically the DOI of an earlier manuscript. Locators refer to the stated version and printed pages where specified. A metadata-only check does not verify a claim about a full-text conclusion. Every entry's source subsection narrows the provenance claim accordingly.

Statement licence: CC-BY-4.0 · Source file

Contributions 1

Econ-610-A1 · Exploration · LLM-generated · gpt 6 astra ultra ·

GPT 6 Astra Ultra attempt on Econ-610

Learning off-path detection and choosing credible forgiveness

Problem ID: C73-606. Status: Unresolved. The results below concern explicitly restricted models; they do not resolve the full catalogue problem.

Question and normalization

The catalogue concerns how information about off-path punishment and forgiveness changes cooperation in repeated games, with a fully specified monitoring process. Fudenberg and Levine [FL] motivate off-path learning and cooperation. We solve a public-monitoring automaton, give a confidence-based learning rule for its unknown monitoring parameters, and separately calculate the effect of correlated monitoring errors. These restricted results do not estimate subjects' behavior or claim convergence of arbitrary learning algorithms.

Stage payoffs are $(2,2)$ at $(C,C)$, $(1,1)$ at $(D,D)$, $(0,3)$ at $(C,D)$, and $(3,0)$ at $(D,C)$. The last played period $N$ satisfies ${\mathbb{P}}(N\ge t)=\beta^{t-1}$, $0<\beta<1$, independently of actions and monitoring. There is no additional time discount. The cooperation statistic is \[ C=(1-\beta){\mathbb{E}}\sum_{t=1}^N{\mathbf 1}\{a_{1t}=a_{2t}=C\}. \] Equivalently it is the discounted occupancy of mutual cooperation in the continuing game. Survival supplies the factor $\beta^{t-1}$ exactly once; it is not applied again to observed surviving rounds.

A public-monitoring game with forgiveness

Each period produces a common signal, good or bad, conditionally independent over time given actions. The complete kernel is \[ {\mathbb{P}}(\mathrm{bad}\mid C,C)=\alpha, \quad{\mathbb{P}}(\mathrm{bad}\mid D,C)={\mathbb{P}}(\mathrm{bad}\mid C,D)=d, \quad{\mathbb{P}}(\mathrm{bad}\mid D,D)=1, \] where $0<\alpha<1/2$ and $\alpha\le d\le1$. Public state $G$ prescribes mutual cooperation; state $B$ prescribes mutual defection. In $G$, a bad signal moves the next surviving round to $B$, while a good signal retains $G$. In $B$, a fresh public coin moves the next surviving round to $G$ with probability $f\in[0,1]$, regardless of actions or signals; otherwise it remains in $B$. Start in $G$.
Theorem (Exact incentives and cooperative occupancy): Let $V_G,V_B$ denote one player's expected undiscounted payoff until random termination under this automaton. Then \[ V_G-V_B=\frac1{1-\beta+\beta(\alpha+f)}. \] The automaton is a perfect public equilibrium, and remains optimal against deviations using additional private history, if and only if \[ f\le f_{\max}:=d-2\alpha-\frac{1-\beta}{\beta}. \tag{1} \] If the right side is negative, no forgiveness rate in this automaton family supports cooperation. Its correctly normalized cooperation is \[ C_G(f)=\frac{1-\beta+\beta f}{1-\beta+\beta(\alpha+f)}. \tag{2} \]
Proof: The continuation equations are \[ V_G=2+\beta[(1-\alpha)V_G+\alpha V_B],\qquad V_B=1+\beta[fV_G+(1-f)V_B]. \] Subtract them to obtain the value difference. In $G$, a unilateral deviation to $D$ gains one now and changes the bad-signal probability from $\alpha$ to $d$. It is unprofitable exactly when $1\le\beta(d-\alpha)(V_G-V_B)$. Substitution and rearrangement give (1). In $B$, deviating from $D$ to $C$ loses one immediately while leaving the public reset rule unchanged, so it is strictly unprofitable.

These inequalities suffice for arbitrary multi-period deviations. Given the opponent's public strategy, each possible own action has current payoff plus $\beta$ times the prescribed continuation value at most the current state's $V$. Iterate that inequality along any deviating strategy for a finite number of rounds. The bounded continuation values make the remaining term vanish as the horizon increases. Thus no deviation, including one conditioning on additional private information, improves value. Conversely, violating (1) supplies the profitable one-period deviation just calculated.

For occupancy, let $C_G,C_B$ be normalized discounted time in state $G$ starting from the two states. They satisfy $C_G=(1-\beta)+\beta[(1-\alpha)C_G+\alpha C_B]$ and $C_B=\beta[fC_G+(1-f)C_B]$. Solving gives (2), and $C_B=\beta f/[1-\beta+\beta(\alpha+f)]$. The equivalence between geometric survival and these discounted recursions follows by conditioning on independent continuation after each round. ∎

Corollary (The most forgiving credible rule): If $f_{\max}\ge0$, the unique cooperation-maximizing equilibrium in this automaton family has $f=f_{\max}$ and \[ C_G(f_{\max})=\frac{d-2\alpha}{d-\alpha}. \tag{3} \] Existence is equivalent to $\beta\ge1/(1+d-2\alpha)$; this requires $d>2\alpha$. Formula (3) is independent of $\beta$ within this feasible region, although the selected forgiveness rate is not.
Proof: Because $\beta<1$ and $d\le1$, $f_{\max}<1$. Differentiating (2) gives $C_G'(f)=\beta^2\alpha/[1-\beta+\beta(\alpha+f)]^2>0$. Thus its maximum over the nonempty feasible interval $[0,f_{\max}]$ is unique at the upper endpoint. Substitute (1) into (2) to obtain (3). Solving $f_{\max}\ge0$ gives the stated continuation condition. Since $(1-\beta)/\beta>0$, feasibility implies $d>2\alpha$. ∎
The patience cancellation in (3) is specific to maximizing forgiveness in this two-state family from a cooperative initial state. It is not a claim that patience never affects cooperation. At a fixed $f$, or in another strategy family, the comparative static differs.

A specified off-path learning rule

Suppose $\alpha,d$ are unknown. Before play, a shared audit supplies independent signals from $n$ prescribed $(C,C)$ rounds and $n$ prescribed unilateral-deviation rounds. These are experimental calibration observations, not endogenous on-path deviations. Let $\widehat\alpha_n,\widehat d_n$ be their bad-signal frequencies. For $0<\delta<1$ define \[ r_n=\sqrt{\frac{\log(2\pi^2n^2/(3\delta))}{2n}},\quad \alpha_n^+=\min\{1,\widehat\alpha_n+r_n\},\quad d_n^-=\max\{0,\widehat d_n-r_n\}, \] \[ \widehat f_n=d_n^--2\alpha_n^+-\frac{1-\beta}{\beta}. \] The learning rule chooses the cooperative automaton with $f=\widehat f_n$ when $\widehat f_n\ge0$, and otherwise chooses permanent mutual defection. Both players use the same audit and decision rule. This is an explicit confidence-set learning model, not a Bayesian interpretation of unspecified beliefs or a behavioral claim about experimental subjects.
Theorem (Uniformly credible learning and convergence): With probability at least $1-\delta$, every policy selected at every audit size $n$ is an equilibrium of the true monitoring game. If the true margin $f_{\max}>0$, then whenever $6r_n<f_{\max}$ on this event, the cooperative automaton is selected and \[ 0\le f_{\max}-\widehat f_n\le6r_n, \] \[ 0\le C_G(f_{\max})-C_G(\widehat f_n) \le\frac{6\beta^2\alpha r_n}{(1-\beta+\beta\alpha)^2}. \tag{4} \] For every fixed true game with $f_{\max}>0$, the chosen forgiveness and cooperation converge almost surely to the optima in the corollary.
Proof: For a mean of $n$ independent Bernoulli variables, the exponential moment bound for a variable in $[0,1]$ gives ${\mathbb{P}}(|\widehat p-p|>r)\le2e^{-2nr^2}$. At the specified $r_n$, summing this probability over the two audit streams and all $n\ge1$ gives $\sum_n6\delta/(\pi^2n^2)=\delta$. On the simultaneous event, $d_n^-\le d$ and $\alpha_n^+\ge\alpha$. Therefore $\widehat f_n\le f_{\max}$, and any nonnegative selected rate satisfies the incentive condition. Permanent defection is itself an equilibrium because defection is a strict stage best response and its continuation never changes with actions.

The same event gives $d-d_n^-\le2r_n$ and $\alpha_n^+-\alpha\le2r_n$, hence $f_{\max}-\widehat f_n\le6r_n$. If this is smaller than the true positive margin, the rate is nonnegative. Integrating the derivative from the corollary, bounded above by its value at zero, proves (4). Finally, both sample means converge almost surely: for any fixed positive tolerance the Hoeffding failure probabilities are summable, so the probability of infinitely many such failures is zero; intersect over rational tolerances. Since $r_n\to0$, $\widehat f_n\to f_{\max}>0$, eventually the cooperative choice is made, and continuity of (2) proves convergence. ∎

Observing only prescribed $(C,C)$ and $(D,D)$ play cannot identify $d$: every such history has the same distribution for all values of $d$ with fixed $\alpha$. Yet (1) depends on $d$. Two complete monitoring kernels can therefore agree on all this on-path evidence and disagree about whether the proposed cooperation is incentive compatible. The audit addresses precisely that missing off-path information. Its costs and feasibility would need to be included in an experimental implementation.

Matched marginal errors do not determine cooperation

For a separate mechanical-policy comparison, let each player start cooperative and switch permanently to defection upon the first private signal saying the opponent defected. Every opponent action is misclassified with marginal probability $\rho\in(0,1)$, independently over dates. Specify either independent error coins across players, or one common error coin that flips both players' observations. This defines the signal kernel at every action profile by applying the indicated flips to the true opponent actions. No equilibrium claim is made for these private grim policies.

Joint cooperation continues to the next date exactly when neither player receives a false defection signal while both cooperate. Once either switches, mutual cooperation can never return. Therefore the two normalized cooperation values are \[ C_{\rm independent}=\frac{1-\beta}{1-\beta(1-\rho)^2},\qquad C_{\rm common}=\frac{1-\beta}{1-\beta(1-\rho)}. \] To verify them, the probability of mutual cooperation at date $t$, conditional on survival, is respectively $[(1-\rho)^2]^{t-1}$ and $(1-\rho)^{t-1}$. Sum the geometric series with the single survival factor. The common-error value is strictly larger despite identical marginal accuracy. This calculation isolates why a lone error probability is insufficient to specify a monitoring treatment.

A population experiment must randomize the complete information or monitoring intervention by pair and measure the correctly normalized outcome. The results here neither identify participants' perceived punishment strategies from choices nor validate an unmeasured belief-report process. Those empirical learning and out-of-sample prediction tasks remain unresolved.

Completion Estimate

58%

This is a post hoc, heuristic estimate for the full catalogue target. The attempt solves credible forgiveness and normalized cooperation in a public-monitoring automaton, proves an audited learning guarantee, and demonstrates the effect of correlated monitoring errors. The experimental cooperation target still needs participants' measured beliefs and learning responses, pair-randomized information effects, and out-of-sample validation in new private or public monitoring regimes.

References

  • [FL] D. Fudenberg and D. K. Levine (2016), "Whither Game Theory? Towards a Theory of Learning in Games," Journal of Economic Perspectives 30(4), 151--170, especially pp.162--165. doi:10.1257/jep.30.4.151. The source motivates off-path learning; the automaton, audit rule, and monitoring comparisons above are fully specified restricted calculations.

Go to contribution →