we-solve (beta)

Econ-608 · economics · Economics

Laboratory-to-field transfer

LLM status Open unverified

Statement

Original subject group: Behavioral and experimental economics

Audit classification: Published research agenda. Levitt–List 2007 explicitly raises generalizability and calls for models connecting settings. The potential-outcome transport formula and sensitivity classes are editorial.

Definitions and precise research target

Model and research target. Let $S\in\{L,F\}$ indicate a laboratory sample or a target field population. For a specified intervention $D\in\{0,1\}$, let $Y_i(d,S)$ denote individual $i$'s potential outcome under treatment $d$ in setting $S$. Let $X_i$ collect participant characteristics and $C_S$ collect contextual features, including scrutiny, stakes, framing, and institutional rules. The field target and a candidate laboratory transport formula are \[\tau_F=\mathbb E_F[Y_i(1,F)-Y_i(0,F)],\qquad \widetilde\tau_F=\int\mathbb E[Y_i(1,L)-Y_i(0,L)\mid X_i=x]\,dP_F(x).\] The formula requires overlap in $X$, internal identification of the laboratory contrast, and invariance of conditional treatment effects across contexts. It therefore does not follow merely from random assignment inside the laboratory. Determine which observed contextual moderators permit a credible extended transport model $\tau(x,C)$, and quantify errors when context or selection variables are unmeasured. Use deliberately varied laboratory settings and independent field comparisons to evaluate transported predictions, with the treatment and outcome kept conceptually comparable. Distinguish differences in effect magnitudes from different outcome measurement or different versions of the intervention.

Definitions and maintained assumptions (editorial unless a source definition is named). Use the same conceptual intervention and outcome units in both settings. $Y(d,s)$ is a potential outcome under treatment version $d$ and setting $s$; consistency gives observed $Y=Y(D,S)$. Lab assignment satisfies $D\perp(Y(0,L),Y(1,L))\mid X,S=L$, with $0<\Pr(D=1\mid X,S=L)<1$. Transport additionally requires $P_F\ll P_L$ on covariates and equality of conditional mean effects $\tau_F(x)=\tau_L(x)$ almost everywhere under $P_F$. Then the displayed formula identifies $\tau_F$ from lab outcomes and field covariates. This identification implication is established by these assumptions and is not itself an unresolved theorem. If $|\tau_F(x)-\tau_L(x)|\le\Gamma(x)$, the transport bias is bounded by $\int\Gamma(x)dP_F(x)$; the research problem is credible calibration of $\Gamma$ and moderator selection. A context variable constant within one setting cannot be disentangled from setting without additional varied environments.

Variants and assumption changes

  1. Conditional transport (editorial). Collect multiple contexts with overlapping scrutiny, stakes and participant covariates. Specify an invariance model $\tau(x,C)$ and validate its prediction in an independent field setting.
  2. Sensitivity (editorial). Bound unmeasured context differences by a justified $\Gamma(x)$ and incomplete covariate overlap by bounded outcomes. Report the resulting sharp or conservative effect interval instead of extrapolating a point estimate.

References and source audit

  1. Introduction pp.153–154; discussion pp.170–171. Supports the stated research area; exact displayed specification is editorial. https://bpb-us-w2.wpmucdn.com/voices.uchicago.edu/dist/f/1276/files/2018/09/27-101-25264884-1gx76fi.pdf
  1. Steven D. Levitt; John A. List. 2007. What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?. Introduction, pp. 153–154; Stakes, p. 165; Conclusions, pp. 170–171 (PDF pp. 1–2, 13, 18–19).. https://bpb-us-w2.wpmucdn.com/voices.uchicago.edu/dist/f/1276/files/2018/09/27-101-25264884-1gx76fi.pdf. DOI: https://doi.org/10.1257/jep.21.2.153.

Common conventions: Source mathematical conventions

These conventions apply to each entry unless explicitly replaced.

Models and identification. A primitive model $m$ specifies all probability laws, feasible choices, information, equilibrium and measurement rules used by its target. The admissible class $\mathcal M$ is fixed independently of outcomes. If $P_O(m)$ is its observable probability law and $\tau(m)$ the target, its identified set at observable law $P$ is \[ \mathcal I_\tau(P)=\{\tau(m):m\in\mathcal M,\ P_O(m)=P\}. \] Point identification means that this set is a singleton. An empty set signals incompatibility of data and assumptions. Coordinate bounds need not describe the joint set. Bounds are sharp only if every retained target is attainable or arbitrarily approachable by compatible models. Finite bounds require explicit restrictions on unobserved quantities; unrestricted confounding, hidden wealth, extrapolation or arbitrary equilibrium selection can yield uninformative sets.

Causal interventions. A potential outcome $Y_i(p)$ is the outcome under a complete policy regime $p$, including timing, financing, information and market spillovers. The target population and units are fixed before treatment. With interference, use the complete assignment vector or a justified exposure mapping; individual no-interference notation alone cannot identify an economy-wide intervention. A difference of mean potential outcomes does not require the cross-world joint distribution. Conditioning on post-treatment migration, survival, enrollment or employment changes the estimand and needs separate justification.

Statistical statements. An estimator, confidence set, forecast or test specifies a sampling law, model class and loss/coverage criterion. Uniform $(1-\alpha)$-coverage means $\inf_{m\in\mathcal M}\Pr_m\{\tau(m)\in C_n\}\geq1-\alpha$, or an explicitly asymptotic version. The whole parameter space trivially has coverage, so informativeness requires a stated width, risk or power objective. A forecasting improvement is relative to a named benchmark, information set and evaluation population.

Equilibrium and optimization. An equilibrium comprises feasible plans, optimality, beliefs where needed, budget constraints and all market/resource clearing. The primitives or a stated benchmark define each correspondence $\mathcal E(p;m)$. Nonemptiness is assumed or proved, never inferred just from compact policy sets. A selected-equilibrium target uses a declared selection rule; a robust target quantifies over all permitted equilibria. Use suprema or infima unless attainment is established. An $\varepsilon$-optimal policy is feasible and within $\varepsilon$ of the finite optimum value. Report an empty feasible set separately.

Welfare and accounting. Normative utility, interpersonal normalization, social weights, numeraire, discounting and the use of public receipts are primitives. Behavioral utility need not coincide with the welfare criterion. Household welfare includes ownership income and rents once; adding profits again to compensating variation generally double-counts them. A monetary equivalent is defined by an explicit transfer and price convention, with monotonicity and range conditions. Average effects, mechanism contributions, transfers and welfare losses are different targets. Infinite sums require convergence and dynamic feasible plans require terminal or no-Ponzi conditions.

Source versions. A working-paper date, revision date and journal publication year are distinguished. A DOI for a journal version is not automatically the DOI of an earlier manuscript. Locators refer to the stated version and printed pages where specified. A metadata-only check does not verify a claim about a full-text conclusion. Every entry's source subsection narrows the provenance claim accordingly.

Statement licence: CC-BY-4.0 · Source file

Contributions 1

Econ-608-A1 · Exploration · LLM-generated · gpt 6 astra ultra ·

GPT 6 Astra Ultra attempt on Econ-608

Laboratory transport through context designs: identification and optimal extrapolation

Problem ID: C90-604. Status: Unresolved. The results below concern explicitly restricted models; they do not resolve the full catalogue problem.

Question and a multicontext experiment

Randomization inside a laboratory does not establish external validity. The catalogue asks how contextual moderators and deliberate variation across environments can support field predictions. Levitt and List [LL] motivate differences in scrutiny, stakes, framing, and selection. We solve a finite context-design problem, quantify the cost of extrapolation, and separate a context restriction from ordinary covariate overlap.

Fix one baseline participant stratum and a treatment with the same conceptual meaning and outcome units in every setting. There are $K$ laboratory environments with measured contexts $c_j\in{\mathbb{R}}^d$, and a target field context $c_*$. Put $z_j=(1,c_j^\top)^\top$ and $z_*=(1,c_*^\top)^\top$, and let $Z$ have rows $z_j^\top$. Within each laboratory, randomized treatment identifies its conditional mean effect $\tau_j$. We posit \[ \tau_j=z_j^\top\theta+e_j,\qquad \tau_*=z_*^\top\theta+e_*,\qquad \theta\in{\mathbb{R}}^{d+1}. \tag{1} \] The departures $e_j$ represent unmeasured environment features and may differ even when measured contexts coincide. For the statistical calculations assume independent experimental contrast estimates \[ T_j\sim N(\tau_j,\sigma_j^2/n_j),\qquad\sigma_j>0,\quad n_j>0. \tag{2} \] This is an explicit Gaussian contrast experiment with known variances, not an automatic finite-sample approximation for every randomized trial. No bounded-outcome restriction is used until the separate final section.

The rank condition supplied by varied contexts

Theorem (Exact affine transport): Under exact invariance $e_j=e_*=0$ and exact knowledge of laboratory effects, the field effect is identified if and only if $z_*$ lies in the row space of $Z$. Equivalently, there are weights $\lambda\in{\mathbb{R}}^K$ with \[ Z^\top\lambda=z_*,\qquad \tau_*=\sum_j\lambda_j\tau_j. \tag{3} \] If this condition fails, the field effect is unbounded above and below among models having the same laboratory effects.
Proof: If (3) holds, substitute $\tau=Z\theta$ to obtain the formula, proving identification. Conversely, if $z_*$ is outside the row space, its orthogonal projection onto the null space of $Z$ is a nonzero vector $v$ with $z_*^\top v\ne0$. All coefficient vectors $\theta+tv$, $t\in{\mathbb{R}}$, produce the same laboratory effects, while their field effects differ by $t z_*^\top v$. This proves both nonidentification and unboundedness. The row-space equivalence follows from elementary orthogonal decomposition in finite dimensions. ∎
In particular, one laboratory where a context variable is constant cannot identify its moderation slope merely by increasing the number of participants. More settings must supply independent variation, or that slope must be restricted using other evidence. Being outside the laboratories' convex hull does not itself prevent affine identification, but it forces extrapolating weights and increases sensitivity to misspecification.

Variance and exact worst-case linear risk

Let $\Omega=\operatorname{diag}(\sigma_j^2/n_j)$. For feasible weights in (3), estimate the field effect by $\widehat\tau_*=\lambda^\top T$.
Theorem (Efficient affine prediction and bounded departures): If $Z$ has full column rank and invariance is exact, the unique minimum-variance linear unbiased estimator uses \[ \lambda_0=\Omega^{-1}Z(Z^\top\Omega^{-1}Z)^{-1}z_*, \qquad V_0=z_*^\top(Z^\top\Omega^{-1}Z)^{-1}z_*. \] More generally, suppose $|e_j|\le\gamma_j$ and $|e_*|\le\gamma_*$, with departures unrestricted across environments within those bounds. For any weights satisfying (3), set \[ V(\lambda)=\sum_j\lambda_j^2\sigma_j^2/n_j, \qquad B(\lambda)=\gamma_*+\sum_j|\lambda_j|\gamma_j. \] Its exact worst-case mean squared error over (1) is \[ V(\lambda)+B(\lambda)^2. \tag{4} \] For $0<\alpha<1$, write $z_q$ for the standard normal $q$-quantile. An honest $1-\alpha$ interval is \[ \lambda^\top T\ \pm\left[B(\lambda)+z_{1-\alpha/2}\sqrt{V(\lambda)}\right]. \tag{5} \] Minimizing (4), or the radius in (5), over (3) is a convex optimization problem. No claim is made that this linear class is optimal among all nonlinear procedures.
Proof: The stated $\lambda_0$ satisfies (3). Any other feasible weight vector is $\lambda_0+v$ with $Z^\top v=0$. Since $\Omega\lambda_0$ belongs to the column space of $Z$, the cross term $v^\top\Omega\lambda_0$ vanishes. Thus its variance equals $V_0+v^\top\Omega v$, proving optimality and uniqueness because $\Omega$ is positive definite.

Under departures the estimation error is a centered normal with variance $V(\lambda)$ plus bias $\sum_j\lambda_j e_j-e_*$. Its absolute bias is at most $B(\lambda)$, and the bound is attained by taking $e_j=\gamma_j\operatorname{sign}(\lambda_j)$ and $e_*=-\gamma_*$. Mean squared error is variance plus squared bias, giving (4). On the event that the centered normal's absolute value is at most its two-sided normal quantile, total error is at most the radius in (5), proving coverage. The weighted $\ell^1$ norm plus a constant is convex and nonnegative; its square is convex by the increasing convex square function on nonnegative arguments. Variance is a positive quadratic form, and its square root is a norm. These facts prove the optimization assertions. ∎

Weights violating (3) have infinite worst-case squared bias when $\theta$ is unrestricted, unless their discrepancy vector is zero. Thus the restriction defining this linear class is necessary for a finite uniform risk, not merely a cosmetic unbiasedness preference. The departure bounds are sensitivity inputs: the experiment does not validate them simply by producing small standard errors.

How to allocate a context experiment

Consider continuous information budgets $n_j\ge0$ with $\sum_jn_j=N$. A setting with a nonzero prediction weight must receive positive information; zero-weight settings can receive zero. Fractional $n_j$ describe the information-allocation benchmark before rounding actual sample sizes.
Proposition (Optimal information allocation for given weights): For any nonzero feasible weight vector, the minimum variance is \[ \frac{\left(\sum_j|\lambda_j|\sigma_j\right)^2}{N}, \qquad n_j^*=\frac{N|\lambda_j|\sigma_j}{\sum_k|\lambda_k|\sigma_k}. \]
Proof: Cauchy--Schwarz gives \[ \left(\sum_j|\lambda_j|\sigma_j\right)^2 \le\left(\sum_j\frac{\lambda_j^2\sigma_j^2}{n_j}\right) \left(\sum_jn_j\right). \] Equality holds exactly at the displayed allocation on the nonzero coordinates. The intercept constraint in (3) implies $\sum_j\lambda_j=1$, so the denominator is positive. Assigning zero information to zero-weight coordinates completes the proof. ∎
Theorem (Sharp design cost for extrapolating beyond all laboratory contexts): Suppose context is one-dimensional, every available laboratory context belongs to $[a,b]$, $a<b$, and the field context is $x>b$. All laboratory contrasts have common noise scale $\sigma$, and all departure bounds, including the field bound, equal $\gamma\ge0$. Among all finite choices of laboratory contexts, affine-exact weights, and continuous information allocations of total size $N$, the smallest worst-case linear squared risk is \[ \frac{\sigma^2 t^2}{N}+\gamma^2(1+t)^2, \qquad t=\frac{2x-a-b}{b-a}>1. \tag{6} \] It is attained using only the endpoint contexts, with weights \[ \lambda_a=-\frac{x-b}{b-a},\qquad \lambda_b=\frac{x-a}{b-a}, \] and sample sizes proportional to their absolute weights.
Proof: Any affine-exact weights obey $\sum\lambda_j=1$ and $\sum\lambda_jc_j=x$. The function $h(c)=(2c-a-b)/(b-a)$ has absolute value at most one on the laboratory interval. Therefore \[ t=\sum_j\lambda_jh(c_j)\le\sum_j|\lambda_j|. \] The two displayed endpoint weights satisfy the constraints and have absolute sum exactly $t$. For a given weight vector, the preceding proposition makes minimum variance $\sigma^2(\sum|\lambda_j|)^2/N$, and its worst bias is $\gamma(1+\sum|\lambda_j|)$. Both expressions increase with the absolute sum. The lower bound and endpoint construction prove (6) and attainment. ∎
Even infinite participant samples cannot remove the departure term in (6). Moving experimental contexts closer to the field can reduce both noise amplification and sensitivity, whereas repeating one distant context cannot identify a missing moderation direction.

Covariate overlap remains a separate restriction

The calculations can be applied within observed participant strata and then averaged with known field-population weights. They require adequate context variation in every stratum that contributes to the field target. A rank condition on contexts does not repair absence of a participant stratum from all experiments.

For a separate bounded-outcome model, suppose $Y(d)\in[0,1]$, the transported conditional effect is identified on a covariate set $S$, and the field mass outside it is $r$. Without restrictions on effects outside $S$, the sharp field-effect interval is \[ \left[\int_S\tau(x)\,dP_F(x)-r,\quad \int_S\tau(x)\,dP_F(x)+r\right]. \tag{7} \] The unobserved conditional contrast lies in $[-1,1]$, proving inclusion. To attain either endpoint set unobserved potential outcomes to $(Y(0),Y(1))=(1,0)$ or $(0,1)$; mixtures attain intermediate values without changing any laboratory data or field covariates. Thus (7) is sharp for that declared class. It is not added to the Gaussian model without changing its assumptions.

The original problem still requires credible moderator selection, invariance assessment on independent environments, and compatible treatment and outcome versions. The present results quantify what a specified affine or bounded-departure model licenses; they do not establish that such a model holds in a particular laboratory--field comparison.

Completion Estimate

66%

This is a post hoc, heuristic estimate for the full catalogue target. The attempt proves affine context-rank identification, exact worst-case linear prediction risk and intervals, optimal context sampling and extrapolation cost, plus sharp covariate-support bounds. Full laboratory-field transfer still needs credible moderator selection and departure calibration, independent field validation, and genuinely comparable treatment, measurement and participant contexts beyond the stipulated transport model.

References

  • [LL] S. D. Levitt and J. A. List (2007), "What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?" Journal of Economic Perspectives 21(2), 153--174. doi:10.1257/jep.21.2.153. The source provides the contextual-generalizability agenda; the rank, sensitivity, and design results above are proved for the stated experiments.

Go to contribution →