Valid causal inference with machine learning

Original identifier: E5 (atlas item 105). Field: Econometrics and causal inference. Classification: editorial research family with established restricted solutions; current open status not certified.

Definitions and assumptions

Observe independent source draws $W_i=(X_i,D_i,Y_i)$, $i=1,\ldots,n$, and an independent target sample $X_j^Q$, $j=1,\ldots,n_0$, with a stated ratio $n_0/n$ as both sizes grow. Target outcomes are unobserved. Treatment $D_i\in\{0,1\}$ satisfies consistency, conditional unconfoundedness and propensity $e_P(x)\in[\eta,1-\eta]$, $\eta>0$. The full target potential-outcome law is denoted $Q$, although only its covariate marginal is sampled. Define $\tau_P(x)=\mathbb E_P[Y(1)-Y(0)\mid X=x]$, and analogously $\tau_Q$. A measurable policy $\pi:\mathcal X\to\{0,1\}$ belongs to a prespecified class $\Pi_n$. With $m_{0,Q}(x)=\mathbb E_Q[Y(0)\mid X=x]$, its absolute value is $V_Q(\pi)=\mathbb E_Q[m_{0,Q}(X)+\pi(X)\tau_Q(X)]$. For supplied cost $c(x)\ge0$ and budget $B$, require the feasible class $\Pi_n^B(Q)=\{\pi\in\Pi_n:\mathbb E_Q[c(X)\pi(X)]\le B\}$ to be nonempty, for example by admitting the zero-treatment policy with $B\ge0$.

Formal research question

Construct honest inference for adaptively selected policy value and regret under target shift. Outcomes satisfy $Y(d)\in[a,b]$ for fixed finite $a<b$. On source-target overlap, impose $|\tau_Q(x)-\tau_P(x)|\le\delta(x)$, $\delta(x)\ge0$. To identify absolute values from covariate-only target data, additionally require target overlap and baseline transport $m_{0,Q}(x)=m_{0,P}(x)$, or use explicitly bounded baseline departures. Effect-transport restrictions alone constrain incremental value $V_Q^{\rm inc}(\pi)=\mathbb E_Q[\pi(X)\tau_Q(X)]$ and policy differences, not the unobserved baseline. For a declared source/target law class $\mathcal M_n$, seek \[ \liminf_n\inf_{(P,Q)\in\mathcal M_n} \Pr\{L_n(\pi)\le V_Q(\pi)\text{ for all }\pi\in\Pi_n\} \ge1-\alpha,\qquad\alpha\in(0,1). \] The learner must output $\widehat\pi\in\Pi_n^B(Q)$, or establish a stated simultaneous high-probability budget guarantee when $Q$ is estimated. Report regret on the feasibility event and account for any budget-violation probability. Characterize regret $\sup_{\pi\in\Pi_n^B(Q)}V_Q(\pi)-V_Q(\widehat\pi)$, separating sampling error, policy complexity and transport ambiguity; replace absolute value by incremental value when baseline transport is unavailable.

Interpretation and scope

A policy learned on a training sample can be evaluated on an independent sample conditional on its training realization. Reusing evaluation data for subgroup search or hyperparameter selection requires simultaneous control or another independent evaluation stage. Pointwise confidence intervals for $\tau_P(x)$ do not automatically provide simultaneous bands or valid intervals for the best discovered subgroup. An inference target can be source CATE, a subgroup average, selected-policy value or an oracle comparison; these are distinct. Outside target overlap, predictive confidence cannot replace causal assumptions. Evaluation on unseen environments requires treating the environment distribution as part of the experiment.

Source audit and status

[S1] supplies orthogonal-score inference for specified low-dimensional causal parameters, [S2] establishes causal-forest estimation and pointwise inference under conditions, and [S3] reviews ML methods for economists. These references demonstrate existing solutions and limitations. The original broad wording is narrowed here to adaptive policy evaluation with transport sensitivity; this formulation is editorial and the references alone establish no universal or current open-status claim.

References

},leftmargin=*]
1. Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey and James Robins (2018). Double/debiased machine learning for treatment and structural parameters. Econometrics Journal 21(1), C1–C68. https://doi.org/10.1111/ectj.12097. Locator: Publisher or institutional abstract and bibliographic record. Support and limits: Orthogonal scores and cross-fitting under nuisance-rate and sampling restrictions; not automatic protection against dependence or misspecification.

2. Stefan Wager and Susan Athey (2018). Estimation and Inference of Heterogeneous Treatment Effects using Random Forests. Journal of the American Statistical Association 113(523), 1228–1242. https://doi.org/10.1080/01621459.2017.1319839. Locator: Publisher or institutional abstract and bibliographic record. Support and limits: Pointwise asymptotic causal-forest inference under unconfoundedness and regularity; not arbitrary adaptive subgroup inference.

3. Susan Athey and Guido W. Imbens (2019). Machine Learning Methods That Economists Should Know About. Annual Review of Economics 11, 685–725. https://doi.org/10.1146/annurev-economics-080217-053433. Locator: Publisher or institutional abstract and bibliographic record. Support and limits: Review of prediction, causal heterogeneity and policy learning; not a blanket inference guarantee.

Common conventions: Source mathematical conventions

Unless an entry states otherwise, agent and firm sets are finite. State and action spaces are measurable, strategies and outcome maps are measurable, probability laws are specified, and expectations exist for the targets under discussion. Infinite-horizon discount factors lie in $(0,1)$ and discounted objectives are finite. Norms, tolerances and complexity input sizes must be specified for computational comparisons. Constants or primitives that appear locally are inputs, not empirical facts asserted by this catalogue.

Identification. A statistical model $m\in\mathcal M$ implies an observable law $P_m$ and target $\theta(m)$. At law $P$, its identified set is \[ \Theta(P;\mathcal M)=\{\theta(m):m\in\mathcal M,\ P_m=P\}. \] Point identification holds when this set is a singleton, or equivalently when $P_{m_1}=P_{m_2}$ implies $\theta(m_1)=\theta(m_2)$ for all compatible models. Sharp bounds mean bounds attained, or approached when the boundary is not attained, by compatible models. Writing this set is a definition, not a solution: the substantive task is to establish informative restrictions and derive or estimate the set. An unrestricted-confounding model may give only trivial bounds.

Inference. Identification, estimability and valid uncertainty quantification are separate questions. Fix $\alpha\in(0,1)$. A confidence procedure $C_n$ over a declared law class $\mathcal P$ has asymptotic uniform coverage at level $1-\alpha$ when \[ \liminf_{n\to\infty}\inf_{P\in\mathcal P}\Pr_P\{\theta(P)\in C_n\}\geq1-\alpha. \] For set-identified targets the coverage event must be defined separately, for example coverage of the full identified set. If an entry uses identification-indexed models instead of uniquely indexed laws, the same coverage convention is applied over those models. Pointwise limits do not establish uniform validity, and asymptotic validity does not guarantee finite-sample precision. Sample sizes, dependence structures and weak-identification sequences are part of each application.

Counterfactuals. $Y_i(a)$ is a potential outcome under a specified intervention, including its timing, implementation and versions. It is not $Y_i$ conditional on voluntarily choosing $a$. A full intervention vector permits interference; shorthand scalar treatment notation does not silently impose no interference. Assignment, selection, attrition, anticipation and measurement must be specified. A claim about an instrument's local effect is not automatically a population policy effect.

Economic equilibrium. A counterfactual model includes best responses, constraints, feasibility, market clearing, state transitions and expectations consistent with the stated information. When several equilibria exist, either a declared selector is an input or the target is set-valued. Comparisons across policy regimes must use the stated selection convention. Reduced-form relationships are not presumed invariant after an equilibrium-changing intervention.

Optimization and welfare. Optimization is over the stated feasible set. If attainment is not established, $\sup$ or $\inf$ and $\varepsilon$-optimal choices replace an asserted maximizer or minimizer. For example, with value $V=\sup_{a\in\mathcal A}W(a)<\infty$, an $\varepsilon$-optimal set is $\{a\in\mathcal A:W(a)\geq V-\varepsilon\}$ for $\varepsilon>0$. Benefits, costs, risk and health or environmental outcomes can be aggregated only using declared units and conversion weights. Interpersonal utility weights, the treatment of fiscal transfers and foreign profits, discounting, and the behavioral welfare criterion are explicit inputs. A comparative-static derivative additionally requires existence and appropriate differentiability of the selected counterfactual.

Sources. Local labels [S1], [S2], and so forth restart in every question. Locators identify the relevant abstract, model, section or result at the level actually checked; a bibliographic or abstract-level verification is not represented as inspection of an entire proof. Working papers are distinguished from later publications and changing versions. Source summaries are paraphrased. All URL checks and editorial formulations refer to the audit date stated above.