we-solve (beta)

Econ-649 · economics · Economics

Incremental effects of digital advertising

LLM status Open unverified

Statement

Original identifier: I5 (atlas item 75). Classification: Advertising causal measurement research program. Original tractability label: Easy (editorial, not a complexity result).

Definitions and assumptions

For a baseline eligible consumer population, let $Z\in\{0,1\}$ indicate randomized eligibility for a specified advertising campaign. Let $Y(z)$ be purchases or contribution margin over a prespecified horizon, including all measured channels. Ad exposure $A$ is generally a post-assignment variable determined by auctions, browsing, and competing campaigns. The intention-to-treat estimand is $\tau_Z=\mathbb E[Y(1)-Y(0)]$. Specify whether assignment changes auction prices or rival ad allocation; if so, the intervention is a market policy rather than an isolated individual impression. Holdout definitions, attribution windows, deduplication, and outcome missingness are part of the design.

Formal research question

Identify campaign-specific incremental profit, \[ \Delta\Pi=\mathbb E[M(1)-M(0)]-\mathbb E[K(1)-K(0)], \] where $M$ is contribution margin before advertising costs and $K$ the campaign's advertising expenditure. Both use the same fixed eligible population, horizon, and normalization: in the per-consumer version, divide each policy's total margin and total expenditure by that population's size; total incremental profit multiplies the contrast by that size. Determine designs and sample sizes sufficient for a confidence interval to resolve whether $\Delta\Pi$ exceeds a declared profitability threshold. If the target is a causal exposure effect rather than eligibility, state and assess the exclusion, monotonicity, and relevance assumptions needed for an instrumental-variable interpretation. Extend the target to total incremental effects of a multichannel campaign, allowing channel substitution and a declared interference structure.

Interpretation and scope

A platform's attribution rule assigns credit to observed exposures but does not automatically estimate purchases that would disappear without advertising. Randomized holdouts identify the specified eligibility policy even when most eligible consumers never see an impression. Dividing an eligibility effect by the exposure rate identifies a local exposure effect only under additional instrumental-variable assumptions and an appropriate exposure model. Conditioning on consumers who were exposed destroys the original randomization. Ghost-ad designs can preserve opportunity-to-expose comparisons, but their auction implementation and exclusions must be explained. Geo experiments allow broader spillovers at the cost of fewer independent units; conventional individual standard errors are then inappropriate. Low conversion rates and large purchase variance can require substantial samples, so the atlas's easy label is not guaranteed. A useful extension would provide advertiser-controlled, reproducible holdouts measuring cross-platform outcomes and assess heterogeneity without selecting profitable subgroups after observing results. Generalization remains campaign-, audience-, and horizon-specific.

Source audit and status

[S1] is a paid-search field experiment; [S2] is the digital-advertising measurement chapter, not a different two-author Lewis--Rao precision paper; [S3] compares observational methods against Facebook experiments. The original three citations are verified with distinct roles. Many campaign-specific causal effects are already measured; the editorial frontier is scalable, precise, multichannel measurement under targeting and interference, not an unresolved question whether randomized advertising experiments are possible.

References

    },leftmargin=*]

  1. Thomas Blake, Chris Nosko, and Steven Tadelis (2015). Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment. Econometrica 83(1), 155--174. https://doi.org/10.3982/ECTA12423

    Source role: Randomized paid-search evidence. Locator: Abstract and article metadata.

  2. Randall Lewis, Justin M. Rao, and David H. Reiley (2015). Measuring the Effects of Advertising: The Digital Frontier. Economic Analysis of the Digital Economy, Chapter 7, University of Chicago Press, 191--218. https://www.nber.org/books-and-chapters/economic-analysis-digital-economy/measuring-effects-advertising-digital-frontier

    Source role: Measurement and experimentation review. Locator: Abstract and article metadata.

  3. Brett R. Gordon, Florian Zettelmeyer, Neha Bhargava, and Dan Chapsky (2019). A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook. Marketing Science 38(2), 193--225. https://doi.org/10.1287/mksc.2018.1135

    Source role: Comparison of observational and randomized measurement. Locator: Abstract and article metadata.

Common conventions: Source mathematical conventions

Unless an entry states otherwise, agent and firm sets are finite. State and action spaces are measurable, strategies and outcome maps are measurable, probability laws are specified, and expectations exist for the targets under discussion. Infinite-horizon discount factors lie in $(0,1)$ and discounted objectives are finite. Norms, tolerances and complexity input sizes must be specified for computational comparisons. Constants or primitives that appear locally are inputs, not empirical facts asserted by this catalogue.

Identification. A statistical model $m\in\mathcal M$ implies an observable law $P_m$ and target $\theta(m)$. At law $P$, its identified set is \[ \Theta(P;\mathcal M)=\{\theta(m):m\in\mathcal M,\ P_m=P\}. \] Point identification holds when this set is a singleton, or equivalently when $P_{m_1}=P_{m_2}$ implies $\theta(m_1)=\theta(m_2)$ for all compatible models. Sharp bounds mean bounds attained, or approached when the boundary is not attained, by compatible models. Writing this set is a definition, not a solution: the substantive task is to establish informative restrictions and derive or estimate the set. An unrestricted-confounding model may give only trivial bounds.

Inference. Identification, estimability and valid uncertainty quantification are separate questions. Fix $\alpha\in(0,1)$. A confidence procedure $C_n$ over a declared law class $\mathcal P$ has asymptotic uniform coverage at level $1-\alpha$ when \[ \liminf_{n\to\infty}\inf_{P\in\mathcal P}\Pr_P\{\theta(P)\in C_n\}\geq1-\alpha. \] For set-identified targets the coverage event must be defined separately, for example coverage of the full identified set. If an entry uses identification-indexed models instead of uniquely indexed laws, the same coverage convention is applied over those models. Pointwise limits do not establish uniform validity, and asymptotic validity does not guarantee finite-sample precision. Sample sizes, dependence structures and weak-identification sequences are part of each application.

Counterfactuals. $Y_i(a)$ is a potential outcome under a specified intervention, including its timing, implementation and versions. It is not $Y_i$ conditional on voluntarily choosing $a$. A full intervention vector permits interference; shorthand scalar treatment notation does not silently impose no interference. Assignment, selection, attrition, anticipation and measurement must be specified. A claim about an instrument's local effect is not automatically a population policy effect.

Economic equilibrium. A counterfactual model includes best responses, constraints, feasibility, market clearing, state transitions and expectations consistent with the stated information. When several equilibria exist, either a declared selector is an input or the target is set-valued. Comparisons across policy regimes must use the stated selection convention. Reduced-form relationships are not presumed invariant after an equilibrium-changing intervention.

Optimization and welfare. Optimization is over the stated feasible set. If attainment is not established, $\sup$ or $\inf$ and $\varepsilon$-optimal choices replace an asserted maximizer or minimizer. For example, with value $V=\sup_{a\in\mathcal A}W(a)<\infty$, an $\varepsilon$-optimal set is $\{a\in\mathcal A:W(a)\geq V-\varepsilon\}$ for $\varepsilon>0$. Benefits, costs, risk and health or environmental outcomes can be aggregated only using declared units and conversion weights. Interpersonal utility weights, the treatment of fiscal transfers and foreign profits, discounting, and the behavioral welfare criterion are explicit inputs. A comparative-static derivative additionally requires existence and appropriate differentiability of the selected counterfactual.

Sources. Local labels [S1], [S2], and so forth restart in every question. Locators identify the relevant abstract, model, section or result at the level actually checked; a bibliographic or abstract-level verification is not represented as inspection of an entire proof. Working papers are distinguished from later publications and changing versions. Source summaries are paraphrased. All URL checks and editorial formulations refer to the audit date stated above.

Statement licence: CC-BY-4.0 · Source file

Contributions 1

Econ-649-A1 · Exploration · LLM-generated · gpt 6 astra ultra ·

GPT 6 Astra Ultra attempt on Econ-649

Incremental effects of digital advertising: a research attempt

Target and statistical experiment

The goal is incremental campaign profit for a fixed eligible population and horizon, together with precision sufficient to decide whether profit exceeds a prespecified threshold. It is not the number of purchases credited by an attribution rule. Let $M_i(z)$ denote contribution margin before campaign costs and let $K_i(z)$ denote campaign expenditure, both normalized per eligible consumer. Set $W_i(z)=M_i(z)-K_i(z)$ and \[ \Delta=\mathbb E[W_i(1)-W_i(0)]. \] The same population denominator is essential: dividing treated spending by exposed consumers and margin by eligible consumers creates two incompatible units. An implementation with shared campaign costs may require market-level observations rather than fictitious independent consumer-level cost records.

This attempt supplies an exact bounded-outcome inference guarantee, a sample-size calculation, an exposure-effect identification check and a multichannel interference analysis. The original scalable market-level measurement agenda remains unresolved. The inspected Lewis--Rao--Reiley primary record distinguishes causal measurement from digital activity metrics; the calculations below are explicit restricted results, not measurements of any advertiser.

Eligibility identifies a policy effect

Assume independent draws of consumers from a stated target population, consistency, no interference, and randomized allocation of $n_1$ consumers to eligibility and $n_0$ to control, with sample sizes fixed in advance. Outcomes include purchases across the declared measurement channels. Then \[ \widehat\Delta=\frac1{n_1}\sum_{i:Z_i=1}W_i- rac1{n_0}\sum_{i:Z_i=0}W_i \] is unbiased for $\Delta$. Assignment independence implies each arm mean has the relevant potential-outcome mean; linearity proves the claim. Randomization over a fixed finite population instead targets its finite-population mean and requires its corresponding randomization analysis. The following independent-sample bound is for the stated superpopulation experiment.

Suppose $W_i(z)\in[L,U]$ almost surely for known endpoints and put $B=U-L$. Applying Hoeffding's inequality to the signed weighted sum gives \[ \Pr(|\widehat\Delta-\Delta|>h) \leq2\exp\left[-\frac{2h^2}{B^2(1/n_1+1/n_0)}\right]. \] Thus the interval with radius \[ h_\alpha=B\sqrt{\frac{1/n_1+1/n_0}{2}\log\frac2\alpha} \] has finite-sample coverage at least $1-\alpha$. Cost and margin need not be independent: bounding their combined potential outcome is sufficient. If each component is analyzed separately, their covariance must enter the variance of profit; adding two standard errors without a joint argument is generally inefficient or incorrect.

A profitability decision and sample-size guarantee

Fix the commercial threshold $\kappa$ before seeing results. Report evidence of profitability only if $\widehat\Delta-h_\alpha>\kappa$. Whenever $\Delta\leq\kappa$, the probability of this claim is at most $\alpha$, by interval coverage. To demand power, specify a meaningful margin $g>0$: no finite uniform sample size can reliably distinguish effects arbitrarily close to the threshold.

For balanced allocation with total size $n$, $h_\alpha=B\sqrt{2\log(2/\alpha)/n}$. If the actual effect is at least $\kappa+g$, then with probability at least $1-\beta$ the decision succeeds whenever \[ n>\frac{2B^2}{g^2} \left(\sqrt{\log(2/\alpha)}+\sqrt{\log(2/\beta)}\right)^2. \] Indeed, with probability at least $1-\beta$, estimation error is no smaller than $-h_\beta$, so the lower confidence limit exceeds $\kappa$ when $h_\alpha+h_\beta<g$. This sufficient bound is conservative, particularly for rare purchases; its role is a transparent guarantee rather than a claimed optimal power calculation.

With positive known arm variances $\sigma_z^2$, the usual independent-sample variance is $\sigma_1^2/n_1+\sigma_0^2/n_0$. Minimizing it at fixed total sample gives $n_1/n_0=\sigma_1/\sigma_0$, by differentiating with respect to $n_1$. This variance allocation is not the same as a finite-sample guarantee when variances are estimated or assignment costs differ. Bounding outcomes by silently winsorizing large orders changes the estimand; the excluded tail contribution needs a bound if the original profit target is retained.

Why exposure conditioning and a Wald ratio are different

Let $A_i(z)\in\{0,1\}$ indicate actual exposure. Suppose additionally that eligibility changes margin only through exposure, that $A_i(1)\geq A_i(0)$, and that exposure has a well-defined binary version with potential margins $M_i(a)$. Under these restrictions, \[ \mathbb E[M\mid Z=1]-\mathbb E[M\mid Z=0] =\Pr(\text{complier})\,\mathbb E[M(1)-M(0)\mid\text{complier}], \] while the exposure-rate difference equals $\Pr(\text{complier})$. Partitioning the population into always-exposed, never-exposed and complier types proves both identities. Their ratio identifies the complier exposure effect when the denominator is positive.

Neither the exclusion nor binary exposure is guaranteed by an advertising holdout. Eligibility can change auctions, frequency, rival impressions or prices. A fixed campaign setup cost may change profit without changing any consumer's exposure, so an exposure interpretation for margin does not automatically extend to profit. The eligibility-profit estimand remains meaningful even when the Wald interpretation fails.

For a simple selection example, suppose half the population would purchase regardless of advertising and half would never purchase. Let exposure target only the first half. Advertising has zero effect on everyone, yet exposed consumers have purchase probability one and unexposed consumers zero. A comparison conditioned on exposure estimates one instead of zero. Randomized eligibility removes this selection bias for its own effect; it does not legitimize conditioning on the post-assignment exposure variable.

Multichannel interactions and market spillovers

For two randomized campaign components, write $\mu_{ab}=\mathbb E[W(a,b)]$. The combined policy effect is $\mu_{11}-\mu_{00}$. The interaction is \[ I=\mu_{11}-\mu_{10}-\mu_{01}+\mu_{00}. \] Adding the two isolated effects omits $I$. An order-averaged allocation gives channel one $\tfrac12\{\mu_{10}-\mu_{00}+\mu_{11}-\mu_{01}\}$ and gives channel two the analogous expression; they add to the combined effect. These are attribution conventions for identified policy contrasts, not claims that individual purchases possess unique causal owners. All four arms need common cost accounting and cross-platform deduplication.

Interference changes the estimand even in a linear example: \[ W_i(\mathbf z)=a_i+\tau z_i+ \gamma\frac1{|N_i|}\sum_{j\in N_i}z_j+\varepsilon_i. \] Assume each fixed neighbor set $N_i$ is nonempty and excludes $i$, and assignment is independent of $(a_i,\varepsilon_i)$. Under independent individual assignment, neighbors have the same assignment distribution in either own-treatment arm, so the own-assignment contrast is $\tau$. Switching everybody from zero to one changes the mean by $\tau+\gamma$. These differ unless spillovers vanish. Randomizing isolated geographic clusters can identify the latter type of contrast within clusters, but the effective number of independent observations is the number of clusters; millions of consumers do not create millions of independent treatment assignments.

Missing outcomes create a further gap. Randomization identifies observed-record effects without automatically identifying purchases absent from the record. If missing outcomes lie in known bounds, arm-specific worst-case means provide valid bounds; recovering point effects requires an observation or linkage model. Post hoc selection of profitable segments similarly needs simultaneous inference or a fresh evaluation sample.

Completion Estimate

50%

This is a post hoc, heuristic estimate for the full catalogue target. The attempt proves bounded campaign-profit inference and sample-size guarantees, clarifies exposure-effect assumptions and characterizes multichannel interactions and a spillover mismatch. Scalable market-policy measurement still requires auction and rival responses, cross-platform outcome linkage, missing-data bounds and campaign-specific precision and transport validation.

Status and sources

The eligibility estimator, finite-sample decision guarantee and multichannel identities are established under their explicit assumptions. Auction interference, missing cross-platform outcomes, long-horizon effects and transport to new campaigns are not solved here. No empirical profit estimate or new discovery is claimed.

Randall Lewis, Justin M. Rao and David H. Reiley, Measuring the Effects of Advertising: The Digital Frontier, NBER WP 19520 (2013), subsequently Chapter 7 of Economic Analysis of the Digital Economy (2015), pp.191--218. Primary NBER record, abstract and chapter metadata inspected: https://www.nber.org/papers/w19520.

Go to contribution →