GPT 6 Astra Ultra attempt on Econ-650

Experimenter-demand and Hawthorne effects: a research attempt

Target and distinguishable interventions

Let $A\in\{0,1\}$ be substantive treatment, $D\in\{-,0,+\}$ a randomized researcher-expectation message, and $H\in\{0,1\}$ awareness of observation. Potential outcomes are $Y(a,d,h)$ on a common scale. The target without experimental demand, $\tau^*=\mathbb E[Y^*(1)-Y^*(0)]$, is not automatically equal to the effect in a neutral-message arm. A neutral message may leave expectations unchanged from recruitment, and awareness can alter behavior without implying a preferred response.

Assume independent sampling, consistency, no interference, and positive-probability factorial randomization. Then the observed arm mean \[ \mu_{adh}=\mathbb E[Y\mid A=a,D=d,H=h] =\mathbb E[Y(a,d,h)] \] is identified, and so are treatment, message and awareness contrasts within the support of the experiment. Randomization does not by itself identify $Y^*$, a potential outcome under a different psychological or recruitment regime. This attempt proves sharp bounds under a declared mean-bracketing model, derives simultaneous inference and exhibits an observational equivalence when that assumption is omitted.

Sharp mean bounds and their interpretation

Normalize outcomes to $[0,1]$. Suppose the substantive assumptions are \[ l_a:=\mu_{a,-,1}\leq m_a^*:=\mathbb E Y^*(a) \leq u_a:=\mu_{a,+,1}\quad(a=0,1). \] If $l_a>u_a$, these population restrictions are inconsistent with the observed law. Otherwise subtraction yields \[ \tau^*\in[l_1-u_0,\ u_1-l_0]. \tag{1} \] These are sharp bounds in the class imposing only these mean inequalities and the experimentally identified potential-outcome marginals. To prove sharpness, choose any pair $(m_0^*,m_1^*)$ in the two intervals and append Bernoulli variables with those means independently of the observed potential outcomes. This preserves the entire experimental law and meets every stated restriction. The difference of two intervals fills exactly (1). Additional individual-level monotonicity, mediation or common-response restrictions may shrink the class, but they are not present in this proof.

For example, let $(l_0,u_0)=(0.3,0.5)$ and $(l_1,u_1)=(0.4,0.8)$. The identified set is $[-0.1,0.5]$. A positive treatment effect in every observed message arm would not necessarily force this interval to be positive: the demand-free treatment and control means may occupy opposite ends of their separate brackets. The uncertainty is about a comparison of two latent means, not just the sign of an experimental regression coefficient.

A sensitivity version permits violations of the brackets by $\epsilon_a\geq0$. Define \[ L_a=\max\{0,l_a-\epsilon_a\},\qquad U_a=\min\{1,u_a+\epsilon_a\}. \] The same proof gives the sharp interval $[L_1-U_0,U_1-L_0]$ whenever both intervals are nonempty. This calculation makes explicit how much distrust of the manipulation is needed to reverse a substantive conclusion. It does not estimate the sensitivity parameters from randomization.

Simultaneous uncertainty for the identified set

Suppose the four bracket arms have fixed independent-sample sizes $n_j$, $j=1,\ldots,4$, and sample means $\widehat\mu_j$. For bounded outcomes, Hoeffding and a union bound imply simultaneous coverage of all four means with probability at least $1-\alpha$ using \[ r_j=\sqrt{\frac{\log(8/\alpha)}{2n_j}},\qquad [\underline\mu_j,\overline\mu_j] =[\max(0,\widehat\mu_j-r_j),\min(1,\widehat\mu_j+r_j)]. \] On this common event, the entire interval (1) lies in \[ [\underline\mu_{1,-,1}-\overline\mu_{0,+,1}, \overline\mu_{1,+,1}-\underline\mu_{0,-,1}]. \tag{2} \] Thus (2) has a full-identified-set coverage interpretation under the bracket model. Separate unadjusted confidence intervals for the two endpoints do not automatically provide that joint guarantee. For the sensitivity model, expand the component bands by the announced $\epsilon_a$ before subtracting.

Sampling uncertainty cannot repair a population violation of the bracket premise. If the estimated direction is reversed, report the evidence and uncertainty rather than sorting the two means and pretending the manipulation worked as intended. Clustered observation or repeated within-person measures require corresponding dependence-aware bands; the independent-sample exponent above would otherwise overstate precision.

A null manipulation does not identify absence of demand

Construct an experiment in which $Y(a,d,h)$ is Bernoulli with mean $1/2$ for every randomized arm, irrespective of $a,d,h$. The full observed law is the same in the following models. Model I appends demand-free outcomes with means $(0,1)$, giving $\tau^*=1$. Model II appends demand-free outcomes with means $(1,0)$, giving $\tau^*=-1$. Since $Y^*$ is never observed and no link to it is assumed, both models are compatible. Every message and awareness contrast is zero in either model.

This is a nonidentification construction in the unrestricted latent-outcome class, not a behavioral claim that such extreme distortions are plausible. It shows which missing assumption is doing the work: a link between the randomized manipulations and demand-free behavior. If mean bracketing is imposed in this example, it forces both demand-free means to $1/2$ and the effect to zero. The dramatic change in conclusion comes from the bracket restriction, not from additional experimental information.

Measuring perceived researcher intent can test whether messages changed the intended belief. It cannot, without further restrictions, establish that outcome changes occurred only through that belief. Messages can also change factual knowledge, trust or anxiety. Conditioning on the measured belief is post-treatment conditioning and can select different participants across arms. A mediation analysis would need its own intervention definition and confounding assumptions.

Separating awareness and message effects

Define an awareness contrast at a fixed treatment and message by $\eta_{ad}=\mu_{ad1}-\mu_{ad0}$. An interaction of awareness with treatment is \[ I_d=(\mu_{1d1}-\mu_{0d1})-(\mu_{1d0}-\mu_{0d0}). \] Factorial randomization identifies both. They separate the effects of two assigned interventions statistically, although naming either contrast a pure psychological mechanism still requires exclusions. If awareness cannot ethically or practically be randomized to zero, its unobserved arm should not be filled in from the neutral-message group.

An independent administrative outcome can help detect survey reporting changes, provided its recording and linkage process is comparable across arms. If $Y^{\rm reported}=Y^{\rm behavior}+B$ in a common scale and treatment does not change administrative measurement error, contrasting both measurements can isolate changes in reporting bias. Without that measurement restriction, a difference can reflect record coverage instead. Nor does administrative measurement erase the effect of being recruited into a study.

To transport the interval to an ordinary policy setting, suppose a justified mean restriction bounds each setting discrepancy by $\delta_a$. Then the effect interval expands by $\delta_0+\delta_1$ on either side, clipped to $[-1,1]$. An awareness experiment in one task cannot justify zero discrepancies in every task, population or horizon.

Completion Estimate

45%

This is a post hoc, heuristic estimate for the full catalogue target. The attempt establishes sharp demand-free effect bounds and joint inference under mean bracketing, demonstrates unrestricted nonidentification and separates assigned awareness and message contrasts. Credible bracketing and psychological mechanism isolation still require independently supported belief and exclusion restrictions, comparable administrative measurement and validation when transporting to ordinary policy settings.

Status and checked source

The bounds, sharpness construction and simultaneous uncertainty guarantee are proved for their declared classes. Credible bracketing, mechanism isolation and external transport remain substantive unresolved parts of the catalogue target. No universal magnitude of experimenter demand is inferred.

Jonathan de Quidt, Johannes Haushofer and Christopher Roth (2018), Measuring and Bounding Experimenter Demand, American Economic Review 108(11), 3266--3302. Primary publisher record and author-hosted manuscript inspected for the demand-manipulation and bounding framework: https://www.aeaweb.org/articles?id=10.1257/aer.20171330, https://jondequidt.com/pdfs/deQuidt_Haushofer_Roth_demand_final.pdf.