we-solve (beta)

Econ-637 · economics · Economics

Direction and novelty

LLM status Open unverified

Statement

Original subject group: Innovation and AI economics

Audit classification: Background; proposed extension. The source is an automation/patenting study, not direct provenance for a randomized AI novelty theorem. Novelty scores and the experiment are explicitly editorial.

Definitions and precise research target

Study research projects begun by firms in a specified technology sector during a defined enrollment year. Let $A_i\in\{0,1\}$ denote randomized access to an AI discovery tool, with the same baseline research budget in each arm. For each project, define $V_i(a)\geq0$ as externally validated scientific or commercial value over three years and $R_i(a)\in[0,1]$ as novelty relative to a frozen pre-intervention knowledge corpus, judged using a preregistered blinded procedure. Failed projects have $V_i(a)=0$. Define novel discovery value $Y_i(a)=V_i(a)R_i(a)$, total-value effect $\tau_V=E[V_i(1)-V_i(0)]$, and novelty-weighted effect $\tau_Y=E[Y_i(1)-Y_i(0)]$. Also record projects' disciplinary and application categories before outcomes to measure changes in research direction without redefining categories after treatment. Estimate these quantities for eligible projects, accounting for connected research-team spillovers through cluster assignment. Patent counts or textual distance alone are insufficient substitutes for validated discovery value. Does AI increase $\tau_Y$, or mainly expand valuable refinements with low measured novelty, and which project-selection changes mediate the difference? This specified experiment extends existing patent evidence; neither the novelty score nor a universal direction-of-innovation claim is supplied as an established open theorem.

Definitions, assumptions and mathematical qualifications. An eligible project has the same inclusion rule and baseline resource allowance under both policies, with AI/model access precisely specified. Novelty $R_i(a)$ is a fixed scoring functional comparing a validated result to a frozen baseline corpus and blinded review protocol; it is not an intrinsic property inferred from word distance. Require finite $E[V_i(a)]$ and define failed-project novelty convention, with $V_iR_i=0$ regardless of that convention. For a fixed technology-category partition, report the joint distribution of output category, value and novelty. A project-selection mediator is itself treatment affected and needs a distinct causal intervention to identify mediation.

Variants and assumption changes

  1. Editorial, existing projects. Randomize AI access after project selection and estimate gross and novelty-weighted validated value effects for this fixed cohort.
  2. Editorial, direction choice. Randomize access before firms allocate an identical research budget among a fixed category set; measure reallocations and market-level novel value with cluster spillovers, without conditioning on the chosen project category.

References and source audit

Checked source. NBER Working Paper 34240 (September 2025), indexed primary abstract. The cited work studies automation and innovation shifts; it does not supply this AI novelty experiment. Primary metadata and abstract indexed; direct record returned an access error. https://www.nber.org/papers/w34240.
  1. Lin William Cong; Yao Lu; Hanqing Shi; Wu Zhu. 2025. Automation-Induced Innovation Shift. NBER Working Paper 34240 (September 2025), indexed primary abstract. https://www.nber.org/papers/w34240. DOI: https://doi.org/10.3386/w34240.

Common conventions: Source mathematical conventions

These conventions apply to each entry unless explicitly replaced.

Models and identification. A primitive model $m$ specifies all probability laws, feasible choices, information, equilibrium and measurement rules used by its target. The admissible class $\mathcal M$ is fixed independently of outcomes. If $P_O(m)$ is its observable probability law and $\tau(m)$ the target, its identified set at observable law $P$ is \[ \mathcal I_\tau(P)=\{\tau(m):m\in\mathcal M,\ P_O(m)=P\}. \] Point identification means that this set is a singleton. An empty set signals incompatibility of data and assumptions. Coordinate bounds need not describe the joint set. Bounds are sharp only if every retained target is attainable or arbitrarily approachable by compatible models. Finite bounds require explicit restrictions on unobserved quantities; unrestricted confounding, hidden wealth, extrapolation or arbitrary equilibrium selection can yield uninformative sets.

Causal interventions. A potential outcome $Y_i(p)$ is the outcome under a complete policy regime $p$, including timing, financing, information and market spillovers. The target population and units are fixed before treatment. With interference, use the complete assignment vector or a justified exposure mapping; individual no-interference notation alone cannot identify an economy-wide intervention. A difference of mean potential outcomes does not require the cross-world joint distribution. Conditioning on post-treatment migration, survival, enrollment or employment changes the estimand and needs separate justification.

Statistical statements. An estimator, confidence set, forecast or test specifies a sampling law, model class and loss/coverage criterion. Uniform $(1-\alpha)$-coverage means $\inf_{m\in\mathcal M}\Pr_m\{\tau(m)\in C_n\}\geq1-\alpha$, or an explicitly asymptotic version. The whole parameter space trivially has coverage, so informativeness requires a stated width, risk or power objective. A forecasting improvement is relative to a named benchmark, information set and evaluation population.

Equilibrium and optimization. An equilibrium comprises feasible plans, optimality, beliefs where needed, budget constraints and all market/resource clearing. The primitives or a stated benchmark define each correspondence $\mathcal E(p;m)$. Nonemptiness is assumed or proved, never inferred just from compact policy sets. A selected-equilibrium target uses a declared selection rule; a robust target quantifies over all permitted equilibria. Use suprema or infima unless attainment is established. An $\varepsilon$-optimal policy is feasible and within $\varepsilon$ of the finite optimum value. Report an empty feasible set separately.

Welfare and accounting. Normative utility, interpersonal normalization, social weights, numeraire, discounting and the use of public receipts are primitives. Behavioral utility need not coincide with the welfare criterion. Household welfare includes ownership income and rents once; adding profits again to compensating variation generally double-counts them. A monetary equivalent is defined by an explicit transfer and price convention, with monotonicity and range conditions. Average effects, mechanism contributions, transfers and welfare losses are different targets. Infinite sums require convergence and dynamic feasible plans require terminal or no-Ponzi conditions.

Source versions. A working-paper date, revision date and journal publication year are distinguished. A DOI for a journal version is not automatically the DOI of an earlier manuscript. Locators refer to the stated version and printed pages where specified. A metadata-only check does not verify a claim about a full-text conclusion. Every entry's source subsection narrows the provenance claim accordingly.

Statement licence: CC-BY-4.0 · Source file

Contributions 1

Econ-637-A1 · Exploration · LLM-generated · gpt 6 astra ultra ·

GPT 6 Astra Ultra attempt on Econ-637

Direction and novelty: research attempt

Scope and status

The catalogue asks whether randomized access to an AI research tool changes both externally validated project value and novelty-weighted value. Its baseline unit is a project enrolled before assignment, with the same research budget in either arm, a frozen comparison corpus, and failed projects assigned value zero. Write $V(a)\geq0$, $R(a)\in[0,1]$, and $Y(a)=V(a)R(a)$. The targets are $\tau_V={\mathbb{E}}[V(1)-V(0)]$ and $\tau_Y={\mathbb{E}}[Y(1)-Y(0)]$, together with changes in research direction. This attempt derives identification, sharp bounds, and a limitation of the stated moment assumptions. It supplies no project outcomes and does not resolve the empirical targets.

Joint measurement is essential

Independent assignment and complete follow-up identify the joint arm distributions of $(V,R)$, hence both targets whenever ${\mathbb{E}} V(a)<\infty$. In particular, the correct arm outcome is the within-project product $VR$. Multiplying the two sample means targets a different quantity: \[ {\mathbb{E}}[VR]={\mathbb{E}}[V]{\mathbb{E}}[R]+{\operatorname{Cov}}(V,R). \] The covariance exists because $0\leq VR\leq V$ and $R$ is bounded, even when $V$ has no second moment. An increase in average novelty and an increase in average value need not identify the change in their product if only marginal summaries are retained.

Suppose temporarily that validation imposes an actual upper bound $V\leq M$, where $M>0$ is common to both arms. Let $v={\mathbb{E}} V$ and $r={\mathbb{E}} R$.

Proposition: Given only these bounds and means, the sharp interval for ${\mathbb{E}} VR$ is \[ \max\{0,v+Mr-M\}\ \leq\ {\mathbb{E}} VR\ \leq\ \min\{v,Mr\}. \tag{1} \]
Proof: Set $X=V/M$. Pointwise on $[0,1]^2$, $XR\geq0$, $XR\geq X+R-1$, $XR\leq X$, and $XR\leq R$. Taking expectations proves the bounds. For attainment, choose binary $X,R$ with success probabilities $v/M,r$. Their joint success probability can be any number in $[\max(0,v/M+r-1),\min(v/M,r)]$. The four cell probabilities are nonnegative exactly on this interval. Thus both endpoints and every intermediate value are achievable. ∎
Applying (1) arm by arm gives the sharp effect interval $[L_1-U_0,U_1-L_0]$ if no additional cross-arm restrictions are imposed. Potential-outcome pairs with any specified arm distributions can be coupled independently, so this subtraction does not introduce a hidden feasibility problem.

For example, let $M=2$, $v=1$, and $r=1/2$ in both arms. An arm with equal probabilities on $(V,R)=(2,1),(0,0)$ has ${\mathbb{E}} VR=1$; an arm with equal probabilities on $(2,0),(0,1)$ has ${\mathbb{E}} VR=0$. Thus even identical marginal distributions permit an effect of either sign. A simpler illustration of differing objectives is deterministic baseline $(V,R)=(1,1)$ and treatment $(2,1/5)$: total value rises by one while novelty-weighted value falls by $3/5$. Neither outcome is a contradiction.

If the complete separate marginal distributions are known, but project matching has been lost, stronger sharp bounds are \[ \int_0^1 Q_V(u)Q_R(1-u)\,du \leq {\mathbb{E}} VR \leq \int_0^1 Q_V(u)Q_R(u)\,du, \tag{2} \] where $Q$ denotes the generalized quantile function. This result only needs integrable $V$. To prove it, write \[ {\mathbb{E}} VR=\int_0^\infty\int_0^1{\mathbb{P}}(V>s,R>t)\,dt\,ds. \] Each joint exceedance probability lies between the Fréchet intersection bounds. Taking both variables as increasing functions of one uniform variable simultaneously attains all upper bounds; reversing the uniform for $R$ simultaneously attains the lower bounds. Nonnegative integration justifies the argument even with unbounded $V$. These are bounds on missing dependence, not a substitute for random assignment.

Finite means do not supply a uniform precision guarantee

The canonical integrability assumption identifies the estimands and supports pointwise laws of large numbers. It does not imply a usable, distribution-free convergence rate for an untrimmed mean.
Proposition: Over all integrable project-value distributions, there is no uniformly consistent estimator of $\tau_Y$, even with perfect randomization and a perfectly known novelty score.
Proof: Under a null model, both arms have $V=0$ and $R=1$. For sample size $n$, define an alternative with control outcomes unchanged and treated $V=n^2$ with probability $n^{-2}$, otherwise zero; set $R=1$ throughout. The alternative effect is one. Couple the assignment vectors identically in the two experiments. Regardless of the number assigned treatment, the probability of observing any nonzero outcome is at most $n/n^2=1/n$. Hence the total variation distance between their observable-data laws is at most $1/n$. An estimator accurate within $1/3$ under both models would distinguish effects zero and one with probability tending to one, contradicting this indistinguishability. Every alternative has a finite first moment; allowing the alternative to depend on $n$ is precisely what tests uniform consistency. ∎
A verified bound $M$, a justified tail envelope, or stronger moment restrictions can support honest precision statements. Winsorization instead changes the estimand unless its omitted tail is bounded. Cluster assignment also requires cluster-level inference: the number of projects is not the number of independent assignments when teams share tools or ideas.

Score error and research direction

If the operational score $\widehat R$ satisfies $|\widehat R-R|\leq\delta$ for every project, then \[ |{\mathbb{E}}[V(a)\widehat R(a)]-{\mathbb{E}}[V(a)R(a)]|\leq\delta{\mathbb{E}} V(a). \] Consequently the absolute error in the novelty-weighted treatment effect is bounded by $\delta({\mathbb{E}} V(1)+{\mathbb{E}} V(0))$. This is a value-weighted calibration requirement: an apparently small score error on a few exceptionally valuable projects can matter greatly. Blinding, the frozen corpus, and a common validation rubric support the interpretation of $R$; they are not consequences of randomizing tool access.

For mutually exclusive fixed categories $c$, let $\pi_{ac}$ be the arm's project share and $\mu_{ac}={\mathbb{E}}[Y\mid A=a,C=c]$. If both category shares are positive, the exact symmetric decomposition is \[ \tau_Y=\sum_c (\pi_{1c}-\pi_{0c})\frac{\mu_{1c}+\mu_{0c}}2 +\sum_c (\mu_{1c}-\mu_{0c})\frac{\pi_{1c}+\pi_{0c}}2. \tag{3} \] Expanding both products cancels the cross terms and gives $\sum_c\pi_{1c}\mu_{1c}-\pi_{0c}\mu_{0c}$. Equation (3) separates composition and within-category accounting changes. It is not a causal mediation formula: category choice is post-assignment, and projects observed in a category may differ across arms. A category absent from one arm leaves its conditional mean undefined and makes this decomposition convention-dependent, although the total effect remains identified.

In the variant where firms receive tools before selecting projects, the stable randomized unit should instead be the firm or connected research market. Its outcome is the sum of validated novelty-weighted outputs from the common budget, including zero from abandoned lines. Restricting analysis to the projects that happen to be funded in each arm compares treatment-dependent populations. The after-selection variant avoids this particular selection problem but answers a different intervention question.

Completion Estimate

60%

This is a post hoc, heuristic estimate for the full catalogue target. The attempt establishes joint value-novelty measurement, sharp bounds from missing dependence, score-error limits, a composition accounting identity, and the failure of uniform precision under mere integrability. The full innovation target still requires validated three-year project outcomes, a reproducible blinded novelty rubric, defensible tail restrictions and cluster inference, plus causal study of research-direction and project-selection changes.

Source relationship and remaining gap

The inspected author manuscript by Cong, Lu, Shi, and Zhu, Automation-Induced Innovation Shift, is the May 2024 draft hosted in the AEA conference programme, at https://www.aeaweb.org/conference/2025/program/paper/S7SHZQ4n. Its abstract and introductory discussion concern automation exposure and the direction of innovation. That is useful motivation for separating direction from output volume. It is not evidence for the randomized project-level effects here, and the inspected draft should not be confused with the later NBER version named in the catalogue.

The calculations establish what joint data and additional tail restrictions would permit. They leave unresolved the signs and magnitudes of the actual three-year effects, whether a reproducible novelty measure captures scientific novelty, and the spillover-adjusted market effects of research reallocation. No claim that AI increases novelty follows from the bounds or the accounting identity.

Go to contribution →