we-solve (beta)

Econ-644 · economics · Economics

AI-generated marketplace summaries

LLM status Open unverified

Statement

Original subject group: Digital, platform and data economics

Audit classification: Published research agenda. The source motivates average and distributional effects of AI summaries. Product-cluster assignment does not alone justify scalar potential outcomes for multi-product viewers.

Definitions and precise research target

Consider product reviews on an e-commerce platform and the population of users viewing eligible products during a prespecified six-month experiment. Randomly assign each product-market cluster to $a\in\{0,1\}$, where $a=1$ displays an AI-generated review summary and $a=0$ displays the existing review interface. Hold access to full reviews available in both regimes. Let $Y_i(a)$ be buyer $i$'s independently measured money-valued postpurchase satisfaction minus price and search cost, assigning zero gross satisfaction to the outside option. Include returns and unmet expectations in the satisfaction measurement. Define $\tau=E[Y_i(1)-Y_i(0)]$ and $\tau_g=E[Y_i(1)-Y_i(0)\mid G_i=g]$, where $G_i$ is a prespecified buyer group and expectation averages over the experiment's eligible-user population. Compare summaries with audited review evidence to measure omitted adverse information and factual errors. Account for seller adaptation and review manipulation by separately estimating immediate effects and effects after the six-month deployment. Specify exposure mappings for buyers who view products in multiple clusters; otherwise report interference sensitivity bounds. Determine whether summaries improve match value and whether gains concentrate among particular users or sellers. Engagement is a secondary outcome, because more clicks alone do not define better matches.

Definitions, assumptions and mathematical qualifications. The eligible population includes all viewers, not just purchasers. A summary policy includes model version, prompt, review cutoff, refresh rule and full-review access. With several clusters viewed by one person, $Y_i(z)$ is indexed by the full assignment vector $z$; a scalar $Y_i(a)$ is valid only under no relevant cross-cluster interference or a prespecified exposure mapping. Define satisfaction using one monetary elicitation or utility model across arms; outside-option gross value and purchase expenditure are zero but incurred search cost remains. Separate immediate outcomes from six-month policy outcomes incorporating seller/review responses.

Variants and assumption changes

  1. Editorial, fixed review corpus. Freeze eligible reviews and seller offers and randomize summaries, identifying a direct information effect under no cross-cluster spillovers.
  2. Editorial, adaptive review ecosystem. Randomize geographically complete markets, permit new reviews and seller responses, and include factual-error, omitted-adverse-information and group-welfare outcomes after six months.

References and source audit

Checked source. 2025 conference draft, Section 1.1, fourth research direction, PDF p. 5. Full primary text retrieved. https://conference.nber.org/confer/2025/DTs25/farronato.pdf.
  1. Chiara Farronato. 2025. Digital Platforms as Information Aggregators: Implications for their Design and Regulation. 2025 conference draft, Section 1.1, fourth research direction, PDF p. 5. https://conference.nber.org/confer/2025/DTs25/farronato.pdf.

Common conventions: Source mathematical conventions

These conventions apply to each entry unless explicitly replaced.

Models and identification. A primitive model $m$ specifies all probability laws, feasible choices, information, equilibrium and measurement rules used by its target. The admissible class $\mathcal M$ is fixed independently of outcomes. If $P_O(m)$ is its observable probability law and $\tau(m)$ the target, its identified set at observable law $P$ is \[ \mathcal I_\tau(P)=\{\tau(m):m\in\mathcal M,\ P_O(m)=P\}. \] Point identification means that this set is a singleton. An empty set signals incompatibility of data and assumptions. Coordinate bounds need not describe the joint set. Bounds are sharp only if every retained target is attainable or arbitrarily approachable by compatible models. Finite bounds require explicit restrictions on unobserved quantities; unrestricted confounding, hidden wealth, extrapolation or arbitrary equilibrium selection can yield uninformative sets.

Causal interventions. A potential outcome $Y_i(p)$ is the outcome under a complete policy regime $p$, including timing, financing, information and market spillovers. The target population and units are fixed before treatment. With interference, use the complete assignment vector or a justified exposure mapping; individual no-interference notation alone cannot identify an economy-wide intervention. A difference of mean potential outcomes does not require the cross-world joint distribution. Conditioning on post-treatment migration, survival, enrollment or employment changes the estimand and needs separate justification.

Statistical statements. An estimator, confidence set, forecast or test specifies a sampling law, model class and loss/coverage criterion. Uniform $(1-\alpha)$-coverage means $\inf_{m\in\mathcal M}\Pr_m\{\tau(m)\in C_n\}\geq1-\alpha$, or an explicitly asymptotic version. The whole parameter space trivially has coverage, so informativeness requires a stated width, risk or power objective. A forecasting improvement is relative to a named benchmark, information set and evaluation population.

Equilibrium and optimization. An equilibrium comprises feasible plans, optimality, beliefs where needed, budget constraints and all market/resource clearing. The primitives or a stated benchmark define each correspondence $\mathcal E(p;m)$. Nonemptiness is assumed or proved, never inferred just from compact policy sets. A selected-equilibrium target uses a declared selection rule; a robust target quantifies over all permitted equilibria. Use suprema or infima unless attainment is established. An $\varepsilon$-optimal policy is feasible and within $\varepsilon$ of the finite optimum value. Report an empty feasible set separately.

Welfare and accounting. Normative utility, interpersonal normalization, social weights, numeraire, discounting and the use of public receipts are primitives. Behavioral utility need not coincide with the welfare criterion. Household welfare includes ownership income and rents once; adding profits again to compensating variation generally double-counts them. A monetary equivalent is defined by an explicit transfer and price convention, with monotonicity and range conditions. Average effects, mechanism contributions, transfers and welfare losses are different targets. Infinite sums require convergence and dynamic feasible plans require terminal or no-Ponzi conditions.

Source versions. A working-paper date, revision date and journal publication year are distinguished. A DOI for a journal version is not automatically the DOI of an earlier manuscript. Locators refer to the stated version and printed pages where specified. A metadata-only check does not verify a claim about a full-text conclusion. Every entry's source subsection narrows the provenance claim accordingly.

Statement licence: CC-BY-4.0 · Source file

Contributions 1

Econ-644-A1 · Exploration · LLM-generated · gpt 6 astra ultra ·

GPT 6 Astra Ultra attempt on Econ-644

AI-generated marketplace summaries Information value and interference in marketplace summary experiments

Policy, cohort, and welfare target

The target is the six-month consumer-welfare effect of a specified AI-summary policy, including users who do not buy. Fix eligibility before the summary is displayed, and retain everyone in that cohort. Freeze the model version, prompt, review cutoff, and refresh rule. For user $i$, let $Y_i$ be independently assessed money-valued satisfaction less price and search cost, including returns and disappointing purchases. The no-purchase alternative has zero gross consumption value and price but still incurs any search cost. This attempt establishes a rational-information benchmark, an explicit failure example, and an estimator under an interference model. It does not determine the actual six-month effect or seller response.

A benchmark in which optional summaries cannot reduce welfare

Consider a fixed review corpus, fixed offers, a finite state space $\Omega$, a finite action set including no purchase, and utility $u(a,\omega)$. Let $R$ denote the complete review information and let a summary $S$ be produced by a stochastic kernel depending only on $R$. With correct beliefs and no processing costs, write \[ V(R)={\mathbb{E}}\max_a{\mathbb{E}}[u(a,\omega)\mid R],\qquad V(S)={\mathbb{E}}\max_a{\mathbb{E}}[u(a,\omega)\mid S]. \] Since $\omega,R,S$ form a Markov chain, conditional averaging implies \[ V(R)\ge V(S). \tag{1} \] To prove this, choose an $S$-optimal action for each signal. An observer of $R$ can simulate $S$ from its known kernel and take that action. This achieves $V(S)$, while choosing an optimal action given $R$ does at least as well. Equivalently, apply convexity of the maximum to posterior expected utilities. The summary need not add statistical information to add value when acquiring or interpreting the underlying reviews is costly.

A stronger conclusion applies to the catalogue's optional-access design. Let $\mathcal P_0$ be the feasible information-acquisition and action policies under the old interface, with their full search costs. If every old policy remains available at the same cost after summaries are added, then $\mathcal P_0\subseteq\mathcal P_1$, so \[ \sup_{\pi\in\mathcal P_1}{\mathbb{E}}[u(\pi,\omega)-c(\pi)] \ge\sup_{\pi\in\mathcal P_0}{\mathbb{E}}[u(\pi,\omega)-c(\pi)]. \tag{2} \] No informational ordering between summaries is needed for (2). Its proof is set inclusion, and its assumptions are demanding: correct decision models, free ability to ignore summaries, unchanged old-interface costs, and no price or seller-quality response. Keeping a link to all reviews does not establish those behavioral and institutional conditions.

A checked negative-welfare example

Let product quality be high with probability $1/4$ and low otherwise. Gross values are three and zero, and the price is one, giving purchase surpluses two and minus one. A full review perfectly reveals quality and costs $0.1$ to read. Under the control interface, reading and buying only high-quality products gives expected welfare \[ (1/4)2-0.1=0.4, \] which exceeds both no search/no purchase, with payoff zero, and uninformed purchase, with payoff $-0.25$. The optimal purchase rate is $1/4$.

Suppose a misleading summary causes consumers to believe both that the high-quality probability is $0.8$ and that the full review contains no information beyond the summary. Their perceived immediate purchase surplus is $0.8(2)+0.2(-1)=1.4$. Under these stated beliefs, further inspection leaves that perceived surplus unchanged and adds cost $0.1$, so buying immediately is subjectively optimal. Actual mean welfare becomes \[ (1/4)2+(3/4)(-1)=-0.25, \] a decrease of $0.65$, while purchases rise from $1/4$ to one. Full reviews remain available throughout. This example intentionally violates correct beliefs in (2), through both quality beliefs and beliefs about incremental information. It shows why greater conversion and nominal review access are insufficient welfare evidence. It does not claim that all AI summaries produce these errors.

The control cohort's nonbuyers have paid the review cost. Dropping them would replace $0.4$ by a selected purchaser average and conceal the proper policy comparison. Audits should therefore measure omitted adverse information and consumer interpretation as well as literal falsehoods. A technically true summary can change beliefs about what further search would reveal.

A design-based result allowing cross-cluster exposure

Let clusters $j=1,\ldots,J$ be independently assigned $Z_j\sim \operatorname{Bernoulli}(p)$, with $0<p<1$. For each baseline user fix a nonempty set $N_i$ of potentially relevant clusters before assignment. Assume the six-month potential outcome is $Y_i(z_{N_i})$: all interference relevant to that user is confined to this set. Define \[ E_i^1=\{Z_j=1\ \forall j\in N_i\},\quad E_i^0=\{Z_j=0\ \forall j\in N_i\}, \] with probabilities $\pi_i^1=p^{|N_i|}$ and $\pi_i^0=(1-p)^{|N_i|}$. The finite-cohort contrast between full exposure to summaries and no exposure is \[ \tau_N=\frac1n\sum_i[Y_i(\mathbf1_{N_i})-Y_i(\mathbf0_{N_i})]. \] Then the Horvitz--Thompson estimator \[ \widehat\tau_N=\frac1n\sum_i\left\{ \frac{\mathbf1(E_i^1)Y_i^{\rm obs}}{\pi_i^1} -\frac{\mathbf1(E_i^0)Y_i^{\rm obs}}{\pi_i^0}\right\} \tag{3} \] is unbiased. On $E_i^a$, consistency and the exposure restriction make the observed outcome equal the corresponding fixed potential outcome; taking expectations cancels the exposure probability in each summand. No independence between users is needed for this expectation calculation.

Exposure overlap can nevertheless make precision poor. Under the extra bound $|Y_i(z)|\le B$, define \[ c_j=\frac{B}{n}\sum_{i:j\in N_i} \left(\frac1{\pi_i^1}+\frac1{\pi_i^0}\right). \] Changing one assignment $Z_j$ can change only summands for users with $j\in N_i$. Each such summand, before division by $n$, takes one of the three values $Y_i(\mathbf1)/\pi_i^1$, zero, or $-Y_i(\mathbf0)/\pi_i^0$; their pairwise distances are bounded by the quantity defining $c_j$. Bounded differences for independent cluster assignments therefore gives \[ {\mathbb{P}}(|\widehat\tau_N-\tau_N|\ge t) \le2\exp\left(-\frac{2t^2}{\sum_jc_j^2}\right). \tag{4} \] This is conservative but fully design-based under the stated assumptions. For one cluster per user, weights are ordinary inverse assignment probabilities. For users linked to many clusters, the exponential growth of inverse exposure probabilities explains why more observations may still leave little useful precision.

The set $N_i$ must include potential exposure routes, not merely products a user actually viewed after treatment. Summary-induced search can change those realized sets. Selecting $N_i$ after observing views would invalidate the assignment-probability calculation and could condition on a treatment mediator. Likewise, market-wide prices or sellers influenced by clusters outside $N_i$ invalidate the assumed outcome mapping.

What remains for the original problem

A frozen-corpus trial estimates a direct interface effect only under its fixed offers and review supply. A market-level six-month trial can include seller adaptation, review manipulation, and changing availability, but its assignment unit must encompass the relevant feedback. One cannot transfer (3) to that ecosystem without expanding its exposure sets or defending a new interference restriction. Broad exposure sets may make (4) vacuous; that is an identification-and-design limitation, not a reason to silently assume individual randomization.

Farronato's 2025 conference paper, Section 1.1, explicitly identifies AI aggregation of textual reviews as a research direction and discusses average/distributional effects and interference. That source motivates this problem; it does not supply the welfare sign. Equations (1)--(4) are restricted decision-theoretic and randomization calculations. Actual monetary satisfaction, search costs, misleading omissions, group heterogeneity, and seller adaptation remain to be measured under a specified policy. The original empirical welfare question is unresolved.

Completion Estimate

40%

This is a post hoc, heuristic estimate for the full catalogue target. The attempt proves optional-information welfare benchmarks, constructs a misleading-summary welfare loss and gives unbiased full-exposure estimation with design-based interference bounds. Actual immediate and six-month group welfare, factual omissions and seller or review adaptation still require measured outcomes and exposure designs spanning the relevant marketplace feedback.

References

Go to contribution →