The question everyone asks about artificial intelligence is the wrong one
The debate about artificial general intelligence has settled into a familiar groove. How many parameters will the next model have? How much compute will it burn? How many reasoning steps can it chain together before it loses the thread? The numbers climb, the benchmarks fall, and the assumption hardens that more is the direction of travel. A paper now circulating among researchers asks a different question entirely, and in doing so it quietly reassigns a job that has belonged to human judgment for as long as complex work has existed: deciding what is worth doing at all.
The framework, described in a Perspective on conditional interaction, argues that the next stage of progress may not come from scaling the resource-intensive executive that handles reasoning and planning — what the authors call System-2 — but from pairing it with a lightweight predictor, System-1, that evaluates candidate interactions before any of them are executed. [1] The authors’ answer to their own question is a qualified yes The executive then acts only on what the predictor admits. The human role that this displaces is not the doing of the work. It is the prior act of triage: the experienced practitioner’s sense that this experiment will teach us something and that one will not, that this line of inquiry has legs and that one is a dead end, that this task is worth the budget and that one is not.
The loop that replaces the instinct
The mechanism is disarmingly simple on paper. System-1 takes a candidate — a prompt, a task configuration, a query, a tool call, together with its context — and returns a score rather than a solution. A threshold admits the candidates whose predicted utility clears a bar. System-2 executes only those. The outcomes, once observed, flow back and refit the predictor. The authors call this the predict-gate-execute-update loop, and they note that it spans training, test-time reasoning, adaptive decision-making, and agentic tool use, with utility and cost defined by whatever application is in view.
What makes this more than an engineering trick is the economics it encodes. Under ideal screening and equal execution costs, the paper shows that execution compute falls by a factor approaching one over the density of utility-relevant interactions — a quantity the authors denote rho-u. That density measures the fraction of candidates whose true utility clears a declared floor. When most candidates are duds, the gate pays for itself many times over. When prediction errors, exploration, and distribution shift creep in, the saving shrinks, and the authors bound the selection regret by the predictor’s error and the utility margin. The arithmetic is clean. The implication is not.
What the veteran sampler actually does
Consider the reinforcement-learning pipeline. A model is trained on prompts, generating a group of responses for each and weighting the per-token gradients by how much better or worse each response did relative to the group average. The paper points out that when every response in a group receives the same reward, every advantage term collapses to zero. The prompt is degenerate: it provides no relative policy-gradient signal while consuming the full rollout budget. Only prompts of intermediate difficulty — hard enough that some attempts fail, easy enough that some succeed — yield the mixed outcomes that carry learning. A substantial fraction of any prompt pool, the authors observe, is degenerate.
In a conventional setup, a human curator or an automated heuristic decides which prompts enter the pool. The conditional-interaction framework moves that decision inside the system. The predictor learns to estimate, before generation, which prompts will produce non-zero advantage, and the gate spends the generation budget only on those. The judgment about what is worth training on — a judgment that once belonged to whoever designed the curriculum — becomes a learned function of the current policy state.
The physical world raises the stakes
The digital regime is cheap by comparison. In robotics and experimental science, the paper notes, interactions cannot be freely parallelized and consume tangible time tied to physical dynamics. A generalist agent trained by domain randomization performs across a simulation parameter space whose effective dimensionality governs how many rollouts are needed for generalization. The qualitative insight the authors draw is stark: in high-dimensional spaces, which tasks are evaluated matters as much as how many are. The engineer who once chose which scenarios to test, drawing on years of hard-won intuition about where the failures hide, is replaced by a predictor that scores candidate configurations before the simulator spins them up.
The paper extends the same logic to scientific discovery, where data are produced one experiment, actuation, or measurement at a time, at the pace of the underlying process, and no amount of computational power can accelerate a chemical reaction. Here the gatekeeper function is not merely an efficiency measure. It is the difference between a research program that converges and one that exhausts its budget on questions already answered. The human who performed that triage — the principal investigator who knew which experiment was worth the bench time — is the role under pressure.
The parrot that learns to feed itself
The paper sketches a trajectory it calls the super parrot hypothesis, and it is here that the displacement of human judgment becomes most explicit. In the first stage, roughly 2020 to 2022, the foundation model behaves as a stochastic parrot, imitating human text without grounded understanding. In the second stage, 2022 to 2025, chain-of-thought prompting and reinforcement learning with verifiable rewards let it develop reasoning that humans can still follow. The authors hypothesize a third stage in which the system encodes information in self-coded representations and improves itself through self-evolution, beyond human interpretation.
At each stage, a larger share of what the system learns from is produced by the system itself: synthetic instructions, self-generated reasoning traces scored by automatic verifiers, and ultimately self-chosen experiments. The trend is toward AI for AI. And every self-generated interaction is paid for with the system’s own compute, yet many of them — degenerate prompts, reasoning tokens that leave the answer unchanged, self-proposed tasks already mastered — teach little. The super parrot therefore needs an internal judgment of value before execution. That judgment is precisely the role the paper assigns to System-1. The human who once decided what the model should practice next is written out of the loop, replaced by a learned predictor that the model trains on its own outcomes.
The scaling coordinate nobody was measuring
The paper’s central hypothesis is that the density of utility-relevant interactions is a measurable coordinate of scaling efficiency, and that informative compute is a better description of scaling than total compute. [1] Classical scaling laws express performance through parameters, data, and compute. The authors propose that rho-u is not an additional resource but an efficiency coordinate describing how well those three convert into capability — generated by the learning dynamics rather than supplied from outside.
The distinction matters because it reframes what a scaling curve means. Uninformative interactions waste both computation and the data slots they occupy, so the effective resources under unconditional training are discounted by rho-u. The paper is careful to separate two interpretations of this fraction. The pool fraction is a property of the candidate distribution at the current state, produced by the learning dynamics, and no gate can change it. The executed fraction is the share of interactions actually paid for that are informative. Unconditional training executes the pool, so the two coincide. A perfect gate drives the executed fraction to one. Conditional interaction raises the executed fraction, so that at fixed parameters, data, and compute, more of the budget tends to buy interactions that matter.
The authors go further, predicting that rho-u decays as the system improves, and that this decay may account for part of the diminishing returns and of the scaling exponent. If that holds, then a portion of what the field has attributed to the exhaustion of data and compute is really the exhaustion of useful interactions — a subtler constraint that a gate can address where brute force cannot.
The three decisions at scale
Integrating a cheap predictor with a costly executor introduces three decisions, and each one absorbs a task that humans have traditionally performed. The first is compute allocation: which prompts, tasks, or requests merit rollouts, trials, or extensive thought chains. The second is storage: which samples, buffers, caches, and context entries should be preserved. The third is feedback: how costly outcomes should adjust the inexpensive judgment that allocated them. The paper formalizes this as a fractional knapsack, in which candidates are admitted in order of value per unit cost, with a multiplier fixed by the budget. The threshold rule that falls out of the Lagrangian is the gate.
What is striking is how naturally the framework swallows the orchestration problem. System-1 interprets a task semantically before any of it is executed and can therefore decide who should execute it and how much they may spend. The pool of available System-2 models or agents, together with their compute budgets, becomes the candidate set; their predicted contribution becomes the utility; their inference cost becomes the price. The routing and budget-allocation rule that results is a joint decision that a human operations lead would once have made by hand, drawing on knowledge of which model handles which kind of request and where the money is best spent.
The static cascade versus the co-designed pair
The paper draws a sharp line between a static cascade and true co-design, and the distinction turns on feedback. In a static cascade, a standalone fast model classifies each request and hands the hard minority to a large model. The paper cites a commercial product described by its developers as a “System One,” returning calibrated probabilities over answers defined in advance, as a clear instance of the pattern. [1] Such products illustrate commercial interest in fast, structured judgment models compatible with the System-1 role, though the authors are careful to note that a single product does not establish an industry-wide convergence.
A static cascade is trained once, routes only at the inference layer, and does not receive the executive’s outcomes. That suffices when its predictions are stationary but fails in the setting the paper cares about, because how informative an interaction is for the current System-2 changes with every update. In co-design, System-1 predicts utility relative to the current executive, acts in training as well as at inference, schedules compute and storage rather than a single call path, and is recalibrated online by the executive’s realized rewards, gradients, and outcomes. The paper notes that a cascade retrained online on its outcomes moves toward co-design, so the distinction is not the presence of two models but feedback and joint optimization. The human intermediary who once read the reports and adjusted the routing table is replaced by a loop that adjusts itself.
The interface where oversight migrates
Beyond the gate, the paper specifies three interfaces that any co-designed pair must settle, and each one is a place where a human function has quietly relocated. The knowledge interface determines what is distilled from System-2 into System-1: where the executive repeatedly solves a class of candidates, the solution itself, not only its value, can be amortized into the predictor, so that System-1 answers routine cases outright and escalates the rest. The representation interface determines what System-1 observes: using the executive’s own features buys accuracy at the price of sharing its blind spots, whereas encoding the candidate independently keeps the two error profiles uncorrelated, and conditioning the predictor on context lets it adapt to a new task, scenario, or user without retraining the executive.
The third interface is the one with the most consequence for human oversight. The feedback interface determines which executive outcomes supervise the predictor — realized rewards, gradients, verifier verdicts, or an audit sample of suppressed candidates — and thereby fixes which of its own errors the predictor can ever observe. The paper notes that because System-1 sees every candidate before System-2 acts, cost, irreversibility, and safety constraints can be imposed before a capable executive commits. That is a governance function, and it has moved from a human reviewer to a scoring module.
The audit sample as the last human trace
The paper is candid about the failure modes, and its candor sharpens the point. Gating has no guaranteed advantage when candidates are homogeneous in utility, when predicting their utility costs about as much as executing them, or when execution is cheap. Dense pretraining and minibatch stochastic gradient descent over independent, identically distributed data often come close to these conditions. But the framework applies only where interactions are individually costly, heterogeneous in informativeness, and predictable before execution by a model cheaper than execution itself.
The most insidious failure is misspecification: the predictor systematically suppresses the very interactions that would reveal its own error. The paper’s safeguard is an audit sample of suppressed candidates, executed anyway so that the predictor’s misses can be detected. It is a small concession, and it is the last place where a human-style check survives in the architecture. The paper treats the censoring formally, noting that the predictor’s training data are gathered by a changing policy and censored by the gate itself, so the familiar convergence rates borrowed from independent sampling need not hold. The audit is what keeps the loop from closing on its own blind spots.
Why a cheap predictor can be right at all
The paper spends considerable effort on a question that a skeptic would raise immediately: why should a lightweight model be able to predict the value of an expensive interaction at all? Three complementary answers are offered, and each one is a different account of how expert judgment gets compressed. Information-theoretically, the quantity gated on is expected information gain net of cost, with epistemic action selection as its limiting case. Generatively, an interaction is a transition of a world or policy model, and System-1 is an amortized projection of that model onto a single scalar, which is why valuation is cheaper than simulation. Control-theoretically, executing only what the gate admits is receding-horizon control of a learner whose own dynamics are still being learned.
The authors are careful to label these connections as interpretive rather than separate contributions. Each supplies insight and, under its own assumptions, a guarantee, but none is a derivation of the framework. What they share is a picture of judgment as compression: the veteran’s instinct is a cheap approximation of an expensive evaluation, and the predictor is the same approximation made explicit, trainable, and scalable.
The two clocks
The paper’s formal treatment of co-design rests on an operator decomposition. One operator improves the policy along gradients sampled through the gate; the other fits the predictor to realized utilities. Each maps the pair of policy and predictor parameters to a new pair and updates one component, so their composition is well defined: the predictor is refitted first, and the executive then steps along gradients sampled through the refitted gate. The authors call this decomposition the load-bearing object of co-design, because neither operator is defined without the other, and they present it as a class of algorithms rather than a single one.
The two operators run on different clocks. The predictor adapts quickly to realized utilities, while the executive improves slowly along the gated gradients. Because better predictions change which interactions are executed, and hence the supervision the predictor receives, one might worry that the coupled loop diverges. Two-timescale stochastic approximation gives conditions under which such coupled updates converge, and The paper notes that MPTS trains the two systems jointly in this way in practice, but a convergence guarantee for the general architecture remains open. The paper is honest that the loop resembles self-play along the informativeness axis rather than adversarial reward, and that the separation of timescales is what keeps it stable.
The ranking rule and the calibrated mean
A modest proposition carries an outsized practical lesson. Assume equal costs, a budget of B executions, and that each candidate’s utility depends on the predictor outputs only through its own. Then, among all selection rules that see only the predictor outputs, executing the B candidates with the largest posterior mean maximizes expected executed utility and hence the efficiency ratio. The proposition says nothing about which predictor to learn; its practical content is that the calibrated mean of the predictor, not an uncalibrated score, is the right ranking signal.
The authors derive this from an information-bottleneck objective that trades predictor complexity against prediction informativeness. Whatever representation is learned, the question of how to use it has a clean answer: rank by the posterior mean. The paper is careful to note that the proposition is modest, but the modesty is the point. It converts a vague hope — that a cheap model can stand in for expensive evaluation — into a precise condition on calibration, and calibration is something that can be measured, audited, and reported.
How large should the judge be
Current practice, the paper observes, sizes the executive and treats any selector as an afterthought. The efficiency formula suggests this is incomplete: a larger predictor lowers its error rates but raises its scoring cost, and where the balance lies depends on how expensive the executive is. The authors prove a comparative-statics result: under fixed error curves, if scaling the executive multiplies every execution cost by a factor, the set of optimal predictor sizes is non-decreasing in that factor in the strong set order, and the maximal efficiency ratio is non-decreasing as well.
The mechanism the proof isolates is that scaling the executive dilutes the predictor’s own cost relative to execution, but not the cost of its mistakes, which grows with the executive itself, so accuracy becomes relatively cheaper to buy. The result is qualitative: it fixes the direction of co-scaling, not its rate, since the error curves are empirical. Measuring them, and hence a compute-optimal ratio between the two systems in the spirit of the parameter-to-data ratio, is the co-design counterpart of a scaling law. The paper flags that the assumptions do most of the work and that the monotonicity need not hold when scaling the executive changes the prediction problem itself — when a stronger executive changes the distribution of utilities, the separation between useful and dead interactions, or the cost structure.
The frontier the pair is aiming at
The design objective is stated as a Pareto frontier over cost, latency, and task success. Each surface is the best task success probability attainable within a cost and latency budget. The co-designed pair would spend both only where they change the answer, so its frontier is hypothesized to dominate: the same success more cheaply, or more of it at the same budget. Suppressing the uninformative fraction acts on every axis at once, since fewer rollouts and gradients reduce compute and data, and routing each task only to the executives it needs cuts the capacity activated per task.
At deployment the same logic holds: training-time gating lowers cost, test-time gating improves quality of service with lower latency, and concentrating both on informative interactions raises task success per unit of either. The paper frames this as the comprehensive co-design objective, and the framing is telling. The goal is not to build a better executive. It is to build a pair whose combined spending is allocated by a judgment that no human supplies.
What the framework does not claim
The paper is disciplined about its scope, and the discipline is worth taking seriously. Conditional interaction is a statement about how to spend a trustworthy informativeness signal, not a claim that one always exists. The framework does not substitute exploration when the predictor is uncertain. In regimes where the utility is misspecified, the right object is an exploratory value of information, such as an upper-confidence or Thompson criterion, which replaces the raw score with an uncertainty-adjusted one. That limit favors the most uncertain candidates rather than sampling the pool uniformly; undirected exploration needs a separate mixture with a uniform policy.
The authors identify the most insidious regimes as those where the predictor’s errors concentrate precisely where it has no data and the interactions that would reveal them are never executed. The audit sampling is the minimal safeguard, and the paper treats the censoring formally. There is no promise of salvation here, and no claim that the gate replaces human judgment in every setting. The claim is narrower and more consequential: in the settings where interactions are costly, uneven in value, and predictable before execution, the judgment about what is worth doing can be learned, and once learned, it can be scaled.
The three eras and the shift in what counts
The paper places its proposal in a sequence. In the first era, roughly 2012 to 2020, progress came mainly from scaling model capacity. In the second, roughly 2020 to 2025, from scaling data and computation. For the third era, beginning around 2026, the authors emphasize a next-scaling lens: co-designing a cost-effective predictor that prioritizes interactions by expected utility and a resource-intensive executive for complicated tasks. This emphasis becomes increasingly consequential, they argue, for foundation models and embodied agents as they depend on generation, computation, tool use, and environmental interaction, and thus on the cost of each.
The historical framing is doing real work. It suggests that the constraint has moved from the supply of capability to the allocation of effort, and that the discipline which once governed the allocation of human effort — the experienced manager’s sense of where to point the team — is now the object of engineering. The paper does not say that human judgment disappears. It says that the judgment becomes a trained module, scored against realized outcomes, and sized against the executive it serves.
The density that decays
The paper’s prediction is that rho-u decays as the system improves, and that this decay may account for part of the diminishing returns and of the scaling exponent. Because rho-u depends on the declared utility and its floor, the meaningful quantity is its relative decay from the initial state. If that prediction holds, then scaling laws are better described in informative compute than in compute alone. The implication is that a system which keeps buying more total compute while the density of useful interactions falls is paying for effort that teaches it less and less — and that the fix is not a bigger executive but a sharper gate.
The authors propose measuring the decay across thresholds and across proxies such as the expected gradient norm or the learnability score, and treating agreement across those measures as a robustness diagnostic rather than a theoretical invariance. They are explicit that tail probabilities at different thresholds need not decay proportionally, and that gradient norm and learning progress are not equivalent. Measurement also needs a protocol: because the gate censors what is observed, the density is estimated on an audit pool drawn independently of the gate, with repeated rollouts per candidate, confidence intervals, and re-estimation as the policy changes.
The last open variable
What remains unsettled is the thing the whole framework depends on and cannot guarantee: the quality of the predictor’s judgment under distribution shift, when the executive it serves is itself changing. The paper’s regret bounds degrade gracefully, linearly in the predictor’s error and the batch size, once twice the predictor’s error reaches the gap between the best and second-best candidates. The screening gain approaches its ideal value only as the predictor’s cost relative to execution goes to zero. The co-scaling result fixes a direction but not a rate. The convergence of the coupled loop remains an open problem for the general architecture.
Every one of these open variables is a place where the old human function could reassert itself, and the paper does not pretend otherwise. The audit sample, the calibration check, the decision about which utility to declare — these are the residual seats of judgment in a system designed to automate judgment. Whether they remain residual or become the new center of gravity is the question the next era will answer. The framework’s contribution is to have made the question precise: not how much compute a system can bring to bear, but how well it knows what is worth spending compute on. That knowledge was once the mark of the experienced practitioner. It is now a coordinate on a scaling curve, and the curve is being fitted.