The Product Team That Shipped in a Day
Product teams have spent the past decade drowning in their own telemetry. Session replays, funnel charts, cohort analyses, support tickets, and code repositories sit in separate systems, each one a partial answer to the same question: what should we build next? The data exists. The insight rarely survives the journey from dashboard to roadmap to shipped code. Amplitude, a company built on helping teams see how people actually use their products, has been testing whether an agent can close that gap — not by generating more charts, but by carrying an observation all the way through to a measured change.
The agent is called Wave. It is described as roughly 80% powered by OpenAI models, including GPT-6.1 Sol, GPT-6 Luna, and GPT-Image-2.5 Flare. [1] A team hands it a product area and the metrics it wants to move. Wave then pulls together Amplitude data, session replays, customer feedback, and source code, ranks opportunities, drafts solutions, and coordinates implementation with people and coding agents, including Codex. After launch, it tracks outcomes and feeds what it learns into the next round. The concrete promise is time: Amplitude estimates Wave can help teams ship product improvements in 1-2 days, down from roughly a month.
That compression is the claim worth examining. Not because speed is inherently virtuous, but because the distance between noticing a problem and fixing it is, in my view, where most product work quietly dies. An insight that takes four weeks to act on has four weeks to be forgotten, deprioritized, or overtaken by a reorganization. In my reading, Wave’s design rests on the premise that the bottleneck was never the analysis, but the handoff.
What the Homepage Experiment Actually Showed
The most useful evidence comes from Amplitude’s own front door. The company’s homepage featured generic suggestion cards meant to help visitors start a conversation with an AI agent. The cards were ignored. In a sample of 100 session replays, 80 users typed their own questions, while only 12 selected a suggestion. That is a finding no dashboard would have volunteered — it required someone, or something, to watch what people did rather than what the interface assumed.
Wave proposed personalizing the cards using recent charts, dashboards, or the page a visitor had arrived from. It produced an illustration of the change and a technical plan that preserved free typing and fell back to generic suggestions when no personalization context was available. After a brief human review, the team handed the work to a coding agent and ran an A/B test. Messages sent through homepage suggestion cards increased 160 % relative to control after one day.
Eric Carlson, a Senior Principal AI Engineer at Amplitude, called it “a great example of how non-groundbreaking things that just get ignored can increase the adoption of features in unexpected ways.” [1]. The modesty is the point. This was not a new product line or a model breakthrough. It was a small interface element that users had been silently routing around, discovered by combing through replays at a scale no human reviewer would sustain over time, then fixed and measured within a cycle short enough to attribute the effect cleanly.
Splitting Work Across Models by Task
Wave’s workflow places genuinely different demands on the models underneath it. A deep investigation into code and analytics needs strong reasoning and conclusions that hold up under scrutiny. A small evaluation check repeated hundreds of times needs to be fast and cheap. Amplitude evaluates quality, latency, and cost for each task, then assigns models accordingly — a division of labor that, by analogy, mirrors how a well-run engineering team allocates senior and junior time.
For the Opportunity Investigator, Amplitude scored models on whether they corroborated findings, avoided false positives, incorporated evaluator feedback, recovered from tool failures, and kept claims aligned with evidence. GPT-6.1 Sol scored 4.06 out of 5 overall, against 3.62, 3.52, and 2.94 for three alternatives. [1] It led on all five investigation criteria and had the fastest median investigation time, at 54 minutes. [1].
The Data Science Agent evaluation tells a more textured story. GPT-6.1 Sol scored 67.5 out of 100, slightly behind another leading model’s 69.3, but at a mean model cost of $1.28 versus $2.63 per dispatch. It also scored higher on tool-contract compliance, estimation and uncertainty, and evidence grounding. That is a trade-off a production system can reason about: a marginal quality difference in one direction, a roughly two-to-one cost advantage in the other, and a better record on the dimensions that determine whether an agent’s output can be trusted without a human rechecking every claim.
For high-volume work such as scanning code and running small evaluation checks, Amplitude uses GPT-6 Luna to keep latency and cost down. The architecture reserves expensive reasoning for the tasks that need it most — a principle that sounds obvious until you notice how often AI deployments apply their largest model to their smallest job.
Mockups That Stay Faithful to the Product
Visual proposals occupy an awkward place in product work. A wireframe that misrepresents the existing interface wastes review time; one that is too abstract invites misinterpretation. Amplitude evaluated OpenAI’s image generation model on solution wireframes across 15 opportunities, scoring communication quality, product fidelity, instruction adherence, text and UI legibility, and visual coherence. GPT-Image-2.5 Flare scored 80.3 out of 100, against 63.9 for another image generation model. [1] The approach matters more than the number. Wave generates mockups using screenshots from session replays as visual references, which grounds the output in the actual product rather than a generic template. When the agent later hands work to a coding agent, the technical plan and acceptance criteria travel with it. Jacob Newman, a Principal AI Product Manager at Amplitude, describes the idea as “a unified platform with agents that comb through massive amounts of session replays, analytics, and code to find opportunities to improve products.” [1].
Where Humans Still Decide
Automation levels are not uniform, and Amplitude is explicit about it. Most teams send bug fixes straight to coding agents, while new features and bigger bets get a human review. That split is a reasonable reading of where trust currently sits: a fix is a bounded correction to known behavior, while a new feature is a hypothesis about what users want, and hypotheses still benefit from a person willing to own them.
The path from discovery to execution is short for changes that reach launch. Among customer-launched experiments, flags, and guides, one in four went live the same day Wave surfaced the opportunity, half in under four days, and two in three within a week. These figures are interpolated from supplied milestones, and the curve is illustrative rather than a set of individual observations — a caveat worth keeping in view, since early beta cohorts tend to be self-selected for teams already inclined to move fast.
What the Agent Cannot Do
Wave produced a plan that anticipated a failure mode — what to show when there is no context to personalize with. It did not decide that suggestion cards were worth keeping, or that the 160 % lift justified the added complexity, or that the experiment should run longer than a day. Those judgments stayed with the team, which is where they belong.
The deeper constraint is upstream. Wave works on products that already generate session replays, analytics, and code — the raw material it combs through. A team without instrumentation, or with instrumentation that captures the wrong events, gives the agent nothing to investigate. In my reading, the tool amplifies an existing feedback loop; it does not create one. That is a less glamorous framing than “self-improving products,” but it is the one the evidence supports. The gain is real and measurable: a month of handoff compressed into days, a silent user behavior surfaced, a small fix shipped and validated. In my view, what remains unproven is whether those gains compound into products that are meaningfully better, or merely faster to adjust. The homepage card is a promising data point, not a verdict.