Skip to main content
ConeReplay
U.S. Provisional 64/043,722 · Private beta

Ship AI agents with confidence.

Pre-deploy regression testing for multi-agent LLM systems. Replay proposed prompt, model, and agent changes against your recorded production traces — byte-identical where it doesn't need to change, using about 41% fewer tokens than replaying the whole run.

Request beta accessPaid migration audit →

The counterfactual gap.

Agent regressions surface in production. Faithful replay answers “what happened.” Naive full re-execution reintroduces sampling noise and burns tokens. Neither tells you whether your proposed prompt change would have broken customer 27 in yesterday's traces.

How it works.

  1. 1

    Record.

    Wrap your LangGraph app. ConeReplay captures every LLM call, tool invocation, and cross-agent message — each tagged with a per-turn causal vector clock.

  2. 2

    Replay with a twist.

    Propose a change. ConeReplay computes the causal cone of affected events. Out-of-cone events are served byte-identically from the recording. In-cone events re-execute with deterministic, content-hash-derived seeds.

  3. 3

    Review.

    Get a divergence report per trace — or aggregated across a corpus — straight into your PR. Ship only changes that do what you intended.

<1s
Cone computation for a 100-turn trace, single core
41%
Fewer provider tokens than full replay, measured live (up to 86% on late edits)
100%
Reproducibility across repeated replays

<1s and 100% measured on the v0.1 synthetic benchmark corpus. 41% measured live on 9 sanitized synthetic trace/edit cases, 598 provider calls each on gpt-4o-mini (41.2%) and claude-haiku-4-5 (40.9%); per-case range 11% to 86%.

Quickstart.

Install the published SDK, record a trace, then replay a proposed change against that corpus. The LangGraph extra is the install that matches the import below.

pip install 'conereplay[langgraph]'

# Record production traces
from conereplay import Recorder
from conereplay.sdk.langgraph import attach_to_langgraph_app

recorder = Recorder(store_path="./traces")
attach_to_langgraph_app(my_app, recorder)

# Replay a proposed change against the recorded corpus
conereplay corpus --traces ./traces --modify change.json --report report.html

Start with a paid pilot: the migration audit.

Model deprecations are forced migrations — another wave lands every few months. Before you flip the switch, know exactly which of your production conversations would have gone differently on the new model.

Two weeks, fixed scope

We take a slice of your recorded production traces, re-run them on the target model, and report — no open-ended engagement.

Per-trace outcome diff

Exactly which conversations end differently on the new model, side by side with the original run.

A go / no-go you can defend

Walk into the migration review with evidence from your own traffic, not a benchmark from someone else's.

Fixed scope, fixed price. Two pilot slots per month.

Ask about a pilot →

Request beta access.

We're onboarding design partners running multi-agent apps in production — LangGraph today, CrewAI and AutoGen next. Tell us about your stack and we'll be in touch within a day.

Further reading.