How AI Shipped 3,000 Changes to Speed Up Claude

Overview

Anthropic engineers recently executed a rapid two-week sprint that accelerated the core user experience of claude.ai and the Claude desktop application by approximately 3x. This initiative was driven by direct user feedback indicating that the platform felt sluggish, a concern the engineering team took seriously.

Rather than relying solely on traditional human-led optimization, the team integrated Claude (specifically a beta version of their internal research model, comparable in capability to Opus 5.5) into their daily workflow via Slack. Claude was tasked with identifying performance bottlenecks, constructing benchmarks, and shipping code changes autonomously.

The results were significant. Over the course of fourteen days, the team merged more than 3,000 changes without causing a single customer-facing incident or requiring a rollback. This collaboration between human oversight and AI-driven automation has saved an estimated tens of thousands of user-hours of waiting time daily.

The Sprint Strategy and Initial Goals

The project began with a clear mandate established in a dedicated Slack channel. The standing instructions directed Claude to facilitate all performance-related activities for the website and desktop app, including monitoring deployments, assessing telemetry, and proposing new optimization projects. While the long-term goal is full autonomy, the team recognized that human steering was necessary for this initial trial.

Measuring the Right Things

The first critical step was defining what "speed" meant for the user. The team analyzed usage data through the Datadog MCP server to identify the four highest-impact user journeys: launching the app, starting a new conversation, loading an existing conversation, and sending a message.

These journeys accounted for roughly 95% of user activity. To ensure accurate measurement, the engineers added instrumentation to create direct baselines for thirteen distinct metrics across both web and desktop environments. They disambiguated client-side work from server-side work to get a true picture of latency.

The sprint kicked off with a list of about twenty hand-picked projects. Claude estimated the impact of each in milliseconds, and the team aggregated these figures to set aggressive targets. The ambition was high, but the confidence in the measurement infrastructure was higher.

Early Successes

The initial phase of the sprint exceeded expectations rapidly. By day three, the team had hit twelve of their thirteen initial targets. Many of the planned optimizations landed early, including:

• Faster Launches: Engineers baked a static composer directly into the HTML, allowing users to begin typing during React initialization. They also precompiled a V8 code cache for the desktop shell’s main process to avoid recompilation.
• Smoother Navigation: The composer was kept mounted between conversations to prevent unnecessary unmounting. Sessions were prefetched when users hovered over them in the sidebar, and sidebar re-renders were cut by 90%.

However, Claude quickly identified additional opportunities beyond the original scope. These organic discoveries ramped up into full workstreams that far exceeded the initial targets, prompting the team to set new goals and look for further metrics to optimize.

Iterating Beyond Wall-Clock Timing

A major challenge in software performance is that wall-clock time is noisy. Milliseconds can fluctuate due to background processes, network variance, and system load, making it difficult to use as a strict CI gate. The team needed deterministic metrics that could serve as reliable guardrails.

Sam Attard, one of the engineers on the project, proposed exploring non-time-based measurements. The team shifted focus to lower-level metrics that could be measured consistently in a lab environment.

Sam: "What can we do instead of wall-clock timing? Can we measure JS instruction counts for instance?"

This question sparked a new wave of experimentation. The team realized that while browser paths lack direct instruction counting under Chromium, other deterministic counts were available.

The Ladder of Deterministic Metrics

In a matter of minutes, the team established five distinct measurement threads, each focusing on a different metric:

1. Instruction Counts: Using Valgrind with Node’s --predictable flag to count literal JS instructions for pure-JS hot paths.
2. V8 Call Counts: Leveraging V8’s precise coverage to count function calls.
3. React Commits: Measuring the number of React component commits per interaction.
4. Style Recalculations: Tracking layout and style recalc counts.
5. DOM Mutations: Counting direct mutations to the Document Object Model.

Each new benchmark had two jobs: it had to be a metric Claude could move in the lab, and it had to serve as a guardrail in continuous integration that could only improve (ratchet down). If a benchmark proved flaky or failed to correlate with actual user latency, the team discarded it rather than letting Claude optimize for the wrong outcome.

The team insisted on proof of correlation. They required Claude to demonstrate that hill climbing against these internal metrics resulted in measurable wall-clock improvements. Benchmarks that could not prove this link were unshipped to ensure integrity.

Results and Impact

The combination of high-level journey tracking and low-level deterministic metrics created a robust feedback loop. Claude operated asynchronously, often working for hours or overnight, validating prototypes without waiting for human review in real-time.

Key Performance Gains

The impact on the 75th percentile of user experience was dramatic. The following metrics illustrate the speed improvements achieved during the sprint:

• Time to Typeable Page: On a fresh load of claude.ai, the time dropped from 3.1 seconds to 0.55 seconds.
• New Claude Code Session: Starting a new session decreased from 0.8 seconds to 0.3 seconds.
• Loading a Cowork Session: Loading a cloud session in Claude Cowork fell from 2.6 seconds to 0.73 seconds.

These improvements were not limited to the initial targets. The total volume of changes—over 3,000 merges—demonstrates the scale of the optimization effort. The team maintained a zero-incident rate, proving that AI-assisted bulk changes could be deployed safely when coupled with rigorous human oversight and strong measurement frameworks.

Lessons Learned

This sprint highlights a shifting paradigm in software engineering. By treating AI as an autonomous agent within a constrained, well-measured environment, teams can iterate significantly faster than traditional development cycles allow.

The key takeaway is the importance of observability before automation. The team did not ask Claude to speed up the app without first establishing how to measure speed. By creating a ladder of deterministic metrics, they ensured that every improvement was real, correlated, and safe for users. As the team noted, once Claude could measure something, it could make it faster, leading them to continuously find new things to measure.