CI 70% Faster: Why Hand-Written Algorithms Beat AI Agents

Overview

In an era where coding agents promise to accelerate software delivery, a growing counter-movement suggests that human ingenuity still holds the edge in critical performance optimizations. A recent case study from engineer Serhii Shchoholiev demonstrates how hand-writing an algorithm reduced CI pipeline latency by 70%, outperforming the default suggestions of AI-driven development tools.

The incident highlights a recurring tension in modern engineering: while AI agents excel at boilerplate and rapid prototyping, they often lack the deep contextual understanding required to solve complex, combinatorial problems. Shchoholiev’s journey from a failed AI-generated solution to a custom Rust implementation offers a compelling argument for retaining human oversight in high-stakes infrastructure decisions.

The Problem with Automated Solutions

The story begins with a typical modern developer workflow. After adopting coding agents, the codebase expanded exponentially, leading to a surge in test counts and a corresponding degradation in CI speed. Engineers waited hours for PRs to merge, creating a bottleneck for product delivery. When prompted with a simple goal to speed up the pipeline, an AI agent proposed the standard industry solution: test sharding.

The agent quickly added a sharding flag to the test runner, reducing initial CI time to ten minutes. However, the stability of this solution was illusory. Within days, execution times became erratic, fluctuating between ten minutes and forty. Profiling revealed that the sharding algorithm distributed tests based on an obscure hash rather than actual execution duration. This led to severe load imbalance, with a single shard consuming disproportionate time while others idled.

"The solution isn't to switch models or reasoning levels - it's to use your brain."

This instability exposed a fundamental limitation of off-the-shelf AI tools. While effective at applying known patterns, the agent failed to recognize that naive hash-based sharding is insufficient for heterogeneous test suites. The engineer realized that addressing the root cause required a custom scheduling algorithm rather than a configuration tweak.

Designing a Custom Scheduler

Rather than adjusting parameters, Shchoholiev approached the issue as a classic computer science problem: multiprocessor scheduling. The objective was to distribute tests (processes) across CI runners (processors) to minimize total execution time. The initial approach involved sorting tests into shards using a max heap for tests and a min heap for workers, keyed by total duration.

While this greedy algorithm achieved a respectable $O(n \log n)$ complexity, it proved unsustainable. As the test suite grew, execution time scaled linearly, eventually returning to the original slow state. The engineer recognized that a static number of shards was the wrong constraint. Since compute resources can scale almost indefinitely, the goal shifted from balancing across a fixed number of shards to keeping each shard under a specific target time.

This change introduced new complexities. Tests are atomic units and cannot be split across machines. If a single test exceeds the target duration, the number of shards must be adjusted accordingly. Initial attempts to fix overflow issues by rerunning the entire algorithm resulted in quadratic time complexity, $O(n^2 \log n)$, which was worse than the original problem.

Optimizing Data Structures

To resolve the performance bottlenecks, the team needed a data structure capable of efficiently handling three operations: finding the smallest remaining capacity greater than a test’s duration, removing a shard, and inserting an updated shard.

A hash map offered fast insertion and removal but required linear time for search. A sorted array with binary search provided logarithmic search times but suffered from linear time complexity during insertion and deletion. The optimal solution lay in a balanced binary search tree, which provided $O(\log n)$ performance for all three operations.

By switching to a balanced BST, the overall algorithm complexity returned to $O(n \log n)$, ensuring that CI times would remain stable even as the test suite grew exponentially. The engineer implemented the solution in Rust, choosing the language for its zero-latency overhead and ability to run as a standalone binary without external dependencies.

The Human Element in Engineering

The successful resolution of the CI bottleneck underscores a broader philosophical point about the role of AI in software development. Shchoholiev noted that writing the code by hand felt refreshingly different from the "vibecoding" paradigm, where developers prompt agents and wait for results.

The iterative process of defining interfaces, writing unit tests, and solving small logical puzzles provided a steady stream of dopamine. This engagement with the problem space allowed for deeper understanding and more elegant solutions than those produced by black-box agents. The experience serves as a reminder that while AI can automate routine tasks, critical infrastructure optimizations often require the creative problem-solving and contextual awareness that only humans can provide.