The AI Distillation Arms Race: Why Model Theft Is Hard to Stop

Overview

American AI giants are facing a new threat to their dominance, one that operates not through hacking infrastructure but by quietly harvesting the intellectual output of their most valuable assets. In September, Anthropic and OpenAI released detailed reports exposing coordinated campaigns by foreign developers to "distill" their flagship models.

This practice involves using large, expensive models as teachers to train smaller, cheaper student models that mimic their capabilities. While distillation is a legitimate technique in computer science, the recent accusations suggest it is being weaponized as a form of industrial espionage. The core conflict highlights a growing tension between the open nature of API access and the proprietary secrets embedded within model weights and reasoning patterns.

"The text that it produces in response to your prompt is being stolen by some rival companies. But on a conceptual level, what you are stealing are capabilities." — Alexander Panfilov, Max Planck Institute for Intelligent Systems

What Is AI Distillation?

To understand the controversy, one must first look at the analogy behind the term. Just as a pot still separates alcohol from fermented mash by heating and cooling vapors, AI distillation condenses a massive, resource-heavy model into a compact version that retains its essential qualities. The process begins with a large model, known as the teacher, which generates responses to a wide array of prompts.

These responses are then fed into a smaller model, or student, to guide its training. This allows the student to learn from the teacher’s knowledge without undergoing the original, costly process of collecting data, finding patterns, and receiving human feedback. It is a form of reverse engineering that turns the output of an AI into the input for a copycat.

According to Yevgeniy Vorobeychik, a computer science professor at Washington University in St. Louis, teachers often reveal more than just final answers. They may provide fine-grained information, such as the probability assigned to each possible response. In some cases, as OpenAI noted, they release encrypted records of the step-by-step reasoning their powerful models use before arriving at a conclusion. This granular data is gold for anyone looking to replicate high-level performance at a fraction of the cost.

The Economic Stakes

The primary motivation behind these alleged thefts is economic efficiency. Smaller models are cheaper to run, faster to deploy, and easier to integrate into consumer applications. The concept itself is not new; researchers have studied neural network distillation since the 2000s to improve efficiency. However, the stakes have changed dramatically with the rise of large language models (LLMs).

In early 2025, the technique entered the mainstream spotlight when DeepSeek, a then-obscure Chinese startup, released a reasoning model called R1. The company claimed it had trained the model for less than $300,000, a tiny fraction of the billions spent by American labs. Allegations quickly surfaced that DeepSeek had trained R1 using responses extracted from OpenAI’s systems.

Since then, the term has become synonymous with accusations of model theft, largely directed at Chinese developers like DeepSeek and Moonshot AI, the maker of Kimi. In their September reports, Anthropic and OpenAI labeled these activities as illicit and unauthorized. While these companies have denied the accusations, the damage to trust and competitive advantage is significant.

American firms argue that if rivals can piggyback on their investments in compute and data, the incentive to innovate evaporates. The high valuations of companies like Anthropic and OpenAI rely on the exclusivity of their technological edge. If that edge can be siphoned off via API queries, the business model faces an existential threat.

How the Theft Occurs

Distillation typically requires thousands of prompts and corresponding responses to train a student model effectively. This process usually involves supervised fine-tuning, where the teacher’s answers serve as the target for the student to emulate. Over many iterations, the training nudges the student’s internal weights to match the teacher’s behavior.

Anthropic and OpenAI have described several creative and deceptive methods used to gather this data. According to Anthropic, some actors routed their own customers’ prompts through Claude, the company’s AI, and then saved the resulting responses to train their own models. This effectively turned paying customers into unwitting数据采集 points for competitors.

Other techniques involve crafting specific queries designed to extract sensitive information or reasoning traces that the models were not intended to reveal so fully. The attackers operate in a gray area, technically accessing the model through legitimate APIs but using the volume and nature of their requests to extract proprietary knowledge.

Why It Is Hard to Stop

Stopping distillation is exceptionally difficult because it exploits the very interface companies want the world to use. APIs are designed for accessibility and ease of integration. Restricting access too heavily could stifle legitimate innovation and alienate developers who rely on these tools for building new applications.

Furthermore, distinguishing between legitimate usage and malicious distillation is challenging. A company using an API to build a helpful chatbot looks similar to an attacker using the same API to harvest training data. The volume of requests may be high, but without clear behavioral markers, it is hard to block traffic without collateral damage.

There is also a broader paradox at play. Anthropic and OpenAI have themselves faced accusations of training their models on data scraped from the internet without authorization. This doubles standard complicates the moral high ground when accusing others of extracting value from their systems.

Experts suggest that no technical fix will fully solve the problem. As Alexandru Panfilov notes, the issue is not just about stolen text but stolen capabilities. Once a model knows how another model thinks, replicating its abilities becomes a matter of resources rather than original research.

Looking Ahead

The distillation arms race is likely to intensify as AI models become more capable and valuable. American companies are beginning to implement detection systems, rate limits, and watermarking techniques to identify and deter suspicious activity. However, these measures are often seen as a cat-and-mouse game where defenders must always stay one step ahead.

For the broader AI community, the situation raises critical questions about intellectual property in the age of generative AI. Who owns the output of a model? Can the patterns learned by a model be considered proprietary? These legal and ethical frameworks have not kept pace with the technology.

As the industry evolves, the balance between open access and proprietary protection will remain a central tension. The recent accusations from Anthropic and OpenAI signal that major players are willing to fight to protect their competitive edges, but they may find that in a digital world, keeping the contents of a black box secret is becoming increasingly impossible.