June-July 2026: Latest Trends - Can AI Companies Create a World Where Models Cannot Be Stolen?

Hello, this is Kuro-Pug🐾
This time, we're talking about distillation. As someone who loves alcohol, the word 'distillation' immediately makes me think of whiskey, but today's distillation is almost a literal translation.

The safe hasn't been cracked. The model weights haven't leaked.
Yet, the intelligence might have been stolen.
The protagonist of this story isn't a spy sneaking into a server room. It's a seemingly legitimate intruder who keeps throwing massive amounts of questions at an API.

According to a Reuters report on June 24, 2026, Anthropic claimed in a June 10 letter to U.S. Senate Banking Committee Chairman Tim Scott and Ranking Member Elizabeth Warren that operators 'associated with' Alibaba and its research institute Alibaba Qwen used approximately 25,000 unauthorized accounts to conduct over 28.8 million interactions with Claude between April 22 and June 5, 2026, aiming to extract Claude's capabilities.

This is an incident that Anthropic has characterized as the 'largest-scale distillation attack against the company to date.'
Let me add a disclaimer here first. As of now, this is a one-sided claim from Anthropic. At the time of the Reuters report, Alibaba had not responded to requests for comment. The phrasing in Anthropic's letter also refers to 'operators associated with Alibaba and its research institute Alibaba Qwen,' and it has not been confirmed that Alibaba itself systematically led the attack. The possibility that a third party abused Alibaba's ecosystem or brand cannot be ruled out at this stage.

Even so, this incident is a watershed moment for the AI industry.
This is because what is happening here is the outline of a new attack surface in the LLM API era that goes far beyond a 'violation of user terms of service.'

Chapter 1: What is Distillation in the First Place?

First, I don't want to start by treating distillation as a villain.

Knowledge distillation is originally a legitimate compression and acceleration technique that transfers the behavior of a strong teacher model to a smaller student model. Classically, Hinton, Vinyals, and Dean (2015) "Distilling the Knowledge in a Neural Network" is a representative example, where the idea was to compress the knowledge of huge ensembles or high-performance models into more manageable models.

Frontier AI companies still use distillation daily to make their models smaller and faster. Anthropic is no exception. Haiku could be called a distilled version of Opus (though strictly speaking, the pipeline is more complex, but conceptually it holds). If AI companies were to reject distillation itself, they would be shooting themselves in the foot.

The problem is not the technology. It's where you collect the output from and what you train on.
Chapter 2: So, What is the Problem?

There are two points of contention.
First: Are they using another company's API output as a training target for their own competing model?
This is an action explicitly prohibited by Anthropic's Help Center. Using Claude's output to train competing models, general-purpose chatbots, or open-ended generative models, or using the output as a training target, is a violation of the terms of service. On the other hand, use for non-competing downstream tasks such as sentiment analysis, classification, summarization, information extraction, semantic search, and anomaly detection is explicitly permitted as an example.

Second: Is it being done systematically, on a large scale, and over a long period?
If someone mixed one or two of Claude's outputs into their training data, it could be dismissed as an accident or internal rogue behavior. But a scale of 25,000 accounts, 28.8 million requests, 44 days, and concentration on specific capability areas is no longer an accident; it is a strategy.

In other words, the question here is not 'Did they distill Claude?' but 'Did they systematically operate in a way that aimed to replicate competing general-purpose capabilities for the purpose of distilling Claude?'
If we don't distinguish this, the discussion becomes very crude.

Chapter 3: The Modus Operandi of Mission: Distill Possible

Let me write this a bit like a movie.
The spy doesn't break into the safe. Instead, they come to the reception desk every day and ask questions with a smile.
One day it's code correction, another day it's agent design, another day it's breaking down long-duration tasks, another day it's explaining reasoning steps, and another day it's how to write evaluation rubrics.
A single question looks normal. In fact, they look like a diligent engineer or a sincere researcher.

However,
tens of thousands of accounts
tens of millions of requests
concentrated on the same capability area
with similar prompt structures
thrown with synchronized access patterns
At this point, it is no longer normal usage, and theoutline of 'capability extraction'emerges.
According to Anthropic's claims, the targets in the Alibaba incident were Claude's most valuable areas—software engineering, advanced reasoning, and long-duration task management. Anthropic positions this as a strategy to accelerate the time it takes for China to catch up to Anthropic's advanced model, Mythos Preview.

By the way, in another case announced by Anthropic in February 2026—a campaign by DeepSeek, Moonshot, and MiniMax—the scales were over 150,000, over 3.4 million, and over 13 million respectively, totaling about 16.5 million. The 28.8 million in the Alibaba incident exceeds that total in a single campaign.

The scale has gone up by an order of magnitude.
Chapter 4: This is the LLM Version of the Long-Standing Model Extraction Problem

Let me add some technical depth.
In 2016, Tramèr et al. showed in "Stealing Machine Learning Models via Prediction APIs" (USENIX Security) that ML models could be replicated solely from public prediction APIs. The issue raised at the time was that ML-as-a-Service is not 'secure because it's a black box.'

What has changed in the LLM era?
Traditional model extraction was about reconstructing decision boundaries—that is, 'if you collect enough API probability outputs or labels, you can create a similar classifier.' It was centered on classification problems.

In LLMs, the output itself—code, reasoning chains, evaluation criteria, tool usage policies, long-form answers—has commercial value as training data.
In other words,
Traditional: Reconstruction of boundaries—stealing 'which class to classify into'
LLM Era: Collection of behaviors, skills, and work procedures—stealing 'how to think and how to write'
This is a qualitative leap. What is being stolen is not a 'judgment' but 'work'.

And to make matters worse, attackers no longer even need to reproduce it themselves. As Anthropic's investigation suggests, in the Chinese gray market, stolen credentials and model replacements are combined to resell access to the Claude API at a 90% discount, and the user's prompts and outputs themselves are further resold as AI training data.
Distillation attacks have already become an underground economy.

Chapter 5: Why is Detection Difficult?

This is the core of LLM API Security.
In SQL injection, the payload smells. You can see ' OR 1=1 -- if you look.
In DDoS, the traffic spikes. The graph tells you.
But in a distillation attack, if you look at just one request, it looks like a 'hardworking user'.

Anthropic explains that through proxy services and hydra-cluster-type unauthorized account networks, massive access is distributed across many accounts and blends into legitimate usage.
In other words, defense is absolutely not enough with simple Rate Limits.

What is needed is:
Cross-account correlation: Are different accounts really different people?
Prompt structure fingerprinting: Is it templated, or is the vocabulary distribution biased?
Bias in capability areas: Normal users don't just hit agentic reasoning this much.
Time synchronization: Are tens of thousands of accounts only waking up during specific time slots?
Payment and infrastructure indicators: Do billing paths, IP blocks, and device fingerprints belong to the same cluster?
Output harvesting patterns: Anomalies in streaming interruption rates, retry rates, and long-form request rates.
You need to look at these across the board.

To rephrase for engineers, Token Bucket is not enough; you need:
Semantic Rate Limit (throttling based on semantic proximity)
Capability Budget (budget management by capability category)
Account Graph Analysis (relationship graph analysis between accounts)
Prompt Shape Fingerprint (fingerprinting of prompt structure)
Output Harvesting Detection (detection of output collection patterns)
You can't protect it without doing all of this. There was a time when Web API Security was covered by the OWASP Top 10. LLM API Security is not enough even with the OWASP Top 10 for LLM.

Chapter 6: Defense Measures — LLM API Security Stack

Let me summarize the responses Anthropic has announced.
Dedicated classifiers to detect distillation patterns
Behavioral fingerprinting (behavioral fingerprinting per account/session)
Detection of chain-of-thought elicitation (identifying prompts that intentionally try to extract reasoning chains)
Detection of massive account coordination (dismantling hydra clusters)
Intelligence sharing with other companies, cloud providers, and authorities
Strengthened verification for education, research, and startup accounts
Multi-layered defense at the API, product, and model levels
What I want to highlight is that this is organized as a problem that can never be solved by a single company alone. Anthropic is requesting the Senate to strengthen threat intelligence sharing with the U.S. government, with EO 14409 in the background. In short, LLM API Security has already entered a framework of cross-industry and state involvement, rather than just a vendor's own security operations.

Chapter 7: Regulation and Security

On June 2, 2026, the U.S. government issued Executive Order 14409, "Promoting Advanced Artificial Intelligence Innovation and Security." There are three pillars.
Strengthening cyber defense of government information systems
A voluntary framework for the secure deployment of frontier AI models (a voluntary framework providing the government with access for up to 30 days before release)
Strengthening crackdowns on criminal acts abusing AI
EO 14409 does not regulate export controls themselves (I note this because it is often misunderstood). Export control is a separate movement. On June 12, 2026, two days after Anthropic sent the letter, the Department of Commerce imposed restrictions on Anthropic's latest Mythos / Fable 5 models due to concerns about their use by military and intelligence agencies in China and other countries of concern, and Anthropic disabled access to those models globally.
In other words, in a few weeks in June 2026, a chain of events occurred:
6/2: Frontier model evaluation framework via Executive Order
6/9: Pentagon adds Alibaba to the 'Chinese military companies' list (Alibaba is challenging this)
6/10: Anthropic sends letter to Senate regarding Alibaba distillation incident
6/12: Department of Commerce restricts distribution of Mythos / Fable 5
6/24: Reuters reports on the letter
The Anthropic letter is placed within this flow.

Distillation attacks are no longer just a violation of terms of service. They stand at the intersection of export control, cloud access, frontier model evaluation, and national security.

Chapter 8: But is that possible in the era of open models?

Let me balance this out.
Honestly, capabilities cannot be completely contained.
The moment a model is released as an API, it leaks its behavior to the world. Even if you protect the weights, the answers, reasoning styles, code correction capabilities, and agent design capabilities are observable. Things that are observable can be approximated if there is a sufficient number of samples. This is an implication of information theory.
In this era where open-weight models exist in large numbers globally, synthetic data generation technology is maturing, and evaluation benchmarks are becoming shared assets, the scenario of 'our model's capabilities are completely secret' no longer holds.

Therefore, what we should aim for is not a 'world where models cannot be stolen.'
Raising the cost of stealing
Detecting early
Layering terms, technology, infrastructure, and policy
Continuously reproducing our competitive advantage on the premise that it will be stolen
These are the four points. It is not Mission Impossible. It is Mission: Distill Possible. Perfect defense is impossible, but making attacks high-cost is possible, and that is a realistic strategy.

And one more important perspective at the end.
The fact that cutting-edge models like Claude are wanted as training data is not a humiliation, but an evaluation. No one steals what is not worth stealing. The problem is not the theft itself, but that the incentive for development investment is structurally damaged as a result of being stolen. In the long run, that makes AI research as a whole poorer.

Therefore, this is not just a story of 'U.S.-China confrontation.'
It is a story for the entire AI industry about how to design the structure of 'those who create capabilities' and 'those who harvest capabilities.'

Conclusion

For AI companies from now on, what needs to be protected is not just model weights.
What leaks beyond the API,'behavior',is also intellectual property, a security asset, and a target for attack.
Thieves no longer break into safes.
They ask questions.
Millions of times.
Sources

Anthropic Official "Detecting and countering misuse of AI" (Published February 2026, DeepSeek / Moonshot / MiniMax incident, definition of distillation, detection/defense measures)

Reuters (Reported June 24, 2026, Anthropic letter, claims of Alibaba / Qwen-related campaign, over 28.8 million requests / approx. 25,000 unauthorized accounts / April 22 - June 5, 2026)

CNBC / Tom's Hardware (Supplementary report on the same incident, targeted capability areas: software engineering / advanced reasoning / long-duration task management)

Claude Help Center (Prohibition of using Claude output for training competing models, permitted non-competing uses)

Hinton, Vinyals, Dean (2015) "Distilling the Knowledge in a Neural Network"

Tramèr et al. (2016) "Stealing Machine Learning Models via Prediction APIs" (USENIX Security)

White House Executive Order 14409 (June 2, 2026, "Promoting Advanced Artificial Intelligence Innovation and Security")

Department of Commerce Mythos / Fable 5 distribution restrictions (June 12, 2026)

Pentagon "Chinese military companies" list update (June 2026, Alibaba added — the company is challenging this)

#AI #LLM #Claude #Anthropic #Alibaba #Qwen #AISecurity #LLMSecurity #API #APISecurity #DistillationAttack #KnowledgeDistillation #ModelExtraction #AISecurity #Engineer #LLMEngineer #GenerativeAI #MachineLearning #CyberSecurity #ZeroTrust #USChinaAICompetition #FrontierModels #MythosPreview #Fable5 #EO14409 #ExecutiveOrder #