The Point: The term “distillation attacks” for Chinese hacking attempts risks permanently tarnishing a legitimate AI research technique. Precise terminology is essential to avoid stigmatizing fundamental development methods.
Calling Chinese hacker attempts “distillation attacks” is misleading. It risks permanently linking a fundamental AI development technique to criminal behavior—and could endanger important research in the process.
The term “distillation attacks” is a poor choice for the current security incidents. While Chinese labs are indeed hacking and manipulating APIs to extract more intelligence from model services—and countering this is critical for America’s AI lead—this naming permanently links the entire concept of distillation to malicious behavior, even though distillation itself is a fundamental technique necessary to broadly distribute AI capabilities across academia and industry.
A similar terminology collapse already occurred in the open-source versus open-weights debate: all terms merged into “open models”—hardly anyone in the broader AI community still understands the distinction. Terminology is crucial: less informed people who shape and define technology are constrained by the available terms. Without careful discussion about distillation, we risk perceiving this versatile research technique as manipulation or even criminality.
Anthropic recently published a blog post about “distillation attacks” by three Chinese labs. These labs use the distillation technique, where a weaker model is trained on the outputs of a stronger one—a legitimate method, but manipulatively deployed in this case.