Anthropic Identifies and Blocks Researchers Using “Claude” for Biological Weapons Research

Anthropic, the developer of the AI model “Claude,” has released a comprehensive report detailing the identification and subsequent suspension of researchers who were utilizing its AI models to advance research related to the potential development of biological weapons. Published on Thursday, the report presents five detailed case studies illustrating how current AI models can be misused. The Complexity of Bio-Security and Intent Anthropic highlights the inherent difficulty in identifying “biological misuse,” as the technical boundaries between legitimate research (such as vaccine development) and the development of biological weapons can be indistinguishable. Jacob Klein, Head of Threat Intelligence at Anthropic, told The New York Times that confronting these threats is rarely a matter of dealing with cartoonish actors stating malicious intent; rather, it involves subtle, complex interactions. Given the high-stakes potential consequences, Anthropic stated that it has adopted a policy of extreme caution in all such instances, resulting in the termination of the users’ accounts. Key Case Studies and Security Concerns: Chikungunya Virus Research: In a critical case from May, Anthropic’s biosafety classification system flagged a request asking “Claude” to draft a research proposal or grant application for “gain-of-function” research on the Chikungunya virus—a virus for which there is currently no approved vaccine or treatment. The proposal specifically aimed to enhance the virus’s transmissibility and its capacity to evade immune system responses. The project’s affiliation with a military research institution further heightened the company’s concerns. Toxins and Avian Influenza: The report also details concerns regarding a separate project involving gain-of-function research on avian influenza. Additionally, in another incident, a researcher utilized “Claude” to generate an atlas of toxic peptides found in venoms and to create a “generative pipeline” designed to optimize the toxic properties of those substances. Broader Scope of AI Misuse The scope of Anthropic’s report extends beyond biological threats. The findings document unauthorized use of AI models for various concerning applications, including: Deployment of AI as a tool for surveillance and monitoring. Creation of software vulnerabilities and exploitation workflows. Production of propaganda and deceptive advertising content. Development and optimization of conventional weapons systems. Anthropic confirmed that the accounts of the users involved in these activities have been blocked. The company stated that the insights gained from these investigations will be leveraged to refine the safety protocols of its models and prevent similar misuse in the future. Due to the sensitivity and security implications of these investigations, Anthropic has withheld the identities of the individuals and institutions linked to the research.