AI Biosecurity: The Dual Dilemma of Open-Source Models and Synthetic Screening

Overview

As artificial intelligence accelerates biological discovery, it also brings unprecedented biosecurity challenges. At the Global Challenges Project’s Biosecurity Workshop held in Washington, D.C. in September 2026, researchers noted that with advancing computational power, AI can now fold, design, and predict the functions of previously unforeseen proteins.

While the probability of designing lethal biological weapons remains low, Anthropic’s recent disclosure of malicious actors using Claude to design harmful proteins sends a clear warning. Additionally, tech leaders such as Dario Amodei, Sam Altman, and Elon Musk have called for slowing the pace of AI development and strengthening regulation. Faced with exponential technological progress, traditional biosecurity defense frameworks urgently need restructuring.

The Dilemma of Open-Source Models and AI Safety Frameworks

The current AIxBio ecosystem is deeply driven by open science, with the vast majority of cutting-edge biological protein models (including function predictors and sequence generators) released as open weights. While this model greatly accelerates scientific research, it fundamentally undermines the security paradigm of traditional closed-source large language models.

In traditional models, safety is usually implemented through prompt rejection at the API layer or linear probe training at the model level. However, open-weight models allow users to directly download and run models locally, rendering API-based control measures ineffective.

“We must redefine AI safety in biology along two directions: one is embedding protective measures into the model weights themselves so they can withstand malicious fine-tuning strategies; the other is strengthening control over the digital-physical interface.”

This means that future safety defenses must be moved upstream into the model architecture, ensuring that even under unauthorized fine-tuning, the model retains resistance to dangerous requests.

The Digital-Physical Interface: Challenges in Synthesis Screening

The transition from computational design to physical entities is the key bottleneck in current biological design and also the first line of defense for biosecurity. Currently, third-party DNA synthesis providers mainly rely on sequence-based screening, filtering harmful content by comparing ordered sequences against known danger databases.

However, in the context of AI being able to design entirely new functional sequences from scratch (de novo), sequence-based screening is becoming increasingly ineffective. New dangerous sequences may share no similarity with known hazards, thereby bypassing detection.

To address this challenge, researchers are exploring the following solutions:

1. Structure-based screening: Leveraging AI’s powerful protein structure prediction capabilities to compare structural features of the ordered sequences with structures of known dangerous proteins.
2. Mechanistic interpretability analysis: Utilizing current advances in mechanistic interpretability to decompose biological AI models into interpretable features, thereby detecting whether the model’s outputs contain features corresponding to known dangerous motifs.

Evaluation Difficulties Brought by Agent Frameworks

The emergence of agentic AI further blurs the boundary between the digital world and physical laboratories. Anthropic’s recent research demonstrates that after integrating specialized biological AI models, Claude can act as a fully autonomous “AI scientist,” independently designing de novo proteins, with experimental hit rates comparable to those of human scientists.

This discovery reveals two key risks: first, we may not fully understand the true capability boundaries of current AI models, making it impossible to accurately assess their safety; second, executing biological threats no longer relies solely on the tacit knowledge and intuition unique to human scientists, as agentic systems may interact more closely with the physical world.

Facing this trend, the biosecurity field urgently needs to establish stricter evaluation mechanisms to monitor the potential abuse risks of agents in complex biological experiments, and to build a dynamic and robust defense system against a backdrop where technological iteration far outpaces security development.