NIST: GLM-5.3 is Top PRC Cyber AI but Trails US Frontier

Overview

The National Institute of Standards and Technology (NIST) has released a comprehensive assessment of GLM-5.3, an artificial intelligence model developed by the People’s Republic of China-based company Z.ai. The report, published on September 17, 2026, evaluates the model’s capabilities in cybersecurity tasks, highlighting its status as the most advanced open-weight AI for cyber operations to date.

However, the assessment also reveals a significant performance gap between GLM-5.3 and leading American artificial intelligence models. While GLM-5.3 leads its PRC peers, it trails the current U.S. frontier by approximately four months in aggregate cyber capability metrics. This gap underscores the ongoing technological competition between Washington and Beijing in the realm of autonomous offensive and defensive cyber tools.

Key Findings on Cyber Capabilities

GLM-5.3 was officially released by Z.ai on August 14, 2026, with its model weights made publicly available two weeks later. The Collaborative Center for AI Stability and Integrity (CAISI) at NIST analyzed the model across four rigorous benchmarks designed to test vulnerability discovery and exploit development.

GLM-5.3 is the most cyber-capable open-weight model released to date.

Despite this distinction, the data indicates that U.S. models maintain a substantial lead. When measured against a composite score of CAISI’s cyber evaluations, which accounts for task difficulty, the best-performing U.S. models outpace GLM-5.3 by a margin equivalent to four months of rapid technological advancement.

The assessment utilized an Item Response Theory (IRT) framework to generate a "cyber capability index." This statistical model treats each language model as having a latent capability level and each benchmark question as having a latent difficulty level. The probability of a model succeeding on a task is derived from the difference between these two values. According to the methodology, a 400-point increase on the capability index correlates to a tenfold increase in the statistical odds of solving complex cyber tasks.

Benchmark Performance Analysis

CAISI evaluated GLM-5.3 alongside the top-performing models from both the United States and China. The benchmarks tested included SEC-Bench Pro, ExploitBench, ExploitGym, and a private CAISI benchmark known as OSS-Fuzz. These tests required models to act as autonomous agents, utilizing bash and Python environments to locate security flaws and develop working exploits.

The results showed a clear hierarchy in performance:

• SEC-Bench Pro: In this test involving 183 tasks on browser source code, GLM-5.3 achieved a success rate of 40.4%. In contrast, the best U.S. model scored 90.2%, while the best previously evaluated PRC model scored only 27.3%.
• ExploitBench: Focused on developing exploits for known bugs in the V8 JavaScript engine, GLM-5.3 achieved a score of 61.1% (9.8 out of 16). The U.S. frontier model reached a perfect 100%, demonstrating superior proficiency in turning vulnerabilities into functional code execution.
• ExploitGym (Userspace): On a larger set of 502 tasks involving real-world open-source projects, GLM-5.3 succeeded in 9.4% of cases. The U.S. benchmark leader achieved a 44.4% success rate, more than quadrupling GLM-5.3’s performance.
• OSS-Fuzz: In this private evaluation involving 297 tasks where models had to find defects without prior descriptions or crash examples, GLM-5.3 scored 7.7%. The U.S. best-in-class model scored 23.2%, while previous PRC models struggled at just 2.4%.

Benchmark	GLM-5.3	U.S. Frontier Best	PRC Frontier Best
SEC-Bench Pro	40.4%	90.2%	27.3%
ExploitBench	61.1%	100.0%	32.2%
ExploitGym	9.4%	44.4%	2.6%
OSS-Fuzz	7.7%	23.2%	2.4%

All models were tested as agents in a ReAct harness with maximum reasoning settings. For U.S. models, cyber safeguards were disabled during the evaluation to ensure a direct comparison of raw capability. The PRC frontier best scores in this report are largely attributed to Kimi K3, which previously held the top position for open-weight models evaluated by CAISI.

Methodology and Implications

The rigorous methodology employed by CAISI ensures that the comparisons reflect genuine computational ability rather than superficial coding skills. The benchmarks were designed to be difficult, with high-difficulty tasks requiring multi-step reasoning and prolonged agent interaction. For instance, in ExploitBench, agents were allowed up to 300 turns to plan, execute, and refine their exploits, mirroring the implementation details of the original benchmark papers.

The findings have significant implications for national security and technological policy. The four-month lag between PRC and U.S. cyber AI capabilities suggests that while China is making rapid strides in open-weight model development, the United States retains a decisive edge in high-stakes offensive and defensive cyber operations.

The assessment highlights that while open-weight models are becoming more capable, U.S. frontier models remain the gold standard for cyber AI performance.

CAISI noted that the assessment does not compare GLM-5.3 to unreleased models, which may possess stronger capabilities. As both nations continue to invest heavily in AI research, the gap in cyber-specific competencies remains a critical metric for understanding the current state of the digital arms race. The release of detailed benchmarks and methodological notes by NIST provides a transparent baseline for future evaluations and international dialogue on AI safety and security standards.