Semgrep: GLM 5.2 Outperforms Claude on Its Own Cyber Benchmarks

According to Semgrep's IDOR (Insecure Direct Object Reference) detection benchmark results, Zhipu AI's open-weight model GLM 5.2 recorded an F1 score of 39%, surpassing Claude Code (32%). This is an unusual result in which an open-weight model outperformed a frontier model in an environment provided with only a simple prompt.

Key features of GLM 5.2:

• MoE (Mixture-of-Experts) architecture: Total parameters are approximately 750 billion, but active parameters per token are optimized to the 40 billion level, reducing inference cost.
• Extended context: Supports up to 1M (1 million) tokens, enabling reliable context retention even over long agentic task trajectories.
• High cost-efficiency: Can be operated at approximately 1/6 the cost compared to existing frontier models.
• Open weight: Released under the MIT license, making local execution and fine-tuning easy in security-sensitive environments.

This result suggests not only the performance of the model itself, but also how important the role of the harness (scaffolding) wrapping the model is in security tasks. Semgrep's multimodal pipeline recorded the highest performance, with F1 scores of 53~61% in a structured environment.