How Catastrophic Is Your LLM

C3LLM is a framework that, instead of measuring LLM safety with a single score, probabilistically certifies catastrophic risk across the entire space of conversations. It models the failure patterns of multi-turn conversations—which existing red-teaming can easily miss—as a graph, and applies Clopper-Pearson confidence intervals to the attack success rate to compute lower and upper bounds.

A conversation is represented as a graph with prompts as nodes and semantically connected relationships as edges. This allows the framework to simulate different levels of attacker capability, such as independent sampling, semantically chained path sampling, and adaptive steering sampling, and for each conversation sequence, a separate ChatGPT-based classifier determines whether the LLM's response is catastrophic.

UIUC researchers applied this framework to closed models such as Claude-Sonnet-4 and Nova Premier, as well as open-weight models. As a result, Claude-Sonnet-4 and Nova Premier were relatively safer, while Mistral-Large and DeepSeek-R1 showed higher risk.

In particular, Nova Premier showed a low risk level thanks to its built-in guardrails, which frequently blocked potentially dangerous outputs. In contrast, DeepSeek-R1 produced a certified lower bound exceeding 70% under the RNwJ distribution in cybercrime scenarios, clearly revealing the differences in safety across models.

The researchers open-sourced this framework, enabling industry and academia to conduct more rigorous LLM safety research.