Building Safeguards for Claude

Anthropic is focused on building Safeguards to prevent misuse and stop real-world harm so that Claude is used in ways that elevate humanity's potential. The Safeguards team, composed of experts in policy, product, data science, threat intelligence, and engineering, builds a defense system across the model's entire lifecycle.

The key strategies are as follows:

• Policy development: The team designs the Usage Policy and analyzes potential harms across five dimensions—physical, psychological, economic, societal, and individual autonomy—through the Unified Harm Framework. It also checks for policy weaknesses through Policy Vulnerability Testing, conducted in collaboration with external experts.
• Claude's training: The team works with the fine-tuning team to intervene from the training stage so that Claude does not engage in inappropriate behavior. When issues arise, they use methods such as updating reward models or adjusting the system prompts of deployed models.

In addition, to improve the accuracy of election information, the team is working with domain experts to deepen understanding of sensitive areas, such as partnering with the Institute for Strategic Dialogue to introduce banners that guide users to authoritative sources of information.