AI Model Risk Assessment Standard RiskRubric.ai

To address the challenge of selecting security- and safety-verified models among the vast number of models on the Hugging Face Hub, RiskRubric.ai has been released. This initiative, led by Cloud Security Alliance and Noma Security, evaluates AI model risk in a standardized way.

RiskRubric.ai evaluates models across 6 core axes: transparency, reliability, security, privacy, safety, and reputation. Using Noma Security's automation technology, it performs the following verifications:

• 1,000+ reliability tests: Checking the model's consistency and its ability to handle edge cases
• 200+ adversarial security probes: Testing for jailbreaks and prompt injection
• Automated code scanning and documentation review: Verifying model components and training data
• Privacy and safety evaluation: Testing for data leakage and harmful content generation

Results are calculated as a 0-100 score for each category, and ultimately provided as an A-F grade. This allows developers to filter and select models suited to a specific use case (e.g., a privacy-enhanced model for medical use).

According to initial analysis, many open source models scored higher than closed models in terms of transparency. However, since mid-range (C/D grade) models may have security gaps, it is recommended that companies set a minimum score threshold (e.g., 75 points) before deploying a model.