Red-Teaming Guide for Exploring LLM Vulnerabilities

LLMs generate realistic text, but they carry the risk of exhibiting inappropriate behavior, such as leaking personal information, showing bias, producing hate speech, or generating misinformation.

Red-teaming is an evaluation method that deliberately induces vulnerabilities in a model to uncover these risk factors. It is also closely related to jailbreaking, which involves breaking down a model's guardrails.

The differences between adversarial attacks and red-teaming are as follows:

• Adversarial attacks: Use random strings that humans cannot understand to degrade the model's performance.
• Red-teaming: Uses natural language prompts to induce the model to generate harmful content.

In the red-teaming process, the key challenge is balancing helpfulness, where the model follows the user's instructions well, with harmlessness, where it avoids giving harmful responses. Major techniques include roleplay attacks, which induce the model to play malicious characters.