🤗 Evaluate Adds Bias Evaluation Feature

As language models grow in scale, the issue of Bias inherent in models and training data is becoming increasingly important. To detect and understand this, Hugging Face has introduced new bias measurement metrics to the 🤗 Evaluate library.

This update focuses on evaluating bias by leveraging the text generation capabilities of Causal Language Models (CLMs) such as GPT-2 and BLOOM. The evaluation workflow proceeds in the following two stages:

• Prompting: Feed the model using a predefined set of prompts hosted on 🤗 Datasets
• Evaluation: Measure the generated outputs with specific metrics using 🤗 Evaluate

The three main harmful language-related tasks addressed are as follows:

• Toxicity: Evaluates the likelihood of the model generating problematic content such as hate speech
• Polarity: Measures the leaning of the generated text
• Hurtfulness: Evaluates whether the model generates content that could hurt others

For example, by using prompts from the WinoBias dataset to measure the difference in GPT-2's generated outputs according to changes in gender pronouns (he/she) with a Toxicity score, the model's bias can be quantified.