A Holistic Approach to Detecting Harmful Content in Real-World Environments

We propose a holistic approach to building robust and useful natural language classification systems for real-world content moderation. The success of the system depends on a carefully designed series of steps.

The main process is as follows:

• Designing a Taxonomy and labeling guidelines
• Data quality control
• Building an Active Learning pipeline to capture rare cases
• Applying various methodologies to increase model robustness and prevent overfitting

This moderation system was trained to detect a wide range of harmful content categories, including sexual content, hate speech, violence, self-harm, and harassment. This approach can be universally applied to various classification taxonomies, and can produce high-quality classifiers that outperform existing off-the-shelf models.