AI Briefing[ICML 2022] Part 1: Long-Tail Distribution Learning
Key point
We examine the concept and various methodologies of Long-Tail Distribution learning to address the data imbalance problem.
Details
Real-world industrial data often has Long-Tail Distribution (LTD) characteristics, where samples of certain classes are abundant (Head-class) while samples of certain classes are very scarce (Long-tail-class). This kind of data imbalance causes the model to be biased toward the majority classes during training, resulting in degraded performance on the minority classes.
Methodologies to address this can be broadly categorized into five types.
- Re-Sampling: A data-level approach, including Under-Sampling, which removes data to balance sample counts, and Over-Sampling, which duplicates or generates data.
- Cost-Sensitive Learning: An algorithm-level approach that applies differential Loss values per class to improve the model's generalization performance.
- Transfer Learning: An approach that leverages learned knowledge.
- Representation Learning: An approach that improves feature extraction capability.
- Decoupled Training: An approach that separates and optimizes the training process.
This article covers everything from these traditional techniques to SOTA methodologies from the latest Deep Learning models, and in particular introduces the PASCL (Partial and Asymmetric Supervised Contrastive Learning) paper presented at ICML 2022.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Sakana AI Releases 'Augmented Lagrangian Predictive Coding' Capable of Training 1000-Layer Neural Networks