[ICML 2021] Part 2: Generative Models for OOD Detection

Generative models estimate the distribution of data, and judge input data with low likelihood as Out-of-distribution(OOD) data. However, recent research has raised the issue that even models that estimate the distribution well assign high likelihood to OOD data.

This paper analyzes the cause of this phenomenon from the perspective of Hierarchical VAE. Experimental results confirmed that low-level features from lower layers struggle to distinguish between In-distribution and OOD data, while high-level features from higher layers can effectively distinguish between the two types of data.

When training a model using the existing ELBO(Evidence Lower Bound) formula, low-level features dominate the loss function, degrading OOD detection performance. To address this, a new OOD detection score was proposed, achieving a SOTA(State-of-the-art) record.