LG AI Research GTC 2022 Review: Personalized Image/Language Model and Instant NeRF

This reviews the latest AI technology trends covered in NVIDIA GTC 2022 sessions. The main topics are the Personalized Image/Language Model, which finely reflects user needs, and Instant NeRF, which maximizes 3D reconstruction performance.

The Personalized Image/Language Model goes beyond the limitations of existing models that generate generic images, and can reflect specific concepts provided by the user (e.g., a photo of a particular cat). To achieve this, it proposes an Images-to-word Mapper, a network that maps concept images into the text embedding space to replace specific token embeddings, enabling users to generate or retrieve the precise images they want.

Instant NeRF dramatically improves the slow optimization time and runtime performance that were long-standing problems of existing NeRF (Neural Radiance Fields). Existing NeRF takes 3D coordinates and direction as input and represents RGB and density through an MLP, but it had the limitation of requiring enormous computation for rendering. NVIDIA overcame this and drew attention by presenting technology that enables training within a few seconds.