LG AI Research 398

At NeurIPS 2023, AI-driven artistic creation—particularly audio and music generation technology—was a major topic of discussion. Google DeepMind's Lyria generates 30-second tracks that reflect a specific artist's voice and style, while Meta's Audiobox performs various audio tasks based on text, including speech, sound effects, and vocal style transformation.

LG AI Research's Data Intelligence (DI) Lab also presented research on an interface that generates 4-bar MIDI loops based on music metadata. This technology takes in various metadata such as instrument, tempo, and average pitch, enhancing the efficiency of music researchers and creators.

Multimodal generation research generally follows a structure consisting of a Prior that connects different modalities and an Encoder/Decoder that compresses data. This is similar to a VAE, but is designed to separate the training of the autoencoder and the Prior in order to maximize reconstruction fidelity and enable sampling from a meaningful space.

To process data efficiently, Discrete Representation technology plays a key role. Representative models include the following technologies:

• VQ-VAE: An autoencoder-family model that compresses and reconstructs data, representing each element with a finite number of codes
• VQ-GAN: A model that improves upon the performance of VQ-VAE
• Residual Vector Quantizer: A technology that is establishing itself as the standard for discrete representation in the audio domain