LeRobot Unveils New Dataset Format Based on Video Encoding

Unlike LLMs, the robotics field faces difficulties in securing vast amounts of high-quality data and managing it efficiently. Previously, visual data was stored by saving individual frames in PNG format, but this approach suffered from severe data redundancy and very large file sizes.

To address this, Hugging Face proposes the LeRobotDataset format. This format stores visual modalities using modern video codecs, providing the following performance improvements.

• Data Compression: Reduces dataset size to an average of 14% of the original, with compression down to 0.2% in the best case.
• Improved Loading Speed: Single-frame decoding speed is similar to PNG, but when decoding multiple consecutive frames, it is 25%~50% faster than PNG.

This technology leverages the principles of Spatial Compression, which exploits similarities within an image, and Temporal Compression, which records only the differences between frames. It is also designed for native integration with the Hugging Face Hub, making data sharing and visualization easy.