[AAAI-25] ImagePiece: A New Content-Aware Re-tokenization Technique for Efficient ViT - LG AI Research BLOG

Existing Vision Transformer (ViT) models have a limitation in that dividing images into fixed-size patches generates many non-semantic patches, such as background regions, leading to increased computational cost. In particular, for images like industrial-site images that contain a lot of repetitive background, unnecessary tokens increase and efficiency drops sharply.

ImagePiece, proposed by LG AI Research, is a new re-tokenization technique inspired by NLP's Subword Tokenization approach. Unlike existing methods that simply remove or merge tokens, this technique reconstructs patches that can carry meaning when combined with surrounding information into a single 'Semantic Unit'.

**The key features of ImagePiece are as follows:

• Content-Aware: Accurately determines the importance of patches to efficiently handle non-semantic tokens.
• Re-tokenization: Minimizes information loss by combining non-semantic patches with surrounding information into a single meaningful token.
• Improved Efficiency and Performance: Enhances the model's structural efficiency while resolving the information loss problem present in existing token-pruning methods (DynamicViT, EViT, etc.), thereby improving performance.