LG AI Research Presents ImagePiece Study for Efficient Image Recognition

Existing Vision Transformers (ViTs) often tokenize specific patches of an image that include meaningless backgrounds or irregular structures, resulting in excessive computational cost. In particular, existing token pruning methods have limitations in that they lose important information or dilute meaningful representations.

ImagePiece, proposed by LG AI Research, draws inspiration from NLP's Subword Tokenization approach and uses a strategy of re-tokenizing meaningless visual tokens into 'meaningful units.' Rather than simply reducing tokens, this technique newly redefines the meaning of tokens, reducing computation while preserving the model's representational power.

The core process of ImagePiece is as follows:

• Token importance evaluation: Calculates the importance of each token based on the Attention Score with the [CLS] token.
• Non-meaningful token merging: Merges low-importance tokens with similar tokens using the Bipartite Soft Matching technique.
• Meaning re-evaluation: Re-evaluates the Attention Score including the merged tokens, preserving tokens that have newly gained meaning while removing unnecessary tokens.

In addition, leveraging the fact that spatially adjacent patches are likely to be similar, LG AI Research introduced a Local Coherence Bias module. Through this, feature similarity between adjacent tokens is strengthened based on Overlapping Convolution, guiding meaningless tokens to be merged more naturally and meaningfully.