Efficient Few-Shot Learning Framework SetFit Released

SetFit is an efficient few-shot fine-tuning framework that leverages Sentence Transformers to achieve high accuracy with just 8-16 labeled examples per class.

Unlike existing approaches, it requires no separate prompts or verbalisers, instead generating rich embeddings directly through a Sentence Transformer.

Key Features and Advantages:

• Fast training and inference: Since it does not use large-scale models (T0, GPT-3, etc.), training is much faster and more efficient.
• Multilingual support: Based on multilingual Sentence Transformers, it enables text classification across various languages.

Two-Stage Training Process:

1. Contrastive Fine-tuning: Using a small amount of data, the Sentence Transformer is fine-tuned so that similar sentences are placed closer together and different sentences farther apart.
2. Classification Head Training: A classifier is trained using the generated embeddings as input.

Performance Metrics: On the RAFT benchmark, SetFit (Roberta Large) matched the performance of the 11B-parameter T-Few model despite having only 355M parameters, and recorded higher accuracy than GPT-3.