Research on LoRA-Based Fine-Tuning Poisoning Prevention Technique

A new approach has been proposed to defend against data poisoning and backdoor attacks that occur during the fine-tuning process. While existing methods focused on detecting malicious data, this research takes an approach that restricts the range of updates the model can learn to a subspace learned from trusted LoRA adapters.

Applying this technique provides the following benefits.

• Attack Blocking: Malicious updates end up in geometrically unreachable regions, drastically lowering the attack success rate
• Utility Preservation: The model's adaptation performance is maintained for tasks included in the adapter pool
• General Applicability: Can defend against risks that may arise during user-data-based fine-tuning or continual learning in on-device AI

As a result of the research team's experiments using 196 public LoRA adapters, the method showed strong defense performance even against adaptive attacks, and the related paper and code have been released.