Making Unlearning More Efficient: Cutting Compute Costs with Low-Impact Points

Existing machine learning unlearning methods treated all data points slated for deletion equally. Apple researchers challenged this. Is it really necessary to remove data points that have almost no impact on model training?

The research team used influence functions to identify subsets of training data that have negligible influence on model outputs, in both language models and vision models. They proposed an efficient framework that removes these low-influence points beforehand, prior to unlearning.

In real-world use cases, this approach achieved compute cost reductions of up to 50%. Amid growing privacy concerns, it presents a new path to securing both privacy protection and computational efficiency at the same time.