Reducing Unlearning Costs: Leveraging Low-Impact Data Points to Cut Computational Expenses

As data privacy issues in machine learning become increasingly important, the significance of Unlearning technology, which removes specific data points from trained models, is growing. However, existing state-of-the-art unlearning methods have the limitation of treating all data within the 'forget set' with equal weight.

This research raises the question of whether it is necessary to remove data points that have negligible impact on model training. The researchers conducted a comparative analysis of Influence Functions across language and vision tasks, identifying data subsets that have almost no effect on model outputs.

Based on these insights, they propose an efficient unlearning framework that pre-reduces the dataset size before performing unlearning. Experimental results in real-world environments showed that this approach achieved a reduction in computational costs of up to approximately 50%.