AI BriefingDP-FedSOFIM: Second-Order-Based Federated Learning Optimization
Key point
It proposes a federated learning optimization technique that applies second-order computation without increasing clients' communication load or privacy cost.
Details
Existing Differential Privacy (DP)-based federated learning mainly relies on first-order derivatives (Gradient), and under strict privacy budgets, noise tends to overwhelm valid gradient information, degrading learning efficiency.
DP-FedSOFIM has the following distinguishing features:
- Minimized client load: It transmits only the same level of data (Gradient vector) as the existing DP-FedGD, without increasing client-side memory or communication volume.
- Server-side curvature estimation: Curvature is estimated using a Fisher Proxy on the server rather than the client. The server maintains privacy-applied aggregated values via EMA (Exponential Moving Average), and uses the Sherman-Morrison formula to efficiently compute the Preconditioned step without directly constructing the matrix.
- Efficient optimization: Instead of computing the full Hessian matrix, it leverages curvature signals along the running direction, providing better learning dynamics even in environments where communication bottlenecks occur.
As a result, this approach improves training convergence speed and efficiency by leveraging the benefits of second-order computation, while maintaining the clients' level of privacy protection.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.