Riemannian SVRG Using Barzilai–Borwein Method as Second-Order Approximation for Federated Learning

Abstract

In this paper, we propose a modified RFedSVRG method by incorporating the Barzilai–Borwein (BB) method to approximate second-order information on the manifold for Federated Learning (FL). Moreover, we use the BB strategy to obtain self-adjustment of step size. We show the convergence of our methods under some assumptions. The numerical experiments on both synthetic and real datasets demonstrate that the proposed methods outperform some used methods in FL in some test problems.

Keywords

• second-order approximation
• Riemannian manifolds
• non-convex constraints
• stochastic variance reduced gradient
• Barzilai–Borwein method
• federated learning