Subgradient Descent Learning Over Fading Multiple Access Channels With Over-the-Air Computation

Abstract

We focus on a distributed learning problem in a communication network, consisting of $N$ distributed nodes and a central parameter server (PS). The PS is responsible for performing the computation based on data received from the nodes, which are transmitted over a multiple access channel (MAC). The objective function for this problem is the sum of the local loss functions of the nodes. This problem has gained attention in the field of distributed sensing systems, as well as in the area of federated learning (FL) recently. However, current approaches to solving this problem rely on the assumption that the loss functions are continuously differentiable. In this paper, we first address the case where this assumption does not hold. We develop a novel algorithm called Sub-Gradient descent Multiple Access (SGMA) to solve the learning problem over MAC. SGMA involves each node transmitting an analog shaped waveform of its local subgradient over MAC, and the PS receiving a superposition of the noisy analog signals, resulting in a bandwidth-efficient over-the-air (OTA) computation used to update the learned model. We analyze the performance of SGMA and prove that it has a convergence rate that approaches that of the centralized subgradient algorithm in large networks. Simulation results using real datasets show the effectiveness of SGMA.

Keywords

• Distributed learning
• gradient descent (GD)-type learning
• subgradient methods
• federated learning (FL)
• multiple access channel (MAC)
• over-the-air (OTA) computation