A proposal on centralised and distributed optimisation via proportional–integral–derivative controllers (PID) control perspective

Abstract

Abstract Motivated by the excellent performance of proportional–integral–derivative controllers (PIDs) in the field of control, the authors injected the philosophy of PID into optimisation and introduced two types of novel PID optimisers from a continuous‐time view, which benefit from the idea that discrete‐time optimisation algorithm can be modelled as a continuous dynamical system/controlled system. For centralised optimisation, the authors discuss the idea of the first‐order PID optimiser and the second‐order accelerated PID optimiser. Furthermore, this framework is extended into distributed optimisation settings, and a distributed PID optimiser is proposed. Finally, some numerical examples are given to verify our ideas.

Keywords

• control
• deep learning
• deep neural network
• machine learning