Accelerated Gradient Descent Driven by Lévy Perturbations

Abstract

In this paper, we mainly consider two kinds of perturbed accelerated gradient descents driven by Lévy perturbations, which is of great importance for enhancing the global search ability. By using Lévy representation, Lévy perturbations can be divided into two parts: small jumps and large jumps, whose properties are then carefully discussed. By introducing the concept of attraction domain for local minima, Makovian transition properties are proven for the proposed two perturbed accelerated gradient descents with different infinitesimal matrices. Finally, all the results are extended to the vector case and two simulation examples are provided to validate all the conclusions.

Keywords

• accelerated gradient descent
• Lévy perturbations
• global optimization
• Markovian transition property