Adaptive Magnetic Hamiltonian Monte Carlo

Abstract

Magnetic Hamiltonian Monte Carlo (MHMC) is a Markov Chain Monte Carlo method that expands on Hamiltonian Monte Carlo (HMC) by adding a magnetic field to Hamiltonian dynamics. This magnetic field offers a great deal of flexibility over HMC and encourages more efficient exploration of the target posterior. This results in faster convergence and lower autocorrelations in the generated samples compared to HMC. However, as with HMC, MHMC is sensitive to the user specified trajectory length and step size. Automatically setting the parameters of MHMC is yet to be considered in the literature. In this work, we present the Adaptive MHMC (A-MHMC) algorithm which extends MHMC in that it automatically sets the parameters of MHMC and thus eliminates the need for the user to manually set a trajectory length and step size. The trajectory length adaptation is based on an extension of the No-U-Turn Sampler (NUTS) methodology to incorporate the magnetic field present in MHMC, while the step size is set via dual averaging during the burn-in period. Empirical results based on experiments performed on jump diffusion processes calibrated to real world financial market data, a simulation study using multivariate Gaussian distributions and real world benchmark datasets modelled using Bayesian Logistic Regression show that A-MHMC outperforms MHMC and NUTS on an effective sample size basis. In addition, A-MHMC provides significant relative speed up (up to 40 times) over MHMC and produces similar time normalised effective samples sizes relative to NUTS.

Keywords

• Bayesian logistic regression
• jump diffusion processes
• adaptive
• Magnetic Hamiltonian Monte Carlo
• Markov Chain Monte Carlo