Utilising Partial Momentum Refreshment in Separable Shadow Hamiltonian Hybrid Monte Carlo

Abstract

Sampling using integrator-dependent shadow Hamiltonian’s has been shown to produce improved sampling properties relative to Hamiltonian Monte Carlo. The shadow Hamiltonian’s are typically non-separable, requiring the expensive generation of momenta, with the recent trend being to utilise partial momentum refreshment. Separable Shadow Hamiltonian Hybrid Monte Carlo (S2HMC) employs a canonical transformation which results in the Hamiltonian being separable and makes use of a processed leapfrog integrator. In this work, we combine the benefit of sampling using S2HMC with partial momentum refreshment to create the Separable Shadow Hamiltonian Hybrid Monte Carlo with Partial Momentum Refreshment (PS2HMC) algorithm which leaves the target distribution invariant. Numerical experiments across various targets show that the proposed algorithm outperforms S2HMC and Shadow Hamiltonian Monte Carlo with partial momentum refreshment. Comprehensive analysis is performed on the Banana shaped distribution, multivariate Gaussian distributions of various dimensions, Bayesian logistic regression and Bayesian neural networks.

Keywords

• Bayesian neural networks
• Bayesian logistic regression
• Hamiltonian Monte Carlo
• partial momentum refreshment
• shadow Hamiltonian Monte Carlo
• Markov Chain Monte Carlo