Adaptively Setting the Path Length for Separable Shadow Hamiltonian Hybrid Monte Carlo

Abstract

Hybrid Monte Carlo (HMC) has been widely applied to numerous posterior inference problems in machine learning and statistics. HMC has two main practical issues, the first is the deterioration in acceptance rates as the system size increases and the second is its sensitivity to two user-specified parameters: the step size and trajectory length. The former issue is addressed by sampling from an integrator-dependent modified or shadow density and compensating for the induced bias via importance sampling. The latter issue is addressed by adaptively setting the HMC parameters, with the state-of-the-art method being the No-U-Turn Sampler (NUTS). We combine the benefits of NUTS with those attained by sampling from the shadow density, by adaptively setting the trajectory length and step size of Separable Shadow Hamiltonian Hybrid Monte Carlo (S2HMC). This leads to a new algorithm, adaptive S2HMC (A-S2HMC), that shows improved performance over S2HMC and NUTS across various targets and leaves the target density invariant.

Keywords

• Hamiltonian Monte Carlo
• shadow Hamiltonian Monte Carlo
• Markov Chain Monte Carlo
• jump-diffusion processes
• NUTS