Exact Expressions for Kullback–Leibler Divergence for Multivariate and Matrix-Variate Distributions

Abstract

The Kullback–Leibler divergence is a measure of the divergence between two probability distributions, often used in statistics and information theory. However, exact expressions for it are not known for multivariate or matrix-variate distributions apart from a few cases. In this paper, exact expressions for the Kullback–Leibler divergence are derived for over twenty multivariate and matrix-variate distributions. The expressions involve various special functions.

Keywords

• beta function
• gamma function
• matrix-variate beta function
• matrix-variate confluent hypergeometric function
• matrix-variate gamma function
• matrix-variate Gauss hypergeometric function