Mutual information rate between stationary Gaussian processes

Abstract

Mutual information rate is an extension of the notion of mutual information to pairs of stationary stochastic processes. This quantity is defined as the time-averaged mutual information between corresponding segments of a pair of stationary stochastic processes, as the segment length tends to infinity. The contribution of this paper is to establish fairly mild regularity conditions for the existence of mutual information rate between Gaussian processes, and to rigorously derive an explicit formula to express this quantity in terms of the power and cross-power spectral densities of these Gaussian processes.

Keywords

• Entropy rate
• Power spectral density
• Szegö’s theorem
• Toeplitz matrix