A Wavelet-Based Computational Framework for a Block-Structured Markov Chain with a Continuous Phase Variable

Abstract

We consider the computing issues of the steady probabilities for block-structured discrete-time Markov chains that are of upper-Hessenberg or lower-Hessenberg transition kernels with a continuous phase set. An effective computational framework is proposed based on the wavelet transform, which extends and modifies the arguments in the literature for quasi-birth-death (QBD) processes. A numerical procedure is developed for computing the steady probabilities based on the fast discrete wavelet transform, and several examples are presented to illustrate its effectiveness.

Keywords

• Markov chains
• stationary distribution
• wavelet transform
• numerical algorithm