Note on stability estimation of stochastic difference equations

Abstract

Stability estimates are proposed for two variants of Markov processes defined by stochastic difference equations: uncontrolled and controlled. Processes of this type are widely used in applications where their “governing distributions” are known only approximately, for example, as statistical estimates obtained from real data. Therefore, the problem of estimating deviations of output characteristics arises. The Kantorovich metric is used to measure the variations of probability distributions that govern the processes. In the uncontrolled case, the Kantorovich distance between the stationary distributions of the initial process and its perturbation is evaluated. On the other hand, the control processes being compared are endowed with an expected total discounted cost, and the inequality for the corresponding stability index is obtained. The stability index measures the increase in costs when using the control policy optimal for the “approximating process.”

Keywords

• kantorovich metric
• markov-controlled process
• stability index
• stability inequality
• stochastic difference equations
• 90c40
• 90c31