Filtering of hidden Markov renewal processes by continuous and counting observations

Abstract

This paper introduces a subclass of Markov renewal processes (MRPs) and presents a solution to the optimal filtering problem in a stochastic observation system, where the state is modeled by an MRP and observed indirectly through noisy measurements. The MRPs considered here can be interpreted as continuous-time Markov chains (CTMCs) with a finite set of abstract states representing distributions of random vectors. The paper outlines the probabilistic properties of MRPs, emphasizing the ability to express any arbitrary function of the MRP as the solution to a linear stochastic differential system (SDS) with a martingale on the right-hand side (RHS). Using these properties, an optimal filtering problem is formulated in stochastic observation systems, where the hidden state belongs to the class of MRPs, and the observations consist of both diffusion and counting components. The drift terms in all observations depend on the system state. An optimal filtering estimate for a scalar function of the MRP is provided through the solution of an SDS with innovation processes on the RHS. Additionally, the paper presents a version of the Kushner-Stratonovich equation, describing the evolution of the conditional probability density function (PDF). To demonstrate the practical application of the estimation method, the paper presents a communications-related example, focusing on monitoring the qualitative state and numerical characteristics of a network channel using noisy observations of round-trip time (RTT) and packet loss flow. The paper also highlights the robustness of the filtering algorithm in scenarios where the MRP distribution is uncertain.

Keywords

• hidden markov model
• markov renewal process
• martingale representation
• optimal filtering problem
• kushner-stratonovich equation