Optimal control of multiple Markov-switching stochastic systems with numerical applications

Abstract

In this article the authors set up an optimal control framework for a hybrid stochastic system with dual or multiple Markov switching diffusion processes, while Markov chains governing these switching diffusions are not identical as assumed in the existing literature. For this purpose, the authors first combine two or more separate Markov chains into one compound Markov chain within continuous time context and give out concrete procedures about how to estimate corresponding generator matrix of the newly combined Markov chain via a modified EM algorithm, then the HJB equations for the optimal control problem with synthetic Markov switching diffusion are stated. As an application and illustration, the authors lay out an adapted numerical solution scheme for the problem about optimal consumption–investment decisions under multiple cyclical or switching markets. Relevant economic analysis and interpretations are provided as results.

Keywords

• Dynamic programming
• Continuous time
• Combined Markov chains
• Regime switching
• Optimal consumption–investment