Optimal control of nonlinear Markov jump systems by control parametrisation technique

Abstract

Abstract This paper considers an optimal control problem of nonlinear Markov jump systems with continuous state inequality constraints. Due to the presence of continuous‐time Markov chain, no existing computation method is available to solve such an optimal control problem. In this paper, a derandomisation technique is introduced to transform the nonlinear Markov jump system into a deterministic system, which simultaneously gives rise to an equivalent deterministic dynamic optimisation problem. The control parametrisation technique is then used to partition the time horizon into a sequence of subintervals such that the control function is approximated by a piecewise constant function consistent with the partition. The heights of the piecewise constant function on the corresponding subintervals are taken as decision variables to be optimised. In this way, the approximate dynamic optimisation problem is an optimal parameter selection problem, which can be viewed as a finite dimensional optimisation problem. To solve it using a gradient‐based optimisation method, the gradient formulas of the cost function and the constraint functions are derived. Finally, a real‐world practical problem involving a bioreactor tank model is solved using the method proposed.