A Low Complexity Approach to Model-Free Stochastic Inverse Linear Quadratic Control

Abstract

In this paper, we present a Model-Free Stochastic Inverse Optimal Control (IOC) algorithm for the discrete-time infinite-horizon stochastic linear quadratic regulator (LQR). Our proposed algorithm exploits the richness of the available system trajectories to recover the control gain $K$ and cost function parameters $(Q,R)$ in a low (space, sample, and computational) complexity manner. By leveraging insights on the stochastic LQR, we guarantee well-posedness of the Model-Free Stochastic IOC LQR via satisfaction of the Certainty Equivalence optimality conditions. The exact solution of the control gain $K$ is recovered via a deterministic, low complexity Least Squares approach. Using $K$ , we solve a completely model-free non-iterative SemiDefinite Programming (SDP) problem to obtain a unique (up to a scalar ambiguity) $(Q,R)$ , in which optimality and feasibility are jointly ensured. Via derivation of the sample complexity bounds, we show that the non-asymptotic performance of the Model-Free Stochastic IOC LQR can be characterized by the signal-to-noise (SNR) ratio of the finite set of system state and input signals. We present a model-based version of the algorithm for the special case where $(A,B)$ is available, and we, further, provide the extension to the Stochastic Model-Free IOC linear quadratic tracking (LQT) case.

Keywords

• Inverse optimal control
• knowledge acquisition
• linear systems
• optimization
• semidefinite programming (SDP)
• statistical learning