Output Feedback H∞ Control for Linear Discrete-Time Multi-Player Systems With Multi-Source Disturbances Using Off-Policy Q-Learning

• Main Authors: Zhenfei Xiao, Jinna Li, Ping Li
• Format: Article
• Language: English
• Published: IEEE 2020-01-01
• Series: IEEE Access
• Subjects: Output feedback; H∞ control; adaptive dynamic programming; game theory; reinforcement learning
• Online Access: https://ieeexplore.ieee.org/document/9261350/

In this paper, a data-driven optimal control method based on adaptive dynamic programming and game theory is presented for solving the output feedback solutions of the H_∞ control problem for linear discrete-time systems with multiple players subject to multi-source disturbances. We first transform the H_∞ control problem into a multi-player game problem following the theoretical solutions according to game theory. Since the system state may not be measurable, we derive the output feedback based control policies and disturbances through mathematical operations. Considering the advantages of off-policy reinforcement learning (RL) over on-policy RL, a novel off-policy game Q-learning algorithm dealing with mixed competition and cooperation among players is developed, such that the H_∞ control problem can be finally solved for linear multi-player systems without the knowledge of system dynamics. Moreover, rigorous proofs of algorithm convergence and unbiasedness of solutions are presented. Finally, simulation results demonstrated the effectiveness of the proposed method.