Q-learning with Continuous Action Value in Multi-agent Cooperation

• Main Authors: Yu-Hong Lin, 林咩
• Other Authors: Kao-Shing Hwang
• Format: Others
• Language: en_US
• Online Access: http://ndltd.ncl.edu.tw/handle/36513682200450671073

碩士 === 國立中正大學 === 電機工程所 === 94 === In this thesis, we propose a Q-learning with continuous action space and extend this algorithm to a multi-agent system. We implement this algorithm in a task that there are two robots taking action independently and both are connected with a straight bar. They must cooperate to move to the goal and avoid the obstacles in the environment. Conventional Q-learning needs a pre-defined and discrete state space, so it will have finite states and actions. But it is not practical because in real world the states of the environment and the actions are both continuous, so when we using Q-learning to demonstrate the action in the world, we can’t precisely identify the variances of the different situation in the same state. We use the concept of SRV (Stochastic Real-Valued Unit) to train the action in each state, so the result action will be continuous. It will make the simulation that more close to the real world; also it can fix the pre-defined action space in Q-learning, result in a more ideal learning outcome.