Multi-agent Based Deep Reinforcement Learning for Risk-shifting Portfolio Management

• Main Authors: Lin, Yu-Cen, 林昱岑
• Other Authors: Chen, An-Pin
• Format: Others
• Language: en_US
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/nfe7z3

碩士 === 國立交通大學 === 資訊管理研究所 === 107 === The growing popularity of quantitative trading, pursuing a systematic and algorithmic approach to invest, has drawn considerable attention among traders and investment firms nowadays, especially in the demand of investors for quant hedge fund. In this thesis, we consider the problem of multi-period portfolio selection with realistic transaction cost model, which is one of the major concerns for quant hedge fund managers. We develop a dedicated multi-agent based deep reinforcement learning framework with a two-level nested agent structure to learn an effective portfolio management with different objectives. With the aim of efficiently capturing specific asset property in portfolio and learning risk-shifting behavior automatically in money management, each agent is equipped with elaborating deep policy networks and a special training method that enables the proposed RL agent to learn risk-shifting behaviors with the stable convergence, which is of importance especially in the long-only portfolio management. We find that the introduction of prior knowledge in money management has a significant impact on the risk-shifting behavior of our learning agent, which acts as a guideline during the learning process. Furthermore, our experimental results reveal the effectiveness of our proposed framework, which outperforms all of surveyed well-known or representative portfolio selection strategies on most risk metrics and absolute returns. We obtain a leap of 37% relative improvement in the risk-adjusted Sharpe ratio, as well as with 8% relatively higher in the annual return, over the previous state-of-the-art.