Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning

By G. Jeong, Ha Young Kim. 2019

188 citations recorded by the source corpus.

Abstract We study trading systems using reinforcement learning with three newly proposed methods to maximize total profits and reflect real financial market situations while overcoming the limitations of financial data. First, we propose a trading system that can predict the number of shares to trade. Specifically, we design an automated system that predicts the number of shares by adding a deep neural network (DNN) regressor to a deep Q-network, thereby combining reinforcement learning and a DNN. Second, we study various action strategies that use Q-values to analyze which action strategies are beneficial for profits in a confused market. Finally, we propose transfer learning approaches to prevent overfitting from insufficient financial data. We use four different stock indices—the S&P500, KOSPI, HSI, and EuroStoxx50—to experimentally verify our proposed methods and then conduct extensive research. The proposed automated trading system, which enables us to predict the number of shares with the DNN regressor, increases total profits by four times in S&P500, five times in KOSPI, 12 times in HSI, and six times in EuroStoxx50 compared with the fixed-number trading system. When the market situation is confused, delaying the decision to buy or sell increases total profits by 18% in S&P500, 24% in KOSPI, and 49% in EuroStoxx50. Further, transfer learning increases total profits by twofold in S&P500, 3 times in KOSPI, twofold in HSI, and 2.5 times in EuroStoxx50. The trading system with all three proposed methods increases total profits by 13 times in S&P500, 24 times in KOSPI, 30 times in HSI, and 18 times in EuroStoxx50, outperforming the market and the reinforcement learning model.

Subfields: Electrical and Electronic Engineering, Finance, Management Science and Operations Research, Reinforcement Learning, Economics & Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

In Finance

In Management Science and Operations Research

In Reinforcement Learning

In Economics & Operations Research

Citations in the corpus

Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019 (2019, methodology): “The authors of [123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 114] used S&P500 as their dataset.”
Deep Learning for Financial Applications : A Survey (2020, background): “[49] combined deep Q-learning and DNN to implement price forecasting and they intended to solve three separate problems: Increasing profit in a market, prediction of the number of shares to trade, and preventing overfitting with insufficient financial data.”
Recent advances in reinforcement learning in finance (2021, methodology): “Jeong and Kim (2019) designed a trading system based on DQN, which determines both the trading direction and the number of shares to trade.”
AI in Finance: Challenges, Techniques, and Opportunities (2021, methodology): “• Conducting time series forecasting by transfer learning across multiple sources or modalities of data [83, 116], e.”
Deep reinforcement learning for automated stock trading: an ensemble strategy (2020, background): “The critic-only learning approach, which is the most common, solves a discrete action space problem using, for example, Deep Q-learning (DQN) and its improvements, and trains an agent on a single stock or asset [9, 12, 21].”