A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem

By Zhengyao Jiang, Dixing Xu, Jinjun Liang. 2017

471 citations recorded by the source corpus.

Financial portfolio management is the process of constant redistribution of a fund into different financial products. This paper presents a financial-model-free Reinforcement Learning framework to provide a deep machine learning solution to the portfolio management problem. The framework consists of the Ensemble of Identical Independent Evaluators (EIIE) topology, a Portfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL) scheme, and a fully exploiting and explicit reward function. This framework is realized in three instants in this work with a Convolutional Neural Network (CNN), a basic Recurrent Neural Network (RNN), and a Long Short-Term Memory (LSTM). They are, along with a number of recently reviewed or published portfolio-selection strategies, examined in three back-test experiments with a trading period of 30 minutes in a cryptocurrency market. Cryptocurrencies are electronic and decentralized alternatives to government-issued money, with Bitcoin as the best-known example of a cryptocurrency. All three instances of the framework monopolize the top three positions in all experiments, outdistancing other compared trading algorithms. Although with a high commission rate of 0.25% in the backtests, the framework is able to achieve at least 4-fold returns in 50 days.

Subfields: Finance, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Finance

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Deep Learning for Financial Applications : A Survey (2020, methodology): “[66] Cryptocurrencies, Bitcoin 2014-2017 Price data CNN, RNN, LSTM Accumulative portfolio value,”
Adversarial Deep Reinforcement Learning in Portfolio Management (2018, background): “The explanation can be reviewed in [10] and we are not going to illustrate them here.”
Deep hedging (2018, background): “the introduction of Kallsen and Muhle-Karbe (2015). For example, calculating the exponential utility indifference price for a call option in a Black–Scholes model involves solving a multidimensional non-linear free boundary problem, see e.”
Reinforcement Learning based Recommender Systems: A Survey (2021, background): “This made it possible to apply RL in problems with enormous state and action spaces, including self-driving cars [9, 10], robotics [11], industry automation [12], finance [13], healthcare [14, 15], and RSs.”
Recent advances in reinforcement learning in finance (2021, methodology): “The benchmark strategies include Constantly Rebalanced Portfolio (CRP) (Yu et al., 2019; Jiang et al., 2017) where at each period the portfolio is rebalanced to the initial wealth distribution among the assets, and the buy-and-hold or do-nothing strategy (Park et al., 2020; Aboussalah, 2020), which…”
Finding attractive technical patterns in cryptocurrency markets (2018, background): “Jiang et al. [9] proposed a deep reinforcement learning framework for portfolio optimization and applied it to the cryptocurrency market.”
Learning Action Representations for Reinforcement Learning (2019, background): “For example, maximizing long-term portfolio value in ﬁnance using various trading strategies [Jiang et al., 2017], improving fault tolerance by regulating voltage level of all the units in a large power system [Glavic et al., 2017], and personalized tutoring systems for recommending sequences of…”