Reinforcement Learning in Portfolio Management

Bachelor Thesis, Bachelor in Data Science and Engineering, Universidad Carlos III de Madrid, 2022-2023. Defended September 2023.

This thesis investigates the potential of Deep Reinforcement Learning (DRL) to automate the task of portfolio management. Motivated by the observation that existing machine learning methods in finance act as supplementary tools for human experts rather than as independent decision-making systems, the study designs a DRL-based portfolio management model aiming at both automation and optimization.

Key components of the work include: a Temporal Convolutional Network (TCN) backbone pre-trained on a supervised return-forecasting task and transferred as a frozen state encoder; a Soft Actor-Critic (SACv2) agent with actor variants based on Normal and Dirichlet action distributions over the portfolio simplex; critic variants based on bilinear and single-head attention mechanisms; a confidence score that lets the agent decide whether to rebalance rather than only how, making the agent granularity-agnostic; random asset permutation during training to encourage company-agnostic policies; and mixup data augmentation across price channels.

Agents are benchmarked against equal-weight (with and without rebalancing), minimum-volatility and maximum-Sharpe efficient-frontier portfolios, and a random agent, over S&P 500 data split chronologically into training (1986-2008), validation (2009-2016) and test (2017-2023, unseen companies) partitions, each containing periods of market turbulence.

Limitations, including reliance on a simulated environment and survivorship bias in the data, are discussed in the scope and limitations section.