APPLICATION OF Q-LEARNING IN FINANCIAL MARKETS: MODELLING AND EXPERIMENTAL RESULTS
Authors/Creators
- 1. Assistant Professor, Department of Computer Science & Engineering, Punjabi University, Patiala, India
Description
The rapid growth of algorithmic trading and financial artificial intelligence has motivated the search for adaptive, data-driven decision-making techniques that can outperform traditional trading strategies. This paper investigates the application of Q-learning, a value-based reinforcement learning algorithm, to stock trading and portfolio management. The trading process is modelled as a Markov Decision Process, where states represent market indicators and technical signals, actions correspond to buy, sell, or hold decisions, and rewards are defined in terms of risk-adjusted returns. Using historical stock data obtained from the Yahoo Finance API, Q-learning agent is implemented and backtested against benchmark strategies such as Buy-and-Hold and Random trading. Experimental results demonstrate that the Q-learning framework can achieve competitive performance, with higher cumulative returns and improved Sharpe ratios, while also adapting to dynamic market conditions. The study contributes to the literature by providing a systematic implementation of Q-learning in financial markets, highlighting both its strengths and limitations. Furthermore, challenges such as data non-stationarity, sample efficiency, and risk management are discussed, while outlining potential extensions to advanced methods like Deep Q-Networks and Actor-Critic models. The findings underscore the potential of reinforcement learning as a promising paradigm for intelligent financial decision-making and provide valuable insights for traders, researchers, and policymakers.
Files
22-Brahmaleen Kaur Sidhu - Online - IERJ4101660508.pdf
Files
(387.0 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:00a90ca45a364249e1effbc8d1d78f12
|
387.0 kB | Preview Download |
Additional details
Identifiers
Related works
- Is published in
- Journal: 2454-9916 (EISSN)
References
- Cont, R. (2001). Empirical properties of asset returns: stylized facts and statistical issues. Quantitative Finance, 1(2), 223–236. https://doi.org/10.1080/713665670
- Dixon, M. F., Halperin, I., & Bilokon, P. (2020). Machine Learning in Finance. Springer International Publishing. https://doi.org/10.1007/978-3-030-41068-1
- Fischer, T. G. (2018). Reinforcement learning in financial markets - a survey (12; FAU Discussion Papers in Economics). https://www.iwf.rw.fau.de/research/iwf-discussion-paper-series/
- Huang, C. Y. (2018). Financial Trading as a Game: A Deep Reinforcement Learning Approach. In arXiv. http://arxiv.org/abs/1807.02787
- Jiang, Z., Xu, D., & Liang, J. (2017). A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem. In arXiv. http://arxiv.org/abs/1706.10059
- Markowitz, H. (1952). PORTFOLIO SELECTION*. The Journal of Finance, 7(1), 77–91. https://doi.org/10.1111/j.1540-6261.1952.tb01525.x
- Moody, J., & Saffell, M. (2001). Learning to trade via direct reinforcement. IEEE Transactions on Neural Networks, 12(4), 875–889. https://doi.org/10.1109/72.935097
- Sutton, R. S. ., & Barto, A. G. . (2020). Reinforcement learning : an introduction. The MIT Press.
- Tsantekidis, A., Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., & Iosifidis, A. (2017). Forecasting Stock Prices from the Limit Order Book Using Convolutional Neural Networks. 2017 IEEE 19th Conference on Business Informatics (CBI), 7–12. https://doi.org/10.1109/CBI.2017.23
- Watkins, C. J. C. H., & Dayan, P. (1992). Technical Note: Q-Learning. Machine Learning, 8(3–4), 279–292. https://doi.org/10.1023/A:1022676722315
- Wu, M.-E., Syu, J.-H., Lin, J. C.-W., & Ho, J.-M. (2021). Portfolio management system in equity market neutral using reinforcement learning. Applied Intelligence, 51(11), 8119–8131. https://doi.org/10.1007/s10489-021-02262-0
- Zhang, Z., Zohren, S., & Roberts, S. (2020). Deep Reinforcement Learning for Trading. The Journal of Financial Data Science, 2(2), 25–40. https://doi.org/10.3905/jfds.2020.1.030