Published January 10, 2025 | Version v1
Publication Open

Hybrid-Quantum Neural Architecture Search for The Proximal Policy Optimization Algorithm

Authors/Creators

  • 1. ROR icon University of Aleppo

Description

Recent studies in quantum machine learning advocated the use of hybrid models to assist with the limitations of
the currently existing Noisy Intermediate Scale Quantum (NISQ) devices, but what was missing from most of them
was the explanations and interpretations of the choices that were made to pick those exact architectures and the
differentiation between good and bad hybrid architectures, this research attempts to tackle that gap in the literature
by using the Regularized Evolution algorithm to search for the optimal hybrid classical-quantum architecture for
the Proximal Policy Optimization (PPO) algorithm, a well-known reinforcement learning algorithm, ultimately
the classical models dominated the leaderboard with the best hybrid model coming in eleventh place among
all unique models, while we also try to explain the factors that contributed to such results, and for some models
to behave better than others in hope to grasp a better intuition about what we should consider good practices for
designing an efficient hybrid architecture.

Files

Hybrid-Quantum Neural Architecture Search for The Proximal Policy Optimization Algorithm.pdf

Files (1.6 MB)

Additional details

Software

Repository URL
https://github.com/moustafa7zada/Quantum-Hybrid-NAS-via-Regularized-Evolution
Programming language
Python

References

  • Killoran, N.: Hybrid Quantum-Classical Machine Learning. Xanadu. https://www.youtube.com/watch?v= t9ytqPTij7k Accessed 2020-11-17
  • Chen, S.Y.-C.: Asynchronous training of quantum reinforcement learning. arXiv. arXiv:2301.05096 [quant-ph] (2023). https://doi.org/10.48550/arXiv.2301.05096 . http://arxiv.org/abs/2301.05096 Accessed 2024-11-26
  • Chen, H.-Y., Chang, Y.-J., Liao, S.-W., Chang, C.-R.: Deep-Q Learning with Hybrid Quantum Neural Network on Solving Maze Problems. arXiv. arXiv:2304.10159 [quant-ph] (2023). https://doi.org/10.48550/arXiv.2304. 10159 . http://arxiv.org/abs/2304.10159 Accessed 2024-11-22
  • Chen, H.-Y., Chang, Y.-J., Liao, S.-W., Chang, C.-R.: Deep-Q Learning with Hybrid Quantum Neural Network on Solving Maze Problems. arXiv. arXiv:2304.10159 [quant-ph] (2023). https://doi.org/10.48550/arXiv.2304. 10159 . http://arxiv.org/abs/2304.10159 Accessed 2024-11-22
  • Drăgan, T.-A., Monnet, M., Mendl, C.B., Lorenz, J.M.: Quantum Reinforcement Learning for Solving a Stochas- tic Frozen Lake Environment and the Impact of Quantum Architecture Choices. arXiv. arXiv:2212.07932 [quant-ph] (2022). https://doi.org/10.48550/arXiv.2212.07932 . http://arxiv.org/abs/2212.07932 Accessed 2024- 11-26
  • Real, E., Aggarwal, A., Huang, Y., Le, Q.V.: Regularized Evolution for Image Classifier Architecture Search. arXiv. arXiv:1802.01548 [cs] (2019). https://doi.org/10.48550/arXiv.1802.01548 . http://arxiv.org/abs/1802. 01548 Accessed 2024-11-22
  • Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms. arXiv. arXiv:1707.06347 [cs] (2017). https://doi.org/10.48550/arXiv.1707.06347 . http://arxiv.org/abs/1707. 06347 Accessed 2024-11-22
  • Huang, S., Dossa, R.F.J., Raffin, A., Kanervisto, A., Wang, W.: The 37 implementation details of proximal policy optimization. In: ICLR Blog Track (2022). https://iclr-blog-track.github.io/2022/03/25/ppo-implementation- details/. https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/
  • Bergholm, V., Izaac, J., Schuld, M., Gogolin, C., Ahmed, S., Ajith, V., Alam, M.S., Alonso-Linaje, G., Akash- Narayanan, B., Asadi, A., Arrazola, J.M., Azad, U., Banning, S., Blank, C., Bromley, T.R., Cordier, B.A., Ceroni, J., Delgado, A., Di Matteo, O., Dusko, A., Garg, T., Guala, D., Hayes, A., Hill, R., Ijaz, A., Isacsson, T., Ittah, D., Jahangiri, S., Jain, P., Jiang, E., Khandelwal, A., Kottmann, K., Lang, R.A., Lee, C., Loke, T., Lowe, A., McKiernan, K., Meyer, J.J., Montañez-Barrera, J.A., Moyard, R., Niu, Z., O'Riordan, L.J., Oud, S., Panigrahi, A., Park, C.-Y., Polatajko, D., Quesada, N., Roberts, C., Sá, N., Schoch, I., Shi, B., Shu, S., Sim, S., Singh, A., Strandberg, I., Soni, J., Száva, A., Thabet, S., Vargas-Hernández, R.A., Vincent, T., Vitucci, N., Weber, M., Wierichs, D., Wiersema, R., Willmann, M., Wong, V., Zhang, S., Killoran, N.: PennyLane: Automatic differen- tiation of hybrid quantum-classical computations. arXiv. arXiv:1811.04968 [physics, physics:quant-ph] (2022). http://arxiv.org/abs/1811.04968 Accessed 2024-11-23
  • Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym. arXiv preprint arXiv:1606.01540 (2016
  • White, C., Safari, M., Sukthanker, R., Ru, B., Elsken, T., Zela, A., Dey, D., Hutter, F.: Neural Architecture Search: Insights from 1000 Papers. arXiv. arXiv:2301.08727 [cs, stat] (2023). https://doi.org/10.48550/arXiv. 2301.08727 . http://arxiv.org/abs/2301.08727 Accessed 2024-11-22