NEASQC project
Published February 24, 2021 | Version 1.0
Project deliverable Open

D6.1 QNLP design and specification

Creators

  • 1. ICHEC
  • 2. Tilde

Contributors

  • 1. University of Leiden
  • 2. CESGA

Description

Understanding the applicability of NISQ-era devices for a variety of problems is of the utmost importance to better develop and utilise these devices for real-world use-cases. In this document we motivate the use of quantum computing models for natural-language processing tasks, focussing on comparison with existing methods in the classical natural language processing (NLP) community. We define the current state of these NISQ devices, and define methods of interest that will allow us to exploit the resources to implement NLP tasks, by encoding and processing data in a hybrid classical-quantum workflow. For this, we outline the high-level architecture of the solution, and provide a modular design for ease of implementation and extension.

Files

NEASQC_D6.1_QNLP-design-and-formal-specification.pdf

Files (719.3 kB)

Additional details

Funding

NEASQC – NExt ApplicationS of Quantum Computing 951821
European Commission

References

  • Östlund, S. a. ((1995)). Östlund, Stellan, and Stefan Rommer. Thermodynamic limit of density matrix renormalization. Physical review letters 75.19 .
  • A. Cichocki, N. L.-H. (2016). Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges. arXiv:1609.00893.
  • Andrey Kardashin, A. U. (2018). Quantum Machine Learning Tensor Network States. arXiv:1804.02398.
  • Artetxe, M., & Schwenk, H. (2019). Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, (lpp. 3197-3203).
  • Artetxe, M., & Schwenk, H. (2019). Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond. Transactions of the Association for Computational Linguistics, 597-610.
  • Bai, G., Yang, Y., & Chiribella, G. (2020). Quantum compression of tensor network states. New Journal of Physics, 43015.
  • Balodis, K., & Deksne, D. (2019). Fasttext-based intent detection for inflected languages. Information, 10(5). doi:10.3390/info10050161
  • Bergholm, V., Izaac, J., Schuld, M., Gogolin, C., Alam, M. S., Ahmed, S., . . . Killoran, N. (2018). PennyLane: Automatic differentiation of hybrid quantum-classical computations. arXiv: 1811.04968.
  • Bharti, K. e. (2021). Noisy intermediate-scale quantum (NISQ) algorithms. arXiv:2101.08448.
  • Biamonte, J., & Bergholm, V. (2017). Tensor Networks in a Nutshell. arXiv: 1708.00006.
  • Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135-146.
  • Braun, D., Mendez, A. H., Matthes, F., & Langen, M. (2017). Evaluating natural language understanding services for conversational question answering systems. Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, (lpp. 174-185).
  • Brown, S. (bez datuma). The C4 model for visualising software architecture. Ielādēts no https://c4model.com/
  • Cerezo, M., Sharma, K., Arrasmith, A., & Coles, P. J. (2020). Variational Quantum State Eigensolver. arXiv: 2004.01372.
  • Chan, G. K.-L., Keselman, A., Nakatani, N., Li, Z., & White, S. R. (2016). Matrix product operators, matrix product states, and ab initio density matrix renormalization group algorithms. The Journal of Chemical Physics, 14102.
  • Choquette, A., Paolo, A. D., Barkoutsos, P. K., Sénéchal, D., Tavernelli, I., & Blais, A. (2020). Quantum-optimal-control-inspired ansatz for variational quantum algorithms. arXiv : 2008.01098.
  • Coecke, B., Felice, G. d., Meichanetzidis, K., & Toumi, A. (2020). Foundations for Near-Term Quantum Natural Language Processing. arXiv: 2012.03755.
  • Coecke, B., Sadrzadeh, M., & Clark, S. (2010). Mathematical Foundations for a Compositional Distributional Model of Meaning. arXiv: 1003.4394.
  • Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020). A Survey of the State of Explainable AI for Natural Language Processing. Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, (lpp. 447-459).
  • Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805.
  • Farhi, E., Goldstone, J., & Gutmann, S. (2014). A Quantum Approximate Optimization Algorithm. arXiv: 1411.4028.
  • Feng, F., Yang, Y., Cer, D., Arivazhagan, N., & Wang, W. (2020). Language-agnostic BERT sentence embedding. arXiv preprint arXiv:2007.01852.
  • Gambetta, J. M. (2020. gada 15. 09). IBM's Roadmap For Scaling Quantum Technology. Ielādēts no https://www.ibm.com/blogs/research/2020/09/ibm-quantum-roadmap/
  • Grimsley, H. R., Economou, S. E., Barnes, E., & Mayhall, N. J. (2019). An adaptive variational algorithm for exact molecular simulations on a quantum computer. Nature Communications, 3007.
  • Herasymenko, Y., & O'Brien, T. E. (2019). A diagrammatic approach to variational quantum ansatz construction. arXiv: 1907.08157.
  • Howard, J., & Ruder, S. (2018). Universal Language Model Fine-tuning for Text Classification. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), (lpp. 328-339).
  • Huggins, W., Patil, P., Mitchell, B., Whaley, K. B., & Stoudenmire, E. M. (2019). Towards quantum machine learning with tensor networks. Quantum Science and Technology, 24001.
  • IBM. (2020). IBM's Roadmap For Scaling Quantum Technology. Ielādēts no https://www.ibm.com/blogs/research/2020/09/ibm-quantum-roadmap/
  • IonQ. (2020). Scaling IonQ's Quantum Computers: The Roadmap. Ielādēts no https://ionq.com/posts/december-09-2020-scaling-quantum-computer-roadmap
  • Jurafsky, D., & Martin, J. H. (2017). Speech and Language Processing (3rd ed. draft). Prentice Hall.
  • Kissinger, A., & van de Wetering, J. (2020). PyZX: Large Scale Automated Diagrammatic Reasoning. Electronic Proceedings in Theoretical Computer Science, 229–241.
  • Kjaergaard, M., Schwartz, M. E., Braumüller, J., Krantz, P., Wang, J. I.-J., Gustavsson, S., & Oliver, W. D. (2020). Superconducting Qubits: Current State of Play. Annual Review of Condensed Matter Physics, 369-395.
  • Lambek, J. (2006). Pregroups and natural language processing. The Mathematical Intelligencer, 41-48.
  • Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019). ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. International Conference on Learning Representations.
  • McArdle, S. e. (2020). Quantum computational chemistry. Reviews of Modern Physics , 015003.
  • McClean, J. R., Romero, J., Babbush, R., & Aspuru-Guzik, A. (2016). The theory of variational hybrid quantum-classical algorithms. New Journal of Physics, 23023.
  • Meichanetzidis, K., Gogioso, S., Felice, G. D., Chiappori, N., Toumi, A., & Coecke, B. (2020). Quantum Natural Language Processing on Near-Term Quantum Computers. arXiv: 2005.04147.
  • Meichanetzidis, K., Toumi, A., Felice, G. d., & Coecke, B. (2020). Grammar-Aware Question-Answering on Quantum Computers. arXiv: 2012.03756.
  • Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
  • O'Riordan, L. J., Doyle, M., Baruffa, F., & Kannan, V. (2020). A hybrid classical-quantum workflow for natural language processing. Machine Learning: Science and Technology, 15011.
  • Orús, R. (2014). A practical introduction to tensor networks: Matrix product states and projected entangled pair states. Annals of Physics, 117-158.
  • Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2002). BLEU: a Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 311-318.
  • Pennington, J., Socher, R., & Manning, C. D. (2014). Glove: Global vectors for word representation. Proceedings of 2014 conference on empirical methods in natural language processing (EMNLP), (lpp. 1532-1543).
  • Penrose, R. (1971). Applications of negative dimensional tensors. Combinatorial Mathematics and its Applications (lpp. 221-244). Academic Press.
  • Peruzzo, A., McClean, J., Shadbolt, P., Yung, M.-H., Zhou, X.-Q., Love, P. J., . . . O'Brien, J. L. (2014). A variational eigenvalue solver on a photonic quantum processor. Nature Communications, 4213.
  • Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. Proceedings of NAACL-HLT, (lpp. 2227-2237).
  • Popović, M. (2015). chrF: character n-gram F-score for automatic MT evaluation. Proceedings of the Tenth Workshop on Statistical Machine Translation, 392-395.
  • Preskill, J. (2018). Quantum Computing in the NISQ era and beyond. Quantum, 2, 79.
  • Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. Retrieved from OpenAI blog: http://www.persagen.com/files/misc/radford2019language.pdf
  • Schollwöck, U. (2011). The density-matrix renormalization group in the age of matrix product states. Annals of Physics, 96-192.
  • Schwenk, H., Wenzek, G., Edunov, S., Grave, E., & Joulin, A. (2019). Ccmatrix: Mining billions of high-quality parallel sentences on the web. arXiv preprint arXiv:1911.04944.
  • Stanojević, M., & Sima'an, K. (2014). Stanojević, Miloš; Sima'an, Khalil. Proceedings of the Ninth Workshop on Statistical Machine Translation, 414–419.
  • Thompson, B., & Koehn, P. (2019). Vecalign: Improved sentence alignment in linear time and space. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), (lpp. 1342-1348).
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., . . . Polosukhin, I. (2017). Attention is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, 6000-6010.
  • Vicentini, F., Biella, A., Regnault, N., & Ciuti, C. (2019). Variational Neural-Network Ansatz for Steady States in Open Quantum Systems. Physical Review Letters, 250503.
  • Wall, M. L., Abernathy, M. R., & Quiroz, G. (2010). Generative machine learning with tensor networks: benchmarks on near-term quantum computers. arXiv: 2010.03641.
  • Wetering, J. v. (2020). ZX-calculus for the working quantum computer scientist. arXiv: 2012.13966.
  • White, S. R. (1993). Density-matrix algorithms for quantum renormalization groups. Physical Review B 48.14.