Open Research Europe

Part of EU Open Research Repository
Published October 12, 2023 | Version 1
Journal article Open

Multi3Generation: Multitask, Multilingual, and Multimodal Language Generation

Authors/Creators

  • 1. University of Alicante, Alicante,, San Vicente del Raspeig, 03690, Spain
  • 2. Instituto de Engenharia de Sistemas e Computadores: Investigac¸ao e Desenvolvimento (INESC-ID), Lisboa, Lisboa, Rua Alves Redol, 9, 1000-029, Portugal
  • 3. CoDesign Lab EU, Örebro University, Örebro, Örebro County, S-701 82, Sweden
  • 4. Centro Singular de Investigacion en Tecnolox´ıas Intelixentes (CiTIUS), Universidade de Santiago de Compostela, Santiago de Compostela, Galicia, 15782, Spain
  • 5. Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing, National Research Council, STIIMA CNR, Bari, via Lembo 38F, 70124, Italy
  • 6. Universite de Franche-Comte, Besançon, Bourgogne-Franche-Comté, 30-32, rue Megevand, 25030, France
  • 7. Amsterdam Public Health, Methodology and Mental Health, Amsterdam, North Holland, The Netherlands
  • 8. Vilnius University, Vilnius, Akademijos st. 4, 08412, Lithuania
  • 9. Department of Information and Electronic Engineering, International Hellenic University, Thesaloniki, 57400, Greece
  • 10. Kharkiv National University of Radio Electronics, Ukraine, Nauky Ave, 14, 61166, Ukraine
  • 11. ILC CNR "A. Zampolli", Pisa, Via G. Moruzzi, 1, 56124, Italy
  • 12. Koç University, Istanbul, Sarıyer, 34450 ˙, Turkey

Description

The purpose of this article is to highlight the critical importance of language generation today. In particular, language generation is explored from the following three aspects: multi-modality, multilinguality, and multitask, which all of them play crucial role for Natural Language Generation (NLG) community. We present the activities conducted within the Multi3Generation COST Action (CA18231), as well as current trends and future perspectives for multitask, multilingual and multimodal language generation.

Files

openreseurope-3-17604.pdf

Files (976.0 kB)

Name Size Download all
md5:f90053c1a609b04c063a6dd347418676
976.0 kB Preview Download

Additional details

Cites
10.1201/9781420085938 (DOI)
10.48550/arXiv.2303.04226 (DOI)
10.1613/jair.5477 (DOI)
10.48550/arXiv.1706.03762 (DOI)
10.1007/978-3-319-55002-2_18 (DOI)
10.1007/s10579-021-09561-5 (DOI)
10.17771/PUCRio.TradRev.30591 (DOI)
10.1007/978-3-030-41505-1_36 (DOI)
10.1007/978-3-030-70629-6_12 (DOI)
10.1007/s13218-017-0511-y (DOI)
10.1613/jair.1.12918 (DOI)
10.1007/978-3-031-24349-3_12 (DOI)
10.3233/FAIA200434 (DOI)
10.1016/j.artint.2021.103522 (DOI)
10.48550/arXiv.1806.07376 (DOI)
10.1609/aaai.v35i7.16796 (DOI)
10.48550/arXiv.1911.03705 (DOI)
10.18653/v1/2020.acl-main.703 (DOI)
10.5555/3298023.3298212 (DOI)
10.18653/v1/P19-1470 (DOI)
10.1609/aaai.v33i01.33013027 (DOI)
10.15388/DAMSS.13.2022 (DOI)
10.3390/robotics8040100 (DOI)
10.1098/rstb.2018.0027 (DOI)
10.3390/s23136054 (DOI)

References

  • Indurkhya N, Damerau FJ (2010). Handbook of Natural Language Processing. doi:10.1201/9781420085938
  • Cao Y, Li S, Liu Y (2023). A comprehensive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt. doi:10.48550/arXiv.2303.04226
  • Gatt A, Krahmer E (2018). Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. J Artif Intell Res. doi:10.1613/jair.5477
  • Vaswani A, Shazeer N, Parmar N (2017). Attention is all you need. doi:10.48550/arXiv.1706.03762
  • Paaß G, Giesselbach S (2023). Foundation Models for Natural Language Processing: Pre-trained Language Models Integrating Media.
  • Azunre P (2021). Transfer Learning for Natural Language Processing.
  • Rothman D (2021). Transformers for Natural Language Processing: Build innovative deep neural network architectures for NLP with Python, PyTorch, TensorFlow, BERT, RoBERTa, and more.
  • Han R, Lu X, Xu J (2014). On big data benchmarking.
  • Mota C, Barreiro A, Raposo F (2016). eSPERTo's Paraphrastic Knowledge Applied to Question-Answering and Summarization. doi:10.1007/978-3-319-55002-2_18
  • Barreiro A, Mota C, Baptista J (2022). Linguistic Resources for Paraphrase Generation in Portuguese: a Lexicon-grammar Approach. Language Resources and Evaluation. doi:10.1007/s10579-021-09561-5
  • Barreiro A, Rebelo-Arnold I, Mota C (2023). Parafrasário: A variety-based paraphrasary for portuguese.
  • Barreiro A, Mota C (2017). e-PACT: eSPERTo Paraphrase Aligned Corpus of EN-EP/BP Translations. Tradução em Revista. doi:10.17771/PUCRio.TradRev.30591
  • Barreiro A, Mota C (2018). Paraphrastic Variance between European and Brazilian Portuguese.
  • Barreiro A, Rebelo-Arnold I, Batista F (2020). One Book, Two Language Varieties. doi:10.1007/978-3-030-41505-1_36
  • Mota C, Santos D, Barreiro A (2021). Paraphrasing Emotions in Portuguese. doi:10.1007/978-3-030-70629-6_12
  • Bhatt M, Kersting K (2017). Semantic interpretation of multi-modal human-behaviour data - making sense of events, activities, processes. Künstliche Intell. doi:10.1007/s13218-017-0511-y
  • Erdem E, Kuyu M, Yagcioglu S (2022). Neural natural language generation: A survey on multilinguality, multimodality, controllability and learning. J Artif Intell Res. doi:10.1613/jair.1.12918
  • Bhatt M, Suchan J (2023). Artificial Visual Intelligence: Perceptual Commonsense for Human-Centred Cognitive Technologies. doi:10.1007/978-3-031-24349-3_12
  • Bhatt M, Suchan J (2020). Cognitive vision and perception. doi:10.3233/FAIA200434
  • Suchan J, Bhatt M, Varadarajan S (2021). Commonsense visual sensemaking for autonomous driving - on generalised neurosymbolic online abduction integrating vision and semantics. Artif Intell. doi:10.1016/j.artint.2021.103522
  • Suchan J, Bhatt M, Vardarajan S (2018). Semantic analysis of (reflectional) visual symmetry: A human-centred computational model for declarative explainability. Advances in Cognitive Systems. doi:10.48550/arXiv.1806.07376
  • Saichyshyna N, Maksymenko D, Turuta O (2023). Extension Multi30K: Multimodal dataset for integrated vision and language research in Ukrainian.
  • Bosselut A, Rashkin H, Sap M (2019). COMET: commonsense transformers for automatic knowledge graph construction.
  • Liu Y, Wan Y, He L (2021). Kg-bart: Knowledge graph-augmented bart for generative commonsense reasoning. doi:10.1609/aaai.v35i7.16796
  • Lin BY, Zhou W, Shen M (2019). Commongen: A constrained text generation challenge for generative commonsense reasoning. doi:10.48550/arXiv.1911.03705
  • Lewis M, Liu Y, Goyal N (2020). BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. doi:10.18653/v1/2020.acl-main.703
  • Speer R, Chin J, Havasi C (2017). Conceptnet 5.5: An open multilingual graph of general knowledge. doi:10.5555/3298023.3298212
  • Bosselut A, Rashkin H, Sap M (2019). COMET: Commonsense transformers for automatic knowledge graph construction. doi:10.18653/v1/P19-1470
  • Sap M, Le Bras R, Allaway E (2019). Atomic: An atlas of machine commonsense for reasoning. doi:10.1609/aaai.v33i01.33013027
  • Lewis P, Perez E, Piktus A (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems.
  • Karpukhin V, Oğuz B, Min S (2020). Dense passage retrieval for open-domain question answering.
  • Estevez-Velarde S, Montoyo A, Cruz YA (2019). Demo application for leto: Learning engine through ontologies.
  • Korvel G, Katsalis A, Diamantaras K (2022). Enrich knowledge graphs and test pre-trained language models in graph2seq tasks. doi:10.15388/DAMSS.13.2022
  • Koncel-Kedziorski R, Bekal D, Luan Y (2019). Text Generation from Knowl- edge Graphs with Graph Transformers.
  • Barreiro A, Mota C (null). A multilingual paraphrasary of multiwords.
  • Breque M, De Nul l, Petridis A (2021). Industry 5.0: towards a sustainable, human-centric and resilient european industry. European commission, directorate-general for research and innovation.
  • Matheson E, Minto R, Zampieri EGG (2019). Human-robot collaboration in manufacturing applications: A review. Robotics. doi:10.3390/robotics8040100
  • Foster ME (2019). Natural language generation for social robotics: opportunities and challenges. Philos Trans R Soc Lond B Biol Sci. doi:10.1098/rstb.2018.0027
  • Modoni GE, Sacco M (2023). A human digital-twin-based framework driving human centricity towards industry 5.0. Sensors (Basel). doi:10.3390/s23136054
  • Reiter E, Dale R (2000). Building Natural Language Generation Systems.
  • Devlin J, Chang MW, Lee K (2019). BERT: Pre-training of deep bidirectional transformers for language understanding.
  • Radford A, Wu J, Child R (2019). Language models are unsupervised multitask learners.
  • Brown TB, Mann B, Ryder N (2020). Language models are few-shot learners. CoRR.
  • Gatt A, Reiter E (2009). SimpleNLG: A realisation engine for practical applications.
  • Corbelle JG, Bugarín-Diz A, Alonso-Moral J (2022). Dealing with hallucination and omission in neural natural language generation: A use case on meteorology.
  • Belz A, Thomson C, Reiter E (2023). Missing information, unresponsive authors, experimental flaws: The impossibility of assessing the reproducibility of previous human evaluations in NLP.