Published December 27, 2022 | Version v1

USING THE BERT MODEL FOR SEARCH AUTOMATION IN ENGINEERING ACTIVITIES

Authors/Creators

  • 1. Certivity GmbH

Description

In today’s digital world, where the amount of data created and shared daily is overwhelming, it becomes more and more challenging for engineers to find relevant information when trying to solve their technical problems or improve their technology. Current search and knowledge management applications rely heavily on NLP-based automation. The recent advances in NLP transfer learning have resulted in powerful models, such as BERT, which perform well on NLP tasks in the general domain. In this work, we evaluate different approaches to adapting BERT to the domain of engineering. We compare multiple domain-specific models in their ability to identify new technologies and assign topics to engineering articles. Our experiments show that the domain-adaptation strategy of further pre-training on domain-specific data without vocabulary extension leads to the best performance in this tasks solutions. After the evaluation, we describe the challenges and the limitations of our approach and provide directions for future research.

Files

DEIT_1-1_Poluektov.pdf

Files (730.9 kB)

Name Size
md5:70f3d6b8a314ab166f863d25ed31a163
730.9 kB Preview Download

Additional details

Related works