Predicting Research Trends From Arxiv

Knowing trends in research has been a long-standing dream of scientists. Projects on popular research topics often lead to higher acceptance rates at conferences and journals, as well as funding application approvals. Further, knowing future research trends immediately also has implications for society as a whole, because these trends will most likely directly affect the labour market, technological orientation and bias, consumer end products, as well as cultural metaphors and definitions of the human identity (this is even more true for fields such as artificial intelligence, as we focus on here). However, with the accelerating number of papers made available each year, it becomes ever more difficult to digest the incoming information and thereby identify topics that will have long-term scientific impact. We have developed an automatic system whose goal is to uncover important research trends, and, therefore, aims at helping researchers better plan their academic endeavours. To illustrate our system, we crawl papers published in the Machine Learning (cs.LG) and natural language processing (cs.CL) categories of Arxiv, with information about how often they were cited. In this dataset, we identify influential papers and categorize them by hand and automatically. Using Arxiv papers for our exploration appears promising, because Arxiv is a very popular pre-print (and post-print) server for scientific publications, whose impact has, moreover, considerably increased over the last few years.