The Commercial Potential of Science

[coming soon: commercial and scientific potential predictions for over 30 million articles, published worldwide]

This dataset introduces a novel index designed to predict the commercial potential of scientific articles. The index captures the probability that an article will be used by firms for the development of marketable products or processes. In addition to commercial potential, the dataset also introduces an index to predict scientific potential—the likelihood that an article will be relevant for the advance of science, regardless its commercial application. 

The indices are crucial for researchers focused on understanding 1) the production of science with commercial potential and 2) the pathway from academic research to market innovations and the factors that influence the commercial viability of scientific discoveries.

Citation Information: If you use this dataset, please cite the article: “Masclans-Armengol, R., Hasan, S., & Cohen, W. M. (2024). Measuring the Commercial Potential of Science. NBER working paper”

Components of the Dataset: The dataset encompasses indices for over 5.2 million articles that meet the following criteria:

• Publication year: 2000 to 2020
• Published under 126 U.S. universities
• Articles in the applied and natural sciences and engineering fields

Data is delivered via a single csv file. Each row contains information for a scientific article, with the following variables:

• ‘doi’: Digital Object Identifier—unique article identifier that can be used to match to other data sources, such as OpenAlex, Dimensions, or Web of Science.
• ‘compot’: commercial potential index.
• ‘scipot‘: scientific potential index.

To develop the commercial potential index, we employed SciBert (Beltagy et al., 2019), a Large Language Model for scientific understanding. We fine tune SciBert with deep neural networks to classify scientific articles based on their potential for commercial application. We trained 20 predictive models, one per year, using the text of an academic article’s abstract to generate ex-ante, out-of-sample, and out-of-training-time-period predictions of any given scientific article’s commercial potential.

Following the same methodology, we compute the scientific potential index.

Licensing and Contact Information: The dataset and its components are distributed under a Creative Commons Attribution Non-Commercial license.

Acknowledgments: We thank The Technology Opportunity Lab at Duke University and the Kauffman Foundation for funding the creation of this dataset.