BIP! DB: A Dataset of Impact Measures for Scientific Publications

This dataset contains citation-based impact indicators (a.k.a, "measures") for ~151M distinct DOIs that correspond to scientific articles. In particular, for each DOI, we have calculated the following indicators (organized in categories based on the semantics of the impact aspect that they better capture):

• Influence indicators (i.e., indicators of the "total" impact of each article; how established the article is in general)

  • Citation Count: The total number of citations of the article, the most well-known influence indicator.

  • PageRank score: An influence indicator based on the PageRank¹, a popular network analysis method. PageRank estimates the influence of each article based on its centrality in the whole citation network. It alleviates some issues of the Citation Count indicator (e.g., two articles with the same number of citations can have significantly different PageRank scores if the aggregated influence of the articles citing them is very different - the article receiving citations from more influential articles will get a larger score).  

• Popularity indicators (i.e., indicators of the "current" impact of each article; how popular the article is currently)

  • RAM score: A popularity indicator based on the RAM² method. It is essentially a Citation Count where recent citations are considered as more important. This type of “time awareness” alleviates problems of methods like PageRank, which are biased against recently published articles (new articles need time to receive a number of citations that can be indicative for their impact).

  • AttRank score: A popularity indicator based on the AttRank³ method. AttRank alleviates PageRank’s bias against recently published papers by incorporating an attention-based mechanism, akin to a time-restricted version of preferential attachment, to explicitly capture a researcher’s preference to read papers which received a lot of attention recently.

• Impulse indicators (i.e., indicators of the initial momentum that the article receives after its publication)

  • Incubation Citation Count (3-year CC): This impulse indicator is a time-restricted version of the Citation Count, where the time window length is fixed for all papers and the time window depends on the publication date of the paper, i.e., only citations 3 years after each paper’s publication are counted.

More details about the aforementioned impact indicators, the way they are calculated and their interpretation can be found here and in the respective references.

From version 5.1 onward, the impact indicators are calculated in two levels:

• The DOI level (assuming that each DOI corresponds to a distinct scientific article).
• The OpenAIRE-id level (leveraging DOI synonyms based on OpenAIRE's deduplication algorithm⁴ - each distinct article has its own OpenAIRE id). 

Previous versions of the dataset only provided the scores at the DOI level.

Also, from version 7 onward, for each article in our files we also offer an impact class, which informs the user about the percentile into which the article score belongs compared to the impact scores of the rest articles in the database. The impact classes are: C1 (in top 0.01%), C2 (in top 0.1%), C3 (in top 1%), C4 (in top 10%), and C5 (in bottom 90%).

Finally, before version 10, the calculation of the impact scores (and classes) was based on a citation network having one node for each article with a distinct DOI that we could find in our input data sources. However, from version 10 onward, the nodes are deduplicated using the most recent version of the OpenAIRE article deduplication algorithm. This enabled a correction of the scores (more specifically, we avoid counting citation links multiple times when they are made by multiple versions of the same article). As a result, each node in the citation network we build is a deduplicated article having a distinct OpenAIRE id. We still report the scores at DOI level (i.e., we assign a score to each of the versions/instances of the article), however these DOI-level scores are just the scores of the respective deduplicated nodes propagated accordingly (i.e., all version of the same deduplicated article will receive the same scores). We have removed a small number of instances (having a DOI) that were assigned (by error) to multiple deduplicated records in the OpenAIRE Graph.

For each calculation level (DOI / OpenAIRE-id) we provide five (5) compressed CSV files (one for each measure/score provided) where each line follows the format  “identifier <tab> score <tab> class”. The parameter setting of each measure is encoded in the corresponding filename. For more details on the different measures/scores see our extensive experimental study⁵ and the configuration of AttRank in the original paper.³ Files for the OpenAIRE-ids case contain the keyword "openaire_ids" in the filename.  

From version 9 onward, we also provide topic-specific impact classes for DOI-identified publications. In particular, we associated those articles with 2nd level concepts from OpenAlex (284 in total); we chose to keep only the three most dominant concepts for each publication, based on their confidence score, and only if this score was greater than 0.3. Then, for each publication and impact measure, we compute its class within its respective concepts. We provide finally the "topic_based_impact_classes.txt" file where each line follows the format “identifier <tab> concept <tab> pagerank_class <tab> attrank_class <tab> 3-cc_class <tab> cc_class”.

The data used to produce the citation network on which we calculated the provided measures have been gathered from the OpenAIRE Graph v6.0.0, including data from (a) the OpenCitations’ COCI dataset (Jan-2023 version), (b) a MAG^6,7 snapshot from Dec-2021, and (c) a Crossref snapshot from May-2023 (before version 10, these sources were gathered independently). The union of all distinct citations that could be found in these sources have been considered. In addition, versions later than v.10 leverage the filtering rules described here to remove from the dataset DOIs with problematic metadata.

References:

1. R. Motwani L. Page, S. Brin and T. Winograd. 1999. The PageRank Citation Ranking: Bringing Order to the Web. Technical Report. Stanford InfoLab.

2. Rumi Ghosh, Tsung-Ting Kuo, Chun-Nan Hsu, Shou-De Lin, and Kristina Lerman. 2011. Time-Aware Ranking in Dynamic Citation Networks. In Data Mining Workshops (ICDMW). 373–380

3. I. Kanellos, T. Vergoulis, D. Sacharidis, T. Dalamagas, Y. Vassiliou: Ranking Papers by their Short-Term Scientific Impact. CoRR abs/2006.00951 (2020)

4. P. Manghi, C. Atzori, M. De Bonis, A. Bardi, Entity deduplication in big data graphs for scholarly communication, Data Technologies and Applications (2020).

5. I. Kanellos, T. Vergoulis, D. Sacharidis, T. Dalamagas, Y. Vassiliou: Impact-Based Ranking of Scientific Publications: A Survey and Experimental Evaluation. TKDE 2019 (early access)

6. Arnab Sinha, Zhihong Shen, Yang Song, Hao Ma, Darrin Eide, Bo-June (Paul) Hsu, and Kuansan Wang. 2015. An Overview of Microsoft Academic Service (MA) and Applications. In Proceedings of the 24th International Conference on World Wide Web (WWW '15 Companion). ACM, New York, NY, USA, 243-246. DOI=http://dx.doi.org/10.1145/2740908.2742839

7. K. Wang et al., “A Review of Microsoft Academic Services for Science of Science Studies”, Frontiers in Big Data, 2019, doi: 10.3389/fdata.2019.00045    

Find our Academic Search Engine built on top of these data here. Further note, that we also provide all calculated scores through BIP! Finder’s API. 

Terms of use: These data are provided "as is", without any warranties of any kind. The data are provided under the Creative Commons Attribution 4.0 International license.

More details about BIP! DB can be found in our relevant peer-reviewed publication:

Thanasis Vergoulis, Ilias Kanellos, Claudio Atzori, Andrea Mannocci, Serafeim Chatzopoulos, Sandro La Bruzzo, Natalia Manola, Paolo Manghi: BIP! DB: A Dataset of Impact Measures for Scientific Publications. WWW (Companion Volume) 2021: 456-460

We kindly request that any published research that makes use of BIP! DB cite the above article.