The State of Unpaywall: Analyzing the Consistency of Open Access Data

These result highlight the difficulties of identifying the open access status of a publication. Especially for less rigid OA subgroups like Hybrid and Bronze OA the classification task is a process of iterating over improved algorithms. Generally, it can be assumed that these iterations lead towards a more accurate reflection of the true OA status. This process, however, has implications for the academic users of Unpaywall data. Studies that use these data to analyze OA status and especially OA subgroups should be aware that the reliability of the data and reproducibility of the results are dependent on time and infrastructural design choice. This observation poses essential background information for OA studies that rely on Unpaywall data at a single point in time.

For the OA transformation, the results also highlight the importance of author-choice based contributions. Publisher-choice based contributions appear to be harder to identify but also volatile in their status over time. For Open Access studies, these findings provide empirical reasons for caution when including data on Bronze OA into their analysis. For the OA transformation in general, the findings highlight authors as the key contributors to a successful transformation.