Standard Errors? Measuring the extent of quotation errors in political science.

In this poster we will show the method we have used to check the correctness of a controlled selection of citations from the top five journals in political science (2009 and 2019). We initially downloaded all references from each year, using Web of Science. From these we randomly selected 50 references from each journal. These 50 were then checked by the research team, expanding the dataset when a reference was cited more than once in the document. The research team consists of 3 research librarians (LEB, MRJ and SS) and a professor of political science (JM). The work is still ongoing, but the method is established. Currently, we have completed work on the dataset from 2019 and we look forward to presenting our preliminary results. The poster will illustrate the workflow from choosing the references to be controlled, downloading the references from Web of Science and comparing the contents of the article under scrutiny (the “document”) with the contents of the citation being checked (the “reference”). When the citation from the document has been checked, we code the reference in our spreadsheet (Fully substantiated, partially substantiated, unsubstantiated and impossible to substantiate). In addition we also code other relevant attributes such as string citation (yes/no), reference type (book/ conference/article/other), pagination (yes/no), reference location in document (page number), location in reference (page number), negative/positive citation. Our work draws heavily on Smith & Cumberledge (2020) and their work checking citations in general science journals. Their results from studying quotation errors suggest that these errors are more widespread than is commonly assumed. Our preliminary results show that political science also have some skeletons in the closet. This work is part of an ongoing process where the library seeks to work closely with researchers, seeking to utilize the strength of all involved. We have previously published a study on the state of political science using bibliometric methods (Jensen & Moses 2021). Once analysis is completed, we plan on publishing our findings in a journal article and publish our dataset in a data repository so that our work can be replicated.

REFERENCES: Smith, N. & Cumberledge, A. (2020). Quotation errors in general science journals. Proceedings of the Royal Society. A, Mathematical, physical, and engineering sciences, 2020 (476), 1-7. http://dx.doi.org/10.1098/rspa.2020.0538 Jensen, M.R. & Moses, J.W. (2021). The state of political science, 2020. European Political Science, 2021 (20), 14-33. https://doi.org/10.1057/s41304-020-00297-4