A conceptual framework for computational reproductions: Formal definitions and epistemic functions

Reproductions, which we define as “redoing activities” that work with the same data and aim to keep the computation as similar as possible, enable the verification of a scientific work. We seek to advance the understanding and conduct of reproductions by providing a conceptual framework and guidance to reproducers. In the first part, the epistemic function of reproductions and direct replications with regard to mistakes – and faults as a consequence – is discussed. In the second part, the logic underlying reproductions is formulated, accompanied by formal definitions of central terms. The third and final part will provide practical guidance on reporting reproductions. In short, a reproduction should involve checking the coherence of relevant instructions, performing another computation, and comparing the results for consistency. Evaluating the consistency means exploring the impact of computational choices on results before suspecting faults as a last resort. Computational choices are necessitated by obstacles such as underspecification and grounds for conflicting choices. Regarding faults, four cases are emphasized: (1) incoherent descriptions, (2) different data, (3) different source code, and (4) incorrect reporting of results. Separate from evaluating the consistency of results, reproductions can investigate their support for a claim. Finally, improper reporting practices can reduce the epistemic value of reproductions. Therefore, reproducers should also report the scope of their reproduction by detailing, among other things, which materials were consulted and to what extent any data were already preprocessed.