Readme file for the Datasets used in Purgar et al. (2024) Nature Ecology and Evolution: Supporting study registration to reduce research waste.

This data package contains three data tables (datasets):

1)	Dataset 1: EcoEvo Registered Reports Journals
Lists 24 journals that publish research in ecology and evolutionary biology and offer registered reports publication type as of August 2023. 

2)	Dataset 2: Meta-studies on impact of registration
Dataset of 26 meta-studies that quantified the impact of pre-registration or registered reports on the methodological quality of research, completeness of reporting, or on the features of obtained results. 

3)	Dataset 3: Meta-studies on outcome reporting bias
Dataset of 11 meta-studies that quantified the outcome reporting bias by comparing pre-registration with the resulting publication.


Methods on how the data were obtained, and the variables used in these datasets are described below.

1) Dataset 1: EcoEvo Registered Reports Journals

How the dataset was obtained

To obtain this dataset, we used Open Science Frameworks (OSF) list of journals that offer Registered Reports, and extracted those that publish ecological and evolutionary biology research (25 journals). The list is kept by the OSF here Registered Reports (cos.io) (under tab ‘Participating journals’) and we accessed the list on the 20th of May 2023. We added one additional journal to the list, Nature Ecology and Evolution, as this journal just recently adopted Registered Reports, and was not entered in the OSF table. We have checked each journal additionally to double check if they do offer registered reports, number of registered reports (Stage 1 or 2) that were published prior to 10th of July 2023, to examine their instructions to authors, and to check whether they clearly state what type of registered reports they support (e.g. experimental research, meta-analyses etc.). We could not detect any reference to registered reports in one of the journals (Frontiers in Plant Science). Further, one of the journals (BMC Ecology) listed at the OSF list has merged with another journal (BMC Ecology and Evolution). While BMC Ecology had registered reports, we could not determine whether BMC Ecology and Evolution specifically supports this type of contribution. We have thus emailed both journals and they confirmed they do not support registered reports. Our final list contains 24 journals for which we could confirm the acceptance of registered reports.

Variables with description and possible values in []:

Journal		= Title of the journal [free text]

Link to Instructions for Authors  = Web link to instructions for authors [free text]

Year of adoption 	= Year in which registered reports were introduced to specific journal [free text]

Introduced as		= Information on how registered reports policy was introduced, e.g. editorial, and a link to introduction [‘announcement’, ‘editorial’, ‘blog post’ and free text], NA denotes cases where we could not find the information on RR introduction policy.]

Number of published registered reports 	= Denotes findable number of published registered reports as of August 2023.  If a journal offered a search tool that enabled targeting registered reports, we noted the number of observed published registered reports (e.g. 0, 1, 2 etc.). If, however, we could not search by article type because there was no such option offered on the journal website, we denoted these with ‘NA’.

Explicitly states supported type of studies (e.g. experimental, observational, replications) for RR= Denotes which type of study is supported as registered report, e.g. experimental, observational, replications, meta-analyses etc. [‘yes’ or ‘no’ plus free text copy-pasted from journal policy]

Note		= Additional relevant information [free text]


2) Dataset 2: Meta-studies on impact of registration

How the dataset was obtained

We conducted an exploratory survey to find meta-studies that evaluated the effect of registration (pre-registration and registered reports) on any aspect of study's methodological or reporting quality, and features of study results (effect size, statistical significance etc.). The aim of this survey was a quick scan of the existing literature to provide some evidence to support (or not) the claims provided in the Perspective, and not a systematic and comprehensive search for all the literature published on the topic. Thus, the survey was not registered and can be used as a starting point for a comprehensive systematic review. 
We searched for meta-studies that compared pre-registered studies or registered reports with standard published literature. We have also aimed to detect studies that compared results registered in the registry with those reported in the related publication.
We conducted a search of published literature on June 13th, 2023, using the Web of Science (WoS) Core Collection, accessed through Ruder Boskovic Institute, Zagreb, Croatia. The search string was defined based on keywords, Boolean, and adjacency operators, and was searched for in All Fields (Field Tag “ALL”). The search string was as follows:

((ALL=(Register* OR registrat* OR RR OR registry)) AND ALL=(“standard literature“ OR “Standard publishing“ OR “Published literature“ OR “Published articles“ “Published work“ OR “Published reports“ OR “published trials“ OR “published studies“ OR “published research“ OR  “unregistered studies“ OR “unregistered trials“ OR  “non registered studies“ OR “not registered studies“ OR “non registered trials“ OR “not registered trials“)) AND ALL=(“research quality“ OR “publication bias“ OR “questionable research practices“ OR “reporting quality“ OR “quality of reporting“ OR “transparency“ OR “positive study findings“ OR “positive results“ OR “effect estimates“ OR “effect size estimates“ OR “treatment effects“ OR “positive study findings“ OR “statistically significant“ OR  “selective reporting“ OR “result reporting“ OR transparent OR reproducible).

We supplemented this with 11 meta-studies already known to us. The WoS search led to 800 results which were exported to Rayyan. Title and abstract screening were done by AC and MP. 50 articles were double-screened with 100% agreement rate. Overall, 69 articles passed to full-text screening out of which 25 were included in the final sample. As we were interested in the potential effects registration has on study design, publication bias, and reporting we included all meta-studies that compared i) registered reports and non-registered reports (N=5), ii) registered reports and pre-registered studies (N=1), and iii) pre-registered studies and non-pre-registered studies (N=21). Note that this category (ii) was deemed relevant post-hoc and not prior to our search. We also included studies that compared discrepancies between the results reported in the pre-registration and its resulting publication (N=12). 

Data extraction from the final list of meta-studies (25 from WoS search, and 11 from prior knowledge) was done by AC and MP.

Variables with description and possible values in []:

Paper	= Title of the paper (meta-study) [free text]

DOI	= Digital Object Identifier [free text]

Field	= Study field [medicine, psychology, psychology - parapsychology]

PR or RR = Denotes whether the study focused on pre-registration (PR) or registered reports (RR) [PR, RR]

Compared	= What was compared to what [PR (prospective vs retrospective) vs non-PR, PR (prospective) vs non-PR, PR prospective vs retrospective, PR vs non-PR, RR vs non-RR, RR vs non-RR (but only replication studies), PR vs RR]

Effect on	= What part of study feature has the meta-study examined [design, reporting, results]

Effect on detailed	= Detailed description of the effect examined [sample size, risk of bias, spin, result direction (in favour of hypothesis or not), effect size, quality score, quality score based on PEDro, statistical significance, methodological reporting, reporting important methodological details associated with risk of bias (and likely lower RoB), reporting on methodological aspects, transparent reporting]

Method	= General methodology of the study [free text, copy-pasted from the meta-study]

Results		= Results, as presented in paper [free text, copy-pasted from the meta-study]

Other info	= Other potentially relevant or interesting information [free text, copy-pasted from the meta-study]

From	= Denotes how study was identified: via literature review or from our previous knowledge [LitRew, Previous]

References
1. Allen C, Mehler DMA (2019) Open science challenges, benefits and tips in early career and beyond. PLoS Biol. 17(5), e3000246 (2019).
2. Brohmer, H., Eckerstorfer, L., V., van Aert, R. C. M. & Corcoran, K. Do Behavioral Observations Make People Catch the Goal? A Meta-Analysis on Goal Contagion. Int. Rev. Soc. Psychol. 34, (2021).
3. Dechartres, A., Ravaud, P., Atal, I., Riveros, C., & Boutron, I. Association between trial registration and treatment effect estimates: a meta-epidemiological study. BMC medicine, 14(1), 100 (2016).
4. Emdin, C., Odutayo, A., Hsiao, A., Shakir, M., Hopewell, S., Rahimi, K. & Altman, D.G. Association of cardiovascular trial registration with positive study findings: Epidemiological Study of Randomized Trials (ESORT). JAMA Intern Med 175(2), 304–307 (2015).
5. Farquhar, C. M., Showell, M. G., Showell, E. A. E., Beetham, P., Baak, N., Mourad, S. & Jordan, V. M. B. Clinical trial registration was not an indicator for low risk of bias. J. Clin. Epidemiol. 84, 47–53 (2017).
6. Gartlehner, G., Emprechtinger, R., Hackl, M., Jutz, F. L., Gartlehner, J. E., Nonninger, J. N., Klerings, I., & Dobrescu, A. I. Assessing the magnitude of reporting bias in trials of homeopathy: a cross-sectional study and meta-analysis. BMJ evidence-based medicine, 27(6), 345–351 (2022).
7. Gopal, A. D., Wallach, J. D., Aminawung, J. A., Gonsalves, G., Dal-Ré, R., Miller, J. E. & Ross, J. S. Adherence to the International Committee of Medical Journal Editors’ (ICMJE) prospective registration policy and implications for outcome integrity: a cross-sectional analysis of trials published in high-impact specialty society journals. Trials 19, 448 (2018).
8. Hamm, M. P., Hartling, L., Milne, A., Tjosvold, L., Vandermeer, B., Thomson, D., Curtis, S., & Klassen, T. P. A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007. BMC pediatrics, 10, 96 (2010).
9. Kaplan, R. M., & Irvin, V. L. Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time. PloS One, 10(8), e0132382 (2015).
10. Kvarven, A., Strømland, E., & Johannesson, M. Comparing meta-analyses and preregistered multiple-laboratory replication projects. Nature human behaviour, 4(4), 423–434 (2020).
11. Odutayo, A., Emdin, C.A., Hsiao, A.J., Shakir, M., Copsey, B., Dutton, S., Chiocchia, V., Schlussel, M., Dutton, P., Roberts, C., Altman, D.G. & Hopewell, S. Association between trial registration and positive study findings: cross sectional study (Epidemiological Study of Randomized Trials—ESORT). BMJ 356 (2017).
12. Papageorgiou, S. N., Xavier, G. M., Cobourne, M. T., & Eliades, T. Registered trials report less beneficial treatment effects than unregistered ones: a meta-epidemiological study in orthodontics. J. Clin. Epidemiol., 100, 44–52 (2018).
13. Pinto, R. Z., Elkins, M. R., Moseley, A. M., Sherrington, C., Herbert, R. D., Maher, C. G., Ferreira, P. H. & Ferreira, M. L. Many randomized trials of physical therapy interventions are not adequately registered: a survey of 200 published trials. Phys Ther 93, 299–309 (2013).
14. Rasmussen, N., Lee, K., & Bero, L. Association of trial registration with the results and conclusions of published trials of new oncology drugs. Trials, 10, 116 (2009).
15. Riemer, M., Kranke, P., Helf, A., Mayer, D., Popp, M., Schlesinger, T., Meybohm, P., & Weibel, S. Trial registration and selective outcome reporting in 585 clinical trials investigating drugs for prevention of postoperative nausea and vomiting. BMC anesthesiology, 21(1), 249 (2021).
16. Schäfer, T. & Schwarz, M. A. The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases. Front. Psychol. 10, 813 (2019).
17. Scheel, A. M., Schijen, M. R. M. J. & Lakens, D. An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports. Adv. Methods Pract. Psychol. Sci. 4, (2021).
18. Seehra, J., Khraishi, H. & Pandis, N. Studies with statistically significant effect estimates are more frequently published compared to non-significant estimates in oral health journals. BMC Med Res Methodol 23, 6 (2023). 
19. Shaw, R., Ni, M., Pillar, M. & Tejani, A. M. Are antidepressant and antipsychotic drug trials registered? A cross-sectional analysis of registration and reporting of methodologic characteristics. Account. Res. 25, 301–309 (2018).
20. Soderberg, C. K., Errington, T. M., Schiavone, S. R., Bottesini, J., Thorn, F. S., Vazire, S., Esterling, K. M. & Nosek, B. A. Initial evidence of research quality of registered reports compared with the standard publishing model. Nat. Hum. Behav. 5, 990-997 (2021).
21. Tan, A. C., Jiang, I., Askie, L., Hunter, K., Simes, R. J., & Seidler, A. L. Prevalence of trial registration varies by study characteristics and risk of bias. J. Clin. Epidemiol. 113, 64–74 (2019).
22. Tharyan, P., George, A.T., Kirubakaran, R. & Barnabas, J.P. Reporting of methods was better in the Clinical Trials Registry-India than in Indian journal publications. J Clin Epidemiol 66(1), 10–22 (2013).
23. Trinquart, L., Dunn, A. G., & Bourgeois, F. T. (2018). Registration of published randomized trials: a systematic review and meta-analysis. BMC medicine, 16(1), 173. 
24. Wiseman, R., Watt, C., & Kornbrot, D. Registered reports: an early example and analysis. PeerJ 7, e6232 (2019).
25. Won, J., Kim, S., Bae, I. & Lee, H. Trial registration as a safeguard against outcome reporting bias and spin? A case study of randomized controlled trials of acupuncture. PLOS ONE 14, (2019).
26. Ye Q. M., Chen Z. G., Chen L. L., Si-Tu B., Mai Y. Y., Xiao J. Y., Yang Y. J. Quality assessment and its influencing factors of lung cancer clinical research registration: a cross-sectional analysis. J Thorac. Dis. 14(9), 3471-3487 (2022).


3) Dataset 3: Meta-studies on outcome reporting bias

The dataset was obtained via same exploratory survey as Dataset 2. 

Variables with description and possible values in []:

Paper	= Title of the paper (meta-study) [free text]

DOI	= Digital Object Identifier [free text]

Field	= Study field [medicine, psychology, psychology - parapsychology]

Compared	= What was compared to what [PR vs resulting published article]

Effect on	= What part of study feature has the meta-study examined [outcome reporting]

Effect on detailed	= Detailed description of the effect examined [result direction (in favour of hypothesis or not), effect size, statistical significance, reporting or not on serious adverse effects, reporting of all key elements (according to three experts, for the flow of participants, efficacy results, adverse events, and serious adverse events)]

Method	= General methodology of the study [free text, copy-pasted from the meta-study]

Results		= Results, as presented in paper [free text, copy-pasted from the meta-study]

Other info	= Other potentially relevant or interesting information [free text, copy-pasted from the meta-study]

From	= Denotes how study was identified: via literature review or from our previous knowledge [LitRew, Previous]


References
1. Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group. Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review. PLoS ONE 8, e66844 (2013).
2. Grégory, J., Créquit, P., Vilgrain, V., Boutron, I., & Ronot, M. Published trials of TACE for HCC are often not registered and subject to outcome reporting bias. JHEP reports : innovation in hepatology, 3(1), 100196 (2020).
3. Karimian, Z., Mavoungou, S., Salem, JE. et al. The quality of reporting general safety parameters and immune-related adverse events in clinical trials of FDA-approved immune checkpoint inhibitors. BMC Cancer 20, 1128 (2020). https://doi.org/10.1186/s12885-020-07518-5
4. Killeen, S., Sourallous, P., Hunter, I. A., Hartley, J. E., & Grady, H. L. Registration rates, adequacy of registration, and a comparison of registered and published primary outcomes in randomized controlled trials published in surgery journals. Annals of surgery, 259(1), 193–196 (2014).
5. Pinto, R. Z., Elkins, M. R., Moseley, A. M., Sherrington, C., Herbert, R. D., Maher, C. G., Ferreira, P. H. & Ferreira, M. L. Many randomized trials of physical therapy interventions are not adequately registered: a survey of 200 published trials. Phys Ther 93, 299–309 (2013).
6. Riemer, M., Kranke, P., Helf, A., Mayer, D., Popp, M., Schlesinger, T., Meybohm, P., & Weibel, S. Trial registration and selective outcome reporting in 585 clinical trials investigating drugs for prevention of postoperative nausea and vomiting. BMC anesthesiology, 21(1), 249 (2021).
7. Riveros, C., Dechartres ,A., Perrodeau, E., Haneef, R., Boutron, I., & Ravaud, P. Timing and Completeness of Trial Results Posted at ClinicalTrials.gov and Published in Journals. PLOS Medicine 11 (12), 10(12): e1001566 (2013).
8. Roest, A. M., de Jonge, P., Williams, C. D., de Vries, Y. A., Schoevers, R. A., & Turner, E. H. Reporting Bias in Clinical Trials Investigating the Efficacy of Second-Generation Antidepressants in the Treatment of Anxiety Disorders: A Report of 2 Meta-analyses. JAMA psychiatry, 72(5), 500–510 (2015).
9. Su, C. X., Han, M., Ren, J., Li, W. Y., Yue, S. J., Hao, Y. F., & Liu, J. P. Empirical evidence for outcome reporting bias in randomized clinical trials of acupuncture: comparison of registered records and subsequent publications. Trials, 16, 28 (2015).
10. Turner E. H., Knoepflmacher D., Shapley L. Publication Bias in Antipsychotic Trials: An Analysis of Efficacy Comparing the Published Literature to the US Food and Drug Administration Database. PLoS Med 9(3), e1001189 (2012).
11. Won, J., Kim, S., Bae, I. & Lee, H. Trial registration as a safeguard against outcome reporting bias and spin? A case study of randomized controlled trials of acupuncture. PLOS ONE 14, (2019).