ID,Title,Date,PostType,Permalink,URL,Title_1,FunctionsOfTheProject,AreasOfInnovation,TypesOfOutputs,Format,Transparency,Process,DisciplinaryFields,OutcomesConsidered,DemographicVariablesOfUsers,x_type_of_project,x_job_website,x_parent_listing,x_link_to_source_code,x_status,x_job_date,x_end_date,x_operated_by,x_goals,x_review_of_code_or_data,x_manuscript_hosting,x_notes_on_review_features,x_eligible_reviewers,x_tags_or_badges,x_criteria_for_inclusion,x_cost,x_explanation_of_cost,x_number_of_commented_outputs,x_metrics,x_results_summary,x_results_url,x_source_of_data,x_independent_variables,x_preregistration_url,x_verified,Status 491,PREreview,12/28/2018,job_listing,https://reimaginereview.asapbio.org/listing/prereview/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2018/12/PREreview_logo_new.png,PREreview,Feedback to authors,Reviewer training|Transparency,Preprints,Free-form commenting,,Comment indexing|Reviewer or editor initiates review,Life Sciences,,,Project,https://www.prereview.org,,,Active,,,"Daniela Saderi, Samantha Hindle, Monica Granados",Lack of comments to preprints; lack of transparency and diversity in peer review; lack of formal peer review training; lack of wide adoption of preprints.,No,No,"Full report, quick report (short paragraph). For PREreview 2.0 and Rapid PREreview (Outbreak Science) we will expand these to being structured templates. Currently PREreviews receive DOIs and are linked under respective abstracts on bioRxiv.org.",Currently anyone with email address who signes up on Authorea. For PREreview 2.0 anyone with Orcid ID. We will allow users to display pseudonyms.,No,,0,,Unknown,"Yes, we will more with the new PREreview 2.0 (under development).",,,,NaN,,Unverified,publish 520,preLights,1/6/2019,job_listing,https://reimaginereview.asapbio.org/listing/prelights/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/01/prelights_logo_square.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/01/infographic_2_preLights.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/01/preLights_infographic_Palfy.png,preLights|preLights|preLights,Curation of interesting work|Feedback to authors,Reviewer training,Preprints,Free-form commenting,Open identities,Pre-publication review|Reviewer or editor initiates review,Life Sciences,,,Project,https://prelights.biologists.com/,,,Active,2/20/2018,,The Company of Biologists,"1) Since the launch of bioRxiv, the number of posted preprints has been exponentially growing and is expected to further increase in the future. This will make it more and more difficult for researchers to keep up with the preprint literature. preLights aims to help scientists by selecting and summarizing interesting preprints for them. 2) Despite the growing popularity of preprints, public commenting on preprints hasn't taken off. At preLights, early-career researchers give personal opinions about preprints (which are often in their field of expertise). In addition, anyone can comment on the preLights posts and by also encouraging preprint authors to provide 'author responses',  preLights aims to facilitate discussions about preprints. 3) Finally, preLights aims to promote early-career researchers and give them a platform where they can practice their scientific writing and reviewing skills. These ECRs also play an important role in raising awareness and advocating for preprints.",,No,'News and views' type of summary with personal opinion + questions to the authors; freeform commenting by anyone,"preLighters are selected through an application process, in which they have to write a preprint highlight, a short bio and explain their motivation to join",No,,0,,Unknown,We track usage of the service and participation of the selected 'preLighters' (Currently we receive cc. 2K views/week on the website and have 130 active 'preLighters'). We also track the number of preprint author comments (currently cc. 30% of posts have author comments),,,,NaN,,Unverified,publish 748,Peeriodicals.com,2/5/2019,job_listing,https://reimaginereview.asapbio.org/listing/peeriodicals-com-2/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/Screenshot-from-2019-02-05-09-50-06-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/Screenshot-from-2019-02-13-22-35-03.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/Screenshot-from-2019-02-13-22-30-58.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/Screenshot-from-2019-02-13-22-50-48.png,Peeriodicals.com|Peeriodicals.com|Peeriodicals.com|Peeriodicals.com,Curation of interesting work|Feedback to authors,Costs|Incentives and recognition for reviewing|Speed|Transparency,Journal accepted manuscripts|Other scholarly outputs|Preprints,Free-form commenting|Structured review form,Open interaction,Comment indexing|Comment moderation|Post-publication review|Reviewer or editor initiates review,,,,Project,https://peeriodicals.com/,,,Active,2/1/2018,,The PubPeer Foundation,"The traditional journal has changed remarkably little in centuries and many people feel that scientific publishing is stuck in a rut, subject to a corporatist drift, and is not serving science optimally. The advent of preprints in many fields beyond those served by the ArXiv is liberating the dissemination of research, but most other journal functions have not been replaced effectively. Now you—all researchers—have the opportunity to select and certify research according to your own criteria. We expect peeriodical subject matters and editorial policies to be extremely varied. Some peeriodicals may wish to target narrow domains, while others will adopt a generalist approach. Some peeriodicals will be inclusive, focusing on discovery, whereas others may aim to enforce stringent quality criteria, prioritising certification. The point is that all approaches are permitted and supported—we hope you will innovate! You can create multiple peeriodicals. It will be users and readers who decide which peeriodicals they find useful and interesting. Users can sign up to receive alerts from any peeriodical they wish.",,,"A peeriodical has one or more editors. Anybody can set-up a peeriodical and either operate it alone or invite colleagues to form an editorial board or community. The editors can select ""manuscripts""—existing papers or preprints—to consider, either spontaneously or through suggestions from other researchers, including of course the authors. Note that there is no obligation that the manuscript be recent; for instance, we expect that some peeriodicals could focus on underappreciated classics. After all, predictions about scientific impact are generally more accurate for the past than the future. If the editors wish, they can solicit reviews for the manuscript via the Peeriodicals interface. Reviews will be published and the referees will have the option of posting anonymously or signing their review. Editors may decide at any time to accept, reject or comment on the manuscript, taking into account the comments received. They may of course suggest improvements to the manuscript or underlying study. If they justify their decision, their editorial decision will also be published.",,,"How will Peeriodicals fit into the publishing landscape? We see them as a space without entry barriers in which researchers can innovate and explore new approaches to scientific dissemination, in parallel to the traditional publishing industry. There are related and complementary initiatives, notably the overlay journals promoted by Tim Gowers, exemplified by Discrete Analysis, but also Science Open Collections, PLoS Channels, the APPRAISE initiative and Peer Community in... Each of these projects has their own specificities and goals. Nobody yet knows exactly what the future will look like, but we strongly believe that we are about to experience a period of rapid evolution in the dissemination of science and we hope that Peeriodicals will inspire and help you to share your imagination and expertise with the whole research community.",0,,"100-1,000",,,,,NaN,,Verified,publish 771,Hypothesis,2/12/2019,job_listing,https://reimaginereview.asapbio.org/listing/hypothesis/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/HypothesisIcon.png,Hypothesis [In Progress],Curation of interesting work|Feedback to authors,Quality of review|Reviewer training|Speed|Transparency,Journal accepted manuscripts|Other scholarly outputs|Preprints|Privately shared manuscripts,Annotations|Free-form commenting|Structured review form,Double blind|Open identities|Open interaction|Open participation|Open reports|Single blind,Author initiates review|Comment indexing|Comment moderation|Journal integration|Post-publication review|Pre-publication review|Professional editors|Reviewer or editor initiates review,All disciplines,,,Project,http://hypothes.is,,https://github.com/hypothesis/h,Active,,,,"Open annotation enables a conversation across web-based content through the creation of private, group, or public in-line annotations. Integration with manuscript submission systems can enable editors, reviewers, and authors to engage in all types of peer review (from traditional single-blind or double-blind to open) in either pre-publication or post-publication form. The tool can also be used outside of a manuscript submission system by publications who wish to undertake peer review experiments. One example is In Review, via which Biomed Central and Research Square offer authors the ability to opt into community feedback in parallel with traditional peer review. Feedback on pre-prints is an ideal use case for providing suggestions (public or in private groups) on early versions of articles. Because it works anywhere on the web, open annotation is an ideal tool to enable overlay journals. It can also raise the visibility of peer review reports posted as supplementary materials.",Yes,No,"'-Works with either pre-publication or post-publication peer review -Enables open peer review -Enables traditional peer review either within manuscript submission system integrations or as a stand alone. Publisher can assign reviewer accounts for reviewer anonymity. -Can be used to post peer review reports or summaries -Can be used to provide feedback on content anywhere on the web in html, PDF, EPBU, or data formats.","This depends on the site integration (publisher, preprint service, etc.). They may select an Open Group (world-readable, world-writeable) or a Restricted Group (world-readable, but writeable only by those indicated by the group owner).",Yes,"Anyone can use the tool through implementation of public or private groups. Should a publisher wish to host their own branded and moderated layer, there is a cost. Please contact us for details.",NaN,"We offer a “freemium” service for those wanting to sign up and use the basic annotation tool. If publishers wish to host their own branded and moderated annotation layer, there is a charge. There is also a charge to integrate with SSO accounts. We also offer commercial products and services including SaaS hosting, software development and consulting services for organizations and enterprises wanting to implement a more robust digital annotation solution. We charge an annual SaaS hosting fee that covers 12 months of service. We use the number of documents added per year as a proxy for publisher size. Publishers can deploy across all content or specify content types or specific journals, for example. Deployment is back to volume 1, issue 1 or earliest copyright year for books. Pricing for software development and consulting services is negotiable, based on the deliverables and resource requirements of the proposed scope of work. For more information on pricing, contact Heather Staines, Director of Partnerships, at heather@hypothes.is","10,000+","We use Metabase, a data analytics tool, to track usage and participation of the Hypothesis tool on an aggregated basis, by sector, and by customer including total # of annotations, # of active annotators, # of new registered users, and# of new groups created. We also generate reports for customers that include information on publicly visible annotations (annotator, date, content annotated, text selection, annotation content) as well as private and group annotations (date, content annotated). We can also provide information on the top public annotators, top annotated documents, and more. We constantly communicate with our partners to see how they are using the tool and to solicit feedback for improvements and new features.","Hypothesis has integrations with all major hosting platforms, but an integration is not necessary for the tool to be utilized by end users. Content which is currently annotated (4.8M as of 2/28/2019) includes scholarly content and main stream web content.",,,NaN,,Verified,publish 774,Plaudit,2/13/2019,job_listing,https://reimaginereview.asapbio.org/listing/plaudit/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/favicon.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/extfx2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/extch1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/extch2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/extfx1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/favicon_white.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/integrated.171a160e.png,Plaudit|Plaudit|Plaudit|Plaudit|Plaudit|Plaudit|Plaudit,Curation of interesting work|Validation of soundness,Costs|Incentives and recognition for reviewing|Speed|Transparency,Journal accepted manuscripts|Other scholarly outputs|Preprints,Annotations|Quantitative scores,Open identities|Open interaction|Open participation|Open reports,Journal integration|Post-publication review|Reviewer or editor initiates review,All disciplines,,,Project,https://plaudit.pub,,https://gitlab.com/Flockademic/plaudit,Active,2/4/2019,,Flockademic,"Plaudit was created to improve the incentive structure in academia, in which researchers feel pressured to publish their work in ""top-tier"", but often paywalled journals, or journals with a substantial publishing fee. By providing an alternative signal of quality that still relies on the expertise of your academic peers, the hope is that that pressure can be relieved.",No,No,,"Anyone with an ORCID iD, willing to expose their endorsement to public scrutiny.",,,0,,Unknown,All endorsements are made publicly available as open data through CrossRef Event Data,,,,NaN,,Verified,publish 781,Peer Community In,2/13/2019,job_listing,https://reimaginereview.asapbio.org/listing/peer-community-in/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/logo_PCI.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/Essai-17.png,Peer Community In IN PROGRESS|Peer Community In IN PROGRESS,Curation of interesting work|Feedback to authors|Validation of soundness,Costs|Incentives and recognition for reviewing|Quality of review|Transparency,Preprints,Free-form commenting,Double blind|Open identities|Open reports|Single blind,Author initiates review,All disciplines,,,Project,https://peercommunityin.org,,,Active,1/1/2017,,Peer Community In non profit organization,"The motivation behind the “Peer Community in” project is the establishment of a high-quality, free, public system for identifying high-quality preprints by a specific recommendation obtained after rigorous peer-review. The aim is for this system to be recognized both within, and, subsequently beyond the community, including by funding and research agencies. This project should lead to a new scientific publication system, in which preprints are deposited in free, open archives, and, if appropriate, reviewed and awarded a recommendation publicly guaranteeing their scientific quality. This recommendation could replace the current evaluation and editing processes of scientific journals, which are often opaque and very costly for research institutions.",Yes,No,"Editorial process (reviews, editorial decisions, authors' replies) is published if the paper is accepted. Nothing is published if the paper rejected. Reviewers can be anonymous if they want to. Authors can be anonymous to reviewers if they want. Editors must sign a recommendation text in case of acceptance.",New editors of a PCI are nominated by a current editor of this PCI and approved by the Managing Board of the PCI. Reviewers are selected by Editors.,No,"At least an Editor of the PCI has to find the article interesting (for any reasons: interesting question, interesting data, methods, results, discussion, etc.) to handle its evaluation.",0,,"100-1,000","> 500 Editors, 3 PCIs, >50 000 pages visited/year overall PCIs",,,,NaN,,Verified,publish 790,SciRate,2/14/2019,job_listing,https://reimaginereview.asapbio.org/listing/scirate/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/1550116511.png,SciRate,Curation of interesting work|Feedback to authors|Validation of soundness,Quality of review|Transparency,Preprints,Free-form commenting,Open identities|Open interaction|Open participation,Pre-publication review,All disciplines,,,Project,https://scirate.com/,,https://github.com/scirate/scirate,Active,,,SciRate Collaboration,To create a collaborative network of people evaluating and discussing papers.,,No,,,No,"No criteria, papers from the arXiv are displayed.",0,,"10,000+",,,,,NaN,,Verified,publish 792,Author-led peer-review trial by eLife,2/15/2019,job_listing,https://reimaginereview.asapbio.org/listing/author-led-peer-review-trial-by-elife/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/eLife-2.png,Author-led peer-review trial by eLife,Feedback to authors,Incentives and recognition for reviewing|Quality of review|Speed|Transparency,Journal accepted manuscripts,Structured review form,Open identities|Open reports|Single blind,Pre-publication review|Reviewer or editor initiates review,Life Sciences,Editorial decision outcomes|Author opt-in rate or submission rate>Reviewer invitation acceptance rate|Total time to publication,Career stage|Gender|Geographic regions,Trial,https://elifesciences.org/articles/36545,,,Complete,6/26/2018,8/8/2018,eLife,"Three main motivations:
  1. Builds on the consultative approach to peer review that eLife has developed by removing the gatekeeping role of peer review
  2. Could reduce waste and unnecessary work when articles are submitted and reviewed by multiple journals before publication
  3. Encourages evaluation of research based on the more than just the journal name in which the work is published
Examples of data we hope to gather:
  1. percentage of authors who undertake the trial process
  2. percentage of reviewers who agree to participate, and who sign their reviews
  3. percentage of revised papers that address all the concerns raised during review
  4. effect on decision-making before in-depth review
  5. are more papers are ultimately published
  6. speed to publication
",,,The gatekeeping role of peer review is removed. All peer-reviewed articles are published (unless the author chooses to withdraw their article).,,Yes,The articles are labelled as Research Communications to indicate that they are part of the trial.,NaN,There is a publication fee as there is for all published research articles in eLife,"100-1,000",,"In “Peer Review: First results from a trial at eLife” (linked below), we compare the outcomes of 313 trial submissions with 665 regular submissions received during the same period of time. That is, almost a third of authors opted in to the trial approach. So far we have evaluated the first part of the editorial process (whether a paper is sent for in-depth review). The success rates at this first step for male and female last authors in the trial were similar (22.6% and 22.1% respectively), but late-career last authors fared better than their early- and mid-career colleagues. Further data will be posted and discussed covering the rest of the trial process.",https://elifesciences.org/inside-elife/262c4c71/peer-review-first-results-from-a-trial-at-elife,,NaN,,Verified,publish 794,mSphereDirect,2/15/2019,job_listing,https://reimaginereview.asapbio.org/listing/mspheredirect/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/og_image.jpg,mSphereDirect IN PROGRESS,Feedback to authors|Validation of soundness,Speed|Transparency,Journal accepted manuscripts,Free-form commenting|Structured review form,Open identities|Open interaction|Open participation,Author initiates review|Pre-publication review,Life Sciences,,,Trial,https://msphere.asm.org/content/1/6/e00307-16,,,Complete,1/11/2017,2/28/2019,mSphere,"mSphereDirect was a response to the scientific community’s concerns about the current scholarly publishing process: peer review is too slow, the publishing process is too opaque, and journal editors may be too far outside their field when evaluating manuscripts. mSphereDirect put authors in the driver’s seat to secure reviews from suitable experts as rapidly as possible. Its objectives were to support the rapid communication of science, to give authors more control over the peer review process, and to provide guaranteed fast decisions. Additionally, mSphereDirect  increased transparency by publishing reviewer names with original research papers accepted via the mSphereDirect pathway. In this way, mSphereDirect merged two crucial publication concepts: relevant, quality peer review and vetted, rapid decisions.",Yes,No,,Authors selected their own reviewers.,No,,NaN,,10-100,,"This experiment is ending February 2019. A number of issues and concerns surfaced during the experiment. Uptake was low. Authors felt awkward asking colleagues to review their manuscripts, and conversely, some scientists were uncomfortable serving as nonblind reviewers. Additionally, a significant number of the reviews were superficial and uncritical. To maintain editorial standards, manuscripts were rejected from authors who technically followed all the rules, but for which the significance of the work was in question.",https://msphere.asm.org/content/3/6/e00678-18,,NaN,,Verified,publish 796,bims: Biomed news,2/17/2019,job_listing,https://reimaginereview.asapbio.org/listing/biomed-news/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/biomednewslogo.jpg,IN PROGRESS bims: Biomed news IN PROGRESS,Curation of interesting work,Incentives and recognition for reviewing|Speed,Journal accepted manuscripts,,Open identities|Open participation|Open reports,,Life Sciences,,,Project,http://biomed.news/,,,Active,2/4/2017,,"The Open Library Society, Inc.","We aim to disseminate biomedical information through regularly scheduled subject specific reports. We want a system for fast dissemination of research results. We want to promote high quality research by having expert reviewers. At this time, Biomed news is based on PubMed. At this time, our reports are mainly geared to the academic and research communities. This does not have to be the case. PubMed has a lot of papers of more general interest. We can for example think of reports aimed at medical doctors, that filter for papers that have clinical relevance. Such reports may be run by MDs who normally don't do any research. We can think of reports aimed at patients that filter for advice on how to cope with a disease. Such reports may be run by patients, who have no prior biomedical training.",No,No,"Selectors initially choose abstracts relevant to the topic report, these inform the learning for the subsequent issues of the report.  Selectors can also customize the abstracts sent to e-mail subscribers by removing abstracts and also by providing a ranking order.",Topic experts are selected based on topic interest.,No,"At this time, we consider all new items that have been made available during the last week. We remove items that we can not date to the current or the past year.",0,,"1,000-10,000",The amount of time a selector spends on each issue can be tracked.,"Since our founder fixed a major bug on 2018-11-14, no selector has lost interest in maintaining their report. Some selectors maintain several reports, this allows reports of different scopes to be compared by the same selector.",http://biomed.news/reports,,NaN,,Verified,publish 799,Select Crowd Review,2/19/2019,job_listing,https://reimaginereview.asapbio.org/listing/select-crowd-review/,https://reimaginereview.asapbio.org/wp-content/uploads/2019/02/Screen-Shot-2020-02-11-at-7.51.09-PM.png|https://reimaginereview.asapbio.org/wp-content/uploads/listing-uploads/logo/2020/03/Thieme-filestage_logo2-1.jpg,Screen Shot 2020-02-11 at 7.51.09 PM|Thieme-filestage_logo2-1.jpg,Feedback to authors|Validation of soundness,Bias in review|Quality of review|Reviewer training|Speed|Transparency,Privately shared manuscripts,Annotations|Free-form commenting,Single blind,Author initiates review|Pre-publication review|Professional editors,Chemistry,,,Project,https://www.thieme.de/de/thieme-chemistry/select-crowd-review-136859.htm,,,Active,1/1/2017,,Thieme & Filestage,"In the first year after the introduction of Select Crowd Review for SYNLETT, experience has proved that the innovative process delivers substantive feedback to the authors, with the same or higher quality than classical peer review does, in only a few days. This allows editors to decide about the acceptance of a manuscript much faster and therefore shortens the time from submission to publication. We will continue to improve Select Crowd Review together with editors, authors, and reviewers. This innovative process will not just benefit SYNLETT and SynOpen, but potentially the wider scientific community.",Yes,Yes,,"Reviewers (members of the 'crowd') are selected by a member of the editorial board of the journal, based upon their expertise, motivation, and experience in reviewing and in synthetic organic chemistry.",No,,0,,"100-1,000",,,https://www.nature.com/news/crowd-based-peer-review-can-be-good-and-fast-1.22072,,NaN,,Verified,publish 815,biOverlay,2/25/2019,job_listing,https://reimaginereview.asapbio.org/listing/bioverlay/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/logo.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/logo.png,biOverlay|logo.png,Curation of interesting work|Feedback to authors|Validation of soundness,Bias in review|Costs|Incentives and recognition for reviewing|Quality of review|Reviewer training|Speed|Transparency,Journal accepted manuscripts|Other scholarly outputs|Preprints,Free-form commenting|Structured review form,Open identities|Open participation|Open reports|Single blind,Post-publication review|Pre-publication review|Reviewer or editor initiates review,All disciplines|Computer science|Life Sciences,Author opt-in rate or submission rate>Reviewer invitation acceptance rate,,Project,https://bioverlay.org,,,Complete,2/21/2018,1/23/2020,,"",No,No,,"Reviewers are selected by editors based on relevant expertise, similar to journal peer review. Editors self-nominate by submitting a CV and are reviewed by a the board of managing editors.",No,Associate Editors are asked to consider what the most impactful item for review is at the time that they select a preprint or publication to send out for review. An editor has viewed it as the scholarly output most worthy of review at that time.,0,,10-100,,"We expected that preprints reviewed on biOverlay would be among the highest profile contributions within our various disciplines. Many preprints covered by biOverlay were published in numerous journals that seek to publish work of broad interest, suggesting that our associate editors were choosing papers that others found interesting as well. We’re closing biOverlay now for a few reasons. First, it may have made it more difficult for authors to publish strong papers. We are aware of one case where reviews from biOverlay were used, but the authors, in a subsequent blog post, noted that they were then expected to respond to five reviewers. We would prefer that editors and authors take all comments under consideration and take a nuanced perspective to request that authors focus their efforts on the most salient ones. While some comments were overlapping, this didn’t make an apparently strong paper of great general interest more straightforward to publish. Second, without information as to the current submission/review status of a preprint, it was unclear if biOverlay reviews would be helpful. Third, without a clear impact on subsequent publication, potential reviewers lacked one incentive to accept our invitations. All of these factors were cited by potential reviewers when declining invitations. We found that posting a preprint is insufficient to signal to potential reviewers that authors welcome feedback. New efforts, such as Review Commons from EMBO and ASAPbio, eLife’s preprint review trial, and PREreview’s live-streamed journal clubs provide author-initiated platforms to produce a similar outcome. Additionally, bioRxiv and other preprint servers could provide a way for authors to signal that they welcome feedback or report a manuscript’s submission/review status.",https://www.bioverlay.org/post/2020-01-experiment-summary/,,NaN,,Verified,publish 822,Nature open peer review trial,2/26/2019,job_listing,https://reimaginereview.asapbio.org/listing/nature-open-peer-review-trial/,https://reimaginereview.asapbio.org/wp-content/uploads/2019/02/Nature_RGB_red.png|https://reimaginereview.asapbio.org/wp-content/uploads/2019/02/nature-padded.png,Nature_RGB_red|nature padded,Curation of interesting work|Feedback to authors|Validation of soundness,Quality of review|Transparency,Preprints,Free-form commenting,Open identities|Open participation|Open reports,Author initiates review|Comment moderation|Journal integration|Pre-publication review,All disciplines,,,Trial,https://www.nature.com/nature/peerreview/debate/nature05535.html,,,Complete,6/1/2006,9/30/2006,Nature,"""[P]eer review is never perfect and we need to keep it subjected to scrutiny as community expectations and new opportunities evolve. In particular, we felt that it was time to explore a more participative approach.""",,Yes,,Anyone,,,NaN,,10-100,"The fraction of authors opting in, the number of comments and page views, and editor and author surveys","""We sent out a total of 1,369 papers for review during the trial period. The authors of 71 (or 5%) of these agreed to their papers being displayed for open comment. Of the displayed papers, 33 received no comments, while 38 (54%) received a total of 92 technical comments. Of these comments, 49 were to 8 papers. The remaining 30 papers had comments evenly distributed. The most commented-on paper received 10 comments (an evolution paper about post-mating sexual selection). There is no obvious time bias: the papers receiving most comments were evenly spread throughout the trial, and recent papers did not show any waning of interest. The trial received a healthy volume of online traffic: an average of 5,600 html page views per week and about the same for RSS feeds. However, this reader interest did not convert into significant numbers of comments.""",https://www.nature.com/nature/peerreview/debate/nature05535.html,,NaN,,Verified,publish 823,Nature double blind peer review trial,2/26/2019,job_listing,https://reimaginereview.asapbio.org/listing/nature-double-blind-peer-review-trial/,,,Curation of interesting work|Feedback to authors|Validation of soundness,Bias in review,,,Double blind|Single blind,Pre-publication review|Professional editors,All disciplines,,,Trial,https://researchintegrityjournal.biomedcentral.com/articles/10.1186/s41073-018-0049-z,,,Complete,3/1/2015,2/28/2017,Nature,"""Double-blind peer review has been proposed as a possible solution to avoid implicit referee bias in academic publishing. The aims of this study are to analyse the demographics of corresponding authors choosing double-blind peer review and to identify differences in the editorial outcome of manuscripts depending on their review model.""",,,,,,,NaN,,"10,000+","Authors's preference for single- or double-blind review by journal, gender, country, and institution. Editorial decision outcomes","""Author uptake for double-blind submissions was 12% (12,631 out of 106,373). We found a small but significant association between journal tier and review type (p value < 0.001, Cramer’s V = 0.054, df = 2). We had gender information for 50,533 corresponding authors and found no statistically significant difference in the distribution of peer review model between males and females (p value = 0.6179). We had 58,920 records with normalised institutions and a THE rank, and we found that corresponding authors from the less prestigious institutions are more likely to choose double-blind review (p value < 0.001, df = 2, Cramer’s V = 0.106). In the ten countries with the highest number of submissions, we found a large significant association between country and review type (p value < 0.001, df = 10, Cramer’s V = 0.189). The outcome both at first decision and post review is significantly more negative (i.e. a higher likelihood for rejection) for double-blind than single-blind papers (p value < 0.001, df = 1, Cramer’s V = 0.112 for first decision; p value < 0.001; df = 1, Cramer’s V = 0.082 for post-review decision).""",https://researchintegrityjournal.biomedcentral.com/articles/10.1186/s41073-018-0049-z,,NaN,,Verified,publish 824,Interactive Public Peer Review,2/26/2019,job_listing,https://reimaginereview.asapbio.org/listing/interactive-public-peer-review/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/icon_public_peer_review_wt_80x80-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/Public_Peer_Review_Chart_accelerated_access_review_post_discussion_editor_decision.png,Interactive Public Peer Review|Interactive Public Peer Review,Feedback to authors|Validation of soundness,Quality of review|Speed|Transparency,Journal accepted manuscripts|Preprints,Free-form commenting|Structured review form,Open interaction|Open participation|Open reports,Author initiates review|Comment indexing|Journal integration|Pre-publication review|Reviewer or editor initiates review,All disciplines,,,Project,https://publications.copernicus.org/services/public_peer_review.html,,,Active,9/3/2001,,Copernicus Publications,"The process aims to provide both rapid scientific exchange and thorough quality assurance. Through the immediate posting of manuscripts after a swift access review, scientists receive a fast record of their research as a preprint (discussion paper). The Interactive Public Peer Review enhances transparency as referee comments, author comments, and the comments of the scientific community are published online and are openly accessible. However, the process meets the criteria of traditional quality assurance as papers undergo revisions and are only published as final revised papers in the journal after final acceptance by the editor. In summary, the process fosters scientific discussion, maximizes the effectiveness and transparency of scientific quality assurance, enables rapid dissemination of new scientific results, and makes scientific publications freely accessible (Pöschl 2012, 10.3389/fncom.2012.00033, van Edig 2016, https://doi.org/10.3233/978-1-61499-649-1-28). This process is applied successfully in 20 of the 42 journals we publish.",,Yes,"In the access review, editors need to assess whether a manuscript is within the scope of the journal, whether it meets basic scientific quality, and whether it contributes something new to the respective field. After approval of a manuscript for Interactive Public Peer Review, it is posted as a discussion paper (preprint) in the journal’s discussion forum and is citable through a DOI. During the discussion phase different parties can engage in an iterative and developmental reflective process. In the interactive discussion, comments are posted by designated referees (anonymous or named). In addition, all interested members of the scientific community (named) can contribute as well. All participants are encouraged to stimulate further debate rather than simply preserve their position. The overall goal is to enhance the manuscript and thereby maximize the impact of the article. Usually, every discussion paper receives at least two comments by the referees, which are requested by the editor. Authors are encouraged to participate actively by posting author comments as a direct reply to referee comments and short comments from the scientific community. After the end of the open discussion, authors need to publish a response to all comments in case they have not done so during the discussion phase. In journals with post-discussion editor decision, the editor – based on the authors’ responses – either invites the authors to submit a revised manuscript or directly rejects the manuscript. If necessary the editor may also consult referees as is done during traditional peer review. In journals without post-discussion editor decision, authors need to submit their revised manuscript 4 to 8 weeks after the discussion. Taking the access peer review and interactive public discussion into account, the editor either directly accepts/rejects the revised manuscript for publication in the journal or consults referees again. If needed, revisions may be requested during peer-review completion until a final decision about acceptance/rejection is reached.","At least two referees, nominated by the editor, review the discussion paper. In addition, the scientific community is invited to join the interactive public discussion prior to publication. The following types of interactive comments can be submitted for instant non-peer-reviewed appearance alongside the discussion paper: a) Short comments (SCs) can be posted by any registered member of the scientific community (free online registration). Such comments are posted under the name of the person commenting. b) Referee comments (RCs) can only be posted by the referees peer-reviewing the discussion paper. Referees can choose to stay anonymous or to disclose their name. c) Editor comments (ECs) can only be posted by the editor handling the review process of the respective discussion paper. d) Author comments (ACs) can only be posted by the contact author of the discussion paper on behalf of all co-authors. All interactive comments are fully citable, paginated, and archived. All comments receive DOIs.",,"Journals have to select whether they want to apply the Interactive Public Peer Review or not. Hence, if a journal opts for it, all manuscripts submitted to this journal will follow this process.",0,"If a paper is accepted after Interactive Public Peer Review, the article processing charges of the respective journal apply.","10,000+","We provide the article level metrics (downloads, views), impact (citations), saves (bookmarks), and discussion information (social media) for discussion papers as well as final revised papers.","Interactive Public Peer Review has been successfully applied since 2001. Since then, the basic concept has remained unchanged even though additional features have been implemented (accelerated access review, post-discussion editor decision, post-discussion report publication, no more typesetting of the discussion papers). At the moment, 20 out of 42 journals we publish apply this peer-review process. Several journals have switched from applying a traditional peer review to the Interactive Public Peer Review.",,,NaN,,Verified,publish 827,Episciences,2/26/2019,job_listing,https://reimaginereview.asapbio.org/listing/episciences/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/episcience.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/docEN_workflow_publication.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/03/schemaEpisciences_eng.png|https://reimaginereview.asapbio.org/wp-content/uploads/listing-uploads/gallery/2020/01/episciencesWorkflowNewEn.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/episcience.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/03/schemaEpisciences_eng.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/docEN_workflow_publication.png,IN PROGRESS Episciences|IN PROGRESS Episciences|Episciences|episciencesWorkflowNewEn.png|episcience.png|schemaEpisciences_eng.png|docEN_workflow_publication.png,Curation of interesting work|Feedback to authors|Validation of soundness,Costs|Speed|Transparency,Preprints,Free-form commenting|Quantitative scores|Structured review form,Single blind,Journal integration|Post-publication review|Pre-publication review,All disciplines,,,Project,https://www.episciences.org/,,,Active,1/1/2013,,CCSD (Center for Direct Scientific Communication) [CNRS ; INRA ; INRIA ; University of Lyon],"The objectives are to achieve open journals and implement open access to digital versions of articles. The epijournals could be either new titles or existing ones wishing to join the platform. The Episciences.org platform could host epijournals of any scientific disciplines. All data handled and generated by the platform or by its users are under the full control of the academic trustees.",Yes,No,,,No,,0,,"1,000-10,000",,,,,NaN,,Verified,publish 831,Faculty Opinions,2/26/2019,job_listing,https://reimaginereview.asapbio.org/listing/facultyopinions/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/F1000Prime-web.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/F1000Prime-web.png|https://reimaginereview.asapbio.org/wp-content/uploads/2019/02/Screen-Shot-2020-05-24-at-5.31.29-PM.png|https://reimaginereview.asapbio.org/wp-content/uploads/2019/02/Screen-Shot-2020-05-24-at-5.31.07-PM.png,F1000Prime|F1000Prime-web.png|Screen Shot 2020-05-24 at 5.31.29 PM|Screen Shot 2020-05-24 at 5.31.07 PM,Curation of interesting work|Validation of soundness,Incentives and recognition for reviewing|Transparency,Journal accepted manuscripts|Other scholarly outputs|Preprints,Annotations|Free-form commenting|Quantitative scores|Structured review form,Open identities|Open reports,Comment indexing|Post-publication review,Life Sciences,,,Project,https://facultyopinions.com/prime/home?,,,Active,,,Science Now,Faculty Opinions intends to provide a route for qualitative (post-publication) review and commenting - the USP of Faculty Opinions is that the recommendations are provided by independent experts.  Faculty Opinions intends to serve as a complement to more quantitative article-based citation metrics.,,No,"All recommendations are assigned a DOI and are citable and useable in other platforms, and pushed openly to the Crossref article record; user commenting functionality in development and coming soon 2019.",All recommendations have to be provided by an appointed Faculty member.  There are currently c8000 virtual Faculty Members – all are peer nominated and approved by the appointed Heads of each Faculty – all must be Associate Professor (or equivalent) and above.  F1000 manages the services but has no influence in who is appointed nor what is recommended – experts are independent.  ,,"A research output (could be an article, a pre-print, an other type of research output – and we are working to encourage recommendation of a broader range of research output types) has to be recommended by an appointed Faculty Member to be included in Faculty Opinions; any conflicts of interest associated with recommending a specific output must be openly declared.",NaN, It varies depending on the type of the institution and number of active users; though F1000 offers free 30-day trials to all potentially interested clients. Please contact for more information. https://f1000.com/subscriptions ,"10,000+",,,,,NaN,,Verified,publish 847,reffit,2/27/2019,job_listing,https://reimaginereview.asapbio.org/listing/reffit-2/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/reffit-logo-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/2019-02-26-202959_1259x898_scrot-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/2019-02-26-203742_1281x887_scrot.png,reffit|reffit|reffit,Curation of interesting work|Feedback to authors|Validation of soundness,Bias in review|Incentives and recognition for reviewing|Quality of review|Speed|Transparency,Journal accepted manuscripts|Other scholarly outputs,Annotations|Free-form commenting|Quantitative scores,Open identities|Open participation|Single blind,Comment rating (meta-evaluation)|Post-publication review,Computer science|Life Sciences,,,Trial,https://reffit.com,,https://github.com/imalsogreg/reffit,In Preparation,,,Greg Hale,"Mainstream adoption of a critical attitude toward published science Beautiful, impactful, featureful medium for expression Reffit's goal is to absorb the low-key, informal and honest tone of journal club criticism onto a platform where the discussion can be shared and distilled. The comments are free-form, but the site allows commenters to specify whether their comment was positive or negative, and whether it referred to a paper's reproducibility, novelty or coolness, and that metadata is used to get a numerical summary of those feelings.",No,No,,Completely open. A reputation system for reviews being planned.,Yes,"Any user can initiate a discussion about any paper. Inclusion doesn't mean much, but frontpage sorting suggests broad engagement.",0,,10-100,,,,,NaN,,Unverified,publish 851,F1000 Research,2/27/2019,job_listing,https://reimaginereview.asapbio.org/listing/f1000-research/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/F1000Research-400x400.png,F1000 Research,,Bias in review|Costs|Incentives and recognition for reviewing|Quality of review|Reviewer training|Speed|Transparency,,Structured review form,Open identities|Open interaction|Open reports,Author initiates review|Post-publication review,Life Sciences,,,Project,https://f1000research.com,,,Complete,,,,"One of the key tenets of the open research publishing model is to improve the peer review process. Under this model, the role of the invited reviewers is not to accept or reject an article for publication – their remit is to help authors improve their work. This transparent process should encoruage more constructive reviews and civil dialogue between authors and reviewers, who communicate directly with each other, without an editor arbitrating decisions, or acting as gatekeeper. The peer review process becomes a central part of the article and with it being open it allowers readers to understand the full context of the article and see how other experts viewed the work.",,,,,,,NaN,,"1,000-10,000",,,,,NaN,,Verified,publish 859,Involving patients and the public in the peer review process,2/27/2019,job_listing,https://reimaginereview.asapbio.org/listing/involving-patients-and-the-public-in-the-peer-review-process/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/Presentation1-3.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/final-banner-blue-bg.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/icon.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/final-banner-blue-bg-1.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/patient.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/patient-1.jpg,Involving patients and the public in the peer review process|Involving patients and the public in the peer review process – IN PROGRESS|Involving patients and the public in the peer review process – IN PROGRESS|Involving patients and the public in the peer review process – IN PROGRESS|Involving patients and the public in the peer review process – IN PROGRESS|Involving patients and the public in the peer review process – IN PROGRESS,,Quality of review|Transparency,Journal accepted manuscripts|Privately shared manuscripts,Free-form commenting,Open identities|Open reports,Journal integration|Reviewer or editor initiates review,Life Sciences,,,Trial,https://www.bmj.com/campaign/patient-partnership,,,Active,,,BMJ,"Involving patients and the public in the research process has the potential to increase the quality and value of the research and reduce waste. Many grant funding organisations now incorporate members of the public in their review processes or panels, for example, the UK’s National Institute for Health Research (NIHR) and the Patient-Centered Outcomes Research Institute (PCORI) in the USA. By contrast, biomedical journals still do very little to involve patients and the public in their peer review processes. In 2014 The BMJ adopted an innovative strategy to coproduce its content with patients and the public. To ensure research is appropriate and relevant to end users, The BMJ now systematically incorporates patient and public review of research manuscripts (and some other article types) alongside traditional scientific peer review. Reviews are signed and published alongside accepted articles. We have a series of research projects planned to evaluate this important initiative.",,,"Patient and public reviewers are registered on the manuscript tracking software (ScholarOne) and invited by editors to review at the same time as peer reviewers, directly from the platform.",All patients and members of the public can sign up as a reviewer by completing our registration form (https://www.bmj.com/about-bmj/resources-reviewers/guidance-patient-reviewers). There are no inclusion/exclusion criteria.,,,NaN,,0,We recently surveyed patient reviewers to hear their views on the experience (see below). We also track time taken to review and acceptance to review rates.,"We have received a high level of engagement with the initiative. Acceptance to review rates are good and reviews are timely. Reviewers' experiences are largely positive but they have also given us feedback on how to improve their experience. We have published the results from a survey describing early experiences (https://bmjopen.bmj.com/content/8/9/e023357). Further research into the value added by patient and public review is planned.  ",https://bmjopen.bmj.com/content/8/9/e023357,,NaN,,Verified,publish 873,Publons,2/27/2019,job_listing,https://reimaginereview.asapbio.org/listing/publons/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/square-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/Stats-profile-public-profile-1650x900-1-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/blue_white_shadow.png,IN PROGRESS Publons|IN PROGRESS Publons|IN PROGRESS Publons,,Incentives and recognition for reviewing|Quality of review|Reviewer training|Transparency,,,,,,,,Project,https://publons.com/,,,Active,,,,"",,,,,,,NaN,,0,,,,,NaN,,Verified,publish 878,Effects of training on quality of peer review: randomised controlled trial,2/27/2019,job_listing,https://reimaginereview.asapbio.org/listing/effects-of-training-on-quality-of-peer-review-randomised-controlled-trial-2/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/BMJ-Logo-for-website.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/F1.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/training.jpg,Effects of training on quality of peer review: randomised controlled trial – IN PROGRESS|Effects of training on quality of peer review: randomised controlled trial – IN PROGRESS|Effects of training on quality of peer review: randomised controlled trial – IN PROGRESS,,Quality of review|Reviewer training,Privately shared manuscripts,,,,Life Sciences,,,Trial,https://www.bmj.com/content/328/7441/673,,,Complete,,,BMJ,"Many studies have illustrated the inadequacies of peer review and its limitations in improving the quality of research papers. However, few studies have evaluated interventions that try to improve peer review. We conducted a randomised controlled trial to examine the effects of training. Training that would be feasible for reviewers to undergo and for a journal to provide would have to be short or provided at a distance. Although the effectiveness of short educational interventions is questionable, some brief interventions have been shown to be successful (depending on what is being taught and the methods used). We aimed to determine whether reviewers for The BMJ who underwent training would produce reviews of better quality than those who received no training; whether face to face training would be more beneficial than a self taught package; and whether any training effect would last at least six months. The training materials for the trial are freely available on bmj.com (https://www.bmj.com/about-bmj/resources-reviewers/training-materials).",,,,All UK reviewers who had recently completed a review at the time of the study were eligible and invited to take part in a research project.,,,NaN,,10-Jan,,"Reviewers in the self taught group scored higher in review quality after training than did the control group, but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on time taken to review but was associated with an increased likelihood of recommending rejection. (1) Schroter S, Black N, Evans S, Smith R, Carpenter J, Godlee F. Effects of training on the quality of peer review: A randomised controlled trial. BMJ 2004;328:673-5.  (2) Schroter S, Black N, Evans S, Godlee F, Osorio L, Smith R. What errors do peer reviewers detect, and does training improve their ability to detect them?  J R Soc Med 2008:101: 507-14.",https://www.bmj.com/content/328/7441/673,,NaN,,Verified,publish 895,Impact of assessing during peer review the consistency between the CONSORT checklist & the submitted manuscript on the completeness of reporting,2/28/2019,job_listing,https://reimaginereview.asapbio.org/listing/impact-of-assessing-during-peer-review-the-consistency-between-the-consort-checklist-the-submitted-manuscript-on-the-completeness-of-reporting/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/BMJ-Logo-for-website-1.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/Logo-Nuevo-Resize.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/David.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/Logo-square.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/new-square.png,Evaluating the impact of assessing during peer review the CONSORT checklist submitted by authors – IN PROGRESS|Evaluating the impact of assessing during peer review the CONSORT checklist submitted by authors – IN PROGRESS|Evaluating the impact of assessing during peer review the CONSORT checklist submitted by authors – IN PROGRESS|Evaluating the impact of assessing during peer review the consistency between the CONSORT checklist & the manuscript submitted by authors on the completeness of reporting – IN PROGRESS|Evaluating the impact of assessing during peer review the consistency between the CONSORT checklist & the manuscript submitted by authors on the completeness of reporting – IN PROGRESS,Feedback to authors,Quality of review|Transparency,,Free-form commenting,Open identities|Open reports,Journal integration|Reviewer or editor initiates review,Life Sciences,,,Trial,https://clinicaltrials.gov/ct2/show/NCT03751878,,,Active,12/1/2018,7/31/2019,David Blanco's PhD project under supervision of Erik Cobo at Universitat Politècnica de Catalunya and Jamie Kirkham at Liverpool University and in collaboration with BMJ Open.,"Randomised trials are considered the gold standard in medical research. The CONSORT Statement aims to improve the quality of reporting of randomised trials. Without transparent reporting, readers cannot judge the reliability of trial findings, therefore these findings cannot inform clinical practice. Different stakeholders, including biomedical journals, have taken different actions to try to improve author adherence to CONSORT. The most popular one is to instruct authors to submit a completed CONSORT checklist with page numbers indicating where the CONSORT items are addressed when they submit their manuscript. However, this measure alone has proven not to be effective. In this study, we intend to assess whether evaluating the checklists submitted by authors and providing feedback to them as part of the peer review process improves the completeness of reporting of published trials.",,Yes,"Randomised controlled trials in the intervention group will receive an additional peer review (conducted by the principal investigator) focused entirely on the consistency of reporting between the submitted manuscript and the completed CONSORT checklist. Where inconsistencies are found, authors of these manuscripts will be asked for changes in relation to the items. Trials in the control group will undergo the usual peer review process.","Eligibility criteria: Manuscripts are eligible for our study if (i) they have been submitted to BMJ Open, (ii) they are original research submissions reporting the results of a randomised trial and (iii) they have passed the first editorial filter and have been subsequently sent out for peer review.",,,NaN,,0,,"This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 676207. Results will be available n September 2019.",,,NaN,https://clinicaltrials.gov/ct2/show/NCT03751878,Verified,publish 904,Transparent Peer Review,2/28/2019,job_listing,https://reimaginereview.asapbio.org/listing/transparent-peer-review/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/Wiley-logo-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/chris-tpr-blog-pic-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/Wiley-logo-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/Wiley-Logo-Square.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/Wiley-Logo-Square.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/02/chris-tpr-blog-pic-1.png,Transparent Peer Review|Transparent Peer Review|Transparent Peer Review|Transparent Peer Review|Wiley-Logo-Square.png|chris-tpr-blog-pic-1.png,,Incentives and recognition for reviewing|Quality of review|Transparency,Journal accepted manuscripts|Other scholarly outputs,Structured review form,Double blind|Open identities|Open reports|Single blind,Professional editors,All disciplines,,,Project,https://www.wiley.com/network/researchers/submission-and-navigating-peer-review/progressing-towards-transparency-more-journals-join-our-transparent-peer-review-pilot,,,Active,9/13/2018,,"Operated by Wiley, together with Publons and ScholarOne (both part of Clarivate Analytics).","Wiley is committed to moving towards greater openness and reproducibility of research, including increasing transparency in peer review. A transparent peer review workflow shows readers the process behind editorial decision making, increases accountability, and helps recognise the work of editors and peer reviewers.   Journals joining the pilot have the flexibility to incorporate transparency into their existing workflows and processes without making other major changes to how they conduct peer review. When combined with author choice (and peer reviewer choices) this approach provides a very flexible way to open-up peer review.   In recognising the need to  change the conversation around transparent peer review and learn more, it’s vital that any initiative has the capability to scale and be compatible with different peer review models, across diverse subject disciplines and publisher workflows. We have taken a collaborative approach working with Publons and ScholarOne to enable this.",,Yes,,Editors invite reviewers to participate. The reviewers are then asked to agree to transparent peer review and it is optional to have their name disclosed.,,,0,,"100-1,000","It is possible to track how many people view the peer review history, linked from the article to Publons.","The pilot started in September 2018 with the inclusion of the first journal Clinical Genetics.  As of end January 2019, 77% of authors have agreed to transparent peer review (417 out of 528 submissions) and 23% of reviewers have signed their reports. These initial results will be presented at the Open Science Conference in Berlin (March 2019).   The pilot was extended in January 2019 to include a further ten journals, and we will be reviewing how the pilot is progressing in due course.",https://www.open-science-conference.eu/%20,,NaN,,Verified,publish 907,Effect of open peer review on quality of reviews and on reviewers'recommendations: a randomised trial,2/28/2019,job_listing,https://reimaginereview.asapbio.org/listing/effect-of-open-peer-review-on-quality-of-reviews-and-on-reviewersrecommendations-a-randomised-trial/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/BMJ-Logo-for-website-2.jpg,Effect of open peer review on quality of reviews and on reviewers’recommendations: a randomised trial – IN PROGRESS,,Quality of review|Speed|Transparency,,Free-form commenting,Open identities,Journal integration|Reviewer or editor initiates review,Life Sciences,,,Trial,https://www.bmj.com/content/318/7175/23,,,Complete,,,BMJ,"Previous RCTs at The BMJ failed to confirm that blinding of reviewers to authors' identities improved the quality of reviews so we decided to experiment with open identities peer review. To us it seemed unjust that authors should be “judged” by reviewers hiding behind anonymity: either both should be unknown or both known, and it is impossible to blind reviewers to the identity of authors all of the time. We therefore conducted a randomised trial to confirm that open review did not lead to poorer quality opinions than traditional review and that reviewers would not refuse to review openly (because open review would then be unworkable).",No,,,"Consecutive manuscripts received by the BMJ and sent by editors for peer review during the first seven weeks of 1998 were eligible for inclusion. Four potential clinical reviewers were selected by one of the BMJ's 13 editors. Two of these four reviewers were chosen to review the manuscript. The other two were kept in reserve in case a selected reviewer declined. The selected reviewers were randomised either to be asked to have their identity revealed to authors (intervention group) or to remain anonymous (control group), forming a paired sample.",,,NaN,,"100-1,000",A questionnaire survey was undertaken of the authors of a cohort of manuscripts submitted for publication to find out their views on open peer review.,"Asking reviewers to consent to being identified to the author had no important effect on the quality of the review, the recommendation regarding publication, or the time taken to review, but it significantly increased the likelihood of reviewers declining to review. Most authors were in favour of open peer review. (van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers' recommendations: A randomised trial. BMJ 1999; 318: 23 – 27.)",https://www.bmj.com/content/318/7175/23,,NaN,,Verified,publish 911,Axios Review,2/28/2019,job_listing,https://reimaginereview.asapbio.org/listing/axios-review/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/02/blue-X-2.png,Axios Review,Curation of interesting work|Feedback to authors|Validation of soundness,Bias in review|Costs|Quality of review|Speed,Preprints|Privately shared manuscripts,Annotations|Free-form commenting|Structured review form,Double blind|Single blind,Author initiates review|Journal integration|Pre-publication review,Life Sciences,,,Project,https://dynamicecology.wordpress.com/2015/03/05/axios-review-is-working-you-should-try-it/,,,Complete,2/1/2013,2/23/2017,,"The current process for publishing scientific research is highly inefficient: - Researchers need to publish their papers in high profile journals to win funding. - High profile journals receive many thousands of research papers every year. - 70 to 95% of these papers are rejected, the majority for poor fit or lack of novelty. - Researchers must then submit their papers to another journal. - Most papers are assessed again and again by different journals before being published. - This takes up researcher time, raises the cost of journals, and slows the progress of science.",No,No,"Axios Review applies the long-established brokerage model to facilitate the interaction between authors and journals: - Scientists send us their papers and suggest target journals. - We use standard editor-led anonymous peer review to evaluate the research. - We offer the paper to the appropriate target journals. - The journals use our evaluation to decide whether they want the paper. - Once we have found an interested journal, authors revise their paper and send it in. - Journals either re-review the paper or accept it for publication immediately Scientists can thus submit to a journal that wants their paper, and journals receive papers they actually want.","Axios staff recruited academic Editors, the Editors picked reviewers (with occasional help from Axios staff)",No,Articles reviewed by Axios generally appeared in normal academic journals. Authors were asked to add a line thanking Axios Review for feedback to their acknowledgements,250,"The fee covered i) editorial office & staff salaries, ii) overheads, and iii) promotional activities.","100-1,000",We collected lots of metrics.,"We received 470 submissions, with almost 200 of those arriving in 2016. At the last time of checking (early 2017) 130 had been published in academic journals, mostly high profile journals in ecology and evolution. We consistently found that 85% of Axios papers were accepted at the interested journal (compared to under 30% for the current system). Half of the accepted Axios papers are not re-reviewed by the journal, greatly increasing the speed of the publishing process. Most Axios papers were published with fewer than 2 rounds of review, compared to over 5 in the current system. Excluding the time authors spend on revisions, we averaged 3 months between the paper arriving at Axios and being published in a journal.",,,NaN,,Verified,publish 963,Peerage of Science,3/5/2019,job_listing,https://reimaginereview.asapbio.org/listing/peerage-of-science/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/peerage_logo.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/03/infograph1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/03/infograph2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/03/infograph3.png,Peerage of Science|Peerage of Science|Peerage of Science|Peerage of Science,Feedback to authors|Validation of soundness,Bias in review|Costs|Incentives and recognition for reviewing|Quality of review|Reviewer training|Speed|Transparency,Other scholarly outputs|Preprints|Privately shared manuscripts,Free-form commenting|Quantitative scores|Structured review form,Double blind|Open interaction|Open participation|Single blind,Author initiates review|Comment rating (meta-evaluation)|Journal integration,All disciplines|Life Sciences,,,Project,https://peerageofscience.org,,,Active,11/1/2011,,Peerage of Science Ltd,"Once upon a time in a land probably far, far away from where you are now, a postdoc was (as they often are) sadly noticing another year commencing on the peer review process of his manuscript. It was his own fault of course: it had been his own darn choice to start the descent down the journal prestige ladder from the top, where the steps are most slippery. Yet, it was dawning on the young scientist that this was also his illusions about scientific peer review meeting reality. Perhaps he should have paid more attention to the words of Richard Horton: ""We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong."" Let’s change that.",Yes,Yes,"https://www.peerageofscience.org/how-it-works/overview/   https://www.peerageofscience.org/how-it-works/process-flow/    ","Reviewers: external verification that there exists at least one article where the person is the first or corresponding author, published in a PubMed- or ISI -indexed journal. Once verified, person gets Peer status, and thereafter can access and engage any manuscript, except those submitted by authors affiliated with Peer via institution or co-authorship with last 3 years. Editors: appointed by participating journals.",,"Peerage of Science itself does not publish or publicly display the outcomes, but facilitates the selection processes of participating publishers. Access to manuscripts and peer reviews is strictly limited to validated Peers and editors. Submissions are pre-screened before posting to the community, and articles judged pseudoscience are rejected. Peerage of Science itself does not appoint or solicit reviewers, nor does it limit the number of reviewers. Some manuscripts fail to get reviews at all (ca 26% of submissions), most get at least one, while some have received eight full Essays.",0,,"1,000-10,000",,,,,NaN,,Verified,publish 976,protocols.io,3/6/2019,job_listing,https://reimaginereview.asapbio.org/listing/protocols-io/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/Screen-Shot-2019-03-05-at-5.17.17-PM.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/Screen-Shot-2019-03-05-at-5.19.41-PM.png,protocols.io|protocols.io,Feedback to authors|Validation of soundness,Costs|Speed|Transparency,Other scholarly outputs,Annotations|Free-form commenting,Open identities|Open interaction|Open participation,Post-publication review|Reviewer or editor initiates review,,,,Project,https://www.protocols.io/,,,Active,2/11/2014,,protocols.io,"The protocols.io platform was inspired by the frustration of one of the co-founders regarding lack of a central space to share corrections and optimizations of previously-published methods. After launching, the team behind protocols.io realized that most research papers do not contain adequate descriptions of the methods. So the mission of the company evolved into encouraging culture change towards better reporting of the details upon publication and then enabling scientists to keep the methods up-to-date.",,,"Handled by competent reviewers with good editors, pre-publication peer review improves the submitted papers. However, improving quality is not the same as ensuring it. A typical reviewer is unlikely to verify the scripts and rerun analyses reported in a genomics paper. Similarly, it would be highly unusual for a reviewer of an experimental molecular biology paper to dash to the lab and attempt to reproduce the reported results as part of checking the paper. The peer review that can truly verify any given paper is the gradual process of replication and extension of the published work over time. When authors of a given paper or other researchers follow up on a publication – then and only then – do we really catch mistakes, improve, and verify the work. This is why it is so important to report not only the results, but the code, data, and the detailed methods used to produce these results. This is why the efforts of publishers like GigaScience are so important. We were driven to create protocols.io by the desire to establish a central place where scientists can share and discover optimizations and corrections of published methods; our hope is that through versioning and collaborative sharing post-publication, methods can evolve and remain up-to-date, long after the original publication. Excellent example of such community review: https://www.protocols.io/view/cut-run-targeted-in-situ-genome-wide-profiling-wit-mgjc3un",Any registered user is allowed to comment.,Yes,"We try to expose all possible metrics that may be helpful in assessing whether a protocol is likely to work. Each public protocol has a dedicated ""metrics"" tab with this information. Example: https://www.protocols.io/view/cut-run-targeted-in-situ-genome-wide-profiling-wit-mgjc3un/metrics",0,"The protocols.io platform is free to read and share for public content. For private collaboration, there is a limit of 5 free private protocols per user, requiring a $5/month subscription above that.","100-1,000",,,,,NaN,,Verified,publish 982,Publishing review reports (PRR),3/6/2019,job_listing,https://reimaginereview.asapbio.org/listing/publishing-review-reports-prr/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/03/41467_2018_8250_Fig4_HTML.png,Publishing review reports (PRR),,Bias in review|Incentives and recognition for reviewing|Quality of review|Speed|Transparency,Journal accepted manuscripts,Free-form commenting,Open identities|Open reports|Single blind,Comment indexing|Professional editors|Reviewer or editor initiates review,,,,Trial,https://www.nature.com/articles/s41467-018-08250-2,,,Complete,11/10/2014,12/31/2017,Elsevier,"Our aim was to understand whether knowing that their report would be published affected the referees’ willingness to review, the type of recommendations, the turn-around time and the tone of the report. These are all aspects that must be considered when assessing the viability and sustainability of open peer review. By reconstructing the gender and academic status of referees, we also wanted to understand whether these innovations were perceived differently by certain categories of scholars",Yes,Yes,,Editors are appointed for the journals. They choose reviewers and manage the peer review process.,No,,NaN,,"10,000+","Yes, article page usage data on Science Direct shows one out of three click on the pilot journal article pages is on the peer review reports. Surveying editors of participating journals shows editors are using published peer review reports as a resource material for training young reviewers.","we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.",https://www.nature.com/articles/s41467-018-08250-2,,NaN,,Verified,publish 1015,Grassroots Journals,3/6/2019,job_listing,https://reimaginereview.asapbio.org/listing/grassroots-journals/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/logo_grassroots.png,Grassroots Journals,,Costs|Quality of review|Transparency,Other scholarly outputs,Annotations|Free-form commenting|Quantitative scores|Structured review form,Open identities|Open interaction|Open participation|Open reports|Single blind,Comment moderation|Post-publication review|Professional editors|Reviewer or editor initiates review,All disciplines|Earth & space sciences,,,Project,https://grassroots.is/,,https://github.com/grassrootsjournals,In Preparation,,,Victor Venema,Breaking the power of the traditional publishers and bringing the assessment what good science is back to the scientific community.,,No,,"The editors of a Grassroots Journal invite experts to review, but everyone else is also welcome to add comments.",,,0,,10-Jan,,,,,NaN,,Verified,publish 1022,Science Open Reviewed,3/6/2019,job_listing,https://reimaginereview.asapbio.org/listing/science-open-reviewed/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/Twitter-picture-1.jpg,Science Open Reviewed,Curation of interesting work|Feedback to authors|Validation of soundness,Bias in review|Costs|Incentives and recognition for reviewing|Quality of review|Transparency,Journal accepted manuscripts|Preprints,Free-form commenting,Open identities,Author initiates review|Journal integration|Pre-publication review,All disciplines,,,Trial,https://science-open-reviewed.com/webapp/,,,NA,,,,"The mission of SciOR is to promote the progress of science as a mission for discovery, rather than a mission for elitism.  Specifically, SciOR provides support for extension and refinement of mechanisms and opportunities for: (1) increasing reviewer incentive and reputation rewards, and thus increasing the pool of available and willing reviewers; and (2) promoting transparent and accountable open peer-review, and thus limiting the incidence of biased and poor quality reviews often associated with blind reviewing.",,No,,"Authors invite reviewers or respond to potential reviewers who have contacted the author. Reviewers are eligible if they hold a PhD, or are registered in a PhD program at an accredited institution",No,"Authors submit their reviewed paper to the journal of their choice, together with the reviews, and the identification of the reviewers.",0,,0,,,,,NaN,,Verified,publish 1062,In Review,3/7/2019,job_listing,https://reimaginereview.asapbio.org/listing/in-review/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/03/mac-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/03/mac-1.png,In Review – In Progress|mac-1.png,,Speed|Transparency,Preprints,Annotations|Free-form commenting,,Journal integration|Professional editors|Reviewer or editor initiates review,,,,Project,https://www.springernature.com/gp/authors/campaigns/in-review,,,Active,,,Springer Nature and Research Square,"Created in partnership with Research Square,  In Review gives researchers the ability to share their work in a citeable way on submission to Springer Nature journals. This allows them to engage the community in discussion to help improve their article either through the public commenting tool or the Hypothes.is open annotation tool. We anticipate early sharing will help authors strengthen their manuscripts as well as spur collaboration opportunities.  In Review also gives authors a way to track the status of their manuscript in real-time through the peer review process, including when reviewers have been invited, and reviewer reports have been received. This increased transparency has been well received by authors who have opted-in to the service since October 2018.",,Yes,"In Review is integrated with journals. The platform hosts information about the peer review timeline and status publicly. In addition, there is a private author dashboard which enables authors to read the peer reviews reports and decision letters. Authors can also use the dashboard to recommend reviewers. One important feature is that manuscripts that are not accepted for publication at the journal will remain on the platform as preprints. This allows authors to continue to share and get feedback on their manuscript, but also give them the freedom to submit to other journals.  The peer review history is only available privately through the author dashboard. As peer review becomes more open over time, full peer reviews may be made available publicly on the platform.",Anyone is able to comment on an article.,Yes,All submissions undergo an Integrity Check at the journal it was submitted.  This is denoted with a checkmark icon on the article page. A drop-down list shows the checks that have been completed.,0,,"10,000+",We track author uptake of the service (averages a 36% as at November 2019) and unique article views.,"As at November 2019, In Review is implemented for 125 Springer Nature journals.  There are more than 10,000 manuscripts on the Research Square platform (including revised versions of manuscripts posted).  has been rolled out on eight BMC journals.   Springer Nature will be rolling this out across many more journals in 2020.",,,NaN,,Verified,publish 1140,WikiJournal User Group,3/19/2019,job_listing,https://reimaginereview.asapbio.org/listing/wikijournal-user-group/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/245px-WikiJournal_logo.svg_-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/03/800px-WikiJournal_of_Science_publishing_pipeline_wiki_first.svg_-1.png,WikiJournal User Group|WikiJournal User Group,Feedback to authors|Validation of soundness,Costs|Quality of review|Transparency,Journal accepted manuscripts|Preprints,Annotations|Free-form commenting,Double blind|Open identities|Open interaction|Open participation|Open reports|Single blind,Author initiates review|Post-publication review|Professional editors,,,,Project,https://en.wikiversity.org/wiki/WikiJournal_User_Group,,https://www.mediawiki.org/,Active,3/28/2014,,"Thomas Shafee, EiC WikiJSci","The main original goal of WikiJournals is to encourage and recognize contributions to Wikipedia by academics. To do this, the WikiJournals allow Wikipedia articles to have named authors, DOIs, a stable PDF version of record, and to undergo academic peer review. Moreover, WikiJournals strive to achieve a broad thematic scope that covers practically all fields of research, and to provide a zero cost, open access publishing venue for the fields that lack one. And finally, WikiJournals implement recognized best practices, in particular a form of open peer review where not only the reviews are made public, but also anyone can contribute.",,Yes,"Thanks to the MediaWiki platform, peer review at WikiJournals is extremely flexible. While comments themselves are always made public, the authors, reviewers and other participants may be named, pseudonymous or anonymous. Interactions between all the participants are possible before, during and after the review process. Reviewers may choose to perform small corrections themselves, rather than suggesting them to authors. And authors are able to revert modifications thanks to version control.","Editors are selected for their professional credentials as researchers, their experience with academic publishing, and their experience with open projects and in particular Wikipedia. Reviewers are selected for their status as experts in the reviewed article's subject matter. As in Wikipedia, anyone is eligible to comment: it is not even necessary to create an account.",,"Articles are accepted for publication (inc, PDF, doi, indexing etc) once they have been been assessed by at least two external peer reviewers (invited by the editorial board) and their responses and edits have been agreed to by both the peer reviewers and editorial board. Suitable material is integrated into Wikipedia, as a 'living version' where it is available for post-publication review and freely editable by anyone.",0,"All editors are currently volunteers, hosting costs supported by Wikimedia Foundation",10-100,"Currently tracked are: The number of times that an article is read in the journal and on Wikipedia (for those integrated into Wikipedia); Number of pre-publication peer reviewers; Post-publication peer review; The Social media impact of each article by AltMetric.com. Later, an impact factor will be sought by ISI Web of Science.","As of 2019: 49 submitted articles,  36 accepted for publication, 12 declined, 1 withdrawn.25 currently in review. Of the 36 published, 27 integrated into Wikipedia (of which 8 were adapted by updating/overhauling existing Wikipedia content). . Mean number of external peer reviewers = 3.1 per article, additional suggestions from editors = 1.2 per article, spontaneous pre-publication suggestions from readers = 0.1 per article. Extensive post-publication commentary performed on 'living' Wikipedia versions of articles. 75% of peer reviewers choose to have their identity open. WikiJournal articles integrated in full or part into Wikipedia receive 6.9 million views per annum so far. Combined citations 121 per G-scholar. Combined AltMetric score >200.",,,NaN,,Verified,publish 1150,"Karma Credits (and Blockchain/Cryptocurrency Incentives) for Reviewers, Editors, Authors",3/26/2019,job_listing,https://reimaginereview.asapbio.org/listing/karma-credits-and-blockchain-cryptocurrency-incentives-for-reviewers-editors-authors/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/karma.png,"Karma Credits for Reviewers, Editors, Authors",,Incentives and recognition for reviewing,,,,,,,,Project,https://jmir.zendesk.com/hc/en-us/articles/115001104247,,,Active,3/1/2016,,JMIR Publications,"Incentivize peer-reviewers, editors, and authors, and provide a fair remuneration in form of APC discounts to our community members, commensurating with their productivity and involvement.",,,,,,,NaN,,"1,000-10,000",,,,,NaN,,Verified,publish 1155,Blockchain for Peer Review,3/26/2019,job_listing,https://reimaginereview.asapbio.org/listing/blockchain-for-peer-review/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/03/Screen-Shot-2019-03-26-at-15.07.41.png,Blockchain for Peer Review,,Incentives and recognition for reviewing|Quality of review|Speed|Transparency,,,,,,,,Trial,https://www.blockchainpeerreview.org/,,,In Preparation,,,,"We believe that most of the problems around the peer review process are caused by the fact that, due to the confidential nature of the process, peer review is traditionally siloed and a 'black box'. The lack of transparency can lead to fraud and manipulation, results in a lack of recognition for reviewers, and hinders the exchange of review history between journals. Moreover, the lack of coordination between journals and publishers, especially on shared reviewers, leads to many inefficiencies. The project's aim is to build a shared infrastructure on a not-for-profit basis that will allow all participants in he ecosystem (publishers, institutions, funders, and of course editors, authors and reviewers themselves) to exchange information around the process, while respecting the confidential nature of the process.",,,,,,,NaN,,0,,,,,NaN,,Verified,publish 1182,Peer Feedback,4/3/2019,job_listing,https://reimaginereview.asapbio.org/listing/peer-feedback/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/Peer-feedback-evaluated-preprint-1.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/Peer-feedback-evaluated-preprint-1.jpg,Peer Feedback|Peer-feedback-evaluated-preprint-1.jpg,Feedback to authors|Validation of soundness,Quality of review|Speed|Transparency,Preprints,Free-form commenting,Open reports,Author initiates review|Journal integration|Professional editors,Life Sciences,,,Project,https://asapbio.org/peer-feedback,,,NA,,,"Ron Vale, Tony Hyman, and Jessica Polka","By conducing peer review before journal submission, Peer Feedback aims to make review more constructive by focusing attention on the science in a manuscript, not its suitability for a given journal. It also aims to reduce repeated peer reviewing that can occur during serial journal submission.",,No,,,,"Upon receipt, submissions are subjected to an initial screening process that checks for adherence to relevant publishing guidelines such as plagiarism. However, the existence of an Evaluated Preprint by itself does not imply quality; readers would have to evaluate the reviews themselves.",NaN,,,,,,,NaN,,Verified,publish 1185,HIRMEOS - open peer review experiment,4/4/2019,job_listing,https://reimaginereview.asapbio.org/listing/hirmeos-open-peer-review-experiment/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/04/HIRMEOS-LOGO2-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/Enssib-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/Casa-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/ENS-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/OEP-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/04/HIRMEOS-LOGO2-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/Enssib-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/Casa-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/ENS-2.png|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/OEP-2.png,HIRMEOS – open peer review experiment|HIRMEOS – open peer review experiment|HIRMEOS – open peer review experiment|HIRMEOS – open peer review experiment|HIRMEOS – open peer review experiment|HIRMEOS-LOGO2-2.png|Enssib-2.png|Casa-2.png|ENS-2.png|OEP-2.png,Feedback to authors|Validation of soundness,Bias in review|Quality of review|Speed|Transparency,Other scholarly outputs,Annotations,Open identities|Open interaction|Open participation|Open reports,Comment moderation|Post-publication review,Humanities|Social science,,,Trial,https://oep.hypotheses.org/2122,,,Complete,2/7/2019,6/1/2019,"OpenEdition (CNRS, EHESS, Aix-Marseille Université)","The objective is to create a space for scientific conversation around publications. We want to allow readers to share their opinions, their knowledge linked to books subjects, directly at the time of reading. We expect participants to write scientific, constructive and argued annotations in order to: Offer a critique of the ideas that make up the texts, achieve fact-checking in the books, open new avenues for reflection, add elements enriching the publications, and interact with each other (reply feature). We also want to offer an innovative service which could be useful to explore the value of opening new peer review processes, allowing users from across the humanities and social sciences, to develop new usages.",,,,,,,NaN,,"100-1,000",,,https://zenodo.org/record/3275651#.Xk_rBErjKUk,,NaN,,Verified,publish 1254,PLOS Channels,4/11/2019,job_listing,https://reimaginereview.asapbio.org/listing/plos-channels/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/04/DhdLdZ9h_400x400.jpg,PLOS Channels,Curation of interesting work,,Journal accepted manuscripts|Other scholarly outputs|Preprints,,,Post-publication review,All disciplines,,,Project,https://channels.plos.org/,,,Active,3/6/2017,,PLOS,"The project is the result of community feedback who wanted a single location to discover research and news in their field, and also authors of PLOS ONE who were concerned their papers were being ""lost"" in a megajournal. Users reported being overwhelmed by the amount of literature (both peer-reviewed and not) and wanted guidance from trusted and respected leaders in their field as to the most interesting and impactful content on an easy to use and regularly updated platform. User-led design resulted in the decision to be publisher agnostic. Publishers have previously created showcases for their own content but these were reported by users as of limited use given it showed just a fraction of the relevant material.",,No,"Channel Editors are provided with a regular content report to review. The content report is generated via PubMed and PLOS search. When submitting to PLOS journals, authors can nominate their submission for inclusion in the Channel once published. Any user can nominate content for the Channel by email or via social media. Direct nominations are noted to Channel Editors in the report.",Channel Editors are recruited by PLOS.,,"Any content that is judged to be in the Channel's scope can be included. Inclusion is entirely at the discretion of Channel Editors, who are volunteers and experts in their field.",0,,"1,000-10,000",We track site visits and interactions; author nominations when submitting to PLOS; impact on ALMs for PLOS content.,,,,NaN,,Verified,publish 1262,SciPost,4/13/2019,job_listing,https://reimaginereview.asapbio.org/listing/scipost/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/04/logo_scipost_with_bgd_square.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/04/Screenshot-2019-04-02-at-11.16.21-AM.png,SciPost|SciPost,Curation of interesting work|Feedback to authors|Validation of soundness,Costs|Incentives and recognition for reviewing|Quality of review|Transparency,Journal accepted manuscripts|Other scholarly outputs|Preprints,Free-form commenting|Structured review form,Open participation|Open reports,Comment indexing|Comment moderation|Comment rating (meta-evaluation)|Post-publication review|Pre-publication review|Professional editors|Reviewer or editor initiates review,All disciplines|Physics,,,Project,https://scipost.org,,https://code.scipost.org,Active,4/20/2016,,SciPost Foundation,"SciPost was born out of the recognition that the currently available publishing infrastructure does not properly serve the interests of science, and that these are best served by scientists themselves. The initiative aims to reform all aspects related to scientific publishing. In particular, SciPost's mission is  to:  ",Yes,Yes,"Our peer-witnessed refereeing procedure is described in detail on our author guidelines page. https://scipost.org/submissions/author_guidelines    ",Editors are Fellows of our Editorial Colleges. Reviewers are invited among the community. Professional scientists are further able to volunteer reports and comments.,Yes,,0,We operate using a consortial sponsorship model.,"100-1,000",,,,,NaN,,Verified,publish 1277,"Open Peer Review, BMC Group",4/17/2019,job_listing,https://reimaginereview.asapbio.org/listing/open-peer-review-bmc-group-in-progress/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/04/BMC-Logo-Part-of-SN_02_NS_RGB.png,"Open Peer Review, BMC Group",Feedback to authors|Validation of soundness,Bias in review|Incentives and recognition for reviewing|Quality of review|Transparency,Journal accepted manuscripts|Other scholarly outputs,Free-form commenting|Structured review form,Open identities|Open reports,Pre-publication review|Professional editors|Reviewer or editor initiates review,,"Editorial decision outcomes|Author opt-in rate or submission rate>Reviewer invitation acceptance rate|Author opt-in rate or submission rate>Reviewer recommendation outcomes – ie accept, accept with revisions, reject, etc (if applicable)",,Project,https://www.biomedcentral.com/about/advancing-peer-review,,,Active,3/1/2001,,"BMC Group, Springer Nature","Peer review is central to the publishing process and has a fundamental role to play in maintaining the integrity of the published literature and advancing discovery. At BMC, we have always supported innovation in peer review. The intention with open peer review is to make the reviewer (and the Editor) more accountable for peer review, and to provide recognition of the work that peer reviewers do. There have been relatively few large-scale studies looking at the impact of open peer review, and so data from the BMC Group is useful for learning about the effectiveness and benefits of open peer review.",,,,,,,NaN,,"10,000+",,"A study in BMJ Open in 2015 found that the quality of peer review reports was slightly higher in BMC Infectious Diseases (open peer review) compared with BMC Microbiology (single-blind peer review). However, no effect was found in the Journal of Inflammation when it operated open vs single-blind review. This study also showed that reviewer recommendations were similar in BMC Infectious Diseases and BMC Microbiology, suggesting no difference between open and single blind peer review.  However, in the Journal of Inflammation, it was found that reviewers were more likely to recommend acceptance under open peer review compared with single blind.",https://bmjopen.bmj.com/content/5/9/e008707,,NaN,,Verified,publish 1325,"Transparent peer review, Nature Communications",5/30/2019,job_listing,https://reimaginereview.asapbio.org/listing/transparent-peer-review-nature-communications/,https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_logo/2019/05/Twitter_400px.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/job-manager-uploads/job_gallery/2019/05/ncomms13626-f1.jpg,"Transparent peer review, Nature Communications|Transparent peer review, Nature Communications",Curation of interesting work|Feedback to authors|Validation of soundness,Bias in review|Incentives and recognition for reviewing|Quality of review|Reviewer training|Transparency,Journal accepted manuscripts,Free-form commenting,Double blind|Open reports|Single blind,Pre-publication review|Professional editors,All disciplines,,,Project,https://www.nature.com/articles/ncomms13626,,,Active,1/1/2016,,Nature Communications,"Our intention is to open up the peer review process at the journal, and to provide information on the expert reports that led to our editorial decision to publish a paper. Moreover, the reviewer reports also can serve as a background on the merits of a study, and inform about the discussion between authors and our reviewers.",,Yes,"Reviewers choose to sign reports; if report is not signed, confidentiality of reviewer is maintained.",Professional Editors invite reviewers to evaluate manuscript.,No,,0,,"10,000+",Rate at which authors are opting in to the publication of the peer review files; author opt-in rate across different research areas.,"On average, around 60% of our authors are voluntarily opting in to publishing the peer review history of their paper. We believe that this is a very encouraging result, given also the diversity of our content across so many research areas. Looking at the data for different research fields, not surprisingly, there is some variation. In areas where more open peer review is commonplace and reviewers often sign their reports by name, author opt-in is particularly strong. Other areas may not necessarily be that familiar with publishing reviewer reports, and might therefore still be hesitant to do so. However, in almost all research areas the majority of papers that we publish will now contain a peer review file.",https://www.nature.com/articles/ncomms13626,,NaN,,Verified,publish 1527,Journal of Open Source Software: Bot-assisted Open Peer Review,10/24/2019,job_listing,https://reimaginereview.asapbio.org/listing/journal-of-open-source-software-bot-assisted-open-peer-review/,https://reimaginereview.asapbio.org/wp-content/uploads/listing-uploads/logo/2019/10/logo.jpg|https://reimaginereview.asapbio.org/wp-content/uploads/listing-uploads/gallery/2019/10/unnamed.png,logo.jpg|unnamed.png,Feedback to authors,Costs|Quality of review|Reviewer training|Speed|Transparency,Journal accepted manuscripts,Free-form commenting|Structured review form,Open identities|Open interaction|Open reports,Author initiates review|Automation|Pre-publication review,,,,Project,https://joss.theoj.org,,https://github.com/openjournals/joss,Active,5/5/2016,,Open Journals,The primary goal of JOSS is to provide a mechanism for receiving career credit as an author of open source research software through citations to JOSS papers. A secondary (but very important) motivation is to improve the quality of open source research software available.,Yes,Yes,,,,"JOSS publishes articles about research software. This definition includes software that: solves complex modeling problems in a scientific context (physics, mathematics, biology, medicine, social science, neuroscience, engineering); supports the functioning of research instruments or the execution of research experiments; extracts knowledge from large data sets; offers a mathematical library, or similar. In addition, we have the following criteria for 'in-scope' submissions: ",0,,"100-1,000",No.,In the past three years we have published close to 750 papers.,,,NaN,,Verified,publish 1675,Review Commons,11/28/2019,job_listing,https://reimaginereview.asapbio.org/listing/review-commons/,https://reimaginereview.asapbio.org/wp-content/uploads/listing-uploads/logo/2019/11/Review-Commons-Logo_2-1.png|https://reimaginereview.asapbio.org/wp-content/uploads/listing-uploads/gallery/2019/11/RC-blue.png|https://reimaginereview.asapbio.org/wp-content/uploads/listing-uploads/gallery/2019/11/RC-screengrab.png,Review-Commons-Logo_2-1.png|RC-blue.png|RC-screengrab.png,Feedback to authors|Validation of soundness,Quality of review|Speed|Transparency,Preprints,Structured review form,Open reports,Author initiates review|Journal integration|Pre-publication review|Professional editors,Life Sciences,,,Trial,https://reviewcommons.org,,,Active,12/9/2019,,EMBO and ASAPbio,"Review Commons will: ",,No,"For more details about the process, please see the FAQ.","An independent Managing Editor will liaise with the authors and assign experienced professional editors at EMBO Press to run a high-quality, in-depth peer-review process. The EMBO editors will only invite referees, but not assess the manuscript for their journals. Reviewers may co-review manuscripts with colleagues, eg members of their research groups.",,Review Commons will only review well-developed scholarly research papers that represent a clear advance for the field. A managing editor in conjunction with an academic editorial board makes this decision independently of the selection criteria of affiliate journals.,0,"During the one-year trial, the project is supported by a grant to ASAPbio from The Leona M. and Harry B. Helmsley Charitable Trust, and there is no cost to authors or journals. Assuming that we find that the platform improves the process of publication, we will also develop a sustainable financial plan so that the service can continue beyond this initial period.",,"Our goal is to increase transparency, speed, and efficiency of the review process. We will evaluate the engagement of the community by measuring the submission rate. To assess the efficiency of the process, we will measure the time from submission to publication in a journal, and, very importantly, the average number of reviewers needed per published manuscript. To measure transparency, we will monitor the fraction of authors who post their reviews and replies to bioRxiv.",,,,NaN,,Verified,publish 1688,o'Peer,11/28/2019,job_listing,https://reimaginereview.asapbio.org/listing/opeer/,https://reimaginereview.asapbio.org/wp-content/uploads/listing-uploads/logo/2019/11/oPeer-logo-1.gif,oPeer-logo-1.gif,Feedback to authors|Validation of soundness,Bias in review|Costs|Incentives and recognition for reviewing|Quality of review|Speed|Transparency,Other scholarly outputs|Preprints,Annotations|Free-form commenting|Quantitative scores|Structured review form,Double blind|Open identities|Open interaction|Open participation|Open reports|Single blind,Author initiates review|Automation|Comment indexing|Comment rating (meta-evaluation)|Post-publication review,All disciplines|Physics,,,Project,http://opeer.org,,,Active,1/8/2014,,Jess H. Brewer,"The original idea (insofar as it IS original) is depicted as a presentaion at http://opeer.org/oPeer-foo.pdf The possible extension of the concept to broader societal impact is discussed at http://jick.ca/?cat=3 and at https://medium.com/predict/trust-2c9841f33994",Yes,,"The entire process is managed cybernetically as a database of submissions, reviews and ""credibility indices"" or reviewers.  All authors and referees are free to either identify themselves publicly or remain anonymous to readers, but the computer knows who they are.  This is necessary in order to accumulate a ""credibility index"" for every author/reviewer based on others' evaluations of their submissions, reviews and comments.  Algorithms will have to be refined and protected constantly to discourage ""gaming"" of the system.","Anyone registered with the system can submit, review or comment. Their reviews and comments will carry ""weight"" based on their own accumulated credibility.",No,"All submissions and reviews will be open access on the Internet.  Rather than ""tags and badges"", each registered participant will, over time, accumulate detailed ""credibility indices"" based on others' evaluations of their submissions, reviews and comments.",0,"Operation of this database and its algorithms on a societalo scale will be expensive, but it should be free to all users. This will obviously require governmental support.",10-100,,"So far almost no one has used my ""toy"" demo site -- luckily for me, as I'm running it on my own server for free!  It's meant to merely illustrate how this might be done on a grand scale if the idea catches on.  With luck, someone will ""steal"" my rudimentary implementation and turn it into a game-changing service.  Whether they make money from that or get the government to fund it is of no concern to me.",http://opeer.org,The database on the website itself.,NaN,,Unverified,publish 2150,The Winnower,7/19/2020,job_listing,https://reimaginereview.asapbio.org/listing/the-winnowerin-progress/,https://reimaginereview.asapbio.org/wp-content/uploads/2020/07/Screen-Shot-2020-07-19-at-12.12.09-PM.png|https://reimaginereview.asapbio.org/wp-content/uploads/2020/07/Screen-Shot-2020-07-19-at-12.14.53-PM.png|https://reimaginereview.asapbio.org/wp-content/uploads/2020/07/Screen-Shot-2020-07-19-at-12.18.03-PM.png|https://reimaginereview.asapbio.org/wp-content/uploads/2020/07/Screen-Shot-2020-07-19-at-12.20.29-PM.png|https://reimaginereview.asapbio.org/wp-content/uploads/2020/07/Screen-Shot-2020-07-19-at-12.31.15-PM.png,Screen Shot 2020-07-19 at 12.12.09 PM|Screen Shot 2020-07-19 at 12.14.53 PM|Screen Shot 2020-07-19 at 12.18.03 PM|Screen Shot 2020-07-19 at 12.20.29 PM|Screen Shot 2020-07-19 at 12.31.15 PM,Curation of interesting work|Feedback to authors|Validation of soundness,Costs|Incentives and recognition for reviewing|Quality of review|Reviewer training|Speed|Transparency,Journal accepted manuscripts|Other scholarly outputs|Preprints,Free-form commenting|Quantitative scores,Open identities|Open interaction|Open participation|Open reports,Author initiates review|Comment moderation|Comment rating (meta-evaluation)|Post-publication review|Reviewer or editor initiates review,All disciplines,,,Project,https://thewinnower.com/,,,Active,5/27/2014,,,The Winnower looks to turn the current system upside down with a platform that will foster transparency and communication by using open post-publication peer review for every paper we publish.,No,Yes,"With The Winnower, the scientific publishing process has become a ""do it yourself"" (DIY) endeavor. Authors upload their finished work to the website, where it is then immediately available to be read and reviewed by the Winnower community. Once reviewed, authors will have the ability to revise their works before a final paper is posted and given a digital object identifier (DOI). Additionally, reviews will be open for the duration of the paper's life, while article-level metrics and the reviews themselves will track the importance and accuracy of the paper. Content published with The Winnower will be licensed under a CC BY 4.0 license.",All who register,Yes,"Publish your research, class essays, peer reviews, grants, letters, how-to articles, conference summaries, conference talks, blogs - whatever format you choose.",25,https://thewinnower.com/memberships,"1,000-10,000",,,,,NaN,,Unverified,publish 2157,PeerLibrary,7/4/2020,job_listing,https://reimaginereview.asapbio.org/listing/peerlibrary/,https://reimaginereview.asapbio.org/wp-content/uploads/listing-uploads/logo/2020/07/tumblr_inline_nmdrpfpWB11to7fi6_400.png|https://reimaginereview.asapbio.org/wp-content/uploads/2020/07/Screen-Shot-2020-07-05-at-3.30.02-PM.png,tumblr_inline_nmdrpfpWB11to7fi6_400.png|Screen Shot 2020-07-05 at 3.30.02 PM,Curation of interesting work|Feedback to authors|Validation of soundness,Bias in review|Quality of review|Speed|Transparency,Journal accepted manuscripts|Other scholarly outputs|Preprints|Privately shared manuscripts,Annotations,Open identities|Open interaction|Open participation|Open reports,Author initiates review|Comment indexing|Post-publication review|Reviewer or editor initiates review,,,,Project,https://peerlibrary.org/,,https://github.com/peerlibrary/peerlibrary,Active,7/1/2014,,,Provide independent open source platform utilizing modern technologies to focus on collaboration around reading of academic paper. High quality implementation and great user experience.,,Yes,,,,,NaN,,"1,000-10,000",,,,,NaN,,Verified,publish