{"id":3397,"date":"2020-11-02T00:00:00","date_gmt":"2020-11-02T00:00:00","guid":{"rendered":"http:\/\/pl-asapbio.local\/putting-peers-at-the-heart-of-peer-review\/"},"modified":"2025-03-28T21:36:53","modified_gmt":"2025-03-28T21:36:53","slug":"putting-peers-at-the-heart-of-peer-review","status":"publish","type":"post","link":"https:\/\/asapbio.org\/putting-peers-at-the-heart-of-peer-review\/","title":{"rendered":"Putting peers at the heart of peer review"},"content":{"rendered":"<h2 class=\"wp-block-heading\" id=\"block-142df3fd-7c2a-4e47-a4b7-a9b99f94ec74\">Organizer<\/h2>\n<p id=\"block-97a13903-aeab-4407-969a-7d261d018e16\">Life Science Editors (Helen Pickersgill, Angela Andersen, Marie Bao, Carol Featherstone, Shawna Hiley, Sabbi Lall, Li-Kuo Su)<\/p>\n<h4 class=\"wp-block-heading\" id=\"block-e94c12a1-ee82-409b-8158-14207fc0dc88\">Website or social media links<\/h4>\n<p><a rel=\"noreferrer noopener\" target=\"_blank\" href=\"http:\/\/www.lifescienceeditors.com\/\">www.lifescienceeditors.com<\/a><\/p>\n<h4 class=\"wp-block-heading\" id=\"block-7c45ccf9-b544-41da-870a-6baf9c08b8c9\">Current stage of development<\/h4>\n<p id=\"block-3a3c24fc-54af-4274-808b-bcbf23b7961e\">&nbsp;Idea<\/p>\n<h4 class=\"wp-block-heading\" id=\"block-faf68aec-8f33-4b83-ba7a-e666cdfdba3a\">Project duration<\/h4>\n<p id=\"block-f132a9fe-e868-40fe-8939-e692f120471f\">Indefinitely<\/p>\n<div class=\"wp-block-group has-background\" style=\"background-color:#f5e8e8\">\n<div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<h2 class=\"wp-block-heading\">Update<\/h2>\n<h4 class=\"wp-block-heading\">How has your project changed? <\/h4>\n<p>We introduced an <span style=\"text-decoration: underline;\">additional option<\/span> to incentivize authors to initiate peer review. As an alternative to requesting traditional formal evaluation of their preprint that could potentially be used by a journal, authors can request <span style=\"text-decoration: underline;\">informal guidance<\/span> on how to further develop and improve the impact of their study before journal submission, to support a key reason that authors post preprints. We expect that this opportunity will encourage both preprint deposition and peer review as it <span style=\"text-decoration: underline;\">provides a novel service<\/span> not formally available elsewhere. <\/p>\n<p>We all know that low quality peer review has damaging consequences on science and scientists. High quality peer review requires a panel of experts with relevant experience covering both technical and conceptual aspects of the paper, as well as oversight to ensure thoroughness and professionalism of each review. For submissions to journals, an editor selects appropriate reviewers and oversees the quality of the reviews. Our proposal creates this valuable peer review process for preprints by having an author-selected Liaison Team assume these responsibilities. To ensure accountability and high quality, the Liaison Team is named, and we added that they <span style=\"text-decoration: underline;\">meet key criteria<\/span> (see below). Ultimately, ensuring <span style=\"text-decoration: underline;\">high quality peer review<\/span> will effectively curate preprints by segregating strong preprints from those that do not garner sufficient interest for review. Instead of providing an overall rating of the preprint, the Liaison Team will write a non- quantitative Impact Summary , given that a single rating cannot capture all aspects. The time taken to complete reviews will be monitored to ensure a timely process and identify reliable reviewers .<\/p>\n<p>Instead of providing an overall rating of the preprint, the Liaison Team will write a non- quantitative <span style=\"text-decoration: underline;\">Impact Summary<\/span>, given that a single rating cannot capture all aspects. <\/p>\n<p>The time taken to complete reviews will be monitored to <span style=\"text-decoration: underline;\">ensure a timely process and identify reliable reviewers<\/span> .<\/p>\n<h4 class=\"wp-block-heading\">Have you integrated any feedback received?<\/h4>\n<ol class=\"wp-block-list\">\n<li> To <span style=\"text-decoration: underline;\">prevent gaming of the system<\/span> we will specify criteria to avoid real and perceived conflicts of interest in the selection of Liaison Teams by authors, as is standard at scientific journals. In addition:\n<ul>\n<li>The Liaison Team comprises multiple members, which should minimise individual biases. <\/li>\n<li>The Liaison Team is identified, which discourages authors from nominating close friends, and gives the team a strong incentive to do a good job \u2013 their reputation is at stake. <\/li>\n<li>Readers can rate the preprints, summaries and reviews. Disconnects could indicate gaming. Liaison Teams consistently rating papers higher than readers could be identified and excluded. <\/li>\n<li>The stakes of the preprint review process are lower \u2013 it\u2019s not an accept or reject decision \u2013 so there is less incentive to game the system. <\/li>\n<li>The scientific peer review system already works on trust. <\/li>\n<\/ul>\n<\/li>\n<li>To <span style=\"text-decoration: underline;\">manage power dynamics<\/span>, the process <span style=\"text-decoration: underline;\">formalizes the contribution<\/span> of graduate students and postdocs by having <span style=\"text-decoration: underline;\">authorship<\/span> on the Liaison Team (and reviews, if they choose to reveal their identity). <\/li>\n<li>To <span style=\"text-decoration: underline;\">minimize implicit bias<\/span>, we will provide <span style=\"text-decoration: underline;\">educational materials<\/span> and a <span style=\"text-decoration: underline;\">structured review template<\/span>.<\/li>\n<\/ol>\n<h4 class=\"wp-block-heading\">Have you started any collaborations?<\/h4>\n<p>Our proposal is a collaboration between LSE and <a href=\"https:\/\/asapbio.org\/peer-review-pipeline-for-preprint-servers\" data-type=\"URL\" data-id=\"https:\/\/asapbio.org\/peer-review-pipeline-for-preprint-servers\">Dr. Joseph Wade<\/a>, and we are discussing the proposal with biorxiv\/medrxiv. Endorsement and collaboration with the NIH, NCBI and Open Access journals would increase value.<\/p>\n<\/div>\n<\/div>\n<h2 class=\"wp-block-heading\" id=\"block-a0c5359e-b8ea-4ae5-acf2-d869d3477452\">Project aims<\/h2>\n<h3 class=\"wp-block-heading\" id=\"block-b16f8d89-68f8-473c-ba51-11ad593ed203\">Overview of the challenge to overcome<\/h3>\n<p id=\"block-5725c18c-c243-489a-b1a7-ee76332222da\">Peer review is central to the scientific process, but is managed primarily by journals behind closed doors, limiting its value to the community. Reviews of rejected papers, and of most published papers, are rarely posted. Peer review of preprints side-steps journal constraints and democratizes the evaluation of science by scientists. Peer review would improve preprint quality and curation but, without incentives and standards, the number and quality of reviews varies widely. Additionally, refereed preprints and their reviews are difficult to find. We propose to integrate peer review directly into preprint servers and hand control of this process back to authors and their peers.<\/p>\n<h3 class=\"wp-block-heading\" id=\"block-df0c1c6c-77c4-4d4f-bc0b-7bff15fa5084\">The ideal outcome or output of the project<\/h3>\n<p id=\"block-9fa74fd6-0c6c-4b8c-97f8-8854e4fa8318\">A system to generate consistent, valuable, discoverable, and open peer reviews. Most preprints would have reviews.&nbsp;<\/p>\n<h3 class=\"wp-block-heading\" id=\"block-85d7b965-e58f-4696-a081-7e62160c280b\">Description of the intervention<\/h3>\n<p>&nbsp;The author submits a preprint and can nominate a \u201cLiaison team\u201d: a lab\/team headed by a PI to handle the peer review process. The team approach promotes training and diversity, and reduces biases. The server automatically sends an invitation: if it is accepted the process moves forward; if not, the author can suggest an alternative team or cancel the process.<br \/>\u2022 The Liaison team receives information on managing constructive peer review and invites multiple individuals\/labs to review the preprint. The author can suggest reviewers.<br \/>\u2022 Reviewers have a unique identifier and can remain anonymous. Reviewers are searchable by reviews, scientific expertise, and peer review experience with grants, journals, and\/or preprints.<br \/>\u2022 Reviewers are invited to watch videos on how to write a review (from the NIH) and on implicit bias. They receive a structured template to facilitate review, reduce bias, and improve consistency and searchability. Reviewers complete their review within two weeks and rate the preprint (1\u20135 stars).<br \/>\u2022 The Liaison team summarizes the reviews, gives an overall rating, links their summary and the reviews to the preprint. The reviews and summary are citable.<br \/>\u2022 Authors can rate the Liaison and the Reviews and respond.<br \/>\u2022 Refereed preprints are highlighted on the server and can be searched and sorted.<br \/>\u2022 Readers can rate the reviews (1-5 stars) and the preprint (1-5 stars; this rating is separate from the reviewer rating).&nbsp;<br \/>\u2022 Invitations from peers, citable outputs, and ratings create incentives.<\/p>\n<figure class=\"wp-block-image size-large\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"800\" src=\"https:\/\/asapbio.org\/wp-content\/uploads\/2025\/03\/LifeScienceEditors_peer-2-1024x800.png\" alt=\"\" class=\"wp-image-5167\" srcset=\"https:\/\/asapbio.org\/wp-content\/uploads\/2025\/03\/LifeScienceEditors_peer-2-1024x800.png 1024w, https:\/\/asapbio.org\/wp-content\/uploads\/2025\/03\/LifeScienceEditors_peer-2-300x234.png 300w, https:\/\/asapbio.org\/wp-content\/uploads\/2025\/03\/LifeScienceEditors_peer-2-768x600.png 768w, https:\/\/asapbio.org\/wp-content\/uploads\/2025\/03\/LifeScienceEditors_peer-2.png 1360w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/figure>\n<h3 class=\"wp-block-heading\" id=\"block-44726407-0bf6-4c6d-a7d6-200b1dd32600\">Plan for monitoring project outcome<\/h3>\n<p>At baseline, 3 months, 12 months assess the number of: preprints with reviews, reviews with ratings, responses from authors, downloads and citations of preprints.&nbsp;<\/p>\n<h3 class=\"wp-block-heading\" id=\"block-00e094ab-ddce-462e-81a8-9630109a1c42\">What\u2019s needed for success<\/h3>\n<h4 class=\"wp-block-heading\" id=\"block-583dd79d-bfbd-4082-bb25-1db33b869eba\">Additional technology development<\/h4>\n<p>Technology to monitor and sort ratings, post reviews and responses, ensure reviewer anonymity.<\/p>\n<h4 class=\"wp-block-heading\" id=\"block-75b3c918-7a23-4762-93db-c2628201cb4b\">Feedback, beta testing, collaboration, endorsement<\/h4>\n<p>Survey authors, reviewers, readers, journal editors, preprint servers.<br \/>Collaborate with preprint servers, NIH.<\/p>\n<h4 class=\"wp-block-heading\" id=\"block-1055ba2c-1030-45ef-a4f6-528849ed4119\">Funding<\/h4>\n<p>Funding to develop training materials, algorithms, functionalities of preprint severs.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Organizer Life Science Editors (Helen Pickersgill, Angela Andersen, Marie Bao, Carol Featherstone, Shawna Hiley, Sabbi Lall, Li-Kuo Su) Website or social media links www.lifescienceeditors.com Current stage of development &nbsp;Idea Project [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":2086,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[48],"tags":[],"class_list":["post-3397","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-preprint-sprint-proposals"],"acf":[],"_links":{"self":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts\/3397","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/comments?post=3397"}],"version-history":[{"count":1,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts\/3397\/revisions"}],"predecessor-version":[{"id":3398,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts\/3397\/revisions\/3398"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/media\/2086"}],"wp:attachment":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/media?parent=3397"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/categories?post=3397"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/tags?post=3397"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}