{"id":3651,"date":"2018-01-19T00:00:00","date_gmt":"2018-01-19T00:00:00","guid":{"rendered":"http:\/\/pl-asapbio.local\/peer-feedback\/"},"modified":"2025-03-28T21:38:15","modified_gmt":"2025-03-28T21:38:15","slug":"peer-feedback","status":"publish","type":"post","link":"https:\/\/asapbio.org\/peer-feedback\/","title":{"rendered":"Peer Feedback"},"content":{"rendered":"

By Ron Vale, Tony Hyman, and Jessica Polka<\/span><\/em><\/p>\n

Summary<\/i><\/b><\/p>\n

We propose the creation of a scientist-driven, journal-agnostic peer review service that produces an \u201cEvaluated Preprint\u201d and facilitates subsequent publication in a journal.<\/span><\/p>\n

\"\"<\/a><\/p>\n

<\/span><\/p>\n

Introduction<\/i><\/b><\/p>\n

Scientists have a love-hate relationship with peer review. Sadly, this relationship has been drifting towards the latter over time. Much of the problem with peer review, as it currently stands, is its dual function in: 1) technical evaluation of data and improving the quality, interpretation, and presentation of a scientific work, and 2) impact evaluation, helping editors assess where the papers stands in the field and its suitability for the journal. We feel that recently peer review has become consumed with this latter journal \u201cgatekeeping\u201d function. As a symptom of this trend, it is not uncommon for authors to receive short reviews proclaiming that the study is \u201cmore suitable for specialized journal\u201d or requesting a list of experiments to \u201craise its impact.\u201d<\/span><\/p>\n

The time has come to disentangle the two roles of peer review. <\/span>Peer Feedback<\/span><\/i> is a proposed scientist-driven service that tackles the first issue by providing <\/span>high quality reviews that help scientists to improve their work and expedite subsequent publication<\/span><\/i>.<\/span><\/p>\n

How would Peer Feedback work?<\/i><\/b><\/p>\n

In brief, <\/span>Peer Feedback<\/span><\/i> could work as follows, although we look to the scientific community now for input on its merit and how best to refine its operation.<\/span><\/p>\n

    \n
  1. Scientists submit their work to a preprint server or <\/span>Peer Feedback<\/span><\/i> and request evaluation. The authors suggest a list of possible referees from an extensive <\/span>Peer Feedback<\/span><\/i> Review Board, or outside referees if their discipline is not well-represented on the Board. Upon receipt, submissions are subjected to an initial screening process that checks for adherence to relevant publishing guidelines such as plagiarism. <\/span>Peer Feedback<\/span><\/i> will select and contact referees, offering them the opportunity to review the paper. When two of them agree to review, they are given two weeks to complete their reports using a template that focuses on constructive technical evaluation After this period, reviewers then have another week to consult with one another online, adopting a general strategy implemented by eLife and EMBO J. Importantly, reviewers are encouraged to engage junior colleagues, such as postdocs, in review, recognizing the respective strengths that senior and junior reviewers bring to evaluation. All reviewers will receive credit for their contribution in a database, which can be used as evidence of public service. A template for peer review will be created to guide the evaluation, and direct manuscript annotation also will be explored.<\/span><\/li>\n
  2. After constructive reviews are obtained, authors (or institutions or funders) pay for <\/span>Peer Feedback<\/span><\/i>. Reviewers are paid for services, recognizing their work in providing high quality review. However, in lieu of payment, referees are encouraged to receive credits for using the system as authors, or donate their credits to scientists from under-resourced institutions. Overall, this system creates a transparent fee structure and promotes timely performance by the referees. Initially, ASAPbio plans to cover some of the cost for authors in order to increase participation. The default will be an \u201copen pipeline\u201d (review reports and the revised manuscript are assigned DOIs and publicly disclosed, regardless of whether the reviewer signs them).<\/span><\/li>\n
  3. After <\/span>Peer Feedback<\/span><\/i>, the authors can respond to the referees and revise their work. We also will explore the possibility of online author-referee consultation during a defined revision period. The revised paper, the reviews, and author response will be linked together to form an \u201c<\/span>Evaluated Preprint,<\/span>\u201d which provides several choices for authors:<\/span>\n
      \n
    1. The Evaluated Preprint can be included as evidence of productivity and rigor in grants, job applications, and promotions, providing added value beyond a preprint. <\/span><\/li>\n
    2. The Evaluated Preprint could, in some cases, become a final product. The research community would have open access to the work and the reviews, potentially in partnership with a publication platform.<\/span><\/li>\n
    3. The Evaluated Preprint could be used to facilitate journal matchmaking and streamline publication. We envision a number of journals partnering with <\/span>Peer Feedback<\/span><\/i>. For authors who opt-in, a \u201cDashboard\u201d available to journal editors would allow participating journals to view the Evaluated Preprint, journal submission status, and reviewer identity, if not public. Through this Dashboard, partnering journals could invite an Evaluated Preprint for submission and indicate whether they will accept it in its revised form or request additional experiments or peer review. The former scenarios would ideally result in a lower publication cost to the author.<\/span><\/li>\n
    4. The author may decide to submit to another journal that is not part of the <\/span>Peer Feedback<\/span><\/i> journal network. Even in this case, the author presumably comes in with a stronger submission and a greater chance of acceptance. The journal editor also may use the Evaluated Preprint in order to better assess whether the paper is a good match for the journal.<\/span><\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

      We anticipate that the majority of papers that pass through <\/span>Peer Feedback<\/span><\/i> will not need to be reviewed again, or could be reviewed only for journal suitability. We also hope that the system will promote better journal matching with less rejection since editors will have better information on the submission.<\/span><\/p>\n

      We propose that reviewers will be selected through partnerships with several scientific societies. The Review Board will include members of the partnering societies who voluntarily agree to participate as potential reviewers. Reviewers will receive scores on their reviews (by author, other reviewer, and community). Incorporating such data, we will create a system of referee awards that can be cited as evidence of community service. Performance will be monitored and reviewers of poor standing (poor scores and\/or time to review) will be dismissed.<\/span><\/p>\n

      Who Benefits from Peer Feedback?<\/i><\/b><\/p>\n

      Science<\/span><\/i><\/p>\n