- \n
- OASPA announcement<\/a><\/li>\n
- Project website<\/a><\/li>\n
- Digital Science Blog Post: \u201cDoes creating a rapid reviewer pool improve trust in COVID-19 peer review?\u201d<\/a><\/li>\n
- Digital Science Blog Post: \u201cCreating a rapid reviewer pool\u2026..and trying to use it\u201d<\/a><\/li>\n
- Poster (Zenodo)<\/a><\/li>\n<\/ul>\n
Current stage of development<\/h4>\n
Ready to be implemented<\/p>\n
Project duration<\/h4>\n
6 months<\/p>\n
\n\nUpdate<\/h2>\n
How has your project changed?<\/h3>\n
Since the kickoff, we designed and began disseminating a survey<\/a> among Editors or individuals in related roles associated with publishers part of the \u201cC19 Rapid Review group (C19 RR)<\/a>\u201c. The goal is to better understand editorial needs and inform next steps in designing and implementing a seamless workflow integration between community reviews\u2014preprint reviews authored by researchers in a journal-independent fashion\u2014of COVID-19 preprints published on Outbreak Science Rapid PREreview (OSr<\/em>PRE)<\/a> and journal-organized peer review. <\/p>\n
The responses we collected so far highlight that for Editors to trust community reviews they would need to know the identity of the reviewers, particularly their names, affiliation, expertise, together with an explicit declaration of no existing competing interests. Currently on the OSr<\/em>PRE platform, reviewers can choose to use a pseudonym to sign their reviews and, while they are not directly asked in each review to explicitly declare competing interests, they are invited to do so when they agree to our Code of Conduct<\/a>. <\/p>\n
Furthermore, we learned that Editors would like to have a pipeline that would automatically inform them if there are preprint reviews available for a particular paper they are handling, with links to those reviews, as well as an easy way to contact the authors of those reviews. <\/p>\n
While we are waiting to hear more from Editors, we have already begun designing solutions that would allow for Editors to trust and access the reviews and reviewers, thereby helping them triage COVID-19 manuscripts submitted to their journals.<\/p>\n
Have you integrated any feedback received?<\/h3>\n
During the breakout sessions, Sprint participants were interested in better understanding our plan around API integration, and suggested the utility of a way for us to integrate with preprint servers to enable author-driven requests of community feedback. This workflow is very important to us as it is key to close the loop between authors and community reviewers, and also a way to signal editors that authors welcome the feedback. Since the kickoff, we have taken steps towards outlining key API elements in our new upcoming platform that can allow for easy integration across many different preprint servers. <\/p>\n
Additionally, participants highlighted the importance of community building efforts to train volunteer community reviewers in constructive peer review to foster engagement and raise interest. Specific suggestions included connecting with research communities that are already active in the space (e.g<\/em>., NewPI slack, FuturePI slack), and leverage early-career researchers and journal clubs to enable collaborative reviews. PREreview has already been working on tools and resources to facilitate these connections and behaviors, and we will make sure to engage the community of volunteer reviewers in these activities. <\/p>\n
Have you started any collaborations?<\/h3>\n
The ASAPbio Sprint was a great opportunity for us to learn more about existing efforts and find areas of overlapping interest with other projects. <\/p>\n
To ensure the development of our API follows standards that will maximize the impact of our work across different third party platforms, our team team connected with Kathleen Shearer, Executive Director of COAR, and established a collaboration through which our work will serve as a testing bed for COAR\u2019s standard metadata model.<\/p>\n<\/div>\n<\/div>\n
Project aims<\/h2>\n
Background information on current practices<\/h3>\n
In May 2020 a group of STM publishing organisations made a direct call to the whole community to maximize the efficiency of peer review of COVID-19 research ( \u201cCOVID-19 Publishers\u2019 Open Letter of Intent \u2013 Rapid Review<\/em><\/a>\u201d). The community responded quickly, with over 1700 researchers signed up to a rapid reviewer pool (volunteer rapid reviewers). All are committed to quickly reviewing COVID-19 manuscripts, to have their reports shared between publishers if manuscripts are transferred and for them to be openly published. <\/p>\n
The group of organisations involved is now greater than 20 and wishes to focus on engaging the volunteer rapid reviewers in the review of COVID-19 preprints. Outbreak Science Rapid PREreview<\/a> (OSr<\/em>PRE), an independent preprint review platform, has been a member of the collaboration since May. To extend the aims of the group, OSr<\/em>PRE has prototyped a COVID-19 preprint review dashboard.<\/a> The dashboard allows readers to filter recommended reviews, show preprints ready for journal-organized peer review, and highlights those that have data and code (screenshot below). The group wishes to support this further and connect the rapid reviews with journal Editors to decrease assessment time and help triage incoming manuscripts.<\/p>\n
![\"\"](\"https:\/\/asapbio.org\/wp-content\/uploads\/2025\/03\/cc19-2-1024x509.png\")
<\/figure>\nChallenges<\/h3>\n
- \n
- Train and support engagement of the volunteer rapid reviewers with preprints<\/li>\n
- Maximise impact of preprints with Editors by: a) highlightCOVID-19 preprints that have reviews; b) identify COVID-19 preprints recommended for peer review; c) identify preprints that have open data\/code; d) identify COVID-19 preprint reviewers directly to Editors.<\/li>\n<\/ul>\n
The ideal outcome or output of the project<\/h3>\n
- \n
- A community engagement strategy to train, manage, and support the volunteer rapid reviewers group<\/li>\n
- Seamless workflow connecting reviews of COVID-19 preprints with Editors<\/li>\n
- Surveys for editorial teams and volunteer rapid reviewers<\/li>\n<\/ul>\n
Description of the intervention<\/h3>\n
An engagement strategy <\/strong>includes:<\/p>\n
- \n
- Inviting volunteer rapid reviewers to review COVID-19 preprints on OSr<\/em>PRE <\/li>\n
- Organizing live-streamed preprint journal clubs and webinars to train the volunteer rapid reviewers <\/li>\n
- Offering badges for volunteer rapid reviewers to be recognized as active contributors<\/li>\n<\/ul>\n
Workflow integration<\/strong> <\/p>\n
Easy interventions are needed to connect Editors and content\/reviewers including:<\/p>\n
- \n
- A demo guide and 1:1 support for Editors in how to use the current COVID-19 dashboard<\/li>\n
- Weekly summary reports and ongoing support<\/li>\n<\/ul>\n
An API integration between OSr<\/em>PRE and the journal submission software would:<\/p>\n
- \n
- Allow authors and\/or editors to send request calls for rapid reviews of submitted COVID-19 manuscripts<\/li>\n
- Allow authors and\/or editors to be notified when new\/requested rapid reviews are published<\/li>\n
- Allow authors to recognize if an OSr<\/em>PRE user is a volunteer rapid reviewer and connect with them<\/li>\n<\/ul>\n
Research<\/strong>: <\/p>\n
We will design the following surveys:<\/p>\n
- \n
- Editorial teams in the collaboration to understand their needs<\/li>\n
- Volunteer rapid reviewers active on OSrPRE<\/li>\n<\/ul>\n
Metrics<\/strong>:<\/p>\n
To measure the impact of our efforts we will:<\/p>\n
- \n
- Track engagement on the dashboard<\/li>\n
- Count the number of rapid reviews of COVID-19 preprints <\/li>\n
- Count the number of journal editors who report to regularly use the dashboard<\/li>\n
- Count the number of rapid review requests from journal editors\/manuscript authors<\/li>\n
- Count the number of volunteer rapid reviewers who are active on OSr<\/em>PRE <\/li>\n
- Measure the extent and impact of our engagement strategy via opt-in surveys <\/li>\n<\/ul>\n
What\u2019s needed for success<\/h3>\n
Additional technology development<\/h4>\n
API integrations \u2014 OSr<\/em>PRE has already a basic API that would need to be customized for the purposes described above.<\/p>\n
Feedback, beta testing, collaboration, endorsement<\/h4>\n
We welcome feedback, beta testing on the dashboard and integration, collaboration, endorsement.<\/p>\n
Funding<\/h4>\n
Some of the more technologically-heavy interventions described above are contingent on funding (up to $50,000 to cover for a part-time project manager and development).<\/p>\n","protected":false},"excerpt":{"rendered":"
Organizers Daniela Saderi (PREreview Co-Founder and Director) and Sarah Greaves (STM Publishing Consultant) Website or social media links OASPA announcement Project website Digital Science Blog Post: \u201cDoes creating a rapid […]<\/p>\n","protected":false},"author":2,"featured_media":2065,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[48],"tags":[],"class_list":["post-3386","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-preprint-sprint-proposals"],"acf":[],"_links":{"self":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts\/3386","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/comments?post=3386"}],"version-history":[{"count":1,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts\/3386\/revisions"}],"predecessor-version":[{"id":3387,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts\/3386\/revisions\/3387"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/media\/2065"}],"wp:attachment":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/media?parent=3386"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/categories?post=3386"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/tags?post=3386"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- Measure the extent and impact of our engagement strategy via opt-in surveys <\/li>\n<\/ul>\n
- Inviting volunteer rapid reviewers to review COVID-19 preprints on OSr<\/em>PRE <\/li>\n
- Project website<\/a><\/li>\n