{"id":3324,"date":"2021-07-30T00:00:00","date_gmt":"2021-07-30T00:00:00","guid":{"rendered":"http:\/\/pl-asapbio.local\/developing-a-taxonomy-to-describe-preprint-review-processes\/"},"modified":"2025-03-28T21:31:26","modified_gmt":"2025-03-28T21:31:26","slug":"developing-a-taxonomy-to-describe-preprint-review-processes","status":"publish","type":"post","link":"https:\/\/asapbio.org\/developing-a-taxonomy-to-describe-preprint-review-processes\/","title":{"rendered":"Developing a taxonomy to describe preprint review processes"},"content":{"rendered":"

By Victoria Yan<\/em><\/p>\n

2022-01-27 update: This work has now been completed. Please see our blog post announcing PReF<\/a>.<\/p>\n

Why develop a preprint review taxonomy?<\/h2>\n

Dozens of projects organizing peer review of preprints are active or being developed. In this landscape full of new possibilities, differentiating among innovative forms of preprint review is challenging. Furthermore, these reviews are often hard to discover and not directly linked to the preprint. In order to tackle the challenges of preprint review terminology and enhancing preprint review discovery, the ASAPbio Taxonomy working group is working on creating a simple, reader-friendly system to classify preprint review processes.<\/p>\n

The working group, made up of 14 members, represents a diverse group of preprint servers representatives, publishers, and technologists. The next steps of the working group are to finalize the taxonomy language intelligible to a larger audience and to develop a simple visual display for readers.<\/p>\n

<\/div>\n

FeedbackASAP breakout session<\/h2>\n

We presented the draft preprint taxonomy at the ASAPbio #FeedbackASAP<\/a> meeting on July 21st, 2021. The draft taxonomy consists of 8 terms, with their associated help text description and options (see table below). First, we stress-tested the draft taxonomy for intelligibility by asking participants to apply the classification categories to Review Commons<\/a> (an ASAPbio-EMBO project that organizes journal-independent peer review). Through this exercise, we received suggestions that for the term \u201cReviewer selected by,\u201d the process by which reviewers are selected should also be captured. For instance, reviewers maybe are first nominated by authors prior to editor selection. As a group, we noted that how \u201cCompeting interest\u201d is handled at various preprint review projects is diverse and unclear. Participants noted that conflict of interest can be extremely important in a small number of cases. On the other hand, in most situations, colleagues or friends should not be precluded from providing feedback on preprints. However, it is important that it becomes a norm to declare any potential competing interest. <\/p>\n

<\/div>\n
\n\n\n\n\n\n\n\n\n\n\n\n
Name of Field<\/strong><\/td>\nHelp text<\/strong><\/td>\nOptions<\/strong><\/td>\n<\/tr>\n
Review requested by<\/td>\nWho submits or initiates the feedback process?<\/td>\nAuthors
Non-authors<\/td>\n<\/tr>\n
Reviewer selected by<\/td>\nWho selects the feedback providers?<\/td>\nAuthors
Editor, service, or community
Self-nominated<\/td>\n<\/tr>\n
Review coverage<\/td>\nDoes the feedback cover the entire paper or only a certain section or aspect?<\/td>\nFull paper
Specific aspects<\/td>\n<\/tr>\n
Reviewer identity known to<\/td>\nAre the identities of reviewers known to everyone (public), editors or service, or noone?<\/td>\nPublic
Editor or service
No one<\/td>\n<\/tr>\n
Competing interests<\/td>\nIs a declaration of competing interest required?<\/td>\nDisplayed 
Checked
Not included<\/td>\n<\/tr>\n
Public interaction<\/td>\nIs there an opportunity for the public to engage as an integral part of the process?<\/td>\nIncluded
Not included<\/td>\n<\/tr>\n
Opportunity for author response<\/td>\nIs the author\u2019s response included as an integral part of the process?<\/td>\nIncluded
Not included<\/td>\n<\/tr>\n
Recommendation<\/td>\nDoes the service provide a binary decision (accept\/reject or recommendation) or a scalar rating after the review process?<\/td>\nBinary decision
Other scale or rating
None<\/td>\n<\/tr>\n<\/tbody>\n<\/table>
The draft version of the preprint feedback taxonomy as of the breakout session on July 21, 2021<\/figcaption><\/figure>\n
<\/div>\n

Where can the taxonomy be useful? <\/h2>\n

In our next exercise, the group was asked to consider which terms displayed in which locations would be useful from the perspective of readers, authors, reviewers, and publishers. We found that for readers, most terms in the taxonomy are helpful when displayed adjacent to individual preprints and their linked reviews. On the other hand, authors and reviewers might find it helpful to have \u201cAuthor response\u201d, \u201cPublic interaction\u201d, \u201cReview coverage\u201d, and \u201cReview requested by\u201d displayed on descriptions of a given preprint review service. Specifically, \u201cReviewer identity known to\u201d is important for the reviewers. Lastly, from the perspective of publishers, \u201cReviewer identity known to\u201d was found to be most important on preprints and linked reviews. <\/p>\n

<\/div>\n

Which aspects are important in preprint review?<\/h2>\n

Finally, we asked the group to rank the terms from most important to least important. Some of the terms such as \u201cReview coverage\u201d and \u201cAuthor response,\u201d were unanimously considered important. Interestingly, the terms \u201cRecommendation\u201d and \u201cReview requested by\u201d were more polarized and found on both ends of the scale. The group noted that peer review of preprints can be used for much more than providing an editorial decision for the publisher. For example, it can be an early form of feedback aimed at improving the work, or it can provide additional context for the preprint. The target audience of review in the case of preprints is shifting away from editors and towards authors and readers. Therefore, less emphasis is placed on an editorial decision as the primary endpoint of review. <\/p>\n

<\/div>\n

Developing a simple and useful taxonomy<\/h2>\n

We had several additional suggestions for taxonomy, both new terms (for example, the public availability of the review and the process of reviewer selection) and additional levels of detail. As a group, we came to the conclusion that an essential aspect of the development of the preprint review taxonomy is maintaining focus on what problems can the taxonomy solve. For improving the clarity and discoverability of review processes, we will be mindful of the level of granularity that can be pragmatically implemented and develop a simple taxonomy that is well-suited for widespread adoption.  If you have additional feedback on the taxonomy, we welcome your input in the comments below!<\/p>\n

Photo by UX Indonesia<\/a> on Unsplash<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

By Victoria Yan 2022-01-27 update: This work has now been completed. Please see our blog post announcing PReF. Why develop a preprint review taxonomy? Dozens of projects organizing peer review […]<\/p>\n","protected":false},"author":2,"featured_media":1953,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[47,49,44],"tags":[],"class_list":["post-3324","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-feedbackasap","category-peer-review","category-preprints"],"acf":[],"_links":{"self":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts\/3324","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/comments?post=3324"}],"version-history":[{"count":1,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts\/3324\/revisions"}],"predecessor-version":[{"id":4421,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/posts\/3324\/revisions\/4421"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/media\/1953"}],"wp:attachment":[{"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/media?parent=3324"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/categories?post=3324"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/asapbio.org\/wp-json\/wp\/v2\/tags?post=3324"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}