The Path to Inclusive Science Paved with Preprints?
The Open Science movement, characterised by the open sharing of ideas, theories, methods, data, and evidence to form the basis for a collaborative and innovative global research system, is gaining ground across the world – no doubt, accelerated by the unprecedented sharing of scientific insights during the peak of the Covid-19 pandemic. The crisis clearly showed that by opening up research outputs early to wider review and feedback, we stand to create more agile research ecosystems that are capable of delivering effective solutions to global challenges.
Dr Jessica Polka heads ASAPbio, a scientist-driven non-profit promoting transparency and innovation in life science communication.
One important lever for change in the way science is share and communicated is the use of preprints – the advance versions of scientific papers that are published before the formal peer review process. Preprints are thought to allow for the faster exchange of research, and enable a more open, collaborative, and inclusive research culture.
In this conversation with Dr Jessica Polka, Executive Director and Co-Founder of ASAPbio (Accelerating Science and Publication in biology), we talk about the productive use of preprints, as well as the critical role for transparent and open peer review in making research more accessible, diverse, inclusive and equitable.
Question: What drove you towards working to make scientific publishing more open and transparent?
Jessica Polka: As a postdoc, I felt that the incentives for early career researchers often run counter to the goals of efficient and collaborative knowledge generation – for instance, the push to place greater importance on publishing in a high-impact journal rather than on ensuring reproducibility, to avoid sharing data for years until it forms a “complete story,” or to value shallow metrics above social impact. I saw preprints as a practical way for many researchers to engage with open science, which would not only benefit individual researchers but also the research enterprise as a whole.
However, open communication about research doesn’t end when someone posts a preprint. Thus, we expanded our work at ASAPbio to include open peer review, both within and outside of journals.
Q: Could you tell me a bit more about preprints and how they could improve research culture and output?
JP: Preprints remove barriers to sharing, reading, and collaborating. In contrast to a traditional journal article that might be hidden from all but a handful of peer reviewers for months or even years before publication, preprints enable everyone around the world to have rapid access to new research. This puts authors in control of dissemination. This system is also compatible with traditional journals, thus enabling people to participate in both open science and more conventional workflows simultaneously.
Furthermore, when people share a journal article, they’re putting out a product that is more or less “set in stone”, and any comments or suggestions for improvement are not likely to be incorporated. By contrast, when someone shares a preprint, they can choose to share a draft in provisional form at a time when they can incorporate feedback and improve their work, maybe even add new collaborators to the project. This creates a genuinely productive dialogue.
“The traditional peer review system in journals is built on trust: authors and readers trust that when a paper is published in a given journal, it has been through a rigorous process. In turn, the editors of journals select reviewers they know they can rely on, creating a “club” of sorts.”
What are the benefits for individual researchers, especially early career researchers?
JP: Preprints allow everyone to participate in providing feedback on a paper. Besides social media and the commenting functionalities of preprint servers, there is a thriving ecosystem of projects that provide peer review on preprints, ranging from projects that highlight interesting preprints (e.g., preLights), to those that automate screening (e.g., ASWG), and provide editor-organised review (e.g., Review Commons, Peer Community In). Many of these projects, such as PREreview and ASAPbio’s own Crowd preprint review activities, are directly beneficial to early career researchers who might be more comfortable commenting anonymously, collaborating with others, or covering specific areas of a paper. And because these projects invite researchers to comment on a paper outside the context of a journal, reviewers can focus on the quality and merit of the science as opposed to whether it meets certain criteria for publication, which can only improve research outputs as whole.
Q: What do you think are the challenges in making the broader peer review system more inclusive?
JP: The traditional peer review system in journals is built on trust: authors and readers trust that when a paper is published in a given journal, it has been through a rigorous process. In turn, the editors of journals select reviewers they know they can rely on, creating a “club” of sorts. Furthermore, as the identities of reviewers and editors are often not known (which is sometimes necessary to protect the vulnerable from retaliation), there’s the potential for favouritism and bias.
Q: Do preprints make research more inclusive?
JP: Preprints are free to read and to post, and the screening process of preprint servers is more “light-weight” than peer review at a journal. This lowers barriers to sharing research, and it means that anyone, not just people invited by a journal, can act as a peer reviewer. However, there are disparities in who is posting preprints, with more representation from select countries and institutions (see Abdill et al.). We recognise that preprints alone aren’t a complete solution, and we are working towards broader cultural change in how research is created, communicated, and assessed.
“…since peer review decisions have significant impact on the authors under review, often making or breaking opportunities for funding, hiring, and promotion, it’s important that peer review proceeds fairly.”
Q: What are some first steps we can take within the peer review system to increase equity and opportunities for underrepresented groups (women, researchers from the global south, unaffiliated researchers) in research?
JP: First, we need a stronger evidence base from which we can recommend interventions. Journals and peer review projects could collect more demographic information from peer reviewers, authors, and editors to ensure that interventions can be studied more systematically. Tools such as PREreview’s bias reflection guide, for example, could be integrated into review workflows, which could help to counteract homophily.
Finally, we need better systems for recommending (and building trust in) reviewers that come from outside a given editor’s network. For example, public reviews, whether on published articles or on preprints (see our Preprint Reviewer Recruitment Network), could serve as work samples to demonstrate that a researcher would make a strong reviewer.
Q: What are some of the “success stories” from ASAPbio that have led to an increase in transparency in the peer review system?
JP: After our 2018 meeting on peer review, over 300 journals signed an open letter committing to enabling the publication of peer review reports alongside published articles. This surfaces the important scholarship involved in peer review, helps readers better understand the paper, and the transparency improves the integrity of the peer review process. On the preprint side, dozens of researchers have signed a pledge to publish reviews they have written alongside preprints. In addition to the many benefits to preprints listed above, this action opens the door to the reuse of peer reviews, which can serve as a catalyst for more public conversations about research.
“As a postdoc, I felt that the incentives for early career researchers often run counter to the goals of efficient and collaborative knowledge generation…”
Q: In your opinion, how would improving diversity, inclusion, and equity in the peer review system contribute to scientific progress?
JP: Peer review can improve the robustness and clarity of the vast body of scientific literature so it’s important that it works as well as possible. Evidence has shown that diverse groups are better at solving problems. From this standpoint, it’s vital that we bring a variety of perspectives into editorial processes.
Furthermore, since peer review decisions have significant impact on the authors under review, often making or breaking opportunities for funding, hiring, and promotion, it’s important that peer review proceeds fairly. Ultimately, fair processes can help to preserve the diversity we need to solve important research questions.
This interview is part of the ALLEA Digital Salon Women in Science Series. Dr Jessica Polka will moderate a panel discussion at the upcoming webinar on Inclusion, Diversity, Equity and Accessibility (IDEA) in the scholarly peer review system, co-organised by ALLEA, GYA (The Global Young Academy) and STM (International Association of Scientific, Technical and Medical Publishers). You can find out more about this webinar, which will tackle the challenges and opportunities for improving IDEA in peer review here.
About Jessica Polka
Dr Jessica Polka serves as Executive Director of ASAPbio, a researcher-driven non-profit organisation working to promote innovation and transparency in life sciences publishing in areas such as preprinting and open peer review. Prior to this, she performed postdoctoral research in the department of Systems Biology at Harvard Medical School following a PhD in Biochemistry & Cell Biology from UCSF. Dr Polka is also a Plan S Ambassador, an affiliate of the Knowledge Futures Group, an independent non-profit organisation that works to make knowledge more open and accesible, and a steering committee member of Rescuing Biomedical Research.
Read More by Jessica Polka
Preprinting the COVID-19 pandemic
