Diversifying peer review by adding junior scientists

Initiatives to train and include early-career researchers in peer review may help improve science’s quality control.

13 September 2019

DigitalVision Vectors/Getty

Their paper about the evolution of malaria was in review for what seemed like an eternity. Every month, Susan Perkins and her then-graduate student Spencer Galen would check in with the editors.

The problem, evidently, was a lack of peer reviewers. “But no one ever asks us to review,” Perkins recalled Galen saying.

That struck a chord. Inspired by the success of DiversifyEEB, an initiative focused on improving representation of women and minorities in ecology and evolutionary biology, Perkins, a curator and professor of microbial genomics at the American Museum of Natural History, created a database of early-career researchers available to be reviewers.

“A lot of editors would say, ‘I’d be happy to use them; I just don’t know how to find them,’” Perkins said. Since she launched the list in February 2018, it has grown to more than 1,100 early-career researchers in ecology, evolution, behavior, and systematics — and more than 200 editors have accessed it.

Peer review is designed to act as a quality check on science. When a paper is submitted to a journal, editors recruit two or more volunteer experts to critique the work to decide whether it’s suitable for publication.

Beyond offering just “yes” or “no” opinions, reviewers can provide constructive criticism that improves the science. But reviewers are often senior scientists who are statistically more likely to be old, white, and male.

Advocates say employing more early-career researchers in peer review — from scientists in the final years of their Ph.D. to young principal investigators — could help diversify and improve science.

In the past few years, efforts to include these scientists in peer review have sprung up across multiple fields. Some, like the Early Career Reviewer Program at the National Institutes of Health, open up reviewer spots that often go to senior researchers.

Others, like the Journal of Neuroscience’s mentorship program and an online course from the journal Nature, offer peer review training.

"Like poetry, love or justice”"

The push to include early-career researchers in peer review is deeply tied to other diversity initiatives, according to Elisa De Ranieri, editor-in-chief of Nature Communications.

“Diversity and inclusion in general have become more of a hot topic on everybody’s tongue,” she said. “Then of course, this is when we are trying to focus our efforts.”

Science may be methodical, but many researchers regard peer review as something closer to an art. After all, every reviewer makes subjective assessments on facets of the work, and every editor weighs reports differently.

“Peer review is impossible to define in operational terms,” as Richard Smith, a former editor of the British Medical Journal, put it in 2006. “Peer review is thus like poetry, love, or justice.”

And just like democracy, Smith added, peer review is “a system full of problems, but the least worst we have.”

One ongoing problem: Reviewers, unpaid and unrecognized for what is usually anonymous work, are fatigued.

“Peer review is the service we mutually provide to each other,” said Perkins. “So if you’re submitting publications, you understand that people are giving up their time to review your work so there’s a certain amount of reciprocity needed in that system.”

But when senior researchers are overburdened as reviewers, and junior scientists are underused, the system fails to be reciprocal. Attempts to address reviewer fatigue have focused on incentives such as public recognition or small payments. But these efforts don’t help a key cause of fatigue: the small pool of reviewers that editors keep selecting.

Formal initiatives like the NIH’s Early Career Reviewer Program, which attempts to diversify peer reviewers of grant applications, are one way to improve representation.

But changing editor habits may be more important in the long run. “As an editor, I can always think of four or five people,” said Stephen Heard, an evolutionary biologist at the University of New Brunswick in Canada and an associate editor of the American Naturalist, a peer reviewed journal published by the University of Chicago.

“But of course my recollection is slanted to the old fogies like me that I’ve seen at conferences over the years.”

Breaking out of old habits

In a 2017 blog post, Heard and another colleague gave editors advice on how to break these habits, such as checking department websites for younger researchers and inviting senior scientists to suggest junior colleagues.

Beyond alleviating reviewer fatigue, skewing the reviewer pool younger would diversify science, many experts suggest. This is in part because while demographics vary across fields, younger researchers are more likely to be women, identify as LGBTQ, and be racially diverse compared to their older counterparts.

And as Undark reported in July, calling on reviewers with more varied perspectives may help guard against the publishing of blatantly flawed or biased work. Of course, scientists fresh out of their Ph.D. programs don’t necessarily have more time than their more-seasoned colleagues, but they are often more willing to review.

Additionally, early-career scientists are perceived by many editors as making excellent reviewers. Though there’s no hard data, some editors Undark spoke to agreed that these researchers write reports that are generally more thorough and technical than their more senior counterparts.

Heard also said that in his experience, younger reviewers are better at giving kinder, more constructive critiques. Without decades of collaborations, scientists who are just starting out may also have fewer conflicts of interest.

Diversifying peer review doesn’t just help the scientific community at large or journals that need reviewers, De Ranieri said. It also gives new researchers valuable experience. Marina Picciotto, a Yale University neuroscientist and editor of the Journal of Neuroscience, agreed.

“My own works gets infinitely better as I watch myself react to other people’s work,” she said. “I see what I love and whether it’s somebody’s writing or it’s somebody’s set of controls or their conceptualization of an idea.”

Impostor syndrome

In what is otherwise a highly trained profession, the fact that science largely lacks formal avenues to teach peer review stands out. Although some advisers give their graduate students and postdoctoral fellows the basics, many don’t view it as their responsibility.

A 2018 study of 264 early career scientists found that more than half received no help reviewing from their advisers.

Young researchers are instead expected to learn by doing. And when the opportunity finally arrives, the responsibility can be overwhelming.

When Riley Brady, a PhD student at the University of Colorado Boulder reviewed his first paper, he said he felt like an impostor.

“I [was] given this paper and it’s like, ‘Oh my God,’” says Brady. “The entire scientific enterprise is built upon peer review and now they’re trusting me to review this paper?“

According to Brady, there aren’t any programs to teach peer review specifically in his fields — climate science and oceanography — so he had to rely on encouragement and a few tips from his adviser.

Meanwhile, other organizations are also trying to fill in gaps like these. Sense about Science, a nonprofit that promotes scientific literacy among the public, and Nature both have tutorials that teach peer review basics. And dozens of researchers have completed the *Journal of Neuroscience’*s mentorship program since it started in early 2018.

“There’s a huge need, and a huge desire among our junior colleagues to become involved in the peer review process,” Picciotto said. “And we think that training can really help with that.”

Although most journals have existing reviewer databases, using them to target junior scientists is complex and could run afoul of laws like the European Union’s General Data Protection Regulation, a far-reaching set of regulations that protect the private data of E.U. citizens, according to De Ranieri.

For now, researchers may have to do much of the legwork by signing up with lists like Perkins’ and searching out relevant courses. But with pushes for science to become more transparent and diverse, there will likely be more opportunities for early career researchers.

“We want to include them; they want to be included,” says De Ranieri. “It’s just a matter of ‘How do we make this happen?’”

Dan Garisto is a science journalist based in New York. He writes about physics and has been published in outlets including Scientific American, Symmetry, Science News, Hakai, and News from Science.

This article was originally published on Undark. Read the original article.