Alum is architect of influential science-watchdog site
After three years, Brandon Stell reveals role in establishing controversial website
The mastermind behind a formerly anonymously run website that serves as a crowd-sourced watchdog of peer-reviewed scientific journal articles turns out to be Brandon Stell, a 1997 graduate of the 做厙輦⑹.
Stell recently disclosed that he is the architect of泭, the controversial online public forum where scientists can anonymously critique each others published research. The sites users have exposed many errors in published works.
While a student at CU-Boulder, Brandon Stell joined discussions of published, peer-reviewed research. He noted a high frequency of errors or other problems in the work, and he thought it was a shame that such feedback could not be shared widely. Eventually, this experience helped him hatch the idea for what became PubPeer. Photo courtesy of Brandon Stell.
The idea for PubPeer originated when Stell was an undergraduate at CU-Boulder studying ecology, populations and organismal biology. Stell regularly attended meetings to discuss recently published research.
The discussion groups dissected papers, scrutinizing each experiment and conclusion. This in-depth review of peer-reviewed research always revealed a few flaws.
Ive never been to a journal club meeting that didnt turn up one or two issues about a paper, says Stell.
Stell enjoyed the occasionally brutal discussions, but he left the meetings disappointed. The insights, remaining questions and possible flaws regarding the research hung in the air with no resolution.
I always thought that the authors could maybe clarify an issue if they had a voice at the journal club, but they didnt. And, when issues were brought up in the meeting, no other scientists in that field ever heard about it, says Stell.
That is really important because scientists build off of each others work.
We thought, if we could create this community where people are discussing science that will take the focus off of where science is published and put it on the content or the data that were publishing.
Stell longed for a centralized, public forum for scientists to discuss and critique each others published research.
The idea that grew into PubPeer percolated for years. Throughout his graduate studies in neuroscience at the University of California Los Angeles and a post-doctoral fellowship in Paris, Stell ranted about this idea in hallways, perpetually asking colleagues why someone else hadnt already launched such a forum.
He even contacted Google Scholar, a specialized search engine for academic and scholarly literature, hoping to encourage it to establish such a site. His email went unanswered.
He caught a break when Richard Smith came to work in his lab in Paris. Richard and his brother George, who provides technical expertise, helped Stell launch the site in 2012. (PubPeer has revealed the brothers names but not their professional affiliations.)
The site published anonymous critiques for an obvious reason: Criticism is rarely complimentary. And because peers in the same area of expertise review ones research, any critique carries the risk of offending a colleague. This could have significant consequences for the critics career.
If some of my colleagues papers were critiqued, people who were reviewing my grants and my papers and basically deciding my career, if their papers were being critiqued on the site and they knew that I was behind the site, they probably wouldnt be very happy about that, Stell explains.
It would also put pressure on me to remove comments that people were unhappy with. So, we were anonymous, and because we were anonymous, it made it difficult to promote the site. You cant really do that anonymously.
Despite the anonymity and lack of promotion, the site gained popularity. In fact, once anonymous commenting was enabled, the sites use more than tripled in less than two years (). One success of PubPeer is that it allows other scientists in a field, who are building on that published research, to learn about other researchers critiques and concerns.
When there are problems that are noticed, then other scientists [in that field] are aware of it勳鳥鳥梗餃勳硃喧梗梭聆泭because theres now this forum, explains Stell.
Stell and colleagues saw that this was going to have broader implications.
If scientists dont know that there are problems with a research article, this has major cost, says Stell.
Theres an economic cost were wasting taxpayer money trying to reproduce or build on [research] that cant be built upon. It also has a human cost because graduate students and post-docs could be wasting their careers trying to build on something that cant be built upon. If a student spends three years trying to build on something thats not very solid, and in the end cant do it, it makes it very difficult for their scientific career.
With a system like PubPeer, as soon as theres a problem with a paper, people who use the research as a foundation for future initiatives, including policymakers, health-care providers, and pharmaceutical companies know about it.
Additionally, PubPeer provides an avenue for focusing scientists attention on the fundamental issue: science.
We thought, if we could create this community where people are discussing science that will take the focus off of where science is published and put it on the content or the data that were publishing, explains Stell.
I feel that scientists are on the edge of an interesting moment in science where were collectively questioning the role of journals.
Publication is not the end of the road, according to Stell. Its not, OK Ive been through the review and publication process, the papers now in a coveted research journal, I can cash it in for jobs, or grant money.
If the data arent solid, theres a conversation thats going to happen about that.
Roni Dengler is a writer for CUs Science Buffs blog and a Ph.D. candidate in the Molecular, Cellular and Developmental Biology Department.
泭