This post was updated on February 27, 2018.
Update: NLM appreciates all of the input we have received in response to our February 1, 2018, announcement that PubMed Commons is being discontinued. Thank you for your thoughtful comments. We are heartened to hear that many of you found it to be a useful service.
As we mentioned in the announcement, comments on articles indexed in PubMed will continue to be visible on PubMed and PubMed Commons through March 2, 2018, after which time they will be available for download from NCBI’s website. NLM again thanks all of you who participated in PubMed Commons for your interest and effort.
PubMed Commons has been a valuable experiment in supporting discussion of published scientific literature. The service was first introduced as a pilot project in the fall of 2013 and was reviewed in 2015. Despite low levels of use at that time, NIH decided to extend the effort for another year or two in hopes that participation would increase. Unfortunately, usage has remained minimal, with comments submitted on only 6,000 of the 28 million articles indexed in PubMed.
While many worthwhile comments were made through the service during its 4 years of operation, NIH has decided that the low level of participation does not warrant continued investment in the project, particularly given the availability of other commenting venues.
The discontinuation plan is as follows:
- New comments will be accepted through February 15, 2018.
- Comments will continue to be visible on the PubMed and PubMed Commons websites through March 2, 2018.
- Users wishing to access the comments after March 2, 2018, will be able to download them from NCBI’s website.
Many thanks to all of you who participated in this experimental effort to enhance the opportunities for interaction about published biomedical literature.
This is too bad. I feel the decision happens at a moment where the cultural shift is actually happening, and when the feature would be most needed… Anyway, you mention that “the availability of other commenting venues.” Would you care to elaborate? Because as far as I know, this is was or less the only place where papers could be commented in a public sphere…
eLetter in Science attached to Peterson article:
Just as thermometers provide a helpful index of health status, so can be viewed the various resources providing indices of scientific worth. Among the many considerations guiding a busy academic’s decision to review a paper, grant application, or even the curriculum vitae of a future Nobelist (i.e. pre-publication review), is the availability of these resources.
Gregory Peterson’s note (March 16th) extolling the virtues of post-publication peer-review was written before the demise of an invaluable reviewer resource, the NCBI’s PubMed Commons (March 2nd). This was unique in providing a link from the PubMed abstract of a paper directly to the post-publication comments on that paper that had accumulated over the five years since the inception of PubMed Commons in 2013.
PubMed abstracts remain a popular first port-of-call for those checking lists of references. While the loss of PubMed Commons now decreases an abstract’s utility, cutting the link to past comments compounds the problem in appearing to scornfully dismiss the hard work of numerous non-anonymous contributors, who were themselves required to have publishing credentials. In this respect PubMed Commons differed from other post-publication review sites (e.g. PubPeer) that allowed anonymity and demanded no credentials. Furthermore, with internet browsers other than Firefox and Chrome (for which there are apps) there was only a one-way link (e.g. from PubPeer to PubMed), not the converse.
We weep for the loss of an important scientific thermometer which, unlike resources where torrents of anonymous comments flow free, has been an invaluable post-publication peer-review resource making that pre-publication review “yes” decision so much easier.
I’m flabbergasted by this management of grocer. Are the 28 millions out of 6000 articles really worth it? Probably not, the usual wisdom says there are only 1% of interesting papers. It is way more expensive to maintain this huge heap of junk than some sparse comments, which only need the code to be installed and not so often updated. Even if there were only one comment, it could have a tremendous value and impact, they aren’t sausages on a shelf. The authors of the preprints surely haven’t time enough to systematically scan Internet searching the other “venues” where they’ll find what concern them most directly.
I also think this is a wrong decision, but by now it is clear that no amount of reasoning, cajoling, or pleading is going to reverse it. Clearly there is some reason why it is imperative that these comments be removed from public scrutiny, and further comments be prohibited. The lame excuse of expense of upkeep, is just that. Hard to see what the reason could be, though- the policy preventing anonymous comments would seem to shift any legal liability to the commenters, and leave PHS blameless.
Because as far as I know, this is was or less the only place where papers could be commented in a public sphere…
giandrasoft, E. Gaudrain, and others: a widely-used website for post-publication peer review is PubPeer:
http://www.pubpeer.com/
Widely used but not public (i.e. note “.com” domain) and commenters are not identified. Pubpeer serves an often-useful but quite distinct purpose relative to PubMed Commons.
I am saddened to learn that PubMed Commons is being killed off – just as I learn that it exists! As one commenter observed, there’s probably no use in complaining. But to reason the program should stop because it isn’t well-subscribed is ridiculous. Problem is, people don’t know about it and it needs to be easily accessible. Why hasn’t it been advertised or promoted? I suspect the number of people who post there is proportional to the number who know about it. Lots of clinicians and others comment on articles in Medscape, and I find the published comments to be thoughtful and useful. Why would it be different at PubMed? I say, offer a comments feature that’s visible and accessible to anyone who reads a paper.
Regarding original material I posted to PubMed Commons and nowhere else, I have been citing the URL of my comment on PubMed Commons on subsequent PubMed-indexed publications I have written. Will readers be able to click on those URLs see my original comment on PubMed Commons?
Links from PubMed to PubMed Commons were cut. Original PMC entries are stored at NCBI as difficult-to-access Excel files. Fortunately the Hypothesis folk have created an independent resource. I refer to my PMC entries using the following URL: https://hypothes.is/search?q=tag%3APubMedCommonsArchive+forsdyke
Madness. I have (or had?) made two comments on papers on PMC. I’m not bragging when I say I think it takes a certain degree of bravery to make non-anonymous comments on such a forum. It is clear that a scientist is taking a professional risk to publicly highlight the weaknesses in their peers’ work: Academic research is (for better or worse) reliant upon scientists to make value judgements on each other. For both publications and funding applications scientists are peer-reviewed, and to publicly criticise the work of other scientists risks future retribution (either willing or unconscious). From my own experience I made the comments because I wanted to give scientists from other fields the benefit of our observations on papers that may be difficult to appraise without our depth of knowledge or experience. Without this, the potential limitations of a paper may take years to be exposed (if they ever are). In short, I put science before my own interests.
As another commenter pointed out, PMC had a very high bar before allowing anyone to comment: non-anonymity and a track record of publication. I think this was a very good idea and gave PMC a unique strength over other post-publication commentary sites. However, they (NIH) surely must have known that this would limit uptake for the reasons I’ve given above. But this limited uptake will undoubtedly be balanced out by the care people take before they put their names to any comments. Quality over quantity.
I despair at how many papers in our field go unchallenged in public, but that the community ‘know’ to be flawed. This knowledge MUST have an outlet, and it must be as close as possible to the source of knowledge (NCBI) to allow others to benefit from it. If no-one has the gumption to speak up out then how much time and money is wasted by those who read the papers believing them to be fact?
From a wider perspective the root cause of this limited uptake of PMC is the reluctance of scientists to embrace criticism. If we all accept we make mistakes, and that it is science that will benefit if they are exposed, then we will welcome others’ criticisms of our work. If on the other hand we see criticism as a threat to ourselves and something that must be resisted and avenged then science itself is doomed to wallow in a mire of misinformation for many, many more years to come. The choice is ours, and the decision to pull PMC has made it even harder to make the right one.
Mike Tatham (University of Dundee) m.tatham@dundee.ac.uk
I agree with your assessment of the situation and fully understand your last remark: The choice is ours, and the decision to pull PMC has made it even harder to make the right one. I also trust that criticism is what makes science evolve. Perhaps scientists would make right choices if, early in their career, they were educated to better understand the consequences of making wrong choices.
Also, the word ‘wrong’ has part of its understanding dictated from a moral point of view, in this case the morality of truth. In other words, truth should be the morality of the individual scientist. Search for truth precludes search for recognition by the scientific community as a whole. However, when truth is no longer valued by the whole, including editors of peer-reviewed journals, the quest for truth is substituted by the quest for recognition by any means for personal gain. In this context, criticizing papers threatens the credibility of authors and editors, as well as the education system. As such it is not welcome. Unless we are going back to strict ethical standards and start valuing criticism, I do not see a way out of the present madness.
I totally agree. We need to resist many things as scientists if our understanding of the universe (I presume you mean this by the term “Truth” – although some scientists don’t like the word) is to progress efficiently. Putting ‘truth’ before our self-interests is incredibly difficult, and is not helped by the system we work in, as you point out. In fact there are many reasons why this is hard:
Journals resist criticism because retraction is seen as a bad reflection on their review standards (It shouldn’t be).
Universities foster a hero culture where ‘achievements’ are trumpeted on their web-sites and press releases (which just encourages egotism).
Various groups and agencies give scientists ‘prizes’ and ‘awards’ (although this also fosters egotism).
We create fuzzy metrics to calculate the value of scientists, like h-index or research-gate scores (again encouraging comparison among individuals).
I could go on. In general I think the research community is guilty on many levels for creating an environment where it is harder and harder to put ‘truth’ before self-interests, and I don’t see that changing any time soon. I’ve noticed this is not unique to science and seems to be something humans indulge in in pretty much any field of endeavour. But of all the different things humans do, science is particularly crippled by egotism and more should be done and said about it. But what we get whenever we open a journal, or read a university press-release or hear about ‘successful’ scientists on the TV, is more hero-worshipping. Why is no-one pointing out the negative consequences of this?
For what little it is worth I have personally vowed to ignore all attempts to create myths about myself by rejecting awards and prizes, but this does not make headlines and I can’t put letters after my name to show it, so the act is largely impotent. But if more ‘famous’ and respected scientists did this publicly I think we might start to shift the culture from the cult of the individual to the cult of ‘truth’.
What do you think?
Mike
Thanks for the info. Good views and source of information.