<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/farewell-then-pubmed-commons/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://nsaunders.wordpress.com/2018/03/15/farewell-then-pubmed-commons/"> R – What You're Doing Is Rather Desperate</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>PubMed Commons, the NCBI’s experiment in comments for PubMed articles, <a rel="nofollow" href="https://ncbiinsights.ncbi.nlm.nih.gov/2018/02/01/pubmed-commons-to-be-discontinued/" target="_blank">has been discontinued</a>. Thoroughly too, with all traces of it expunged from the NCBI website.</p>
<p><a rel="nofollow" href="https://nsaunders.wordpress.com/2016/12/02/an-analysis-of-contributions-to-pubmed-commons/" target="_blank">Last time I wrote about the service</a>, I concluded “all it needs now is more active users, more comments per user and a real API.” None of those things happened. Result: “NIH has decided that the low level of participation does not warrant continued investment in the project, particularly given the availability of other commenting venues.”</p>
<p>NLM <a rel="nofollow" href="https://www.nlm.nih.gov/pubs/techbull/ma18/brief/ma18_pubmed_commons_discontinued.html" target="_blank">also write</a> that “all comments are archived on our FTP site.” A CSV file is available <a rel="nofollow" target="_blank">at this location</a>. So is it good for anything?</p>
<p><span></span></p>
<p>The CSV archive contains only 6 fields: CommentId, PubmedId, DateCreated, FirstName, LastName and Content. This is unfortunate, as a lot of information has been lost. For example:</p>
<ul>
<li>user IDs, to disambiguate user names</li>
<li>comment up- and down-votes</li>
<li>threading, showing which comments were replies to other comments</li>
<li>information regarding comment moderation</li>
</ul>
<p>However, there is still information to be extracted from the file. Here’s <a rel="nofollow" href="https://github.com/neilfws/PubMed/blob/master/commons/code/R/pubmed_commons_archive.md" target="_blank">a summary document</a> at Github. We can see, for example that:</p>
<p>Comments per year never exceeded the maximum achieved in the first full year of operation (2014) and declined to a minimum in 2017<br>
<img src="https://i2.wp.com/github.com/neilfws/PubMed/raw/master/commons/output/archive/by-year-1.png?w=450&ssl=1" data-recalc-dims="1"></p>
<p>Comments per month also declined to a minimum in 2017, rarely surpassing 150 and often falling below 100.<br>
<img src="https://i0.wp.com/github.com/neilfws/PubMed/raw/master/commons/output/archive/by-year-month-1.png?w=450&ssl=1" data-recalc-dims="1"></p>
<p>We can count comments per article showing that the most-commented, with 33 comments, is: “<a rel="nofollow" href="https://www.ncbi.nlm.nih.gov/pubmed/?term=26933091" target="_blank">When is Science Ultimately Unreliable?</a>” You will never know now, from looking at PubMed, that this article was controversial and caused debate.</p>
<p>We can count comments per author showing that the “winner” is Lydia Maniatis, with 248 comments. You will never know now, from looking at PubMed, what inspired her and others or precisely how they interacted.</p>
<p>We can at least analyse the comment text; this simple word cloud highlights the prevalance of human clinical studies in publications that generated debate.<br>
<img src="https://i1.wp.com/github.com/neilfws/PubMed/raw/master/commons/output/archive/comment-words-1.png?w=450&ssl=1" data-recalc-dims="1"></p>
<p><strong>For more data</strong><br>
I re-ran my code a few days before PubMed Commons closed its doors, to generate a richer data file (commons.csv) that <a rel="nofollow" href="https://github.com/neilfws/PubMed/tree/master/commons/data" target="_blank">you can find here</a>. It contains 7619 comments, which I believe is only 10 less than the NCBI archive. I also re-ran my report one final time and you can <a rel="nofollow" href="https://github.com/neilfws/PubMed/blob/master/commons/code/R/pubmedcommons.md" target="_blank">see the results here</a>.</p>
<p>It is a shame, in my opinion, that NCBI never fully committed to PubMed Commons, and that this same attitude is apparent in their approach to archiving the data. I guess it was an interesting if flawed experiment.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://nsaunders.wordpress.com/2018/03/15/farewell-then-pubmed-commons/"> R – What You're Doing Is Rather Desperate</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/NiuQDvPwpHw" height="1"></a></div>