<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/a-partial-remedy-to-the-reproducibility-problem/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://apicdn.viglink.com/api/click?format=go&key=949efb41171ac6ec1bf7f206d57e90b8&loc=https%3A%2F%2Fwww.r-bloggers.com%2Ffeed%2F&out=https%3A%2F%2Fmatloff.wordpress.com%2F2017%2F05%2F31%2Fa-partial-remedy-to-the-reproducibility-problem%2F"> Mad (Data) Scientist</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Several years ago, John Ionnidis jolted the scientific establishment with <a rel="nofollow" href="http://apicdn.viglink.com/api/click?format=go&key=949efb41171ac6ec1bf7f206d57e90b8&loc=https%3A%2F%2Fwww.r-bloggers.com%2Ffeed%2F&out=https%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC1182327%2F" target="_blank">an article</a> titled, “Why Most Published Research Findings Are False.” He had concerns about inattention to statistical power, multiple inference issues and so on. Most people had already been aware of all this, of course, but that conversation opened the floodgates, and many more issues were brought up, such as hidden lab-to-lab variability. In addition, there is the occasional revelation of outright fraud.</p>
<p>Many consider the field to be at a crisis point.</p>
<p>In the 2014 JSM, Phil Stark organized <a rel="nofollow" href="http://apicdn.viglink.com/api/click?format=go&key=949efb41171ac6ec1bf7f206d57e90b8&loc=https%3A%2F%2Fwww.r-bloggers.com%2Ffeed%2F&out=https%3A%2F%2Fww2.amstat.org%2Fmeetings%2Fjsm%2F2014%2Fonlineprogram%2FActivityDetails.cfm%3FSessionID%3D210570" target="_blank">a last-minute session</a> on the issue, including Marcia McNutt, former editor of <em>Science </em>and Yoav Benjamini of multiple inference methodology fame. The session attracted a standing-room-only crowd.</p>
<p>In this post, Reed Davis and I are <a rel="nofollow" href="http://apicdn.viglink.com/api/click?format=go&key=949efb41171ac6ec1bf7f206d57e90b8&loc=https%3A%2F%2Fwww.r-bloggers.com%2Ffeed%2F&out=https%3A%2F%2Fgithub.com%2Fmatloff%2Frevisit" target="_blank">releasing the prototype</a> of an R package that we are writing, <strong>revisit,</strong> with the goal of partially remedying the statistical and data wrangling aspects of this problem. It is assumed that the authors of a study have supplied (possibly via carrots or sticks) not only the data but also the complete code for their analyses, from data cleaning up through formal statistical analysis.</p>
<p>There are two main aspects:</p>
<ul>
<li>The package allows the user to “replay” the authors’ analysis, and most importantly, explore other alternate analyses that the authors may have overlooked. The various alternate analyses may be saved for sharing.</li>
<li>Warn of statistical   errors, such as: overreliance on p-values; need for multiple inference procedures; possible distortion due to outliers; etc.</li>
</ul>
<p>The term <em>user</em> here could refer to several different situations:</p>
<ul>
<li>The various authors of a study, collaborating and trying different analyses during the course of the study.</li>
<li>Reviewers of a paper submitted for publication on the results of the study.</li>
<li>Fellow scientists who wish to delve further into the study after it is published.</li>
</ul>
<p>The package has text and GUI versions. The latter is currently implemented as an RStudio add-in.</p>
<p>The package is on my GitHub site, and has a fairly extensive README file introducing the goals and usage.</p>
<p> <a rel="nofollow" href="http://apicdn.viglink.com/api/click?format=go&key=949efb41171ac6ec1bf7f206d57e90b8&loc=https%3A%2F%2Fwww.r-bloggers.com%2Ffeed%2F&out=http%3A%2F%2Ffeeds.wordpress.com%2F1.0%2Fgocomments%2Fmatloff.wordpress.com%2F1847%2F" target="_blank"><img border="0" alt="" src="http://feeds.wordpress.com/1.0/comments/matloff.wordpress.com/1847/"></a> <img border="0" alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" data-recalc-dims="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://apicdn.viglink.com/api/click?format=go&key=949efb41171ac6ec1bf7f206d57e90b8&loc=https%3A%2F%2Fwww.r-bloggers.com%2Ffeed%2F&out=https%3A%2F%2Fmatloff.wordpress.com%2F2017%2F05%2F31%2Fa-partial-remedy-to-the-reproducibility-problem%2F"> Mad (Data) Scientist</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://apicdn.viglink.com/api/click?format=go&key=949efb41171ac6ec1bf7f206d57e90b8&loc=https%3A%2F%2Fwww.r-bloggers.com%2Ffeed%2F&out=https%3A%2F%2Ffeedburner.google.com%2Ffb%2Fa%2Fmailverify%3Furi%3DRBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="http://apicdn.viglink.com/api/click?format=go&key=949efb41171ac6ec1bf7f206d57e90b8&loc=https%3A%2F%2Fwww.r-bloggers.com%2Ffeed%2F&out=https%3A%2F%2Fwww.r-project.org%2F" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="http://apicdn.viglink.com/api/click?format=go&key=949efb41171ac6ec1bf7f206d57e90b8&loc=https%3A%2F%2Fwww.r-bloggers.com%2Ffeed%2F&out=https%3A%2F%2Fwww.r-users.com%2F" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/6Dsz0BCKEbM" height="1"></a></div>