<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/statcheck-an-r-package-to-check-statistical-results-in-psychology-papers/"></div></div><div><a data-url="https://www.r-bloggers.com/statcheck-an-r-package-to-check-statistical-results-in-psychology-papers/" data-counturl="https://www.r-bloggers.com/statcheck-an-r-package-to-check-statistical-results-in-psychology-papers/" data-count="horizontal" data-text="Statcheck: an R package to check statistical results in psychology papers" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2016/10/statcheck.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>The <a target="_blank" href="http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124">results of many scientific papers are wrong</a>. There are many reasons for this, including <a target="_blank" href="https://www.statnews.com/2016/05/09/john-oliver-bad-science/">p-hacking,</a> <a target="_blank" href="http://blog.revolutionanalytics.com/2012/03/bad-science-at-strata-2012.html">publication bias,</a> and the general <a target="_blank" href="http://blog.revolutionanalytics.com/2016/01/rs-role-in-science-breakthrough-reproducibility-of-psychology-studies.html">inability to replicate</a> results. But there's another, more mundane cause: incorrect calculation of p-values in statistical tests. This could be caused by simple transcription errors when plugging numbers into a statistical tool, incorrect rounding, or misapplication of the test itself (say, applying a two-sided test when a 1-sided p-value is appropriate). Such errors should be picked up in the peer review process, but given that even <a target="_blank" href="http://fivethirtyeight.com/features/not-even-scientists-can-easily-explain-p-values/">expert statisicians sometimes struggle to explain p-values</a>, it's not surprising that some errors get through.</p>
<p>That's why Michèle B. Nuijten, a PhD student at Tilburg University, created the <a target="_blank" href="https://mbnuijten.com/statcheck/">R package statcheck</a>. Given a paper to be published in a psychology journal, statcheck searches for statistical results from \(t\), \(F\), \(r\), \(\chi^2\), and \(Z\) tests, and compares the published p-value to a value calculated by R. This is possible only because the <a target="_blank" href="http://www.apastyle.org/">American Psychological Association Style Guide</a> has a very specific format for reporting statistical results, listing the p-value next to the reported test statistic. Statcheck also attempts to detect if the surrounding language mentions a &quot;one-sided&quot; or &quot;one-tailed&quot; test and calculates the p-value in R accordingly (although this process isn't perfect). Anyone can use statcheck by uploading a PDF or HTML version of their paper to the <a target="_blank" href="http://statcheck.io/">statcheck web application</a>, or by <a target="_blank" href="http://rpubs.com/michelenuijten/202816">using the statcheck function within R</a> directly.</p>
<div><a target="_blank" href="http://a5.typepad.com/6a0105360ba1c6970c01bb093edd35970d-pi"><img title="Comparison" alt="Comparison" src="http://a5.typepad.com/6a0105360ba1c6970c01bb093edd35970d-800wi" border="0"></a>
<div>statcheck compares reported p-values with computed p-values and reports discrepancies, noting whether the difference would have changed the outcome of the test.</div>
</div>
<p>Nuijten recounts the origins and development of statcheck in an interesting <a target="_blank" href="http://retractionwatch.com/2015/11/17/making-it-easier-and-more-automated-to-find-errors-a-guest-post-from-the-co-developer-of-statcheck/">article in Retraction Watch</a>. One major surprise: <a target="_blank" href="http://link.springer.com/article/10.3758/s13428-015-0664-2">when they applied statcheck</a> to p-values reported in eight major psychology journals from 1985 to 2013:</p>
<blockquote>
<p>Half of the papers in psychology contain at least one statistical reporting inconsistency, and one in eight papers contain an inconsistency that might have affected the statistical conclusion.</p>
</blockquote>
<p>Since then, they've further automated statcheck by <a target="_blank" href="http://retractionwatch.com/2016/09/02/heres-why-more-than-50000-psychology-studies-are-about-to-have-pubpeer-entries/">automatically sharing the results of its analyses for 50,000 papers at PubPeer</a>. <a target="_blank" href="http://www.vox.com/science-and-health/2016/9/30/13077658/statcheck-psychology-replication">Not everyone was pleased</a> by the notifications (a former president of the Association for Psychological Science called it '<a target="_blank" href="http://www.businessinsider.com/susan-fiske-methodological-terrorism-qa-2016-9?r=UK&IR=T">methodological terrorism</a>'), but the process did reveal more inconsistencies in published papers.</p>
<p>For more on statcheck, check out its website at the link below.</p>
<p>Michèle B. Nuijten: <a target="_blank" href="https://mbnuijten.com/statcheck/">R package “statcheck”: Extract statistics from articles and recompute p values (Epskamp &amp; Nuijten, 2016)</a></p>
</div>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/statcheck-an-r-package-to-check-statistical-results-in-psychology-papers/"></div></div><div><a data-url="https://www.r-bloggers.com/statcheck-an-r-package-to-check-statistical-results-in-psychology-papers/" data-counturl="https://www.r-bloggers.com/statcheck-an-r-package-to-check-statistical-results-in-psychology-papers/" data-count="vertical" data-text="Statcheck: an R package to check statistical results in psychology papers" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2016/10/statcheck.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/nuPoOI8ewgQ"></a></div>