<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/how-reproducible-data-analysis-scripts-can-help-you-route-around-data-sharing-blockers/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://blog.ouseful.info/2017/03/14/how-reproducible-data-analysis-scripts-can-help-you-route-around-data-sharing-blockers/"> Rstats – OUseful.Info, the blog…</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>For aaaagggggggeeeeeeesssssss now, I’ve been wittering on about how just publishing “open data” is okay insofar as it goes, but it’s often not that helpful, or at least, not as useful as it could be. Yes, it’s a <em>Good Thing</em> when a dataset is published in support of a report; but have you ever tried <em>reproducing</em> the charts, tables, or summary figures mentioned in the report from the data supplied along with it?</p>
<p>If a report is generated “from source” using something like Rmd (RMarkdown), which can blend text with analysis code <em>and a means to import the data used in the analysis</em>, as well as the automatically generated outputs, (such as charts, tables, or summary figures) obtained by executing the code over the loaded in data, third parties can see exactly how the data was turned into reported facts. And if you need to run the analysis again with a more recent dataset, you can do. (See <a target="_blank" rel="nofollow" href="https://github.com/psychemedia/parlihacks/blob/master/Rmd/gplsoa.Rmd">here</a> for an example.)</p>
<p>But publishing details about how to do the lengthy first mile of any piece of data analysis – finding the data, loading it in, and then <em>cleaning and shaping it enough so that you can actually start to use it</em> – has additional benefits too.</p>
<p>In the above linked example, the Rmd script links to a local copy of a dataset I’d downloaded onto my local computer. But if I’d written a properly reusable, reproducible script, I should have done at least one of the following two things:</p>
<ul>
<li><em>either</em> added a local copy of the data to the repository and checked that the script correctly linked relatively to it;</li>
<li><em>and/or</em> provided the original download link for the datafile (and the HTML web page on which the link could be found) and loaded the data in from that URL.</li>
</ul>
<p>Where the license of a dataset allows sharing, the first option is always a possibility. But where the license <em>does not</em> allow sharing on, the second approach provides a <em>de facto</em> way of sharing the data without actually sharing it <em>directly</em> yourself. I may not be giving you a copy of the data, but I am giving you some of the means by which you can obtain a copy of the data for yourself.</p>
<p>As well as getting round licensing requirements that limit sharing of a dataset but allow downloading of it for personal use, this approach can also be handy in other situations.</p>
<p>For example, where a dataset is available from a particular URL but authentication is required to access it (this often needs a few more tweaks when trying to write the reusable downloader! A stop-gap is to provide the URL in reproducible report document and explicitly instruct the reader to download the dataset locally using their own credentials, then load it in from the local copy).</p>
<p>Or as <a target="_blank" rel="nofollow" href="https://twitter.com/LWpaulbivand/status/841233890232684544">Paul Bivand pointed out via Twitter</a>, in situations <em>“where data is secure like pupil database, so replication needs independent ethical clearance”</em>. In a similar vein, we might add where data is <em>commercial</em>, and replication may be forbidden, or where additional costs may be incurred. And where the data includes personally identifiable information, such as data published under a DPA exemption as part of a public register, it may be easier all round <em>not</em> to publish your own copy or copies of data from such a register.</p>
<p>Sharing recipes also means you can share pathways to the inclusion of <em>derived</em> datasets, such as named entity tags extracted from a text using free, but non-shareable, (or at least, <em>attributable</em>) license key restricted services, such as the named entity extraction services operated by <em>Thomson Reuters OpenCalais</em>, <em>Microsoft Cognitive Services</em>, <em>IBM Alchemy</em> or <em>Associated Press</em>. That is, rather than tagging your dataset and then sharing and analysing the <em>tagged</em> data, publish a recipe that will allow a third party to tag the original dataset themselves and then analyse it.</p>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/15968/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/15968/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://blog.ouseful.info/2017/03/14/how-reproducible-data-analysis-scripts-can-help-you-route-around-data-sharing-blockers/"> Rstats – OUseful.Info, the blog…</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/jmx5w-iD6KQ"></a></div>