<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/r-hero-saves-backup-city-with-archivist-and-github/"></div></div><div><a data-url="http://www.r-bloggers.com/r-hero-saves-backup-city-with-archivist-and-github/" data-counturl="http://www.r-bloggers.com/r-hero-saves-backup-city-with-archivist-and-github/" data-count="horizontal" data-text="R Hero saves Backup City with archivist and GitHub" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://r-addict.com/2016/06/13/RHero-Saves-Backup-City-With-archivist-github.html"> http://r-addict.com</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><img alt="" src="http://i2.wp.com/r-addict.com/images/fulls/archivist_github.png?w=456" data-recalc-dims="1">Have you ever suffered because of the impossibility of reproducing graphs, tables or analysis’ results in R? Have you ever bothered yourself for not being able to share R objects (i.e., plots or final analysis models) within your reports, posters or articles? Or maybe simply you have too many objects you can’t manage to store in a convenient and handy way? Now you can share partial results of analysis, provide hooks to valuable R objects within articles, manage analysis’ results and restore objects’ pedigree with <a target="_blank" href="http://pbiecek.github.io/archivist/Posts.html">archivist</a> package and its extension <a target="_blank" href="http://r-addict.com/archivist.github/">archivist.github</a>, allautomatically through <a target="_blank" href="https://github.com/">GitHub</a> without closing RStudio. If you are tired of archiving results by yourself, then read how you can became an R Hero with the <a target="_blank" href="http://r-addict.com/archivist.github/">archivist.github</a> package power.</p>
<h1>R Hero archiving power</h1>
<p>Recently I’ve visited Backup City, a data analysis mecca in the middle of Reproducible Research RLand. That’s where I ovearheared a feverish discussion between R Hero and commissar O’Rdon. You can read the story of their meeting at the opening comic.</p>
<h1>archivist.gitub: archivist and GitHub integration</h1>
<p>archivist.github is a package with tools for archiving, managing and sharing R objects via GitHub and is the extension of the archivist. You can install package from <a target="_blank" href="https://cran.r-project.org/web/packages/archivist.github/index.html">CRAN</a></p>
<figure>
<pre><code data-lang="r"><span>install.packages</span><span>(</span><span>'archivist.github'</span><span>)</span></code></pre>
</figure>
<p>I have prepared a workflow graph to visualize functionalities of archivist.github</p>
<p><img alt="" src="http://i0.wp.com/r-addict.com/images/fulls/archivist_github_workflow.png?w=456" data-recalc-dims="1"></p>
<p>and provide explanation of core powers in this post.</p>
<p>After you’ve created a GitHub developer application (the process is described at <a target="_blank" href="http://r-addict.com/archivist.github/#21_oauth_open_autorization">archivist.github: 2.1 OAuth open autorization</a>, set: Homepage URL – http://github.com, Authorization callback URL – http://localhost:1410) you will be able to automatically create repositories on GitHub from R console.<br>
Below is an example on how to authorise with GitHub API (using your application Client ID and Client Secret), create a GitHub repository with <a target="_blank" href="http://pbiecek.github.io/archivist/staticdocs/Repository.html">archivist-like Repository</a> and automatically archive R object on GitHub</p>
<figure>
<pre><code data-lang="r"><span># I saved some variables earlier # no to provide them publicly
</span><span>load</span><span>(</span><span>&quot;github_token.rda&quot;</span><span>);</span><span>load</span><span>(</span><span>&quot;password.rda&quot;</span><span>)</span>

<span>library</span><span>(</span><span>archivist.github</span><span>)</span>
<span># this can be done only in interactive mode
# so have to be done before knitr compilation
#
# authoriseGitHub(ClientID, ClientSecret,
# scope = c(&quot;public_repo&quot;, &quot;delete_repo&quot;)) ->
# github_token #
# repository creation
</span><span>createGitHubRepo</span><span>(</span><span>repo</span> <span>=</span> <span>&quot;RHero&quot;</span><span>,</span> 
                 <span>user</span> <span>=</span> <span>&quot;archivistR&quot;</span><span>,</span> 
                 <span>password</span> <span>=</span> <span>password</span><span>,</span>
                 <span>github_token</span> <span>=</span> <span>github_token</span><span>,</span>
                 <span>default</span> <span>=</span> <span>TRUE</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">[1] &quot;archivistR&quot;</code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span>#</span> <span>-></span> <span>https</span><span>://</span><span>github.com</span><span>/</span><span>archivistR</span><span>/</span><span>RHero</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span># parameters can be set globally,
# so you will not have to specify # them for each call
</span><span>aoptions</span><span>(</span><span>&quot;password&quot;</span><span>,</span> <span>password</span><span>)</span>
<span>aoptions</span><span>(</span><span>&quot;user&quot;</span><span>,</span> <span>&quot;archivistR&quot;</span><span>)</span> 
<span>aoptions</span><span>(</span><span>&quot;repo&quot;</span><span>,</span> <span>&quot;RHero&quot;</span><span>)</span>
<span>aoptions</span><span>(</span><span>&quot;github_token&quot;</span><span>,</span> <span>github_token</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span># archiving on GitHub
</span><span>archive</span><span>(</span><span>iris</span><span>,</span> <span>alink</span> <span>=</span> <span>TRUE</span><span>)</span></code></pre>
</figure>
<p><a target="_blank" href="https://raw.githubusercontent.com/archivistR/RHero/master/gallery/ff575c261c949d073b2895b05d1097c3.rda"><code>archivist::aread('archivistR/RHero/ff575c261c949d073b2895b05d1097c3')</code></a></p>
<p><img alt="" src="http://i0.wp.com/r-addict.com/images/fulls/github_R.jpg?w=456" data-recalc-dims="1"></p>
<p>One can check that the artifact is really on GitHub and that the <a target="_blank" href="https://github.com/archivistR/RHero/commits/master">commit was performed</a> (with great help of <a target="_blank" href="https://github.com/ropensci/git2r">git2r</a> package)</p>
<figure>
<pre><code data-lang="r"><span># sometimes GitHub need more # time to react
</span><span>Sys.sleep</span><span>(</span><span>60</span><span>)</span>
<span># show archived objects with their hashes
</span><span>archivist</span><span>::</span><span>showRemoteRepo</span><span>()</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">                           md5hash                             name         createdDate
1 ff575c261c949d073b2895b05d1097c3                             iris 2016-06-13 16:41:17
2 2e7b44a1845602a5e3a4898b618b4aa6 2e7b44a1845602a5e3a4898b618b4aa6 2016-06-13 16:41:17</code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span># one can check how many commits have been performed so far
</span><span>length</span><span>(</span>
   <span>jsonlite</span><span>::</span><span>fromJSON</span><span>(</span>
      <span>rawToChar</span><span>(</span>
<span>httr</span><span>::</span><span>GET</span><span>(</span><span>'https://api.github.com/repos/archivistR/RHero/commits'</span><span>)</span><span>$</span><span>content</span><span>)</span>
      <span>)</span><span>$</span><span>sha</span>
<span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">[1] 2</code></pre>
</figure>
<p>Each object (referred as <a target="_blank" href="http://pbiecek.github.io/archivist/staticdocs/archivist-package.html">artifact</a>) is archived with it’s metadata and <a target="_blank" href="http://pbiecek.github.io/archivist/staticdocs/md5hash.html">md5hash</a> in case someone would like to <a target="_blank" href="http://r-addict.com/archivist.github/#23_archiving_and_exploring_example">restore or search for archived objects</a> within <a target="_blank" href="http://pbiecek.github.io/archivist/staticdocs/Repository.html">Repository</a>.</p>
<h1>Partial results archiving and objects’ pedigree restoration</h1>
<p><a target="_blank" href="https://github.com/pbiecek/archivist/graphs/contributors">We</a> have prepared extended version of <a target="_blank" href="https://cran.r-project.org/web/packages/magrittr/vignettes/magrittr.html">pipe – %>%</a> operator <a target="_blank" href="http://pbiecek.github.io/archivist/staticdocs/magrittr.html">%a%</a> so that every partial result of analysis workflow can be archived. Below is an example of workflow archiving for <a target="_blank" href="http://rtcga.github.io/RTCGA/">RTCGA</a> (about which I wrote <a target="_blank" href="http://r-addict.com/2016/05/04/RTCGA-Quick-Guide.html">here</a>) RNASeq data (genes’ expression) (broader example can be find <a target="_blank" href="http://r-addict.com/archivist.github/#3_advanced_example_with_rtcga">here</a>) and it’s pedigree restoration</p>
<figure>
<pre><code data-lang="r"><span># This sets `silent=TRUE` in saveToRepo
# which is used by %a% . There will be # no warning printed about archiving # the same artifact or it's data twice.
</span><span>aoptions</span><span>(</span><span>'silent'</span><span>,</span> <span>TRUE</span><span>)</span>
<span>aoptions</span><span>(</span><span>'repoDir'</span><span>,</span> <span>'RHero'</span><span>)</span>
 <span># %a% archives to Local Repos, that's
</span><span>#</span> <span>why</span> <span>we</span> <span>need</span> <span>'repoDir'</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span>#
# information about genes' expressions
</span><span>library</span><span>(</span><span>RTCGA.rnaseq</span><span>);</span> <span>data</span><span>(</span><span>BRCA.rnaseq</span><span>)</span>
<span>library</span><span>(</span><span>dplyr</span><span>)</span>
<span>BRCA.rnaseq</span> <span>%</span><span>a</span><span>%</span>
    <span>select</span><span>(</span><span>`TP53|7157`</span><span>,</span> 
           <span>bcr_patient_barcode</span><span>)</span> <span>%</span><span>a</span><span>%</span>
    <span># bcr_patient_barcode contains a key to </span>    <span># merge patients between various datasets
</span>    <span>rename</span><span>(</span><span>TP53</span> <span>=</span> <span>`TP53|7157`</span><span>)</span> <span>%</span><span>a</span><span>%</span>
    <span>filter</span><span>(</span><span>substr</span><span>(</span><span>bcr_patient_barcode</span><span>,</span> 
                  <span>14</span><span>,</span> <span>15</span><span>)</span> <span>==</span> <span>&quot;01&quot;</span> <span>)</span> <span>-></span> 
    <span># 01 at the 14-15th position tells </span>    <span># these are cancer sample
</span>    <span>BRCA.rnaseq.TP53</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span># knitr: results='asis'.
# give hooks to objects
</span><span>ahistory</span><span>(</span><span>BRCA.rnaseq.TP53</span><span>,</span>
         <span>format</span> <span>=</span> <span>&quot;kable&quot;</span><span>,</span>
         <span>alink</span> <span>=</span> <span>TRUE</span> <span>)</span> </code></pre>
</figure>
<table>
<thead>
<tr>
<th></th>
<th>call</th>
<th>md5hash</th>
</tr>
</thead>
<tbody>
<tr>
<td>4</td>
<td>env[[nm]]</td>
<td><a target="_blank" href="https://raw.githubusercontent.com/archivistR/RHero/master/gallery/63678e012c5b7f40966c32eec91f828b.rda">63678e012c5b7f40966c32eec91f828b</a></td>
</tr>
<tr>
<td>3</td>
<td>select(<code>TP53&amp;#124;7157</code>, bcr_patient_barcode)</td>
<td><a target="_blank" href="https://raw.githubusercontent.com/archivistR/RHero/master/gallery/4a85ce61229dd743b911d7edab0310b3.rda">4a85ce61229dd743b911d7edab0310b3</a></td>
</tr>
<tr>
<td>2</td>
<td>rename(TP53 = <code>TP53&amp;#124;7157</code>)</td>
<td><a target="_blank" href="https://raw.githubusercontent.com/archivistR/RHero/master/gallery/103f2b82c41956e9f6437b3a0cd68679.rda">103f2b82c41956e9f6437b3a0cd68679</a></td>
</tr>
<tr>
<td>1</td>
<td>filter(substr(bcr_patient_barcode, 14, 15) == “01”)</td>
<td><a target="_blank" href="https://raw.githubusercontent.com/archivistR/RHero/master/gallery/1da5a026aae19e0d0467ba3773679e28.rda">1da5a026aae19e0d0467ba3773679e28</a></td>
</tr>
</tbody>
</table>
<figure>
<pre><code data-lang="r"><span># it uses global user and repo
</span><span>#</span> <span>if</span> <span>they</span> <span>are</span> <span>not</span> <span>specified</span></code></pre>
</figure>
<p>Column with <code>[[env]]</code> is the object before transformations. We are working on using original names for objects in <a target="_blank" href="https://github.com/pbiecek/archivist/issues/269">this issue</a>.</p>
<p>This operation does not archive objects automatically on GitHub as this is functionality from base archivist package. One have to upload objects with</p>
<figure>
<pre><code data-lang="r"><span>pushGitHubRepo</span><span>()</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span># one can check how many commits have been performed so far
</span><span>length</span><span>(</span>
   <span>jsonlite</span><span>::</span><span>fromJSON</span><span>(</span>
      <span>rawToChar</span><span>(</span>
<span>httr</span><span>::</span><span>GET</span><span>(</span><span>'https://api.github.com/repos/archivistR/RHero/commits'</span><span>)</span><span>$</span><span>content</span><span>)</span>
      <span>)</span><span>$</span><span>sha</span>
<span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">[1] 3</code></pre>
</figure>
<h1>Overload <code>print()</code> to use <code>archive()</code></h1>
<p>After global parameters specification (aoptions() function sets ‘user’, ‘repo’, and ‘password’ parameters for each archivist.github and archivist function globally) we don’t have to use <code>archive</code> function after each call to provide hooks in rmarkdown reports. We can overload <code>print()</code> function for specific classes so that after printing objects will be also evaluated with <code>archive</code> function.</p>
<figure>
<pre><code data-lang="r"><span>addHooksToPrintGitHub</span><span>(</span><span>class</span> <span>=</span> <span>&quot;lm&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span># knitr: results='asis'
</span><span>pld</span> <span>&lt;-</span> <span>aread</span><span>(</span><span>&quot;MarcinKosinski/Museum/04eb0bdc12&quot;</span><span>)</span>
<span>(</span><span>lm.D9</span> <span>&lt;-</span> <span>lm</span><span>(</span><span>weight</span> <span>~</span> <span>group</span><span>,</span>
             <span>data</span> <span>=</span> <span>pld</span><span>))</span></code></pre>
</figure>
<p>Load: <a target="_blank" href="https://raw.githubusercontent.com/archivistR/RHero/master/gallery/2b639023bc41e289aa21d790d5876736.rda"><code>archivist::aread('archivistR/RHero/2b639023bc41e289aa21d790d5876736')</code></a></p>
<p>Call:<br>
lm(formula = weight ~ group, data = pld)</p>
<p>Coefficients:<br>
(Intercept) groupTrt<br>
5.032 -0.371</p>
<figure>
<pre><code data-lang="r"><span>(</span><span>lm.D90</span> <span>&lt;-</span> <span>lm</span><span>(</span><span>weight</span> <span>~</span> <span>group</span> <span>-</span> <span>1</span><span>,</span>
              <span>data</span> <span>=</span> <span>pld</span><span>))</span></code></pre>
</figure>
<p>Load: <a target="_blank" href="https://raw.githubusercontent.com/archivistR/RHero/master/gallery/a33c804ff1d0b652210a39e2071d1e14.rda"><code>archivist::aread('archivistR/RHero/a33c804ff1d0b652210a39e2071d1e14')</code></a></p>
<p>Call:<br>
lm(formula = weight ~ group – 1, data = pld)</p>
<p>Coefficients:<br>
groupCtl groupTrt<br>
5.032 4.661</p>
<p>This is the GitHub equivalent for local archiving with <a target="_blank" href="http://pbiecek.github.io/archivist/staticdocs/addHooksToPrint.html">addHooksToPrint</a></p>
<h1><a target="_blank" href="http://marcinkosinski.github.io/archivist.github/#6_feedback_and_notes">Feedback and Notes</a></h1>
<p>If you have any comments or user request, please see <a target="_blank" href="http://marcinkosinski.github.io/archivist.github/#6_feedback_and_notes">Feedback and Notes</a> section to be aware of our future plans.</p>
<p>More examples can be checked at <a target="_blank" href="http://r-addict.com/archivist.github/">archivist.github Tutorial</a> or you can learn more during <a target="_blank" href="http://r-addict.com/2016/06/13/RHero-Saves-Backup-City-With-archivist-github.html">@pbiecek</a> talk <a target="_blank" href="http://schedule.user2016.org/event/7BYx/how-to-use-the-archivist-package-to-boost-reproducibility-of-your-research">How to use the archivist package to boost reproducibility of your research</a> at <a target="_blank" href="http://r-addict.com/2016/06/13/schedule.user2016.org/">useR2016 Conference</a>.</p>
<p>If you’d like to meet more R Heroes then restore message that was archived for commisar O’Rdon with</p>
<figure>
<pre><code data-lang="r"><span># library(archivist.github)
</span><span>cat</span><span>(</span><span>aread</span><span>(</span><span>&quot;MarcinKosinski/archivist.github/eRum2016/010e596&quot;</span><span>))</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">European R users meeting (eRum 2016) will take place between October 12th and 14th.

We already have confirmed great invited speakers such as: Rasmus Bååth, Romain François, Ulrike Grömping, Matthias Templ, and Heather Turner, as well as strong representation from Poland: Przemysław Biecek, Marek Gągolewski, Jakub Glinka, Katarzyna Kopczewska, and Katarzyna Stąpor. We are planning a meeting of more than 200 useRs from all across Europe working in different areas of the industry, academy, and government.

On behalf of organising committee, chaired by Maciej Beręsewicz, we want to invite you to be a part of this historical meeting by proposing a workshop, submitting a regular or lightning talk, presenting a poster, or just attending the activities we are preparing for the meeting.

You will find more details about the registration process on the website www.erum.ue.poznan.pl.

If you have any questions do not hesitate to ask through erum@konf.ue.poznan.pl.

See you in Poznań.

Source: http://www.r-bloggers.com/european-r-users-meeting-meeting-of-r-heroes-poznan-12-14-10-2016/</code></pre>
</figure>
<p><img alt="" src="http://i2.wp.com/r-addict.com/images/fulls/eRum.png?w=456" data-recalc-dims="1"></p>
<h1>Thanks</h1>
<p>Paintings were made by <a target="_blank" href="https://pedzlenie.github.io/">pedzlenie</a>.</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/r-hero-saves-backup-city-with-archivist-and-github/"></div></div><div><a data-url="http://www.r-bloggers.com/r-hero-saves-backup-city-with-archivist-and-github/" data-counturl="http://www.r-bloggers.com/r-hero-saves-backup-city-with-archivist-and-github/" data-count="vertical" data-text="R Hero saves Backup City with archivist and GitHub" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://r-addict.com/2016/06/13/RHero-Saves-Backup-City-With-archivist-github.html"> http://r-addict.com</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/Ri_ZiMZTuwA"></a></div>