<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/improving-automatic-document-production-with-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://itsalocke.com/improving-automatic-document-production-with-r/"> R – Locke Data</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div></div>
<p>In this post, I describe the latest iteration of my automatic document production with R. It improves upon the methods used in <a target="_blank" rel="nofollow" href="https://github.com/stephlocke/Rtraining">Rtraining</a>, and previous work on this topic can read by going to the <a target="_blank" rel="nofollow" href="https://itsalocke.com/tag/auto-deploying-r-documentation/">auto deploying R documentation</a> tag.</p>
<p>I keep banging on about this area because reproducible research / analytical document pipelines is an area I’ve a keen interest in. I see it as a core part of <a target="_blank" rel="nofollow" href="https://itsalocke.com/dataops-its-a-thing-honest/">DataOps</a> as it’s vital for helping us ensure our models and analysis are correct in data science and boosting our productivity.</p>
<p>Even after (or because of) a few years of off and on again development to the process, Rtraining had a number of issues:</p>
<ul>
<li>build time was long because all presentations and their respective dependencies were required</li>
<li>if a single presentation broke, any later presentations would not get generated</li>
<li>the presentation build step was in “after_success” so didn’t trigger notifications</li>
<li>the build script for the presentations did a lot of stuff I thought could be removed</li>
<li>the dynamic <a target="_blank" rel="nofollow" href="https://stephlocke.info/Rtraining/">index page</a> sucks</li>
</ul>
<p>This post covers how I’m attempting to fix all bar the last problem (more on that in a later post).</p>
<p>With the problems outlined, let’s look at my new <a target="_blank" rel="nofollow" href="https://github.com/lockedata/pres-stub">base solution</a> and how it addresses these issues.</p>
<h2>Structure</h2>
<p>I have built a template that can be used to generate multiple presentations and publish them to a <code>docs/</code> directory for online hosting by GitHub. I can now use this template to produce category repositories, based on the folders in <code>inst/slides/</code> in Rtraining. I can always split them out further at a later date.</p>
<p>The new repo is structured like so:</p>
<ul>
<li>Package infrastructure
<ul>
<li><code>DESCRIPTION</code> – used for managing dependencies primarily</li>
<li><code>R/</code> – store any utility functions</li>
<li><code>.Rbuildignore</code> – avoid non-package stuff getting checked</li>
</ul>
</li>
<li>Presentations
<ul>
<li><code>pres/</code> – directory for storing presentation .Rmd files</li>
<li><code>pres/_output.yml</code> – file with render preferences</li>
</ul>
</li>
<li>Output directory
<ul>
<li><code>docs/</code> – directory for putting generated slides in</li>
</ul>
</li>
<li>Document generation infrastructure
<ul>
<li><code>.travis.yml</code> – used to generate documents every time we push a commit</li>
<li><code>buildpres.sh</code> – shell script doing the git workflow and calling R</li>
<li><code>buildpres.R</code> – R script that performs the render step</li>
</ul>
</li>
</ul>
<h2>Presentations</h2>
<ul>
<li>My Rtraining repo contained all presentations in the <code>inst/slidedecks/</code> directory with further categories. This meant that if someone installed Rtraining, they’d get all the decks. I think this is a sub-optimal experience for people, especially because it mean installing so many packages, and I’ll be focusing instead on improving the web delivery.</li>
<li>Render requirements are now stored in an <code>_output.yml</code> file instead of being hard coded into the render step so that I can add more variant later</li>
<li>I’m currently using a modified version of the <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=revealjs"><code>revealjs</code></a> package as I’ve built a heavily customised theme. As I’m not planning on any of these presentation packages ever going on CRAN, I can use the <code>Remotes</code> field in <code>DESCRIPTION</code> to describe the location. This simplifies the code significantly.</li>
</ul>
<h2>Document generation</h2>
<figure class="wp-caption"><img sizes="(max-width: 485px) 100vw, 485px" alt="Automatic document generation with R" class="wp-image-62216" src="https://i1.wp.com/itsalocke.com/wp-content/uploads/2017/05/processflow.png?w=450" data-recalc-dims="1"><br>
<figcaption class="wp-caption-text">Automatic document generation with R</figcaption>
</figure>
<h3>Travis</h3>
<p>I use <a target="_blank" rel="nofollow" href="https://travis-ci.org/">travis-ci</a> to perform the presentation builds. The instructions I provide travis are:</p>
<pre><code>language: r<br>
cache: packages<br>
latex: false<br>
warnings_are_errors: false<br>
install: <br>
  - R -e 'install.packages(&quot;devtools&quot;)'<br>
  - R -e 'devtools::install_deps(dep = T)'<br>
  - R CMD build --no-build-vignettes --no-manual .<br>
  - R CMD check --no-build-vignettes --no-manual  *tar.gz<br>
  - Rscript -e 'devtools::install(pkg = &quot;.&quot;)'<br>
before_script:<br>
  - chmod +x ./buildpres.sh<br>
script:<br>
  - ./buildpres.sh<br>

</code></pre>
<p>One important thing to note here is that I used some arguments on my package build and check steps along with <code>latex: false</code> to drastically reduce the build time as I have no intention of producing PDFs normally.</p>
<p>The <code>install</code> section is the prep work, and then the <code>script</code> section does the important bit. Now if there are errors, I’ll get notified!</p>
<h3>Bash</h3>
<p>The script that gets executed in my Travis build:</p>
<ul>
<li>changes over to a GITHUB_PAT based connection to the repo to facilitate pushing changes and does some other config</li>
<li>executes the R render step</li>
<li>puts the R execution log in <code>docs/</code> for debugging</li>
<li>commits all the changes using the important prefix <code>[CI SKIP]</code> so we don’t get infinite build loops</li>
<li>pushes the changes</li>
</ul>
<pre><code>#!/bin/bash<br>
AUTHORNAME=&quot;Steph&quot;<br>
AUTHOREMAIL=&quot;Steph@itsalocke.com&quot;<br>
GITURL=&quot;https://$GITHUB_PAT@github.com/$TRAVIS_REPO_SLUG.git&quot;<br>
<br>
git remote set-url origin $GITURL<br>
git pull<br>
git checkout master<br>
git config --global user.name $AUTHORNAME<br>
git config --global user.email $AUTHOREMAIL<br>
<br>
R CMD BATCH './buildpres.R'<br>
<br>
cp buildpres.Rout docs/<br>
<br>
git add docs/<br>
git commit -am &quot;[ci skip] Documents produced in clean environment via Travis $TRAVIS_BUILD_NUMBER&quot;<br>
git push -u  --quiet origin master<br>

</code></pre>
<h3>R</h3>
<p>The R step is now very minimal in that it works out what presentations to generate, then loops through them and builds each one according to the options specified in <code>_output.yml</code></p>
<pre><code>library(rmarkdown)<br>
slides=list.files(&quot;pres&quot;,&quot;*.Rmd&quot;,full.names=TRUE)<br>
<br>
for (f in slides) render(f,output_dir = &quot;docs&quot;)<br>

</code></pre>
<h2>Next steps for me</h2>
<p>This work has substantially mitigated most of the issues I had with my previous Rtraining workflow. I now have to get all my slide decks building under this new process.</p>
<p>I will be writing about making an improved presentation portal and how to build and maintain your own substantially modified revealjs theme at a later date.</p>
<p>The modified workflow and scripts also have implications on my <a target="_blank" rel="nofollow" href="https://github.com/lockedata/pRojects">pRojects</a> package that I’m currently developing along with <a target="_blank" rel="nofollow" href="https://twitter.com/jonmcalder">Jon Calder</a>. I’d be very interested to hear from you if you have thoughts on how to make things more streamlined.</p>
<p>The post <a target="_blank" rel="nofollow" href="http://itsalocke.com/improving-automatic-document-production-with-r/">Improving automatic document production with R</a> appeared first on <a target="_blank" rel="nofollow" href="http://itsalocke.com/">Locke Data</a>. Locke Data are a data science consultancy aimed at helping organisations get ready and get started with data science.</p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://itsalocke.com/improving-automatic-document-production-with-r/"> R – Locke Data</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/dWSQctYijmc"></a></div>