<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/13-much-faster-package-re-installation-via-binaries/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://dirk.eddelbuettel.com/blog/2017/12/13#013_faster_package_installation_two"> Thinking inside the box </a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Welcome to the thirteenth post in the <em>ridiculously rapid R recommendation</em> series, or <span><em>R</em><sup>4</sup></span> for short. A few days ago we <a rel="nofollow" href="http://dirk.eddelbuettel.com/blog/2017/11/27#011_faster_package_installation_one" target="_blank">riffed on faster installation thanks to ccache</a>. Today we show another way to get equally drastic gains for some (if not most) packages.</p>
<p>In a nutshell, there are two ways to get your R packages off <a rel="nofollow" href="https://cran.r-project.org/" target="_blank">CRAN</a>. Either you install as a binary, or you use source. Most people do not think too much about this as on Windows, binary is the default. So why wouldn’t one? Precisely. (Unless you are on Windows, and you develop, or debug, or test, or … and need source. Another story.) On other operating systems, however, source is the rule, and binary is often unavailable.</p>
<p>Or is it? Exactly how to find out what <em>is</em> available will be left for another post as we <em>do</em> have a tool just for that. But today, just hear me out when I say that <em>binary is often an option even when source is the default</em>. And it matters. See below.</p>
<p>As a (mostly-to-always) Linux user, I sometimes whistle between my teeth that we &quot;lost all those battles&quot; (<em>i.e.</em> for the desktop(s) or laptop(s)) but &quot;won the war&quot;. That topic merits a longer post I hope to write one day, and I won’t do it justice today but my main gist that <em>everybody</em> (and here I mean mostly developers/power users) now at least <em>also</em> runs on Linux. And by that I mean that we <em>all</em> test our code in Linux environments such as <em>e.g.</em> <a rel="nofollow" href="https://travis-ci.org/" target="_blank">Travis CI</a>, and that many of us run deployments on cloud instances (AWS, GCE, Azure, …) which are predominantly based on Linux. Or on local clusters. Or, if one may dream, the <a rel="nofollow" href="https://www.top500.org/" target="_blank">top500</a> And on and on. And frequently these are Ubuntu machines.</p>
<p>So here is an Ubuntu trick: Install from binary, and save loads of time. As an illustration, consider the chart below. It carries over the logic from <a rel="nofollow" href="http://dirk.eddelbuettel.com/blog/2017/11/27#011_faster_package_installation_one" target="_blank">the ‘cached vs non-cached’ compilation post</a> and contrasts two ways of installing: from source, or as a binary. I use pristine and empty Docker containers as the base, and rely of course on the official <a rel="nofollow" href="https://hub.docker.com/_/r-base/" target="_blank">r-base image</a> which is supplied by <a rel="nofollow" href="https://www.carlboettiger.info/" target="_blank">Carl Boettiger</a> and yours truly as part of our <a rel="nofollow" href="https://www.rocker-project.org/" target="_blank">Rocker Project</a> (and for which we have a <a rel="nofollow" href="http://rjournal.github.io/archive/2017/RJ-2017-065/index.html" target="_blank">forthcoming R Journal piece</a> I might mention). So for example the timings for the <code>ggplot2</code> installation were obtained via</p>
<pre><code>time docker run --rm -ti r-base  /bin/bash -c 'install.r ggplot2'</code></pre>
<p>and</p>
<pre><code>time docker run --rm -ti r-base  /bin/bash -c 'apt-get update &amp;&amp; apt-get install -y r-cran-ggplot2'</code></pre>
<p>Here <code>docker run --rm -ti</code> just means to launch Docker, in ‘remove leftovers at end’ mode, use terminal and interactive mode and invoke a shell. The shell command then is, respectively, to install a <a rel="nofollow" href="https://cran.r-project.org/" target="_blank">CRAN</a> package using <code>install.r</code> from my <a rel="nofollow" href="http://dirk.eddelbuettel.com/code/littler.html" target="_blank">littler</a> package, or to install the binary via <code>apt-get</code> after updating the apt indices (as the Docker container may have been built a few days or more ago).</p>
<p>Let’s not focus on Docker here—it is just a convenient means to an end of efficiently <em>measuring</em> via a simple (wall-clock counting) <code>time</code> invocation. The key really is that <code>install.r</code> is just a wrapper to <code>install.packages()</code> meaning <em>source</em> installation on Linux (as used inside the Docker container). And <code>apt-get install ...</code> is how one gets a binary. Again, I will try post another piece to determine how one finds <em>if</em> a suitable binary for a CRAN package exists. For now, just allow me to proceed.</p>
<p>So what do we see then? Well have a look:</p>
<div>
<img src="https://i1.wp.com/dirk.eddelbuettel.com/blog/2017/12/13/binary_vs_source_impact.png?w=456" data-recalc-dims="1">
</div>
<p>A few things stick out. <a rel="nofollow" href="http://dirk.eddelbuettel.com/code/rquantlib.html" target="_blank">RQuantLib</a> really is a monster. And <a rel="nofollow" href="http://dplyr.tidyverse.org/" target="_blank">dplyr</a> is also fairly heavy—both rely on <a rel="nofollow" href="http://dirk.eddelbuettel.com/code/rcpp.html" target="_blank">Rcpp</a>, <a rel="nofollow" href="http://dirk.eddelbuettel.com/code/bh.html" target="_blank">BH</a> and lots of templating. At the other end, <a rel="nofollow" href="https://github.com/Rdatatable/data.table/wiki" target="_blank">data.table</a> is still a marvel. No external dependencies, and just plain C code make the source installation <em>essentially</em> the same speed as the binary installation. Amazing. But I digress.</p>
<p>We should add that one of the source installations also required installing additional libries: QuantLib is needed along with Boost for <a rel="nofollow" href="http://dirk.eddelbuettel.com/code/rquantlib.html" target="_blank">RQuantLib</a>. Similar for another package (not shown) which needed curl and libcurl.</p>
<p>So what is the upshot? <em>If you can, consider binaries.</em> I will try to write another post how I do that <em>e.g.</em> for <a rel="nofollow" href="https://travis-ci.org/" target="_blank">Travis CI</a> where all my tests us binaries. (Yes, I know. This mattered more in the past when they did not cache. It still matters today as you a) do not need to fill the cache in the first place and b) do not need to worry about details concerning compilation from source which still throws enough people off. But yes, you can of course survive as is.)</p>
<p>The same approach is equally valid on AWS and related instances: I answered <em>many</em> <a rel="nofollow" href="https://stackoverflow.com/" target="_blank">StackOverflow</a> questions where folks were failing to compile &quot;large-enough&quot; pieces from source on minimal installations with minimal RAM, and running out of resources and failed with bizarre errors. In short: <em>Don’t</em>. Consider binaries. It saves time and trouble.</p>
<p>
This post by <a rel="nofollow" href="http://dirk.eddelbuettel.com/" target="_blank">Dirk Eddelbuettel</a> originated on his <a rel="nofollow" href="http://dirk.eddelbuettel.com/blog/" target="_blank">Thinking inside the box</a> blog. Please report excessive re-aggregation in third-party for-profit settings.</p>
<p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://dirk.eddelbuettel.com/blog/2017/12/13#013_faster_package_installation_two"> Thinking inside the box </a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/0DSs_fbyNy4" height="1"></a></div>