<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/remote-processing-using-futures/"></div></div><div><a data-url="https://www.r-bloggers.com/remote-processing-using-futures/" data-counturl="https://www.r-bloggers.com/remote-processing-using-futures/" data-count="horizontal" data-text="Remote Processing Using Futures" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.jottr.org/2016/10/remote-processing-using-futures.html"> jottR</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>A new version of the <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=future">future</a> package has been released and is available on CRAN. With futures, it is easy to <em>write R code once</em>, which later <em>the user can choose</em> to parallelize using whatever resources s/he has available, e.g. a local machine, a set of local notebooks, a set of remote machines, or a high-end compute cluster.</p>
<table align="center" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td><img src="https://i0.wp.com/3.bp.blogspot.com/-C4F2HQQYTmU/V_174RGSP2I/AAAAAAABQro/ot71_jFkMdAQd3UF5feN5sJMYnFrpbdlgCLcB/s320/early_days_video_call.jpg?w=456&ssl=1" border="0" data-recalc-dims="1"></td>
</tr>
<tr>
<td>The future provides comfortable and friendly long-distance interactions.</td>
</tr>
</tbody>
</table>
<p>The new version, future 1.1.1, provides:</p>
<ul>
<li>
<p><strong>Much easier usage of remote computers / clusters</strong></p>
<ul>
<li>If you can SSH to the machine, then you can also use it to resolve R expressions remotely.</li>
<li>Firewall configuration and port forwarding are no longer needed. </li>
</ul>
</li>
<li>
<p><strong>Improved identification of global variables</strong></p>
<ul>
<li>Corner cases where the package previously failed to identify and export global variables are now also handled. For instance, variable <code>x</code> is now properly identified as a global variable in expressions such as <code>x$a &lt;- 3</code> and <code>x[1,2,4] &lt;- 3</code> as well as in formulas such as <code>y ~ x | z</code>.</li>
<li>Global variables are by default identified automatically, but can now also be specified manually, either by their names (as a character vector) or by their names and values (as a named list).</li>
</ul>
</li>
</ul>
<p>For full details on updates, please see the <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=future">NEWS</a> file. The future package installs out-of-the-box on all operating systems.</p>
<h2>Example: Remote graphics rendered locally</h2>
<p>To illustrate how simple and powerful remote futures can be, I will show how to (i) set up locally stored data, (ii) generate <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=plotly">plotly</a>-enhanced <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=ggplot2">ggplot2</a> graphics based on these data using a remote machine, and then (iii) render these plotly graphics in the local web browser for interactive exploration of data.</p>
<p>Before starting, all we need to do is to verify that we have SSH access to the remote machine, let's call it <code>remote.server.org</code>, and that it has R installed:</p>
<pre><code>{local}: ssh remote.server.org<br>{remote}: Rscript --version<br>R scripting front-end version 3.3.1 (2016-06-21)<br>{remote}: exit<br>{local}: exit<br></code></pre>
<p>Note, it is highly recommended to use <a target="_blank" rel="nofollow" href="https://en.wikipedia.org/wiki/Secure_Shell#Key_management">SSH-key pair authentication</a> so that login credentials do not have to be entered manually.</p>
<p>After having made sure that the above works, we are ready for our remote future demo. The following code is based on an online <a target="_blank" rel="nofollow" href="https://plot.ly/ggplot2/">plotly example</a> where only a few minor modifications have been done:</p>
<pre><code>library(&quot;plotly&quot;)<br>library(&quot;future&quot;)<br><br>## %&lt;-% assignments will be resolved remotely<br>plan(remote, workers = &quot;remote.server.org&quot;)<br><br>## Set up data (locally)<br>set.seed(100)<br>d &lt;- diamonds[sample(nrow(diamonds), 1000), ]<br><br>## Generate ggplot2 graphics and plotly-fy (remotely)<br>gg %&lt;-% {<br>  p &lt;- ggplot(data = d, aes(x = carat, y = price)) +<br>         geom_point(aes(text = paste(&quot;Clarity:&quot;, clarity)), size = 4) +<br>         geom_smooth(aes(colour = cut, fill = cut)) + facet_wrap(~ cut)<br>  ggplotly(p)<br>}<br><br>## Display graphics in browser (locally)<br>gg<br></code></pre>
<p>The above renders the plotly-compiled ggplot2 graphics in our local browser. See below screenshot for an example.</p>
<p>This might sound like magic, but all that is going behind the scenes is a carefully engineered utilization of the <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=globals">globals</a> and the parallel packages, which is then encapsulated into the unified API provided by the future package. First, a future assignment (<code>%&lt;-%</code>) is used for <code>gg</code>, instead of a regular assignment (<code>&lt;-</code>). That tells R to use a future to evaluate the expression on the right-hand side (everything within <code>{ ... }</code>). Second, since we specified that we want to use the remote machine <code>remote.server.org</code> to resolve our futures, that is where the future expression is evaluated. Third, necessary data is automatically communicated between our local and remote machines. That is, any global variables (<code>d</code>) and functions are automatically identified and exported to the remote machine and required packages (<code>ggplot2</code> and <code>plotly</code>) are loaded remotely. When resolved, the value of the expression is automatically transferred back to our local machine afterward and is available as the value of future variable <code>gg</code>, which was formally set up as a promise. </p>
<table align="center" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td><img src="https://i0.wp.com/1.bp.blogspot.com/-Ghf5LwcKuvA/V_18txv9UkI/AAAAAAABQrw/dom1CUcNRJQYrCrXv5qKaD5OcvP2d8iMwCLcB/s1600/future_1.1.1-example_plotly.png?w=456&ssl=1" border="0" data-recalc-dims="1"></td>
</tr>
<tr>
<td>An example of remote futures: This ggplot2 + plotly figure was generated on a remote machine and then rendered in the local web browser where it is can be interacted with dynamically.</td>
</tr>
</tbody>
</table>
<p><em>What's next?</em> Over the summer, I have received tremendous feedback from several people, such as (in no particular order) <a target="_blank" rel="nofollow" href="https://github.com/krlmlr">Kirill Müller</a>, <a target="_blank" rel="nofollow" href="https://github.com/gdevailly">Guillaume Devailly</a>, <a target="_blank" rel="nofollow" href="https://github.com/clarkfitzg">Clark Fitzgerald</a>, <a target="_blank" rel="nofollow" href="https://github.com/michaelsbradleyjr">Michael Bradley</a>, <a target="_blank" rel="nofollow" href="https://github.com/thomasp85">Thomas Lin Pedersen</a>, <a target="_blank" rel="nofollow" href="https://github.com/alexvorobiev">Alex Vorobiev</a>, <a target="_blank" rel="nofollow" href="https://github.com/hrbrmstr">Bob Rudis</a>, <a target="_blank" rel="nofollow" href="https://github.com/RebelionTheGrey">RebelionTheGrey</a>, <a target="_blank" rel="nofollow" href="https://github.com/wrathematics">Drew Schmidt</a> and <a target="_blank" rel="nofollow" href="https://github.com/gaborcsardi">Gábor Csárdi</a> (sorry if I missed anyone, please let me know). This feedback contributed to some of the new features found in future 1.1.1. However, there're many great <a target="_blank" rel="nofollow" href="https://github.com/HenrikBengtsson/future/issues">suggestions and wishes</a> that didn't make it in for this release – I hope to be able to work on those next. Thank you all.</p>
<p>Happy futuring!</p>
<h2>Links</h2>
<ul>
<li>future package:
<ul>
<li>CRAN page: <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=future">https://cran.r-project.org/package=future</a></li>
<li>GitHub page: <a target="_blank" rel="nofollow" href="https://github.com/HenrikBengtsson/future">https://github.com/HenrikBengtsson/future</a></li>
</ul>
</li>
<li>future.BatchJobs package:
<ul>
<li>CRAN page: <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=future.BatchJobs">https://cran.r-project.org/package=future.BatchJobs</a></li>
<li>GitHub page: <a target="_blank" rel="nofollow" href="https://github.com/HenrikBengtsson/future.BatchJobs">https://github.com/HenrikBengtsson/future.BatchJobs</a></li>
</ul>
</li>
<li>doFuture package:
<ul>
<li>CRAN page: <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=doFuture">https://cran.r-project.org/package=doFuture</a></li>
<li>GitHub page: <a target="_blank" rel="nofollow" href="https://github.com/HenrikBengtsson/doFuture">https://github.com/HenrikBengtsson/doFuture</a></li>
</ul>
</li>
</ul>
<h2>See also</h2>
<ul>
<li><a target="_blank" rel="nofollow" href="http://www.jottr.org/2016/07/a-future-for-r-slides-from-user-2016.html">A Future for R: Slides from useR 2016</a></li>
</ul>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/remote-processing-using-futures/"></div></div><div><a data-url="https://www.r-bloggers.com/remote-processing-using-futures/" data-counturl="https://www.r-bloggers.com/remote-processing-using-futures/" data-count="vertical" data-text="Remote Processing Using Futures" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.jottr.org/2016/10/remote-processing-using-futures.html"> jottR</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/dvw6_YdEjhg"></a></div>