<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/tutorial-using-r-for-scalable-data-analytics/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2017/03/tutorial-scaling-r.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>At the recent Strata conference in San Jose, several members of the Microsoft Data Science team presented the tutorial <a target="_blank" rel="nofollow" href="https://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/55806">Using R for Scalable Data Analytics: Single Machines to Spark Clusters</a>. The materials are all available online, including the <a target="_blank" rel="nofollow" href="https://cdn.oreillystatic.com/en/assets/1/event/193/Using%20R%20for%20scalable%20data%20analytics_%20From%20single%20machines%20to%20Hadoop%20Spark%20clusters%20Presentation.pdf">presentation slides</a> and <a target="_blank" rel="nofollow" href="https://github.com/Azure/Azure-MachineLearning-DataScience/tree/master/Misc/StrataSanJose2017">hands-on R scripts</a>. You can follow along with the materials at home, using the <a target="_blank" rel="nofollow" href="https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-data-science-linux-dsvm-intro">Data Science Virtual Machine for Linux</a>, which provides all the necessary components like Spark and Microsoft R Server. (If you don't already have an Azure account, you can get $200 credit with the <a target="_blank" rel="nofollow" href="https://azure.microsoft.com/en-us/offers/ms-azr-0044p/">Azure free trial</a>.)</p>
<p>The tutorial covers many different techniques for training predictive models at scale, and deploying the trained models as predictive engines within production environments. Among the technologies you'll use are <a target="_blank" rel="nofollow" href="https://github.com/Azure/Azure-MachineLearning-DataScience/tree/master/Misc/StrataSanJose2017/Code/MRS">Microsoft R Server running on Spark</a>, the <a target="_blank" rel="nofollow" href="https://github.com/Azure/Azure-MachineLearning-DataScience/tree/master/Misc/StrataSanJose2017/Code/SparkR">SparkR package</a>, the <a target="_blank" rel="nofollow" href="https://github.com/Azure/Azure-MachineLearning-DataScience/tree/master/Misc/StrataSanJose2017/Code/sparklyr">sparklyr package</a> and H20 (via the <a target="_blank" rel="nofollow" href="https://github.com/h2oai/rsparkling">rsparkling package</a>). It also touches on some non-Spark methods, like the <a target="_blank" rel="nofollow" href="https://github.com/Azure/Azure-MachineLearning-DataScience/blob/master/Misc/StrataSanJose2017/Code/bigmemory/bigmemory.R">bigmemory</a> and <a target="_blank" rel="nofollow" href="https://github.com/Azure/Azure-MachineLearning-DataScience/tree/master/Misc/StrataSanJose2017/Code/ff">ff</a> packages for R (and various other packages that make use of them), and <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2017/03/doazureparallel.html">using the foreach package for coarse-grained parallel computations</a>. You'll also learn how to create prediction engines from these trained models using the <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2017/03/running-your-r-code-azure.html">mrsdeploy package</a>.</p>
<p><a target="_blank" rel="nofollow" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b8d2720db6970c-pi"><img title="Mrsdeploy" alt="Mrsdeploy" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b8d2720db6970c-800wi" border="0"></a></p>
<p>The tutorial also includes <a target="_blank" rel="nofollow" href="https://github.com/Azure/Azure-MachineLearning-DataScience/tree/master/Misc/StrataSanJose2017/Performance_Comparison">scripts for comparing the performance</a> of these various techniques, both for training the predictive model:</p>
<p><a target="_blank" rel="nofollow" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c8e7b5ff970b-pi"><img title="Training" alt="Training" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c8e7b5ff970b-800wi" border="0"></a></p>
<p>and for generating predictions from the trained model:</p>
<p><a target="_blank" rel="nofollow" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c8e7b635970b-pi"><img title="Scoring" alt="Scoring" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c8e7b635970b-800wi" border="0"></a></p>
<p>(The above tests used 4 worker nodes and 1 edge node, all with with 16 cores and 112Gb of RAM.)</p>
<p>You can find the tutorial details, including slides and scripts, at the link below.</p>
<p>Strata + Hadoop World 2017, San Jose: <a target="_blank" rel="nofollow" href="https://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/55806">Using R for scalable data analytics: From single machines to Hadoop Spark clusters</a></p>
<p> </p>
<p> </p>
</div>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2017/03/tutorial-scaling-r.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/yV-JVJw965M"></a></div>