<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/a-simple-way-to-set-up-a-sparklyr-cluster-on-azure/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2018/01/azure-sparklyr-aztk.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>The <a rel="nofollow" href="http://spark.rstudio.com/" target="_blank">SparklyR package</a> from RStudio provides a high-level interface to Spark from R. This means you can create R objects that point to data frames stored in the Spark cluster and apply some familiar R paradigms (like <a rel="nofollow" href="http://dplyr.tidyverse.org/" target="_blank">dplyr</a>) to the data, all the while leveraging Spark's distributed architecture without having to worry about memory limitations in R. You can also access the <a rel="nofollow" href="http://spark.apache.org/docs/latest/mllib-guide.html" target="_blank">distributed machine-learning algorithms</a> included in Spark directly from R functions. </p>
<p>If you don't happen to have a cluster of Spark-enabled machines set up in a nearby well-ventilated closet, you can easily set one up in your favorite cloud service. For Azure, one option is to launch a <a rel="nofollow" href="https://docs.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-overview" target="_blank">Spark cluster in HDInsight</a>, which also includes the extensions of <a rel="nofollow" href="https://docs.microsoft.com/en-us/azure/hdinsight/r-server/r-server-overview/?WT.mc_id=Revolutions-blog-davidsmi" target="_blank">Microsoft ML Server</a>. While this service recently had a <a rel="nofollow" href="https://azure.microsoft.com/en-us/blog/azure-hdinsight-announcements-significant-price-reduction-and-amazing-new-capabilities/?WT.mc_id=Revolutions-blog-davidsmi" target="_blank">significant price reduction</a>, it's still more expensive than running a &quot;vanilla&quot; Spark-and-R cluster. If you'd like to take the vanilla route, a new guide details <a rel="nofollow" href="https://github.com/Azure/aztk/wiki/SparklyR-on-Azure-with-AZTK" target="_blank">how to set up Spark cluster on Azure for use with SparklyR</a>.</p>
<p><a rel="nofollow" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb09e948c2970d-pi" target="_blank"><img border="0" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb09e948c2970d-800wi" alt="AZTK" title="AZTK"></a></p>
<p>All of the details are provided in the link below, but the guide basically provides the <a rel="nofollow" href="https://github.com/Azure/aztk/blob/master/docs/00-getting-started.md" target="_blank">Azure Distributed Data Engineering Toolkit</a> shell commands to <a rel="nofollow" href="https://github.com/Azure/aztk/blob/master/docs/10-clusters.md" target="_blank">provision a Spark cluster</a>, connect SparklyR to the cluster, and then interact with it via RStudio Server. This includes the ability to launch the cluster with <a rel="nofollow" href="https://docs.microsoft.com/en-us/azure/batch/batch-low-pri-vms/?WT.mc_id=Revolutions-blog-davidsmi" target="_blank">pre-emptable low-priority VMs</a>, a cost-effective option (up to 80% cheaper!) for non-critical workloads. Check out the details at the link below.</p>
<p>Github (Azure): <a rel="nofollow" href="https://github.com/Azure/aztk/blob/master/docs/00-getting-started.md" target="_blank">How to use SparklyR on Azure with AZTK</a></p>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2018/01/azure-sparklyr-aztk.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/Gr3ltzu34IE" height="1"></a></div>