<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/data-manipulation-with-sparklyr-on-azure-hdinsight/"></div></div><div><a data-url="https://www.r-bloggers.com/data-manipulation-with-sparklyr-on-azure-hdinsight/" data-counturl="https://www.r-bloggers.com/data-manipulation-with-sparklyr-on-azure-hdinsight/" data-count="horizontal" data-text="Data Manipulation with sparklyr on Azure HDInsight" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2016/11/data-manipulation-with-sparklyr-on-azure-hdinsight.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>by Ali Zaidi, Data Scientist at Microsoft</p>
<h1>Apache Spark and a Tale of APIs</h1>
<p>Spark is an exceptionally popular processing engine for distributed data. Dealing with data in distributed storage and programming with concurrent systems often requires learning complicated new paradigms and techniques. Statisticans and data scientists familiar wtih R are unlikely to have much experience with such systems. Fortunately, Spark is a very intuitive distributed computing platform, and there are APIs available in R and Python that are more familiar to the average data scientist than scala. </p>
<p>One of the most intriguing APIs available for Spark for R users is the <a target="_blank" rel="nofollow" href="https://github.com/rstudio/sparkapi"><code>sparkapi package</code></a> developed by RStudio and the associated <a target="_blank" rel="nofollow" href="http://spark.rstudio.com/"><code>sparklyr</code></a> project, which provides a very similar <code>dplyr</code> syntax for manipulating Spark DataFrames.</p>
<h2>Azure HDInsight</h2>
<p>Azure HDInsight is the Microsoft developed Apache Hadoop distribution for the cloud. The Hadoop ecosystem is a wild zoo of components, and configuring them together can be a difficult job. Microsoft’s Azure HDInsight service gives data scientists and analysts the most exciting capabilities of distributed computing on the cloud without the hassle of configuration and management. In this post, I will show how we can utilize premium Spark clusters with a complete installation of Microsoft R’s stack for doing data manipulation at scale.</p>
<h3>Provisioning</h3>
<p>To provision Azure HDInsight Spark clusters with Microsoft R Server, follow the instructions on the <a target="_blank" rel="nofollow" href="https://azure.microsoft.com/en-us/documentation/articles/hdinsight-hadoop-r-server-get-started/">Azure Documentation Page</a>, or my <a target="_blank" rel="nofollow" href="https://github.com/akzaidi/R-cadence/tree/master/Spark">Spark course repository</a>. To create a cluster, simply navigate to your portal and select a new HDInsight cluster:</p>
<p><a target="_blank" rel="nofollow" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c8ad5435970b-pi"><img title="Portal-blade3" alt="Portal-blade3" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c8ad5435970b-800wi" border="0"></a></p>
<p>On the Azure portal page, you will be asked to select a subscription, and also provide some security credentials to access your cluster via a secure terminal connection (SSH).</p>
<p>Make sure you select the premium option to get the awesome features of Microsoft R Server:</p>
<p><a target="_blank" rel="nofollow" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb095012b8970d-pi"><img title="Premiumspark" alt="Premiumspark" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb095012b8970d-800wi" border="0"></a></p>
<p>A follow up blog post will show you how to develop scalable machine learning models using the Microsoft R Server machine learning libraries.</p>
<h2>Spark DataFrames and Spark SQL</h2>
<p>In order to accommodate the variety of workloads a data scientist encounters during her work, Spark contains four core pieces aimed at a specific use-case:</p>
<p><a target="_blank" rel="nofollow" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c8ad173f970b-pi"><img title="Spark-core" alt="Spark-core" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c8ad173f970b-800wi" border="0"></a></p>
<p>The component aimed at data manipulation workloads is the <a target="_blank" rel="nofollow" href="http://spark.apache.org/docs/latest/sql-programming-guide.html">Spark SQL component</a>. Inside the Spark SQL context, Spark developers create DataFrames, which are a schematic versions of RDDs (Resilient Distributed Datasets). RDDs are the standard primitive of data in the Spark ecosystem, and can be thought of as distributed lists. However, just as lists are somewhat too general for the workloads necessitated by data manipulation in R, RDDs are also too general for data analysis workloads in Spark. By injecting a schema of columns (type schema and equality of rows), DataFrames are born. </p>
<h3>Functional Programming and the Art of Being Lazy</h3>
<p>The majority of Spark is written in Scala (~80% of Spark core), which is a functional programming language. Functional programming languages emphasize functional purity (the output only depends on the inputs) and strive to avoid side-effects. One important component of most functional programming languages is their lazy evaluation. While it might seem odd that we would appreciate laziness from our computing tools, lazy evaluation is an effective way of ensuring computations are evaluated in the most efficient manner possible. </p>
<p>Lazy evaluation allows Spark SQL to highly optimize the queries. When a user submits a query to Spark SQL, Spark composes the components of the SQL query into a logical plan. The logical plan is basically a recipe Spark SQL creates in order to evaluate the desired query. Spark SQL then submits the logical plan to its highly optimized engine called <a target="_blank" rel="nofollow" href="https://databricks.com/blog/2015/04/13/deep-dive-into-spark-sqls-catalyst-optimizer.html">Catalyst</a>, which optimizes this plan into a physical plan of action that is executed inside Spark computation engine (a series of coordinating JVMs). </p>
<h2>Spark SQL and <code>dplyr</code></h2>
<p>Some of the more R-enthusiastic readers might already see where this is going. Since Spark SQL is Spark’s core component of doing data analysis, why can’t we just use our favorite SQL-like paradigm in R to connect to Spark as an inteface? For many an R user, the only package that brings up memories of love and joy for data analysis is <a target="_blank" rel="nofollow" href="https://github.com/hadley/dplyr"><code>dplyr</code></a>.</p>
<p>More than just being an awesome package with a simple to understand <em>grammar</em>, <code>dplyr</code> is highly extensible and works hand-in-hand with SQL. It supports working with many SQL databases by utilizing that database engine as it’s backend, and converting/translating all of it’s verbs into SQL statements. For more information on working with databases using <code>dplyr</code>, see the package’s <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/dplyr/vignettes/databases.html">vignette on databases</a>. Take a look at <code>dplyr</code>‘s <code>translate_sql</code> function to see how it converts <code>dplyr</code> verbs into SQL statements. </p>
<p>The <code>sparklyr</code> package takes advantage of dplyr’s SQL connectivity and R’s <a target="_blank" rel="nofollow" href="http://adv-r.had.co.nz/Functions.html#lazy-evaluation">lazy evaluation procedures</a> by sending all the <code>dplyr</code> operations to Spark SQL. Therefore, as an R user, you don’t need to write anything but <code>dplyr</code> to get the awesomeness of Spark SQL and Catalyst!</p>
<h2>Spark Context</h2>
<p>In order to start using Spark as our computation engine, we need to define a Spark context (or Spark Session). We will use the <code>sparklyr</code> function <code>spark_connect</code> to create a Spark SQL context. There are a slew of arguments you can use to modify the parameters of your Spark SQL context, but we will stick to the defaults. By specifying <code>yarn-client</code> as our master, we are telling Spark we want a yarn client application. Full an overview of the environments on which you can run spark, take a look <a target="_blank" rel="nofollow" href="http://spark.apache.org/docs/latest/running-on-yarn.html">at the documentation page here</a>.</p>
<p>
<div>
<pre>library(sparklyr)
sc <span>&lt;-</span> spark_connect(<span>&quot;yarn-client&quot;</span>)
</pre>
</div>
<h2>Import Into Spark DataFrames</h2>
<p>In my cluster’s storage account on Azure HDInsight, I have loaded data from <a target="_blank" rel="nofollow" href="http://www.freddiemac.com/news/finance/sf_loanlevel_dataset.html">Freddie Mac’s single family loan database</a>. I then use the <code>spark_read_csv</code> function to read my folder of mortgage originations data into a Spark DataFrame. </p>
<p>
<div>
<pre>origins <span>&lt;-</span> file.path(<span>&quot;wasb://mrs-spark@alizaidi.blob.core.windows.net&quot;</span>,
                     <span>&quot;user/RevoShare/alizaidi/Freddie/Acquisition&quot;</span>)
freddie_origins <span>&lt;-</span> spark_read_csv(sc,
                                  path <span>=</span> origins,
                                  name <span>=</span> <span>'freddie_origins'</span>,
                                  header <span>=</span> <span>FALSE</span>,
                                  delimiter <span>=</span> <span>&quot;|&quot;</span>
                                  )
</pre>
</div>
<h2>Using <code>dplyr</code> to Query our Spark DataFrame</h2>
<p>The resulting object from the above operation is a Spark DataFrame. If we peek at it’s class, we notice that in addition to being a <code>tbl_spark</code> object, it is also of class <code>tbl_sql</code>, so all <code>dplyr</code> methods are converted to <code>Spark SQL</code> statements and run on the spark application defined through the spark context <code>sc</code>.</p>
<p>
<div>
<pre>class(freddie_origins)
</pre>
</div>
<p>
<div>
<pre> ## [1] &quot;tbl_spark&quot; &quot;tbl_sql&quot;   &quot;tbl_lazy&quot;  &quot;tbl&quot;
</pre>
</div>
<p>
<div>
<pre>library(dplyr)
freddie_origins <span>%>%</span> head
</pre>
</div>
<p>
<div>
<pre> ## Source:   query [?? x 25]
 ## Database: spark connection master=yarn-client app=sparklyr local=FALSE
 ## 
 ##      V1     V2    V3     V4    V5    V6    V7    V8    V9   V10    V11
 ##   &lt;chr>  &lt;int> &lt;chr>  &lt;int> &lt;int> &lt;chr> &lt;int> &lt;chr> &lt;dbl> &lt;chr>  &lt;int>
 ## 1   751 199910     N 202909    NA   000     1     O    71    20 180000
 ## 2   733 199909     N 202908 29540   000     1     O    51       116000
 ## 3   755 199905     N 202904 29540    30     1     O    95    38 138000
 ## 4   669 200206     N 202901    NA   000     1     O    80    33 162000
 ## 5   732 199904     N 202903 17140   000     1     O    25    10  53000
 ## 6   715 199904     N 202903 17140   000     1     O    67    35  91000
 ## # ... with 14 more variables: V12 &lt;int>, V13 &lt;dbl>, V14 &lt;chr>, V15 &lt;chr>,
 ## #   V16 &lt;chr>, V17 &lt;chr>, V18 &lt;chr>, V19 &lt;int>, V20 &lt;chr>, V21 &lt;chr>,
 ## #   V22 &lt;int>, V23 &lt;int>, V24 &lt;chr>, V25 &lt;chr>
</pre>
</div>
<p>Therefore, we can do any our common <code>dplyr</code> operations directly on this Spark DataFrame, and through the magic of method dispatch in R, the operations are converted to SQL and sent to Catalyst for evaluation in Spark SQL.</p>
<p>
<div>
<pre>freddie_rename <span>&lt;-</span> freddie_origins <span>%>%</span> rename(
                          credit_score <span>=</span> V1,
                          first_payment <span>=</span> V2,
                          first_home <span>=</span> V3,
                          maturity <span>=</span> V4,
                          msa <span>=</span> V5,
                          mi_perc <span>=</span> V6,
                          num_units <span>=</span> V7,
                          occ_status <span>=</span> V8,
                          cltv <span>=</span> V9,
                          dti <span>=</span> V10,
                          upb <span>=</span> V11,
                          ltv <span>=</span> V12,
                          orig_rate <span>=</span> V13,
                          channel <span>=</span> V14,
                          ppm <span>=</span> V15,
                          prod_type <span>=</span> V16,
                          state <span>=</span> V17,
                          prop_type <span>=</span> V18,
                          post_code <span>=</span> V19,
                          loan_number <span>=</span> V20,
                          loan_purpose <span>=</span> V21,
                          orig_term <span>=</span> V22,
                          num_borrowers <span>=</span> V23,
                          seller <span>=</span> V24,
                          servicer <span>=</span> V25
                          )
freddie_rename <span>%>%</span> head
</pre>
</div>
<p>
<div>
<pre>
 ## Source:   query [?? x 25]
 ## Database: spark connection master=yarn-client app=sparklyr local=FALSE
 ## 
 ##   credit_score first_payment first_home maturity   msa mi_perc num_units
 ##          &lt;chr>         &lt;int>      &lt;chr>    &lt;int> &lt;int>   &lt;chr>     &lt;int>
 ## 1          751        199910          N   202909    NA     000         1
 ## 2          733        199909          N   202908 29540     000         1
 ## 3          755        199905          N   202904 29540      30         1
 ## 4          669        200206          N   202901    NA     000         1
 ## 5          732        199904          N   202903 17140     000         1
 ## 6          715        199904          N   202903 17140     000         1
 ## # ... with 18 more variables: occ_status &lt;chr>, cltv &lt;dbl>, dti &lt;chr>,
 ## #   upb &lt;int>, ltv &lt;int>, orig_rate &lt;dbl>, channel &lt;chr>, ppm &lt;chr>,
 ## #   prod_type &lt;chr>, state &lt;chr>, prop_type &lt;chr>, post_code &lt;int>,
 ## #   loan_number &lt;chr>, loan_purpose &lt;chr>, orig_term &lt;int>,
 ## #   num_borrowers &lt;int>, seller &lt;chr>, servicer &lt;chr>
</pre>
</div>
<p>The origination date is buried inside the loan number field. We will pick it out by indexing the loan number substring:</p>
<p>
<div>
<pre>freddie_rename <span>%>%</span> select(loan_number)
</pre>
</div>
<p>
<div>
<pre> ## Source:   query [?? x 1]
 ## Database: spark connection master=yarn-client app=sparklyr local=FALSE
 ## 
 ##     loan_number
 ##           &lt;chr>
 ## 1  F199Q1000001
 ## 2  F199Q1000002
 ## 3  F199Q1000003
 ## 4  F199Q1000004
 ## 5  F199Q1000005
 ## 6  F199Q1000006
 ## 7  F199Q1000007
 ## 8  F199Q1000008
 ## 9  F199Q1000009
 ## 10 F199Q1000010
 ## # ... with more rows
</pre>
</div>
<p>Observe that year field is augmented with the quarter field and then some additional loan identification numbers. We will extract that by using some very simple string operations:</p>
<p>
<div>
<pre>freddie_rename <span>&lt;-</span> freddie_rename <span>%>%</span> 
  mutate(orig_date <span>=</span> substr(loan_number, <span>3</span>, <span>4</span>),
         year <span>=</span> as.numeric(substr(loan_number, <span>3</span>, <span>2</span>)))
freddie <span>&lt;-</span> freddie_rename <span>%>%</span> 
  mutate(orig_year <span>=</span> paste0(ifelse(year <span>&lt;</span> <span>10</span>, <span>&quot;200&quot;</span>, 
                                   ifelse(year <span>></span> <span>16</span>, <span>&quot;19&quot;</span>,
                                          <span>&quot;20&quot;</span>)), year))
freddie <span>&lt;-</span> freddie <span>%>%</span> 
  mutate(orig_year <span>=</span> substr(orig_year, <span>1</span>, <span>4</span>))
freddie <span>%>%</span> head
</pre>
</div>
<p>
<div>
<pre> ## Source:   query [?? x 28]
 ## Database: spark connection master=yarn-client app=sparklyr local=FALSE
 ## 
 ##   credit_score first_payment first_home maturity   msa mi_perc num_units
 ##          &lt;chr>         &lt;int>      &lt;chr>    &lt;int> &lt;int>   &lt;chr>     &lt;int>
 ## 1          751        199910          N   202909    NA     000         1
 ## 2          733        199909          N   202908 29540     000         1
 ## 3          755        199905          N   202904 29540      30         1
 ## 4          669        200206          N   202901    NA     000         1
 ## 5          732        199904          N   202903 17140     000         1
 ## 6          715        199904          N   202903 17140     000         1
 ## # ... with 21 more variables: occ_status &lt;chr>, cltv &lt;dbl>, dti &lt;chr>,
 ## #   upb &lt;int>, ltv &lt;int>, orig_rate &lt;dbl>, channel &lt;chr>, ppm &lt;chr>,
 ## #   prod_type &lt;chr>, state &lt;chr>, prop_type &lt;chr>, post_code &lt;int>,
 ## #   loan_number &lt;chr>, loan_purpose &lt;chr>, orig_term &lt;int>,
 ## #   num_borrowers &lt;int>, seller &lt;chr>, servicer &lt;chr>, orig_date &lt;chr>,
 ## #   year &lt;dbl>, orig_year &lt;chr>
</pre>
</div>
<h2>Calculate Average Credit Score by Year</h2>
<p>Now that we have created features defining the quarter and year of the mortgage origination, we can create a summary of average credit attributes by year. For example, here’s the average credit score by year for each state:</p>
<p>
<div>
<pre>fico_year <span>&lt;-</span> freddie <span>%>%</span> group_by(orig_year, state) <span>%>%</span> 
  summarise(ave_fico <span>=</span> mean(credit_score)) <span>%>%</span> collect
fico_year <span>%>%</span> head
</pre>
</div>
<p>
<div>
<pre> ## Source: local data frame [6 x 3]
 ## Groups: orig_year [6]
 ## 
 ##   orig_year state ave_fico
 ##       &lt;chr> &lt;chr>    &lt;dbl>
 ## 1      2008    NM 730.7395
 ## 2      2012    WI 769.2008
 ## 3      2011    KS 763.1039
 ## 4      2006    WI 728.6378
 ## 5      2014    AR 750.9074
 ## 6      2015    MI 752.5942
</pre>
</div>
<p>Even though we are using Spark to do most of computation, we can still proceed as though we were working entirely in R. For example, we could create a function for evaluating the average credit attribute, which will make it easier to obtain different summarizations:</p>
<p>
<div>
<pre>year_state_sum <span>&lt;-</span> <span>function</span>(val <span>=</span> <span>&quot;credit_score&quot;</span>) {
  library(lazyeval) <br>
  year_state <span>&lt;-</span> freddie <span>%>%</span> group_by(orig_year, state) <span>%>%</span> 
    summarise_(sum_val <span>=</span> interp(<span>~</span>mean(var), var <span>=</span> as.name(val)))
  year_state <span>&lt;-</span> year_state <span>%>%</span> collect
  names(year_state)[<span>3</span>] <span>&lt;-</span> paste0(<span>&quot;ave_&quot;</span>, val)
  <span>return</span>(year_state)
}
</pre>
</div>
<h2>Summarize, Plot, …?, Profit!</h2>
<p>While our original dataset was very large, our resulting aggregated result is pretty small. The length of the output is just the number of years of data we have times the number of states:</p>
<p>$$<br>
\text{number of rows} = \sum years \times\sum states<br>
$$</p>
<p>Therefore, we could collect that object into a local R data.frame and plot it using the plethora of plotting libraries in R. In this case, we have data at the state-level, so I will use the awesome <a target="_blank" rel="nofollow" href="https://github.com/ramnathv/rMaps"><code>rMaps</code></a> to make a htmlwidget containing average debt-to-income by state and year:</p>
<p>
<div>
<pre>library(rMaps)
year_state_sum(<span>&quot;dti&quot;</span>) <span>%>%</span> 
  mutate(year <span>=</span> as.numeric(orig_year)) <span>%>%</span> 
  rMaps<span>::</span>ichoropleth(ave_dti <span>~</span> state, data <span>=</span> .,
                     animate <span>=</span> <span>&quot;year&quot;</span>,
                     geographyConfig <span>=</span> list(popupTemplate <span>=</span> <span>&quot;#!function(geo, data) {</span>
<span> return '&lt;div class=\&quot;hoverinfo\&quot;>&lt;strong>'+</span>
<span> data.state + '&lt;br>' + 'Average DTI in '+ data.year + ': ' +</span>
<span> data.ave_dti.toFixed(2)+</span>
<span> '&lt;/strong>&lt;/div>';}!#&quot;</span>)) <span>-></span> state_dti
state_dti<span>$</span>save(<span>&quot;StateMapDTI.html&quot;</span>, cdn <span>=</span> <span>T</span>)
htmltools<span>::</span>includeHTML(<span>&quot;StateMapDTI.html&quot;</span>)
</pre>
</div>
<p><iframe height="520" width="500" allowfullscreen src="http://blog.revolutionanalytics.com/downloads/StateMapDTI.html"></iframe></p>
<p><a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/downloads/StateMapDTI.html">Click here for a larger version without the scrollbar.</a></p>
<h1>The End</h1>
<p>Thank you for reading! If this post was interesting, I welcome you to check out my course on R and Spark <a target="_blank" rel="nofollow" href="https://github.com/akzaidi/R-cadence/wiki">here</a> and my in-development booklet on <a target="_blank" rel="nofollow" href="https://bookdown.org/alizaidi/mrs-spark-ml/">Spark and R</a>. I also spoke about Spark and R at the recent Spark Summit in Brussels, and you can <a target="_blank" rel="nofollow" href="https://www.youtube.com/watch?v=BtJMF2QkLE8">find the recording of that talk here</a>. Thanks!</p>
</div>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/data-manipulation-with-sparklyr-on-azure-hdinsight/"></div></div><div><a data-url="https://www.r-bloggers.com/data-manipulation-with-sparklyr-on-azure-hdinsight/" data-counturl="https://www.r-bloggers.com/data-manipulation-with-sparklyr-on-azure-hdinsight/" data-count="vertical" data-text="Data Manipulation with sparklyr on Azure HDInsight" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2016/11/data-manipulation-with-sparklyr-on-azure-hdinsight.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/dhywpiLs8Jg"></a></div>