<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/build-a-gradient-boosted-trees-model-with-microsoft-r-server/"></div></div><div><a data-url="http://www.r-bloggers.com/build-a-gradient-boosted-trees-model-with-microsoft-r-server/" data-counturl="http://www.r-bloggers.com/build-a-gradient-boosted-trees-model-with-microsoft-r-server/" data-count="horizontal" data-text="Build a Gradient Boosted Trees Model with Microsoft R Server" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2016/05/build-a-gradient-boosted-trees-model-with-mrs.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>by Yuzhou Song, Microsoft Data Scientist</p>
<p>R is an open source, statistical programming language with millions of users in its community. However, a well-known weakness of R is that it is both single threaded and memory bound, which limits its ability to process big data. With <a target="_blank" href="https://www.microsoft.com/en-us/server-cloud/products/r-server/">Microsoft R Server</a> (MRS), the enterprise grade distribution of R for advanced analytics, users can continue to work in their preferred R environment with following benefits: the ability to scale to data of any size, potential speed increases of up to one hundred times faster than open source R.</p>
<p>In this article, we give a walk-through on how to build a gradient boosted tree using MRS. We use a simple fraud data data set having approximately 1 million records and 9 columns. The last column “fraudRisk” is the tag: 0 stands for non-fraud and 1 stands for fraud. The following is a snapshot of the data.</p>
<p><a target="_blank" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb08f1cebb970d-pi"><img title="Y1" alt="Y1" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb08f1cebb970d-800wi" border="0"></a></p>
<p><strong>Step 1: Import the Data</strong></p>
<p>At the very beginning, load “RevoScaleR” package and specify directory and name of data file. Note that this demo is done in local R, thus, I need load “RevoScaleR” package.</p>
<p><span>library(RevoScaleR)<br>data.path &lt;- &quot;./&quot;<br>file.name &lt;- &quot;ccFraud.csv&quot;<br>fraud_csv_path &lt;- file.path(data.path,file.name)</span></p>
<p>Next, we make a data source using RxTextData function. The output of RxTextData function is a data source declaring type of each field, name of each field and location of the data.</p>
<p><span>colClasses &lt;- c(&quot;integer&quot;,&quot;factor&quot;,&quot;factor&quot;,&quot;integer&quot;,<br>                &quot;numeric&quot;,&quot;integer&quot;,&quot;integer&quot;,&quot;numeric&quot;,&quot;factor&quot;)<br>names(colClasses)&lt;-c(&quot;custID&quot;,&quot;gender&quot;,&quot;state&quot;,&quot;cardholder&quot;,<br></span><span>                     &quot;balance&quot;,&quot;numTrans&quot;,&quot;numIntlTrans&quot;,<br>                     &quot;creditLine&quot;,&quot;fraudRisk&quot;)<br></span><span>fraud_csv_source &lt;- RxTextData(file=fraud_csv_path,colClasses=colClasses)</span></p>
<p>With the data source (“fraud_csv_source” above), we are able to take it as input to other functions, just like inputting a data frame into regular R functions. For example, we can put it into <span>rxGetInfo</span> function to check the information of the data:</p>
<p><span>rxGetInfo(fraud_csv_source, getVarInfo = TRUE, numRows = 5)</span></p>
<p>and you will get the following:</p>
<p><a target="_blank" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b8d1d7f76e970c-pi"><img title="Y2" alt="Y2" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b8d1d7f76e970c-800wi" border="0"></a></p>
<p><strong>Step 2: Process Data</strong></p>
<p>Next, we demonstrate how to use an important function, <span>rxDataStep</span>, to process data before training. Generally, rxDataStep function transforms data from an input data set to an output data set. Basically there are three main arguments of that function: inData, outFile and transformFunc. inData can take either a data source (made by RxTextData shown in step 1) or a data frame. outFile takes data source to specify output file name, schema and location. If outFile is empty, rxDataStep function will return a data frame. transformFunc takes a function as input which will be used to do transformation by <span>rxDatastep</span> function. If the function has arguments more than input data source/frame, you may specify them in the transformObjects argument.</p>
<p>Here, we make a training flag using <span>rxDataStep</span> function as an example. The data source fraud_csv_source made in step 1 will be used for inData. We create an output data source specifying output file name “ccFraudFlag.csv”:</p>
<p><span>fraud_flag_csv_source &lt;- RxTextData(file=file.path(data.path,<br>                                    &quot;ccFraudFlag.csv&quot;))</span></p>
<p>Also, we create a simple transformation function called “make_train_flag”. It creates training flag which will be used to split data into training and testing set:</p>
<p><span>make_train_flag &lt;- function(data){<br>          data &lt;- as.data.frame(data)<br>          set.seed(34)<br>          data$trainFlag &lt;- sample(c(0,1),size=nrow(data),<br>                                   replace=TRUE,prob=c(0.3,0.7))<br>          return(data)<br> }</span></p>
<p>Then, use <span>rxDateStep</span> to complete the transformation:</p>
<p><span>rxDataStep(inData=fraud_csv_source,<br>           outFile=fraud_flag_csv_source,<br>           transformFunc=make_train_flag,overwrite = TRUE)</span></p>
<p>again, we can check the output file information by using rxGetInfo function:</p>
<p><span>rxGetInfo(fraud_flag_csv_source, getVarInfo = TRUE, numRows = 5)</span></p>
<p>we can find the trainFlag column has been appended to the last:</p>
<p><a target="_blank" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb08f1ced2970d-pi"><img title="Y3" alt="Y3" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb08f1ced2970d-800wi" border="0"></a></p>
<p>Based on the trainFlag, we split the data into training and testing set. Thus, we need specify the data source for output:</p>
<p><span>train_csv_source &lt;- RxTextData(<br>                    file=file.path(data.path,&quot;train.csv&quot;))</span></p>
<p><span>test_csv_source &lt;- RxTextData(<br>                   file=file.path(data.path,&quot;test.csv&quot;))</span></p>
<p>Instead of creating a transformation function, we can simply specify the rowSelection argument in <span>rxDataStep</span> function to select rows satisfying certain conditions:</p>
<p><span>rxDataStep(inData=fraud_flag_csv_source,<br></span><span>           outFile=train_csv_source,<br></span><span>           reportProgress = 0,<br></span><span>           rowSelection = (trainFlag == 1),<br></span><span>           overwrite = TRUE)</span></p>
<p><span>rxDataStep(inData=fraud_flag_csv_source,<br></span><span>           outFile=test_csv_source,<br></span><span>           reportProgress = 0,<br></span><span>           rowSelection = (trainFlag == 0),<br></span><span>           overwrite = TRUE)</span></p>
<p>A well-known problem for fraud data is the extremely skewed distribution of labels, i.e., most transactions are legitimate while only a very small proportion are fraud transactions. In the original data, good/bad ratio is about 15:1. Directly using original data to train a model will result in a poor performance, since the model is unable to find the proper boundary between “good” and “bad”. A simple but effective solution is to randomly down sample the majority.  The following is the down_sample transformation function down sampling majority to a good/bad ratio 4:1. The down sample ratio is pre-selected based on prior knowledge but can be finely tuned based on cross validation as well.</p>
<p><span>down_sample &lt;- function(data){<br></span><span>     data &lt;- as.data.frame(data)<br></span><span>     data_bad &lt;- subset(data,fraudRisk == 1)<br></span><span>     data_good &lt;- subset(data,fraudRisk == 0)<br>     # good to bad ratio 4:1<br></span><span>     rate &lt;- nrow(data_bad)*4/nrow(data_good) <br></span><span>     set.seed(34)</span></p>
<p><span>  data_good$keepTag &lt;- sample(c(0,1),replace=TRUE,<br>       size=nrow(data_good),prob=c(1-rate,rate))</span></p>
<p><span>  data_good_down &lt;- subset(data_good,keepTag == 1)<br></span><span>  data_good_down$keepTag &lt;- NULL<br></span><span>  data_down &lt;- rbind(data_bad,data_good_down)<br></span><span>  data_down$trainFlag &lt;- NULL<br></span><span>  return(data_down)<br></span><span>}</span></p>
<p>Then, we specify the down sampled training data source and use <span>rxDataStep</span> function again to complete the down sampling process:</p>
<p><span>train_downsample_csv_source &lt;- RxTextData(<br>                  file=file.path(data.path,<br>                  &quot;train_downsample.csv&quot;),<br>                  colClasses = colClasses)</span></p>
<p><span>rxDataStep(inData=train_csv_source,<br></span><span>           outFile=train_downsample_csv_source,<br></span><span>           transformFunc = down_sample,<br></span><span>           reportProgress = 0,<br></span><span>           overwrite = TRUE)</span></p>
<p><strong>Step 3: Training</strong></p>
<p>In this step, we take the down sampled data to train a gradient boosted tree. We first use <span>rxGetVarNames</span> function to get all variable names in training set. The input is still the data source of down sampled training data. Then we use it to create a formula which will be used later:</p>
<p><span>training_vars &lt;- rxGetVarNames(train_downsample_csv_source)</span></p>
<p><span>training_vars &lt;- training_vars[!(training_vars %in% <br>                               c(&quot;fraudRisk&quot;,&quot;custID&quot;))]</span></p>
<p><span>formula &lt;- as.formula(paste(&quot;fraudRisk~&quot;, <br>                      paste(training_vars, collapse = &quot;+&quot;)))</span></p>
<p>The <span>rxBTrees</span> function is used for building gradient boosted tree model. formula argument is used to specify label column and predictor columns. data argument takes a data source as input for training. lossFunction argument specifies the distribution of label column, i.e., “bernoulli” for numerical 0/1 regression, “gaussian” for numerical regression, and “multinomial” for two or more class classification. Here we choose “multinomial” as 0/1 classification problem. Other parameters are pre-selected, not finely tuned:</p>
<p><span>boosted_fit &lt;- rxBTrees(formula = formula,<br></span><span>               data = train_downsample_csv_source,<br></span><span>               learningRate = 0.2,<br></span><span>               minSplit = 10,<br></span><span>               minBucket = 10,<br>               # small number of tree for testing purpose<br></span><span>               nTree = 20, <br></span><span>               seed = 5,<br></span><span>               lossFunction =&quot;multinomial&quot;,<br></span><span>               reportProgress = 0)</span></p>
<p><strong>Step 4: Prediction and Evaluation</strong></p>
<p>We use <span>rxPredict</span> function to predict on testing data set, but first use <span>rxImport</span> function to import testing data set:</p>
<p><span>test_data &lt;- rxImport(test_csv_source)</span></p>
<p>Then, we take the imported testing set and fitted model object as input for rxPredict function:</p>
<p><span>predictions &lt;- rxPredict(modelObject = boosted_fit,<br></span><span>                         data = test_data,<br></span><span>                         type = &quot;response&quot;,<br></span><span>                         overwrite = TRUE,<br></span><span>                         reportProgress = 0)</span></p>
<p>In <span>rxPrediction</span> function, type=”response” will output predicted probabilities. Finally, we pick 0.5 as the threshold and evaluate the performance:</p>
<p><span>threshold &lt;- 0.5<br></span><span>predictions &lt;- data.frame(predictions$X1_prob)<br></span><span>names(predictions) &lt;- c(&quot;Boosted_Probability&quot;)<br></span><span>predictions$Boosted_Prediction &lt;- ifelse(<br>        predictions$Boosted_Probability > threshold, 1, 0)<br></span><span>predictions$Boosted_Prediction &lt;- factor(<br>        predictions$Boosted_Prediction, <br>        levels = c(1, 0))</span></p>
<p><span>scored_test_data &lt;- cbind(test_data, predictions)</span></p>
<p><span>evaluate_model &lt;- function(data, observed, predicted) {<br></span><span>       confusion &lt;- table(data[[observed]], <br>                          data[[predicted]])<br></span><span>       print(confusion)<br></span><span>       tp &lt;- confusion[1, 1]<br></span><span>       fn &lt;- confusion[1, 2]<br></span><span>       fp &lt;- confusion[2, 1]<br></span><span>       tn &lt;- confusion[2, 2]<br></span><span>       accuracy &lt;- (tp+tn)/(tp+fn+fp tn)<br></span><span>       precision &lt;- tp / (tp + fp)<br></span><span>       recall &lt;- tp / (tp + fn)<br></span><span>       fscore &lt;- 2*(precision*recall)/(precision+recall)</span></p>
<p><span>       metrics &lt;- c(&quot;Accuracy&quot; = accuracy,<br></span><span>               &quot;Precision&quot; = precision,<br></span><span>               &quot;Recall&quot; = recall,<br></span><span>               &quot;F-Score&quot; = fscore)<br></span><span>       return(metrics)<br></span><span>      }</span></p>
<p><em> </em></p>
<p><span>roc_curve &lt;- function(data, observed, predicted) {<br></span><span>             data &lt;- data[, c(observed, predicted)]<br></span><span>             data[[observed]] &lt;- as.numeric(<br>                     as.character(data[[observed]]))</span></p>
<p><span>             rxRocCurve(actualVarName = observed,<br></span><span>                        predVarNames = predicted,<br></span><span>                        data = data)</span></p>
<p><span>             }</span></p>
<p><em> </em></p>
<p><span>boosted_metrics &lt;- evaluate_model(data = scored_test_data,<br></span><span>                   observed = &quot;fraudRisk&quot;,<br></span><span>                   predicted = &quot;Boosted_Prediction&quot;)</span></p>
<p><span>roc_curve(data = scored_test_data,<br></span><span>          observed = &quot;fraudRisk&quot;,<br></span><span>          predicted = &quot;Boosted_Probability&quot;)</span></p>
<p> </p>
<p>The confusion Matrix:</p>
<p> <span>      0             1</span><br><span>0   2752288       67659</span><br><span>1     77117      101796</span></p>
<p>ROC curve is (AUC=0.95):</p>
<p><a target="_blank" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c84e0694970b-pi"><img title="Y4" alt="Y4" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c84e0694970b-800wi" border="0"></a></p>
<p><strong>Summary:</strong></p>
<p>In this article, we demonstrate how to use MRS in a fraud data. It includes how to create a data source by <span>RxTextData</span> function, how to make transformation using <span>rxDataStep</span> function, how to import data using <span>rxImport</span> function, how to train a gradient boosted tree model using <span>rxBTrees</span> function, and how to predict using <span>rxPredict</span> function. </p>
</div>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/build-a-gradient-boosted-trees-model-with-microsoft-r-server/"></div></div><div><a data-url="http://www.r-bloggers.com/build-a-gradient-boosted-trees-model-with-microsoft-r-server/" data-counturl="http://www.r-bloggers.com/build-a-gradient-boosted-trees-model-with-microsoft-r-server/" data-count="vertical" data-text="Build a Gradient Boosted Trees Model with Microsoft R Server" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2016/05/build-a-gradient-boosted-trees-model-with-mrs.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/H4L2wjgfqGI"></a></div>