<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/chaid-v-ranger-v-xgboost-a-comparison-july-27-2018/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://ibecav.github.io/chaidrangerxgboost/"> Chuck Powell</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a rel="nofollow" href="https://ibecav.github.io/chaidtutor1/" target="_blank">In an earlier post</a>, I focused<br>
on an in depth visit with CHAID (Chi-square automatic interaction<br>
detection). Quoting myself, I said “As the name implies it is<br>
fundamentally based on the venerable Chi-square test – and while not the<br>
most powerful (in terms of detecting the smallest possible differences)<br>
or the fastest, it really is easy to manage and more importantly to tell<br>
the story after using it”. In this post I’ll spend a little time<br>
comparing CHAID with a random forest algorithm in the <code>ranger</code> library<br>
and with a gradient boosting algorithm via the <code>xgboost</code> library. I’ll<br>
use the exact same data set for all three so we can draw some easy<br>
comparisons about their speed and their accuracy.</p>
<p>I do believe CHAID is a great choice for some sets of data and some<br>
circumstances but I’m interested in some empirical information, so off<br>
we go.</p>
<h2>Setup and library loading</h2>
<p>If you’ve never used <code>CHAID</code> before you may also not have <code>partykit</code>.<br>
<code>CHAID</code> isn’t on <code>CRAN</code> but I have provided the commented out install<br>
command below. <code>ranger</code> and <code>xgboost</code> are available from CRAN and are<br>
straightforward to install. You’ll also get a variety of messages, none<br>
of which is relevant to this example so I’ve suppressed them.</p>
<div>
<div>
<pre><code><span># install.packages(&quot;partykit&quot;)</span><span>
</span><span># install.packages(&quot;CHAID&quot;, repos=&quot;http://R-Forge.R-project.org&quot;)</span><span>
</span><span># install.packages(&quot;ranger&quot;)</span><span>
</span><span># install.packages(&quot;xgboost&quot;)</span><span>
</span><span>require</span><span>(</span><span>dplyr</span><span>)</span><span>
</span><span>require</span><span>(</span><span>tidyr</span><span>)</span><span>
</span><span>require</span><span>(</span><span>ggplot2</span><span>)</span><span>
</span><span>require</span><span>(</span><span>CHAID</span><span>)</span><span>
</span><span>require</span><span>(</span><span>purrr</span><span>)</span><span>
</span><span>require</span><span>(</span><span>caret</span><span>)</span><span>
</span><span>require</span><span>(</span><span>ranger</span><span>)</span><span>
</span><span>require</span><span>(</span><span>xgboost</span><span>)</span><span>
</span><span>require</span><span>(</span><span>kableExtra</span><span>)</span><span> </span><span># just to make the output nicer</span><span>
</span><span>theme_set</span><span>(</span><span>theme_bw</span><span>())</span><span> </span><span># set theme for ggplot2</span><span>
</span></code></pre>
</div>
</div>
<h2>Predicting customer churn for a fictional TELCO company</h2>
<p>We’re going to use a dataset that comes to us from the <a rel="nofollow" href="https://www.ibm.com/communities/analytics/watson-analytics-blog/using-customer-behavior-data-to-improve-customer-retention/" target="_blank">IBM Watson<br>
Project</a>.<br>
It’s a very practical example and an understandable dataset. A great use<br>
case for the algorithms we’ll be using. Imagine yourself in a fictional<br>
company faced with the task of trying to predict which customers are<br>
going to leave your business for another provider a.k.a. churn.<br>
Obviously we’d like to be able to predict this phenomenon and<br>
potentially target these customers for retention or just better project<br>
our revenue. Being able to predict churn even a little bit better could<br>
save us lots of money, especially if we can identify the key indicators<br>
and influence them.</p>
<p><a rel="nofollow" href="https://ibecav.github.io/chaidtutor1/" target="_blank">In the original posting</a> I spent<br>
a great deal of time explaining the mechanics of loading and prepping<br>
the data. This time we’ll do that quickly and efficiently and if you<br>
need an explanation of what’s going on please refer back. I’ve embedded<br>
some comments in the code where I think they’ll be most helpful. First<br>
we’ll grab the data from the IBM site using <code>read.csv</code>, in this case I’m<br>
happy to let it tag most of our variables as <code>factors</code> since that’s what<br>
we’ll want for our <code>CHAID</code> work.</p>
<div>
<div>
<pre><code><span>set.seed</span><span>(</span><span>2018</span><span>)</span><span>
</span><span>churn</span><span> </span><span>&lt;-</span><span> </span><span>read.csv</span><span>(</span><span>&quot;https://community.watsonanalytics.com/wp-content/uploads/2015/03/WA_Fn-UseC_-Telco-Customer-Churn.csv&quot;</span><span>)</span><span>
</span><span>str</span><span>(</span><span>churn</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## 'data.frame':    7043 obs. of  21 variables:
##  $ customerID      : Factor w/ 7043 levels &quot;0002-ORFBO&quot;,&quot;0003-MKNFE&quot;,..: 5376 3963 2565 5536 6512 6552 1003 4771 5605 4535 ...
##  $ gender          : Factor w/ 2 levels &quot;Female&quot;,&quot;Male&quot;: 1 2 2 2 1 1 2 1 1 2 ...
##  $ SeniorCitizen   : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ Partner         : Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 2 1 1 1 1 1 1 1 2 1 ...
##  $ Dependents      : Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 1 1 1 1 1 1 2 1 1 2 ...
##  $ tenure          : int  1 34 2 45 2 8 22 10 28 62 ...
##  $ PhoneService    : Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 1 2 2 1 2 2 2 1 2 2 ...
##  $ MultipleLines   : Factor w/ 3 levels &quot;No&quot;,&quot;No phone service&quot;,..: 2 1 1 2 1 3 3 2 3 1 ...
##  $ InternetService : Factor w/ 3 levels &quot;DSL&quot;,&quot;Fiber optic&quot;,..: 1 1 1 1 2 2 2 1 2 1 ...
##  $ OnlineSecurity  : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 3 3 3 1 1 1 3 1 3 ...
##  $ OnlineBackup    : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 3 1 3 1 1 1 3 1 1 3 ...
##  $ DeviceProtection: Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 3 1 3 1 3 1 1 3 1 ...
##  $ TechSupport     : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 1 1 3 1 1 1 1 3 1 ...
##  $ StreamingTV     : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 1 1 1 1 3 3 1 3 1 ...
##  $ StreamingMovies : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 1 1 1 1 3 1 1 3 1 ...
##  $ Contract        : Factor w/ 3 levels &quot;Month-to-month&quot;,..: 1 2 1 2 1 1 1 1 1 2 ...
##  $ PaperlessBilling: Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 2 1 2 1 2 2 2 1 2 1 ...
##  $ PaymentMethod   : Factor w/ 4 levels &quot;Bank transfer (automatic)&quot;,..: 3 4 4 1 3 3 2 4 3 1 ...
##  $ MonthlyCharges  : num  29.9 57 53.9 42.3 70.7 ...
##  $ TotalCharges    : num  29.9 1889.5 108.2 1840.8 151.7 ...
##  $ Churn           : Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 1 1 2 1 2 2 1 1 2 1 ...
</code></pre>
</div>
</div>
<p>We have data on 7,043 customers across 21 variables. <code>customerID</code> can’t<br>
really be a predictor but we will use it in a little bit. <code>Churn</code> is<br>
what we want to predict so we have 19 potential predictor variables to<br>
work with. Four of them were not automatically converted to factors so<br>
we’ll have to look into them for <code>CHAID</code>. <a rel="nofollow" href="https://ibecav.github.io/chaidtutor1/" target="_blank">For a review of what the<br>
output means and how CHAID works please refer<br>
back</a>.</p>
<p>Let’s address the easiest thing first. <code>SeniorCitizen</code> is coded zero and<br>
one instead of yes/no so let’s recode that in a nice conservative<br>
fashion and see what the breakdown is.</p>
<div>
<div>
<pre><code><span># Fix senior citizen status</span><span>
</span><span>churn</span><span>$</span><span>SeniorCitizen</span><span> </span><span>&lt;-</span><span> </span><span>recode_factor</span><span>(</span><span>
</span><span>churn</span><span>$</span><span>SeniorCitizen</span><span>,</span><span>
</span><span>`0`</span><span> </span><span>=</span><span> </span><span>&quot;No&quot;</span><span>,</span><span>
</span><span>`1`</span><span> </span><span>=</span><span> </span><span>&quot;Yes&quot;</span><span>,</span><span>
</span><span>.default</span><span> </span><span>=</span><span> </span><span>&quot;Should not happen&quot;</span><span>
</span><span>)</span><span>
</span><span>summary</span><span>(</span><span>churn</span><span>$</span><span>SeniorCitizen</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##   No  Yes 
## 5901 1142
</code></pre>
</div>
</div>
<p>We have three variables left that are numeric, now that we have<br>
addressed senior citizen status. Let’s use a combination of <code>dplyr</code> and<br>
<code>ggplot2</code> to see what the distribution looks like using a density plot.</p>
<div>
<div>
<pre><code><span>churn</span><span> </span><span>%>%</span><span>
</span><span>select_if</span><span>(</span><span>is.numeric</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>gather</span><span>(</span><span>metric</span><span>,</span><span> </span><span>value</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>value</span><span>,</span><span> </span><span>fill</span><span> </span><span>=</span><span> </span><span>metric</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_density</span><span>(</span><span>show.legend</span><span> </span><span>=</span><span> </span><span>FALSE</span><span>)</span><span> </span><span>+</span><span>
</span><span>facet_wrap</span><span>(</span><span> </span><span>~</span><span> </span><span>metric</span><span>,</span><span> </span><span>scales</span><span> </span><span>=</span><span> </span><span>&quot;free&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Warning: Removed 11 rows containing non-finite values (stat_density).
</code></pre>
</div>
</div>
<p><img alt="" src="https://i2.wp.com/ibecav.github.io/images/crx3-1.png?w=456" data-recalc-dims="1"></p>
<p>Well those aren’t the most normal looking distributions and we have this<br>
message <code>## Warning: Removed 11 rows containing non-finite values<br>
(stat_density).</code> which alerts us to the fact that there are some missing<br>
values in our data. Let’s first figure out where the missing data is:</p>
<div>
<div>
<pre><code><span>churn</span><span> </span><span>%>%</span><span>
</span><span>select_if</span><span>(</span><span>anyNA</span><span>)</span><span> </span><span>%>%</span><span> </span><span>summary</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##   TotalCharges   
##  Min.   :  18.8  
##  1st Qu.: 401.4  
##  Median :1397.5  
##  Mean   :2283.3  
##  3rd Qu.:3794.7  
##  Max.   :8684.8  
##  NA's   :11
</code></pre>
</div>
</div>
<p>Now we know that total customer charges is missing 11 entries. Our three<br>
algorithms vary as to how gracefully they handle missing values but at<br>
this point we have several options including:</p>
<ul>
<li>Eliminate the entire customer record if anything is missing</li>
<li>Impute or substitute in some reasonable value like the mean or the<br>
median for missing values</li>
<li>Do some fancier imputation to make sure we substitute in the most<br>
plausible value for <code>TotalCharges</code></li>
</ul>
<p>Elimination is easy, efficient, and conservative and since it is a very<br>
small percentage of our total data set unlikely to cost us a lot of<br>
information for the models that don’t handle missing values well. But<br>
for purposes of this blog post and to help demonstrate some of the<br>
capabilities within <code>caret</code> (since we’re going to use it anyway) we’ll<br>
try median and knn (k nearest neighbor) imputation.</p>
<p>First let’s make a vector that contains the <code>customerID</code> numbers of the<br>
eleven cases in question.</p>
<div>
<div>
<pre><code><span>xxx</span><span> </span><span>&lt;-</span><span> </span><span>churn</span><span> </span><span>%>%</span><span>
</span><span>filter_all</span><span>(</span><span>any_vars</span><span>(</span><span>is.na</span><span>(</span><span>.</span><span>)))</span><span> </span><span>%>%</span><span> </span><span>select</span><span>(</span><span>customerID</span><span>)</span><span>
</span><span>xxx</span><span> </span><span>&lt;-</span><span> </span><span>as.vector</span><span>(</span><span>xxx</span><span>$</span><span>customerID</span><span>)</span><span>
</span><span>xxx</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##  [1] &quot;4472-LVYGI&quot; &quot;3115-CZMZD&quot; &quot;5709-LVOEQ&quot; &quot;4367-NUYAO&quot; &quot;1371-DWPAZ&quot; &quot;7644-OMVMY&quot; &quot;3213-VVOLG&quot; &quot;2520-SGTTA&quot; &quot;2923-ARZLG&quot; &quot;4075-WKNIU&quot; &quot;2775-SEFEE&quot;
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>churn</span><span> </span><span>%>%</span><span> </span><span>filter</span><span>(</span><span>customerID</span><span> </span><span>%in%</span><span> </span><span>xxx</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##    customerID gender SeniorCitizen Partner Dependents tenure PhoneService    MultipleLines InternetService      OnlineSecurity        OnlineBackup    DeviceProtection         TechSupport         StreamingTV     StreamingMovies Contract PaperlessBilling             PaymentMethod MonthlyCharges TotalCharges Churn
## 1  4472-LVYGI Female            No     Yes        Yes      0           No No phone service             DSL                 Yes                  No                 Yes                 Yes                 Yes                  No Two year              Yes Bank transfer (automatic)          52.55           NA    No
## 2  3115-CZMZD   Male            No      No        Yes      0          Yes               No              No No internet service No internet service No internet service No internet service No internet service No internet service Two year               No              Mailed check          20.25           NA    No
## 3  5709-LVOEQ Female            No     Yes        Yes      0          Yes               No             DSL                 Yes                 Yes                 Yes                  No                 Yes                 Yes Two year               No              Mailed check          80.85           NA    No
## 4  4367-NUYAO   Male            No     Yes        Yes      0          Yes              Yes              No No internet service No internet service No internet service No internet service No internet service No internet service Two year               No              Mailed check          25.75           NA    No
## 5  1371-DWPAZ Female            No     Yes        Yes      0           No No phone service             DSL                 Yes                 Yes                 Yes                 Yes                 Yes                  No Two year               No   Credit card (automatic)          56.05           NA    No
## 6  7644-OMVMY   Male            No     Yes        Yes      0          Yes               No              No No internet service No internet service No internet service No internet service No internet service No internet service Two year               No              Mailed check          19.85           NA    No
## 7  3213-VVOLG   Male            No     Yes        Yes      0          Yes              Yes              No No internet service No internet service No internet service No internet service No internet service No internet service Two year               No              Mailed check          25.35           NA    No
## 8  2520-SGTTA Female            No     Yes        Yes      0          Yes               No              No No internet service No internet service No internet service No internet service No internet service No internet service Two year               No              Mailed check          20.00           NA    No
## 9  2923-ARZLG   Male            No     Yes        Yes      0          Yes               No              No No internet service No internet service No internet service No internet service No internet service No internet service One year              Yes              Mailed check          19.70           NA    No
## 10 4075-WKNIU Female            No     Yes        Yes      0          Yes              Yes             DSL                  No                 Yes                 Yes                 Yes                 Yes                  No Two year               No              Mailed check          73.35           NA    No
## 11 2775-SEFEE   Male            No      No        Yes      0          Yes              Yes             DSL                 Yes                 Yes                  No                 Yes                  No                  No Two year              Yes Bank transfer (automatic)          61.90           NA    No
</code></pre>
</div>
</div>
<p>As you look at those eleven records it doesn’t appear they are<br>
“average”! In particular, I’m worried that the <code>MonthlyCharges</code><br>
look small and they have 0 <code>tenure</code> for this group. No way of knowing<br>
for certain but it could be that these are just the newest customers<br>
with very little time using our service. Let’s use our list to do some<br>
comparing of these eleven versus the total population, that will help us<br>
decide what to do about the missing cases. Replacing with the median<br>
value is simple and easy but it may well not be the most accurate<br>
choice.</p>
<div>
<div>
<pre><code><span>churn</span><span> </span><span>%>%</span><span> </span><span>filter</span><span>(</span><span>customerID</span><span> </span><span>%in%</span><span> </span><span>xxx</span><span>)</span><span> </span><span>%>%</span><span> </span><span>summarise</span><span>(</span><span>median</span><span>(</span><span>MonthlyCharges</span><span>))</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##   median(MonthlyCharges)
## 1                  25.75
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>median</span><span>(</span><span>churn</span><span>$</span><span>MonthlyCharges</span><span>,</span><span> </span><span>na.rm</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 70.35
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>churn</span><span> </span><span>%>%</span><span> </span><span>filter</span><span>(</span><span>customerID</span><span> </span><span>%in%</span><span> </span><span>xxx</span><span>)</span><span> </span><span>%>%</span><span> </span><span>summarise</span><span>(</span><span>median</span><span>(</span><span>tenure</span><span>))</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##   median(tenure)
## 1              0
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>median</span><span>(</span><span>churn</span><span>$</span><span>tenure</span><span>,</span><span> </span><span>na.rm</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 29
</code></pre>
</div>
</div>
<p>The median <code>MonthlyCharges</code> are much lower and instead of two years or<br>
so of median tenure this group has none. Let’s use the <code>preProcess</code><br>
function in <code>caret</code> to accomplish several goals. We’ll ask it to impute<br>
the missing values for us using both <code>knnImpute</code> (k nearest neighbors)<br>
and a pure median <code>medianImpute</code>. From the <code>?preProcess</code> help pages:</p>
<blockquote>
<p>k-nearest neighbor imputation is carried out by finding the k closest<br>
samples (Euclidian distance) in the training set. Imputation via<br>
bagging fits a bagged tree model for each predictor (as a function of<br>
all the others). This method is simple, accurate and accepts missing<br>
values, but it has much higher computational cost. Imputation via<br>
medians takes the median of each predictor in the training set, and<br>
uses them to fill missing values. This method is simple, fast, and<br>
accepts missing values, but treats each predictor independently, and<br>
may be inaccurate.</p>
</blockquote>
<p>We’ll also have it transform our numeric variables using <code>YeoJohnson</code><br>
and identify any predictor variables that have near zero variance <code>nzv</code>.</p>
<div>
<div>
<pre><code><span># using k nearest neighbors</span><span>
</span><span>pp_knn</span><span> </span><span>&lt;-</span><span> </span><span>preProcess</span><span>(</span><span>churn</span><span>,</span><span> </span><span>method</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>&quot;knnImpute&quot;</span><span>,</span><span> </span><span>&quot;YeoJohnson&quot;</span><span>,</span><span> </span><span>&quot;nzv&quot;</span><span>))</span><span>
</span><span># simple output</span><span>
</span><span>pp_knn</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Created from 7032 samples and 21 variables
## 
## Pre-processing:
##   - centered (3)
##   - ignored (18)
##   - 5 nearest neighbor imputation (3)
##   - scaled (3)
##   - Yeo-Johnson transformation (3)
## 
## Lambda estimates for Yeo-Johnson transformation:
## 0.45, 0.93, 0.25
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span># more verbose</span><span>
</span><span>pp_knn</span><span>$</span><span>method</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## $knnImpute
## [1] &quot;tenure&quot;         &quot;MonthlyCharges&quot; &quot;TotalCharges&quot;  
## 
## $YeoJohnson
## [1] &quot;tenure&quot;         &quot;MonthlyCharges&quot; &quot;TotalCharges&quot;  
## 
## $ignore
##  [1] &quot;customerID&quot;       &quot;gender&quot;           &quot;SeniorCitizen&quot;    &quot;Partner&quot;          &quot;Dependents&quot;       &quot;PhoneService&quot;     &quot;MultipleLines&quot;    &quot;InternetService&quot;  &quot;OnlineSecurity&quot;   &quot;OnlineBackup&quot;     &quot;DeviceProtection&quot; &quot;TechSupport&quot;      &quot;StreamingTV&quot;      &quot;StreamingMovies&quot;  &quot;Contract&quot;         &quot;PaperlessBilling&quot; &quot;PaymentMethod&quot;    &quot;Churn&quot;           
## 
## $center
## [1] &quot;tenure&quot;         &quot;MonthlyCharges&quot; &quot;TotalCharges&quot;  
## 
## $scale
## [1] &quot;tenure&quot;         &quot;MonthlyCharges&quot; &quot;TotalCharges&quot;
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span># using medians</span><span>
</span><span>pp_median</span><span> </span><span>&lt;-</span><span> </span><span>preProcess</span><span>(</span><span>churn</span><span>,</span><span> </span><span>method</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>&quot;medianImpute&quot;</span><span>,</span><span> </span><span>&quot;YeoJohnson&quot;</span><span>,</span><span> </span><span>&quot;nzv&quot;</span><span>))</span><span>
</span><span>pp_median</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Created from 7032 samples and 21 variables
## 
## Pre-processing:
##   - ignored (18)
##   - median imputation (3)
##   - Yeo-Johnson transformation (3)
## 
## Lambda estimates for Yeo-Johnson transformation:
## 0.45, 0.93, 0.25
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>pp_median</span><span>$</span><span>method</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## $medianImpute
## [1] &quot;tenure&quot;         &quot;MonthlyCharges&quot; &quot;TotalCharges&quot;  
## 
## $YeoJohnson
## [1] &quot;tenure&quot;         &quot;MonthlyCharges&quot; &quot;TotalCharges&quot;  
## 
## $ignore
##  [1] &quot;customerID&quot;       &quot;gender&quot;           &quot;SeniorCitizen&quot;    &quot;Partner&quot;          &quot;Dependents&quot;       &quot;PhoneService&quot;     &quot;MultipleLines&quot;    &quot;InternetService&quot;  &quot;OnlineSecurity&quot;   &quot;OnlineBackup&quot;     &quot;DeviceProtection&quot; &quot;TechSupport&quot;      &quot;StreamingTV&quot;      &quot;StreamingMovies&quot;  &quot;Contract&quot;         &quot;PaperlessBilling&quot; &quot;PaymentMethod&quot;    &quot;Churn&quot;
</code></pre>
</div>
</div>
<p>The <code>preProcess</code> function creates a list object of <code>class</code> preProcess<br>
that contains information about what needs to be done and what the<br>
results of the transformations will be, but we need to apply the<br>
<code>predict</code> function to actually make the changes proposed. So at this<br>
point let’s create two new dataframes <code>nchurn1</code> and <code>nchurn2</code> that<br>
contain the data after the pre-processing has occurred. Then we can see<br>
how the results compare.</p>
<div>
<div>
<pre><code><span>nchurn1</span><span> </span><span>&lt;-</span><span> </span><span>predict</span><span>(</span><span>pp_knn</span><span>,</span><span>churn</span><span>)</span><span>
</span><span>nchurn2</span><span> </span><span>&lt;-</span><span> </span><span>predict</span><span>(</span><span>pp_median</span><span>,</span><span>churn</span><span>)</span><span>
</span><span>nchurn2</span><span> </span><span>%>%</span><span> </span><span>filter</span><span>(</span><span>customerID</span><span> </span><span>%in%</span><span> </span><span>xxx</span><span>)</span><span> </span><span>%>%</span><span> </span><span>summarise</span><span>(</span><span>median</span><span>(</span><span>TotalCharges</span><span>))</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##   median(TotalCharges)
## 1             20.79526
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>median</span><span>(</span><span>nchurn2</span><span>$</span><span>TotalCharges</span><span>,</span><span> </span><span>na.rm</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 20.79526
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>nchurn1</span><span> </span><span>%>%</span><span> </span><span>filter</span><span>(</span><span>customerID</span><span> </span><span>%in%</span><span> </span><span>xxx</span><span>)</span><span> </span><span>%>%</span><span> </span><span>summarise</span><span>(</span><span>median</span><span>(</span><span>TotalCharges</span><span>))</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##   median(TotalCharges)
## 1            -1.849681
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>median</span><span>(</span><span>nchurn1</span><span>$</span><span>TotalCharges</span><span>,</span><span> </span><span>na.rm</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 0.01820494
</code></pre>
</div>
</div>
<p>May also be useful to visualize the data as we did earlier to see how<br>
the transformations have changed the density plots.</p>
<div>
<div>
<pre><code><span>nchurn1</span><span> </span><span>%>%</span><span>
</span><span>select_if</span><span>(</span><span>is.numeric</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>gather</span><span>(</span><span>metric</span><span>,</span><span> </span><span>value</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>value</span><span>,</span><span> </span><span>fill</span><span> </span><span>=</span><span> </span><span>metric</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_density</span><span>(</span><span>show.legend</span><span> </span><span>=</span><span> </span><span>FALSE</span><span>)</span><span> </span><span>+</span><span>
</span><span>facet_wrap</span><span>(</span><span> </span><span>~</span><span> </span><span>metric</span><span>,</span><span> </span><span>scales</span><span> </span><span>=</span><span> </span><span>&quot;free&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="" src="https://i1.wp.com/ibecav.github.io/images/crx9-1.png?w=456" data-recalc-dims="1"></p>
<div>
<div>
<pre><code><span>nchurn2</span><span> </span><span>%>%</span><span>
</span><span>select_if</span><span>(</span><span>is.numeric</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>gather</span><span>(</span><span>metric</span><span>,</span><span> </span><span>value</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>value</span><span>,</span><span> </span><span>fill</span><span> </span><span>=</span><span> </span><span>metric</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_density</span><span>(</span><span>show.legend</span><span> </span><span>=</span><span> </span><span>FALSE</span><span>)</span><span> </span><span>+</span><span>
</span><span>facet_wrap</span><span>(</span><span> </span><span>~</span><span> </span><span>metric</span><span>,</span><span> </span><span>scales</span><span> </span><span>=</span><span> </span><span>&quot;free&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="" src="https://i2.wp.com/ibecav.github.io/images/crx9-2.png?w=456" data-recalc-dims="1"></p>
<p>If you compare the two plots you can see that they vary imperceptibly<br>
except for the y axis scale. There is no warning about missing values<br>
and if you scroll back and compare with the original plots of the raw<br>
variables the shape of <code>tenure</code> and <code>TotalCharges</code> have changed<br>
significantly because of the transformation.</p>
<p>I’m pretty convinced that knn provides a much better approximation of<br>
those eleven missing values than a mere median substitution so let’s<br>
make those changes and move on to comparing models. While we’re at it,<br>
let’s go ahead and remove the unique customer ID number as well. We<br>
really only needed it to compare a few specific cases.</p>
<div>
<div>
<pre><code><span>churn</span><span> </span><span>&lt;-</span><span> </span><span>predict</span><span>(</span><span>pp_knn</span><span>,</span><span>churn</span><span>)</span><span>
</span><span>churn</span><span>$</span><span>customerID</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>str</span><span>(</span><span>churn</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## 'data.frame':    7043 obs. of  20 variables:
##  $ gender          : Factor w/ 2 levels &quot;Female&quot;,&quot;Male&quot;: 1 2 2 2 1 1 2 1 1 2 ...
##  $ SeniorCitizen   : Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 1 1 1 1 1 1 1 1 1 1 ...
##  $ Partner         : Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 2 1 1 1 1 1 1 1 2 1 ...
##  $ Dependents      : Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 1 1 1 1 1 1 2 1 1 2 ...
##  $ tenure          : num  -1.644 0.297 -1.495 0.646 -1.495 ...
##  $ PhoneService    : Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 1 2 2 1 2 2 2 1 2 2 ...
##  $ MultipleLines   : Factor w/ 3 levels &quot;No&quot;,&quot;No phone service&quot;,..: 2 1 1 2 1 3 3 2 3 1 ...
##  $ InternetService : Factor w/ 3 levels &quot;DSL&quot;,&quot;Fiber optic&quot;,..: 1 1 1 1 2 2 2 1 2 1 ...
##  $ OnlineSecurity  : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 3 3 3 1 1 1 3 1 3 ...
##  $ OnlineBackup    : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 3 1 3 1 1 1 3 1 1 3 ...
##  $ DeviceProtection: Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 3 1 3 1 3 1 1 3 1 ...
##  $ TechSupport     : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 1 1 3 1 1 1 1 3 1 ...
##  $ StreamingTV     : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 1 1 1 1 3 3 1 3 1 ...
##  $ StreamingMovies : Factor w/ 3 levels &quot;No&quot;,&quot;No internet service&quot;,..: 1 1 1 1 1 3 1 1 3 1 ...
##  $ Contract        : Factor w/ 3 levels &quot;Month-to-month&quot;,..: 1 2 1 2 1 1 1 1 1 2 ...
##  $ PaperlessBilling: Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 2 1 2 1 2 2 2 1 2 1 ...
##  $ PaymentMethod   : Factor w/ 4 levels &quot;Bank transfer (automatic)&quot;,..: 3 4 4 1 3 3 2 4 3 1 ...
##  $ MonthlyCharges  : num  -1.158 -0.239 -0.343 -0.731 0.214 ...
##  $ TotalCharges    : num  -1.81 0.254 -1.386 0.233 -1.249 ...
##  $ Churn           : Factor w/ 2 levels &quot;No&quot;,&quot;Yes&quot;: 1 1 2 1 2 2 1 1 2 1 ...
</code></pre>
</div>
</div>
<p>One more step before we start using <code>CHAID</code>, <code>ranger</code>, and <code>xgboost</code> and<br>
while we have the data in one frame. Let’s take the 3 numeric variables<br>
and create 3 analogous variables as factors. This is necessary because<br>
<code>CHAID</code> requires categorical a.k.a. nominal data. If you’d like to<br>
review the options for how to “cut” the data <a rel="nofollow" href="https://ibecav.github.io/chaidtutor1/" target="_blank">please refer back to my<br>
earlier post</a>.</p>
<div>
<div>
<pre><code><span>churn</span><span> </span><span>&lt;-</span><span> </span><span>churn</span><span> </span><span>%>%</span><span>
</span><span>mutate_if</span><span>(</span><span>is.numeric</span><span>,</span><span> </span><span>funs</span><span>(</span><span>factor</span><span> </span><span>=</span><span> </span><span>cut_number</span><span>(</span><span>.</span><span>,</span><span> </span><span>n</span><span>=</span><span>5</span><span>,</span><span> </span><span>labels</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>&quot;Lowest&quot;</span><span>,</span><span>&quot;Below Middle&quot;</span><span>,</span><span>&quot;Middle&quot;</span><span>,</span><span>&quot;Above Middle&quot;</span><span>,</span><span>&quot;Highest&quot;</span><span>))))</span><span>
</span><span>summary</span><span>(</span><span>churn</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##     gender     SeniorCitizen Partner    Dependents     tenure        PhoneService          MultipleLines     InternetService             OnlineSecurity              OnlineBackup             DeviceProtection              TechSupport                StreamingTV              StreamingMovies           Contract    PaperlessBilling                   PaymentMethod  MonthlyCharges     TotalCharges       Churn           tenure_factor   MonthlyCharges_factor   TotalCharges_factor
##  Female:3488   No :5901      No :3641   No :4933   Min.   :-1.8439   No : 682     No              :3390   DSL        :2421   No                 :3498   No                 :3088   No                 :3095    No                 :3473   No                 :2810   No                 :2785   Month-to-month:3875   No :2872         Bank transfer (automatic):1544   Min.   :-1.5685   Min.   :-1.929306   No :5174   Lowest      :1481   Lowest      :1420      Lowest      :1409    
##  Male  :3555   Yes:1142      Yes:3402   Yes:2110   1st Qu.:-0.8555   Yes:6361     No phone service: 682   Fiber optic:3096   No internet service:1526   No internet service:1526   No internet service:1526    No internet service:1526   No internet service:1526   No internet service:1526   One year      :1473   Yes:4171         Credit card (automatic)  :1522   1st Qu.:-0.9632   1st Qu.:-0.783551   Yes:1869   Below Middle:1397   Below Middle:1397      Below Middle:1408    
##                                                    Median : 0.1183                Yes             :2971   No         :1526   Yes                :2019   Yes                :2429   Yes                :2422    Yes                :2044   Yes                :2707   Yes                :2732   Two year      :1695                    Electronic check         :2365   Median : 0.2021   Median : 0.018205              Middle      :1408   Middle      :1411      Middle      :1409    
##                                                    Mean   : 0.0000                                                                                                                                                                                                                                                                     Mailed check             :1612   Mean   : 0.0000   Mean   :-0.002732              Above Middle:1350   Above Middle:1407      Above Middle:1408    
##                                                    3rd Qu.: 0.9252                                                                                                                                                                                                                                                                                                      3rd Qu.: 0.8341   3rd Qu.: 0.868066              Highest     :1407   Highest     :1408      Highest     :1409    
##                                                    Max.   : 1.3421                                                                                                                                                                                                                                                                                                      Max.   : 1.7530   Max.   : 1.758003
</code></pre>
</div>
</div>
<p>Okay now we have three additional variables that end in _factor,<br>
they’re like their numeric equivalents only cut into more or less 5<br>
equal bins.</p>
<h2>Training and testing our models</h2>
<p>We’re going to use <code>caret</code> to train and test all three of the algorithms<br>
on our data. We could operate directly by invoking the individual model<br>
functions directly but <code>caret</code> will allow us to use some common steps.<br>
We’ll employ <code>cross-validation</code> a.k.a. <code>cv</code> to mitigate the problem of<br>
over-fitting. <a rel="nofollow" href="https://elitedatascience.com/overfitting-in-machine-learning#how-to-prevent" target="_blank">This article<br>
explains</a><br>
it well so I won’t repeat that explanation here, I’ll simply show you<br>
how to run the steps in <code>R</code>.</p>
<p>This is also a good time to point out that <code>caret</code> has <a rel="nofollow" href="https://topepo.github.io/caret/" target="_blank">extraordinarily<br>
comprehensive documentation</a> which I<br>
used extensively and I’m limiting myself to the basics.</p>
<p>As a first step, let’s just take 30% of our data and put is aside as the<br>
testing data set. Why 30%? Doesn’t have to be, could be as low as 20% or<br>
as high as 40% it really depends on how conservative you want to be, and<br>
how much data you have at hand. Since this is just a tutorial we’ll<br>
simply use 30% as a representative number. I’m going to use <code>caret</code><br>
syntax which is the line with <code>createDataPartition(churn$Churn, p=0.7,<br>
list=FALSE)</code> in it. That takes our data set <code>churn</code> makes a 70% split<br>
ensuring that we keep our outcome variable <code>Churn</code> as close to 70/30 as<br>
we can. <em>This is important because our data is already pretty lop-sided</em><br>
for outcomes. The two subsequent lines serve to take the vector<br>
<code>intrain</code> and produce two separate dataframes, <code>testing</code> and <code>training</code>.<br>
They have 2112 and 4931 customers respectively.</p>
<div>
<div>
<pre><code><span>intrain</span><span> </span><span>&lt;-</span><span> </span><span>createDataPartition</span><span>(</span><span>churn</span><span>$</span><span>Churn</span><span>,</span><span> </span><span>p</span><span>=</span><span>0.7</span><span>,</span><span> </span><span>list</span><span>=</span><span>FALSE</span><span>)</span><span>
</span><span>training</span><span> </span><span>&lt;-</span><span> </span><span>churn</span><span>[</span><span>intrain</span><span>,]</span><span>
</span><span>testing</span><span> </span><span>&lt;-</span><span> </span><span>churn</span><span>[</span><span>-</span><span>intrain</span><span>,]</span><span>
</span><span>dim</span><span>(</span><span>training</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 4931   23
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>dim</span><span>(</span><span>testing</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 2112   23
</code></pre>
</div>
</div>
<h4>CHAID</h4>
<p>Now that we have a training and testing dataset let’s remove the numeric<br>
version of the variables <code>CHAID</code> can’t use.</p>
<div>
<div>
<pre><code><span># first pass at CHAID</span><span>
</span><span># remove numbers</span><span>
</span><span>training</span><span> </span><span>&lt;-</span><span> </span><span>training</span><span> </span><span>%>%</span><span>
</span><span>select_if</span><span>(</span><span>is.factor</span><span>)</span><span>
</span><span>dim</span><span>(</span><span>training</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 4931   20
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>testing</span><span> </span><span>&lt;-</span><span> </span><span>testing</span><span> </span><span>%>%</span><span>
</span><span>select_if</span><span>(</span><span>is.factor</span><span>)</span><span>
</span><span>dim</span><span>(</span><span>testing</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 2112   20
</code></pre>
</div>
</div>
<p>The next step is a little counter-intuitive but quite practical. Turns<br>
out that many models do not perform well when you feed them a <code>formula</code><br>
for the model even if they claim to support a formula interface (as<br>
CHAID does). <a rel="nofollow" href="https://stackoverflow.com/questions/33088893/caret-random-forests-not-working-something-is-wrong-all-the-accuracy-metric" target="_blank">Here’s a Stack Overflow<br>
link</a><br>
that discusses in detail but my suggestion to you is to always separate<br>
them and avoid the problem altogether. We’re just taking our<br>
<code>predictors</code> or <code>features</code> and putting them in <code>x</code> while we put our<br>
outcome in <code>y</code>.</p>
<div>
<div>
<pre><code><span># create response and feature data</span><span>
</span><span>features</span><span> </span><span>&lt;-</span><span> </span><span>setdiff</span><span>(</span><span>names</span><span>(</span><span>training</span><span>),</span><span> </span><span>&quot;Churn&quot;</span><span>)</span><span>
</span><span>x</span><span> </span><span>&lt;-</span><span> </span><span>training</span><span>[,</span><span> </span><span>features</span><span>]</span><span>
</span><span>y</span><span> </span><span>&lt;-</span><span> </span><span>training</span><span>$</span><span>Churn</span><span>
</span></code></pre>
</div>
</div>
<p><code>trainControl</code> is the next function within <code>caret</code> we need to use.<br>
Chapter 5 in the <code>caret</code> doco covers it in great detail. I’m simply<br>
going to pluck out a few sane and safe options. <code>method = &quot;cv&quot;</code> gets us<br>
cross-validation. <code>number = 5</code> is pretty obvious. I happen to like<br>
seeing the progress in case I want to go for coffee so <code>verboseIter =<br>
TRUE</code> (here I will turn it off since the static output is rather<br>
boring), and I play it safe and explicitly save my predictions<br>
<code>savePredictions = &quot;final&quot;</code>. We put everything in <code>train_control</code> which<br>
we’ll use in a minute. <strong>We’ll use this same <code>train_control</code> for all our<br>
models</strong></p>
<div>
<div>
<pre><code><span># set up 5-fold cross validation procedure</span><span>
</span><span>train_control</span><span> </span><span>&lt;-</span><span> </span><span>trainControl</span><span>(</span><span>method</span><span> </span><span>=</span><span> </span><span>&quot;cv&quot;</span><span>,</span><span>
</span><span>number</span><span> </span><span>=</span><span> </span><span>5</span><span>,</span><span>
</span><span># verboseIter = TRUE,</span><span>
</span><span>savePredictions</span><span> </span><span>=</span><span> </span><span>&quot;final&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p>By default <code>caret</code> allows us to adjust three parameters in our <code>chaid</code><br>
model; <code>alpha2</code>, <code>alpha3</code>, and <code>alpha4</code>. As a matter of fact it will<br>
allow us to build a grid of those parameters and test all the<br>
permutations we like, using the same cross-validation process. I’m a bit<br>
worried that we’re not being conservative enough. I’d like to train our<br>
model using p values for alpha that are not .05, .03, and .01 but<br>
instead the de facto levels in my discipline; .05, .01, and .001. The<br>
function in <code>caret</code> is <code>tuneGrid</code>. We’ll use the base <code>R</code> function<br>
<code>expand.grid</code> to build a dataframe with all the combinations and then<br>
feed it to <code>caret</code> in our training via <code>tuneGrid = search_grid</code> in our<br>
call to <code>train</code>.</p>
<div>
<div>
<pre><code><span># set up tuning grid default</span><span>
</span><span>search_grid</span><span> </span><span>&lt;-</span><span> </span><span>expand.grid</span><span>(</span><span>
</span><span>alpha2</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>.05</span><span>,</span><span> </span><span>.01</span><span>,</span><span> </span><span>.001</span><span>),</span><span>
</span><span>alpha4</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>.05</span><span>,</span><span> </span><span>.01</span><span>,</span><span> </span><span>.001</span><span>),</span><span>
</span><span>alpha3</span><span> </span><span>=</span><span> </span><span>-1</span><span>
</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p>Now we can use the <code>train</code> function in <code>caret</code> to train our model! It<br>
wants to know what our <code>x</code> and <code>y</code>’s are, as well as our training<br>
control parameters which we’ve parked in <code>train_control</code>.</p>
<div>
<div>
<pre><code><span>chaid.model</span><span> </span><span>&lt;-</span><span> </span><span>train</span><span>(</span><span>
</span><span>x</span><span> </span><span>=</span><span> </span><span>x</span><span>,</span><span>
</span><span>y</span><span> </span><span>=</span><span> </span><span>y</span><span>,</span><span>
</span><span>method</span><span> </span><span>=</span><span> </span><span>&quot;chaid&quot;</span><span>,</span><span>
</span><span>trControl</span><span> </span><span>=</span><span> </span><span>train_control</span><span>,</span><span>
</span><span>tuneGrid</span><span> </span><span>=</span><span> </span><span>search_grid</span><span>
</span><span>)</span><span>
</span><span>chaid.model</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## CHi-squared Automated Interaction Detection 
## 
## 4931 samples
##   19 predictor
##    2 classes: 'No', 'Yes' 
## 
## No pre-processing
## Resampling: Cross-Validated (5 fold) 
## Summary of sample sizes: 3944, 3946, 3945, 3944, 3945 
## Resampling results across tuning parameters:
## 
##   alpha2  alpha4  Accuracy   Kappa    
##   0.001   0.001   0.7805723  0.3956853
##   0.001   0.010   0.7832088  0.3786619
##   0.001   0.050   0.7767165  0.3956067
##   0.010   0.001   0.7844263  0.3950757
##   0.010   0.010   0.7832088  0.3740510
##   0.010   0.050   0.7767167  0.3883397
##   0.050   0.001   0.7844263  0.3950757
##   0.050   0.010   0.7817889  0.3764934
##   0.050   0.050   0.7783394  0.3921129
## 
## Tuning parameter 'alpha3' was held constant at a value of -1
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were alpha2 = 0.05, alpha3 = -1 and alpha4 = 0.001.
</code></pre>
</div>
</div>
<p>And after roughly two minutes it’s done. Let’s inspect what we have so<br>
far. The output gives us a nice concise summary. 4931 cases with 19<br>
predictors. It gives us an idea of how many of the 4931 cases were used<br>
in the individual folds <code>Summary of sample<br>
sizes: 3944, 3946, 3945, 3944, 3945</code>. If you need a review of what<br>
<code>alpha2</code>, <code>alpha4</code>, and <code>alpha3</code> are please review the <code>?chaid</code> doco.</p>
<p>You’ll notice that I stored the results in an object called<br>
<code>chaid.model</code>. That object has lots of useful information you can access<br>
(it’s a <code>list</code> object of class “train”). As a matter of fact we will be<br>
creating one object per run and then using the stored information to<br>
build a nice comparison later. For now here are some useful examples of<br>
what’s contained in the object…</p>
<ol>
<li>Produce the <code>confusionMatrix</code> across all folds<br>
<code>confusionMatrix(chaid.model)</code></li>
<li>Plot the effect of the tuning parameters on accuracy<br>
<code>plot(chaid.model)</code>. Note that the scaling deceives the eye and the<br>
results are close across the plot</li>
<li>Check on variable importance <code>varImp(chaid.model)</code></li>
<li>How long did it take? Look in <code>chaid.model$times</code></li>
</ol>
<p>If you need a refresher on what these represent <a rel="nofollow" href="https://ibecav.github.io/chaidtutor1/" target="_blank">please see the earlier<br>
post on CHAID</a>.</p>
<div>
<div>
<pre><code><span>confusionMatrix</span><span>(</span><span>chaid.model</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Cross-Validated (5 fold) Confusion Matrix 
## 
## (entries are percentual average cell counts across resamples)
##  
##           Reference
## Prediction   No  Yes
##        No  66.2 14.3
##        Yes  7.2 12.2
##                             
##  Accuracy (average) : 0.7844
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>plot</span><span>(</span><span>chaid.model</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="" src="https://i0.wp.com/ibecav.github.io/images/crx18-1.png?w=456" data-recalc-dims="1"></p>
<div>
<div>
<pre><code><span>varImp</span><span>(</span><span>chaid.model</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## ROC curve variable importance
## 
##                       Importance
## Contract                100.0000
## tenure_factor            95.7457
## OnlineSecurity           81.2110
## TechSupport              74.1473
## TotalCharges_factor      57.0885
## MonthlyCharges_factor    53.3755
## OnlineBackup             48.3382
## DeviceProtection         45.3815
## PaperlessBilling         44.2540
## Partner                  36.7387
## Dependents               35.4481
## PaymentMethod            29.7699
## SeniorCitizen            22.5287
## StreamingMovies          10.3811
## MultipleLines             9.1536
## InternetService           7.8258
## StreamingTV               6.7502
## gender                    0.6105
## PhoneService              0.0000
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>chaid.model</span><span>$</span><span>times</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## $everything
##    user  system elapsed 
## 131.788   1.824 133.819 
## 
## $final
##    user  system elapsed 
##   1.393   0.028   1.425 
## 
## $prediction
## [1] NA NA NA
</code></pre>
</div>
</div>
<p>One of the nice aspects about CHAID as a method is that is relatively<br>
easy to “see”” your model in either text or plot format. While there are<br>
packages that will help you “see” a random forest; by definition (pardon<br>
the pun) it’s hard to see the forest because of all the trees. Simply<br>
“printing” the final model with <code>chaid.model$finalModel</code> gives you the<br>
text representation while you can plot the final model with<br>
<code>plot(chaid.model$finalModel)</code>. As I explained in the earlier post it’s<br>
nice being able to see where your model fits well and where it misses at<br>
a high level.</p>
<div>
<div>
<pre><code><span>chaid.model</span><span>$</span><span>finalModel</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## 
## Model formula:
## .outcome ~ gender + SeniorCitizen + Partner + Dependents + PhoneService + 
##     MultipleLines + InternetService + OnlineSecurity + OnlineBackup + 
##     DeviceProtection + TechSupport + StreamingTV + StreamingMovies + 
##     Contract + PaperlessBilling + PaymentMethod + tenure_factor + 
##     MonthlyCharges_factor + TotalCharges_factor
## 
## Fitted party:
## [1] root
## |   [2] Contract in Month-to-month
## |   |   [3] InternetService in DSL
## |   |   |   [4] TotalCharges_factor in Lowest: No (n = 326, err = 49.7%)
## |   |   |   [5] TotalCharges_factor in Below Middle: No (n = 255, err = 27.5%)
## |   |   |   [6] TotalCharges_factor in Middle, Above Middle, Highest: No (n = 282, err = 15.2%)
## |   |   [7] InternetService in Fiber optic
## |   |   |   [8] tenure_factor in Lowest
## |   |   |   |   [9] OnlineSecurity in No, No internet service: Yes (n = 414, err = 25.4%)
## |   |   |   |   [10] OnlineSecurity in Yes: No (n = 33, err = 42.4%)
## |   |   |   [11] tenure_factor in Below Middle: Yes (n = 413, err = 41.4%)
## |   |   |   [12] tenure_factor in Middle: No (n = 352, err = 44.3%)
## |   |   |   [13] tenure_factor in Above Middle, Highest: No (n = 276, err = 31.9%)
## |   |   [14] InternetService in No: No (n = 378, err = 17.2%)
## |   [15] Contract in One year
## |   |   [16] StreamingMovies in No: No (n = 303, err = 8.3%)
## |   |   [17] StreamingMovies in No internet service: No (n = 268, err = 2.2%)
## |   |   [18] StreamingMovies in Yes: No (n = 457, err = 20.6%)
## |   [19] Contract in Two year
## |   |   [20] InternetService in DSL, No: No (n = 866, err = 1.2%)
## |   |   [21] InternetService in Fiber optic: No (n = 308, err = 8.1%)
## 
## Number of inner nodes:     7
## Number of terminal nodes: 14
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>plot</span><span>(</span><span>chaid.model</span><span>$</span><span>finalModel</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img src="https://i0.wp.com/ibecav.github.io/images/crx19-1.png?w=456" data-recalc-dims="1"></p>
<p>Finally, probably the most important step of all, we’ll take our trained<br>
model and apply it to the <code>testing</code> data that we held back to see how<br>
well it fits this data it’s never seen before. This is a key step<br>
because it reassures us that we have not <a rel="nofollow" href="https://elitedatascience.com/overfitting-in-machine-learning" target="_blank">overfit (if you want a fuller<br>
understanding please consider reading this post on EliteDataScience) our<br>
model</a>.<br>
We’ll take our model we made with the training dataset <code>chaid.model</code><br>
and have it predict against the testing dataset and see how we did with<br>
a <code>confusionMatrix</code></p>
<div>
<div>
<pre><code><span>confusionMatrix</span><span>(</span><span>predict</span><span>(</span><span>chaid.model</span><span>,</span><span> </span><span>newdata</span><span> </span><span>=</span><span> </span><span>testing</span><span>),</span><span> </span><span>testing</span><span>$</span><span>Churn</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Confusion Matrix and Statistics
## 
##           Reference
## Prediction   No  Yes
##        No  1443  331
##        Yes  109  229
##                                           
##                Accuracy : 0.7917          
##                  95% CI : (0.7737, 0.8088)
##     No Information Rate : 0.7348          
##     P-Value [Acc > NIR] : 7.793e-10       
##                                           
##                   Kappa : 0.3878          
##  Mcnemar's Test P-Value : &lt; 2.2e-16       
##                                           
##             Sensitivity : 0.9298          
##             Specificity : 0.4089          
##          Pos Pred Value : 0.8134          
##          Neg Pred Value : 0.6775          
##              Prevalence : 0.7348          
##          Detection Rate : 0.6832          
##    Detection Prevalence : 0.8400          
##       Balanced Accuracy : 0.6693          
##                                           
##        'Positive' Class : No              
## 
</code></pre>
</div>
</div>
<p>Very nice! Our accuracy on <code>testing</code> actually exceeds the accuracy we<br>
achieved in training.</p>
<h4>Random Forest via <code>ranger</code></h4>
<p>One of the nicest things about using <code>caret</code> is that it is pretty<br>
straight-forward to move from one model to another. The amount of work<br>
we have to do while moving from <code>CHAID</code> to <code>ranger</code> and eventually<br>
<code>xgboost</code> is actually quite modest.</p>
<p><code>ranger</code> will accept a mix of factors and numeric variables so our first<br>
step will be to go back and recreate training and testing using the<br>
numeric versions of <code>tenure</code>, <code>MonthlyCharges</code>, and <code>TotalCharges</code><br>
instead of the _factor versions. <code>intrain</code> still holds our list of rows<br>
that should be in <code>training</code> so we’ll follow the exact same process just<br>
keep the numeric versions and arrive at <code>x</code> and <code>y</code> to feed to <code>caret</code><br>
and <code>ranger</code>.</p>
<div>
<div>
<pre><code><span>##### using ranger</span><span>
</span><span># intrain &lt;-createDataPartition(churn$Churn,p=0.7,list=FALSE)</span><span>
</span><span>training</span><span> </span><span>&lt;-</span><span> </span><span>churn</span><span>[</span><span>intrain</span><span>,]</span><span>
</span><span>testing</span><span> </span><span>&lt;-</span><span> </span><span>churn</span><span>[</span><span>-</span><span>intrain</span><span>,]</span><span>
</span><span>dim</span><span>(</span><span>training</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 4931   23
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>dim</span><span>(</span><span>testing</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 2112   23
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>training</span><span> </span><span>&lt;-</span><span> </span><span>training</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>-</span><span>ends_with</span><span>(</span><span>&quot;_factor&quot;</span><span>))</span><span>
</span><span>dim</span><span>(</span><span>training</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 4931   20
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span># testing &lt;-testing %>%</span><span>
</span><span># select(-ends_with(&quot;_factor&quot;))</span><span>
</span><span>dim</span><span>(</span><span>testing</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] 2112   23
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span># create response and feature data</span><span>
</span><span>features</span><span> </span><span>&lt;-</span><span> </span><span>setdiff</span><span>(</span><span>names</span><span>(</span><span>training</span><span>),</span><span> </span><span>&quot;Churn&quot;</span><span>)</span><span>
</span><span>x</span><span> </span><span>&lt;-</span><span> </span><span>training</span><span>[,</span><span> </span><span>features</span><span>]</span><span>
</span><span>y</span><span> </span><span>&lt;-</span><span> </span><span>training</span><span>$</span><span>Churn</span><span>
</span></code></pre>
</div>
</div>
<p>As I mentioned earlier <code>train_control</code> doesn’t have to change at all. So<br>
I’ll just print it to remind you of what’s in there.</p>
<p><code>search_grid</code> is almost always specific to the model and this is no<br>
exception. When we consult <a rel="nofollow" href="https://topepo.github.io/caret/train-models-by-tag.html#random-forest" target="_blank">the documentation for ranger within<br>
caret</a><br>
we see that we can adjust <code>mtry</code>, <code>splitrule</code>, and <code>min.node.size</code>.<br>
We’ll put in some reasonable values for those and then put the<br>
resulting grid into rf_grid. I tried to give ranger’s search grid about<br>
the same amount of flexibility as I did for CHAID.</p>
<div>
<div>
<pre><code><span>##### reusing train_control</span><span>
</span><span>head</span><span>(</span><span>train_control</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## $method
## [1] &quot;cv&quot;
## 
## $number
## [1] 5
## 
## $repeats
## [1] NA
## 
## $search
## [1] &quot;grid&quot;
## 
## $p
## [1] 0.75
## 
## $initialWindow
## NULL
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span># define a grid of parameter options to try with ranger</span><span>
</span><span>rf_grid</span><span> </span><span>&lt;-</span><span> </span><span>expand.grid</span><span>(</span><span>mtry</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>2</span><span>:</span><span>4</span><span>),</span><span>
</span><span>splitrule</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>&quot;gini&quot;</span><span>),</span><span>
</span><span>min.node.size</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>3</span><span>,</span><span> </span><span>5</span><span>,</span><span> </span><span>7</span><span>))</span><span>
</span><span>rf_grid</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##   mtry splitrule min.node.size
## 1    2      gini             3
## 2    3      gini             3
## 3    4      gini             3
## 4    2      gini             5
## 5    3      gini             5
## 6    4      gini             5
## 7    2      gini             7
## 8    3      gini             7
## 9    4      gini             7
</code></pre>
</div>
</div>
<p>Okay, we’re ready to train our model using ranger now. The only<br>
additional line we need (besides changing from <code>chaid</code> to <code>ranger</code> is to<br>
tell it what to use to capture variable importance e.g. “impurity”.</p>
<div>
<div>
<pre><code><span># re-fit the model with the parameter grid</span><span>
</span><span>rf.model</span><span> </span><span>&lt;-</span><span> </span><span>train</span><span>(</span><span>
</span><span>x</span><span> </span><span>=</span><span> </span><span>x</span><span>,</span><span>
</span><span>y</span><span> </span><span>=</span><span> </span><span>y</span><span>,</span><span>
</span><span>method</span><span> </span><span>=</span><span> </span><span>&quot;ranger&quot;</span><span>,</span><span>
</span><span>trControl</span><span> </span><span>=</span><span> </span><span>train_control</span><span>,</span><span>
</span><span>tuneGrid</span><span> </span><span>=</span><span> </span><span>rf_grid</span><span>,</span><span>
</span><span>importance</span><span> </span><span>=</span><span> </span><span>&quot;impurity&quot;</span><span>)</span><span>
</span><span>rf.model</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Random Forest 
## 
## 4931 samples
##   19 predictor
##    2 classes: 'No', 'Yes' 
## 
## No pre-processing
## Resampling: Cross-Validated (5 fold) 
## Summary of sample sizes: 3946, 3944, 3945, 3944, 3945 
## Resampling results across tuning parameters:
## 
##   mtry  min.node.size  Accuracy   Kappa    
##   2     3              0.7990299  0.4368130
##   2     5              0.8000437  0.4380797
##   2     7              0.7976106  0.4298171
##   3     3              0.7963928  0.4342273
##   3     5              0.7980151  0.4365761
##   3     7              0.7986242  0.4360132
##   4     3              0.7970009  0.4343468
##   4     5              0.7982171  0.4358659
##   4     7              0.7943648  0.4228518
## 
## Tuning parameter 'splitrule' was held constant at a value of gini
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were mtry = 2, splitrule = gini and min.node.size = 5.
</code></pre>
</div>
</div>
<p>Now we can run the exact same set of commands as we did with chaid.model<br>
on rf.model.</p>
<div>
<div>
<pre><code><span>confusionMatrix</span><span>(</span><span>rf.model</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Cross-Validated (5 fold) Confusion Matrix 
## 
## (entries are percentual average cell counts across resamples)
##  
##           Reference
## Prediction   No  Yes
##        No  67.1 13.6
##        Yes  6.4 12.9
##                          
##  Accuracy (average) : 0.8
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>plot</span><span>(</span><span>rf.model</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="" src="https://i0.wp.com/ibecav.github.io/images/crx24-1.png?w=456" data-recalc-dims="1"></p>
<div>
<div>
<pre><code><span>varImp</span><span>(</span><span>rf.model</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## ranger variable importance
## 
##                  Overall
## tenure           100.000
## TotalCharges      97.336
## MonthlyCharges    88.321
## Contract          66.882
## OnlineSecurity    43.533
## TechSupport       37.978
## PaymentMethod     34.685
## InternetService   29.435
## OnlineBackup      18.389
## DeviceProtection  14.376
## PaperlessBilling  13.363
## MultipleLines     10.114
## Partner            9.712
## StreamingTV        9.607
## StreamingMovies    8.183
## gender             7.998
## Dependents         7.679
## SeniorCitizen      7.314
## PhoneService       0.000
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>rf.model</span><span>$</span><span>times</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## $everything
##    user  system elapsed 
## 110.338   1.543  18.646 
## 
## $final
##    user  system elapsed 
##   2.205   0.023   0.346 
## 
## $prediction
## [1] NA NA NA
</code></pre>
</div>
</div>
<p>Now, the all important prediction against the testing data set.</p>
<div>
<div>
<pre><code><span>confusionMatrix</span><span>(</span><span>predict</span><span>(</span><span>rf.model</span><span>,</span><span> </span><span>newdata</span><span> </span><span>=</span><span> </span><span>testing</span><span>),</span><span> </span><span>testing</span><span>$</span><span>Churn</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Confusion Matrix and Statistics
## 
##           Reference
## Prediction   No  Yes
##        No  1427  286
##        Yes  125  274
##                                           
##                Accuracy : 0.8054          
##                  95% CI : (0.7879, 0.8221)
##     No Information Rate : 0.7348          
##     P-Value [Acc > NIR] : 2.048e-14       
##                                           
##                   Kappa : 0.4501          
##  Mcnemar's Test P-Value : 2.969e-15       
##                                           
##             Sensitivity : 0.9195          
##             Specificity : 0.4893          
##          Pos Pred Value : 0.8330          
##          Neg Pred Value : 0.6867          
##              Prevalence : 0.7348          
##          Detection Rate : 0.6757          
##    Detection Prevalence : 0.8111          
##       Balanced Accuracy : 0.7044          
##                                           
##        'Positive' Class : No              
## 
</code></pre>
</div>
</div>
<p>Very nice! Once again our accuracy on testing actually exceeds the<br>
accuracy we achieved in training. Looks like we were more accurate than<br>
CHAID but we’ll come back to that after we finish xgboost.</p>
<h4>Extreme Gradient Boosting via <code>xgboost</code></h4>
<p>Moving from <code>ranger</code> to <code>xgboost</code> is even easier than it was from<br>
<code>CHAID</code>.</p>
<p><code>xgboost</code> like <code>ranger</code> will accept a mix of factors and numeric<br>
variables so there is no need to change our training and testing<br>
datasets at all. There’s also no need to change our <code>train_control</code>. As<br>
far as tuning goes <code>caret</code> supports 7 of the <a rel="nofollow" href="http://xgboost.readthedocs.io/en/latest/parameter.html" target="_blank">many<br>
parameters</a> that<br>
you could feed to <code>?xgboost</code>. If you <a rel="nofollow" href="https://topepo.github.io/caret/train-models-by-tag.html#" target="_blank">consult the <code>caret</code> documentation<br>
here</a> under<br>
<code>xgbTree</code> you’ll see them listed. If you don’t provide any tuning<br>
guidance then it will provide a default set of pretty rational initial<br>
values. I initially ran it that way but below for purposes of this post<br>
have chosen only a few that seem to make the largest difference to<br>
accuracy and set the rest to a constant.</p>
<p>One final important note about the code below. Notice in the train<br>
command I am feeding a formula <code>Churn ~ .</code> to <code>train</code>. If you try to<br>
give it the same x = x &amp; y = y syntax I used with <code>ranger</code> it will fail.<br>
That’s because as stated in the doco “xgb.train accepts only an<br>
xgb.DMatrix as the input. xgboost, in addition, also accepts matrix,<br>
dgCMatrix, or name of a local data file.” You could use commands like<br>
<code>xx &lt;- model.matrix(~. -1, data=x)[,-1]</code> &amp; <code>yy &lt;- as.numeric(y) -1</code> to<br>
convert them but since our dataset is small I’m just going to use the<br>
formula interface.</p>
<div>
<div>
<pre><code><span># reusing train_control</span><span>
</span><span>head</span><span>(</span><span>train_control</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## $method
## [1] &quot;cv&quot;
## 
## $number
## [1] 5
## 
## $repeats
## [1] NA
## 
## $search
## [1] &quot;grid&quot;
## 
## $p
## [1] 0.75
## 
## $initialWindow
## NULL
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span># define a grid of parameter options to try with xgboost</span><span>
</span><span>xgb_grid</span><span> </span><span>&lt;-</span><span> </span><span>expand.grid</span><span>(</span><span>nrounds</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>100</span><span>,</span><span> </span><span>150</span><span>,</span><span> </span><span>200</span><span>),</span><span>
</span><span>max_depth</span><span> </span><span>=</span><span> </span><span>1</span><span>,</span><span>
</span><span>min_child_weight</span><span> </span><span>=</span><span> </span><span>1</span><span>,</span><span>
</span><span>subsample</span><span> </span><span>=</span><span> </span><span>1</span><span>,</span><span>
</span><span>gamma</span><span> </span><span>=</span><span> </span><span>0</span><span>,</span><span>
</span><span>colsample_bytree</span><span> </span><span>=</span><span> </span><span>0.8</span><span>,</span><span>
</span><span>eta</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>.2</span><span>,</span><span> </span><span>.3</span><span>,</span><span> </span><span>.4</span><span>))</span><span>
</span><span>xgb_grid</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##   nrounds max_depth min_child_weight subsample gamma colsample_bytree eta
## 1     100         1                1         1     0              0.8 0.2
## 2     150         1                1         1     0              0.8 0.2
## 3     200         1                1         1     0              0.8 0.2
## 4     100         1                1         1     0              0.8 0.3
## 5     150         1                1         1     0              0.8 0.3
## 6     200         1                1         1     0              0.8 0.3
## 7     100         1                1         1     0              0.8 0.4
## 8     150         1                1         1     0              0.8 0.4
## 9     200         1                1         1     0              0.8 0.4
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span># Fit the model with the parameter grid</span><span>
</span><span>xgboost.model</span><span> </span><span>&lt;-</span><span> </span><span>train</span><span>(</span><span>Churn</span><span> </span><span>~</span><span> </span><span>.</span><span>,</span><span> </span><span>training</span><span> </span><span>,</span><span> </span><span>method</span><span> </span><span>=</span><span> </span><span>&quot;xgbTree&quot;</span><span>,</span><span> </span><span>tuneGrid</span><span> </span><span>=</span><span> </span><span>xgb_grid</span><span>,</span><span>
</span><span>trControl</span><span> </span><span>=</span><span> </span><span>train_control</span><span>)</span><span>
</span><span>xgboost.model</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## eXtreme Gradient Boosting 
## 
## 4931 samples
##   19 predictor
##    2 classes: 'No', 'Yes' 
## 
## No pre-processing
## Resampling: Cross-Validated (5 fold) 
## Summary of sample sizes: 3945, 3944, 3945, 3944, 3946 
## Resampling results across tuning parameters:
## 
##   eta  nrounds  Accuracy   Kappa    
##   0.2  100      0.7974010  0.4272991
##   0.2  150      0.7998353  0.4414212
##   0.2  200      0.8002400  0.4465105
##   0.3  100      0.8002410  0.4449462
##   0.3  150      0.8020667  0.4532478
##   0.3  200      0.8028779  0.4565789
##   0.4  100      0.8010528  0.4478758
##   0.4  150      0.8020655  0.4530151
##   0.4  200      0.8024716  0.4546512
## 
## Tuning parameter 'max_depth' was held constant at a value of 1
## Tuning parameter 'gamma' was held constant at a value of 0
## Tuning parameter 'colsample_bytree' was held constant at a value of 0.8
## Tuning parameter 'min_child_weight' was held constant at a value of 1
## Tuning parameter 'subsample' was held constant at a value of 1
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were nrounds = 200, max_depth = 1, eta = 0.3, gamma = 0, colsample_bytree = 0.8, min_child_weight = 1 and subsample = 1.
</code></pre>
</div>
</div>
<p>After a (relatively) brief moment the results are back. Average accuracy<br>
on the <code>training</code> is .8029 which is better than <code>CHAID</code> or <code>ranger</code>. We<br>
can run the same additional commands simply by listing <code>xgboost.model</code>.</p>
<div>
<div>
<pre><code><span>confusionMatrix</span><span>(</span><span>xgboost.model</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Cross-Validated (5 fold) Confusion Matrix 
## 
## (entries are percentual average cell counts across resamples)
##  
##           Reference
## Prediction   No  Yes
##        No  66.5 12.7
##        Yes  7.0 13.8
##                             
##  Accuracy (average) : 0.8029
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>plot</span><span>(</span><span>xgboost.model</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="" src="https://i0.wp.com/ibecav.github.io/images/crx27-1.png?w=456" data-recalc-dims="1"></p>
<div>
<div>
<pre><code><span>varImp</span><span>(</span><span>xgboost.model</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## xgbTree variable importance
## 
##   only 20 most important variables shown (out of 30)
## 
##                                      Overall
## tenure                             100.00000
## InternetServiceFiber optic          64.17202
## ContractTwo year                    48.23551
## InternetServiceNo                   26.81070
## ContractOne year                    10.38092
## OnlineSecurityYes                    7.23882
## StreamingTVYes                       5.04154
## MultipleLinesYes                     2.87755
## StreamingMoviesYes                   2.74796
## TechSupportYes                       2.14009
## PhoneServiceYes                      1.52273
## DependentsYes                        1.33244
## SeniorCitizenYes                     0.50710
## genderMale                           0.23151
## DeviceProtectionYes                  0.03949
## MonthlyCharges                       0.00000
## MultipleLinesNo phone service        0.00000
## StreamingMoviesNo internet service   0.00000
## OnlineBackupNo internet service      0.00000
## PartnerYes                           0.00000
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span>xgboost.model</span><span>$</span><span>times</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## $everything
##    user  system elapsed 
##   4.635   0.038   4.681 
## 
## $final
##    user  system elapsed 
##   0.263   0.002   0.266 
## 
## $prediction
## [1] NA NA NA
</code></pre>
</div>
</div>
<p>Now, the all important prediction against the testing data<br>
set.</p>
<div>
<div>
<pre><code><span>confusionMatrix</span><span>(</span><span>predict</span><span>(</span><span>xgboost.model</span><span>,</span><span> </span><span>newdata</span><span> </span><span>=</span><span> </span><span>testing</span><span>),</span><span> </span><span>testing</span><span>$</span><span>Churn</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Confusion Matrix and Statistics
## 
##           Reference
## Prediction   No  Yes
##        No  1405  262
##        Yes  147  298
##                                          
##                Accuracy : 0.8063         
##                  95% CI : (0.7888, 0.823)
##     No Information Rate : 0.7348         
##     P-Value [Acc > NIR] : 9.038e-15      
##                                          
##                   Kappa : 0.4682         
##  Mcnemar's Test P-Value : 1.731e-08      
##                                          
##             Sensitivity : 0.9053         
##             Specificity : 0.5321         
##          Pos Pred Value : 0.8428         
##          Neg Pred Value : 0.6697         
##              Prevalence : 0.7348         
##          Detection Rate : 0.6652         
##    Detection Prevalence : 0.7893         
##       Balanced Accuracy : 0.7187         
##                                          
##        'Positive' Class : No             
## 
</code></pre>
</div>
</div>
<p>Very nice! Once again our accuracy on testing .8063 actually exceeds<br>
the accuracy we achieved in training. Looks like we were more accurate<br>
than either CHAID or ranger and we’ll focus on the comparison in the<br>
next section.</p>
<h2>Comparing Models</h2>
<p>At this juncture we’re faced with a problem I’ve had before. We’re<br>
drowning in data from the individual <code>confusionMatrix</code> results. <a rel="nofollow" href="http://ibecav.github.io/chaidrangerxgboost/" target="_blank">We’ll<br>
resort to the same <code>purrr</code><br>
solution</a> to give us a far<br>
more legible table of results focusing on the metrics I’m most<br>
interested in. To do that we need to:</p>
<ol>
<li>Make a <code>named list</code> called <code>modellist</code> that contains our 3 models<br>
with a descriptive name for each</li>
<li>Use <code>map</code> from <code>purrr</code> to apply the <code>predict</code> command to each model<br>
in turn to our <code>testing</code> dataset</li>
<li>Pipe those results to a second <code>map</code> command to generate a confusion<br>
matrix comparing our predictions to <code>testing$Churn</code> which are the<br>
actual outcomes.</li>
<li>Pipe those results to a complex <code>map_dfr</code> (<a rel="nofollow" href="https://ibecav.github.io/chaidtutor1/" target="_blank">that I explained<br>
previously</a>) that creates a<br>
dataframe of all the results with each model as a row.</li>
<li>Separately grab the elapsed times for training with commands like<br>
<code>chaid.model$times$everything[[3]]</code></li>
<li>Separately grab the best accuracy for training with commands like<br>
<code>max(chaid.model$results$Accuracy)</code></li>
<li>Then use <code>kable</code> to make a pretty table that is much easier to<br>
understand.</li>
</ol>
<p></p>
<div>
<div>
<pre><code><span>modellist</span><span> </span><span>&lt;-</span><span> </span><span>list</span><span>(</span><span>&quot;CHAID&quot;</span><span> </span><span>=</span><span> </span><span>chaid.model</span><span>,</span><span>
</span><span>&quot;ranger&quot;</span><span> </span><span>=</span><span> </span><span>rf.model</span><span>,</span><span>
</span><span>&quot;xgboost&quot;</span><span> </span><span>=</span><span> </span><span>xgboost.model</span><span>)</span><span>
</span><span>CompareResults</span><span> </span><span>&lt;-</span><span> </span><span>map</span><span>(</span><span>modellist</span><span>,</span><span> </span><span>~</span><span> </span><span>predict</span><span>(</span><span>.x</span><span>,</span><span> </span><span>newdata</span><span> </span><span>=</span><span> </span><span>testing</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>map</span><span>(</span><span>~</span><span> </span><span>confusionMatrix</span><span>(</span><span>testing</span><span>$</span><span>Churn</span><span>,</span><span> </span><span>.x</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>map_dfr</span><span>(</span><span>~</span><span> </span><span>cbind</span><span>(</span><span>as.data.frame</span><span>(</span><span>t</span><span>(</span><span>.x</span><span>$</span><span>overall</span><span>)),</span><span> </span><span>as.data.frame</span><span>(</span><span>t</span><span>(</span><span>.x</span><span>$</span><span>byClass</span><span>))),</span><span> </span><span>.id</span><span> </span><span>=</span><span> </span><span>&quot;Model&quot;</span><span>)</span><span>
</span><span>CompareResults</span><span>[</span><span>1</span><span>,</span><span>&quot;ETime&quot;</span><span>]</span><span> </span><span>&lt;-</span><span> </span><span>chaid.model</span><span>$</span><span>times</span><span>$</span><span>everything</span><span>[[</span><span>3</span><span>]]</span><span>
</span><span>CompareResults</span><span>[</span><span>2</span><span>,</span><span>&quot;ETime&quot;</span><span>]</span><span> </span><span>&lt;-</span><span> </span><span>rf.model</span><span>$</span><span>times</span><span>$</span><span>everything</span><span>[[</span><span>3</span><span>]]</span><span>
</span><span>CompareResults</span><span>[</span><span>3</span><span>,</span><span>&quot;ETime&quot;</span><span>]</span><span> </span><span>&lt;-</span><span> </span><span>xgboost.model</span><span>$</span><span>times</span><span>$</span><span>everything</span><span>[[</span><span>3</span><span>]]</span><span>
</span><span>CompareResults</span><span>[</span><span>1</span><span>,</span><span>&quot;BestTrain&quot;</span><span>]</span><span> </span><span>&lt;-</span><span> </span><span>max</span><span>(</span><span>chaid.model</span><span>$</span><span>results</span><span>$</span><span>Accuracy</span><span>)</span><span>
</span><span>CompareResults</span><span>[</span><span>2</span><span>,</span><span>&quot;BestTrain&quot;</span><span>]</span><span> </span><span>&lt;-</span><span> </span><span>max</span><span>(</span><span>rf.model</span><span>$</span><span>results</span><span>$</span><span>Accuracy</span><span>)</span><span>
</span><span>CompareResults</span><span>[</span><span>3</span><span>,</span><span>&quot;BestTrain&quot;</span><span>]</span><span> </span><span>&lt;-</span><span> </span><span>max</span><span>(</span><span>xgboost.model</span><span>$</span><span>results</span><span>$</span><span>Accuracy</span><span>)</span><span>
</span><span>kable</span><span>(</span><span>CompareResults</span><span>,</span><span> </span><span>&quot;html&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>kable_styling</span><span>(</span><span>bootstrap_options</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>&quot;striped&quot;</span><span>,</span><span> </span><span>&quot;hover&quot;</span><span>,</span><span> </span><span>&quot;condensed&quot;</span><span>,</span><span> </span><span>&quot;responsive&quot;</span><span>))</span><span>
</span></code></pre>
</div>
</div>
<table>
<thead>
<tr>
<th>
<p>Model</p>
</th>
<th>
<p>Accuracy</p>
</th>
<th>
<p>Kappa</p>
</th>
<th>
<p>AccuracyLower</p>
</th>
<th>
<p>AccuracyUpper</p>
</th>
<th>
<p>AccuracyNull</p>
</th>
<th>
<p>AccuracyPValue</p>
</th>
<th>
<p>McnemarPValue</p>
</th>
<th>
<p>Sensitivity</p>
</th>
<th>
<p>Specificity</p>
</th>
<th>
<p>Pos Pred Value</p>
</th>
<th>
<p>Neg Pred Value</p>
</th>
<th>
<p>Precision</p>
</th>
<th>
<p>Recall</p>
</th>
<th>
<p>F1</p>
</th>
<th>
<p>Prevalence</p>
</th>
<th>
<p>Detection Rate</p>
</th>
<th>
<p>Detection Prevalence</p>
</th>
<th>
<p>Balanced Accuracy</p>
</th>
<th>
<p>ETime</p>
</th>
<th>
<p>BestTrain</p>
</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<p>CHAID</p>
</td>
<td>
<p>0.7916667</p>
</td>
<td>
<p>0.3878325</p>
</td>
<td>
<p>0.7737107</p>
</td>
<td>
<p>0.8088094</p>
</td>
<td>
<p>0.8399621</p>
</td>
<td>
<p>1.0000000</p>
</td>
<td>
<p>0</p>
</td>
<td>
<p>0.8134160</p>
</td>
<td>
<p>0.6775148</p>
</td>
<td>
<p>0.9297680</p>
</td>
<td>
<p>0.4089286</p>
</td>
<td>
<p>0.9297680</p>
</td>
<td>
<p>0.8134160</p>
</td>
<td>
<p>0.8677090</p>
</td>
<td>
<p>0.8399621</p>
</td>
<td>
<p>0.6832386</p>
</td>
<td>
<p>0.7348485</p>
</td>
<td>
<p>0.7454654</p>
</td>
<td>
<p>133.819</p>
</td>
<td>
<p>0.7844263</p>
</td>
</tr>
<tr>
<td>
<p>ranger</p>
</td>
<td>
<p>0.8053977</p>
</td>
<td>
<p>0.4501003</p>
</td>
<td>
<p>0.7878577</p>
</td>
<td>
<p>0.8220858</p>
</td>
<td>
<p>0.8110795</p>
</td>
<td>
<p>0.7573566</p>
</td>
<td>
<p>0</p>
</td>
<td>
<p>0.8330414</p>
</td>
<td>
<p>0.6867168</p>
</td>
<td>
<p>0.9194588</p>
</td>
<td>
<p>0.4892857</p>
</td>
<td>
<p>0.9194588</p>
</td>
<td>
<p>0.8330414</p>
</td>
<td>
<p>0.8741194</p>
</td>
<td>
<p>0.8110795</p>
</td>
<td>
<p>0.6756629</p>
</td>
<td>
<p>0.7348485</p>
</td>
<td>
<p>0.7598791</p>
</td>
<td>
<p>18.646</p>
</td>
<td>
<p>0.8000437</p>
</td>
</tr>
<tr>
<td>
<p>xgboost</p>
</td>
<td>
<p>0.8063447</p>
</td>
<td>
<p>0.4681509</p>
</td>
<td>
<p>0.7888346</p>
</td>
<td>
<p>0.8230002</p>
</td>
<td>
<p>0.7892992</p>
</td>
<td>
<p>0.0281863</p>
</td>
<td>
<p>0</p>
</td>
<td>
<p>0.8428314</p>
</td>
<td>
<p>0.6696629</p>
</td>
<td>
<p>0.9052835</p>
</td>
<td>
<p>0.5321429</p>
</td>
<td>
<p>0.9052835</p>
</td>
<td>
<p>0.8428314</p>
</td>
<td>
<p>0.8729419</p>
</td>
<td>
<p>0.7892992</p>
</td>
<td>
<p>0.6652462</p>
</td>
<td>
<p>0.7348485</p>
</td>
<td>
<p>0.7562472</p>
</td>
<td>
<p>4.681</p>
</td>
<td>
<p>0.8028779</p>
</td>
</tr>
</tbody>
</table>
<h2>What do we know?</h2>
<p>Well our table looks very nice but there’s probably still too much<br>
information. What data should we focus on and what conclusions can we<br>
draw from our little exercise in comparative modeling? I will draw your<br>
attention back to this webpage to <a rel="nofollow" href="https://www.dataschool.io/simple-guide-to-confusion-matrix-terminology/" target="_blank">review the terminology for<br>
classification models and how to interpret a confusion<br>
matrix</a>.</p>
<p>So <code>Accuracy</code>, <code>Kappa</code>, and <code>F1</code> are all measures of overall accuracy.<br>
There are merits to each. <code>Pos Pred Value</code>, and <code>Neg Pred Value</code> are<br>
related but different nuanced ideas we’ll discuss in a minute. We’ll<br>
also want to talk about time to complete training our model with <code>ETime</code><br>
and training accuracy with <code>BestTrain</code>.</p>
<p>Let’s use <code>dplyr</code> to select just these columns we want and see what we<br>
can glean from this reduced table.</p>
<div>
<div>
<pre><code><span>CompareResults</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>Model</span><span>,</span><span> </span><span>ETime</span><span>,</span><span> </span><span>BestTrain</span><span>,</span><span> </span><span>Accuracy</span><span>,</span><span> </span><span>Kappa</span><span>,</span><span> </span><span>F</span><span>1</span><span>,</span><span> </span><span>'Pos Pred Value'</span><span>,</span><span> </span><span>'Neg Pred Value'</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>kable</span><span>(</span><span>&quot;html&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>kable_styling</span><span>(</span><span>bootstrap_options</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>&quot;striped&quot;</span><span>,</span><span> </span><span>&quot;hover&quot;</span><span>,</span><span> </span><span>&quot;condensed&quot;</span><span>,</span><span> </span><span>&quot;responsive&quot;</span><span>))</span><span>
</span></code></pre>
</div>
</div>
<table>
<thead>
<tr>
<th>
<p>Model</p>
</th>
<th>
<p>ETime</p>
</th>
<th>
<p>BestTrain</p>
</th>
<th>
<p>Accuracy</p>
</th>
<th>
<p>Kappa</p>
</th>
<th>
<p>F1</p>
</th>
<th>
<p>Pos Pred Value</p>
</th>
<th>
<p>Neg Pred Value</p>
</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<p>CHAID</p>
</td>
<td>
<p>133.819</p>
</td>
<td>
<p>0.7844263</p>
</td>
<td>
<p>0.7916667</p>
</td>
<td>
<p>0.3878325</p>
</td>
<td>
<p>0.8677090</p>
</td>
<td>
<p>0.9297680</p>
</td>
<td>
<p>0.4089286</p>
</td>
</tr>
<tr>
<td>
<p>ranger</p>
</td>
<td>
<p>18.646</p>
</td>
<td>
<p>0.8000437</p>
</td>
<td>
<p>0.8053977</p>
</td>
<td>
<p>0.4501003</p>
</td>
<td>
<p>0.8741194</p>
</td>
<td>
<p>0.9194588</p>
</td>
<td>
<p>0.4892857</p>
</td>
</tr>
<tr>
<td>
<p>xgboost</p>
</td>
<td>
<p>4.681</p>
</td>
<td>
<p>0.8028779</p>
</td>
<td>
<p>0.8063447</p>
</td>
<td>
<p>0.4681509</p>
</td>
<td>
<p>0.8729419</p>
</td>
<td>
<p>0.9052835</p>
</td>
<td>
<p>0.5321429</p>
</td>
</tr>
</tbody>
</table>
<p>Clearly <code>xgboost</code> is the fastest to train a model, more than 30 times<br>
faster than <code>CHAID</code>, and 3 times faster than ranger for this data. Not<br>
really surprising since <a rel="nofollow" href="https://github.com/dmlc/xgboost" target="_blank">xgboost is a very modern set of code designed<br>
from the ground up to be fast and<br>
efficient</a>.</p>
<p>One interesting fact you can glean from all 3 models is that they all<br>
did better on <code>testing</code> than they did on <code>training</code>. This is slightly<br>
unusual since one would expect some differences to be missed but is<br>
likely simply due to a lucky split in our data with more of the<br>
difficult to predict cases falling in <code>training</code> than <code>testing</code>. The<br>
good news is it leaves us feeling comfortable that we did not overfit<br>
our model to the training data, which is why we were conservative in our<br>
fitting and cross validated the training data.</p>
<p>No matter which “accuracy measure” we look at <code>Accuracy</code>, <code>F1</code> or<br>
<code>Kappa</code> the answer is pretty consistent, <code>xgboost</code> “wins” or is the most<br>
accurate. The exception is <code>F1</code> where <code>ranger</code> edges is out by 0.11775%<br>
which means it was correct on about 3 more cases out of 2112 cases in<br>
the <code>testing</code> set.</p>
<p>Notice that the differences in accuracy are not large as percentages<br>
<code>xgboost</code> is 1.4678% more accurate than <code>CHAID</code> or it correctly<br>
predicted 31 more customers. While more accurate is always “better” the<br>
practical significance is also a matter of what the stakes are. If a<br>
wrong prediction costs you $1,000.00 dollars that additional accuracy is<br>
more concerning than a lesser dollar amount.</p>
<p>I also deliberately included <a rel="nofollow" href="https://en.wikipedia.org/wiki/Positive_and_negative_predictive_values" target="_blank">Positive and Negative Predictive<br>
Values</a><br>
the columns labelled <code>Pos Pred Value</code> and <code>Neg Pred Value</code> for a very<br>
specific reason. Notice that <code>CHAID</code> has the highest <code>Pos Pred Value</code><br>
that means is is the most accurate at predicting customers who did not<br>
“churn”. Of the 1,552 customers who did not leave us is correctly<br>
predicted 1,443 of them. <code>xgboost</code> on the other hand was much much<br>
better at <code>Neg Pred Value</code> correctly predicting 298 out of 560 customers<br>
who left us. While <code>Accuracy</code>, <code>Kappa</code> and <code>F1</code> take different<br>
approaches to finding “balanced” accuracy sometimes one case negative or<br>
positive has more important implications for your business and you<br>
should choose those measures.</p>
<p>At least at this point after a possible tl;dr journey we have some<br>
empirical data to inform my original statement about CHAID: “As the name<br>
implies it is fundamentally based on the venerable Chi-square test – and<br>
while not the most powerful (in terms of detecting the smallest possible<br>
differences) or the fastest, it really is easy to manage and more<br>
importantly to tell the story after using it”.</p>
<h2>What don’t we know?</h2>
<ol>
<li>
<p><strong>That this example would apply to other types of datasets.</strong><br>
Absolutely not! This sort of data is almost ideal for CHAID since<br>
it involves a lot of nominal/categorical and/or ordinal data. CHAID<br>
will get much slower faster as we add more columns. More generally<br>
this was one example relatively small dataset more about learning<br>
something about <code>caret</code> and process than a true comparison of<br>
accuracy across a wide range of cases.</p>
</li>
<li>
<p><strong>This is the “best” these models can do with this data</strong> Absolutely<br>
not! I made no attempt to seriously tune any of them. Tried some<br>
mild comparability. Also made no effort to feature engineer or<br>
adjust. I’m pretty certain if you tried you can squeeze a little<br>
more out of all three. Even wth <code>CHAID</code> there’s more we could do<br>
very easily. I arbitrarily divided <code>tenure</code> into 5 equal sized bins.<br>
Why not 10? Why not equidistant instead of equal sized?</p>
</li>
</ol>
<h2>Done!</h2>
<p>I hope you’ve found this useful. I am always open to comments,<br>
corrections and suggestions.</p>
<p>Chuck (ibecav at gmail dot<br>
com)</p>
<h3>License</h3>
<p><a rel="nofollow" href="http://creativecommons.org/licenses/by-sa/4.0/" target="_blank"><img alt="Creative Commons License" src="https://i1.wp.com/i.creativecommons.org/l/by-sa/4.0/88x31.png?w=456&ssl=1" data-recalc-dims="1"></a><br>This<br>
work is licensed under a<br>
<a rel="nofollow" href="http://creativecommons.org/licenses/by-sa/4.0/" target="_blank">Creative<br>
Commons Attribution-ShareAlike 4.0 International License</a>.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://ibecav.github.io/chaidrangerxgboost/"> Chuck Powell</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/2gEopb4lygU" height="1"></a></div>