<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/explaining-predictions-of-machine-learning-models-with-lime-munster-data-science-meetup/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://shirinsplayground.netlify.com/2017/12/lime_sketchnotes/"> Shirin's playgRound</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<h2>Slides from Münster Data Science Meetup</h2>
<p><a rel="nofollow" href="http://res.cloudinary.com/shiring/image/upload/v1513090254/lime_meetup_slides_wvsh6s.pdf" target="_blank">These are my slides</a> from the <a rel="nofollow" href="https://www.meetup.com/Data-Science-Meetup-Muenster/events/244173239/" target="_blank">Münster Data Science Meetup on December 12th, 2017</a>.</p>
<p></p>
<p>My sketchnotes were collected from these two podcasts:</p>
<ul>
<li><a rel="nofollow" href="https://twimlai.com/twiml-talk-7-carlos-guestrin-explaining-predictions-machine-learning-models/" target="_blank">https://twimlai.com/twiml-talk-7-carlos-guestrin-explaining-predictions-machine-learning-models/</a></li>
<li><a rel="nofollow" href="https://dataskeptic.com/blog/episodes/2016/trusting-machine-learning-models-with-lime" target="_blank">https://dataskeptic.com/blog/episodes/2016/trusting-machine-learning-models-with-lime</a></li>
</ul>
<div>
<img alt="Sketchnotes: TWiML Talk #7 with Carlos Guestrin – Explaining the Predictions of Machine Learning Models &amp; Data Skeptic Podcast - Trusting Machine Learning Models with Lime" src="https://i0.wp.com/res.cloudinary.com/shiring/image/upload/v1512627737/lime_sketchnotes_guq6u5.jpg?w=456" data-recalc-dims="1">
<p>Sketchnotes: TWiML Talk #7 with Carlos Guestrin – Explaining the Predictions of Machine Learning Models &amp; Data Skeptic Podcast – Trusting Machine Learning Models with Lime</p>
</div>
<hr>
</div>
<div>
<h2>Example Code</h2>
<ul>
<li>the following libraries were loaded:</li>
</ul>
<pre><code>library(tidyverse)  # for tidy data analysis
library(farff)      # for reading arff file
library(missForest) # for imputing missing values
library(dummies)    # for creating dummy variables
library(caret)      # for modeling
library(lime)       # for explaining predictions</code></pre>
<div>
<h3>Data</h3>
<p>The Chronic Kidney Disease dataset was downloaded from UC Irvine’s Machine Learning repository: <a rel="nofollow" href="http://archive.ics.uci.edu/ml/datasets/Chronic_Kidney_Disease" target="_blank">http://archive.ics.uci.edu/ml/datasets/Chronic_Kidney_Disease</a></p>
<pre><code>data_file &lt;- file.path(&quot;path/to/chronic_kidney_disease_full.arff&quot;)</code></pre>
<ul>
<li>load data with the <code>farff</code> package</li>
</ul>
<pre><code>data &lt;- readARFF(data_file)</code></pre>
<div>
<h4>Features</h4>
<ul>
<li>age – age</li>
<li>bp – blood pressure</li>
<li>sg – specific gravity</li>
<li>al – albumin</li>
<li>su – sugar</li>
<li>rbc – red blood cells</li>
<li>pc – pus cell</li>
<li>pcc – pus cell clumps</li>
<li>ba – bacteria</li>
<li>bgr – blood glucose random</li>
<li>bu – blood urea</li>
<li>sc – serum creatinine</li>
<li>sod – sodium</li>
<li>pot – potassium</li>
<li>hemo – hemoglobin</li>
<li>pcv – packed cell volume</li>
<li>wc – white blood cell count</li>
<li>rc – red blood cell count</li>
<li>htn – hypertension</li>
<li>dm – diabetes mellitus</li>
<li>cad – coronary artery disease</li>
<li>appet – appetite</li>
<li>pe – pedal edema</li>
<li>ane – anemia</li>
<li>class – class</li>
</ul>
</div>
</div>
<div>
<h3>Missing data</h3>
<ul>
<li>impute missing data with Nonparametric Missing Value Imputation using Random Forest (<code>missForest</code> package)</li>
</ul>
<pre><code>data_imp &lt;- missForest(data)</code></pre>
</div>
<div>
<h3>One-hot encoding</h3>
<ul>
<li>create dummy variables (<code>caret::dummy.data.frame()</code>)</li>
<li>scale and center</li>
</ul>
<pre><code>data_imp_final &lt;- data_imp$ximp
data_dummy &lt;- dummy.data.frame(dplyr::select(data_imp_final, -class), sep = &quot;_&quot;)
data &lt;- cbind(dplyr::select(data_imp_final, class), scale(data_dummy, 
                                                   center = apply(data_dummy, 2, min),
                                                   scale = apply(data_dummy, 2, max)))</code></pre>
</div>
<div>
<h3>Modeling</h3>
<pre><code># training and test set
set.seed(42)
index &lt;- createDataPartition(data$class, p = 0.9, list = FALSE)
train_data &lt;- data[index, ]
test_data  &lt;- data[-index, ]

# modeling
model_rf &lt;- caret::train(class ~ .,
  data = train_data,
  method = &quot;rf&quot;, # random forest
  trControl = trainControl(method = &quot;repeatedcv&quot;, 
       number = 10, 
       repeats = 5, 
       verboseIter = FALSE))</code></pre>
<pre><code>model_rf</code></pre>
<pre><code>## Random Forest 
## 
## 360 samples
##  48 predictor
##   2 classes: 'ckd', 'notckd' 
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 5 times) 
## Summary of sample sizes: 324, 324, 324, 324, 325, 324, ... 
## Resampling results across tuning parameters:
## 
##   mtry  Accuracy   Kappa    
##    2    0.9922647  0.9838466
##   25    0.9917392  0.9826070
##   48    0.9872930  0.9729881
## 
## Accuracy was used to select the optimal model using  the largest value.
## The final value used for the model was mtry = 2.</code></pre>
<pre><code># predictions
pred &lt;- data.frame(sample_id = 1:nrow(test_data), predict(model_rf, test_data, type = &quot;prob&quot;), actual = test_data$class) %>%
  mutate(prediction = colnames(.)[2:3][apply(.[, 2:3], 1, which.max)], correct = ifelse(actual == prediction, &quot;correct&quot;, &quot;wrong&quot;))

confusionMatrix(pred$actual, pred$prediction)</code></pre>
<pre><code>## Confusion Matrix and Statistics
## 
##           Reference
## Prediction ckd notckd
##     ckd     23      2
##     notckd   0     15
##                                           
##                Accuracy : 0.95            
##                  95% CI : (0.8308, 0.9939)
##     No Information Rate : 0.575           
##     P-Value [Acc > NIR] : 1.113e-07       
##                                           
##                   Kappa : 0.8961          
##  Mcnemar's Test P-Value : 0.4795          
##                                           
##             Sensitivity : 1.0000          
##             Specificity : 0.8824          
##          Pos Pred Value : 0.9200          
##          Neg Pred Value : 1.0000          
##              Prevalence : 0.5750          
##          Detection Rate : 0.5750          
##    Detection Prevalence : 0.6250          
##       Balanced Accuracy : 0.9412          
##                                           
##        'Positive' Class : ckd             
## </code></pre>
</div>
<div>
<h3>LIME</h3>
<ul>
<li>LIME needs data without response variable</li>
</ul>
<pre><code>train_x &lt;- dplyr::select(train_data, -class)
test_x &lt;- dplyr::select(test_data, -class)

train_y &lt;- dplyr::select(train_data, class)
test_y &lt;- dplyr::select(test_data, class)</code></pre>
<ul>
<li>build explainer</li>
</ul>
<pre><code>explainer &lt;- lime(train_x, model_rf, n_bins = 5, quantile_bins = TRUE)</code></pre>
<ul>
<li>run <code>explain()</code> function</li>
</ul>
<pre><code>explanation_df &lt;- lime::explain(test_x, explainer, n_labels = 1, n_features = 8, n_permutations = 1000, feature_select = &quot;forward_selection&quot;)</code></pre>
<ul>
<li>model reliability</li>
</ul>
<pre><code>explanation_df %>%
  ggplot(aes(x = model_r2, fill = label)) +
    geom_density(alpha = 0.5)</code></pre>
<p><img src="https://i2.wp.com/shirinsplayground.netlify.com/post/2017-12-12_lime_sketchnotes_files/figure-html/unnamed-chunk-13-1.png?w=450&ssl=1" data-recalc-dims="1"></p>
<ul>
<li>plot explanations</li>
</ul>
<pre><code>plot_features(explanation_df[1:24, ], ncol = 1)</code></pre>
<p><img src="https://i2.wp.com/shirinsplayground.netlify.com/post/2017-12-12_lime_sketchnotes_files/figure-html/unnamed-chunk-14-1.png?w=70%25&ssl=1" data-recalc-dims="1"></p>
</div>
<div>
<h3>Session Info</h3>
<pre><code>## Session info -------------------------------------------------------------</code></pre>
<pre><code>##  setting  value                       
##  version  R version 3.4.2 (2017-09-28)
##  system   x86_64, darwin15.6.0        
##  ui       X11                         
##  language (EN)                        
##  collate  de_DE.UTF-8                 
##  tz       </code></pre></div></div>