<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/lime-v0-4-the-kitten-picture-edition-2/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://tensorflow.rstudio.com/blog/lime-v0.4-the-kitten-picture-edition.html"> TensorFlow for R</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>This is guest post contributed by <a rel="nofollow" href="https://www.data-imaginist.com/about/" target="_blank">Thomas Lin Pedersen</a>, creator of the <a rel="nofollow" href="https://github.com/thomasp85/lime" target="_blank">lime</a> package. The post originally appeared on Thomas’ blog at <a rel="nofollow" href="https://www.data-imaginist.com/2018/lime-v0-4-the-kitten-picture-edition/" target="_blank">https://www.data-imaginist.com/2018/lime-v0-4-the-kitten-picture-edition/</a>.</p>
</div>
<p><img align="right" src="https://i1.wp.com/tensorflow.rstudio.com/blog/images/lime-v0-4-the-kitten-picture-edition/lime_logo_small.jpg?w=456&ssl=1" data-recalc-dims="1"></p>
<p>I’m happy to report a new major release of <code>lime</code> has landed on CRAN. <code>lime</code> is an R port of the Python library of the same name by Marco Ribeiro that allows the user to pry open black box machine learning models and explain their outcomes on a per-observation basis. It works by modelling the outcome of the black box in the local neighborhood around the observation to explain and using this local model to explain why (not how) the black box did what it did. For more information about the theory of <code>lime</code> I will direct you to the article <a rel="nofollow" href="https://arxiv.org/abs/1602.04938" target="_blank">introducing the methodology</a>.</p>
<div>
</div>
<div>
<h2>New features</h2>
<p>The meat of this release centers around two new features that are somewhat linked: Native support for keras models and support for explaining image models.</p>
<div>
<h3>keras and images</h3>
<p>J.J. Allaire was kind enough to namedrop <code>lime</code> during his keynote introduction of the <code>tensorflow</code> and <code>keras</code> packages and I felt compelled to support them natively. As keras is by far the most popular way to interface with tensorflow it is first in line for build-in support. The addition of keras means that <code>lime</code> now directly supports models from the following packages:</p>
<ul>
<li><a rel="nofollow" href="https://github.com/topepo/caret" target="_blank">caret</a></li>
<li><a rel="nofollow" href="https://github.com/mlr-org/mlr" target="_blank">mlr</a></li>
<li><a rel="nofollow" href="https://github.com/dmlc/xgboost" target="_blank">xgboost</a></li>
<li><a rel="nofollow" href="https://github.com/h2oai/h2o-3" target="_blank">h2o</a></li>
<li><a rel="nofollow" href="https://github.com/rstudio/keras" target="_blank">keras</a></li>
</ul>
<p>If you’re working on something too obscure or cutting edge to not be able to use these packages it is still possible to make your model <code>lime</code> compliant by providing <code><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/model_support?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">predict_model()</a></code> and <code><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/model_support?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">model_type()</a></code> methods for it.</p>
<p>keras models are used just like any other model, by passing it into the <code><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/lime?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">lime()</a></code> function along with the training data in order to create an explainer object. Because we’re soon going to talk about image models, we’ll be using one of the pre-trained ImageNet models that is available from keras itself:</p>
<div>
<pre><code><span>library</span>(keras)
<span>library</span>(lime)
<span>library</span>(magick)

model &lt;-<span> </span><span><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/application_vgg.html" target="_blank">application_vgg16</a></span>(
  <span>weights =</span> <span>&quot;imagenet&quot;</span>,
  <span>include_top =</span> <span>TRUE</span>
)
model</code></pre>
</div>
<pre><code>Model
______________________________________________________________________________________________
Layer (type)                              Output Shape                         Param #        
==============================================================================================
input_1 (InputLayer)                      (None, 224, 224, 3)                  0              
______________________________________________________________________________________________
block1_conv1 (Conv2D)                     (None, 224, 224, 64)                 1792           
______________________________________________________________________________________________
block1_conv2 (Conv2D)                     (None, 224, 224, 64)                 36928          
______________________________________________________________________________________________
block1_pool (MaxPooling2D)                (None, 112, 112, 64)                 0              
______________________________________________________________________________________________
block2_conv1 (Conv2D)                     (None, 112, 112, 128)                73856          
______________________________________________________________________________________________
block2_conv2 (Conv2D)                     (None, 112, 112, 128)                147584         
______________________________________________________________________________________________
block2_pool (MaxPooling2D)                (None, 56, 56, 128)                  0              
______________________________________________________________________________________________
block3_conv1 (Conv2D)                     (None, 56, 56, 256)                  295168         
______________________________________________________________________________________________
block3_conv2 (Conv2D)                     (None, 56, 56, 256)                  590080         
______________________________________________________________________________________________
block3_conv3 (Conv2D)                     (None, 56, 56, 256)                  590080         
______________________________________________________________________________________________
block3_pool (MaxPooling2D)                (None, 28, 28, 256)                  0              
______________________________________________________________________________________________
block4_conv1 (Conv2D)                     (None, 28, 28, 512)                  1180160        
______________________________________________________________________________________________
block4_conv2 (Conv2D)                     (None, 28, 28, 512)                  2359808        
______________________________________________________________________________________________
block4_conv3 (Conv2D)                     (None, 28, 28, 512)                  2359808        
______________________________________________________________________________________________
block4_pool (MaxPooling2D)                (None, 14, 14, 512)                  0              
______________________________________________________________________________________________
block5_conv1 (Conv2D)                     (None, 14, 14, 512)                  2359808        
______________________________________________________________________________________________
block5_conv2 (Conv2D)                     (None, 14, 14, 512)                  2359808        
______________________________________________________________________________________________
block5_conv3 (Conv2D)                     (None, 14, 14, 512)                  2359808        
______________________________________________________________________________________________
block5_pool (MaxPooling2D)                (None, 7, 7, 512)                    0              
______________________________________________________________________________________________
flatten (Flatten)                         (None, 25088)                        0              
______________________________________________________________________________________________
fc1 (Dense)                               (None, 4096)                         102764544      
______________________________________________________________________________________________
fc2 (Dense)                               (None, 4096)                         16781312       
______________________________________________________________________________________________
predictions (Dense)                       (None, 1000)                         4097000        
==============================================================================================
Total params: 138,357,544
Trainable params: 138,357,544
Non-trainable params: 0
______________________________________________________________________________________________
</code></pre>
<p>The vgg16 model is an image classification model that has been build as part of the ImageNet competition where the goal is to classify pictures into 1000 categories with the highest accuracy. As we can see it is fairly complicated.</p>
<p>In order to create an explainer we will need to pass in the training data as well. For image data the training data is really only used to tell lime that we are dealing with an image model, so any image will suffice. The format for the training data is simply the path to the images, and because the internet runs on kitten pictures we’ll use one of these:</p>
<div>
<pre><code>img &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/magick/topics/editing?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">image_read</a></span>(<span>'https://www.data-imaginist.com/assets/images/kitten.jpg'</span>)
img_path &lt;-<span> </span><span>file.path</span>(<span>tempdir</span>(), <span>'kitten.jpg'</span>)
<span><a rel="nofollow" href="http://www.rdocumentation.org/packages/magick/topics/editing?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">image_write</a></span>(img, img_path)
<span>plot</span>(<span>as.raster</span>(img))</code></pre>
</div>
<div>
<img src="https://i2.wp.com/tensorflow.rstudio.com/blog/images/lime-v0-4-the-kitten-picture-edition/unnamed-chunk-2-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p>As with text models the explainer will need to know how to prepare the input data for the model. For keras models this means formatting the image data as tensors. Thankfully keras comes with a lot of tools for reshaping image data:</p>
<div>
<pre><code>image_prep &lt;-<span> </span><span>function</span>(x) {
  arrays &lt;-<span> </span><span>lapply</span>(x, <span>function</span>(path) {
    img &lt;-<span> </span><span><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/image_load.html" target="_blank">image_load</a></span>(path, <span>target_size =</span> <span>c</span>(<span>224</span>,<span>224</span>))
    x &lt;-<span> </span><span><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/image_to_array.html" target="_blank">image_to_array</a></span>(img)
    x &lt;-<span> </span><span><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/reexports.html" target="_blank">array_reshape</a></span>(x, <span>c</span>(<span>1</span>, <span>dim</span>(x)))
    x &lt;-<span> </span><span><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/imagenet_preprocess_input.html" target="_blank">imagenet_preprocess_input</a></span>(x)
  })
  <span>do.call</span>(abind<span>::</span>abind, <span>c</span>(arrays, <span>list</span>(<span>along =</span> <span>1</span>)))
}
explainer &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/lime?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">lime</a></span>(img_path, model, image_prep)</code></pre>
</div>
<p>We now have an explainer model for understanding how the vgg16 neural network makes its predictions. Before we go along, lets see what the model think of our kitten:</p>
<div>
<pre><code>res &lt;-<span> </span><span>predict</span>(model, <span>image_prep</span>(img_path))
<span><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/imagenet_decode_predictions.html" target="_blank">imagenet_decode_predictions</a></span>(res)</code></pre>
</div>
<pre><code>[[1]]
  class_name class_description      score
1  n02124075      Egyptian_cat 0.48913878
2  n02123045             tabby 0.15177219
3  n02123159         tiger_cat 0.10270492
4  n02127052              lynx 0.02638111
5  n03793489             mouse 0.00852214</code></pre>
<p>So, it is pretty sure about the whole cat thing. The reason we need to use <code><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/imagenet_decode_predictions.html" target="_blank">imagenet_decode_predictions()</a></code> is that the output of a keras model is always just a nameless tensor:</p>
<div>
<pre><code><span>dim</span>(res)</code></pre>
</div>
<pre><code>[1]    1 1000</code></pre>
<div>
<pre><code><span>dimnames</span>(res)</code></pre>
</div>
<pre><code>NULL</code></pre>
<p>We are used to classifiers knowing the class labels, but this is not the case for keras. Motivated by this, <code>lime</code> now have a way to define/overwrite the class labels of a model, using the <code><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/as_classifier?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">as_classifier()</a></code> function. Let’s redo our explainer:</p>
<div>
<pre><code>model_labels &lt;-<span> </span><span>readRDS</span>(<span>system.file</span>(<span>'extdata'</span>, <span>'imagenet_labels.rds'</span>, <span>package =</span> <span>'lime'</span>))
explainer &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/lime?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">lime</a></span>(img_path, <span><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/as_classifier?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">as_classifier</a></span>(model, model_labels), image_prep)</code></pre>
</div>
<blockquote>
<p>There is also an <code><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/as_classifier?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">as_regressor()</a></code> function which tells <code>lime</code>, without a doubt, that the model is a regression model. Most models can be introspected to see which type of model they are, but neural networks doesn’t really care. <code>lime</code> guesses the model type from the activation used in the last layer (linear activation == regression), but if that heuristic fails then <code><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/as_classifier?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">as_regressor()</a></code>/<code><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/as_classifier?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">as_classifier()</a></code> can be used.</p>
</blockquote>
<p>We are now ready to poke into the model and find out what makes it think our image is of an Egyptian cat. But… first I’ll have to talk about yet another concept: superpixels (I promise I’ll get to the explanation part in a bit).</p>
<p>In order to create meaningful permutations of our image (remember, this is the central idea in <code>lime</code>), we have to define how to do so. The permutations needs to be substantial enough to have an impact on the image, but not so much that the model completely fails to recognise the content in every case – further, they should lead to an interpretable result. The concept of superpixels lends itself well to these constraints. In short, a superpixel is a patch of an area with high homogeneity, and superpixel segmentation is a clustering of image pixels into a number of superpixels. By segmenting the image to explain into superpixels we can turn area of contextual similarity on and off during the permutations and find out if that area is important. It is still necessary to experiment a bit as the optimal number of superpixels depend on the content of the image. Remember, we need them to be large enough to have an impact but not so large that the class probability becomes effectively binary. <code>lime</code> comes with a function to assess the superpixel segmentation before beginning the explanation and it is recommended to play with it a bit — with time you’ll likely get a feel for the right values:</p>
<div>
<pre><code><span># default</span>
<span><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/plot_superpixels?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">plot_superpixels</a></span>(img_path)</code></pre>
</div>
<div>
<img src="https://i0.wp.com/tensorflow.rstudio.com/blog/images/lime-v0-4-the-kitten-picture-edition/unnamed-chunk-7-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<div>
<pre><code><span># Changing some settings</span>
<span><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/plot_superpixels?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">plot_superpixels</a></span>(img_path, <span>n_superpixels =</span> <span>200</span>, <span>weight =</span> <span>40</span>)</code></pre>
</div>
<div>
<img src="https://i0.wp.com/tensorflow.rstudio.com/blog/images/lime-v0-4-the-kitten-picture-edition/unnamed-chunk-7-2.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p>The default is set to a pretty low number of superpixels — if the subject of interest is relatively small it may be necessary to increase the number of superpixels so that the full subject does not end up in one, or a few superpixels. The <code>weight</code> parameter will allow you to make the segments more compact by weighting spatial distance higher than colour distance. For this example we’ll stick with the defaults.</p>
<p>Be aware that explaining image models is much heavier than tabular or text data. In effect it will create 1000 new images per explanation (default permutation size for images) and run these through the model. As image classification models are often quite heavy, this will result in computation time measured in minutes. The permutation is batched (default to 10 permutations per batch), so you should not be afraid of running out of RAM or hard-drive space.</p>
<div>
<pre><code>explanation &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/explain?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">explain</a></span>(img_path, explainer, <span>n_labels =</span> <span>2</span>, <span>n_features =</span> <span>20</span>)</code></pre>
</div>
<p>The output of an image explanation is a data frame of the same format as that from tabular and text data. Each feature will be a superpixel and the pixel range of the superpixel will be used as its description. Usually the explanation will only make sense in the context of the image itself, so the new version of <code>lime</code> also comes with a <code><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/plot_image_explanation?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">plot_image_explanation()</a></code> function to do just that. Let’s see what our explanation have to tell us:</p>
<div>
<pre><code><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/plot_image_explanation?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">plot_image_explanation</a></span>(explanation)</code></pre>
</div>
<div>
<img src="https://i0.wp.com/tensorflow.rstudio.com/blog/images/lime-v0-4-the-kitten-picture-edition/unnamed-chunk-8-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p>We can see that the model, for both the major predicted classes, focuses on the cat, which is nice since they are both different cat breeds. The plot function got a few different functions to help you tweak the visual, and it filters low scoring superpixels away by default. An alternative view that puts more focus on the relevant superpixels, but removes the context can be seen by using <code>display = 'block'</code>:</p>
<div>
<pre><code><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/plot_image_explanation?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">plot_image_explanation</a></span>(explanation, <span>display =</span> <span>'block'</span>, <span>threshold =</span> <span>0.01</span>)</code></pre>
</div>
<div>
<img src="https://i2.wp.com/tensorflow.rstudio.com/blog/images/lime-v0-4-the-kitten-picture-edition/unnamed-chunk-9-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p>While not as common with image explanations it is also possible to look at the areas of an image that contradicts the class:</p>
<div>
<pre><code><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/lime/topics/plot_image_explanation?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">plot_image_explanation</a></span>(explanation, <span>threshold =</span> <span>0</span>, <span>show_negative =</span> <span>TRUE</span>, <span>fill_alpha =</span> <span>0.6</span>)</code></pre>
</div>
<div>
<img src="https://i2.wp.com/tensorflow.rstudio.com/blog/images/lime-v0-4-the-kitten-picture-edition/unnamed-chunk-10-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p>As each explanation takes longer time to create and needs to be tweaked on a per-image basis, image explanations are not something that you’ll create in large batches as you might do with tabular and text data. Still, a few explanations might allow you to understand your model better and be used for communicating the workings of your model. Further, as the time-limiting factor in image explanations are the image classifier and not lime itself, it is bound to improve as image classifiers becomes more performant.</p>
</div>
<div>
<h3>Grab back</h3>
<p>Apart from keras and image support, a slew of other features and improvements have been added. Here’s a quick overview:</p>
<ul>
<li>All explanation plots now include the fit of the ridge regression used to make the explanation. This makes it easy to assess how good the assumptions about local linearity are kept.</li>
<li>When explaining tabular data the default distance measure is now <code>'gower'</code> from the <code>gower</code> package. <code>gower</code> makes it possible to measure distances between heterogeneous data without converting all features to numeric and experimenting with different exponential kernels.</li>
<li>When explaining tabular data numerical features will no longer be sampled from a normal distribution during permutations, but from a kernel density defined by the training data. This should ensure that the permutations are more representative of the expected input.</li>
</ul>
</div>
</div>
<div>
<h2>Wrapping up</h2>
<p>This release represents an important milestone for <code>lime</code> in R. With the addition of image explanations the <code>lime</code> package is now on par or above its Python relative, feature-wise. Further development will focus on improving the performance of the model, e.g. by adding parallelisation or improving the local model definition, as well as exploring alternative explanation types such as <a rel="nofollow" href="https://homes.cs.washington.edu/~marcotcr/aaai18.pdf" target="_blank">anchor</a>.</p>
<p>Happy Explaining!</p>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://tensorflow.rstudio.com/blog/lime-v0.4-the-kitten-picture-edition.html"> TensorFlow for R</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/y1FG0iXsdfY" height="1"></a></div>