<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/dalex-which-variables-are-really-important-ask-your-black-box-model/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://smarterpoland.pl/index.php/2018/03/dalex-which-variables-are-important-to-your-black-box/"> SmarterPoland.pl » English</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a rel="nofollow" href="https://xkcd.com/1364/" target="_blank"><img align="right" src="https://i2.wp.com/imgs.xkcd.com/comics/like_im_five.png?w=200&ssl=1" data-recalc-dims="1"></a>Third post from the short series about black-box explainers implemented in the DALEX package. Learn more about DALEX at SER (Warsaw, April 2018), eRum (Budapest, May 2018), WhyR (Wroclaw, June 2018) or UseR (Brisbane, July 2018). </p>
<p>Two weeks ago I wrote about <a rel="nofollow" href="http://smarterpoland.pl/index.php/2018/02/dalex-conditional-single-variable-response-of-your-black-box-model/" target="_blank">single variable conditional responses</a> and last week I wrote about <a rel="nofollow" href="http://smarterpoland.pl/index.php/2018/02/dalex-which-variables-influence-this-single-prediction/" target="_blank">decompositions of a single prediction</a>.</p>
<p>Sometimes we would like to know the general structure of a model, or at least know which variables are the most influential. There is a lot of different approaches to this problem proposed in literature. A nice, simple, and model agnostic approach is described in <a rel="nofollow" href="https://arxiv.org/abs/1801.01489" target="_blank">this article (Fisher, Rudin, and Dominici 2018)</a>. To see how important is variable X let’s permute it’s values and measure the drop in model accuracy.<br>
This procedure is implemented in the <em>DALEX</em> package in the <em>variable_dropout()</em> function. There are some tweaks (for large datasets you do not need to permute all rows while for small datasets you could consider some oversampling) but the idea is the same. </p>
<p>In the figure below you will find variable importances for three models created with the <a rel="nofollow" href="https://www.kaggle.com/jacksonchou/hr-analytics" target="_blank">HR dataset</a>. It is easy to spot that the <em>randomForest</em> model results in the best model and <em>satisfaction_level</em> is the most important variable in all three models.</p>
<p><a rel="nofollow" href="https://i1.wp.com/smarterpoland.pl/wp-content/uploads/2018/02/plot.variable_dropout_explainer-19.png" target="_blank"><img alt="plot.variable_dropout_explainer-19" src="https://i1.wp.com/smarterpoland.pl/wp-content/uploads/2018/02/plot.variable_dropout_explainer-19-1024x633.png?w=450" class="wp-image-8583" data-recalc-dims="1"></a></p>
<p>There are two things that I like in this explainer.</p>
<p>1) Variable effects for a single model are interesting, but ability to compare effects for many modes is even more interesting. In the DALEX you can simply contrast/compare explainers across different models.</p>
<p>2) There is no reason to start variable importance plots in the point 0, since the initial model performance is different for different plots. It is much more informative to present both the initial model performance and drop in the performance resulting from the dropout of a variable.</p>
<p>If you want to learn more about DALEX package and variable importances consult following vignette or the DALEX website.</p>
<p><a rel="nofollow" href="https://i2.wp.com/smarterpoland.pl/wp-content/uploads/2018/02/DALEX_variable_dropout.png" target="_blank"><img alt="DALEX_variable_dropout" src="https://i2.wp.com/smarterpoland.pl/wp-content/uploads/2018/02/DALEX_variable_dropout-1024x783.png?w=450" class="wp-image-8582" data-recalc-dims="1"></a></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://smarterpoland.pl/index.php/2018/03/dalex-which-variables-are-important-to-your-black-box/"> SmarterPoland.pl » English</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/MgxpnefgG5E" height="1"></a></div>