<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/local-goodness-of-fit-plots-wangkardu-explanations-a-new-dalex-companion/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://smarterpoland.pl/index.php/2018/07/local-goodness-of-fit-plots-wangkardu-explanations-a-new-dalex-companion/"> English – SmarterPoland.pl</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><center><img src="https://i1.wp.com/smarterpoland.pl/wp-content/uploads/2018/07/dalex_sydney.jpg?w=100%25" data-recalc-dims="1"></center></p>
<p>The next <a rel="nofollow" href="https://github.com/pbiecek/DALEX" target="_blank">DALEX</a> workshop will take place in 4 days at <a rel="nofollow" href="https://user2018.r-project.org/" target="_blank">UseR</a>. In the meantime I am working on a new explainer for a single observation.<br>
Something like a diagnostic plot for a single observation. Something that extends <a rel="nofollow" href="http://smarterpoland.pl/index.php/2018/06/ceteris-paribus-plots-a-new-dalex-companion/" target="_blank">Ceteris Paribus Plots</a>. Something similar to <a rel="nofollow" href="https://arxiv.org/abs/1309.6392" target="_blank">Individual Conditional Expectation (ICE) Plots</a>. An experimental version is implemented in <a rel="nofollow" href="https://github.com/pbiecek/ceterisParibus" target="_blank">ceterisParibus</a> package.<br>
 <br>
<b>Intro</b></p>
<p>For a single observation, Ceteris Paribus Plots (What-If plots) show how predictions for a model change along a single variable. But they do not tell if the model is well fitted around this observation.</p>
<p>Here is an idea how to fix this:<br>
(1) Take N points from validation dataset, points that are closest to a selected observation (<a rel="nofollow" href="https://cran.r-project.org/web/packages/gower/vignettes/intro.html" target="_blank">Gower distance</a> is used by default).<br>
(2) Plot N <a rel="nofollow" href="https://github.com/pbiecek/ceterisParibus" target="_blank">Ceteris Paribus Plots</a> for these points,<br>
(3) Since we know the true y for these points, then we can plot model residuals in these points.<br>
 <br>
<b>Examples</b></p>
<p>Here we have an example for a random forest model. The validation dataset has 9000 observations. We use N=18 observations closest to the observation of interest to show the model stability and the local goodness-of-fit.</p>
<p><center><a rel="nofollow" href="https://i0.wp.com/smarterpoland.pl/wp-content/uploads/2018/07/local2.png" target="_blank"><img src="https://i0.wp.com/smarterpoland.pl/wp-content/uploads/2018/07/local2.png?w=400" data-recalc-dims="1"></a><br><small>(click to enlarge)</small></center></p>
<p>The empty circle in the middle stands for the observation of interest. We may read its <i>surface</i> component (OX axis, around 85 sq meters), and the model prediction (OY axis, around 3260 EUR).<br>
The thick line stands for Ceteris Paribus Plot for the observation of interest.<br>
Grey points stands for 18 closest observations from the validation dataset while grey lines are their Ceteris Paribus Plots. <br>
Red and blue lines stand for residuals for these neighbours. Corresponding true values of y are marked with red and blue circles. </p>
<p>Red and blue intervals are short and symmetric so one may say that the model is well fitted around the observation of interest.<br>
<span></span><br>
Let’s see an another example for the same model but a different point prediction. This time for a large apartment.</p>
<p><center><a rel="nofollow" href="https://i1.wp.com/smarterpoland.pl/wp-content/uploads/2018/07/local1.png" target="_blank"><img src="https://i1.wp.com/smarterpoland.pl/wp-content/uploads/2018/07/local1.png?w=400" data-recalc-dims="1"></a><br><small>(click to enlarge)</small></center></p>
<p>Residuals are larger, all negative (true values are higher than predictions). The local goodness-of-fit for the model is pretty bad. Other diagnostic tools (see <a rel="nofollow" href="https://mi2-warsaw.github.io/auditor/articles/Intorduction_into_model_audit.html" target="_blank">auditor</a>) show that for this dataset the random forest model is heavily biased for expensive apartments.</p>
<p><b>Wangkardu</b></p>
<p>Side story: In <a rel="nofollow" href="https://www.artgallery.nsw.gov.au/collection/works/4.2003/?tab=about" target="_blank">AGNSW</a> I found a painting named <a rel="nofollow" href="https://www.artgallery.nsw.gov.au/collection/works/4.2003/?tab=about" target="_blank"><i>Wangkardu</i></a> that looks (to me) like above graphs. If you want to use its colors, just add <i>palette = „wangkardu”</i> to the plot function (<a rel="nofollow" href="https://pbiecek.github.io/ceterisParibus/articles/coral.html#wangkardu-explanations" target="_blank">R code here</a>). </p>
<p><center><a rel="nofollow" href="https://i2.wp.com/smarterpoland.pl/wp-content/uploads/2018/07/local4.png" target="_blank"><img src="https://i2.wp.com/smarterpoland.pl/wp-content/uploads/2018/07/local4.png?w=400" data-recalc-dims="1"></a></center><br>
<center><a rel="nofollow" href="https://i1.wp.com/smarterpoland.pl/wp-content/uploads/2018/07/local3.png" target="_blank"><img src="https://i1.wp.com/smarterpoland.pl/wp-content/uploads/2018/07/local3.png?w=400" data-recalc-dims="1"></a><br><small>(click to enlarge)</small></center></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://smarterpoland.pl/index.php/2018/07/local-goodness-of-fit-plots-wangkardu-explanations-a-new-dalex-companion/"> English – SmarterPoland.pl</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/gf4W0Nr5jXc" height="1"></a></div>