<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/abc-random-forests-for-bayesian-parameter-inference/"></div></div><div><a data-url="http://www.r-bloggers.com/abc-random-forests-for-bayesian-parameter-inference/" data-counturl="http://www.r-bloggers.com/abc-random-forests-for-bayesian-parameter-inference/" data-count="horizontal" data-text="ABC random forests for Bayesian parameter inference" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://xianblog.wordpress.com/2016/05/20/abc-random-forests-for-bayesian-parameter-inference/"> R – Xi'an's Og</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><strong><a target="_blank" href="https://xianblog.files.wordpress.com/2016/05/sunse3-e1463646978827.jpg"><img height="470" alt="" width="404" class="wp-image-30683" src="https://xianblog.files.wordpress.com/2016/05/sunse3-e1463646978827.jpg?w=404&h=470"></a>B</strong>efore leaving Helsinki, we <a target="_blank" href="http://arxiv.org/abs/1605.05537">arXived</a> [from the Air France lounge!] the paper Jean-Michel presented on Monday at <a target="_blank" href="https://xianblog.wordpress.com/2016/05/17/abc-in-helsinki-on-board/">ABCruise</a> in <a target="_blank" href="https://xianblog.wordpress.com/2016/05/17/sunrise-on-bothnia/">Helsinki</a>. This paper summarises the experiments Louis conducted over the past months to assess the great performances of a random forest regression approach to ABC parameter inference. Thus validating in this experimental sense the use of this new approach to conducting ABC for Bayesian inference by random forests. (And not <a target="_blank" href="https://xianblog.wordpress.com/2015/10/21/abc-model-choice-via-random-forests-accepted/">ABC model choice</a> as in the <a target="_blank" href="https://xianblog.wordpress.com/2015/10/21/abc-model-choice-via-random-forests-accepted/">Bioinformatics paper</a> with Pierre Pudlo and others.)</p>
<p>I think the major incentives in exploiting the (still mysterious) tool of random forests [against more traditional ABC approaches like <a target="_blank" href="https://xianblog.wordpress.com/2010/04/14/semi-automatic-abc/">Fearnhead and Prangle (2012)</a> on summary selection] are that (i) forests do not require a preliminary selection of the summary statistics, since an arbitrary number of summaries can be used as input for the random forest, even when including a large number of useless white noise variables; (b) there is no longer a tolerance level involved in the process, since the many trees in the random forest define a natural if rudimentary distance that corresponds to being or not being in the same leaf as the observed vector of summary statistics η(y); (c) the size of the reference table simulated from the prior (predictive) distribution does not need to be as large as for in usual ABC settings and hence this approach leads to significant gains in computing time since the production of the reference table usually is the costly part! To the point that deriving a different forest for each univariate transform of interest is truly a minor drag in the overall computing cost of the approach.</p>
<p>An intriguing point we uncovered through Louis’ experiments is that an unusual version of the variance estimator is preferable to the standard estimator: we indeed exposed better estimation performances when using a weighted version of the out-of-bag residuals (which are computed as the differences between the simulated value of the parameter transforms and their expectation obtained by removing the random trees involving this simulated value). Another intriguing feature [to me] is that the regression weights as proposed by Meinshausen (2006) are obtained as an average of the inverse of the number of terms in the leaf of interest. When estimating the posterior expectation of a transform h(θ) given the observed η(y), this summary statistic η(y) ends up in a given leaf for each tree in the forest and all that matters for computing the weight is the number of points from the reference table ending up in this very leaf. I do find this difficult to explain when confronting the case when many simulated points are in the leaf against the case when a single simulated point makes the leaf. This single point ends up being much more influential that all the points in the other situation… While being an outlier of sorts against the prior simulation. But now that I think more about it (after an expensive <a target="_blank" href="https://en.wikipedia.org/wiki/Lapin_Kulta">Lapin Kulta beer</a> in the Helsinki airport while waiting for a change of tire on our airplane!), it somewhat makes sense that rare simulations that agree with the data should be weighted much more than values that stem from the prior simulations and hence do not translate much of an information brought by the observation. (If this sounds murky, blame the beer.) What I found great about this new approach is that it produces a non-parametric evaluation of the cdf of the quantity of interest h(θ) at no calibration cost or hardly any. (An R package is in the making, to be added to the existing R functions of <a target="_blank" href="https://xianblog.wordpress.com/2016/02/12/new-version-of-abcrf/">abcrf</a> we developed for the <a target="_blank" href="https://xianblog.wordpress.com/2014/10/01/abc-model-choice-via-random-forests-expanded/">ABC model choice paper</a>.)</p>
<p>Filed under: <a target="_blank" href="https://xianblog.wordpress.com/category/books/">Books</a>, <a target="_blank" href="https://xianblog.wordpress.com/category/kids/">Kids</a>, <a target="_blank" href="https://xianblog.wordpress.com/category/statistics/r-statistics/">R</a>, <a target="_blank" href="https://xianblog.wordpress.com/category/statistics/">Statistics</a>, <a target="_blank" href="https://xianblog.wordpress.com/category/travel/">Travel</a>, <a target="_blank" href="https://xianblog.wordpress.com/category/university-life/">University life</a>, <a target="_blank" href="https://xianblog.wordpress.com/category/wines/">Wines</a> Tagged: <a target="_blank" href="https://xianblog.wordpress.com/tag/abc-approximation-error/">ABC approximation error</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/abc-in-helsinki/">ABC in Helsinki</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/abcrf/">abcrf</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/abcruise/">ABCruise</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/arxiv/">arXiv</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/baltic-sea/">Baltic Sea</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/bayesian-inference/">Bayesian inference</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/gulf-of-bothnia/">Gulf of Bothnia</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/helsinki/">Helsinki</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/lapin-kulta/">Lapin Kulta</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/out-of-bag-correction/">out-of-bag correction</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/r/">R</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/random-forests/">random forests</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/reference-table/">reference table</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/sunrise/">sunrise</a> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/xianblog.wordpress.com/30682/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/xianblog.wordpress.com/30682/" border="0"></a> <img height="1" alt="" width="1" src="https://pixel.wp.com/b.gif?host=xianblog.wordpress.com&blog=5051449&%23038;post=30682&%23038;subd=xianblog&%23038;ref=&%23038;feed=1" border="0"></p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/abc-random-forests-for-bayesian-parameter-inference/"></div></div><div><a data-url="http://www.r-bloggers.com/abc-random-forests-for-bayesian-parameter-inference/" data-counturl="http://www.r-bloggers.com/abc-random-forests-for-bayesian-parameter-inference/" data-count="vertical" data-text="ABC random forests for Bayesian parameter inference" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://xianblog.wordpress.com/2016/05/20/abc-random-forests-for-bayesian-parameter-inference/"> R – Xi'an's Og</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/ZzK1Qc8gXAE"></a></div>