<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/tractable-bayesian-variable-selection-beyond-normality/"></div></div><div><a data-url="https://www.r-bloggers.com/tractable-bayesian-variable-selection-beyond-normality/" data-counturl="https://www.r-bloggers.com/tractable-bayesian-variable-selection-beyond-normality/" data-count="horizontal" data-text="tractable Bayesian variable selection: beyond normality" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://xianblog.wordpress.com/2016/10/17/tractable-bayesian-variable-selection-beyond-normality/"> R – Xi'an's Og</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a target="_blank" href="https://i0.wp.com/xianblog.files.wordpress.com/2012/02/pict0496.jpg?ssl=1"><img alt="bird tracks in the first snow of Winter, Feb. 05, 2012 " class="wp-image-14579" src="https://i0.wp.com/xianblog.files.wordpress.com/2012/02/pict0496.jpg?resize=337%2C253&ssl=1" data-recalc-dims="1"></a><strong>D</strong>avid Rossell and Francisco Rubio (both from Warwick<a target="_blank" href="https://arxiv.org/abs/1609.01708">) arXived</a> a month ago a <a target="_blank" href="https://arxiv.org/abs/1609.01708">paper</a> on non-normal variable selection. They use two-piece error models that preserve manageable inference and allow for simple computational algorithms, but also characterise the behaviour of the resulting variable selection process under model misspecification. Interestingly, they show that the existence of asymmetries or heavy tails leads to power losses when using the Normal model. The two-piece error distribution is made of two halves of location-scale transforms of the same reference density on the two sides of the common location parameter. In this <a target="_blank" href="https://arxiv.org/abs/1609.01708">paper</a>, the density is either Gaussian or Laplace (i.e., exponential?). In both cases the (log-)likelihood has a nice compact expression (although it does not allow for a useful sufficient statistic). One is the L¹ version versus the other which is the L² version. Which is the main reason for using this formalism based on only two families of parametric distributions, I presume. (As mentioned in an <a target="_blank" href="https://xianblog.wordpress.com/2015/12/08/mixtures-as-exponential-families/">earlier post</a>, I do not consider those distributions as mixtures because the component of a given observation can always be identified. And because as shown in the current <a target="_blank" href="https://arxiv.org/abs/1609.01708">paper</a>, maximum likelihood estimates can be easily derived.) The prior construction follows the <a target="_blank" href="https://xianblog.wordpress.com/2016/09/15/non-local-priors-for-mixtures/">non-local prior</a> principles of Johnson and Rossell (2010, 2012) also discussed in earlier posts. The construction is very detailed and hence highlights how many calibration steps are needed in the process.</p>
<blockquote>
<p><span><em>“Bayes factor rates are the same as when the correct model is assumed [but] model misspecification often causes a decrease in the power to detect truly active variables.”</em></span></p>
</blockquote>
<p>When there are too many models to compare at once, the authors propose a random walk on the finite set of models (which does not require advanced measure-theoretic tools like reversible jump MCMC). One interesting aspect is that moving away from the normal to another member of this small family is driven by the density of the data under the marginal densities, which means moving only to interesting alternatives. But also sticking to the normal only for adequate datasets. In a sense this is not extremely surprising given that the marginal likelihoods (model-wise) are available. It is also interesting that on real datasets, one of the four models is heavily favoured against the others, be it Normal (6.3) or Laplace (6.4). And that the four model framework returns almost identical values when compared with a single (most likely) model. Although not immensely surprising when acknowledging that the frequency of the most likely model is 0.998 and 0.998, respectively.</p>
<blockquote>
<p><em><span>“Our framework represents a middle-ground to add flexibility in a parsimonious manner that remains analytically and computationally tractable, facilitating applications where either p is large or n is too moderate to fit more flexible models accurately.”</span></em></p>
</blockquote>
<p>Overall, I find the experiment quite conclusive and do not object [much] to this choice of parametric family in that it is always more general and generic than the sempiternal Gaussian model. That we picked in our <a target="_blank" href="https://xianblog.wordpress.com/2013/11/14/bayesian-essentials-with-r-available-on-amazon/">Bayesian Essentials</a>, following tradition. In a sense, it would be natural to pick the most general possible parametric family that allows for fast computations, if this notion does make any sense…</p>
<p>Filed under: <a target="_blank" href="https://xianblog.wordpress.com/category/statistics/r-statistics/">R</a>, <a target="_blank" href="https://xianblog.wordpress.com/category/statistics/">Statistics</a>, <a target="_blank" href="https://xianblog.wordpress.com/category/university-life/">University life</a> Tagged: <a target="_blank" href="https://xianblog.wordpress.com/tag/bayesian-essentials-with-r/">Bayesian Essentials with R</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/calibration/">calibration</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/marginal-density/">marginal density</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/maximum-likelihood-estimation/">maximum likelihood estimation</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/parametric-family/">parametric family</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/r/">R</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/two-piece-error-model/">two-piece error model</a>, <a target="_blank" href="https://xianblog.wordpress.com/tag/university-of-warwick/">University of Warwick</a> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/xianblog.wordpress.com/31704/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/xianblog.wordpress.com/31704/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/tractable-bayesian-variable-selection-beyond-normality/"></div></div><div><a data-url="https://www.r-bloggers.com/tractable-bayesian-variable-selection-beyond-normality/" data-counturl="https://www.r-bloggers.com/tractable-bayesian-variable-selection-beyond-normality/" data-count="vertical" data-text="tractable Bayesian variable selection: beyond normality" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://xianblog.wordpress.com/2016/10/17/tractable-bayesian-variable-selection-beyond-normality/"> R – Xi'an's Og</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/p00yeZpyG-g"></a></div>