<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/bayesab-0-7-0-a-primer-on-priors/"></div></div><div><a data-url="https://www.r-bloggers.com/bayesab-0-7-0-a-primer-on-priors/" data-counturl="https://www.r-bloggers.com/bayesab-0-7-0-a-primer-on-priors/" data-count="horizontal" data-text="bayesAB 0.7.0 + A Primer on Priors" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://fportman.com/blog/bayesab-0-dot-7-0-plus-a-primer-on-priors/"> Frank Portman</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2>bayesAB 0.7.0</h2>
<p>Quick announcement that my package for Bayesian AB Testing, bayesAB, has been updated to 0.7.0 on CRAN. Some improvements on the backend as well a few tweaks for a more fluid UX/API. Some links:</p>
<ul>
<li><a target="_blank" href="https://github.com/FrankPortman/bayesAB">bayesAB Github Repo</a></li>
<li><a target="_blank" href="https://cran.r-project.org/package=bayesAB">CRAN Page</a></li>
</ul>
<p>Now, on to the good stuff.</p>
<h2>Why should we care about priors?</h2>
<p>Most questions I’ve gotten since I released bayesAB have been along the lines of:</p>
<ul>
<li>Why/how is Bayesian AB testing better than Frequentist hypothesis AB testing?</li>
<li>Why do I need priors?</li>
<li>Do I really really really need priors?</li>
<li>How do I choose priors?</li>
</ul>
<p>Question 1 has a few objective and a few subjective answers to it. The main benefits are ones that I’ve already highlighted in the README/vignette of the bayesAB package. To briefly summarize, we get direct probabilities for A > B (rather than p-values) and distributions over the parameter estimates rather than point estimates. Finally, we can also leverage <strong>priors</strong> which help with the low sample size and low base rate problems.</p>
<p>To start, let’s go back to what a <strong>prior</strong> actually is in a Bayesian context. There are countless mathematical resources out there (including part of my previous blog post) so I’ll only about this conceptually. Simply put, a <strong>prior</strong> lets you specify some sort of, ahem, <em>prior</em> information about a certain parameter so that the end <strong>posterior</strong> on that parameter encapsualtes both the <strong>data</strong> you saw and the <strong>prior</strong> you inputted. Priors can come from a variety of places including past experiments, literature, and domain expertise into the problem. <a target="_blank" href="http://www.sumsar.net/blog/2015/11/a-bayesian-model-to-calculate-whether-my-wife-is-pregnant/">See this blogpost</a> for a great example of somebody combining their own past data and literature to form very strong priors.</p>
<p><span></span></p>
<p><strong>Priors</strong> can be weak or strong. The weakest prior will be completely <strong>objective</strong> and thus assign an equal probability to each value for the parameter. Examples of this include a Beta(1, 1) prior for the Bernoulli distribution. In these cases, the <strong>posterior</strong> distribution is completely reliant on the <strong>data</strong>. A strong <strong>prior</strong> will convey a very precise belief as to where a parameter’s values may lie. For example:</p>
<div></div>