<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/bayesian-ab-testing-made-easy/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.r-exercises.com/2017/08/21/bayesian_ab_testing_made_easy/"> R-exercises</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>A/B Testing is a familiar task for many working in business analytics. Essentially, A/B Testing is a simple form of hypothesis testing with one control group and one treatment group. Classical frequentist methodology instructs the analyst to estimate the expected effect of the treatment, calculate the required sample size, and perform a test to determine if a large enough effect is observed. This method is somewhat lacking: it only leaves one with point estimates for the control and the treatment groups, and a verdict to reject (effect is observed) or to fail to reject (effect is not observed).</p>
<p>Let’s consider an alternative approach following Bayesian methods with the <code>bayesAB</code> package. Suppose that we have the current version and a proposed version of a web page, each containing a button of interest, and we wish to determine whether the proposed version leads to more clicks on the button of interest. Currently, approximately half of all visitors click the button of interest. Suppose the proposed version of the web page is actually much worse and only 30 percent will click it.</p>
<p>To test this, we randomly assign some visitors to the current and other visitors to the proposed version. Since a visitor either clicks the button of interest or not, we can treat this as a Bernoulli random variable with parameter theta. For the control and the treatment groups, we will assign the same prior distribution on theta, e.g., a beta distribution with mean 0.5. You can think of this as the analog of the null hypothesis.</p>
<p><a rel="nofollow" href="https://i0.wp.com/www.r-exercises.com/wp-content/uploads/2017/08/prior.png" target="_blank"><img sizes="(max-width: 414px) 100vw, 414px" src="https://i0.wp.com/www.r-exercises.com/wp-content/uploads/2017/08/prior-300x129.png?resize=414%2C178" alt="" class="wp-image-7546" data-recalc-dims="1"></a>Now, let’s simulate 20 observations for each group and compare the posterior probabilities for the control and treatment groups. The package automatically computes the probability that the mean of the treatment is greater than the mean of the control.</p>
<pre><code># First collection
control_1 &lt;- rbinom(20, 1, 0.5)
treatment_1 &lt;- rbinom(20, 1, 0.3)

<div><aside><img src="http://www.r-exercises.com/2017/08/21/bayesian_ab_testing_made_easy/"></aside><div><strong>Learn more</strong> about probability functions in the online course <a rel="nofollow" href="http://www.r-exercises.com/product/statistics-with-r-advanced-level/" target="_blank">Statistics with R – Advanced Level</a>. In this course you will learn how to
<ul>
<li>work with different binomial and logistic regression techniques,</li>
<li>know how to compare regression models and choose the right fit,</li>
<li>and much more.</li>
</ul>
</div></div>

# First Analysis
test1 &lt;- bayesTest(treatment_1, control_1, distribution = &quot;bernoulli&quot;, priors = c(&quot;alpha&quot; = 10, &quot;beta&quot; = 10))
print(test1)
summary(test1)
plot(test1)</code></pre>
<pre><a rel="nofollow" href="https://i0.wp.com/www.r-exercises.com/wp-content/uploads/2017/08/posterior_1.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i0.wp.com/www.r-exercises.com/wp-content/uploads/2017/08/posterior_1-300x212.png?resize=300%2C212" alt="" class="wp-image-7539" data-recalc-dims="1"></a><img sizes="(max-width: 361px) 100vw, 361px" src="https://i2.wp.com/www.r-exercises.com/wp-content/uploads/2017/08/probs_1-300x181.png?resize=361%2C218" alt="" class="wp-image-7545" data-recalc-dims="1"></pre>
<p>The treatment posterior distribution is in red and the control posterior is in green. After 40 observations in total, the posteriors have started to separate, and the probability that the treatment is less than the control is approaching 95 percent.</p>
<p>Let’s simulate 20 more observations for each group and compare.</p>
<pre><code># Second Collection
control_2 &lt;- rbind(control_1, rbinom(20, 1, 0.5))
treatment_2 &lt;- rbind(treatment_1, rbinom(20, 1, 0.3))

# Second Analysis
test2 &lt;- bayesTest(treatment_2, control_2, distribution = &quot;bernoulli&quot;, priors = c(&quot;alpha&quot; = 10, &quot;beta&quot; = 10))
print(test2)
summary(test2)
plot(test2)</code></pre>
<p><a rel="nofollow" href="https://i1.wp.com/www.r-exercises.com/wp-content/uploads/2017/08/posterior_2.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i1.wp.com/www.r-exercises.com/wp-content/uploads/2017/08/posterior_2-300x212.png?resize=300%2C212" alt="" class="wp-image-7536" data-recalc-dims="1"></a> <img sizes="(max-width: 361px) 100vw, 361px" src="https://i0.wp.com/www.r-exercises.com/wp-content/uploads/2017/08/props_2-300x181.png?resize=361%2C218" alt="" class="wp-image-7544" data-recalc-dims="1"></p>
<p>We can see that with the additional 40 observations, the distributions have separated more, and the probability that the treatment is less than the control is 98 percent.</p>
<p>In addition to point estimates and a verdict, we have full distributions for each of the parameters, which makes computing prediction intervals, for instance, easy. There’s much more to explore in this package than can be covered in this tutorial, so try getting your hands dirty with a few examples contained in the documentation.</p>
<div>
<h3>Related exercise sets:</h3>
<ol>
<li><a rel="nofollow" href="http://www.r-exercises.com/2017/07/25/hacking-statistics-exercises-part-4/" title="Hacking statistics or: How I Learned to Stop Worrying About Calculus and Love Stats Exercises (Part-4)" target="_blank">Hacking statistics or: How I Learned to Stop Worrying About Calculus and Love Stats Exercises (Part-4) </a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/2017/07/08/hacking-statistics-exercises-part-2/" title="Hacking statistics or: How I Learned to Stop Worrying About Calculus and Love Stats Exercises (Part-2)" target="_blank">Hacking statistics or: How I Learned to Stop Worrying About Calculus and Love Stats Exercises (Part-2) </a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/2016/05/12/cross-tabulation-with-xtabs-exercises/" title="Cross Tabulation with Xtabs exercises" target="_blank">Cross Tabulation with Xtabs exercises </a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/start-here-to-learn-r/" target="_blank">Explore all our (>1000) R exercises</a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/r-courses/" target="_blank">Find an R course using our R Course Finder</a> directory</li>
</ol>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.r-exercises.com/2017/08/21/bayesian_ab_testing_made_easy/"> R-exercises</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/ny-UTmSQXG0" height="1"></a></div>