<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/big-data-solutions-ab-t-test/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://drsimonj.svbtle.com/big-data-solutions-a-b-t-test"> blogR</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a rel="nofollow" href="https://twitter.com/drsimonj" target="_blank">@drsimonj</a> here to share my code for using <a rel="nofollow" href="https://en.wikipedia.org/wiki/Welch%27s_t-test" target="_blank">Welch’s <em>t</em>-test</a> to compare group means using summary statistics.</p>
<h2><a rel="nofollow" href="http://drsimonj.svbtle.com/big-data-solutions-a-b-t-test#motivation_2" target="_blank"> </a>Motivation</h2>
<p>I’ve just started working with A/B tests that use big data. Where once I’d whimsically run <code>t.test()</code>, now my data won’t fit into memory!</p>
<p>I’m sharing my solution here in the hope that it might help others.</p>
<h2><a rel="nofollow" href="http://drsimonj.svbtle.com/big-data-solutions-a-b-t-test#inmemory-data_2" target="_blank"> </a>In-memory data</h2>
<p>As a baseline, let’s start with an in-memory case by comparing whether automatic and manual cars have different Miles Per Gallon ratings on average (using the <code>mtcars</code> data set).</p>
<pre><code>t.test(mpg ~ am, data = mtcars)
#> 
#>  Welch Two Sample t-test
#> 
#> data:  mpg by am
#> t = -3.7671, df = 18.332, p-value = 0.001374
#> alternative hypothesis: true difference in means is not equal to 0
#> 95 percent confidence interval:
#>  -11.280194  -3.209684
#> sample estimates:
#> mean in group 0 mean in group 1 
#>        17.14737        24.39231
</code></pre>
<p>Well… that was easy!</p>
<h2><a rel="nofollow" href="http://drsimonj.svbtle.com/big-data-solutions-a-b-t-test#big-data_2" target="_blank"> </a>Big Data</h2>
<p>The problem with big data is that we can’t pull it into memory and work with R.</p>
<p>Fortunately, we don’t need the raw data to run Welch’s <em>t</em>-test. All we need is the mean, variance, and sample size of each group. So our raw data might have billions of rows, but we only need six numbers.</p>
<p>Here are the numbers we need for the previous example:</p>
<pre><code>library(dplyr)

grp_summary &lt;- mtcars %>% 
  group_by(am) %>% 
  summarise(
    mpg_mean = mean(mpg),
    mpg_var  = var(mpg),
    n        = n()
  )

grp_summary
#> # A tibble: 2 x 4
#>      am mpg_mean  mpg_var     n
#>   </code></pre>