<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/understanding-the-empirical-law-of-large-numbers-and-the-gamblers-fallacy/"></div></div><div><a data-url="https://www.r-bloggers.com/understanding-the-empirical-law-of-large-numbers-and-the-gamblers-fallacy/" data-counturl="https://www.r-bloggers.com/understanding-the-empirical-law-of-large-numbers-and-the-gamblers-fallacy/" data-count="horizontal" data-text="Understanding the empirical law of  large numbers and the gambler’s fallacy" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://science-memo.blogspot.com/2016/08/empirical-law-of-large-numbers-with.html"> Scientific Memo</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>One of the misconceptions in our understanding of statistics, or a <span>counter-intuitive guess, fallacy, appears in the assumption of the existence of the law of averages. Imagine we toss a fair coin many times, most people would think that the number of heads and tails would be balanced over the increasing number of trails, which is wrong. If you don’t, then you might have a very good statistical intuition. Briefly,  we will illustrate this, a kind of <a target="_blank" href="https://en.wikipedia.org/wiki/Gambler%27s_fallacy">gambler’s fallacy </a>with a simple simulation approach and discuss the empirical <a target="_blank" href="https://en.wikipedia.org/wiki/Law_of_large_numbers">law of large numbers</a>.</span><br>
<table cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td><a target="_blank" href="https://i1.wp.com/1.bp.blogspot.com/-P96FxM9B9SU/V59HHZWuW4I/AAAAAAAAC_s/N8Z8AcEQFD4vRmSOFCvDQxXRktP_VIZ3QCLcB/s1600/law_of_large_numbers.png?ssl=1"><img src="https://i1.wp.com/1.bp.blogspot.com/-P96FxM9B9SU/V59HHZWuW4I/AAAAAAAAC_s/N8Z8AcEQFD4vRmSOFCvDQxXRktP_VIZ3QCLcB/s320/law_of_large_numbers.png?resize=320%2C320&ssl=1" border="0" data-recalc-dims="1"></a></td>
</tr>
<tr>
<td>Figure 1: Empirical law of large numbers, ratio of<br>occurances approach to a constant.</td>
</tr>
</tbody>
</table>
<p><b><span>Empirical law of large numbers</span></b></p>
<p><span>If we repeat an experiment long enough, we would approach to expected outcome. The simplest example is a coin-toss experiment, that an expected </span><br><span>fair coin toss would lead to equal likelihood for head and tail, 1 or 0.  This implies that, the ratio of head and tails will approach to one with </span><span><span> increasing number of repeats</span>. Let’s say, we toss the coin $N$ times. The number of  heads and tails would be $n_{1}$ and $n_{0}$. The emprical law of large numbers states</span></p>
<p><span>$ \frac{n_{1}}{n_{0}} \to 1.0 $ if $N$ is very large, $N >>1$</span><br><span><br></span> <br><span>But note that, the absolute difference, $|n_{1}-n_{0}|$ does not approach to any constant, on the contrary, it will increase with increasing number of repeats. This is classic example of gambler’s fallacy that an outcome would balance out as there are more repeats. </span></p>
<p><b><span>Fair coin-toss: Evolution of Bernoulli process</span></b><br>
<table cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td><a target="_blank" href="https://i2.wp.com/2.bp.blogspot.com/-Sj_LPVIuk4M/V59HJKXURQI/AAAAAAAAC_w/nunlUA2CW7cq7fhrz6SNQYQErJ5MBTlNQCLcB/s1600/no_law_of_averages.png?ssl=1"><img src="https://i0.wp.com/2.bp.blogspot.com/-Sj_LPVIuk4M/V59HJKXURQI/AAAAAAAAC_w/nunlUA2CW7cq7fhrz6SNQYQErJ5MBTlNQCLcB/s320/no_law_of_averages.png?resize=320%2C320&ssl=1" border="0" data-recalc-dims="1"></a></td>
</tr>
<tr>
<td>Figure 2: No low of averages. Absolute difference<br>of occurances<br>increases over repeats.</td>
</tr>
</tbody>
</table>
<p><span>The <a target="_blank" href="https://en.wikipedia.org/wiki/Bernoulli_process">Bernoulli process </a>expresses binary outcomes, 1 or 0, i.e., success or failure, true or false. Bernoulli distribution reads,</span></p>
<p><span>$Ber(p, k) = p^{k} (1-p)^{1-k}$ for $k \in \{0,1\}$.</span></p>
<p><span>$p$ is the probability of success. We draw 50K samples from this distribution to get a Bernoulli process with $p=0.5$ and repeat the experiment 50 times, in order to obtain a “generalised” behaviour with uncertainty. This situation corresponds to a fair coin-toss experiment.  </span></p>
<p><b><span>Results</span></b></p>
<p><span>Empirical results of ratio of two outcomes and their absolute difference over repeats are reported in Figure 1 and 2 respectively..</span></p>
<p><b><span>Appendix: Source codes</span></b></p>
<p><span><a target="_blank" href="http://www.r-project.org/">R</a> and <a target="_blank" href="http://adv-r.had.co.nz/Rcpp.html">Rcpp</a> functions are shown in this section to reproduce the plots in this post. Source files are also available on github (<a target="_blank" href="https://github.com/msuzen/scientificMemo/tree/master/lawOfAverages">here</a>). </span> 
<div>
<pre><span>#'</span><br><span>#' Misconception of the law of averages: Understanding the empirical law of large numbers</span><br><span>#' </span><br><span>#' (c) 2016 Mehmet Suzen </span><br><span>#' </span><br><br>rm(<span>list</span><span>=</span>ls())<br><span>set</span><span>.</span>seed(<span>987231</span>)<br>library(Rcpp)          <span># law level implementations</span><br>library(LaplacesDemon) <span># for drawing samples from Bernoulli distribution</span><br>library(matlab)        <span># for tic/toc</span><br><span># Set working directory to script directory </span><br>Rcpp:::sourceCpp(<span>&quot;empirical_law_of_large_numbers.cpp&quot;</span>) <span># diff_binary_vec2</span><br><span>#'</span><br><span>#' Calculate the running difference in a binary vector </span><br><span>#' </span><br>diff_binary_vec  <span>&lt;-</span> function(bvec) {<br> ll <span>&lt;-</span> length(bvec) <br> diff_vec  <span>&lt;-</span> vector(<span>&quot;double&quot;</span>, ll)<br> diff_vec  <span>&lt;-</span> sapply(<span>1</span>:ll, function(i) { <span>abs</span>(i<span>-</span><span>2.0</span><span>*</span><span>sum</span>(bvec[<span>1</span>:i])) } )<br> diff_vec<br>}<br><span>#'</span><br><span>#' Calculate the running ratio in a binary vector </span><br><span>#'</span><br>ratio_binary_vec  <span>&lt;-</span> function(bvec) {<br> ll         <span>&lt;-</span> length(bvec) <br> ratio_vec  <span>&lt;-</span> vector(<span>&quot;double&quot;</span>, ll)<br> ratio_vec  <span>&lt;-</span> sapply(<span>1</span>:ll, function(i) { <span>abs</span>(i<span>/</span><span>sum</span>(bvec[<span>1</span>:i])<span>-</span><span>1.0</span>) } )<br> ratio_vec  <span>&lt;-</span> sapply(ratio_vec, function(rv) {<span>if</span>(<span>is</span><span>.</span>infinite(rv)) { rv <span>&lt;-</span> <span>0</span>; }; rv })<br> ratio_vec<br>}<br><br><span>#' Wall-call timing difference </span><br>tb <span>&lt;-</span> rbern(<span>20000</span>, <span>0.5</span>) <span># a fair-coin</span><br>tic()<br>t1 <span>&lt;-</span> diff_binary_vec(tb)<br>toc()<br>tic()<br>t2 <span>&lt;-</span> diff_binary_vec2(tb)<br>toc()<br>tic()<br>r1 <span>&lt;-</span> ratio_binary_vec(tb)<br>toc()<br>tic()<br>r2 <span>&lt;-</span> ratio_binary_vec2(tb)<br>toc()<br><span>#' </span><br><span>#' Generate Bernoulli Process</span><br><span>#' </span><br>nr        <span>&lt;-</span> <span>50</span>    <span># repeats</span><br>nt        <span>&lt;-</span> <span>20000</span> <span># Bernoulli trails</span><br>tic()<br>bern_df <span>&lt;-</span> data<span>.</span>frame(trail<span>=</span>c(), diff<span>=</span>c(), ratio<span>=</span>c())<br><span>for</span>(i <span>in</span> <span>1</span>:nr) {<br>  cat(<span>&quot;repeat:&quot;</span>,i, <span>&quot;</span><span>\n</span><span>&quot;</span>)<br>  trail   <span>&lt;-</span> rbern(nt, <span>0.5</span>) <span># a fair-coin</span><br>  diff    <span>&lt;-</span> diff_binary_vec2(trail)<br>  ratio   <span>&lt;-</span> ratio_binary_vec2(trail)<br>  bern_df <span>&lt;-</span> rbind(bern_df, cbind(<span>1</span>:nt, diff, ratio))<br>}<br><br><span>#' Now plot ratio and diff evolution with local regression in ggplot2</span><br>names(bern_df) <span>&lt;-</span> c(<span>&quot;trail&quot;</span>, <span>&quot;diff&quot;</span>, <span>&quot;ratio&quot;</span>)<br>library(ggplot2)<br>p_diff <span>&lt;-</span> ggplot(data<span>=</span>bern_df, aes(x<span>=</span>trail, y<span>=</span>diff)) <span>+</span> geom_smooth(formula<span>=</span><span>&quot;y~x&quot;</span>) <span>+</span> <br>          theme(<br>                panel<span>.</span>background <span>=</span> element_blank(), <br>                axis<span>.</span>text<span>.</span>x      <span>=</span> element_text(face<span>=</span><span>&quot;bold&quot;</span>, color<span>=</span><span>&quot;#000000&quot;</span>, size<span>=</span><span>11</span>),<br>                axis<span>.</span>text<span>.</span>y      <span>=</span> element_text(face<span>=</span><span>&quot;bold&quot;</span>, color<span>=</span><span>&quot;#000000&quot;</span>, size<span>=</span><span>11</span>),<br>                axis<span>.</span>title<span>.</span>x     <span>=</span> element_text(face<span>=</span><span>&quot;bold&quot;</span>, color<span>=</span><span>&quot;#000000&quot;</span>, size<span>=</span><span>11</span>),<br>                axis<span>.</span>title<span>.</span>y     <span>=</span> element_text(face<span>=</span><span>&quot;bold&quot;</span>, color<span>=</span><span>&quot;#000000&quot;</span>, size<span>=</span><span>11</span>))  <span>+</span><br>           xlab(<span>&quot;Bernoulli Trails&quot;</span>) <span>+</span> ylab(<span>&quot;Difference between occurance of two outcomes&quot;</span>) <span>+</span><br>           ggtitle(<span>&quot;No Law of Averages: Tail/Heads do not balanced out!&quot;</span>)<br><br>png(<span>file</span><span>=</span><span>&quot;no_law_of_averages.png&quot;</span>)<br>p_diff<br>dev<span>.</span>off()<br><br>p_ratio <span>&lt;-</span> ggplot(data<span>=</span>bern_df, aes(x<span>=</span>trail, y<span>=</span>ratio)) <span>+</span> geom_smooth(formula<span>=</span><span>&quot;y~x&quot;</span>) <span>+</span> <br>          theme(<br>                panel<span>.</span>background <span>=</span> element_blank(), <br>                axis<span>.</span>text<span>.</span>x      <span>=</span> element_text(face<span>=</span><span>&quot;bold&quot;</span>, color<span>=</span><span>&quot;#000000&quot;</span>, size<span>=</span><span>11</span>),<br>                axis<span>.</span>text<span>.</span>y      <span>=</span> element_text(face<span>=</span><span>&quot;bold&quot;</span>, color<span>=</span><span>&quot;#000000&quot;</span>, size<span>=</span><span>11</span>),<br>                axis<span>.</span>title<span>.</span>x     <span>=</span> element_text(face<span>=</span><span>&quot;bold&quot;</span>, color<span>=</span><span>&quot;#000000&quot;</span>, size<span>=</span><span>11</span>),<br>                axis<span>.</span>title<span>.</span>y     <span>=</span> element_text(face<span>=</span><span>&quot;bold&quot;</span>, color<span>=</span><span>&quot;#000000&quot;</span>, size<span>=</span><span>11</span>))  <span>+</span><br>           xlab(<span>&quot;Bernoulli Trails&quot;</span>) <span>+</span> ylab(<span>&quot;Difference between occurance of two outcomes&quot;</span>) <span>+</span><br>           ggtitle(<span>&quot;Empirical Law of Large Numbers: Ratio of Tails/Heads approach to one&quot;</span>)<br><br>png(<span>file</span><span>=</span><span>&quot;law_of_large_numbers.png&quot;</span>)<br>p_ratio<br>dev<span>.</span>off()<br></pre>
</div>
<p>
<div>
<pre><span>#include &lt;Rcpp.h></span><br><span>#include &lt;stdlib.h></span><br><span>using</span> <span>namespace</span> Rcpp;<br><br><span>//</span><br><span>// Sum a numeric vector up to an index</span><br><span>//</span><br><span>double</span> <span>sum_nv</span>(NumericVector x, <span>int</span> s) {<br>  <span>int</span> i<span>=</span><span>0</span>;<br>  <span>double</span> ss<span>=</span><span>0.0</span>;<br>  <span>for</span>(i<span>=</span><span>0</span>;i<span>&lt;</span>(s<span>+</span><span>1</span>);i<span>++</span>) {<br>   ss<span>=</span>ss<span>+</span>x[i];<br>  }<br>  <span>return</span>(ss);<br>}<br><br><span>//</span><br><span>// Calculate the running difference in a Bernoulli process (binary_vec)</span><br><span>//</span><br><span>// [[Rcpp::export]]</span><br><span>//</span><br>NumericVector <span>diff_binary_vec2</span>(NumericVector binary_vec) {<br>  <span>int</span> ll <span>=</span> binary_vec.size();<br>  <span>int</span> i  <span>=</span> <span>0</span>;<br>  NumericVector diff_vec(ll, <span>0.0</span>);<br>  <span>for</span>(i<span>=</span><span>0</span>;i<span>&lt;</span>ll;i<span>++</span>) {<br>    diff_vec[i] <span>=</span> std<span>::</span>abs(i<span>+</span><span>1.0</span><span>-</span><span>2.0</span><span>*</span>sum_nv(binary_vec, i));<br>  }<br>  <span>return</span>(diff_vec);<br>}<br><br><span>//</span><br><span>// Calculate the running ratio in a Bernoulli process (binary_vec)</span><br><span>//</span><br><span>// [[Rcpp::export]]</span><br>NumericVector <span>ratio_binary_vec2</span>(NumericVector binary_vec) {<br>  <span>int</span> ll <span>=</span> binary_vec.size();<br>  <span>int</span> i  <span>=</span> <span>0</span>;<br>  NumericVector ratio_vec(ll, <span>0.0</span>);<br>  <span>for</span>(i<span>=</span><span>0</span>;i<span>&lt;</span>ll;i<span>++</span>) {<br>    ratio_vec[i] <span>=</span> std<span>::</span>abs(((i<span>+</span><span>1.0</span>)<span>/</span>sum_nv(binary_vec, i))<span>-</span><span>1.0</span>);<br>  }<br>  <span>return</span>(ratio_vec);<br>}<br></pre>
</div>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/understanding-the-empirical-law-of-large-numbers-and-the-gamblers-fallacy/"></div></div><div><a data-url="https://www.r-bloggers.com/understanding-the-empirical-law-of-large-numbers-and-the-gamblers-fallacy/" data-counturl="https://www.r-bloggers.com/understanding-the-empirical-law-of-large-numbers-and-the-gamblers-fallacy/" data-count="vertical" data-text="Understanding the empirical law of  large numbers and the gambler’s fallacy" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://science-memo.blogspot.com/2016/08/empirical-law-of-large-numbers-with.html"> Scientific Memo</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/0r9WiXMYS-o"></a></div>