<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/cant-compute-the-standard-deviation-in-your-head-divide-the-range-by-four/"></div></div><div><a data-url="http://www.r-bloggers.com/cant-compute-the-standard-deviation-in-your-head-divide-the-range-by-four/" data-counturl="http://www.r-bloggers.com/cant-compute-the-standard-deviation-in-your-head-divide-the-range-by-four/" data-count="horizontal" data-text="Can’t compute the standard deviation in your head? Divide the range by four." class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.decisionsciencenews.com/2016/06/17/cant-compute-standard-deviation-head-divide-range-four/"> R – Decision Science News</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>TESTING A HEURISTIC TO ESTIMATE STANDARD DEVIATION</p>
<p><a target="_blank" rel="attachment wp-att-5714" href="http://i0.wp.com/www.decisionsciencenews.com/wp-content/uploads/2016/06/range.heuristic.dists_.png"><img alt="range.heuristic.dists.s" class="wp-image-5714" src="http://i0.wp.com/www.decisionsciencenews.com/wp-content/uploads/2016/06/range.heuristic.dists_.s.png?resize=456%2C506" data-recalc-dims="1"></a><br>
<em>Click to enlarge</em></p>
<p>Say you’ve got 30 numbers and a strong urge to estimate their standard deviation. But you’ve left your computer at home. Unless you’re really good at mentally squaring and summing, it’s pretty hard to compute a standard deviation in your head. But there’s a heuristic you can use:</p>
<blockquote><p>Subtract the smallest number from the largest number and divide by four</p>
</blockquote>
<p>Let’s call it the “range over four” heuristic. You could, and probably should, be skeptical. You could want to see how accurate the heuristic is. And you could want to see how the heuristic’s accuracy depends on the distribution of numbers you are dealing with.</p>
<p>Fine.</p>
<p>We generated  random numbers from four distributions, pictured above. We nickname them (from top to bottom): floor, left of center, normalish, and uniform. They’re all beta distributions. If you want more detail, they’re the same beta distributions studied in Goldstein and Rothschild (<a target="_blank" href="http://journal.sjdm.org/13/131029/jdm131029.pdf">2014</a>). See the code below for parameters.</p>
<p>We vary two things in our simulation:<br>
1) The number of observations on which we’re estimating the standard deviation.<br>
2) The distributions from which the observations are drawn</p>
<p>With each sample, we compute the actual standard deviation and compare it to the heuristic’s estimate of the standard deviation. We do this many times and take the average. Because we like the way mape sounds, we used mean absolute percent error (MAPE) as our error metric. Enough messing around. Let’s show the result.</p>
<p><a target="_blank" rel="attachment wp-att-5716" href="http://i0.wp.com/www.decisionsciencenews.com/wp-content/uploads/2016/06/range.heuristic.png"><img alt="range.heuristic.s" class="wp-image-5716" src="http://i2.wp.com/www.decisionsciencenews.com/wp-content/uploads/2016/06/range.heuristic.s.png?resize=456%2C506" data-recalc-dims="1"></a><br>
<em>Click to enlarge</em></p>
<p>There you have it. With about 30 to 40 observations, we could get an average absolute error of less than 10 percent for three of our distributions, even the skewed ones. With more observations, the error grew for those distributions.</p>
<p>With the uniform distribution, error was over 15 percent in the 30-40 observation range. We’re fine with that. We don’t tend to measure too many things that are uniformly distributed.</p>
<p>Another thing that set the uniform distribution apart is that its error continued to go down as more observations were added. Why is this? The standard deviation of a uniform distribution between 0 and 1 is 1/sqrt(12) or <span>0.289</span>. The heuristic, if it were lucky enough to draw 1 and a 0 as its sample range, would estimate the standard deviation as 1/4 or .25. So, the sample size increases, the error for the uniform distribution should drop down to a MAPE of 13.4% and flatten out. The graph shows it is well on its way towards doing so.</p>
<p>Want to play with it yourself? R Code below. Thanks to <a target="_blank" href="http://hadley.nz/">Hadley Wickham</a> for creating tools like dplyr and ggplot2 which take <a target="_blank" href="https://www.r-project.org/">R</a> to the next level.</p>
<p></p>
<p>The post <a target="_blank" rel="nofollow" href="http://www.decisionsciencenews.com/2016/06/17/cant-compute-standard-deviation-head-divide-range-four/">Can’t compute the standard deviation in your head? Divide the range by four.</a> appeared first on <a target="_blank" rel="nofollow" href="http://www.decisionsciencenews.com/">Decision Science News</a>.</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/cant-compute-the-standard-deviation-in-your-head-divide-the-range-by-four/"></div></div><div><a data-url="http://www.r-bloggers.com/cant-compute-the-standard-deviation-in-your-head-divide-the-range-by-four/" data-counturl="http://www.r-bloggers.com/cant-compute-the-standard-deviation-in-your-head-divide-the-range-by-four/" data-count="vertical" data-text="Can’t compute the standard deviation in your head? Divide the range by four." class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.decisionsciencenews.com/2016/06/17/cant-compute-standard-deviation-head-divide-range-four/"> R – Decision Science News</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/xqQRfl3qO3U"></a></div>