<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/timing-grouped-mean-calculation-in-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.win-vector.com/blog/2018/12/timing-grouped-means/"> R – Win-Vector Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>This note is a comment on some of the timings shared in the dplyr-0.8.0 <a rel="nofollow" href="https://www.tidyverse.org/articles/2018/12/dplyr-0-8-0-release-candidate/" target="_blank">pre-release announcement</a>.</p>
<p>The original published timings were as follows:</p>
<p><a rel="nofollow" href="https://www.tidyverse.org/articles/2018/12/dplyr-0-8-0-release-candidate/" target="_blank"><img alt="" src="https://i2.wp.com/raw.githubusercontent.com/WinVector/rqdatatable/master/extras/timings_summarise_mean_dplyr-0-8-0.jpg?w=660&ssl=1" data-recalc-dims="1"></a></p>
<p>With performance metrics: measurements are marketing. So let’s dig in the above a bit.</p>
<p><span></span></p>
<p>
<p>These timings are of the kind of <a rel="nofollow" href="https://cran.r-project.org/web/packages/data.table/vignettes/datatable-benchmarking.html#avoid-microbenchmark...-times100" target="_blank">small task large number of repetition breed</a> that Matt Dowle writes against. So they at first wouldn’t seem that decisive. Except, look at the following:</p>
<ul>
<li>At the time of our reading the example and methods were not shared. To reproduce the work we will need to make our own example (which we share <a rel="nofollow" href="https://github.com/WinVector/rqdatatable/blob/master/extras/TimingGroupedMean.Rmd" target="_blank">here</a>).</li>
<li>The timings are not relative to any other package or system (base-<code>R</code>, <code>data.table</code>, and <code>Pandas</code> being three obvious choices), so may have trouble valuing the results.</li>
<li>The time reported for <code>dplyr</code> on the <code>sum()/n()</code> examples is over a second to process 10,000 rows. This is unbelievably slow, and we fail to reproduce it in our run.</li>
<li>The time reported for <code>dplyr</code> on the <code>mean()</code> examples is about 0.01 seconds. This is a plausible time for this task (about 3 times as long as <code>data.table</code> would take). But it is much faster than is typical for <code>dplyr</code>. We fail to reproduce it in our run, we see <code>dplyr</code> taking closer to 0.06 seconds on this task (or about five times slower).</li>
</ul>
<p>Let’s try to reproduce these timings on a 2018 Dell XPS 13 Intel Core i5, 16GB Ram running Ubuntu 18.04, and also compare to some other packages: <a rel="nofollow" href="https://cran.r-project.org/package=data.table" target="_blank"><code>data.table</code></a> and <a rel="nofollow" href="https://cran.r-project.org/package=rqdatatable" target="_blank"><code>rqdatatable</code></a>.</p>
<p><img border="0" src="https://i0.wp.com/www.win-vector.com/blog/wp-content/uploads/2018/12/present-2.png?w=660" alt="Present 2" title="present-2.png" data-recalc-dims="1"></p>
<p>In this reproduction attempt we see:</p>
<ul>
<li>The <code>dplyr</code> time being around 0.05 seconds. This is about 5 times slower than claimed.</li>
<li>The <code>dplyr</code> <code>sum()/n()</code> time is about 0.2 seconds, about 5 times faster than claimed.</li>
<li>The <code>data.table</code> time being around 0.004 seconds. This is about three times as fast as the <code>dplyr</code> claims, and over ten times as fast as the actual observed <code>dplyr</code> behavior.</li>
</ul>
<p>However, Matt Dowle is also right: comparing at this scale doesn’t tell half the story we see when we try to summarize 10,000,000 rows down to 1,000,000. At this scale <code>data.table</code> still takes under half a second (time not much worth arguing over), yet <code>dplyr</code> takes 10 to 24 seconds! Or that <code>dplyr</code> is no faster than <code>base::tapply()</code> (despite many claims to the contrary).</p>
<p><img border="0" src="https://i1.wp.com/www.win-vector.com/blog/wp-content/uploads/2018/12/present2-2.png?resize=660%2C471" alt="Present2 2" width="450" title="present2-2.png" data-recalc-dims="1"></p>
<p>All code for this benchmark is available <a rel="nofollow" href="https://github.com/WinVector/rqdatatable/blob/master/extras/TimingGroupedMean.Rmd" target="_blank">here</a> and <a rel="nofollow" href="https://github.com/WinVector/rqdatatable/blob/master/extras/TimingGroupedMean.md" target="_blank">here</a>.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.win-vector.com/blog/2018/12/timing-grouped-means/"> R – Win-Vector Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/Udc7Aet2p-U" height="1"></a></div>