<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/why-i-rarely-use-apply/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://privefl.github.io/blog/why-i-rarely-use-apply/"> Florian Privé</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<section>
<p>In this short post, I talk about why I’m moving away from using function <code>apply</code>.</p>
<div>
<h2>With matrices</h2>
<p>It’s okay to use <code>apply</code> with a dense matrix, although you can often use an equivalent that is faster.</p>
<div>
<pre><code>N &lt;-<span> </span>M &lt;-<span> </span><span>8000</span>
X &lt;-<span> </span><span>matrix</span>(<span>rnorm</span>(N <span>*</span><span> </span>M), N)
<span>system.time</span>(res1 &lt;-<span> </span><span>apply</span>(X, <span>2</span>, mean))</code></pre>
</div>
<pre><code>##    user  system elapsed 
##    0.73    0.05    0.78</code></pre>
<div>
<pre><code><span>system.time</span>(res2 &lt;-<span> </span><span>colMeans</span>(X))</code></pre>
</div>
<pre><code>##    user  system elapsed 
##    0.05    0.00    0.05</code></pre>
<div>
<pre><code><span>stopifnot</span>(<span>isTRUE</span>(<span>all.equal</span>(res2, res1)))</code></pre>
</div>
<p>“Yeah, there are <code>colSums</code> and <code>colMeans</code>, but what about computing standard deviations?”</p>
<p>There are lots of <code>apply</code>-like functions in <a rel="nofollow" href="https://cran.r-project.org/package=matrixStats" target="_blank">package {matrixStats}</a>.</p>
<div>
<pre><code><span>system.time</span>(res3 &lt;-<span> </span><span>apply</span>(X, <span>2</span>, sd))</code></pre>
</div>
<pre><code>##    user  system elapsed 
##    0.96    0.01    0.97</code></pre>
<div>
<pre><code><span>system.time</span>(res4 &lt;-<span> </span>matrixStats<span>::</span><span>colSds</span>(X))</code></pre>
</div>
<pre><code>##    user  system elapsed 
##     0.2     0.0     0.2</code></pre>
<div>
<pre><code><span>stopifnot</span>(<span>isTRUE</span>(<span>all.equal</span>(res4, res3)))</code></pre>
</div>
</div>
<div>
<h2>With data frames</h2>
<div>
<pre><code><span>head</span>(iris)</code></pre>
</div>
<pre><code>##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1          5.1         3.5          1.4         0.2  setosa
## 2          4.9         3.0          1.4         0.2  setosa
## 3          4.7         3.2          1.3         0.2  setosa
## 4          4.6         3.1          1.5         0.2  setosa
## 5          5.0         3.6          1.4         0.2  setosa
## 6          5.4         3.9          1.7         0.4  setosa</code></pre>
<div>
<pre><code><span>apply</span>(<span>head</span>(iris), <span>2</span>, identity)</code></pre>
</div>
<pre><code>##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species 
## 1 &quot;5.1&quot;        &quot;3.5&quot;       &quot;1.4&quot;        &quot;0.2&quot;       &quot;setosa&quot;
## 2 &quot;4.9&quot;        &quot;3.0&quot;       &quot;1.4&quot;        &quot;0.2&quot;       &quot;setosa&quot;
## 3 &quot;4.7&quot;        &quot;3.2&quot;       &quot;1.3&quot;        &quot;0.2&quot;       &quot;setosa&quot;
## 4 &quot;4.6&quot;        &quot;3.1&quot;       &quot;1.5&quot;        &quot;0.2&quot;       &quot;setosa&quot;
## 5 &quot;5.0&quot;        &quot;3.6&quot;       &quot;1.4&quot;        &quot;0.2&quot;       &quot;setosa&quot;
## 6 &quot;5.4&quot;        &quot;3.9&quot;       &quot;1.7&quot;        &quot;0.4&quot;       &quot;setosa&quot;</code></pre>
<p><img src="https://i1.wp.com/privefl.github.io/blog/images/as-matrix.jpg?w=450&ssl=1" data-recalc-dims="1"></p>
<p>A DATA FRAME IS NOT A MATRIX (it’s a list).</p>
<p>The first thing that <code>apply</code> does is converting the object to a matrix, which consumes memory and in the previous example transforms all data as strings (because a matrix can have only one type).</p>
<p>What can you use as a replacement of <code>apply</code> with a data frame?</p>
<ul>
<li>
<p>If you want to operate on all columns, since a data frame is just a list, you can use <code>sapply</code> instead (or <code>map*</code> if you are a purrrist).</p>
<div>
<pre><code><span>sapply</span>(iris, typeof)</code></pre>
</div>
<pre><code>## Sepal.Length  Sepal.Width Petal.Length  Petal.Width      Species 
##     &quot;double&quot;     &quot;double&quot;     &quot;double&quot;     &quot;double&quot;    &quot;integer&quot;</code></pre>
</li>
<li>
<p>If you want to operate on all rows, I recommend you to watch <a rel="nofollow" href="https://www.rstudio.com/resources/webinars/thinking-inside-the-box-you-can-do-that-inside-a-data-frame/" target="_blank">this webinar</a>.</p>
</li>
</ul>
</div>
<div>
<h2>With sparse matrices</h2>
<p>The memory problem is even more important when using <code>apply</code> with sparse matrices, which makes using <code>apply</code> very slow for such data.</p>
<div>
<pre><code><span>library</span>(Matrix)

X.sp &lt;-<span> </span><span>rsparsematrix</span>(N, M, <span>density =</span> <span>0.01</span>)

## X.sp is converted to a dense matrix when using `apply`
<span>system.time</span>(res5 &lt;-<span> </span><span>apply</span>(X.sp, <span>2</span>, mean))  </code></pre>
</div>
<pre><code>##    user  system elapsed 
##    0.78    0.46    1.25</code></pre>
<div>
<pre><code><span>system.time</span>(res6 &lt;-<span> </span>Matrix<span>::</span><span>colMeans</span>(X.sp))</code></pre>
</div>
<pre><code>##    user  system elapsed 
##    0.01    0.00    0.02</code></pre>
<div>
<pre><code><span>stopifnot</span>(<span>isTRUE</span>(<span>all.equal</span>(res6, res5)))</code></pre>
</div>
<p>You could implement your own <code>apply</code>-like function for sparse matrices by seeing a sparse matrix as a data frame with 3 columns (<code>i</code> and <code>j</code> storing positions of non-null elements, and <code>x</code> storing values of these elements). Then, you could use a <code>group_by</code>–<code>summarize</code> approach.</p>
<p>For instance, for the previous example, you can do this in base R:</p>
<div>
<pre><code>apply2_sp &lt;-<span> </span><span>function</span>(X, FUN) {
  res &lt;-<span> </span><span>numeric</span>(<span>ncol</span>(X))
  X2 &lt;-<span> </span><span>as</span>(X, <span>&quot;dgTMatrix&quot;</span>)
  tmp &lt;-<span> </span><span>tapply</span>(X2<span>@</span>x, X2<span>@</span>j, FUN)
  res[<span>as.integer</span>(<span>names</span>(tmp)) <span>+</span><span> </span><span>1</span>] &lt;-<span> </span>tmp
  res
}

<span>system.time</span>(res7 &lt;-<span> </span><span>apply2_sp</span>(X.sp, sum) <span>/</span><span> </span><span>nrow</span>(X.sp))</code></pre>
</div>
<pre><code>##    user  system elapsed 
##    0.03    0.00    0.03</code></pre>
<div>
<pre><code><span>stopifnot</span>(<span>isTRUE</span>(<span>all.equal</span>(res7, res5)))</code></pre>
</div>
</div>
<div>
<h2>Conclusion</h2>
<p>Using <code>apply</code> with a dense matrix is fine, but try to avoid it if you have a data frame or a sparse matrix.</p>
</div>
</section>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://privefl.github.io/blog/why-i-rarely-use-apply/"> Florian Privé</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/v3FhAcuGPZE" height="1"></a></div>