<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/rewiring-replyr-with-dplyr/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://eighty20.github.io//package%20exploration/2017/02/16/replyr-dplyr/"> Pareto's Playground</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h1>Introduction of Parameterized dplyr expression</h1>
<p>The usefullness of any small function you write will eventually be judged upon its ability to be generically applied across any arbitrary data. As I explored a blog post from <a target="_blank" rel="nofollow" href="http://www.win-vector.com/blog/2016/12/using-replyrlet-to-parameterize-dplyr-expressions/">Dec 2016</a>, I became a lot more interested in writing dynamic code with dplyr functions that form part of the data wrangling silo in my analytical flow. This ability came with the new <code>replyr</code> package – No longer will I have the need to break up my data processing when columns have to be changed as my code depends on certain column names in my dataset that is currently in use.</p>
<blockquote>
<p>‘replyr allows you to encapsulate complex dplyr expressions without the use of the lazyeval package, which is the currently recommended way to manage dplyr‘s use of non-standard evaluation’</p>
</blockquote>
<p>The example <code>replyr</code> provides works out summary statistics of an arbitrary column, with the ability to group by another column. Imagine you had a quick and easy function which could pump out summary statistics without too much fuss. Lets take a look at the <code>replyr</code> construct of such a function provided by the package maintainers <a target="_blank" rel="nofollow" href="http://www.win-vector.com/blog/">Win-Vector</a>:</p>
<figure>
<pre><code data-lang="r"><span># to install replyr: # devtools::install_github('WinVector/replyr')
</span><span>
</span><span>suppressPackageStartupMessages</span><span>(</span><span>library</span><span>(</span><span>dplyr</span><span>))</span><span>
</span><span>suppressPackageStartupMessages</span><span>(</span><span>library</span><span>(</span><span>replyr</span><span>))</span><span>
</span><span>#
# calculate mean +/- sd intervals and
# median +/- 1/2 IQR intervals
# for arbitrary data frame column, with optional grouping
#
</span><span>
</span><span>dist_intervals</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>dframe</span><span>,</span><span> </span><span>colname</span><span>,</span><span> </span><span>groupcolname</span><span>=</span><span>NULL</span><span>)</span><span> </span><span>{</span><span>
</span><span>mapping</span><span> </span><span>=</span><span> </span><span>list</span><span>(</span><span>col</span><span>=</span><span>colname</span><span>)</span><span>
</span><span>if</span><span>(</span><span>!</span><span>is.null</span><span>(</span><span>groupcolname</span><span>))</span><span> </span><span>{</span><span>
</span><span>dframe</span><span> </span><span>%>%</span><span> </span><span>group_by_</span><span>(</span><span>groupcolname</span><span>)</span><span> </span><span>-></span><span> </span><span>dframe</span><span>
</span><span>}</span><span>
</span><span>let</span><span>(</span><span>alias</span><span>=</span><span>mapping</span><span>,</span><span>
</span><span>expr</span><span>=</span><span>{</span><span>
</span><span>dframe</span><span> </span><span>%>%</span><span> </span><span>summarize</span><span>(</span><span>sdlower</span><span> </span><span>=</span><span> </span><span>mean</span><span>(</span><span>col</span><span>)</span><span>-</span><span>sd</span><span>(</span><span>col</span><span>),</span><span>
</span><span>mean</span><span> </span><span>=</span><span> </span><span>mean</span><span>(</span><span>col</span><span>),</span><span>
</span><span>sdupper</span><span> </span><span>=</span><span> </span><span>mean</span><span>(</span><span>col</span><span>)</span><span> </span><span>+</span><span> </span><span>sd</span><span>(</span><span>col</span><span>),</span><span>
</span><span>iqrlower</span><span> </span><span>=</span><span> </span><span>median</span><span>(</span><span>col</span><span>)</span><span>-0.5</span><span>*</span><span>IQR</span><span>(</span><span>col</span><span>),</span><span>
</span><span>median</span><span> </span><span>=</span><span> </span><span>median</span><span>(</span><span>col</span><span>),</span><span>
</span><span>iqrupper</span><span> </span><span>=</span><span> </span><span>median</span><span>(</span><span>col</span><span>)</span><span>+0.5</span><span>*</span><span>IQR</span><span>(</span><span>col</span><span>))</span><span>
</span><span>})</span><span>
</span><span>}</span></code></pre>
</figure>
<p>For my analysis I will be using the <code>snail</code> dataset.<br>
This dataset contains data on the probability of a snail surviving given certain stimuli such as:</p>
<ul>
<li>exposure in weeks</li>
<li>relative humidity (4 levels)</li>
<li>temperature, in degrees Celsius (3 levels)</li>
<li>deaths</li>
</ul>
<figure>
<pre><code data-lang="r"><span>Snails</span><span> </span><span>&lt;-</span><span> </span><span>read.csv</span><span>(</span><span>&quot;https://vincentarelbundock.github.io/Rdatasets/csv/MASS/snails.csv&quot;</span><span>)</span><span>
</span><span>Snails</span><span> </span><span>%>%</span><span> </span><span>glimpse</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## Observations: 96
## Variables: 7
## $ X        &lt;int> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16...
## $ Species  &lt;fctr> A, A, A, A, A, A, A, A, A, A, A, A, A, A, A, A, A, A...
## $ Exposure &lt;int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2,...
## $ Rel.Hum  &lt;dbl> 60.0, 60.0, 60.0, 65.8, 65.8, 65.8, 70.5, 70.5, 70.5,...
## $ Temp     &lt;int> 10, 15, 20, 10, 15, 20, 10, 15, 20, 10, 15, 20, 10, 1...
## $ Deaths   &lt;int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0,...
## $ N        &lt;int> 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 2...</code></pre>
</figure>
<p>Lets now see how the function outputs with a simple example using the <code>snail</code> dataset.</p>
<figure>
<pre><code data-lang="r"><span>Snails</span><span> </span><span>%>%</span><span> </span><span>dist_intervals</span><span>(</span><span>&quot;Deaths&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">##      sdlower     mean  sdupper iqrlower median iqrupper
## 1 -0.8984496 2.864583 6.627616   -1.125      1    3.125</code></pre>
</figure>
<p>As you can see, the summary statistics for the death column was worked out. For those who noticed the negtive <code>sdlower</code> – no, snails did not wake up, its purely for example purposes.</p>
<p>Expanding this idea of by includeing a grouping variable, we can get the summary statistics per specie:</p>
<figure>
<pre><code data-lang="r"><span>Snails</span><span> </span><span>%>%</span><span> </span><span>dist_intervals</span><span>(</span><span>&quot;Deaths&quot;</span><span>,</span><span> </span><span>&quot;Species&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 2 × 7
##   Species    sdlower     mean  sdupper iqrlower median iqrupper
##    &lt;fctr>      &lt;dbl>    &lt;dbl>    &lt;dbl>    &lt;dbl>  &lt;dbl>    &lt;dbl>
## 1       A -0.5320915 1.708333 3.948758     -1.0    0.5      2.0
## 2       B -0.5477438 4.020833 8.589410     -1.5    2.0      5.5</code></pre>
</figure>
<p>So, we can see the function works well in the sense that you can now write dynamic loops which could apply <code>dplyr</code> functions to an arbitrary column and grouping variable of your choice based on string inputs…</p>
<h2>But that was not the end</h2>
<p>In one part of the <a target="_blank" rel="nofollow" href="http://www.win-vector.com/blog/2016/12/using-replyrlet-to-parameterize-dplyr-expressions/">Win-Vector blog</a> they have the following challenge:</p>
<blockquote>
<p>To write such a function in dplyr can get quite hairy, quite quickly. Try it yourself, and see</p>
</blockquote>
<p>So, with this in mind I have, with some fighting I might add, developed a similar function that has some added benefits beyond that of the <code>replyr</code> function using the <code>dplyr</code> package.</p>
<figure>
<pre><code data-lang="r"><span>dist_intervals_dplyr</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>dframe</span><span>,</span><span> </span><span>col</span><span>,</span><span> </span><span>group</span><span>)</span><span> </span><span>{</span><span>
</span><span>if</span><span>(</span><span>missing</span><span>(</span><span>group</span><span>))</span><span>
</span><span>{</span><span>
</span><span>df</span><span> </span><span>&lt;-</span><span> </span><span>dframe</span><span> </span><span>%>%</span><span> </span><span>select</span><span>(</span><span>.</span><span>,</span><span> </span><span>one_of</span><span>(</span><span>col</span><span>))</span><span> </span><span>}</span><span> </span><span>else</span><span> </span><span>{</span><span>
</span><span>df</span><span> </span><span>&lt;-</span><span> </span><span>dframe</span><span> </span><span>%>%</span><span> </span><span>group_by_</span><span>(</span><span>group</span><span>)</span><span> </span><span>%>%</span><span> </span><span>select</span><span>(</span><span>.</span><span>,</span><span> </span><span>one_of</span><span>(</span><span>c</span><span>(</span><span>group</span><span>,</span><span> </span><span>col</span><span>)))</span><span> </span><span>}</span><span>
</span><span>df</span><span> </span><span>%>%</span><span> </span><span>summarise_each</span><span>(</span><span>funs</span><span>(</span><span>sdlower</span><span> </span><span>=</span><span> </span><span>mean</span><span>(</span><span>.</span><span>)</span><span> </span><span>-</span><span> </span><span>sd</span><span>(</span><span>.</span><span>),</span><span>
</span><span>mean</span><span> </span><span>=</span><span> </span><span>mean</span><span>(</span><span>.</span><span>),</span><span>
</span><span>sdupper</span><span> </span><span>=</span><span> </span><span>mean</span><span>(</span><span>.</span><span>)</span><span> </span><span>+</span><span> </span><span>sd</span><span>(</span><span>.</span><span>),</span><span>
</span><span>iqrlower</span><span> </span><span>=</span><span> </span><span>median</span><span>(</span><span>.</span><span>)</span><span>-0.5</span><span>*</span><span>IQR</span><span>(</span><span>.</span><span>),</span><span>
</span><span>median</span><span> </span><span>=</span><span> </span><span>median</span><span>(</span><span>.</span><span>),</span><span>
</span><span>iqrupper</span><span> </span><span>=</span><span> </span><span>median</span><span>(</span><span>.</span><span>)</span><span>+0.5</span><span>*</span><span>IQR</span><span>(</span><span>.</span><span>)))</span><span>
</span><span>}</span></code></pre>
</figure>
<p>By combining the <code>one_of</code> and the standard standard evaluation of the <code>dplyr</code> functions we have recreated the results of the <code>replyr</code> function:</p>
<figure>
<pre><code data-lang="r"><span>Snails</span><span> </span><span>%>%</span><span> </span><span>dist_intervals_dplyr</span><span>(</span><span>col</span><span> </span><span>=</span><span> </span><span>&quot;Deaths&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">##      sdlower     mean  sdupper iqrlower median iqrupper
## 1 -0.8984496 2.864583 6.627616   -1.125      1    3.125</code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span>Snails</span><span> </span><span>%>%</span><span> </span><span>dist_intervals_dplyr</span><span>(</span><span>&quot;Deaths&quot;</span><span>,</span><span> </span><span>group</span><span> </span><span>=</span><span> </span><span>&quot;Species&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 2 × 7
##   Species    sdlower     mean  sdupper iqrlower median iqrupper
##    &lt;fctr>      &lt;dbl>    &lt;dbl>    &lt;dbl>    &lt;dbl>  &lt;dbl>    &lt;dbl>
## 1       A -0.5320915 1.708333 3.948758     -1.0    0.5      2.0
## 2       B -0.5477438 4.020833 8.589410     -1.5    2.0      5.5</code></pre>
</figure>
<p>The added benefit of using the dplyr notation is the ability to include <em>mutliple</em> columns in the function that you want to summarise:<br>
The <code>replyr</code> approach will not be able to handle this and will you give an error such as:</p>
<figure>
<pre><code data-lang="r"><span>Snails</span><span> </span><span>%>%</span><span> </span><span>dist_intervals</span><span>(</span><span>c</span><span>(</span><span>&quot;Deaths&quot;</span><span>,</span><span> </span><span>&quot;Exposure&quot;</span><span>),</span><span> </span><span>group</span><span> </span><span>=</span><span> </span><span>&quot;Species&quot;</span><span>)</span><span>
</span><span># Error in letprep(alias, strexpr) : </span><span> </span><span>#</span><span> </span><span>wrapr</span><span>:</span><span>let</span><span> </span><span>alias</span><span> </span><span>values</span><span> </span><span>must</span><span> </span><span>all</span><span> </span><span>be</span><span> </span><span>single</span><span> </span><span>strings</span><span> </span><span>(</span><span>not</span><span> </span><span>arrays</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span>Snails</span><span> </span><span>%>%</span><span> </span><span>dist_intervals_dplyr</span><span>(</span><span>c</span><span>(</span><span>&quot;Deaths&quot;</span><span>,</span><span> </span><span>&quot;Exposure&quot;</span><span>),</span><span> </span><span>group</span><span> </span><span>=</span><span> </span><span>&quot;Species&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 2 × 13
##   Species Deaths_sdlower Exposure_sdlower Deaths_mean Exposure_mean
##    &lt;fctr>          &lt;dbl>            &lt;dbl>       &lt;dbl>         &lt;dbl>
## 1       A     -0.5320915         1.370135    1.708333           2.5
## 2       B     -0.5477438         1.370135    4.020833           2.5
## # ... with 8 more variables: Deaths_sdupper &lt;dbl>, Exposure_sdupper &lt;dbl>,
## #   Deaths_iqrlower &lt;dbl>, Exposure_iqrlower &lt;dbl>, Deaths_median &lt;dbl>,
## #   Exposure_median &lt;dbl>, Deaths_iqrupper &lt;dbl>, Exposure_iqrupper &lt;dbl></code></pre>
</figure>
<p>And with a little bit of tidying of the data, you can have a much richer dataset</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>tidyr</span><span>)</span><span>
</span><span>Snails</span><span> </span><span>%>%</span><span> </span><span>dist_intervals_dplyr</span><span>(</span><span>c</span><span>(</span><span>&quot;Deaths&quot;</span><span>,</span><span> </span><span>&quot;Exposure&quot;</span><span>),</span><span> </span><span>group</span><span> </span><span>=</span><span> </span><span>&quot;Species&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>gather</span><span>(</span><span>key</span><span> </span><span>=</span><span> </span><span>Deaths</span><span>,</span><span> </span><span>Metric</span><span>,</span><span> </span><span>-</span><span>Species</span><span>)</span><span> </span><span>%>%</span><span> </span><span>arrange</span><span>(</span><span>Species</span><span>,</span><span> </span><span>Deaths</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 24 × 3
##    Species            Deaths     Metric
##     &lt;fctr>             &lt;chr>      &lt;dbl>
## 1        A   Deaths_iqrlower -1.0000000
## 2        A   Deaths_iqrupper  2.0000000
## 3        A       Deaths_mean  1.7083333
## 4        A     Deaths_median  0.5000000
## 5        A    Deaths_sdlower -0.5320915
## 6        A    Deaths_sdupper  3.9487582
## 7        A Exposure_iqrlower  1.7500000
## 8        A Exposure_iqrupper  3.2500000
## 9        A     Exposure_mean  2.5000000
## 10       A   Exposure_median  2.5000000
## # ... with 14 more rows</code></pre>
</figure>
<p>I do find that applying parameterized dplyr functions an amazing advantage when working with more advanced ETL workflows. This is by no means to say that the <code>replyr</code> package is redundant, but more a declaration to say that <code>dplyr</code> has the ability to dynamically conduct analysis – albeit some fighting to get it to work.</p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://eighty20.github.io//package%20exploration/2017/02/16/replyr-dplyr/"> Pareto's Playground</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/CHIkQwts87A"></a></div>