<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/dplyr-0-5-0/"></div></div><div><a data-url="http://www.r-bloggers.com/dplyr-0-5-0/" data-counturl="http://www.r-bloggers.com/dplyr-0-5-0/" data-count="horizontal" data-text="dplyr 0.5.0" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://blog.rstudio.org/2016/06/27/dplyr-0-5-0/"> RStudio Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I’m very pleased to announce that dplyr 0.5.0 is now available from CRAN. Get the latest version with:</p>
<pre><code>install.packages(&quot;dplyr&quot;)</code></pre>
<p>dplyr 0.5.0 is a big release with a heap of new features, a whole bunch of minor improvements, and many bug fixes, both from me and from the broader dplyr community. In this blog post, I’ll highlight the most important changes:</p>
<ul>
<li>Some breaking changes to single table verbs.</li>
<li>New tibble and dtplyr packages.</li>
<li>New vector functions.</li>
<li>Replacements for <code>summarise_each()</code> and <code>mutate_each()</code>.</li>
<li>Improvements to SQL translation.</li>
</ul>
<p>To see the complete list, please read the <a target="_blank" href="https://github.com/hadley/dplyr/releases/tag/v0.5.0">release notes</a>.</p>
<div>
<h2>Breaking changes</h2>
<p><code>arrange()</code> once again ignores grouping, reverting back to the behaviour of dplyr 0.3 and earlier. This makes <code>arrange()</code> inconsistent with other dplyr verbs, but I think this behaviour is generally more useful. Regardless, it’s not going to change again, as more changes will just cause more confusion.</p>
<pre><code>mtcars %>% 
  group_by(cyl) %>% 
  arrange(desc(mpg))
#> Source: local data frame [32 x 11]
#> Groups: cyl [3]
#> 
#> # A tibble: 32 x 11
#>     mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
#>   &lt;dbl> &lt;dbl> &lt;dbl> &lt;dbl> &lt;dbl> &lt;dbl> &lt;dbl> &lt;dbl> &lt;dbl> &lt;dbl> &lt;dbl>
#> 1  33.9     4  71.1    65  4.22 1.835 19.90     1     1     4     1
#> 2  32.4     4  78.7    66  4.08 2.200 19.47     1     1     4     1
#> 3  30.4     4  75.7    52  4.93 1.615 18.52     1     1     4     2
#> 4  30.4     4  95.1   113  3.77 1.513 16.90     1     1     5     2
#> 5  27.3     4  79.0    66  4.08 1.935 18.90     1     1     4     1
#> ... with 27 more rows</code></pre>
<p>If you give <code>distinct()</code> a list of variables, it now only keeps those variables (instead of, as previously, keeping the first value from the other variables). To preserve the previous behaviour, use <code>.keep_all = TRUE</code>:</p>
<pre><code>df &lt;- data_frame(x = c(1, 1, 1, 2, 2), y = 1:5)

# Now only keeps x variable
df %>% distinct(x)
#> # A tibble: 2 x 1
#>       x
#>   &lt;dbl>
#> 1     1
#> 2     2

# Previous behaviour preserved all variables
df %>% distinct(x, .keep_all = TRUE)
#> # A tibble: 2 x 2
#>       x     y
#>   &lt;dbl> &lt;int>
#> 1     1     1
#> 2     2     4</code></pre>
<p>The <code>select()</code> helper functions <code>starts_with()</code>, <code>ends_with()</code>, etc are now real exported functions. This means that they have better documentation, and there’s an extension mechnaism if you want to write your own helpers.</p>
</div>
<div>
<h2>Tibble and dtplyr packages</h2>
<p>Functions related to the creation and coercion of <code>tbl_df</code>s (“tibble”s for short), now live in their own package: <a target="_blank" href="https://blog.rstudio.org/2016/03/24/tibble-1-0-0/">tibble</a>. See <code>vignette(&quot;tibble&quot;)</code> for more details.</p>
<p>Similarly, all code related to the data table dplyr backend code has been separated out in to a new <a target="_blank" href="https://github.com/hadley/dtplyr">dtplyr</a> package. This decouples the development of the data.table interface from the development of the dplyr package, and I hope will spur improvements to the backend. If both data.table and dplyr are loaded, you’ll get a message reminding you to load dtplyr.</p>
</div>
<div>
<h2>Vector functions</h2>
<p>This version of dplyr gains a number of vector functions inspired by SQL. Two functions make it a little easier to eliminate or generate missing values:</p>
<ul>
<li>Given a set of vectors, <code>coalesce()</code> finds the first non-missing value in each position:
<pre><code>x &lt;- c(1,  2, NA, 4, NA, 6)
y &lt;- c(NA, 2,  3, 4,  5, NA)

# Use this to piece together a complete vector:
coalesce(x, y)
#> [1] 1 2 3 4 5 6

# Or just replace missing value with a constant:
coalesce(x, 0)
#> [1] 1 2 0 4 0 6</code></pre>
</li>
<li>The complement of <code>coalesce()</code> is <code>na_if()</code>: it replaces a specified value with an <code>NA</code>.
<pre><code>x &lt;- c(1, 5, 2, -99, -99, 10)
na_if(x, -99)
#> [1]  1  5  2 NA NA 10</code></pre>
</li>
</ul>
<p>Three functions provide convenient ways of replacing values. In order from simplest to most complicated, they are:</p>
<ul>
<li><code>if_else()</code>, a vectorised if statement, takes a logical vector (usually created with a comparison operator like <code>==</code>, <code>&lt;</code>, or <code>%in%</code>) and replaces <code>TRUE</code>s with one vector and <code>FALSE</code>s with another.
<pre><code>x1 &lt;- sample(5)
if_else(x1 &lt; 5, &quot;small&quot;, &quot;big&quot;)
#> [1] &quot;small&quot; &quot;small&quot; &quot;big&quot;   &quot;small&quot; &quot;small&quot;</code></pre>
<p><code>if_else()</code> is similar to <code>base::ifelse()</code>, but has two useful improvements.<br>
First, it has a fourth argument that will replace missing values:</p>
<pre><code>x2 &lt;- c(NA, x1)
if_else(x2 &lt; 5, &quot;small&quot;, &quot;big&quot;, &quot;unknown&quot;)
#> [1] &quot;unknown&quot; &quot;small&quot;   &quot;small&quot;   &quot;big&quot;     &quot;small&quot;   &quot;small&quot;</code></pre>
<p>Secondly, it also have stricter semantics that <code>ifelse()</code>: the <code>true</code> and <code>false</code> arguments must be the same type. This gives a less surprising return type, and preserves S3 vectors like dates and factors:</p>
<pre><code>x &lt;- factor(sample(letters[1:5], 10, replace = TRUE))
ifelse(x %in% c(&quot;a&quot;, &quot;b&quot;, &quot;c&quot;), x, factor(NA))
#>  [1] NA NA  1 NA  3  2  3 NA  3  2
if_else(x %in% c(&quot;a&quot;, &quot;b&quot;, &quot;c&quot;), x, factor(NA))
#>  [1] &lt;NA> &lt;NA> a    &lt;NA> c    b    c    &lt;NA> c    b   
#> Levels: a b c d e</code></pre>
<p>Currently, <code>if_else()</code> is very strict, so you’ll need to careful match the types of <code>true</code> and <code>false</code>. This is most likely to bite you when you’re using missing values, and you’ll need to use a specific <code>NA</code>: <code>NA_integer_</code>, <code>NA_real_</code>, or <code>NA_character_</code>:</p>
<pre><code>if_else(TRUE, 1, NA)
#> Error: `false` has type 'logical' not 'double'
if_else(TRUE, 1, NA_real_)
#> [1] 1</code></pre>
</li>
<li><code>recode()</code>, a vectorised <code>switch()</code>, takes a numeric vector, character vector, or factor, and replaces elements based on their values.
<pre><code>x &lt;- sample(c(&quot;a&quot;, &quot;b&quot;, &quot;c&quot;, NA), 10, replace = TRUE)

# The default is to leave non-replaced values as is
recode(x, a = &quot;Apple&quot;)
#>  [1] &quot;c&quot;     &quot;Apple&quot; NA      NA      &quot;c&quot;     NA      &quot;b&quot;     NA     
#>  [9] &quot;c&quot;     &quot;Apple&quot;
# But you can choose to override the default:
recode(x, a = &quot;Apple&quot;, .default = NA_character_)
#>  [1] NA      &quot;Apple&quot; NA      NA      NA      NA      NA      NA     
#>  [9] NA      &quot;Apple&quot;
# You can also choose what value is used for missing values
recode(x, a = &quot;Apple&quot;, .default = NA_character_, .missing = &quot;Unknown&quot;)
#>  [1] NA        &quot;Apple&quot;   &quot;Unknown&quot; &quot;Unknown&quot; NA        &quot;Unknown&quot; NA       
#>  [8] &quot;Unknown&quot; NA        &quot;Apple&quot;</code></pre>
</li>
<li><code>case_when()</code>, is a vectorised set of <code>if</code> and <code>else if</code>s. You provide it a set of test-result pairs as formulas: The left side of the formula should return a logical vector, and the right hand side should return either a single value, or a vector the same length as the left hand side. All results must be the same type of vector.
<pre><code>x &lt;- 1:40
case_when(
  x %% 35 == 0 ~ &quot;fizz buzz&quot;,
  x %% 5 == 0 ~ &quot;fizz&quot;,
  x %% 7 == 0 ~ &quot;buzz&quot;,
  TRUE ~ as.character(x)
)
#>  [1] &quot;1&quot;         &quot;2&quot;         &quot;3&quot;         &quot;4&quot;         &quot;fizz&quot;     
#>  [6] &quot;6&quot;         &quot;buzz&quot;      &quot;8&quot;         &quot;9&quot;         &quot;fizz&quot;     
#> [11] &quot;11&quot;        &quot;12&quot;        &quot;13&quot;        &quot;buzz&quot;      &quot;fizz&quot;     
#> [16] &quot;16&quot;        &quot;17&quot;        &quot;18&quot;        &quot;19&quot;        &quot;fizz&quot;     
#> [21] &quot;buzz&quot;      &quot;22&quot;        &quot;23&quot;        &quot;24&quot;        &quot;fizz&quot;     
#> [26] &quot;26&quot;        &quot;27&quot;        &quot;buzz&quot;      &quot;29&quot;        &quot;fizz&quot;     
#> [31] &quot;31&quot;        &quot;32&quot;        &quot;33&quot;        &quot;34&quot;        &quot;fizz buzz&quot;
#> [36] &quot;36&quot;        &quot;37&quot;        &quot;38&quot;        &quot;39&quot;        &quot;fizz&quot;</code></pre>
<p><code>case_when()</code> is still somewhat experiment and does not currently work inside <code>mutate()</code>. That will be fixed in a future version.

<p>I also added one small helper for dealing with floating point comparisons: <code>near()</code> tests for equality with numeric tolerance (<code>abs(x - y) &lt; tolerance</code>).</p>
<pre><code>x &lt;- sqrt(2) ^ 2

x == 2
#> [1] FALSE
near(x, 2)
#> [1] TRUE</code></pre>

<div>
<h2>Predicate functions</h2>
<p>Thanks to ideas and code from <a target="_blank" href="http://github.com/lionel-">Lionel Henry</a>, a new family of functions improve upon <code>summarise_each()</code> and <code>mutate_each()</code>:</p>
<ul>
<li><code>summarise_all()</code> and <code>mutate_all()</code> apply a function to all (non-grouped) columns:
<pre><code>mtcars %>% group_by(cyl) %>% summarise_all(mean)    
#> # A tibble: 3 x 11
#>     cyl      mpg     disp        hp     drat       wt     qsec        vs
#>   &lt;dbl>    &lt;dbl>    &lt;dbl>     &lt;dbl>    &lt;dbl>    &lt;dbl>    &lt;dbl>     &lt;dbl>
#> 1     4 26.66364 105.1364  82.63636 4.070909 2.285727 19.13727 0.9090909
#> 2     6 19.74286 183.3143 122.28571 3.585714 3.117143 17.97714 0.5714286
#> 3     8 15.10000 353.1000 209.21429 3.229286 3.999214 16.77214 0.0000000
#> ... with 3 more variables: am &lt;dbl>, gear &lt;dbl>, carb &lt;dbl></code></pre>
</li>
<li><code>summarise_at()</code> and <code>mutate_at()</code> operate on a subset of columns. You can select columns with:
<ul>
<li>a character vector of column names,</li>
<li>a numeric vector of column positions, or</li>
<li>a column specification with <code>select()</code> semantics generated with the new <code>vars()</code> helper.</li>
</ul>
<pre><code>mtcars %>% group_by(cyl) %>% summarise_at(c(&quot;mpg&quot;, &quot;wt&quot;), mean)
#> # A tibble: 3 x 3
#>     cyl      mpg       wt
#>   &lt;dbl>    &lt;dbl>    &lt;dbl>
#> 1     4 26.66364 2.285727
#> 2     6 19.74286 3.117143
#> 3     8 15.10000 3.999214
mtcars %>% group_by(cyl) %>% summarise_at(vars(mpg, wt), mean)
#> # A tibble: 3 x 3
#>     cyl      mpg       wt
#>   &lt;dbl>    &lt;dbl>    &lt;dbl>
#> 1     4 26.66364 2.285727
#> 2     6 19.74286 3.117143
#> 3     8 15.10000 3.999214</code></pre>
</li>
<li><code>summarise_if()</code> and <code>mutate_if()</code> take a predicate function (a function that returns <code>TRUE</code> or <code>FALSE</code> when given a column). This makes it easy to apply a function only to numeric columns:
<pre><code>iris %>% summarise_if(is.numeric, mean)
#>   Sepal.Length Sepal.Width Petal.Length Petal.Width
#> 1     5.843333    3.057333        3.758    1.199333</code></pre>
</li>
</ul>
<p>All of these functions pass <code>...</code> on to the individual <code>funs</code>:</p>
<pre><code>iris %>% summarise_if(is.numeric, mean, trim = 0.25)
#>   Sepal.Length Sepal.Width Petal.Length Petal.Width
#> 1     5.802632    3.032895     3.934211    1.230263</code></pre>
<p>A new <code>select_if()</code> allows you to pick columns with a predicate function:</p>
<pre><code>df &lt;- data_frame(x = 1:3, y = c(&quot;a&quot;, &quot;b&quot;, &quot;c&quot;))
df %>% select_if(is.numeric)
#> # A tibble: 3 x 1
#>       x
#>   &lt;int>
#> 1     1
#> 2     2
#> 3     3
df %>% select_if(is.character)
#> # A tibble: 3 x 1
#>       y
#>   &lt;chr>
#> 1     a
#> 2     b
#> 3     c</code></pre>
<p><code>summarise_each()</code> and <code>mutate_each()</code> will be deprecated in a future release.</p>
</div>
<div>
<h2>SQL translation</h2>
<p>I have completely overhauled the translation of dplyr verbs into SQL statements. Previously, dplyr used a rather ad-hoc approach which tried to guess when a new subquery was needed. Unfortunately this approach was fraught with bugs, so I have now implemented a richer internal data model. In the short-term, this is likely to lead to some minor performance decreases (as the generated SQL is more complex), but the dplyr is much more likely to generate correct SQL. In the long-term, these abstractions will make it possible to write a query optimiser/compiler in dplyr, which would make it possible to generate much more succinct queries. If you know anything about writing query optimisers or compilers and are interested in working on this problem, please let me know!</p>
</div>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/rstudioblog.wordpress.com/2859/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/rstudioblog.wordpress.com/2859/" border="0"></a> <img alt="" src="http://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/dplyr-0-5-0/"></div></div><div><a data-url="http://www.r-bloggers.com/dplyr-0-5-0/" data-counturl="http://www.r-bloggers.com/dplyr-0-5-0/" data-count="vertical" data-text="dplyr 0.5.0" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://blog.rstudio.org/2016/06/27/dplyr-0-5-0/"> RStudio Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/8fhfb_cbKc4"></a></div></ul></div>