<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/is-r-basesubset-really-that-bad/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.win-vector.com/blog/2018/02/is-r-basesubset-really-that-bad/"> R – Win-Vector Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Is <a rel="nofollow" href="https://cran.r-project.org/" target="_blank"><code>R</code></a> <a rel="nofollow" href="https://www.rdocumentation.org/packages/base/versions/3.4.3/topics/subset?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank"><code>base::subset()</code></a> really that bad?</p>
<p><a rel="nofollow" href="https://commons.wikimedia.org/wiki/File:The_Hitchhiker%27s_Guide_to_the_Galaxy.svg" target="_blank"><img border="0" src="https://i2.wp.com/www.win-vector.com/blog/wp-content/uploads/2018/02/The_Hitchhikers_Guide_to_the_Galaxy.svg_.png?resize=599%2C468" alt="The Hitchhiker s Guide to the Galaxy svg" width="450" title="The_Hitchhiker's_Guide_to_the_Galaxy.svg.png" data-recalc-dims="1"></a></p>
<p><span></span></p>
<p>Notes discussing <code>subset()</code> often refer to the following text (from <code>help(subset)</code>, referred to in examples: <a rel="nofollow" href="https://adv-r.hadley.nz/evaluation.html#base-subset" target="_blank">1</a>, <a rel="nofollow" href="https://community.rstudio.com/t/teaching-dplyr-functions-which-have-base-r-equivalents/3775/20" target="_blank">2</a>):</p>
<blockquote>
<p>
<strong>Warning</strong></p>
<p>This is a convenience function intended for use interactively. For programming it is better to use the standard sub-setting functions like <code>[</code>, and in particular the non-standard evaluation of argument <code>subset</code> can have unanticipated consequences.
</p>
</blockquote>
<p>Is it really obvious the <code>subset()</code> authors or describers are apologizing solely for the implementation, or could they be expressing reservations about wholesale use/implementation of non-standard evaluation in general? It could be a situation similar to the following dialogue from &quot;The Hitchhiker’s Guide to the Galaxy&quot; by Douglas Adams:</p>
<blockquote>
<p>
Zaphod stared at Arthur.</p>
<p>&quot;Did you think of that, Earthman?&quot; he demanded.</p>
<p>&quot;Well,&quot; said Arthur, &quot;all I did was…&quot;</p>
<p>&quot;That’s very good thinking, you know. Turn on the Improbability Drive for a second without first activating the proofing screens. Hey, kid, you just saved our lives, you know that?&quot;</p>
<p>&quot;Oh,&quot; said Arthur, &quot;well, it was nothing really…&quot;</p>
<p>&quot;Was it?&quot; said Zaphod. &quot;Oh well, forget it then. Okay computer, take us in to land.&quot;</p>
<p>&quot;But…&quot;</p>
<p>&quot;I said forget it.&quot;
</p>
</blockquote>
<p><code>subset()</code>‘s code for <code>data.frame</code>s is actually fairly short and clear (and easy to examine via <code>print(subset.data.frame)</code>). Objections could include:</p>
<ol>
<li>The result of <code>subset()</code> depends on more than just the values it is applied to (it also depends on exact code passed to subset, and the contents of various environments).</li>
<li>There is no way to unambiguously refer to column names of the <code>data.frame</code> in question.</li>
<li>There is no way to unambiguously refer to items in <em>arbitrary</em> environments.</li>
</ol>
<p>Let comment on these points in order.</p>
<ol>
<li>The first point is the nature of non-standard evaluation. It will follow you wherever you implement non-<a rel="nofollow" href="https://en.wikipedia.org/wiki/Referential_transparency" target="_blank">referentially transparent</a> evaluation schemes.</li>
<li>The second point is valid, and <a rel="nofollow" href="https://cran.r-project.org/package=rlang" target="_blank">some authors</a> advocate adding a prefix to refer to the <code>data.frame</code>.</li>
<li>The third point is perhaps, here, a bit over-stated. The R convention is: environments nest, they are structured (not spaghetti pointers). The R programmer expects this. If we know both the name we want and the environment we want to de-reference it in, we can often just take an extra step and grab the value in question. Adding the ability to refer to arbitrary environments is powerful, but it is the data equivalent to adding a &quot;<code>goto</code>&quot; statement to a <a rel="nofollow" href="https://en.wikipedia.org/wiki/Structured_programming" target="_blank">structured programming language</a>. From a clean design point of view, all one really needs is another prefix to refer to the expected environment.</li>
</ol>
<p>One can quickly create a variation of subset that allows for both data and environment prefixes. Below is a simple implementation based on the above ideas and minor edits to the original <code>subset()</code> code.</p>
<div>
<pre><code><span>#' Pick a subset of rows, evaluating the subset expression </span>
<span>#' as if the columns of x were in the evaluation environment.</span>
<span>#' References can be forced to the environment with a .e$ prefix</span>
<span>#' and forced to the data frame with a .d$ prefix (failure to</span>
<span>#' lookup returns null).</span>
<span>#' </span>
<span>#' @param x data.frame to work with</span>
<span>#' @param subset logical expression to compute per-row</span>
<span>#' @param env environment to work in</span>
<span>#' @return data.frame that is the specified subset of the rows of x.</span>
<span>#' </span>
<span>#' @seealso \code{\link[base]{subset}} </span>
<span>#' </span>
<span>#' @examples</span>
<span>#' </span>
<span>#' Temp &lt;-90</span>
<span>#' subset_rows(airquality, (.d$Temp > .e$Temp) &amp; !is.na(Ozone))</span>
<span>#' </span>
<span>#' </span>
subset_rows &lt;-<span> </span>function(x, subset, <span>env =</span> <span>parent.frame</span>()) {
  if(!<span>is.data.frame</span>(x)) {
    <span>stop</span>(<span>&quot;subset_rows expected x to be a data.frame&quot;</span>)
  }
  if(<span>missing</span>(subset) ||<span> </span>(<span>nrow</span>(x)&lt;=<span>0</span>)) {
    <span>return</span>(x)
  }
  e &lt;-<span> </span><span>substitute</span>(subset) <span># capture expression</span>
  eval_env &lt;-<span> </span><span>new.env</span>(<span>parent =</span> env)
  <span>assign</span>(<span>&quot;.d&quot;</span>, x, eval_env)   <span># data prefix .d$</span>
  <span>assign</span>(<span>&quot;.e&quot;</span>, env, eval_env) <span># environment prefix .e$</span>
  r &lt;-<span> </span><span>eval</span>(e, <span>envir =</span> x, <span>enclos =</span> eval_env)
  if(!<span>is.logical</span>(r)) {
    <span>stop</span>(<span>&quot;subset_rows predicate must evaluate to logical&quot;</span>)
  }
  r &lt;-<span> </span>r &amp;<span> </span>!<span>is.na</span>(r)
  if(<span>length</span>(r)!=<span>nrow</span>(x)) {
    <span>stop</span>(<span>&quot;subset_rows predicate must have one entry per row&quot;</span>)
  }
  x[r, , drop =<span> </span><span>FALSE</span>]
}</code></pre>
</div>
<p>The idea is:</p>
<ul>
<li>By default names are de-referenced as possible <code>data.frame</code> column names, failing that are matched to the evaluation environment.</li>
<li>Names prefixed by <code>.d$</code> can only refer to columns of the <code>data.frame</code>.</li>
<li>Names prefixed by <code>.e$</code> can only refer to the evaluation environment.</li>
</ul>
<p>The above code has deliberately different semantics than <code>base::subset()</code> (else, why write it): the prefix ability, no &quot;drop&quot;, skips predicate evaluation on zero-row <code>data.frame</code>s, checks predicate length, no use of &quot;<code>...</code>&quot;, and executes in a more deeply nested environment. Beyond style differences the change from <code>base::substitute()</code> are really only the three lines of code directly manipulating environments. We feel we have thrown out a lot of bath water while keeping quite a lot of the baby.</p>
<p>This is enough to work a standard problem: suppose our <code>data.frame</code> has a column named &quot;<code>Temp</code>&quot; and we want to select rows that have a value in the &quot;<code>Temp</code>&quot; column that exceeded the value stored in a variable which is also named &quot;<code>Temp</code>&quot; (plus some other conditions). The common way out of that is to re-design a bit and store a reference to the threshold in a fresh variable named &quot;<code>Temp_bound</code>&quot; (which is why some of the above issues will be unfamiliar to some <code>R</code> users, they are often easy to avoid). The user can specify a variable or column named <code>.d</code> or <code>.e</code> by adding a prefix (such as <code>.e$.e</code>, but this does mean they must specify if they are looking at the <code>data.frame</code> or the environment, for these two name they do not have the luxury of saying &quot;either&quot;).</p>
<p>The prefix style solution is given below:</p>
<div>
<pre><code>Temp &lt;-<span> </span><span>90</span>
Ozone_bound &lt;-<span> </span><span>100</span>
<span>subset_rows</span>(airquality, 
            (.d$Temp ><span> </span>.e$Temp) &amp;<span> </span>
<span> </span>(!<span>is.na</span>(Ozone)) &amp;<span> </span>(Ozone <span> </span>Ozone_bound))</code></pre>
</div>
<pre><code>##     Ozone Solar.R Wind Temp Month Day
## 69     97     267  6.3   92     7   8
## 70     97     272  5.7   92     7   9
## 120    76     203  9.7   97     8  28
## 122    84     237  6.3   96     8  30
## 123    85     188  6.3   94     8  31
## 124    96     167  6.9   91     9   1
## 125    78     197  5.1   92     9   2
## 126    73     183  2.8   93     9   3
## 127    91     189  4.6   93     9   4</code></pre>
<p>We do not actually need the &quot;<code>.d$</code>&quot; prefix on &quot;<code>.d$Temp</code>&quot;, as the rules are the <code>data.frame</code> is checked first. However, adding it lowered the ambiguity to the reader. We know the <code>.d$Temp</code> and <code>.e$Temp</code> must be referring to different columns, as for standard numbers (non <code>NA</code>/<code>NaN</code>) we do not have &quot;<code>Temp > Temp</code>&quot;. Notice we did not need to write &quot;<code>.e$Ozone_bound</code>&quot; (though that would work if we so choose).</p>
<p>We have only shown the solution in an interactive environment, but with a few conventions (such as building up long names) the code works about as well as any other when writing re-usable functions or packages.</p>
<p>I think the above if fairly teachable, and working through it has the side benefit that it gives one a chance learn about <code>R</code> itself.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.win-vector.com/blog/2018/02/is-r-basesubset-really-that-bad/"> R – Win-Vector Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/rNA-rCoEnSg" height="1"></a></div>