<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/use-pseudo-aggregators-to-add-safety-checks-to-your-data-wrangling-workflow/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.win-vector.com/blog/2018/10/use-pseudo-aggregators-to-add-safety-checks-to-your-data-wrangling-workflow/"> R – Win-Vector Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>One of the concepts we teach in both <a rel="nofollow" href="http://www.practicaldatascience.com/" target="_blank">Practical Data Science with R</a> and in our <a rel="nofollow" href="https://winvector.github.io/cdata/" target="_blank">theory of data shaping</a> is the importance of identifying the roles of columns in your data.</p>
<p>For example, to <a rel="nofollow" href="http://www.win-vector.com/blog/2018/10/designing-transforms-for-data-reshaping-with-cdata/" target="_blank">think in terms of multi-row records</a> it helps to identify:</p>
<ul>
<li>Which columns are keys (together identify rows or records).</li>
<li>Which columns are data/payload (are considered free varying data).</li>
<li>Which columns are &quot;derived&quot; (functions of the keys).</li>
</ul>
<p>In this note we will show how to use some of these ideas to write safer data-wrangling code.</p>
<p><span></span></p>
<p>In mathematics the statement &quot;y is a function of x&quot; merely means there is an ideal lookup table with which if you knew the value of <code>x</code>, then you could in principle know the value of <code>y</code>.</p>
<p>For example in the following <a rel="nofollow" href="https://www.r-project.org/" target="_blank"><code>R</code></a> <code>data.frame:</code> <code>y</code> is a function of <code>x</code>, as all rows that have the same value for <code>x</code> also have the same value for <code>y</code>.</p>
<div>
<pre><code>d1 &lt;-<span> </span><span>data.frame</span>(
  <span>x =</span> <span>c</span>(<span>&quot;a&quot;</span>, <span>&quot;a&quot;</span>, <span>&quot;b&quot;</span>, <span>&quot;b&quot;</span>, <span>&quot;c&quot;</span>),
  <span>y =</span> <span>c</span>(<span>10</span>, <span>10</span>, <span>20</span>, <span>20</span>, <span>20</span>), 
  <span>z =</span> <span>c</span>(<span>1</span>, <span>2</span>, <span>1</span>, <span>1</span>, <span>1</span>),
  <span>stringsAsFactors =</span> <span>FALSE</span>)
<span>print</span>(d1)</code></pre>
</div>
<pre><code>##   x  y z
## 1 a 10 1
## 2 a 10 2
## 3 b 20 1
## 4 b 20 1
## 5 c 20 1</code></pre>
<p>Notice if we know the value of <code>x</code> we then, in principle know the value of <code>y</code>. In the same example <code>z</code> is not a function of <code>x</code>, as it does not have this property.</p>
<p>A more concrete example would be: user-name is a function of user-ID. If you know the an individual’s user-ID, then you also (if you have the right lookup table) know the individual’s user-name.</p>
<p>We first taught these concepts in the context of <a rel="nofollow" href="https://en.wikipedia.org/wiki/SQL" target="_blank"><code>SQL</code></a> and <code>SQL</code> grouped aggregation. In <code>SQL</code> once you aggregate on one column, then all other columns in your query must either be the grouping columns, or also aggregated. This is easiest to show in code, but we will use the <code>dplyr</code> package for our example.</p>
<div>
<pre><code><span>library</span>(<span>&quot;dplyr&quot;</span>)

d1 %>%
<span> </span><span>group_by</span>(x) %>%
<span> </span><span>summarize</span>(<span>y =</span> <span>max</span>(y)) %>%
<span> </span><span>ungroup</span>()</code></pre>
</div>
<pre><code>## # A tibble: 3 x 2
##   x         y
##   </code></pre>