<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/lets-have-some-sympathy-for-the-part-time-r-user/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.win-vector.com/blog/2017/08/lets-have-some-sympathy-for-the-part-time-r-user/"> R – Win-Vector Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>When I started writing about methods for better <a rel="nofollow" href="http://www.win-vector.com/blog/2016/12/parametric-variable-names-and-dplyr/" target="_blank">&quot;parametric programming&quot; interfaces for <code>dplyr</code></a> for <a rel="nofollow" href="https://www.r-project.org/" target="_blank"><code>R</code></a> <a rel="nofollow" href="http://dplyr.tidyverse.org/" target="_blank"><code>dplyr</code></a> users in December of 2016 I encountered three divisions in the audience:</p>
<ul>
<li><code>dplyr</code> users who had such a need, and wanted such extensions.</li>
<li><code>dplyr</code> users who did not have such a need (&quot;we always know the column names&quot;).</li>
<li><code>dplyr</code> users who found the then-current fairly complex &quot;underscore&quot; and <a rel="nofollow" href="https://cran.r-project.org/web/packages/lazyeval/index.html" target="_blank"><code>lazyeval</code></a> system sufficient for the task.</li>
</ul>
<p>Needing name substitution is a problem an advanced full-time <code>R</code> user can solve on their own. However a part-time <code>R</code> would greatly benefit from a simple, reliable, readable, documented, and comprehensible packaged solution.<span></span></p>
<h2>Background</h2>
<p>Roughly I suggested two possible methods for making the task easier:</p>
<ul>
<li>Renaming <a rel="nofollow" href="https://en.wikipedia.org/wiki/View_(SQL)" target="_blank">views</a> for <code>data.frame</code>s. I have now implemented the idea as a call-scoped concept in <code>replyr::replyr_apply_f_mapped()</code> (&quot;call-scoped&quot;, meaning the re-mapping lasts for the duration of a function call).</li>
<li>Symbol re-binding by a block-scoped command called <a rel="nofollow" href="https://winvector.github.io/wrapr/reference/let.html" target="_blank"><code>let()</code></a> (<a rel="nofollow" href="https://en.wikipedia.org/wiki/Lisp_(programming_language)#Self-evaluating_forms_and_quoting" target="_blank">a common functional notation</a>; and &quot;block-scoped&quot; meaning the re-mapping lasts for the span of a code-block). I released this solution to <code>CRAN</code> and publicly announced it on <a rel="nofollow" href="http://www.win-vector.com/blog/2016/12/using-replyrlet-to-parameterize-dplyr-expressions/#comment-66361" target="_blank">December 8 2016</a>.</li>
</ul>
<p>I mention dates to point out that this is something I have been inviting public comment on for some time.</p>
<p>Things change. Since the above time:</p>
<ul>
<li>The development version of <code>dplyr</code> incorporated a new <a rel="nofollow" href="https://github.com/tidyverse/rlang" target="_blank"><code>rlang</code>/<code>tidyeval</code></a> package (<a rel="nofollow" href="https://github.com/tidyverse/dplyr/commit/7d34aea17cb6806992acb2b1cc59a5484148aa03" target="_blank">probably around February 14th 2017</a>).</li>
<li><code>rlang</code>/<code>tidyeval</code> was <a rel="nofollow" href="https://cran.r-project.org/src/contrib/Archive/rlang/" target="_blank">released to CRAN on May 2017</a>. Obviously <code>rlang</code>/<code>tidyeval</code> had been under development for some time, but I don’t think the parametric aspect of it was publicly discussed much before <a rel="nofollow" href="https://github.com/tidyverse/dplyr/issues/1600#issuecomment-280453923" target="_blank">February 16, 2017</a> (notice that a formula centric interface was still being contemplated).</li>
<li><code>dplyr</code> <code>0.7.0</code> was released, based on <code>rlang</code>/<code>tidyeval</code> (June 9th, 2017).</li>
<li><code>dplyr</code> excised direct use of <code>lazyeval</code>.</li>
<li>The <code>dplyr</code> &quot;underscore verbs&quot; (or methods) were all deprecated (i.e., no longer advised).</li>
</ul>
<p>The <code>rlang</code>/<code>tidyeval</code> strategy is to capture un-evaluated user expressions (as a new object called a &quot;<a rel="nofollow" href="http://rlang.tidyverse.org/reference/quosure.html" target="_blank"><code>quosure</code></a>&quot;) and <a rel="nofollow" href="http://rlang.tidyverse.org/reference/eval_tidy_.html" target="_blank">evaluate them with new language rules</a> (with new bindings and something called an &quot;<code>overscope</code>&quot;). Also note the <code>rlang</code>/<code>tidyeval</code> strategy is full integration or re-writing of packages in terms of <code>rlang</code>/<code>tidyeval</code>; this isn’t something you mix-in or turn on or off.</p>
<p>Some points I think that have been under-represented in previous discussions include:</p>
<ul>
<li>Not all <code>R</code> users consider themselves to be expert programmers (many are happy calling themselves analysts or statisticians).</li>
<li><code>R</code> is often used in collaborative projects where there are varying levels of programming expertise.</li>
</ul>
<p>The second point I think is particularly interesting. It means:</p>
<blockquote>
<p>An <code>R</code> user who does not consider themselves an expert programmer could be maintaining code that they understand, but could not be expected to create from scratch.</p>
</blockquote>
<p>Or:</p>
<blockquote>
<p>Let’s have some sympathy for the part-time <code>R</code> user.</p>
</blockquote>
<p>This is the point we will emphasize in our new example.</p>
<h2>The example</h2>
<p>The design and discussion of substitution solutions should be driven from concrete realistic use cases. Working from larger examples gives us a taste of what working with each solution is like in practice. So, let’s pretend to discuss social science (instead of programming).</p>
<p>Suppose an analyst, psychologist, medical doctor, or scientist is building an assessment for some aspects of behavior and anxiety.</p>
<p>Often such assessments involve selecting moving through a multiple-choice questionnaire and collecting a number of points that depend on answers selected. One such assessment is the <a rel="nofollow" href="https://en.wikipedia.org/wiki/Generalized_Anxiety_Disorder_7" target="_blank">Generalized Anxiety Disorder 7 questionnaire</a> (or GAD-7). It is a very simple system as can be seen below.</p>
<p><a rel="nofollow" href="https://www.integration.samhsa.gov/clinical-practice/GAD708.19.08Cartwright.pdf" target="_blank"><img border="0" src="https://i0.wp.com/www.win-vector.com/blog/wp-content/uploads/2017/08/GAD708.19.08Cartwright.png?resize=463%2C600" alt="GAD708 19 08Cartwright" title="GAD708.19.08Cartwright.png" data-recalc-dims="1">
<p>One can treat such a test score as a classifier and <a rel="nofollow" href="http://jamanetwork.com/journals/jamainternalmedicine/fullarticle/410326" target="_blank">assess it</a> in terms of sensitivity, specificity, and <a rel="nofollow" href="http://www.win-vector.com/blog/2016/07/a-budget-of-classifier-evaluation-measures/" target="_blank">different correspondence measures</a>.</p>
<p>An obvious extension of such tests is to give a different number of points in different categories for each multiple-choice answer. For example we could imagine such a test where each answer gave a varying number of points in one of two categories called &quot;withdrawal behavior&quot; and &quot;positive re-framing&quot; (both in the sense of <a rel="nofollow" href="https://en.wikipedia.org/wiki/Coping_(psychology)" target="_blank">coping behaviors</a>).</p>
<p>For example, our scientist might record the results of two subjects taking a test as follows:</p>
<div>
<pre><code>d &lt;-<span> </span><span>data.frame</span>(
  <span>subjectID =</span> <span>c</span>(<span>1</span>,                   
                <span>1</span>,
                <span>2</span>,                   
                <span>2</span>),
  <span>surveyCategory =</span> <span>c</span>(
    <span>'withdrawal behavior'</span>,
    <span>'positive re-framing'</span>,
    <span>'withdrawal behavior'</span>,
    <span>'positive re-framing'</span>
  ),
  <span>assessmentTotal =</span> <span>c</span>(<span>5</span>,                 
                      <span>2</span>,
                      <span>3</span>,                  
                      <span>4</span>),
  <span>stringsAsFactors =</span> <span>FALSE</span>
)
  
<span>print</span>(d)</code></pre>
</div>
<pre><code>##   subjectID      surveyCategory assessmentTotal
## 1         1 withdrawal behavior               5
## 2         1 positive re-framing               2
## 3         2 withdrawal behavior               3
## 4         2 positive re-framing               4</code></pre>
<div>
<pre><code><span># or in &quot;wide form&quot;:</span>
<span>library</span>(<span>&quot;cdata&quot;</span>)
<span>moveValuesToColumns</span>(d, 
                    <span>columnToTakeKeysFrom =</span> <span>'surveyCategory'</span>,
                    <span>columnToTakeValuesFrom =</span> <span>'assessmentTotal'</span>,
                    <span>rowKeyColumns =</span> <span>'subjectID'</span>)</code></pre>
</div>
<pre><code>##   subjectID positive re-framing withdrawal behavior
## 1         1                   2                   5
## 2         2                   4                   3</code></pre>
<p>A natural question is: how does one assign weights to each answer? One way would be to administer the test to a number of people the experimenter has classified as having either of the above mentioned behaviors and then performing a <a rel="nofollow" href="http://www.win-vector.com/blog/2011/09/the-simpler-derivation-of-logistic-regression/" target="_blank">logistic regression</a> to map assessment answers to the probability of a given diagnosis for this population. By re-scaling the weights and rounding them to small integers we could have a test point system that is very close to performing a logistic regression classification. We may be able to use the same assessment questions in a much more decisive manner than assigning all questions the same number of points.</p>
<p>This sort of idea is what one would expect from a mixed and collaborating team that includes medical experts, statistics experts, and programmers. After some work our team might work out that scoring the assessment can be done by the simple <code>R</code> <code>dplyr</code> pipeline:</p>
<div>
<pre><code><span>suppressPackageStartupMessages</span>(<span>library</span>(<span>&quot;dplyr&quot;</span>))

scale &lt;-<span> </span><span>0.237</span>
  
d %>%
<span> </span><span>group_by</span>(subjectID) %>%
<span> </span><span>mutate</span>(<span>probability =</span>
           <span>exp</span>(assessmentTotal *<span> </span>scale)/
<span> </span><span>sum</span>(<span>exp</span>(assessmentTotal *<span> </span>scale)))</code></pre>
</div>
<pre><code>## # A tibble: 4 x 4
## # Groups:   subjectID [2]
##   subjectID      surveyCategory assessmentTotal probability
##       </code></pre></a>