<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/dissecting-r-package-utility-belts/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rud.is/b/2018/04/08/dissecting-r-package-utility-belts/"> R – rud.is</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Many R package authors (including myself) lump a collection of small, useful functions into some type of <code>utils.R</code> file and usually do not export the functions since they are (generally) designed to work on package internals rather than expose their functionality via the exported package API. Just like Batman’s utility belt, which can be customized for any mission, any set of utilities in a given R package will also likely be different from those in other packages.</p>
<p>I thought it would be neat to take a look at:</p>
<ul>
<li>just how many packages have one or more <code>util*.R</code> files and what the most common file names are for them;</li>
<li>utility function naming preferences — specifically snake-case, camel-case or dot-case</li>
<li>what the most common “utility” functions names are across the packages</li>
<li>coding style — specifically compare ratios of white space, full-line comments to code size</li>
</ul>
<p>for all the published packages on CRAN.</p>
<p>There are <em>many</em> more questions one can ask and then use this corpus to answer, so we’ll close out the post with a link to it so any intrepid readers can do just that, especially since reproducing the first bit of this post would require a local CRAN mirror (which most folks — rightly so — do not have handy).</p>
<h3>Acquiring and Transforming the Data We Need</h3>
<p><img data-orig-file="https://i1.wp.com/rud.is/b/wp-content/uploads/2018/04/cran.png?fit=730%2C578&amp;ssl=1" data-orig-size="730,578" data-medium-file="https://i1.wp.com/rud.is/b/wp-content/uploads/2018/04/cran.png?fit=300%2C238&amp;ssl=1" data-attachment-id="9499" data-image-meta="{" data-image-description="" src="https://i1.wp.com/rud.is/b/wp-content/uploads/2018/04/cran.png?resize=243%2C192&ssl=1" data-permalink="https://rud.is/b/2018/04/08/dissecting-r-package-utility-belts/cran/" data-large-file="https://i1.wp.com/rud.is/b/wp-content/uploads/2018/04/cran.png?fit=530%2C420&amp;ssl=1" alt="" data-image-title="cran" width="243" data-comments-opened="1" class="wp-image-9499" data-recalc-dims="1" height="192">Since I have local CRAN mirror, it’s just a matter of iterating through all the package <code>tar.gz</code> files in <code>src/contrib</code> and <code>grep</code>ping through a <code>tar</code> listing of each for a pattern like <code>&quot;R/util.*$</code>. That pattern isn’t perfect but it’s quick and we’ll be able to filter out any files it catches that don’t belong. I chose to use a small <code>bash</code> script for this but it’s possible to do this with R as well (an exercise left to the reader). The resultant data file looks a bit like the output from an <code>ls -l</code> (linux-ish) directory listing:</p>
<pre><code>-rw-r--r--  0 hornik users    1658 Jun  5  2016 AHR/R/util.R
-rw-r--r--  0 ligges users   12609 Dec 13  2016 ALA4R/R/utilities_internal.R
-rw-r--r--  0 hornik users       0 Feb 24  2017 AWR.Kinesis/R/utils.R
-rw-r--r--  0 ligges users    4127 Aug 30  2017 AlphaVantageClient/R/utils.R
-rw-r--r--  0 ligges users     121 Jan 19  2017 AmyloGram/R/utils.R
-rw-r--r--  0 ligges users    2873 Jan 17 23:04 DT/R/utils.R
-rw-r--r--  0 ligges users    3055 Jan 17  2017 cleanr/inst/source/R/utils.R
drwxr-xr-x  0 ligges users       0 Sep 24  2017 JGR/java/org/rosuda/JGR/util/
</code></pre>
<p>I made sure to show a few examples of where a better search pattern would have helped ensure lines like the three at the bottom of that listings aren’t included. But, we all often have to deal with imperfect data, so we’ll make sure to deal with that during the ingestion &amp; cleanup process.</p>
<pre><code>library(stringi)
library(hrbrthemes)
library(archive) # devtools::install_github(&quot;jimhester&quot;, &quot;archive&quot;)
library(tidyverse)

# I ran readr::type_convert() once and it returns this column type spec. By using it 
# for subsequent conversions, we'll gain reproducibility and data format change 
# detection capabilities &quot;for free&quot;

cols(
  permsissions = col_character(),
  links = col_integer(),
  owner = col_character(),
  group = col_character(),
  size = col_integer(),
  month = col_character(),
  day = col_integer(),
  year_hr = col_character(),
  path = col_character()
) -> tar_cols

# Now, we parse the tar verbose ('ls -l') listing

stri_read_lines(&quot;~/Data/pkutils.txt&quot;) %>% # stringi was loaded so might as well use it
  stri_split_regex(&quot; +&quot;, 9, simplify = TRUE) %>% # split input into 9 columns
  as_data_frame() %>% # ^^ returns a matrix but data frames are more useful for our work
  set_names(names(tar_cols$cols)) %>% # column names are useful and we can use our colspec for it
  type_convert(col_types = tar_cols) %>% # see comment block before cols()
  mutate(day = sprintf(&quot;%02d&quot;, day)) %>% # now we'll work on getting the date pieces to be a Date
  mutate(year_hr = case_when( # the year_hr field can be either %Y or %H:%M depending on file 'recency'
    stri_detect_fixed(year_hr, &quot;:&quot;) &amp;
      (month %in% c(&quot;Jan&quot;, &quot;Feb&quot;, &quot;Mar&quot;, &quot;Apr&quot;)) ~ &quot;2018&quot;, # if %H:%M but 'starter' months it's 2018
    stri_detect_fixed(year_hr, &quot;:&quot;) &amp;
      (month %in% c(&quot;Dec&quot;, &quot;Nov&quot;, &quot;Oct&quot;, &quot;Sep&quot;, &quot;Aug&quot;, &quot;Jul&quot;, &quot;Jun&quot;)) ~ &quot;2017&quot;, # %H:%M &amp; 'end' months
    TRUE ~ year_hr # already in %Y format
  )) %>%
  mutate(date= lubridate::mdy(sprintf(&quot;%s %s, %s&quot;, month, day, year_hr))) %>% # get a Date
  mutate(pkg = stri_match_first_regex(path, &quot;^(.*)/R/&quot;)[,2]) %>% # extract package name (stri_extract is also usable here)
  mutate(fil = basename(path)) %>% # extrafct just the file name
  filter(!is.na(pkg)) %>% # handle one type of wrongly included file
  filter(!stri_detect_fixed(pkg, &quot;/&quot;)) %>% # ande another
  filter(!is.na(path)) -> xdf # and another; but we're done so we close with an assignment

glimpse(xdf)
## Observations: 1,746
## Variables: 12
## $ permsissions </code></pre>