<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/use-quick-formula-functions-in-purrrmap-base-vs-tidtyverse-idiom-comparisonsexamples/"></div></div><div><a data-url="http://www.r-bloggers.com/use-quick-formula-functions-in-purrrmap-base-vs-tidtyverse-idiom-comparisonsexamples/" data-counturl="http://www.r-bloggers.com/use-quick-formula-functions-in-purrrmap-base-vs-tidtyverse-idiom-comparisonsexamples/" data-count="horizontal" data-text="Use quick formula functions in purrr::map (+ base vs tidtyverse idiom comparisons/examples)" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rud.is/b/2016/07/26/use-quick-formula-functions-in-purrrmap-base-vs-tidtyverse-idiom-comparisonsexamples/"> R – rud.is</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I’ve converted the vast majority of my <code>*apply</code> usage over to <a target="_blank" href="https://github.com/hadley/purrr"><code>purrr</code></a> functions. In an attempt to make this a quick post, I’ll refrain from going into all the benefits of the <code>purrr</code> package. Instead, I’ll show just <em>one</em> thing that’s super helpful: formula functions.</p>
<p>After seeing <a target="_blank" href="http://qz.com/741391/its-not-your-imagination-gun-violence-is-over-the-top-this-summer/">this Quartz article</a> using a visualization to compare the frequency and volume of mass shootings, I wanted to grab the data to look at it with a stats-eye (humans are ++gd at visually identifying patterns, but we’re also ++gd as misinterpreting them, plus stats validates visual assumptions). I’m not going into that here, but will use the grabbing of the data to illustrate the formula functions. Note that there’s quite a bit of “setup” here for just one example, so I guess I kinda am attempting to shill the <code>purrr</code> package and the “piping tidyverse” <em>just a tad</em>. </p>
<p>If you head on over to the <a target="_blank" href="https://www.massshootingtracker.org/data">site with the data</a> you’ll see you can download files for all four years. In <em>theory</em>, these are all individual years, but the names of the files gave me pause:</p>
<ul>
<li><code>MST Data 2013 - 2015.csv</code></li>
<li><code>MST Data 2014 - 2015.csv</code></li>
<li><code>MST Data 2015 - 2015.csv</code></li>
<li><code>Mass Shooting Data 2016 - 2016.csv</code></li>
</ul>
<p>So, they <em>may</em> all be individual years, but the naming consistency isn’t there and it’s better to double check than to assume.</p>
<p>First, we can check to see if the column names are the same (we can eyeball this since there are only four files and a small # of columns):</p>
<pre><code>library(purrr)
library(readr)

list.files() %>% 
  map(read_csv) %>% 
  map(colnames)

## [[1]]
## [1] &quot;date&quot;                        &quot;name_semicolon_delimited&quot;   
## [3] &quot;killed&quot;                      &quot;wounded&quot;                    
## [5] &quot;city&quot;                        &quot;state&quot;                      
## [7] &quot;sources_semicolon_delimited&quot;
## 
## [[2]]
## [1] &quot;date&quot;                        &quot;name_semicolon_delimited&quot;   
## [3] &quot;killed&quot;                      &quot;wounded&quot;                    
## [5] &quot;city&quot;                        &quot;state&quot;                      
## [7] &quot;sources_semicolon_delimited&quot;
## 
## [[3]]
## [1] &quot;date&quot;                        &quot;name_semicolon_delimited&quot;   
## [3] &quot;killed&quot;                      &quot;wounded&quot;                    
## [5] &quot;city&quot;                        &quot;state&quot;                      
## [7] &quot;sources_semicolon_delimited&quot;
## 
## [[4]]
## [1] &quot;date&quot;                        &quot;name_semicolon_delimited&quot;   
## [3] &quot;killed&quot;                      &quot;wounded&quot;                    
## [5] &quot;city&quot;                        &quot;state&quot;                      
## [7] &quot;sources_semicolon_delimited&quot;</code></pre>
<p>A quick inspection of the <code>date</code> column shows it’s in <code>month/day/year</code> format and we want to know if each file only spans one year. This is where the elegance of the formula function comes in:</p>
<pre><code>library(lubridate)

list.files() %>% 
  map(read_csv) %>% 
  map(~range(mdy(.$date))) # &lt;--- the *entire* post was to show this one line <img alt="😉" class="wp-smiley" src="https://s.w.org/images/core/emoji/72x72/1f609.png">

## [[1]]
## [1] &quot;2016-01-06&quot; &quot;2016-07-25&quot;
## 
## [[2]]
## [1] &quot;2013-01-01&quot; &quot;2013-12-31&quot;
## 
## [[3]]
## [1] &quot;2014-01-01&quot; &quot;2014-12-29&quot;
## 
## [[4]]
## [1] &quot;2015-01-01&quot; &quot;2015-12-31&quot;</code></pre>
<p>To break that down a bit:</p>
<ul>
<li><code>list.files()</code> returns a vector of filenames in the current directory</li>
<li>the first <code>map()</code> reads each of those files in and creates a list with four elements, each being a <code>tibble</code> (<code>data_frame</code> / <code>data.frame</code>)</li>
<li>the second <code>map()</code> iterates over those data frames and calls a newly created anonymous function which converts the <code>date</code> column to a proper <code>Date</code> data type then gets the range of those dates, ultimately resulting in a four element list, with each element being a two element vector of <code>Date</code>s</li>
</ul>
<p>For you “basers” out there, this is what that looks like old school style:</p>
<pre><code>fils &lt;- list.files()
dfs &lt;- lapply(fils, read.csv, stringsAsFactors=FALSE)
lapply(dfs, function(x) range(as.Date(x$date, format=&quot;%m/%e/%Y&quot;)))</code></pre>
<p>or</p>
<pre><code>lapply(list.files(), function(x) {
  df &lt;- read.csv(x, stringsAsFactors=FALSE)
  range(as.Date(df$date, format=&quot;%m/%e/%Y&quot;))
})</code></pre>
<p>You eliminate the <code>function(x) { }</code> and get pre-defined vars (either <code>.x</code> or <code>.</code> and, if needed, <code>.y</code>) to compose your <code>map</code>s and pipes very cleanly and succinctly, but still being super-readable. </p>
<p>After performing this inspection (i.e. that each file does contain only a incidents for a single year), we can now automate the data ingestion:</p>
<pre><code>library(rvest)
library(purrr)
library(readr)
library(dplyr)
library(lubridate)

read_html(&quot;https://www.massshootingtracker.org/data&quot;) %>% 
  html_nodes(&quot;a[href^='https://docs.goo']&quot;) %>% 
  html_attr(&quot;href&quot;) %>% 
  map_df(read_csv) %>% 
  mutate(date=mdy(date)) -> shootings</code></pre>
<p>Here’s what that looks like w/o the tidyverse/piping:</p>
<pre><code>library(XML)

doc &lt;- htmlParse(&quot;http://www.massshootingtracker.org/data&quot;) # note the necessary downgrade to &quot;http&quot;

dfs &lt;- xpathApply(doc, &quot;//a[contains(@href, 'https://docs.goo')]&quot;, function(x) {
  csv &lt;- xmlGetAttr(x, &quot;href&quot;)
  df &lt;- read.csv(csv, stringsAsFactors=FALSE)
  df$date &lt;- as.Date(df$date, format=&quot;%m/%e/%Y&quot;)
  df
})

shootings &lt;- do.call(rbind, dfs)
</code></pre>
<p>Even hardcore “basers” may have to admit that the piping/tidyverse version is ultimately better.</p>
<p>Give the <code>purrr</code> package a first (or second) look if you haven’t switched over to it. Type safety, readable anonymous functions and C-backed fast functional idioms will mean that your code may ultimately be purrrfect.</p>
<h3>UPDATE</h3>
<p>I received a question in the comments regarding how I came to that CSS selector for the gdocs CSV URLs, so I made a quick video of the exact steps I took. Exposition below the film.</p>
<p><center><iframe height="315" width="560" allowfullscreen src="https://www.youtube.com/embed/OSmfOIg8jM4"></iframe></center></p>
<p>Right-click “Inspect” in Chrome is my go-to method for finding what I’m after. This isn’t the place to dive deep into the dark art of web page spelunking, but in this case, when I saw there were four similar anchor (<code>&lt;a></code>) tags that pointed to the CSV “files”, I took the easy way out and just built a selector based on the <code>href</code> attribute value (or, more specifically, the characters at the start of the <code>href</code> attribute). However, all four ways below end up targeting the same four elements:</p>
<pre><code>pg &lt;- read_html(&quot;https://www.massshootingtracker.org/data&quot;)

html_nodes(pg, &quot;a.btn.btn-default&quot;)
html_nodes(pg, &quot;a[href^='https://docs.goo']&quot;)
html_nodes(pg, xpath=&quot;.//a[@class='btn btn-default']&quot;)
html_nodes(pg, xpath=&quot;.//a[contains(@href, 'https://docs.goo')]&quot;)</code></pre>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/use-quick-formula-functions-in-purrrmap-base-vs-tidtyverse-idiom-comparisonsexamples/"></div></div><div><a data-url="http://www.r-bloggers.com/use-quick-formula-functions-in-purrrmap-base-vs-tidtyverse-idiom-comparisonsexamples/" data-counturl="http://www.r-bloggers.com/use-quick-formula-functions-in-purrrmap-base-vs-tidtyverse-idiom-comparisonsexamples/" data-count="vertical" data-text="Use quick formula functions in purrr::map (+ base vs tidtyverse idiom comparisons/examples)" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://rud.is/b/2016/07/26/use-quick-formula-functions-in-purrrmap-base-vs-tidtyverse-idiom-comparisonsexamples/"> R – rud.is</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/T68Dfg0E3WA"></a></div>