<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/exploring-2017-retail-store-closings-with-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rud.is/b/2017/03/19/exploring-2017-retail-store-closings-with-r/"> R – rud.is</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>A story about one of the retail chains (J.C. Penny) releasing their list of stores closing in 2017 <a target="_blank" rel="nofollow" href="https://www.yahoo.com/news/jc-penney-releases-full-list-103403009.html">crossed paths</a> with my Feedly reading list today and jogged my memory that there were a number of chains closing many of their doors this year, and I wanted to see the impact that might have on various states.</p>
<p>I’m also doing this to add one more example of:</p>
<ul>
<li>scraping (with content caching)</li>
<li>data cleaning</li>
<li>per-capita normalization</li>
<li>comparing salient information to other indicators</li>
</ul>
<p>to the growing list of great examples out there by the extended R community. Plus, I feel compelled to try to keep up with <a target="_blank" rel="nofollow" href="http://twitter.com/ma_salmon">@ma_salmon</a>’s <a target="_blank" rel="nofollow" href="http://www.masalmon.eu/">blogging pace</a>.</p>
<p>Let’s jump right in…</p>
<pre><code>library(httr)
library(rvest)
library(knitr)
library(kableExtra)
library(ggalt)
library(statebins)
library(hrbrthemes)
library(tidyverse)

options(knitr.table.format = &quot;html&quot;)
update_geom_font_defaults(font_rc_light, size = 2.75)</code></pre>
<p>“Closing” lists of four major retailers — K-Mart, Sears, Macy’s and J.C. Penny — abound (HTML formatting a list seems to be the “easy way out” story-wise for many blogs and newspapers). We can dig a bit deeper than just a plain set of lists, but first we need the data.</p>
<p>The Boston Globe has a nice, predictable, mostly-uniform pattern to their list-closing “stories”, so we’ll use that data. Site content can change quickly, so it makes sense to try to cache content whenever possible as we scrape it. To that end, we’ll use <code>httr::GET</code> vs <code>xml2::read_html</code> since <code>GET</code> preserves all of the original request and response information and <code>read_html</code> returns an external pointer that has no current support for serialization without extra work.</p>
<pre><code>closings &lt;- list(
  kmart = &quot;https://www.bostonglobe.com/metro/2017/01/05/the-full-list-kmart-stores-closing-around/4kJ0YVofUWHy5QJXuPBAuM/story.html&quot;,
  sears = &quot;https://www.bostonglobe.com/metro/2017/01/05/the-full-list-sears-stores-closing-around/yHaP6nV2C4gYw7KLhuWuFN/story.html&quot;,
  macys = &quot;https://www.bostonglobe.com/metro/2017/01/05/the-full-list-macy-stores-closing-around/6TY8a3vy7yneKV1nYcwY7K/story.html&quot;,
    jcp = &quot;https://www.bostonglobe.com/business/2017/03/17/the-full-list-penney-stores-closing-around/vhoHjI3k75k2pSuQt2mZpO/story.html&quot;
)

saved_pgs &lt;- &quot;saved_store_urls.rds&quot;

if (file.exists(saved_pgs)) {
  pgs &lt;- read_rds(saved_pgs)
} else {
  pgs &lt;- map(closings, GET)
  write_rds(pgs, saved_pgs)
}</code></pre>
<p>This is what we get from that scraping effort:</p>
<pre><code>map(pgs, content) %>%
  map(html_table) %>%
  walk(~glimpse(.[[1]]))
## Observations: 108
## Variables: 3
## $ X1 &lt;chr> &quot;300 Highway 78 E&quot;, &quot;2003 US Hwy 280 Bypass&quot;, &quot;3600 Wilson ...
## $ X2 &lt;chr> &quot;Jasper&quot;, &quot;Phenix City&quot;, &quot;Bakersfield&quot;, &quot;Coalinga&quot;, &quot;Kingsb...
## $ X3 &lt;chr> &quot;AL&quot;, &quot;AL&quot;, &quot;CA&quot;, &quot;CA&quot;, &quot;CA&quot;, &quot;CA&quot;, &quot;CO&quot;, &quot;CO&quot;, &quot;CT&quot;, &quot;FL&quot;,...
## Observations: 42
## Variables: 4
## $ X1 &lt;chr> &quot;301 Cox Creek Pkwy&quot;, &quot;1901 S Caraway Road&quot;, &quot;90 Elm St; En...
## $ X2 &lt;chr> &quot;Florence&quot;, &quot;Jonesboro&quot;, &quot;Enfield&quot;, &quot;Lake Wales&quot;, &quot;Albany&quot;,...
## $ X3 &lt;chr> &quot;AL&quot;, &quot;AR&quot;, &quot;CT&quot;, &quot;FL&quot;, &quot;GA&quot;, &quot;GA&quot;, &quot;IN&quot;, &quot;KS&quot;, &quot;KY&quot;, &quot;LA&quot;,...
## $ X4 &lt;chr> &quot;Y&quot;, &quot;N&quot;, &quot;N&quot;, &quot;Y&quot;, &quot;Y&quot;, &quot;N&quot;, &quot;N&quot;, &quot;N&quot;, &quot;Y&quot;, &quot;Y&quot;, &quot;Y&quot;, &quot;Y&quot;,...
## Observations: 68
## Variables: 6
## $ X1 &lt;chr> &quot;Mission Valley Apparel&quot;, &quot;Paseo Nuevo&quot;, &quot;*Laurel Plaza&quot;, &quot;...
## $ X2 &lt;chr> &quot;San Diego&quot;, &quot;Santa Barbara&quot;, &quot;North Hollywood&quot;, &quot;Simi Vall...
## $ X3 &lt;chr> &quot;CA&quot;, &quot;CA&quot;, &quot;CA&quot;, &quot;CA&quot;, &quot;FL&quot;, &quot;FL&quot;, &quot;FL&quot;, &quot;FL&quot;, &quot;FL&quot;, &quot;GA&quot;,...
## $ X4 &lt;int> 385000, 141000, 475000, 190000, 101000, 195000, 143000, 140...
## $ X5 &lt;int> 1961, 1990, 1995, 2006, 1995, 2000, 1977, 1974, 2000, 1981,...
## $ X6 &lt;int> 140, 77, 105, 105, 68, 83, 86, 73, 72, 69, 9, 57, 54, 87, 5...
## Observations: 138
## Variables: 3
## $ Mall/Shopping Center &lt;chr> &quot;Auburn Mall&quot;, &quot;Tannehill Promenade&quot;, &quot;Ga...
## $ City                 &lt;chr> &quot;Auburn&quot;, &quot;Bessemer&quot;, &quot;Gadsden&quot;, &quot;Jasper&quot;...
## $ State                &lt;chr> &quot;AL&quot;, &quot;AL&quot;, &quot;AL&quot;, &quot;AL&quot;, &quot;AR&quot;, &quot;AR&quot;, &quot;AZ&quot;,...</code></pre>
<p>We now need to normalize the content of the lists.</p>
<pre><code>map(pgs, content) %>%
  map(html_table) %>%
  map(~.[[1]]) %>%
  map_df(select, abb=3, .id = &quot;store&quot;) -> closings</code></pre>
<p>We’re ultimately just looking for city/state for this simple exercise, but one could do more precise geolocation (perhaps with <a target="_blank" rel="nofollow" href="https://github.com/hrbrmstr/rgeocodio"><code>rgeocodio</code></a>) and combine that with local population data, job loss estimates, current unemployment levels, etc. to make a <em>real</em> story out of the closings vs just do the easy thing and publish a list of stores.</p>
<pre><code>count(closings, abb) %>%
  left_join(data_frame(name = state.name, abb = state.abb)) %>%
  left_join(usmap::statepop, by = c(&quot;abb&quot;=&quot;abbr&quot;)) %>%
  mutate(per_capita = (n/pop_2015) * 1000000) %>%
  select(name, n, per_capita) -> closings_by_state</code></pre>
<p>(NOTE: you can get the code for the entire Rmd via <a target="_blank" rel="nofollow" href="http://rpubs.com/hrbrmstr/stores_2017">RPubs</a> or <a target="_blank" rel="nofollow" href="https://gist.github.com/hrbrmstr/880f609aa084b56c7f0326a6ee1c4c59">GitHub</a>)</p>
<p><iframe height="1250" width="100%" src="https://rud.is/b/iframes/jcp/just_tabs.html"></iframe></p>
<h3>Compared to unemployment/underutilization</h3>
<p>I’d have used the <a target="_blank" rel="nofollow" href="https://cran.rstudio.com/web/packages/epidata/"><code>epidata</code></a> package to get the current state unemployment data but it’s not <em>quite</em> current enough, so we can either use a package to get data from the Bureau of Labor Statistics or just scrape it. A quick coin-flip says we’ll scrape the data. </p>
<p>We’ll use the U-6 rate since that is an expanded definition of “underutilization” including <em>“total unemployed, plus all marginally attached workers, plus total employed part time for economic reasons, as a percent of the civilian labor force plus all marginally attached workers”</em> and is likely to more representative for the populations working at retail chains. I could be wrong. I only play an economist on <img alt="📺" class="wp-smiley" src="https://s.w.org/images/core/emoji/2/72x72/1f4fa.png">. If you are an economist, please drop a note telling me where I errd in my thinking <img alt="😎" class="wp-smiley" src="https://s.w.org/images/core/emoji/2/72x72/1f60e.png"></p>
<pre><code>pg &lt;- read_html(&quot;https://www.bls.gov/lau/stalt16q4.htm&quot;)

html_nodes(pg, &quot;table#alternmeas16\\:IV&quot;) %>% 
  html_table(header = TRUE, fill = TRUE) %>%
  .[[1]] %>% 
  docxtractr::assign_colnames(1) %>% 
  rename(name=State) %>% 
  as_data_frame() %>% 
  slice(2:52) %>% 
  type_convert() %>% 
  left_join(closings_by_state, by=&quot;name&quot;) %>% 
  filter(!is.na(n)) -> with_unemp

ggplot(with_unemp, aes(per_capita, `U-6`)) +
  geom_label(aes(label=name), fill=&quot;#8c96c6&quot;, color=&quot;white&quot;, size=3.5, family=font_rc) +
  scale_x_continuous(limits=c(-0.125, 6.75)) +
  labs(x=&quot;Closings per-capita (1MM)&quot;, 
       y=&quot;BLS Labor Underutilization (U-6 rate)&quot;,
       title=&quot;Per-capita store closings compared to current BLS U-6 Rate&quot;) +
  theme_ipsum_rc(grid=&quot;XY&quot;)</code></pre>
<p><img alt="" class="wp-image-5186" src="https://i0.wp.com/rud.is/b/wp-content/uploads/2017/03/bls-1.png?resize=780%2C683&ssl=1" data-recalc-dims="1"></p>
<p>If I were a reporter (again, I only play one on <img alt="📺" class="wp-smiley" src="https://s.w.org/images/core/emoji/2/72x72/1f4fa.png">), I think I’d be digging a bit deeper on the impact of these (and the half-dozen or so other) retailers closing locations in New Mexico, Nevada, West Virginia, Wyoming, (mebbe Maine, though I’m super-b ased :-), North Dakota &amp; South Dakota. I also hope <a target="_blank" rel="nofollow" href="http://twitter.com/Marketplace">@Marketplace</a> does a few more stories on the changing retail landscape in the U.S. over the coming months to see if there are any overt consequences to the loss of jobs and anchor stores.</p>
<p>If you end up tinkering with the data, drop a note in the comments if something you discover piques your interest. For those interested in potentially marrying the data up with some additional cartography, there should be enough precision the store lists to get distinct enough lat/lon paris after geocoding (I did a quick test with <code>rgeocodio</code>) to make some interesting map views, especially if you can find more store closing lists.</p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://rud.is/b/2017/03/19/exploring-2017-retail-store-closings-with-r/"> R – rud.is</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/-Eq9JvUXcws"></a></div>