<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/visualizing-possible-cities-for-amazons-new-headquarters-using-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://sharpsightlabs.com/blog/new-amazon-hq/"> r-bloggers – SHARP SIGHT LABS</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a rel="nofollow" href="http://sharpsightlabs.com/join-newsletter-rapid-mastery/" target="_blank"><br>
<img sizes="(max-width: 600px) 100vw, 600px" src="https://i2.wp.com/sharpsightlabs.com/wp-content/uploads/2017/09/amazon_HQ_cities_NOTITLE-600x325.png?w=450" alt="" class="wp-image-3119" data-recalc-dims="1"><br>
</a></p>
<p>Last week, Amazon announced that it has started to search for a new city in which to build a second headquarters.</p>
<p>Among several selection criteria, they indicated that they’re looking for a city with more than 1 million people, and a city with a good pool of tech talent.</p>
<p>While reading about Amazon’s new HQ search on a news website, I encountered a <a href="https://www.cbsnews.com/news/amazons-hq2-cities-second-headquarters-these-cities-are-contenders/">dataset of cities</a> that might qualify: cities with over 1 million people in the metro area, and the corresponding percent of people with college degrees in each city. </p>
<p>The news website already <a href="https://www.cbsnews.com/news/amazons-hq2-cities-second-headquarters-these-cities-are-contenders/">visualized</a> it, but I want to show you how to do this in . It’s an excellent little exercise to show you again how to scrape/wrangle/visualize data.</p>
<p>With that in mind, we’re going to scrape the data, <a href="http://sharpsightlabs.com/blog/dplyr-intro-data-manipulation-with-r/">wrangle it into shape</a> (using  and a few other tools), and then visualize it as a map using .</p>
<p>I’ll preface this by saying that this is an <em>imperfect analysis</em>. We don’t know the full and final selection criteria, and even if we did, a full analysis would be far beyond the scope of a simple blog post. </p>
<p>Having said that, this is a good “first pass” at such an analysis: the quick-and-dirty version.</p>
<p>Furthermore, if you’re getting involved in data science, this will give you some hints about how to use  as an analytical tool. If you’re in marketing, sales, or operations, you could very easily use this quick-and-dirty analysis as a template and starting point for some of your own work.</p>
<p>Ok. Let’s jump in.</p>
<p>First, we’ll just load the packages that we will use.</p>
<pre class="wp-code-highlight">
#==============
# LOAD PACKAGES
#==============

library(rvest)
library(tidyverse)
library(stringr)
library(ggmap)

</pre>
<p>Next, we’ll use several functions from the  package to scrape the data and read it into a dataframe.</p>
<pre class="wp-code-highlight">
#=======
# SCRAPE
#=======
html.amz_cities &lt;- read_html(&quot;https://www.cbsnews.com/news/amazons-hq2-cities-second-headquarters-these-cities-are-contenders/&quot;)


df.amz_cities &lt;- html.amz_cities %>%
  html_nodes(&quot;table&quot;) %>%
  .[[1]] %>% 
  html_table()


# inspect
df.amz_cities %>% head()

</pre>
<p>Next, we’ll change the column names. </p>
<p>When we scraped the data, the column names were not read in properly from the HTML table, so we need to add them manually.</p>
<pre class="wp-code-highlight">
#====================
# CHANGE COLUMN NAMES
#====================

# inspect initial column names
colnames(df.amz_cities)

# assign new column names
colnames(df.amz_cities) &lt;- c(&quot;metro_area&quot;, 'state', 'population_tot', 'bachelors_degree_pct')


# inspect
df.amz_cities %>% head()

</pre>
<p>As it turns out, when we scraped the data, the original column names (the column names that appeared on the website) ended up in the first row of our newly created dataframe.</p>
<p>This is inappropriate, so we need to remove the first row of data.</p>
<pre class="wp-code-highlight">
#==============================================
# REMOVE FIRST ROW
# - when we scraped the data, the column names
#   on the table were read in as the first row
#   of data.
# - Therefore, we need to remove the first row
#==============================================

df.amz_cities &lt;- df.amz_cities %>% filter(row_number() != 1)

</pre>
<p>Now we’re going to modify the data type of two variables. </p>
<p> and  need to be numeric variables, but when we scraped them, they were read in as character variables. This being the case, we’re going to use some techniques to parse/coerce them into numerics.</p>
<pre class="wp-code-highlight">
#===================================================================================
# MODIFY VARIABLES
# - both bachelors_degree_pct and population_tot were scraped as character variables
#    but we need them in numeric format
# - we will use techniques to parse/coerce these variable from char to numeric
#===================================================================================

#--------------------------------
# PARSE AS NUMBER: population_tot
#--------------------------------

df.amz_cities &lt;- mutate(df.amz_cities, population_tot = parse_number(population_tot))


# check
typeof(df.amz_cities$population_tot)

# inspect
df.amz_cities %>% head()


#-----------------------------
# COERCE: bachelors_degree_pct
#-----------------------------

df.amz_cities &lt;- mutate(df.amz_cities, bachelors_degree_pct = as.numeric(bachelors_degree_pct))

</pre>
<p>Next, we’re going to create a variable that contains the city name. </p>
<p>When we read in the data, there was a variable for ‘.’ This variable contains values like “New York-Newark-Jersey City.” The  variable might be useful for some things, but when we geocode our data (to get the lat-long coordinates), these metro names may cause errors. They are too broad. We need a variable that contains specific city names; for geocoding, it will be better if we have a strict city name.</p>
<p>This being the case, we will create a new  variable by extracting the city names from the metro names. To do this, will will use  along with an appropriate regular expression that can pull out the city names that we want.</p>
<pre class="wp-code-highlight">
#=============================================================
# CREATE VARIABLE: city
# - here, we're using the stringr function str_extract() to
#   extract the primary city name from the metro_area variable
# - to do this, we're using a regex to pull out the city name
#   prior to the first '-' character
#=============================================================

df.amz_cities &lt;- df.amz_cities %>% mutate(city = str_extract(metro_area, &quot;^[^-]*&quot;))


</pre>
<p>Now that we have proper city names, we will geocode our data. We will use the  function to retrieve the latitude and longitude for each city. Then we will merge the geo-data back onto the dataframe using .</p>
<pre class="wp-code-highlight">
#=========================================
# GEOCODE
# - here, we're getting the lat/long data
#=========================================

data.geo &lt;- geocode(df.amz_cities$city)

#inspect

data.geo %>% head()
data.geo

#========================================
# RECOMBINE: merge geo data to data frame
#========================================

df.amz_cities &lt;- cbind(df.amz_cities, data.geo)
df.amz_cities


</pre>
<p>Quickly, we’ll use the  function to rename the ‘‘ variable to ‘.’</p>
<pre class="wp-code-highlight">
#==============================================================
# RENAME VARIABLE: lon -> long
# - we'll rename lon to lon, just because 'long' is consistent
#   with the name for longitude in other data sources
#   that we will use
#==============================================================

df.amz_cities &lt;- rename(df.amz_cities, long = lon)


# get column names names
df.amz_cities %>% names()


</pre>
<p>Next, to make our data a little easier to read, we will re-order the variables. We’ll organize it so that the , , and  are first, followed by the geo-data, and then with the analysis variables at the end (population and ‘college degree percent’). </p>
<pre class="wp-code-highlight">
#==========================================
# REORDER COLUMN NAMES
# - here, we're just doing it manually ...
#==========================================

df.amz_cities &lt;- select(df.amz_cities, city, state, metro_area, long, lat, population_tot, bachelors_degree_pct)


# inspect

df.amz_cities %>% head()

</pre>
<p>Now, we’re going to get a map. In order to visualize the data as a map, we need a map of the United States.</p>
<p>To get this, we will <a href="http://sharpsightlabs.com/blog/simple-practice-maps-tidyverse/">use </a>.</p>
<pre class="wp-code-highlight">
#================================================
# GET USA MAP
# - this is the map of the USA states, upon which
#   we will plot our city data points
#================================================

map.states &lt;- map_data(&quot;state&quot;)

</pre>
<p>Finally, we’re ready to plot.</p>
<p>We’ll initially do a “first iteration” to check that everything looks good.</p>
<pre class="wp-code-highlight">
#====================================
# PLOT
# - here, we're actually creating the 
#   data visualizations with ggplot()
#====================================


#------------------------------------------------
# FIRST ITERATION
# - this is just a 'first pass' to check that
#   everything looks good before we take the time
#   to format it
#------------------------------------------------
ggplot() +
  geom_polygon(data = map.states, aes(x = long, y = lat, group = group)) +
  geom_point(data = df.amz_cities, aes(x = long, y = lat, size = population_tot, color = bachelors_degree_pct))
 
</pre>
<p><a rel="nofollow" href="http://sharpsightlabs.com/join-newsletter-rapid-mastery/" target="_blank"><br>
<img sizes="(max-width: 600px) 100vw, 600px" src="https://i2.wp.com/sharpsightlabs.com/wp-content/uploads/2017/09/amazon_HQ_cities_NoFormat-600x275.png?w=450" alt="" class="wp-image-3120" data-recalc-dims="1"><br>
</a></p>
<p>At a high level, everything looks OK. The points are in the right locations, and at a glance, everything looks good.</p>
<p>Keep in mind, that compared to the finalized version below, the ‘first iteration’ is much much simpler to build. This is a great example of the 80/20 rule in data analysis: in this visualization, you can get 80% of the way with only 20% of the total  code. </p>
<p>Now that we have an initial version, we’ll polish it by adding titles, <a href="http://sharpsightlabs.com/blog/format-titles-and-axes-in-ggplot2/">formatting theme elements</a>, and by adjusting the legends.</p>
<pre class="wp-code-highlight">
#--------------------------------------------------
# FINALIZED VERSION (FORMATTED)
# - this is the 'finalized' version with all of the
#   detailed formatting
#--------------------------------------------------

ggplot() +
  geom_polygon(data = map.states, aes(x = long, y = lat, group = group)) +
  geom_point(data = df.amz_cities, aes(x = long, y = lat, size = population_tot, color = bachelors_degree_pct*.01), alpha = .5) +
  geom_point(data = df.amz_cities, aes(x = long, y = lat, size = population_tot, color = bachelors_degree_pct*.01), shape = 1) +
  coord_map(projection = &quot;albers&quot;, lat0 = 30, lat1 = 40, xlim = c(-121,-73), ylim = c(25,51)) +
  scale_color_gradient2(low = &quot;red&quot;, mid = &quot;yellow&quot;, high = &quot;green&quot;, midpoint = .41, labels = scales::percent_format()) +
  scale_size_continuous(range = c(.9, 11),  breaks = c(2000000, 10000000, 20000000),labels = scales::comma_format()) +
  guides(color = guide_legend(reverse = T, override.aes = list(alpha = 1, size = 4) )) +
  labs(color = &quot;Bachelor's Degree\nPercent&quot;
       ,size = &quot;Total Population\n(metro area)&quot;
       ,title = &quot;Possible cities for new Amazon Headquarters&quot;
       ,subtitle = &quot;Based on population &amp; percent of people with college degrees&quot;) +
  theme(text = element_text(colour = &quot;#444444&quot;, family = &quot;Gill Sans&quot;)
        ,panel.background = element_blank()
        ,axis.title = element_blank()
        ,axis.ticks = element_blank()
        ,axis.text = element_blank()
        ,plot.title = element_text(size = 28)
        ,plot.subtitle = element_text(size = 12)
        ,legend.key = element_rect(fill = &quot;white&quot;)
        ) 


</pre>
<p><a rel="nofollow" href="http://sharpsightlabs.com/join-newsletter-rapid-mastery/" target="_blank"><br>
<img sizes="(max-width: 600px) 100vw, 600px" src="https://i1.wp.com/sharpsightlabs.com/wp-content/uploads/2017/09/amazon_HQ_cities-600x391.png?w=450" alt="" class="wp-image-3118" data-recalc-dims="1"><br>
</a></p>
<h3>A quick note: this is not supposed to be a comprehensive analysis</h3>
<p>I want to point out that this is not intended to be comprehensive or conclusive in any way. Without detailed selection criteria, it will be difficult to come to any solid conclusions.</p>
<p>Rather, this is intended to give you a hint of what’s possible using R tools. If you were so inclined, you could certainly extend this into a much more comprehensive analysis by gathering more data and producing more charts. </p>
<h3>Creating great visualizations gets easier once you master your toolkit</h3>
<p>As you progress as a data scientist, you will get better at creating visualizations like this. </p>
<p><a href="http://sharpsightlabs.com/blog/one-thing-master-data-science/">If you practice</a> and <a href="http://sharpsightlabs.com/blog/master-r-obsolete/">master</a> individual  functions, you will be able to create visualizations like this quickly.</p>
<h1>Sign up now, and discover how to rapidly master data science</h1>
<p>To rapidly master data science, you need to master the essential tools.</p>
<p>You need to know what tools are important, which tools are <em>not</em> important, and <em>how to practice</em>.</p>
<p>Sharp Sight is dedicated to teaching you how to master the tools of data science as quickly as possible.</p>
<p><a rel="nofollow" href="http://sharpsightlabs.com/join-newsletter-rapid-mastery/" target="_blank">Sign up now for our email list</a>, and you’ll receive regular tutorials and lessons. </p>
<p>You’ll learn:</p>
<ul>
<li>What data science tools you should learn (and what not to learn)</li>
<li>How to practice those tools</li>
<li>How to put those tools together to execute analyses and machine learning projects</li>
<li>… and more </li>
</ul>
<p>If you <a rel="nofollow" href="http://sharpsightlabs.com/join-newsletter-rapid-mastery/" target="_blank">sign up for our email list <em>right now</em></a>, you’ll also get access to our “Data Science Crash Course” <u>for free</u>.</p>
<p><a rel="nofollow" href="http://sharpsightlabs.com/join-newsletter-rapid-mastery/" target="_blank">
<p><strong>SIGN UP NOW</strong></p>
<p></p></a>
<p>The post <a rel="nofollow" href="http://sharpsightlabs.com/blog/new-amazon-hq/" target="_blank">Visualizing possible cities for Amazon’s new headquarters (using R)</a> appeared first on <a rel="nofollow" href="http://sharpsightlabs.com/" target="_blank">SHARP SIGHT LABS</a>.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://sharpsightlabs.com/blog/new-amazon-hq/"> r-bloggers – SHARP SIGHT LABS</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/RnrP-ZvtCE4" height="1"></a></div>