<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/gg_tweeting-power-outages/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rud.is/b/2017/10/30/gg_tweeting-power-outages/"> R – rud.is</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>As many folks know, I live in semi-rural Maine and we were hit pretty hard with a wind+rain storm Sunday to Monday. The hrbrmstr compound had no power (besides a generator) and no stable/high-bandwidth internet (Verizon LTE was heavily congested) since 0500 Monday and still does not as I write this post.</p>
<p>I’ve played with scraping power outage data from Central Maine Power but there’s a great Twitter account — <a rel="nofollow" href="https://twitter.com/PowerOutage_us" target="_blank">PowerOutage_us</a> — that has done much of the legwork for the entire country. They don’t cover everything and do not provide easily accessible historical data (likely b/c evil folks wld steal it w/o payment or credit) but they do have a site you can poke at and do provide updates via Twitter. As you’ve seen in a previous post, we can use the <code>rtweet</code> package to easily read Twitter data. And, the power outage tweets are regular <em>enough</em> to identify and parse. But raw data is so…<em>raw</em>. </p>
<p>While one <em>could</em> graph data just for one’s self, I decided to marry this power scraping capability with a recent idea I’ve been toying with adding to <code>hrbrthemes</code> or <code>ggalt</code>: <code>gg_tweet()</code>. Imagine being able to take a ggplot2 object and “plot” it to Twitter, fully conforming to Twitter’s stream or card image sizes. By conforming to these size constraints, they don’t get cropped in the timeline view (if you allow images to be previewed in-timeline). This is even more powerful if you have some helper functions for proper theme-ing (font sizes especially need to be tweaked). Enter <code>gg_tweet()</code>.</p>
<h2>Power Scraping</h2>
<p>We’ll cover scraping <a rel="nofollow" href="http://twitter.com/PowerOutage_us" target="_blank">@PowerOutage_us</a> first, but we’ll start with all the packages we’ll need and a helper function to convert power outage estimates to numeric values:</p>
<pre><code>library(httr)
library(magick)
library(rtweet)
library(stringi)
library(hrbrthemes)
library(tidyverse)

words_to_num &lt;- function(x) {
  map_dbl(x, ~{
    val &lt;- stri_match_first_regex(.x, &quot;^([[:print:]]+) #PowerOutages&quot;)[,2]
    mul &lt;- case_when(
      stri_detect_regex(val, &quot;[Kk]&quot;) ~ 1000,
      stri_detect_regex(val, &quot;[Mm]&quot;) ~ 1000000,
      TRUE ~ 1
    ) 
    val &lt;- stri_replace_all_regex(val, &quot;[^[:digit:]\\.]&quot;, &quot;&quot;)
    as.numeric(val) * mul 
  })
}</code></pre>
<p>Now, I can’t cover setting up <code>rtweet</code> OAuth here. The vignette and package web site do that well.</p>
<p>The bot tweets infrequently enough that this is really all we need (though, bump up <code>n</code> as you need to):</p>
<pre><code>outage &lt;- get_timeline(&quot;PowerOutage_us&quot;, n=300)</code></pre>
<p>Yep, that gets the last 300 tweets from said account. It’s amazingly simple. </p>
<p>Now, the outage tweets for the east coast / northeast are not individually uniform but collectively they are (there’s a pattern that may change but you can tweak this if they do):</p>
<pre><code>filter(outage, stri_detect_regex(text, &quot;\\#(EastCoast|NorthEast)&quot;)) %>% 
  mutate(created_at = lubridate::with_tz(created_at, 'America/New_York')) %>% 
  mutate(number_out = words_to_num(text)) %>%  
  ggplot(aes(created_at, number_out)) +
  geom_segment(aes(xend=created_at, yend=0), size=5) +
  scale_x_datetime(date_labels = &quot;%Y-%m-%d\n%H:%M&quot;, date_breaks=&quot;2 hours&quot;) +
  scale_y_comma(limits=c(0,2000000)) +
  labs(
    x=NULL, y=&quot;# Customers Without Power&quot;,
    title=&quot;Northeast Power Outages&quot;,
    subtitle=&quot;Yay! Twitter as a non-blather data source&quot;,
    caption=&quot;Data via: @PowerOutage_us </code></pre>