As many folks know, I live in semi-rural Maine and we were hit pretty hard with a wind+rain storm Sunday to Monday. The hrbrmstr compound had no power (besides a generator) and no stable\/high-bandwidth internet (Verizon LTE was heavily congested) since 0500 Monday and still does not as I write this post.<\/p>\n
I’ve played with scraping power outage data from Central Maine Power but there’s a great Twitter account \u2014\u00a0PowerOutage_us<\/a> \u2014 that has done much of the legwork for the entire country. They don’t cover everything and do not provide easily accessible historical data (likely b\/c evil folks wld steal it w\/o payment or credit) but they do have a site you can poke at and do provide updates via Twitter. As you’ve seen in a previous post, we can use the While one could<\/em> graph data just for one’s self, I decided to marry this power scraping capability with a recent idea I’ve been toying with adding to We’ll cover scraping @PowerOutage_us first, but we’ll start with all the packages we’ll need and a helper function to convert power outage estimates to numeric values:<\/p>\n Now, I can’t cover setting up The bot tweets infrequently enough that this is really all we need (though, bump up Yep, that gets the last 300 tweets from said account. It’s amazingly simple.<\/p>\n Now, the outage tweets for the east coast \/ northeast are not individually uniform but collectively they are (there’s a pattern that may change but you can tweak this if they do):<\/p>\n That pipe chain looks for key hashtags (for my area), rejiggers the time zone, and calls the helper function to, say, convert You can plot that on your own (print @opencpu made it possible shunt plot output to a Now, we just need a way to take a ggplot2 object and shunt it off to twitter. The following So you pass in a ggplot2 object (likely adding the tweet theme to it) and a Twitter status text (there’s a TODO to check the length for 140c compliance) plus choose a style (stream or card, defaulting to stream) and decide on whether you’re cool with the “retina” default.<\/p>\n Unless you tell<\/em> it to send the tweet it won’t, giving you a chance to preview the image before sending, just in case you want to tweak it a bit before committing it to the Twitterverse. It als returns the We can combine the power outage scraper & plotter with the tweeting code and just do:<\/p>\n That was, in-fact, the last power outage tweet I sent.<\/p>\n Ironically, given current levels of U.S. news and public “discourse” on Twitter and some inane machinations in my own area of domain expertise (cyber), Kick the tyres and watch for As many folks know, I live in semi-rural Maine and we were hit pretty hard with a wind+rain storm Sunday to Monday. The hrbrmstr compound had no power (besides a generator) and no stable\/high-bandwidth internet (Verizon LTE was heavily congested) since 0500 Monday and still does not as I write this post. I’ve played with […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[753,91,655],"tags":[810],"class_list":["post-6917","post","type-post","status-publish","format-standard","hentry","category-ggplot","category-r","category-twitter-2","tag-post"],"yoast_head":"\nrtweet<\/code> package to easily read Twitter data. And, the power outage tweets are regular enough<\/em> to identify and parse. But raw data is so…raw<\/em>.<\/p>\nhrbrthemes<\/code> or ggalt<\/code>: gg_tweet()<\/code>. Imagine being able to take a ggplot2 object and “plot” it to Twitter, fully conforming to Twitter’s stream or card image sizes. By conforming to these size constraints, they don’t get cropped in the timeline view (if you allow images to be previewed in-timeline). This is even more powerful if you have some helper functions for proper theme-ing (font sizes especially need to be tweaked). Enter gg_tweet()<\/code>.<\/p>\nPower Scraping<\/h2>\n
library(httr)\r\nlibrary(magick)\r\nlibrary(rtweet)\r\nlibrary(stringi)\r\nlibrary(hrbrthemes)\r\nlibrary(tidyverse)\r\n\r\nwords_to_num <- function(x) {\r\n map_dbl(x, ~{\r\n val <- stri_match_first_regex(.x, "^([[:print:]]+) #PowerOutages")[,2]\r\n mul <- case_when(\r\n stri_detect_regex(val, "[Kk]") ~ 1000,\r\n stri_detect_regex(val, "[Mm]") ~ 1000000,\r\n TRUE ~ 1\r\n ) \r\n val <- stri_replace_all_regex(val, "[^[:digit:]\\\\.]", "")\r\n as.numeric(val) * mul \r\n })\r\n}<\/code><\/pre>\nrtweet<\/code> OAuth here. The vignette and package web site do that well.<\/p>\nn<\/code> as you need to):<\/p>\noutage <- get_timeline("PowerOutage_us", n=300)<\/code><\/pre>\nfilter(outage, stri_detect_regex(text, "\\\\#(EastCoast|NorthEast)")) %>% \r\n mutate(created_at = lubridate::with_tz(created_at, 'America\/New_York')) %>% \r\n mutate(number_out = words_to_num(text)) %>% \r\n ggplot(aes(created_at, number_out)) +\r\n geom_segment(aes(xend=created_at, yend=0), size=5) +\r\n scale_x_datetime(date_labels = "%Y-%m-%d\\n%H:%M", date_breaks="2 hours") +\r\n scale_y_comma(limits=c(0,2000000)) +\r\n labs(\r\n x=NULL, y="# Customers Without Power",\r\n title="Northeast Power Outages",\r\n subtitle="Yay! Twitter as a non-blather data source",\r\n caption="Data via: @PowerOutage_us <https:\/\/twitter.com\/PowerOutage_us>"\r\n ) -> gg<\/code><\/pre>\n1.2+ Million<\/code> to 1200000<\/code>. Finally it builds a mostly complete ggplot2 object (you should make the max Y limit more dynamic).<\/p>\ngg<\/code>). We’re here to tweet, so let’s go into the next section.<\/p>\nMagick Tweeting<\/h2>\n
magick<\/code> device. This means we have really precise control over ggplot2 output size as well as the ability to add other graphical components to a ggplot2 plot using magick<\/code> idioms. One thing we need to take into account is “retina” plots. They are — essentially — double resolution plots (72 => 144 pixels per inch). For the best looking plots we need to go retina, but that also means kicking up base plot theme font sizes a bit. Let’s build on hrbrthemes::theme_ipsum_rc()<\/code> a bit and make a theme_tweet_rc()<\/code>:<\/p>\ntheme_tweet_rc <- function(grid = "XY", style = c("stream", "card"), retina=TRUE) {\r\n \r\n style <- match.arg(tolower(style), c("stream", "card"))\r\n \r\n switch(\r\n style, \r\n stream = c(24, 18, 16, 14, 12),\r\n card = c(22, 16, 14, 12, 10)\r\n ) -> font_sizes\r\n \r\n theme_ipsum_rc(\r\n grid = grid, \r\n plot_title_size = font_sizes[1], \r\n subtitle_size = font_sizes[2], \r\n axis_title_size = font_sizes[3], \r\n axis_text_size = font_sizes[4], \r\n caption_size = font_sizes[5]\r\n )\r\n \r\n}<\/code><\/pre>\ngg_tweet()<\/code> function does not (now) use rtweet<\/code> as I’ll likely add it to either ggalt<\/code> or hrbrthemes<\/code> and want to keep dependencies to a minimum. I may opt-in to bypass the current method since it relies on environment variables vs an RDS file for app credential storage. Regardless, one thing I wanted to do here was provide a way to preview the image before tweeting.<\/p>\nmagick<\/code> object it creates in the event you want to do something more with it:<\/p>\ngg_tweet <- function(g, status = "ggplot2 image", style = c("stream", "card"), \r\n retina=TRUE, send = FALSE) {\r\n \r\n style <- match.arg(tolower(style), c("stream", "card"))\r\n \r\n switch(\r\n style, \r\n stream = c(w=1024, h=512),\r\n card = c(w=800, h=320)\r\n ) -> dims\r\n \r\n dims["res"] <- 72\r\n \r\n if (retina) dims <- dims * 2\r\n \r\n fig <- image_graph(width=dims["w"], height=dims["h"], res=dims["res"])\r\n print(g)\r\n dev.off()\r\n \r\n if (send) {\r\n \r\n message("Posting image to twitter...")\r\n \r\n tf <- tempfile(fileext = ".png")\r\n image_write(fig, tf, format="png")\r\n \r\n # Create an app at apps.twitter.com w\/callback URL of http:\/\/127.0.0.1:1410\r\n # Save the app name, consumer key and secret to the following\r\n # Environment variables\r\n \r\n app <- oauth_app(\r\n appname = Sys.getenv("TWITTER_APP_NAME"),\r\n key = Sys.getenv("TWITTER_CONSUMER_KEY"),\r\n secret = Sys.getenv("TWITTER_CONSUMER_SECRET")\r\n )\r\n \r\n twitter_token <- oauth1.0_token(oauth_endpoints("twitter"), app)\r\n \r\n POST(\r\n url = "https:\/\/api.twitter.com\/1.1\/statuses\/update_with_media.json",\r\n config(token = twitter_token), \r\n body = list(\r\n status = status,\r\n media = upload_file(path.expand(tf))\r\n )\r\n ) -> res\r\n \r\n warn_for_status(res)\r\n \r\n unlink(tf)\r\n \r\n }\r\n \r\n fig\r\n \r\n}<\/code><\/pre>\nTwo Great Tastes That Taste Great Together<\/h2>\n
gg_tweet(\r\n gg + theme_tweet_rc(grid="Y"),\r\n status = "Progress! #rtweet #gg_tweet",\r\n send=TRUE\r\n)<\/code><\/pre>\nNext Steps<\/h2>\n
gg_tweet()<\/code> is likely one of the few ways I’ll be interacting with Twitter for a while. You can ping me on Keybase \u2014\u00a0hrbrmstr<\/a> \u2014 or join the rstats<\/code> Keybase team<\/a> via keybase team request-access rstats<\/code> if you need to poke me for anything for a while.<\/p>\nFIN<\/h2>\n
gg_tweet()<\/code> ending up in ggalt<\/code> or hrbrthemes<\/code>. Don’t hesitate to suggest (or code up) feature requests. This is still an idea in-progress and definitely not ready for prime time without a bit more churning. (Also, words_to_num()<\/code> can be optimized, it was hastily crafted).<\/p>\n","protected":false},"excerpt":{"rendered":"