{"id":6226,"date":"2017-09-09T14:12:27","date_gmt":"2017-09-09T19:12:27","guid":{"rendered":"https:\/\/rud.is\/b\/?p=6226"},"modified":"2018-03-10T07:54:15","modified_gmt":"2018-03-10T12:54:15","slug":"teasing-out-top-daily-topics-with-gdelts-television-explorer","status":"publish","type":"post","link":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/","title":{"rendered":"Teasing Out Top Daily Topics with GDELT’s Television Explorer"},"content":{"rendered":"

Earlier this year, the GDELT Project<\/a> released their Television Explorer<\/a> that enabled API access to closed-caption tedt from television news broadcasts. They’ve done an incredible job expanding and stabilizing the API and just recently released “top trending tables<\/a>” which summarise what the “top” topics and phrases are across news stations every fifteen minutes. You should read that (long-ish) intro as there are many caveats to the data source and I’ve also found that the files aren’t always available (i.e. there are often gaps when retrieving a sequence of files).<\/p>\n

The R newsflash<\/code><\/a> package has been able to work with the GDELT Television Explorer API since the inception of the service. It now has the ability work with this new “top topics” resource directly from R.<\/p>\n

There are two interfaces to the top topics, but I’ll show you the easiest one to use in this post. Let’s chart the top 25 topics per day for the past ~3 days (this post was generated ~mid-day 2017-09-09).<\/p>\n

To start, we’ll need the data!<\/p>\n

We provide start and end POSIXct<\/code> times in the current time zone (the top_trending_range()<\/code> function auto-converts to GMT which is how the file timestamps are stored by GDELT). The function takes care of generating the proper 15-minute sequences.<\/p>\n

library(newsflash) # devtools::install_github("hrbrmstr\/newsflash")\r\nlibrary(hrbrthemes)\r\nlibrary(tidyverse)\r\n\r\nfrom <- as.POSIXct("2017-09-07 00:00:00")\r\nto <- as.POSIXct("2017-09-09 12:00:00")\r\n\r\ntrends <- top_trending_range(from, to)\r\n\r\nglimpse(trends)\r\n## Observations: 233\r\n## Variables: 5\r\n## $ ts                       <dttm> 2017-09-07 00:00:00, 2017-09-07 00:15:00, 2017-...\r\n## $ overall_trending_topics  <list> [<"florida", "irma", "barbuda", "puerto rico", ...\r\n## $ station_trending_topics  <list> [<c("CNN", "BLOOMBERG", "CNBC", "FBC", "FOXNEWS...\r\n## $ station_top_topics       <list> [<c("CNN", "BLOOMBERG", "CNBC", "FBC", "FOXNEWS...\r\n## $ overall_trending_phrases <list> [<"debt ceiling", "legalize daca", "florida key...<\/code><\/pre>\n
The glimpse<\/code> view shows a compact, nested data frame. I encourage you to explore the individual nested elements to see the gems they contain, but we’re going to focus on the station_top_topics<\/code>:<\/p>\n
glimpse(trends$station_top_topics[[1]])\r\n## Variables: 2\r\n## $ Station <chr> "CNN", "BLOOMBERG", "CNBC", "FBC", "FOXNEWS", "MSNBC", "BBCNEWS"\r\n## $ Topics  <list> [<"florida", "irma", "daca", "north korea", "harvey", "united st...<\/code><\/pre>\n
Each individual data frame has the top topics of each tracked station.<\/p>\n
To get the top 25 topics per day, we’re going to bust out this structure, count up the topic “mentions” (not 100% accurate term, but good enough for now) per day and slice out the top 25. It’s a pretty straightforward process with tidyverse ops:<\/p>\n
select(trends, ts, station_top_topics) %>% \r\n  unnest() %>% \r\n  unnest() %>% \r\n  mutate(day = as.Date(ts)) %>% \r\n  rename(station=Station, topic=Topics) %>% \r\n  count(day, topic) %>% \r\n  group_by(day) %>% \r\n  top_n(25) %>% \r\n  slice(1:25) %>% \r\n  arrange(day, desc(n)) %>% \r\n  mutate(rnk = 25:1) -> top_25_trends\r\n\r\nglimpse(top_25_trends)\r\n## Observations: 75\r\n## Variables: 4\r\n## $ day   <date> 2017-09-07, 2017-09-07, 2017-09-07, 2017-09-07, 2017-09-07, 2017-0...\r\n## $ topic <chr> "florida", "irma", "harvey", "north korea", "america", "daca", "chi...\r\n## $ n     <int> 546, 546, 468, 464, 386, 362, 356, 274, 217, 210, 200, 156, 141, 13...\r\n## $ rnk   <int> 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, ...<\/code><\/pre>\n
Now, it’s just a matter of some ggplotting:<\/p>\n
ggplot(top_25_trends, aes(day, rnk, label=topic, size=n)) +\r\n  geom_text(vjust=0.5, hjust=0.5) +\r\n  scale_x_date(expand=c(0,0.5)) +\r\n  scale_size(name=NULL, range=c(3,8)) +\r\n  labs(\r\n    x=NULL, y=NULL, \r\n    title="Top 25 Trending Topics Per Day",\r\n    subtitle="Topic placed by rank and sized by frequency",\r\n    caption="GDELT Television Explorer & #rstats newsflash package github.com\/hrbrmstr\/newsflash"\r\n  ) +\r\n  theme_ipsum_rc(grid="") +\r\n  theme(axis.text.y=element_blank()) +\r\n  theme(legend.position=c(0.75, 1.05)) +\r\n  theme(legend.direction="horizontal")<\/code><\/pre>\n
\"\"<\/a><\/p>\n
Hopefully you’ll have some fun with the new “API”. Make sure to blog your own creations!<\/p>\n
UPDATE<\/strong><\/p>\n
As a result of a tweet by @arnicas, you can find a per-day, per-station view (top 10 only) here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
Earlier this year, the GDELT Project released their Television Explorer that enabled API access to closed-caption tedt from television news broadcasts. They’ve done an incredible job expanding and stabilizing the API and just recently released “top trending tables” which summarise what the “top” topics and phrases are across news stations every fifteen minutes. You should […]<\/p>\n","protected":false},"author":1,"featured_media":6234,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[91],"tags":[800,810],"class_list":["post-6226","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-r","tag-gdelt","tag-post"],"yoast_head":"\nTeasing Out Top Daily Topics with GDELT's Television Explorer - rud.is<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Teasing Out Top Daily Topics with GDELT's Television Explorer - rud.is\" \/>\n<meta property=\"og:description\" content=\"Earlier this year, the GDELT Project released their Television Explorer that enabled API access to closed-caption tedt from television news broadcasts. They’ve done an incredible job expanding and stabilizing the API and just recently released “top trending tables” which summarise what the “top” topics and phrases are across news stations every fifteen minutes. You should […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/\" \/>\n<meta property=\"og:site_name\" content=\"rud.is\" \/>\n<meta property=\"article:published_time\" content=\"2017-09-09T19:12:27+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-03-10T12:54:15+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1656%2C1694&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"1656\" \/>\n\t<meta property=\"og:image:height\" content=\"1694\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"hrbrmstr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hrbrmstr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/\"},\"author\":{\"name\":\"hrbrmstr\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"headline\":\"Teasing Out Top Daily Topics with GDELT’s Television Explorer\",\"datePublished\":\"2017-09-09T19:12:27+00:00\",\"dateModified\":\"2018-03-10T12:54:15+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/\"},\"wordCount\":373,\"commentCount\":10,\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2017\\\/09\\\/Plot_Zoom.png?fit=1656%2C1694&ssl=1\",\"keywords\":[\"gdelt\",\"post\"],\"articleSection\":[\"R\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/\",\"name\":\"Teasing Out Top Daily Topics with GDELT's Television Explorer - rud.is\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2017\\\/09\\\/Plot_Zoom.png?fit=1656%2C1694&ssl=1\",\"datePublished\":\"2017-09-09T19:12:27+00:00\",\"dateModified\":\"2018-03-10T12:54:15+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/#primaryimage\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2017\\\/09\\\/Plot_Zoom.png?fit=1656%2C1694&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2017\\\/09\\\/Plot_Zoom.png?fit=1656%2C1694&ssl=1\",\"width\":1656,\"height\":1694},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/09\\\/09\\\/teasing-out-top-daily-topics-with-gdelts-television-explorer\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/rud.is\\\/b\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Teasing Out Top Daily Topics with GDELT’s Television Explorer\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/\",\"name\":\"rud.is\",\"description\":\""In God we trust. All others must bring data"\",\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/rud.is\\\/b\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\",\"name\":\"hrbrmstr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"width\":460,\"height\":460,\"caption\":\"hrbrmstr\"},\"logo\":{\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\"},\"description\":\"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7\",\"sameAs\":[\"http:\\\/\\\/rud.is\"],\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/author\\\/hrbrmstr\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Teasing Out Top Daily Topics with GDELT's Television Explorer - rud.is","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/","og_locale":"en_US","og_type":"article","og_title":"Teasing Out Top Daily Topics with GDELT's Television Explorer - rud.is","og_description":"Earlier this year, the GDELT Project released their Television Explorer that enabled API access to closed-caption tedt from television news broadcasts. They’ve done an incredible job expanding and stabilizing the API and just recently released “top trending tables” which summarise what the “top” topics and phrases are across news stations every fifteen minutes. You should […]","og_url":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/","og_site_name":"rud.is","article_published_time":"2017-09-09T19:12:27+00:00","article_modified_time":"2018-03-10T12:54:15+00:00","og_image":[{"width":1656,"height":1694,"url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1656%2C1694&ssl=1","type":"image\/png"}],"author":"hrbrmstr","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hrbrmstr","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/#article","isPartOf":{"@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/"},"author":{"name":"hrbrmstr","@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"headline":"Teasing Out Top Daily Topics with GDELT’s Television Explorer","datePublished":"2017-09-09T19:12:27+00:00","dateModified":"2018-03-10T12:54:15+00:00","mainEntityOfPage":{"@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/"},"wordCount":373,"commentCount":10,"publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"image":{"@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1656%2C1694&ssl=1","keywords":["gdelt","post"],"articleSection":["R"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/","url":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/","name":"Teasing Out Top Daily Topics with GDELT's Television Explorer - rud.is","isPartOf":{"@id":"https:\/\/rud.is\/b\/#website"},"primaryImageOfPage":{"@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/#primaryimage"},"image":{"@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1656%2C1694&ssl=1","datePublished":"2017-09-09T19:12:27+00:00","dateModified":"2018-03-10T12:54:15+00:00","breadcrumb":{"@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/#primaryimage","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1656%2C1694&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1656%2C1694&ssl=1","width":1656,"height":1694},{"@type":"BreadcrumbList","@id":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/rud.is\/b\/"},{"@type":"ListItem","position":2,"name":"Teasing Out Top Daily Topics with GDELT’s Television Explorer"}]},{"@type":"WebSite","@id":"https:\/\/rud.is\/b\/#website","url":"https:\/\/rud.is\/b\/","name":"rud.is","description":""In God we trust. All others must bring data"","publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/rud.is\/b\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886","name":"hrbrmstr","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","width":460,"height":460,"caption":"hrbrmstr"},"logo":{"@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1"},"description":"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7","sameAs":["http:\/\/rud.is"],"url":"https:\/\/rud.is\/b\/author\/hrbrmstr\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1656%2C1694&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p23idr-1Cq","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":4968,"url":"https:\/\/rud.is\/b\/2017\/02\/01\/exploring-news-coverage-with-newsflash\/","url_meta":{"origin":6226,"position":0},"title":"Exploring News Coverage With newsflash","author":"hrbrmstr","date":"2017-02-01","format":false,"excerpt":"I was enthused to see a mention of this on the GDELT blog since I've been working on an R package dubbed newsflash to work with the API that the form front-ends. Given the current climate, I feel compelled to note that I'm neither a Clinton supporter\/defender\/advocate nor a ?\u2026","rel":"","context":"In "R"","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":6558,"url":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/","url_meta":{"origin":6226,"position":1},"title":"Retrieve & process TV News chyrons with newsflash","author":"hrbrmstr","date":"2017-10-01","format":false,"excerpt":"The Internet Archive recently announced a new service they've dubbed 'Third Eye'. This service scrapes the chyrons that annoyingly scroll across the bottom-third of TV news broadcasts. IA has a vast historical archive of TV news that they'll eventually process, but --- for now --- the more recent broadcasts from\u2026","rel":"","context":"In "R"","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1200%2C594&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1200%2C594&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1200%2C594&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1200%2C594&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1200%2C594&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":11088,"url":"https:\/\/rud.is\/b\/2018\/07\/26\/two-new-apache-drill-udfs-for-processing-urils-and-internet-domain-names\/","url_meta":{"origin":6226,"position":2},"title":"Two new Apache Drill UDFs for Processing UR[IL]s  and Internet Domain Names","author":"hrbrmstr","date":"2018-07-26","format":false,"excerpt":"Continuing the blog's UDF theme of late, there are two new UDF kids in town: drill-url-tools? for slicing & dicing URI\/URLs (just going to use 'URL' from now on in the post) drill-domain-tools? for slicing & dicing internet domain names (IDNs). Now, if you're an Apache Drill fanatic, you're likely\u2026","rel":"","context":"In "Apache Drill"","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":6164,"url":"https:\/\/rud.is\/b\/2017\/08\/22\/caching-httr-requests-this-means-warc\/","url_meta":{"origin":6226,"position":3},"title":"Caching httr Requests? This means WAR[C]!","author":"hrbrmstr","date":"2017-08-22","format":false,"excerpt":"I've blathered about my crawl_delay project before and am just waiting for a rainy weekend to be able to crank out a follow-up post on it. Working on that project involved sifting through thousands of Web Archive (WARC) files. While I have a nascent package on github to work with\u2026","rel":"","context":"In "R"","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4852,"url":"https:\/\/rud.is\/b\/2017\/01\/08\/2017-01-authored-package-updates\/","url_meta":{"origin":6226,"position":4},"title":"2017-01 Authored Package Updates","author":"hrbrmstr","date":"2017-01-08","format":false,"excerpt":"The rest of the month is going to be super-hectic and it's unlikely I'll be able to do any more to help the push to CRAN 10K, so here's a breakdown of CRAN and GitHub new packages & package updates that I felt were worth raising awareness on: epidata I\u2026","rel":"","context":"In "dplyr"","block_context":{"text":"dplyr","link":"https:\/\/rud.is\/b\/category\/dplyr\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/01\/epi2.png?fit=982%2C1200&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/01\/epi2.png?fit=982%2C1200&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/01\/epi2.png?fit=982%2C1200&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/01\/epi2.png?fit=982%2C1200&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":4753,"url":"https:\/\/rud.is\/b\/2016\/12\/20\/sergeant-a-r-boot-camp-for-apache-drill\/","url_meta":{"origin":6226,"position":5},"title":"sergeant : An R Boot Camp for Apache Drill","author":"hrbrmstr","date":"2016-12-20","format":false,"excerpt":"I recently mentioned that I've been working on a development version of an Apache Drill R package called sergeant. Here's a lifted \"TLDR\" on Drill: Drill supports a variety of NoSQL databases and file systems, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift,\u2026","rel":"","context":"In "Apache Drill"","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/6226","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/comments?post=6226"}],"version-history":[{"count":0,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/6226\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media\/6234"}],"wp:attachment":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media?parent=6226"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/categories?post=6226"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/tags?post=6226"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}