{"id":6558,"date":"2017-10-01T08:38:11","date_gmt":"2017-10-01T13:38:11","guid":{"rendered":"https:\/\/rud.is\/b\/?p=6558"},"modified":"2018-03-07T17:01:09","modified_gmt":"2018-03-07T22:01:09","slug":"retrieve-process-tv-news-chyrons-with-newsflash","status":"publish","type":"post","link":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/","title":{"rendered":"Retrieve &#038; process TV News chyrons with newsflash"},"content":{"rendered":"<p>The Internet Archive <a href=\"http:\/\/blog.archive.org\/2017\/09\/21\/tv-news-chyron-data\/\">recently announced<\/a> a new service they&#8217;ve dubbed &#8216;Third Eye&#8217;. This service scrapes the chyrons that annoyingly scroll across the bottom-third of TV news broadcasts. IA has a vast historical archive of TV news that they&#8217;ll eventually process, but &#8212; for now &#8212; the more recent broadcasts from four channels are readily available. There&#8217;s tons of information about the project <a href=\"https:\/\/archive.org\/details\/third-eye\">on its main page<\/a> where you can interactively work with the API if that&#8217;s how you roll.<\/p>\n<p>Since my <a href=\"https:\/\/github.com\/hrbrmstr\/newsflash\"><code>newsflash<\/code>?<\/a> package already had a &#8220;news&#8221; theme and worked with the joint IA-GDELT project TV data, it seemed to be a good home for a Third Eye interface to live.<\/p>\n<h3>Basic usage<\/h3>\n<p>You can read long-form details of the Third Eye service on their site. The TLDR is that they provide two feeds:<\/p>\n<ul>\n<li>a &#8220;raw&#8221; one which has massive duplicates and tons of errors<\/li>\n<li>a &#8220;clean&#8221; one that filters out duplicates, cleans up the text and is much better to work with<\/li>\n<\/ul>\n<p>You can retrieve either with <code>newsflash::read_chyrons()<\/code> but the default is to use the clean feed. If you are studying text processing and or NLP\/text-cleanup via machine learning, then the raw feed may be very interesting for you. I suspect most data journalists will want to use the clean feed that also powers the IA chyron twitter bots.<\/p>\n<p>Since it&#8217;s the Internet Archive, they&#8217;re awesome at providing metadata about their data. Heck, even their metadata has metadata about metadata. We can use the fact that they provide a metadata feed to enable listing available chyron archive dates:<\/p>\n<pre id=\"cyclops01\"><code class=\"language-r\">library(newsflash) # devtools::install_github(&quot;hrbrmstr\/newsflash&quot;)\r\nlibrary(hrbrthemes)\r\nlibrary(tidyverse)\r\n\r\nlist_chyrons()\r\n## # A tibble: 61 x 3\r\n##            ts    type     size\r\n##        &lt;date&gt;   &lt;chr&gt;    &lt;dbl&gt;\r\n##  1 2017-09-30 cleaned   539061\r\n##  2 2017-09-30     raw 17927121\r\n##  3 2017-09-29 cleaned   635812\r\n##  4 2017-09-29     raw 19234407\r\n##  5 2017-09-28 cleaned   414067\r\n##  6 2017-09-28     raw 12663606\r\n##  7 2017-09-27 cleaned   613474\r\n##  8 2017-09-27     raw 20442644\r\n##  9 2017-09-26 cleaned   659930\r\n## 10 2017-09-26     raw 19942951\r\n## # ... with 51 more rows<\/code><\/pre>\n<p>Reading the chyrons in only requires passing in a <code>Date<\/code> object or a <code>YYYY-mm-dd<\/code> format date string:<\/p>\n<pre id=\"cyclops02\"><code class=\"language-r\">chyrons &lt;- read_chyrons(&quot;2017-09-30&quot;)\r\n\r\n\r\nglimpse(chyrons)\r\n## Observations: 2,729\r\n## Variables: 5\r\n## $ ts       &lt;dttm&gt; 2017-09-30 00:00:00, 2017-09-30 00:00:00, 2017-09-30 00:00:00, 2017-09-30...\r\n## $ channel  &lt;chr&gt; &quot;BBCNEWS&quot;, &quot;CNNW&quot;, &quot;FOXNEWSW&quot;, &quot;BBCNEWS&quot;, &quot;CNNW&quot;, &quot;MSNBCW&quot;, &quot;BBCNEWS&quot;, &quot;CN...\r\n## $ duration &lt;int&gt; 18, 42, 26, 10, 47, 19, 14, 62, 26, 11, 45, 17, 35, 11, 62, 32, 35, 35, 15...\r\n## $ details  &lt;chr&gt; &quot;BBCNEWS_20170929_233000_Race_and_Pace\/start\/1800&quot;, &quot;CNNW_20170929_230000_...\r\n## $ text     &lt;chr&gt; &quot;TRUMP CABINET SECRETARY QUITS\\\\n&#039;MIRACLE NEEDED&#039; ON BREXIT&quot;, &quot;TRUMP BRAGS...<\/code><\/pre>\n<p>You get five columns in a data frame on a successful retrieval:<\/p>\n<ul>\n<li><code>ts<\/code> (<code>POSIXct<\/code>) chyron timestamp<\/li>\n<li><code>channel<\/code> (<code>character<\/code>) news channel the chyron appeared on<\/li>\n<li><code>duration<\/code> (<code>integer<\/code>) see Description<\/li>\n<li><code>details<\/code> (<code>character<\/code>) Internet Archive details path<\/li>\n<li><code>text<\/code> (<code>character<\/code>) the chyron text<\/li>\n<\/ul>\n<p>We&#8217;ll talk about the <code>details<\/code> path in a bit. The <code>text<\/code> is likely what you want, so here&#8217;s a sample:<\/p>\n<pre id=\"cyclops03\"><code class=\"language-r\">head(chyrons$text, 30)\r\n##  [1] &quot;TRUMP CABINET SECRETARY QUITS\\\\n&#039;MIRACLE NEEDED&#039; ON BREXIT&quot;                                                                                                                                                                                            \r\n##  [2] &quot;TRUMP BRAGS ABOUT PUERTO RICO RESPONSE AS FED-UP. SURVIVORS PLEAD FOR ELECTRICITY, WATER, FUEL\\\\nAnderson Cooper&quot;                                                                                                                                      \r\n##  [3] &quot;ALIFORNIA STUDENT SWIPES &#039;MAGA&#039; HAT&quot;                                                                                                                                                                                                                   \r\n##  [4] &quot;US HEALTH SECRETARY QUITS. Mr Price apologised for use O126 private \\\\ufb02ights since May\\\\nUS HEALTH SECRETARY QUITS. Private flights cost taxpayers 4OO,OOO dollars\\\\nLAURA BICKER. Washington&quot;                                                     \r\n##  [5] &quot;HHS SECY. PRICE OUT AFI&#039;ER PRIVATE JET SCANDAL\\\\nTRUMP BRAGS ABOUT PUERTO RICO RESPONSE AS FED-UP. SURVIVORS PLEAD FOR ELECTRICITY, WATER, FUEL&quot;                                                                                                       \r\n##  [6] &quot;TOM PRICE RESIGNS AMID PRIVATE JET SCANDAL&quot;                                                                                                                                                                                                            \r\n##  [7] &quot;US HEALTH SECRETARY QUITS. Private flights cost taxpayers 4OO,OOO dollars\\\\nUS HEALTH SECRETARY QUITS. Government otficials required to take commercial \\\\ufb02ights\\\\nUS HEALTH SECRETARY QUITS. Scandal emerged after...&quot;                            \r\n##  [8] &quot;HHS SECY. PRICE OUT AFI&#039;ER PRIVATE JET SCANDAL&quot;                                                                                                                                                                                                        \r\n##  [9] &quot;TOM PRICE RESIGNS AMID PRIVATE JET SCANDAL\\\\nTRUMP: \\\\\\&quot;I CERTAINLY DON&#039;T LIKE THE OPTICS\\\\\\&quot; OF PRICE SCANDAL&quot;                                                                                                                                        \r\n## [10] &quot;US HEALTH SECRETARY QUITS. Scandal emerged after investigation by Politico magazine\\\\nUS HEALTH SECRETARY QUITS. Tom Price resigned over use of private planes\\\\nUS HEALTH SECRETARY QUITS. Mr Price apologised for use O126...&quot;                       \r\n## [11] &quot;HHS SECY. PRICE OUT AFI&#039;ER PRIVATE JET SCANDAL\\\\nHHS SECY. PRICE OUT AFI&#039;ER PRIVATE JET SCANDAL. . Ryan Nobles (J\\\\\\\\N Washington Correspondent&quot;                                                                                                       \r\n## [12] &quot;BRARIAN REJECTS \\\\\\&quot;RACIST\\\\\\&quot; DR. SEUSS BOOKS I&quot;                                                                                                                                                                                                      \r\n## [13] &quot;TOM PRICE RESIGNS AMID PRIVATE JET SCANDAL\\\\nREPORTER WHO BROKE PRICE SCANDAL SPEAKS OUT&quot;                                                                                                                                                              \r\n## [14] &quot;US HEALTH SECRETARY QUITS. Tom Price resigned over use of private planes\\\\nUS HEALTH SECRETARY QUITS. Scandal emerged after investigation by Politico magazine\\\\nUS HEALTH SECRETARY QUITS. Mr Price apologised for...&quot;                                \r\n## [15] &quot;HHS SECY. PRICE OUT AFI&#039;ER PRIVATE JET SCANDAL&quot;                                                                                                                                                                                                        \r\n## [16] &quot;BIZARRE LIBERAL MELTDOWNS I\\\\nTUCKER &amp; THE CAT IN THE HAT I. . _ &lt; &#039;rnnwnn FAD! cnm tint-\\\\ufb01nk&quot;                                                                                                                                                    \r\n## [17] &quot;TRUMP: \\\\\\&quot;I CERTAINLY DON&#039;T LIKE THE OPTICS\\\\\\&quot; OF PRICE SCANDAL\\\\nTOM PRICE RESIGNS AMID PRIVATE JET SCANDAL&quot;                                                                                                                                        \r\n## [18] &quot;HHS SECY. PRICE OUT AFI&#039;ER PRIVATE JET SCANDAL\\\\nTRUMP BRAGS ABOUT PUERTO RICO RESPONSE AS FED-UP. SURVIVORS PLEAD FOR ELECTRICITY, WATER, FUEL&quot;                                                                                                       \r\n## [19] &quot;BRARIAN REJECTS \\\\\\&quot;RACIST\\\\\\&quot; DR. SEUSS BOOKS I\\\\nBIZARRE LIBERAL MELTDOWN I&quot;                                                                                                                                                                         \r\n## [20] &quot;TRUMP: \\\\\\&quot;I CERTAINLY DON&#039;T LIKE THE OPTICS\\\\\\&quot; OF PRICE SCANDAL&quot;                                                                                                                                                                                     \r\n## [21] &quot;TRUMP BRAGS ABOUT PUERTO RICO RESPONSE AS FED-UP. SURVIVORS PLEAD FOR ELECTRICITY, WATER, FUEL&quot;                                                                                                                                                        \r\n## [22] &quot;BRARIAN REJECTS \\\\\\&quot;RACIST\\\\\\&quot; DR. SEUSS BOOKS I\\\\nSCHOOL LIBRARIAN REJECTS DR. SEUSS. BOOKS GIFTED BY MELANIA TRUMP. . _&#039; tnnx&#039;nkr&quot;                                                                                                                   \r\n## [23] &quot;TRUMP: \\\\\\&quot;I CERTAINLY DON&#039;T LIKE THE OPTICS\\\\\\&quot; OF PRICE SCANDAL\\\\nTOM PRICE RESIGNS AMID PRIVATE JET SCANDAL&quot;                                                                                                                                        \r\n## [24] &quot;YEMEN WAR CRIMES. UN Human Rights Council agrees on investigation\\\\nINIGO MENDEZ DE VIGO. Spanish Education Minister&quot;                                                                                                                                  \r\n## [25] &quot;TRUMP BRAGS ABOUT PUERTO RICO RESPONSE AS FED-UP. SURVIVORS PLEAD FOR ELECTRICITY, WATER, FUEL\\\\nSAN JUAN MAYOR: \\\\\\&quot;THIS IS NOT A GOOD NEWS STORY\\\\\\&quot;\\\\nSAN JUAN MAYOR: \\\\\\&quot;THIS IS NOT A GOOD NEWS STORY\\\\\\&quot;. . Mavor Carmen Yulin Cruz San Juan,...&quot;\r\n## [26] &quot;BRARIAN REJECTS \\\\\\&quot;RACIST\\\\\\&quot; DR. SEUSS BOOKS&quot;                                                                                                                                                                                                        \r\n## [27] &quot;TOM PRICE RESIGNS AMID PRIVATE JET SCANDAL&quot;                                                                                                                                                                                                            \r\n## [28] &quot;TRUMP ASIA TOUR. US President to visit Japan, South Korea and China\\\\nYEMEN WAR CRIMES. UN Human Rights Council agrees on investigation&quot;                                                                                                               \r\n## [29] &quot;SAN JUAN MAYOR: \\\\\\&quot;MAD AS HELL\\\\\\&quot; OVER HURRICANE RESPONSE\\\\nSAN JUAN MAYOR: \\\\\\&quot;MAD AS HELLII OVER HURRICANE RESPONSE. . Dr. Saniav Gupta (J\\\\\\\\N Chief Medical Correspondent&quot;                                                                       \r\n## [30] &quot;SAN JUAN MAYOR: \\\\\\&quot;MAD AS HELL\\\\\\&quot; OVER HURRICANE RESPONSE&quot;<\/code><\/pre>\n<p>Be warned: even the &#8220;clean&#8221; text is often kinda messy.<\/p>\n<p>For now, there are only four channels, so it&#8217;s easy to show a quick example. Since chyrons are <em>supposed<\/em> to be super-important things you need to know NOW, let&#8217;s see how many times Puerto Rico was mentioned on them in the above archive. NOTE: This is a quick example, not a thorough one. I&#8217;m searching for some key letter combinations to see just mentions of something looking like &#8220;Puerto Rico&#8221;. &#8220;San Juan&#8221; and other text that might be associated with the topic aren&#8217;t being considered for this toy example.<\/p>\n<pre id=\"cyclops04\"><code class=\"language-r\">mutate(\r\n  chyrons, \r\n  hour = lubridate::hour(ts),\r\n  text = tolower(text),\r\n  mention = grepl(&quot;erto ri&quot;, text)\r\n) %&gt;% \r\n  filter(mention) %&gt;% \r\n  count(hour, channel) %&gt;% \r\n  ggplot(aes(hour, n)) +\r\n  geom_segment(aes(xend=hour, yend=0)) +\r\n  scale_x_continuous(name=&quot;Hour (GMT)&quot;, breaks=seq(0, 23, 6),\r\n                     labels=sprintf(&quot;%02d:00&quot;, seq(0, 23, 6))) +\r\n  scale_y_continuous(name=&quot;# Chyrons&quot;, limits=c(0,30)) +\r\n  facet_wrap(~channel, scales=&quot;free&quot;) +\r\n  labs(title=&quot;Chyrons mentioning &#039;Puerto Rico&#039; per hour per channel&quot;,\r\n       subtitle=&quot;Chyron date: 2017-09-30&quot;,\r\n       caption=&quot;Source: Internet Archive Third Eye project &amp; &lt;github.com\/hrbrmstr\/newsflash&gt;&quot;) +\r\n  theme_ipsum_rc(grid=&quot;Y&quot;)<\/code><\/pre>\n<p><a href=\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?ssl=1\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"6559\" data-permalink=\"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/chy01\/\" data-orig-file=\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1690%2C836&amp;ssl=1\" data-orig-size=\"1690,836\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"chy01\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=510%2C252&amp;ssl=1\" src=\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?resize=510%2C252&#038;ssl=1\" alt=\"\" width=\"510\" height=\"252\" class=\"aligncenter size-full wp-image-6559\" \/><\/a><\/p>\n<h3>Details, details, details<\/h3>\n<p>Entries in <code>details<\/code> column look like this:<\/p>\n<pre id=\"cyclops05\"><code class=\"language-r\">head(chyrons$details)\r\n## [1] &quot;BBCNEWS_20170929_233000_Race_and_Pace\/start\/1800&quot;                   \r\n## [2] &quot;CNNW_20170929_230000_Erin_Burnett_OutFront\/start\/3600&quot;              \r\n## [3] &quot;FOXNEWSW_20170929_230000_The_Story_With_Martha_MacCallum\/start\/3600&quot;\r\n## [4] &quot;BBCNEWS_20170930_000000_BBC_News\/start\/60&quot;                          \r\n## [5] &quot;CNNW_20170930_000000_Anderson_Cooper_360\/start\/60&quot;                  \r\n## [6] &quot;MSNBCW_20170930_000000_All_In_With_Chris_Hayes\/start\/60&quot;<\/code><\/pre>\n<p>They are path fragments that can be attached to a URL prefix to see the news clip from that station on that day\/time. <code>newsflash::view_clip()<\/code> does that work for you:<\/p>\n<pre id=\"cyclops06\"><code class=\"language-r\">view_clip(chyrons$details[2])<\/code><\/pre>\n<p><iframe style=\"width:100%; height:600px\" seamless src=\"https:\/\/archive.org\/details\/CNNW_20170929_230000_Erin_Burnett_OutFront\/start\/3600\/end\/3660\"><\/iframe><\/p>\n<p>The URL for that is <code>https:\/\/archive.org\/details\/CNNW_20170929_230000_Erin_Burnett_OutFront\/start\/3600\/end\/3660<\/code> in the event the iframe load failed or you really like being annoyed with cable news shows.<\/p>\n<h3>FIN<\/h3>\n<p>Grab the package <a href=\"https:\/\/github.com\/hrbrmstr\/newsflash\">on GitHub<\/a>, kick the tyres and don&#8217;t hesitate to file issues, questions or jump on board with package development. There&#8217;s plenty of room for improvement before it hits CRAN and your ideas are most welcome.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The Internet Archive recently announced a new service they&#8217;ve dubbed &#8216;Third Eye&#8217;. This service scrapes the chyrons that annoyingly scroll across the bottom-third of TV news broadcasts. IA has a vast historical archive of TV news that they&#8217;ll eventually process, but &#8212; for now &#8212; the more recent broadcasts from four channels are readily available. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":6559,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[91],"tags":[810],"class_list":["post-6558","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-r","tag-post"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Retrieve &amp; process TV News chyrons with newsflash - rud.is<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Retrieve &amp; process TV News chyrons with newsflash - rud.is\" \/>\n<meta property=\"og:description\" content=\"The Internet Archive recently announced a new service they&#8217;ve dubbed &#8216;Third Eye&#8217;. This service scrapes the chyrons that annoyingly scroll across the bottom-third of TV news broadcasts. IA has a vast historical archive of TV news that they&#8217;ll eventually process, but &#8212; for now &#8212; the more recent broadcasts from four channels are readily available. [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/\" \/>\n<meta property=\"og:site_name\" content=\"rud.is\" \/>\n<meta property=\"article:published_time\" content=\"2017-10-01T13:38:11+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-03-07T22:01:09+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1690%2C836&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"1690\" \/>\n\t<meta property=\"og:image:height\" content=\"836\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"hrbrmstr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hrbrmstr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/\"},\"author\":{\"name\":\"hrbrmstr\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"headline\":\"Retrieve &#038; process TV News chyrons with newsflash\",\"datePublished\":\"2017-10-01T13:38:11+00:00\",\"dateModified\":\"2018-03-07T22:01:09+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/\"},\"wordCount\":569,\"commentCount\":2,\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2017\\\/10\\\/chy01.png?fit=1690%2C836&ssl=1\",\"keywords\":[\"post\"],\"articleSection\":[\"R\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/\",\"name\":\"Retrieve & process TV News chyrons with newsflash - rud.is\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2017\\\/10\\\/chy01.png?fit=1690%2C836&ssl=1\",\"datePublished\":\"2017-10-01T13:38:11+00:00\",\"dateModified\":\"2018-03-07T22:01:09+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/#primaryimage\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2017\\\/10\\\/chy01.png?fit=1690%2C836&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2017\\\/10\\\/chy01.png?fit=1690%2C836&ssl=1\",\"width\":1690,\"height\":836},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/10\\\/01\\\/retrieve-process-tv-news-chyrons-with-newsflash\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/rud.is\\\/b\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Retrieve &#038; process TV News chyrons with newsflash\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/\",\"name\":\"rud.is\",\"description\":\"&quot;In God we trust. All others must bring data&quot;\",\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/rud.is\\\/b\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\",\"name\":\"hrbrmstr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"width\":460,\"height\":460,\"caption\":\"hrbrmstr\"},\"logo\":{\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\"},\"description\":\"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7\",\"sameAs\":[\"http:\\\/\\\/rud.is\"],\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/author\\\/hrbrmstr\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Retrieve & process TV News chyrons with newsflash - rud.is","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/","og_locale":"en_US","og_type":"article","og_title":"Retrieve & process TV News chyrons with newsflash - rud.is","og_description":"The Internet Archive recently announced a new service they&#8217;ve dubbed &#8216;Third Eye&#8217;. This service scrapes the chyrons that annoyingly scroll across the bottom-third of TV news broadcasts. IA has a vast historical archive of TV news that they&#8217;ll eventually process, but &#8212; for now &#8212; the more recent broadcasts from four channels are readily available. [&hellip;]","og_url":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/","og_site_name":"rud.is","article_published_time":"2017-10-01T13:38:11+00:00","article_modified_time":"2018-03-07T22:01:09+00:00","og_image":[{"width":1690,"height":836,"url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1690%2C836&ssl=1","type":"image\/png"}],"author":"hrbrmstr","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hrbrmstr","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/#article","isPartOf":{"@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/"},"author":{"name":"hrbrmstr","@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"headline":"Retrieve &#038; process TV News chyrons with newsflash","datePublished":"2017-10-01T13:38:11+00:00","dateModified":"2018-03-07T22:01:09+00:00","mainEntityOfPage":{"@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/"},"wordCount":569,"commentCount":2,"publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"image":{"@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1690%2C836&ssl=1","keywords":["post"],"articleSection":["R"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/","url":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/","name":"Retrieve & process TV News chyrons with newsflash - rud.is","isPartOf":{"@id":"https:\/\/rud.is\/b\/#website"},"primaryImageOfPage":{"@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/#primaryimage"},"image":{"@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1690%2C836&ssl=1","datePublished":"2017-10-01T13:38:11+00:00","dateModified":"2018-03-07T22:01:09+00:00","breadcrumb":{"@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/#primaryimage","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1690%2C836&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1690%2C836&ssl=1","width":1690,"height":836},{"@type":"BreadcrumbList","@id":"https:\/\/rud.is\/b\/2017\/10\/01\/retrieve-process-tv-news-chyrons-with-newsflash\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/rud.is\/b\/"},{"@type":"ListItem","position":2,"name":"Retrieve &#038; process TV News chyrons with newsflash"}]},{"@type":"WebSite","@id":"https:\/\/rud.is\/b\/#website","url":"https:\/\/rud.is\/b\/","name":"rud.is","description":"&quot;In God we trust. All others must bring data&quot;","publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/rud.is\/b\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886","name":"hrbrmstr","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","width":460,"height":460,"caption":"hrbrmstr"},"logo":{"@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1"},"description":"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7","sameAs":["http:\/\/rud.is"],"url":"https:\/\/rud.is\/b\/author\/hrbrmstr\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/10\/chy01.png?fit=1690%2C836&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p23idr-1HM","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":6226,"url":"https:\/\/rud.is\/b\/2017\/09\/09\/teasing-out-top-daily-topics-with-gdelts-television-explorer\/","url_meta":{"origin":6558,"position":0},"title":"Teasing Out Top Daily Topics with GDELT&#8217;s Television Explorer","author":"hrbrmstr","date":"2017-09-09","format":false,"excerpt":"Earlier this year, the GDELT Project released their Television Explorer that enabled API access to closed-caption tedt from television news broadcasts. They've done an incredible job expanding and stabilizing the API and just recently released \"top trending tables\" which summarise what the \"top\" topics and phrases are across news stations\u2026","rel":"","context":"In &quot;R&quot;","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1173%2C1200&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1173%2C1200&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1173%2C1200&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1173%2C1200&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/09\/Plot_Zoom.png?fit=1173%2C1200&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":4968,"url":"https:\/\/rud.is\/b\/2017\/02\/01\/exploring-news-coverage-with-newsflash\/","url_meta":{"origin":6558,"position":1},"title":"Exploring News Coverage With newsflash","author":"hrbrmstr","date":"2017-02-01","format":false,"excerpt":"I was enthused to see a mention of this on the GDELT blog since I've been working on an R package dubbed newsflash to work with the API that the form front-ends. Given the current climate, I feel compelled to note that I'm neither a Clinton supporter\/defender\/advocate nor a ?\u2026","rel":"","context":"In &quot;R&quot;","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/02\/clinton_plot-1-1.png?fit=1200%2C600&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":11527,"url":"https:\/\/rud.is\/b\/2018\/09\/17\/access-the-internet-archive-advanced-search-scrape-api-with-wayback-a-links-to-a-new-vignette-pkgdown-site\/","url_meta":{"origin":6558,"position":2},"title":"Access the Internet Archive Advanced Search\/Scrape API with wayback (+ links to a new vignette &#038; pkgdown site)","author":"hrbrmstr","date":"2018-09-17","format":false,"excerpt":"The wayback? package has had an update to more efficiently retrieve mementos and added support for working with the Internet Archive's advanced search+scrape API. Search\/Scrape The search\/scrape interface lets you examine the IA collections and download what you are after (programmatically). The main function is ia_scrape() but you can also\u2026","rel":"","context":"In &quot;R&quot;","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":11648,"url":"https:\/\/rud.is\/b\/2018\/11\/14\/use-github-vulnerability-alerts-to-keep-users-of-your-r-packages-safe\/","url_meta":{"origin":6558,"position":3},"title":"Use GitHub Vulnerability Alerts to Keep Users of Your R Packages Safe","author":"hrbrmstr","date":"2018-11-14","format":false,"excerpt":"Despite their now inherent evil status, GitHub has some tools other repository aggregators do not. One such tool is the free vulnerability alert service which will scan repositories for outdated+vulnerable dependencies. Now, \"R\" is nowhere near a first-class citizen in the internet writ large, including software development tooling (e.g. the\u2026","rel":"","context":"In &quot;Cybersecurity&quot;","block_context":{"text":"Cybersecurity","link":"https:\/\/rud.is\/b\/category\/cybersecurity\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/11\/Screen-Shot-2018-11-14-at-08.43.14.png?fit=1200%2C424&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/11\/Screen-Shot-2018-11-14-at-08.43.14.png?fit=1200%2C424&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/11\/Screen-Shot-2018-11-14-at-08.43.14.png?fit=1200%2C424&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/11\/Screen-Shot-2018-11-14-at-08.43.14.png?fit=1200%2C424&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/11\/Screen-Shot-2018-11-14-at-08.43.14.png?fit=1200%2C424&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":10284,"url":"https:\/\/rud.is\/b\/2018\/05\/16\/ripe-76-selected-talks\/","url_meta":{"origin":6558,"position":4},"title":"RIPE 76 Selected Talks","author":"hrbrmstr","date":"2018-05-16","format":false,"excerpt":"RIPE 76 is going on this week and --- as usual --- there are scads of great talks. The selected ones below are just my (slightly) thinner slice at what may have broader appeal outside pure networking circles. Do not read anything more into the order than the end-number of\u2026","rel":"","context":"In &quot;Cybersecurity&quot;","block_context":{"text":"Cybersecurity","link":"https:\/\/rud.is\/b\/category\/cybersecurity\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":13317,"url":"https:\/\/rud.is\/b\/2022\/01\/28\/a-weatherflow-tempest-udp-broadcast-receiver-in-golang-rust-swift-and-c\/","url_meta":{"origin":6558,"position":5},"title":"A WeatherFlow Tempest UDP Broadcast Receiver in Golang, Rust, Swift, and C++","author":"hrbrmstr","date":"2022-01-28","format":false,"excerpt":"After a Twitter convo about weather stations I picked up a WeatherFlow Tempest. Setup was quick, but the sensor package died within 24 hours. I was going to give up on it but I had written an R package (for the REST API & UDP broadcast interfaces) and C++ utility\u2026","rel":"","context":"In &quot;C++&quot;","block_context":{"text":"C++","link":"https:\/\/rud.is\/b\/category\/c\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/6558","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/comments?post=6558"}],"version-history":[{"count":0,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/6558\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media\/6559"}],"wp:attachment":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media?parent=6558"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/categories?post=6558"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/tags?post=6558"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}