<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/xml-parsing-made-easy-is-that-podcast-getting-longer/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://nsaunders.wordpress.com/2017/10/12/xml-parsing-made-easy-is-that-podcast-getting-longer/"> R – What You're Doing Is Rather Desperate</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Sometime in 2009, I began listening to a science podcast titled <a rel="nofollow" href="http://www.microbe.tv/twiv/" target="_blank">This Week in Virology</a>, or TWiV for short. I thought it was pretty good and listened regularly up until sometime in 2016, when it seemed that most episodes were approaching two hours in duration. I listen to several podcasts when commuting to/from work, which takes up about 10 hours of my week, so I found it hard to justify two hours for one podcast, no matter how good.</p>
<p>Were the episodes really getting longer over time? Let’s find out using R.<br>
<span></span></p>
<p>One thing I’ve learned as a data scientist: management want to see the key points first. So here it is:</p>
<p><a rel="nofollow" href="https://nsaunders.wordpress.com/2017/10/12/xml-parsing-made-easy-is-that-podcast-getting-longer/twiv-joy-1/" target="_blank"><img data-orig-file="https://nsaunders.files.wordpress.com/2017/10/twiv-joy-1.png?w=456" data-orig-size="672,480" data-medium-file="https://nsaunders.files.wordpress.com/2017/10/twiv-joy-1.png?w=456?w=300" data-attachment-id="4967" data-image-meta="{" data-image-description="" src="https://nsaunders.files.wordpress.com/2017/10/twiv-joy-1.png?w=456" data-permalink="https://nsaunders.wordpress.com/2017/10/12/xml-parsing-made-easy-is-that-podcast-getting-longer/twiv-joy-1/" data-large-file="https://nsaunders.files.wordpress.com/2017/10/twiv-joy-1.png?w=456?w=672" alt="" data-image-title="twiv-joy-1" sizes="(max-width: 672px) 100vw, 672px" data-comments-opened="1" class="wp-image-4967" data-recalc-dims="1"></a></p>
<p>Technical people want to see how we got there.</p>
<p>It turns out that the podcast has an RSS feed (in XML format), containing detailed information about every episode to date. Once there was the <code>XML</code> package, which was OK, but now we have <code>rvest</code> and <code>xml2</code>, which makes life much easier.</p>
<p>Full code and a markdown report <a rel="nofollow" href="https://github.com/neilfws/media/tree/master/podcasts/twiv/code/R" target="_blank">can be found at Github</a>, so don’t copy and paste from here. Here are the highlights. First, as data for each episode is found between <code>item</code> tags, it can be extracted from the feed like so:</p>
<pre>
library(rvest)
feed_items &lt;- read_xml(&quot;http://twiv.microbeworld.libsynpro.com/twiv&quot;) %>%
  xml_nodes(&quot;item&quot;)
</pre>
<p>Podcast date is stored in <code>pubDate</code> and duration in <code>itunes:duration</code>, so we can extract the text between those tags:</p>
<pre>
feed_items %>% 
  xml_nodes(&quot;pubDate&quot;) %>% 
  xml_text()

feed_items %>% 
  xml_nodes(&quot;itunes\\:duration&quot;) %>% 
  xml_text()
</pre>
<p>Durations less than 1 hour are represented as minutes:seconds, e.g. “55:42”, as opposed to “02:01:33”. So we can count the colons, prepend “00:” (zero hours) if there is only one, convert to hours, minutes and seconds using <code>lubridate</code> and then convert to numeric seconds. We can also convert the date/time string to class datetime.</p>
<pre>
# assume myDF is a data frame with columns named pubDate, duration
myDF %>%
  mutate(pubDate = as.POSIXct(strptime(pubDate, &quot;%a, %d %b %Y %H:%M:%S +0000&quot;)),
         duration = ifelse(grepl(&quot;:\\d+:&quot;, duration), duration, paste0(&quot;00:&quot;, duration)),
         duration_seconds = as.numeric(hms(duration)))
</pre>
<p>We have dates, we have durations (bar one for episode 124 – not a big deal), we’re good to go.</p>
<p>I tried a scatter plot. I tried summarising mean duration by year. However, in the end I finally found a use for…the “joy plot” – or as it seems to have been renamed, the <a rel="nofollow" href="https://cran.r-project.org/web/packages/ggridges/vignettes/introduction.html" target="_blank">ridgeline plot</a>. I still have mixed feelings about it: isn’t it just a density plot with facets? Should it obscure data by default? On the whole though, I think it has some merit for examining distributions over time.</p>
<p>And so back to the chart at the start, showing TWiV episode duration over time. We can see that the average episode in 2017 is around twice the duration of those in 2008 (close to 100 minutes versus 50). The duration distribution has shifted to the right over time but has come back to the left a little in 2017, from a maximum peak in 2016. So perhaps I’ll be able to listen again soon.</p>
<p>If only TWiV were more like TWiM – This Week in Microbiology.</p>
<p><a rel="nofollow" href="https://nsaunders.wordpress.com/2017/10/12/xml-parsing-made-easy-is-that-podcast-getting-longer/twim-joy2-1/" target="_blank"><img data-orig-file="https://nsaunders.files.wordpress.com/2017/10/twim-joy2-1.png?w=456" data-orig-size="672,480" data-medium-file="https://nsaunders.files.wordpress.com/2017/10/twim-joy2-1.png?w=456?w=300" data-attachment-id="4969" data-image-meta="{" data-image-description="" src="https://nsaunders.files.wordpress.com/2017/10/twim-joy2-1.png?w=456" data-permalink="https://nsaunders.wordpress.com/2017/10/12/xml-parsing-made-easy-is-that-podcast-getting-longer/twim-joy2-1/" data-large-file="https://nsaunders.files.wordpress.com/2017/10/twim-joy2-1.png?w=456?w=672" alt="" data-image-title="twim-joy2-1" sizes="(max-width: 672px) 100vw, 672px" data-comments-opened="1" class="wp-image-4969" data-recalc-dims="1"></a></p>
<p>TWiM rarely strays beyond 90 minutes and this year, forms a nice round peak right about the one hour mark. In my opinion, a sensible maximum for a podcast.</p>
<p>Filed under: <a rel="nofollow" href="https://nsaunders.wordpress.com/category/statistics/r/" target="_blank">R</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/category/statistics/" target="_blank">statistics</a> Tagged: <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/parsing/" target="_blank">parsing</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/podcast/" target="_blank">podcast</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/rss/" target="_blank">rss</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/twiv/" target="_blank">twiv</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/xml/" target="_blank">xml</a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/nsaunders.wordpress.com/4965/" target="_blank"><img border="0" alt="" src="http://feeds.wordpress.com/1.0/comments/nsaunders.wordpress.com/4965/"></a> <img border="0" alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" data-recalc-dims="1"></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://nsaunders.wordpress.com/2017/10/12/xml-parsing-made-easy-is-that-podcast-getting-longer/"> R – What You're Doing Is Rather Desperate</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/ctBQFmylDqQ" height="1"></a></div>