<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/removing-personal-bias-from-flu-severity-estimation-a-k-a-misery-loves-data/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rud.is/b/2017/01/02/removing-personal-bias-from-flu-severity-estimation-a-k-a-misery-loves-data/"> R – rud.is</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>The family got hit pretty hard with the flu right as the Christmas festivities started and we were all pretty much bed-ridden zombies up until today (2017-01-02). When in the throes of a very bad ILI it’s easy to imagine that you’re a victim of a severe outbreak, especially with ancillary data from others that they, too, either just had/have the flu or know others who do. Thankfully, I didn’t have to accept this emotional opine and could turn to the <code>cdcfluview</code> package to see just how this year is measuring up.</p>
<p>Influenza cases are cyclical, and that’s super-easy to see with a longer-view of the CDC national data:</p>
<pre><code>library(cdcfluview)
library(tidyverse)
library(stringi)

flu &lt;- get_flu_data(&quot;national&quot;, years=2010:2016)

mutate(flu, week=as.Date(sprintf(&quot;%s %s 1&quot;, YEAR, WEEK), format=&quot;%Y %U %u&quot;)) %>% 
  select(-`AGE 25-64`) %>% 
  gather(age_group, count, starts_with(&quot;AGE&quot;)) %>% 
  mutate(age_group=stri_replace_all_fixed(age_group, &quot;AGE&quot;, &quot;Ages&quot;)) %>% 
  mutate(age_group=factor(age_group, levels=c(&quot;Ages 0-4&quot;, &quot;Ages 5-24&quot;, &quot;Ages 25-49&quot;, &quot;Ages 50-64&quot;, &quot;Ages 65&quot;))) %>%
  ggplot(aes(week, count, group=age_group)) +
  geom_line(aes(color=age_group)) +
  scale_y_continuous(label=scales::comma, limits=c(0,20000)) +
  facet_wrap(~age_group, scales=&quot;free&quot;) +
  labs(x=NULL, y=&quot;Count of reported ILI cases&quot;, 
       title=&quot;U.S. National ILI Case Counts by Age Group (2010:2011 flu season through 2016:2017)&quot;,
       caption=&quot;Source: CDC ILInet via CRAN cdcfluview pacakge&quot;) +
  ggthemes::scale_color_tableau(name=NULL) +
  hrbrmisc::theme_hrbrmstr(grid=&quot;XY&quot;) +
  theme(legend.position=&quot;none&quot;)</code></pre>
<p><a target="_blank" rel="nofollow" href="https://i2.wp.com/rud.is/b/wp-content/uploads/2017/01/Cursor_and___Development_cdcfluview_-_master_-_RStudio.png?ssl=1"><img alt="" class="wp-image-4807" src="https://i2.wp.com/rud.is/b/wp-content/uploads/2017/01/Cursor_and___Development_cdcfluview_-_master_-_RStudio.png?resize=456%2C232&ssl=1" data-recalc-dims="1"></a></p>
<p>We can use the same data to zoom in on this season:</p>
<pre><code>mutate(flu, week=as.Date(sprintf(&quot;%s %s 1&quot;, YEAR, WEEK), format=&quot;%Y %U %u&quot;)) %>% 
  select(-`AGE 25-64`) %>% 
  gather(age_group, count, starts_with(&quot;AGE&quot;)) %>% 
  mutate(age_group=stri_replace_all_fixed(age_group, &quot;AGE&quot;, &quot;Ages&quot;)) %>% 
  mutate(age_group=factor(age_group, levels=c(&quot;Ages 0-4&quot;, &quot;Ages 5-24&quot;, &quot;Ages 25-49&quot;, &quot;Ages 50-64&quot;, &quot;Ages 65&quot;))) %>%
  filter(week >= as.Date(&quot;2016-07-01&quot;)) %>% 
  ggplot(aes(week, count, group=age_group)) +
  geom_line(aes(color=age_group)) +
  scale_y_continuous(label=scales::comma, limits=c(0,20000)) +
  facet_wrap(~age_group, scales=&quot;free&quot;) +
  labs(x=NULL, y=&quot;Count of reported ILI cases&quot;, 
       title=&quot;U.S. National ILI Case Counts by Age Group (2016:2017 flu season)&quot;,
       caption=&quot;Source: CDC ILInet via CRAN cdcfluview pacakge&quot;) +
  ggthemes::scale_color_tableau(name=NULL) +
  hrbrmisc::theme_hrbrmstr(grid=&quot;XY&quot;) +
  theme(legend.position=&quot;none&quot;)</code></pre>
<p><a target="_blank" rel="nofollow" href="https://i0.wp.com/rud.is/b/wp-content/uploads/2017/01/Cursor_and___Development_cdcfluview_-_master_-_RStudio-2.png?ssl=1"><img alt="" class="wp-image-4806" src="https://i0.wp.com/rud.is/b/wp-content/uploads/2017/01/Cursor_and___Development_cdcfluview_-_master_-_RStudio-2.png?resize=456%2C233&ssl=1" data-recalc-dims="1"></a></p>
<p>So, things are trending up, but how severe is this year compared to others? While looking at the number/percentage of ILI cases is one way to understand severity, another is to look at the mortality rate. The <code>cdcfluview</code> package has a <code>get_mortality_surveillance_data()</code> function, but it’s region-based and I’m really only looking at national data in this post. A helpful individual pointed out a new CSV file at <a target="_blank" rel="nofollow" href="https://www.cdc.gov/flu/weekly/index.htm#MS">https://www.cdc.gov/flu/weekly/index.htm#MS</a> which we can reproducibly programmatically target (so we don’t have to track filename changes by hand) with:</p>
<pre><code>library(rvest)

pg &lt;- read_html(&quot;https://www.cdc.gov/flu/weekly/index.htm#MS&quot;)
html_nodes(pg, xpath=&quot;.//a[contains(@href, 'csv') and contains(@href, 'NCHS')]&quot;) %>% 
  html_attr(&quot;href&quot;) -> mort_ref
mort_url &lt;- sprintf(&quot;https://www.cdc.gov%s&quot;, mort_ref)

df &lt;- readr::read_csv(mort_url)</code></pre>
<p>We can, then, take a look at the current “outbreak” status (when real-world mortality events exceed the model threshold):</p>
<pre><code>mutate(df, week=as.Date(sprintf(&quot;%s %s 1&quot;, Year, Week), format=&quot;%Y %U %u&quot;)) %>% 
  select(week, Expected, Threshold, `Percent of Deaths Due to Pneumonia and Influenza`) %>% 
  gather(category, percent, -week) %>% 
  mutate(percent=percent/100) %>% 
  ggplot() +
  geom_line(aes(week, percent, group=category, color=category)) +
  scale_x_date(date_labels=&quot;%Y-%U&quot;) +
  scale_y_continuous(label=scales::percent) +
  ggthemes::scale_color_tableau(name=NULL) +
  labs(x=NULL, y=NULL, title=&quot;U.S. Pneumonia &amp; Influenza Mortality&quot;,
       subtitle=&quot;Data through week ending December 10, 2016 as of December 28, 2016&quot;,
       caption=&quot;Source: National Center for Health Statistics Mortality Surveillance System&quot;) +
  hrbrmisc::theme_hrbrmstr(grid=&quot;XY&quot;) +
  theme(legend.position=&quot;bottom&quot;)</code></pre>
<p><a target="_blank" rel="nofollow" href="https://i0.wp.com/rud.is/b/wp-content/uploads/2017/01/Cursor_and___Development_cdcfluview_-_master_-_RStudio-1.png?ssl=1"><img alt="" class="wp-image-4808" src="https://i0.wp.com/rud.is/b/wp-content/uploads/2017/01/Cursor_and___Development_cdcfluview_-_master_-_RStudio-1.png?resize=456%2C232&ssl=1" data-recalc-dims="1"></a></p>
<p>That view is for all mortality events from both influenza and pneumonia. We can look at the counts for just influenza as well:</p>
<pre><code>mutate(df, week=as.Date(sprintf(&quot;%s %s 1&quot;, Year, Week), format=&quot;%Y %U %u&quot;)) %>% 
  select(week, `Influenza Deaths`) %>% 
  ggplot() +
  geom_line(aes(week, `Influenza Deaths`), color=ggthemes::tableau_color_pal()(1)) +
  scale_x_date(date_labels=&quot;%Y-%U&quot;) +
  scale_y_continuous(label=scales::comma) +
  ggthemes::scale_color_tableau(name=NULL) +
  labs(x=NULL, y=NULL, title=&quot;U.S. Influenza Mortality (count of mortality events)&quot;,
       subtitle=&quot;Data through week ending December 10, 2016 as of December 28, 2016&quot;,
       caption=&quot;Source: National Center for Health Statistics Mortality Surveillance System&quot;) +
  hrbrmisc::theme_hrbrmstr(grid=&quot;XY&quot;) +
  theme(legend.position=&quot;bottom&quot;)</code></pre>
<p><a target="_blank" rel="nofollow" href="https://i1.wp.com/rud.is/b/wp-content/uploads/2017/01/Cursor_and___Development_cdcfluview_-_master_-_RStudio-3.png?ssl=1"><img alt="" class="wp-image-4816" src="https://i1.wp.com/rud.is/b/wp-content/uploads/2017/01/Cursor_and___Development_cdcfluview_-_master_-_RStudio-3.png?resize=456%2C131&ssl=1" data-recalc-dims="1"></a></p>
<p>It’s encouraging that the overall combined mortality rate is trending downwards and that the mortality rate for influenza is very low. Go. Science.</p>
<p>I’ll be adding a function to <code>cdcfluview</code> to retrieve this new data set a bit later in the year.</p>
<p>Hopefully you’ll avoid the flu and enjoy a healthy and prosperous 2017.</p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://rud.is/b/2017/01/02/removing-personal-bias-from-flu-severity-estimation-a-k-a-misery-loves-data/"> R – rud.is</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/e44C52cI-lM"></a></div>