<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/feels-like-a-dry-winter-but-what-does-the-data-say/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://nsaunders.wordpress.com/2017/10/11/feels-like-a-dry-winter-but-what-does-the-data-say/"> R – What You're Doing Is Rather Desperate</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a rel="nofollow" href="https://nsaunders.wordpress.com/2017/10/11/feels-like-a-dry-winter-but-what-does-the-data-say/2017-09-17-11-21-03/" target="_blank"><img data-orig-file="https://nsaunders.files.wordpress.com/2017/10/2017-09-17-11-21-03.jpg?w=450" data-orig-size="640,427" data-medium-file="https://nsaunders.files.wordpress.com/2017/10/2017-09-17-11-21-03.jpg?w=450?w=300" data-attachment-id="4955" data-image-meta="{" data-image-description="" src="https://nsaunders.files.wordpress.com/2017/10/2017-09-17-11-21-03.jpg?w=450" data-permalink="https://nsaunders.wordpress.com/2017/10/11/feels-like-a-dry-winter-but-what-does-the-data-say/2017-09-17-11-21-03/" data-large-file="https://nsaunders.files.wordpress.com/2017/10/2017-09-17-11-21-03.jpg?w=450?w=640" alt="" data-image-title="2017-09-17 11.21.03" sizes="(max-width: 480px) 100vw, 480px" data-comments-opened="1" class="wp-image-4955" data-recalc-dims="1"></a>A reminder that when idle queries pop into your head, the answer can often be found using R + online data. And a brief excursion into accessing the Weather Underground.</p>
<p>One interesting aspect of Australian life, even in coastal urban areas like Sydney, is that sometimes it just stops raining. For weeks or months at a time. The realisation hits slowly: at some point you look around at the yellow-brown lawns, ovals and “nature strips” and say “gee, I don’t remember the last time it rained.”</p>
<p>Thankfully in our data-rich world, it’s relatively easy to find out whether the dry spell is really as long as it feels. In Australia, meteorological data is readily available via the <a rel="nofollow" href="http://www.bom.gov.au/" target="_blank">Bureau of Meteorology</a> (known as BoM). Another source is the <a rel="nofollow" href="https://www.wunderground.com/" target="_blank">Weather Underground</a> (WU), which has the benefit that there may be data from a personal weather station much closer to you than the BoM stations.</p>
<p>Here’s how you can access WU data using R and see whether your fuzzy recollection is matched by reality.<br>
<span></span></p>
<p>First, you need to create an account at Weather Underground and <a rel="nofollow" href="https://www.wunderground.com/weather/api" target="_blank">sign up for an API key</a>. The free plan gives you 500 calls a day and a maximum rate of 10 calls per minute. I had issues with the confirmation email; you may need to login again after account creation and sign up for the API a second time to get there.</p>
<p>There’s an R package of course, <a rel="nofollow" href="https://github.com/ALShum/rwunderground" target="_blank">rwunderground</a>, which you can install from Github. You can then include the line:</p>
<p><code>WUNDERGROUNDID=myAPIKey</code></p>
<p>in your <code>.Renviron</code> file, replacing with the real value of your key.</p>
<p>The <code>history_daily()</code> function works quite nicely; you give it a location identifier and a date, data summarised by day comes back in a tibble:</p>
<p><code>history_daily(set_location(PWS_id = &quot;INEWSOUT879&quot;), date = &quot;20171010&quot;)</code></p>
<p>You could also use <code>history()</code> or <code>history_range()</code>, which return all observations taken on a date or within a date range, respectively. I figured the latter was the way to go as it automatically takes care of the API rate limit. Unfortunately “in my hands”, much of the data returned (including rainfall) consists of NA values, even though the raw JSON does contain numeric data. I’ve filed <a rel="nofollow" href="https://github.com/ALShum/rwunderground/issues/30" target="_blank">an issue</a> so we’ll see what happens.</p>
<p>In the meantime I did what any self-respecting data scientist does when they require a small dataset precisely once and quickly: ran the query at the website, then copy-paste-edit in a text file.</p>
<p>The rest is pretty straightforward: code and output <a rel="nofollow" href="https://github.com/neilfws/AusWeather/tree/master/wunderground/rainfall2017/code/R" target="_blank">are at Github</a> and the rainfall graph for 2017 to date looks like this.</p>
<p><a rel="nofollow" href="https://nsaunders.wordpress.com/2017/10/11/feels-like-a-dry-winter-but-what-does-the-data-say/plot-data-1/" target="_blank"><img data-orig-file="https://nsaunders.files.wordpress.com/2017/10/plot-data-1.png?w=456" data-orig-size="672,480" data-medium-file="https://nsaunders.files.wordpress.com/2017/10/plot-data-1.png?w=456?w=300" data-attachment-id="4950" data-image-meta="{" data-image-description="" src="https://nsaunders.files.wordpress.com/2017/10/plot-data-1.png?w=456" data-permalink="https://nsaunders.wordpress.com/2017/10/11/feels-like-a-dry-winter-but-what-does-the-data-say/plot-data-1/" data-large-file="https://nsaunders.files.wordpress.com/2017/10/plot-data-1.png?w=456?w=672" alt="" data-image-title="plot-data-1" sizes="(max-width: 672px) 100vw, 672px" data-comments-opened="1" class="wp-image-4950" data-recalc-dims="1"></a></p>
<p>Aside from a heavy shower in August, this weather station has seen barely any significant rainfall events since April. Some notable numbers include: a total of 0.5 mm for September, 35.4 mm in total for May – September and two long periods (27 and 21 days) with no rain at all.</p>
<p>I may not remember the last time it rained, but it’s easy to find out. And this time, my feeling was correct. It’s been one long, dry winter.</p>
<p>Filed under: <a rel="nofollow" href="https://nsaunders.wordpress.com/category/australia/" target="_blank">australia</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/category/statistics/r/" target="_blank">R</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/category/statistics/" target="_blank">statistics</a> Tagged: <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/ggplot2/" target="_blank">ggplot2</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/rainfall/" target="_blank">rainfall</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/sydney/" target="_blank">sydney</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/underground/" target="_blank">underground</a>, <a rel="nofollow" href="https://nsaunders.wordpress.com/tag/weather/" target="_blank">weather</a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/nsaunders.wordpress.com/4949/" target="_blank"><img border="0" alt="" src="http://feeds.wordpress.com/1.0/comments/nsaunders.wordpress.com/4949/"></a> <img border="0" alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" data-recalc-dims="1"></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://nsaunders.wordpress.com/2017/10/11/feels-like-a-dry-winter-but-what-does-the-data-say/"> R – What You're Doing Is Rather Desperate</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/8i8NDut49jU" height="1"></a></div>