I love working with R and have been sharing the love with my friends and colleagues for almost seven years now. I’m one of those really annoying people whose response to most analysis-related questions is “You can do that in R! Five minutes, tops!” or “Three lines of code, I swear!” The problem was that I invariably spent an hour or more showing people how to get the data, load the data, clean the data, transform the data, and join the data, before we could even start the “five minute analysis”. With the advent of tidyverse, data manipulation has gotten much, much easier, but I still find that data manipulation is where most new users get stuck. This is one of the reasons why, when I designed weathercan, I tried as hard as possible to make it simple and straightforward.
weathercan is an R package designed to make it easy to access historical weather data from Environment and Climate Change Canada (ECCC). It downloads, combines, cleans, and transforms the data from multiple weather stations and across long time frames. So when you access ECCC data, you get everything in one dataset. Nifty, eh?
Although downloading data with weathercan is fairly straight forward, weather data often needs to be integrated into other data sets. You may want to combine weathercan data with other types of measurements (e.g., river water samples on a specific day), or summarize and join it with data on other scales (e.g. temporal or spatial). Depending on the other data this can be a tricky step. That’s why I’m going to walk you through some different ways of integrating weather data from weathercan with other data sets.
We’ll also be using several other R packages to do this, so why don’t we load them right now:
# Data manipulation and plotting
library(dplyr)
library(ggplot2)
# Checking data completeness
library(naniar)
# Access to data containing feeder visits by birds
library(feedr)
# Spatial analyses
library(sf)
library(mapview)
Well, I’ve told you it’s easy to get data from weathercan, so let’s start by doing so. For example, if you wanted to download weather data for all of Manitoba, Canada since the New Year, you have only three steps:
1) Load the package:
library(weathercan)
2) Look at the built in stations data set to find the specific stations you’re interested in (you can also use the stations_search() function). Here, we’ll use the dplyr package (part of tidyverse) to filter() stations to only those in the province of Manitoba, which record data at daily intervals, and which have an end date of 2018 or later (which likely means it’s still operational at the date of writing this post). Note that we’ll also be removing some columns (prov, climate_id, WMO_id, TC_id) just for clarity.
mb = 2018) %__%
select(-prov, -climate_id, -WMO_id, -TC_id)
mb
## # A tibble: 70 x 8
## station_name station_id lat lon elev interval start end
## ## 1 BALDUR 3463 49.3 - 99.3 450 day 1962 2018
## 2 BRANDON A 50821 49.9 -100.0 409 day 2012 2018
## 3 BRANDON RCS 49909 49.9 -100.0 409 day 2012 2018
## 4 CARBERRY CS 27741 49.9 - 99.4 384 day 1994 2018
## 5 CYPRESS RIVER RCS 48128 49.6 - 99.1 374 day 2009 2018
## 6 ELKHORN 2 EAST 3460 49.9 -101 498 day 1987 2018
## 7 PORTAGE ROMANCE 45987 50.0 - 98.3 262 day 2007 2018
## 8 PORTAGE LA PRAIRIE … 3519 50.0 - 98.3 259 day 1970 2018
## 9 PORTAGE SOUTHPORT 10884 49.9 - 98.3 273 day 1992 2018
## 10 ROBLIN 27119 51.2 -101 540 day 1996 2018
## # ... with 60 more rows
3) Download all the data from the start of the year for these stations
mb_weather_all %
filter(end __= 2018)
siteA
## # A tibble: 3 x 9
## station_name station_id lat lon elev interval start end distance
## ## 1 WINNIPEG CHA… 43185 49.8 -97.3 238 day 2004 2018 21.1
## 2 WINNIPEG THE… 28051 49.9 -97.1 230 day 1999 2018 22.5
## 3 KLEEFELD (MA… 50897 49.5 -96.9 246 day 2014 2018 24.4
Now, site B:
siteB %
select(-prov, -climate_id, -WMO_id, -TC_id) %__%
filter(end __= 2018)
siteB
## # A tibble: 2 x 9
## station_name station_id lat lon elev interval start end distance
## ## 1 EMERSON AUTO 48068 49.0 -97.2 242 day 2009 2018 30.4
## 2 GRETNA (AUT) 3605 49.0 -97.6 253 day 1885 2018 31.4
We have a selection of stations for each site that are all about the same distance away. Before we choose any we should make sure they have the data we’re interested in.
Above, we already downloaded all the data for Manitoba in this date range (mb_weather), so we can select() the types of data we want and then filter() to the specific station_ids we’re interested in. We can then use the naniar package to easily visualize any missing data from these stations:
# Limit ourselves to temperature and precipitation
mb_weather %
filter(station_id %in% siteA$station_id) %__%
gg_miss_var(show_pct = TRUE, facet = station_id) + labs(title = &quot;Site A: Stations __ 25 km&quot;)
# Stations near site B (__ 35 km)
mb_weather %__%
filter(station_id %in% siteB$station_id) %__%
gg_miss_var(show_pct = TRUE, facet = station_id) + labs(title = &quot;Site B: Stations __ 35 km&quot;)
Except for station 43185, no station has much snow data. However, while 43185 has snow, it doesn’t have any temperature data. So, for site A, unless we’re willing to lose all temperature data, it definitely looks like station 28051 has the most complete data (temperature and precipitation, at least). Neither station at site B has snow data, but both have full temperature data sets, so we can use either station.
Note: Factors other than distance may also play a role in deciding on a station, such as habitat type, elevation, etc. Depending on what you hope to achieve, you may want to consider these when choosing a station.
Now that we’ve decided on the appropriate stations, we need to merge weathercan data from these stations into our stream data.
We’ll start by creating an index data frame. This will hold the information of which site corresponds to which station:
index