What is rrricanes
Why Write rrricanes?
There is a tremendous amount of weather data available on the internet. Much of it is in raw format and not very easy to obtain. Hurricane data is no different. When one thinks of this data they may be inclined to think it is a bunch of map coordinates with some wind values and not much else. A deeper look will reveal structural and forecast data. An even deeper look will find millions of data points from hurricane reconnaissance, computer forecast models, ship and buoy observations, satellite and radar imagery, ...
rrricanes is an attempt to bring this data together in a way that doesn't just benefit R users, but other languages as well.
I began learning R in 2015 and immediately had wished I had a hurricane-specific dataset when Hurricane Patricia became a harmless, but historic hurricane roaming the Pacific waters. I found this idea revisited again as Hurricane Matthew took aim at Florida and the southeast in 2016. Unable to use R to study and consolidate Matthew's data in R led me to begin learning package development. Thus, the birth of rrricanes.
In this article, I will take you on a lengthy tour of the most important features of rrricanes and what the data means. If you have a background working with hurricane data, most of this will be redundant. My aim here is to cover the big ideas behind the package and explain them under the assumption you, the reader, are unfamiliar with the data offered.
rrricanes is not intended to be used in emergency situations. I write this article as areas I have lived or currently live are under the gun from Hurricane Harvey and rrricanes is unable to obtain data due to external issues (I will describe these later). It is designed with the intent of answering questions and exploring ideas outside of a time-sensitive environment.
rrricanes will not be available in CRAN for quite some time. The current schedule is May 15, 2018 (the &quot;start&quot; of the East Pacific hurricane season). This year is soley for testing under real-time conditions.
And rrricanesdata
The NHC archives text products dating back to at least 1998 (some earlier years exist but yet to be implemented in this package). Accessing this data is a time-consuming process on any computer. A limit of 4 requests per second is put in place to avoid being banned (or restricted) from the archives. So, if a hurricane has 20 text products you wish to pull and parse, this will take 5 seconds. Most cyclones have more and some, far more.
rrricanesdata is a compliment package to rrricanes. rrricanesdata contains post-scraped datasets of the archives for all available storms with the exception of advisories issued in the current month.This means you can explore the various datasets without the wait.
rrricanesdata will be updated monthly if an advisory has been issued the previous month. There will be regular monthly updates approximately from May through November - the typical hurricane season. In some cases, a cyclone may develop in the off-season. rrricanesdata will be updated on the same schedule.
ELI5 the Data
This package covers tropical cyclones that have developed in the Atlantic basin (north Atlantic ocean) or East Pacific basin (northeast Pacific east of 140#°W). Central Pacific (140#°W - 180#°W) may be mixed in if listed in the NHC archives.
While traditionally the hurricane season for each basin runs from mid-May or June through November, some cyclones have developed outside of this time frame.
Every tropical cylone (any tropical low whether classified as a tropical depression, tropical storm or hurricane) contains a core set of text products officially issued from the National Hurricane Center. These products are issued every six hours.
Much of this data has changed in format over the years. Some products have been discontinued and replaced by new products or wrapped into existing products. Some of these products are returned in raw text format; it is not cleaned and may contain HTML characters. Other products are parsed with every piece of data extracted and cleaned.
I have done my best to ensure data is high quality. But, I cannot guarantee it is perfect. If you do believe you have found an error, please let me know; even if it seems small. I would rather be notified of a false error than ignore a true one.
The Products
Each advisory product is listed below with an abbreviation in parentheses. Unless otherwise noted, these products are issued every six hours. Generally, the times issued are 03:00, 09:00, 15:00 and 21:00 UTC. Some products may be issued in three-hour increments and, sometimes, two-hour increments. update can be issued at any time.
Storm Discussion (discus) - These are technical discussions centered on the current structure of the cyclone, satellite presentation, computer forecast model tendencies and more. These products are not parsed.
Forecast/Adivsory (fstadv) - This data-rich product lists the current location of the cyclone, its wind structure, forecast and forecast wind structure.
Public Advisory (public) - These are general text statements issued for the public-at-large. Information in these products is a summary of the Forecast/Advisory product along with any watches and warnings issued, changed, or cancelled. Public Advisory products are the only regularly-scheduled product that may be issued intermittently (every three hours and, occasionally, every two hours) when watches and warnings are in effect. These products are not parsed.
Wind Speed Probabilities (wndprb) - These products list the probability of a minimum sustained wind speed expected in a given forecast window. This product replaces the Strike Probabilities product beginning in 2006 (see below).
Updates (update) - Tropical Cyclone Updates may be issued at any time if a storm is an immediate threat to land or if the cyclone undergoes a significant change of strength or structure. The information in this product is general. These products are not parsed.
Discontinued Products
Strike Probabilities (prblty) - List the probability of a tropical cyclone passing within 65 nautical miles of a location within a forecast window. Replaced in 2006 by the Wind Speed Probabilities product.
Position Estimates (posest) - Typically issued as a storm is threatening land but generally rare (see Hurricane Ike 2008, Key AL092008). It is generally just an update of the current location of the cyclone. After the 2011 hurricane season, this product was discontinued; Updates are now issued in their place. These products are not parsed.
Primary Key
Every cyclone has a Key. However, not all products contain this value (prblty, for example). Products issued during and after the 2005 hurricane season contain this variable.
Use Key to tie datasets together. If Key does not exist, you will need to use a combination of Name and Date, depending on your requirements. Keep in mind that, unless a name is retired, names are recycled every seven years. For example, there are multiple cyclones named Katrina but you may want to isolate on Katrina, 2005.
Installation
rrricanes will not be submitted to CRAN until prior to the hurricane season, 2018. It can be installed via github using devtools:
devtools::install_github(&quot;ropensci/rrricanes&quot;, build_vignettes = TRUE)
Optional Supporting Packages
rrricanesdata uses a drat repository to host the large, pre-processed datasets.
install.packages(&quot;rrricanesdata&quot;,
repos = &quot;https://timtrice.github.io/drat/&quot;,
type = &quot;source&quot;)
To use high resolution tracking charts, you may also wish to install the `rnaturalearthhires' package:
install.packages(&quot;rnaturalearthhires&quot;,
repos = &quot;http://packages.ropensci.org&quot;,
type = &quot;source&quot;)
Linux users may also need to install:
libgdal-dev
libproj-dev
libxml2-dev
Get a List of Storms
We start exploring rrricanes by finding a storm (or storms) we wish to analyze. For this, we use get_storms. There are two optional parameters:
years Between 1998 and current year
basins One or both &quot;AL&quot; and &quot;EP&quot;
An empty call to the function will return storms for both the Atlantic and East Pacific basin for the current year.
library(dplyr)
library(rrricanes)
get_storms() %__% print(n = nrow(.))
## # A tibble: 33 x 4
## Year Name Basin
## ## 1 2017 Tropical Storm Arlene AL
## 2 2017 Tropical Storm Bret AL
## 3 2017 Tropical Storm Cindy AL
## 4 2017 Tropical Depression Four AL
## 5 2017 Tropical Storm Don AL
## 6 2017 Tropical Storm Emily AL
## 7 2017 Hurricane Franklin AL
## 8 2017 Hurricane Gert AL
## 9 2017 Hurricane Harvey AL
## 10 2017 Potential Tropical Cyclone Ten AL
## 11 2017 Hurricane Irma AL
## 12 2017 Hurricane Jose AL
## 13 2017 Hurricane Katia AL
## 14 2017 Hurricane Lee AL
## 15 2017 Hurricane Maria AL
## 16 2017 Tropical Storm Adrian EP
## 17 2017 Tropical Storm Beatriz EP
## 18 2017 Tropical Storm Calvin EP
## 19 2017 Hurricane Dora EP
## 20 2017 Hurricane Eugene EP
## 21 2017 Hurricane Fernanda EP
## 22 2017 Tropical Storm Greg EP
## 23 2017 Tropical Depression Eight-E EP
## 24 2017 Hurricane Hilary EP
## 25 2017 Hurricane Irwin EP
## 26 2017 Tropical Depression Eleven-E EP
## 27 2017 Tropical Storm Jova EP
## 28 2017 Hurricane Kenneth EP
## 29 2017 Tropical Storm Lidia EP
## 30 2017 Hurricane Otis EP
## 31 2017 Hurricane Max EP
## 32 2017 Hurricane Norma EP
## 33 2017 Tropical Storm Pilar EP
## # ... with 1 more variables: Link Function get_storms returns four variables:
Year - year of the cyclone.
Name - name of the cyclone.
Basin - basin the cyclone developed (AL for Atlantic, EP for east Pacific).
Link - URL to the cyclone's archive page.
The variables Name and Link are the only variables that could potentially change. For example, you'll notice a Name value of Potential Tropical Cyclone Ten. If this storm became a tropical storm then it would receive a new name and the link to the archive page would change as well.
For this example we will explore Hurricane Harvey.
Text Products
Current Data
Once we have identified the storms we want to retrieve we can begin working on getting the products. In the earlier discussion of the available products, recall I used abbreviations such as discus, fstadv, etc. These are the terms we will use when obtaining data.
The easiest method to getting storm data is the function get_storm_data. This function can take multiple storm archive URLs and return multiple datasets within a list.
ds %
filter(Name == &quot;Hurricane Harvey&quot;) %__%
pull(Link) %__%
get_storm_data(products = c(&quot;discus&quot;, &quot;fstadv&quot;))
This process may take some time (particularly, fstadv products). This is because the NHC website allows no more than 80 connections every 10 seconds. rrricanes processes four links every half second.
rrricanes uses the dplyr progress bar to keep you informed of the status. You can turn this off by setting option dplyr.show_progress to FALSE.
An additional option is rrricanes.working_msg; FALSE by default. This option will show a message for each advisory currently being worked. I primarily added it to help find products causing problems but you may find it useful at some point.
At this point, we have a list - ds - of dataframes. Each dataframe is named after the product.
names(ds)
## [1] &quot;discus&quot; &quot;fstadv&quot;
discus is one of the products that isn't parsed; the full text of the product is returned.
str(ds$discus)
## Classes 'tbl_df', 'tbl' and 'data.frame': 43 obs. of 6 variables:
## $ Status : chr &quot;Potential Tropical Cyclone&quot; &quot;Tropical Storm&quot; &quot;Tropical Storm&quot; &quot;Tropical Storm&quot; ...
## $ Name : chr &quot;Nine&quot; &quot;Harvey&quot; &quot;Harvey&quot; &quot;Harvey&quot; ...
## $ Adv : num 1 2 3 4 5 6 7 8 9 10 ...
## $ Date : POSIXct, format: &quot;2017-08-17 15:00:00&quot; &quot;2017-08-17 21:00:00&quot; ...
## $ Key : chr &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; ...
## $ Contents: chr &quot;\nZCZC MIATCDAT4 ALL\nTTAA00 KNHC DDHHMM\n\nPotential Tropical Cyclone Nine Discussion Number 1\nNWS National&quot;| __truncated__ &quot;\nZCZC MIATCDAT4 ALL\nTTAA00 KNHC DDHHMM\n\nTropical Storm Harvey Discussion Number 2\nNWS National Hurricane&quot;| __truncated__ &quot;\nZCZC MIATCDAT4 ALL\nTTAA00 KNHC DDHHMM\n\nTropical Storm Harvey Discussion Number 3\nNWS National Hurricane&quot;| __truncated__ &quot;\nZCZC MIATCDAT4 ALL\nTTAA00 KNHC DDHHMM\n\nTropical Storm Harvey Discussion Number 4\nNWS National Hurricane&quot;| __truncated__ ...
The fstadv dataframes, however, are parsed and contain the bulk of the information for the storm.
str(ds$fstadv)
## Classes 'tbl_df', 'tbl' and 'data.frame': 43 obs. of 117 variables:
## $ Status : chr &quot;Potential Tropical Cyclone&quot; &quot;Tropical Storm&quot; &quot;Tropical Storm&quot; &quot;Tropical Storm&quot; ...
## $ Name : chr &quot;Nine&quot; &quot;Harvey&quot; &quot;Harvey&quot; &quot;Harvey&quot; ...
## $ Adv : num 1 2 3 4 5 6 7 8 9 10 ...
## $ Date : POSIXct, format: &quot;2017-08-17 15:00:00&quot; &quot;2017-08-17 21:00:00&quot; ...
## $ Key : chr &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; ...
## $ Lat : num 13.1 13 13 13.1 13.1 13.4 13.7 13.8 13.9 14.1 ...
## $ Lon : num -54.1 -55.8 -57.4 -59.1 -61.3 -62.9 -64.1 -65.9 -68.1 -70 ...
## $ Wind : num 30 35 35 35 35 35 35 35 35 30 ...
## $ Gust : num 40 45 45 45 45 45 45 45 45 40 ...
## $ Pressure : num 1008 1004 1005 1004 1005 ...
## $ PosAcc : num 30 30 30 30 30 40 40 30 30 30 ...
## $ FwdDir : num 270 270 270 270 270 275 275 275 275 275 ...
## $ FwdSpeed : num 15 16 16 16 18 18 16 18 19 19 ...
## $ Eye : num NA NA NA NA NA NA NA NA NA NA ...
## $ SeasNE : num NA 60 60 60 75 150 60 60 45 NA ...
## $ SeasSE : num NA 0 0 0 0 0 0 0 0 NA ...
## $ SeasSW : num NA 0 0 0 0 0 0 0 0 NA ...
## $ SeasNW : num NA 60 60 60 60 60 45 60 45 NA ...
## $ NE64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ SE64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ SW64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ NW64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ NE50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ SE50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ SW50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ NW50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ NE34 : num NA 30 50 50 60 60 60 60 0 NA ...
## $ SE34 : num NA 0 0 0 0 0 0 0 0 NA ...
## $ SW34 : num NA 0 0 0 0 0 0 0 0 NA ...
## $ NW34 : num NA 30 50 50 60 60 60 60 60 NA ...
## $ Hr12FcstDate : POSIXct, format: &quot;2017-08-18 00:00:00&quot; &quot;2017-08-18 06:00:00&quot; ...
## $ Hr12Lat : num 13.1 13.1 13.2 13.2 13.3 13.6 14 14 14.1 14.3 ...
## $ Hr12Lon : num -56.4 -58.3 -59.9 -61.7 -63.8 -65.7 -66.8 -68.7 -70.9 -73 ...
## $ Hr12Wind : num 30 35 35 35 35 35 35 35 35 30 ...
## $ Hr12Gust : num 40 45 45 45 45 45 45 45 45 40 ...
## $ Hr12NE64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr12SE64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr12SW64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr12NW64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr12NE50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr12SE50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr12SW50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr12NW50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr12NE34 : num NA 30 50 50 60 60 60 60 60 NA ...
## $ Hr12SE34 : num NA 0 0 0 0 0 0 0 0 NA ...
## $ Hr12SW34 : num NA 0 0 0 0 0 0 0 0 NA ...
## $ Hr12NW34 : num NA 30 50 50 60 60 60 60 60 NA ...
## $ Hr24FcstDate : POSIXct, format: &quot;2017-08-18 12:00:00&quot; &quot;2017-08-18 18:00:00&quot; ...
## $ Hr24Lat : num 13.2 13.4 13.6 13.5 13.6 13.9 14.3 14.3 14.4 14.6 ...
## $ Hr24Lon : num -59.8 -61.6 -63.3 -65.2 -67.3 -69.3 -70.4 -72.7 -74.9 -77 ...
## $ Hr24Wind : num 35 40 40 40 40 40 40 40 40 35 ...
## $ Hr24Gust : num 45 50 50 50 50 50 50 50 50 45 ...
## $ Hr24NE64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr24SE64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr24SW64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr24NW64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr24NE50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr24SE50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr24SW50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr24NW50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr24NE34 : num 50 40 50 50 60 60 60 60 60 60 ...
## $ Hr24SE34 : num 0 0 0 0 30 0 0 0 0 0 ...
## $ Hr24SW34 : num 0 0 0 0 30 0 0 0 0 0 ...
## $ Hr24NW34 : num 50 40 50 50 60 60 60 60 60 60 ...
## $ Hr36FcstDate : POSIXct, format: &quot;2017-08-19 00:00:00&quot; &quot;2017-08-19 06:00:00&quot; ...
## $ Hr36Lat : num 13.5 13.7 13.9 13.9 14 14.2 14.5 14.6 14.9 15.2 ...
## $ Hr36Lon : num -63.2 -65.1 -67 -68.8 -71.1 -73 -74.3 -76.7 -78.7 -80.5 ...
## $ Hr36Wind : num 40 45 45 40 40 40 40 45 45 40 ...
## $ Hr36Gust : num 50 55 55 50 50 50 50 55 55 50 ...
## $ Hr36NE64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr36SE64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr36SW64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr36NW64 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr36NE50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr36SE50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr36SW50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr36NW50 : num NA NA NA NA NA NA NA NA NA NA ...
## $ Hr36NE34 : num 60 60 60 60 60 60 60 70 70 60 ...
## $ Hr36SE34 : num 0 30 30 30 30 0 0 0 0 0 ...
## $ Hr36SW34 : num 0 30 30 30 30 0 0 0 0 0 ...
## $ Hr36NW34 : num 60 60 60 60 60 60 60 70 70 60 ...
## $ Hr48FcstDate : POSIXct, format: &quot;2017-08-19 12:00:00&quot; &quot;2017-08-19 18:00:00&quot; ...
## $ Hr48Lat : num 13.9 14 14.1 14.1 14.3 14.5 14.8 15.2 15.7 16 ...
## $ Hr48Lon : num -66.7 -68.8 -70.9 -72.7 -75 -76.7 -78.1 -80.1 -82.4 -83.8 ...
## $ Hr48Wind : num 45 45 50 50 50 45 45 50 50 45 ...
## $ Hr48Gust : num 55 55 60 60 60 55 55 60 60 55 ...
## $ Hr48NE50 : num NA NA 30 30 30 NA NA 30 30 NA ...
## $ Hr48SE50 : num NA NA 0 0 0 NA NA 0 0 NA ...
## $ Hr48SW50 : num NA NA 0 0 0 NA NA 0 0 NA ...
## $ Hr48NW50 : num NA NA 30 30 30 NA NA 30 30 NA ...
## $ Hr48NE34 : num 60 60 60 60 70 60 60 90 90 70 ...
## $ Hr48SE34 : num 30 30 30 30 30 30 0 50 50 0 ...
## $ Hr48SW34 : num 30 30 30 30 30 30 0 40 40 0 ...
## $ Hr48NW34 : num 60 60 60 60 70 60 60 70 70 70 ...
## $ Hr72FcstDate : POSIXct, format: &quot;2017-08-20 12:00:00&quot; &quot;2017-08-20 18:00:00&quot; ...
## $ Hr72Lat : num 14.5 14.5 14.8 15 15 15.5 16.5 17 17.5 18 ...
## $ Hr72Lon : num -74.5 -76.5 -78.6 -80.2 -82 -83.5 -84.7 -86.5 -88 -89 ...
## $ Hr72Wind : num 55 55 60 60 60 60 55 55 60 45 ...
## $ Hr72Gust : num 65 65 75 75 75 75 65 65 75 55 ...
## [list output truncated]
Each product can also be accessed on its own. For example, if you only wish to view discus products, use the get_discus function. fstadv products can be accessed with get_fstadv. Every products specific function is preceeded by get_.
To understand the variable definitions, access the help file for each of these functions (i.e., ?get_fstadv). They contain full definitions on the variables and their purpose.
As you can see, the fstadv dataframe is very wide. There may be instances you only want to focus on specific pieces of the product. I've developed tidy functions to help trim these datasets:
tidy_fcst
tidy_fcst_wr
tidy_fstadv
tidy_wr
These datasets exist in rrricanesdata as fcst, fcst_wr, adv, and wr, respectively (see below).
Most tropical cyclone forecast/advisory products will contain multiple forecast points. Initially, only three-day forecasts were issued. Beginning the with the 2003 season, 96 hour (five-day) forecasts were issued.
If a storm is not expected to survive the full forecast period, then only relevant forecasts will be issued.
We use tidy_fcst to return these forecast points in a tidy fashion from fstadv.
str(tidy_fcst(ds$fstadv))
## Classes 'tbl_df', 'tbl' and 'data.frame': 283 obs. of 8 variables:
## $ Key : chr &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; ...
## $ Adv : num 1 1 1 1 1 1 1 2 2 2 ...
## $ Date : POSIXct, format: &quot;2017-08-17 15:00:00&quot; &quot;2017-08-17 15:00:00&quot; ...
## $ FcstDate: POSIXct, format: &quot;2017-08-18 00:00:00&quot; &quot;2017-08-18 12:00:00&quot; ...
## $ Lat : num 13.1 13.2 13.5 13.9 14.5 15.5 17 13.1 13.4 13.7 ...
## $ Lon : num -56.4 -59.8 -63.2 -66.7 -74.5 -82 -87.5 -58.3 -61.6 -65.1 ...
## $ Wind : num 30 35 40 45 55 65 65 35 40 45 ...
## $ Gust : num 40 45 50 55 65 80 80 45 50 55 ...
Wind radius values are issued with parameters of 34, 50 and 64. These values are the radius to which minimum one-minute sustained winds can be expected or exist.
A tropical depression will not have associated wind radius values since the maximum winds of a depression are 30 knots. If a tropical storm has winds less than 50 knots, then it will only have wind radius values for the 34-knot wind field. If winds are greater than 50 knots, then it will have wind radius values for 34 and 50 knot winds. A hurricane will have all wind radius fields.
Wind radius values are further seperated by quadrant; NE (northeast), SE, SW and NW. Not all quadrants will have values; particularly if the cyclone is struggling to organize. For example, you will often find a minimal hurricane only has hurricane-force winds (64 knots) in the northeast quadrant.
When appropriate, a forecast/advisory product will contain these values for the current position and for each forecast position. Use tidy_wr and tidy_fcst_wr, respectively, for these variables.
str(tidy_wr(ds$fstadv))
## Classes 'tbl_df', 'tbl' and 'data.frame': 56 obs. of 8 variables:
## $ Key : chr &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; ...
## $ Adv : num 2 3 4 5 6 7 8 9 15 16 ...
## $ Date : POSIXct, format: &quot;2017-08-17 21:00:00&quot; &quot;2017-08-18 03:00:00&quot; ...
## $ WindField: num 34 34 34 34 34 34 34 34 34 34 ...
## $ NE : num 30 50 50 60 60 60 60 0 100 80 ...
## $ SE : num 0 0 0 0 0 0 0 0 0 30 ...
## $ SW : num 0 0 0 0 0 0 0 0 0 20 ...
## $ NW : num 30 50 50 60 60 60 60 60 50 50 ...
str(tidy_fcst_wr(ds$fstadv))
## Classes 'tbl_df', 'tbl' and 'data.frame': 246 obs. of 9 variables:
## $ Key : chr &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; ...
## $ Adv : num 1 1 1 1 1 2 2 2 2 2 ...
## $ Date : POSIXct, format: &quot;2017-08-17 15:00:00&quot; &quot;2017-08-17 15:00:00&quot; ...
## $ FcstDate : POSIXct, format: &quot;2017-08-18 12:00:00&quot; &quot;2017-08-19 00:00:00&quot; ...
## $ WindField: num 34 34 34 34 50 34 34 34 34 34 ...
## $ NE : num 50 60 60 80 30 30 40 60 60 80 ...
## $ SE : num 0 0 30 40 0 0 0 30 30 40 ...
## $ SW : num 0 0 30 40 0 0 0 30 30 40 ...
## $ NW : num 50 60 60 80 30 30 40 60 60 80 ...
Lastly, you may only want to focus on current storm details. For this, we use tidy_fstadv:
str(tidy_fstadv(ds$fstadv))
## Classes 'tbl_df', 'tbl' and 'data.frame': 43 obs. of 18 variables:
## $ Key : chr &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; &quot;AL092017&quot; ...
## $ Adv : num 1 2 3 4 5 6 7 8 9 10 ...
## $ Date : POSIXct, format: &quot;2017-08-17 15:00:00&quot; &quot;2017-08-17 21:00:00&quot; ...
## $ Status : chr &quot;Potential Tropical Cyclone&quot; &quot;Tropical Storm&quot; &quot;Tropical Storm&quot; &quot;Tropical Storm&quot; ...
## $ Name : chr &quot;Nine&quot; &quot;Harvey&quot; &quot;Harvey&quot; &quot;Harvey&quot; ...
## $ Lat : num 13.1 13 13 13.1 13.1 13.4 13.7 13.8 13.9 14.1 ...
## $ Lon : num -54.1 -55.8 -57.4 -59.1 -61.3 -62.9 -64.1 -65.9 -68.1 -70 ...
## $ Wind : num 30 35 35 35 35 35 35 35 35 30 ...
## $ Gust : num 40 45 45 45 45 45 45 45 45 40 ...
## $ Pressure: num 1008 1004 1005 1004 1005 ...
## $ PosAcc : num 30 30 30 30 30 40 40 30 30 30 ...
## $ FwdDir : num 270 270 270 270 270 275 275 275 275 275 ...
## $ FwdSpeed: num 15 16 16 16 18 18 16 18 19 19 ...
## $ Eye : num NA NA NA NA NA NA NA NA NA NA ...
## $ SeasNE : num NA 60 60 60 75 150 60 60 45 NA ...
## $ SeasSE : num NA 0 0 0 0 0 0 0 0 NA ...
## $ SeasSW : num NA 0 0 0 0 0 0 0 0 NA ...
## $ SeasNW : num NA 60 60 60 60 60 45 60 45 NA ...
In release 0.2.1, tidy_fstadv will be renamed to tidy_adv.
One final note on the data: all speed variables are measured in knots, distance variables in nautical miles, and pressure variables in millibars. Functions knots_to_mph and mb_to_in are available for speed/pressure conversions. Function nm_to_sm to convert nautical miles to survey miles will be included in release 0.2.1.
Archived Data
rrricanesdata was built to make it easier to get pre-processed datasets. As mentioned earlier, rrricanesdata will be updated the first of every month if any advisory was issued for the previous month. (As I am now writing this portion in September, all of Hurricane Harvey's advisories - the last one issued the morning of August 31 - exist in rrricanesdata release 0.0.1.4.)
As with rrricanes, rrricanesdata is not available in CRAN (nor will be due to size limitations).
I'll load all datasets with the call:
library(rrricanesdata)
data(list = data(package = &quot;rrricanesdata&quot;)$results[,3])
All core product datasets are available. The dataframes adv, fcst, fcst_wr and wr are the dataframes created by tidy_fstadv, tidy_fcst, tidy_fcst_wr and tidy_wr, respectively.
Tracking Charts
rrricanes also comes with helper functions to quickly generate tracking charts. These charts use rnaturalearthdata (for high resolution maps, use package rnaturalearthhires). These charts are not required - Bob Rudis demonstrates demonstrates succintly - so feel free to experiment.
You can generate a default plot for the entire globe with tracking_chart:
tracking_chart()
You may find this handy when examining cyclones that cross basins (from the Atlantic to east Pacific such as Hurricane Otto, 2016).
tracking_chart takes three parameters (in addition to dots for other ggplot calls):
countries - By default, show country borders
states - By default, show state borders
res - resolution; default is 110nm.
We do not see countries and states in the map above because of the ggplot defaults. Let's try it again:
tracking_chart(color = &quot;black&quot;, size = 0.1, fill = &quot;white&quot;)
We can &quot;zoom in&quot; on each basin with helper functions al_tracking_chart and ep_tracking_chart:
al_tracking_chart(color = &quot;black&quot;, size = 0.1, fill = &quot;white&quot;)
ep_tracking_chart(color = &quot;black&quot;, size = 0.1, fill = &quot;white&quot;)
GIS Data
GIS data exists for some cyclones and varies by year. This is a relatively new archive by the NHC and is inconsistent from storm to storm.
The &quot;gis&quot; functions are as follows:
gis_advisory
gis_latest
gis_prob_storm_surge
gis_windfield
gis_wsp
Another area of inconsistency with these products is how they are organized. For example, gis_advisory, gis_prob_storm_surge and gis_windfield can be retrieved with a storm Key (unique identifier for every cyclone; see fstadv$Key). Except for gis_prob_storm_surge, you can even pass an advisory number (see fstadv$Adv).
gis_wsp requires a datetime value; to access a specific GIS package for a storm's advisory you would need to use a variable such as fstadv$Date, subtract three hours and convert to &quot;%Y%m%d%H&quot; format (&quot;%m&quot;, &quot;%d&quot;, and &quot;%H&quot; are optional).
All above functions only return URL's to their respective datasets. This was done to allow you to validate the quantity of datasets you wish to retrieve as, in some cases, the dataset may not exist at all or there may be several available. Use gis_download with the requested URL's to retrieve your datasets.
Let's go through each of these. First, let's get the Key of Hurricane Harvey:
# Remember that ds already and only contains data for Hurricane Harvey
key % pull(Key) %__% first()
gis_advisory
gis_advisory returns a dataset package containing past and forecast plot points and lines, a forecast cone (area representing where the cyclone could track), wind radius data and current watches and warnings.
gis_advisory takes two parameters:
Key
advisory (optional)
If we leave out advisory we get all related datasets for Hurricane Harvey:
x %
filter(FCSTPRD == 72),
aes(x = long, y = lat, color = FCSTPRD))
Nothing there!
As mentioned earlier, these are experimental products issued by the NHC and they do contain inconsistencies. To demonstrate, I'll use Hurricane Ike advisory 42.
df %
gis_download()
## OGR data source with driver: ESRI Shapefile
## Source: &quot;/var/folders/gs/4khph0xs0436gmd2gdnwsg080000gn/T//Rtmp1wYg2x&quot;, layer: &quot;al092008.042_5day_lin&quot;
## with 2 features
## It has 9 fields
## OGR data source with driver: ESRI Shapefile
## Source: &quot;/var/folders/gs/4khph0xs0436gmd2gdnwsg080000gn/T//Rtmp1wYg2x&quot;, layer: &quot;al092008.042_5day_pgn&quot;
## with 2 features
## It has 9 fields
## OGR data source with driver: ESRI Shapefile
## Source: &quot;/var/folders/gs/4khph0xs0436gmd2gdnwsg080000gn/T//Rtmp1wYg2x&quot;, layer: &quot;al092008.042_5day_pts&quot;
## with 13 features
## It has 20 fields
## OGR data source with driver: ESRI Shapefile
## Source: &quot;/var/folders/gs/4khph0xs0436gmd2gdnwsg080000gn/T//Rtmp1wYg2x&quot;, layer: &quot;al092008.042_ww_wwlin&quot;
## with 5 features
## It has 10 fields
al_tracking_chart(color = &quot;black&quot;, size = 0.1, fill = &quot;white&quot;) +
geom_polygon(data = shp_to_df(df$al092008_042_5day_pgn) %__%
filter(FCSTPRD == 72),
aes(x = long, y = lat))
We do, however, have a 120-hour forecast cone for Hurricane Harvey. Let's go ahead and plot that.
bp +
geom_polygon(data = gis$al092017_019_5day_pgn,
aes(x = long, y = lat), alpha = 0.15)
It's an odd-looking forecast cone, for sure. But this demonstrates the entire area that Harvey could have potentially traveled.
Watches and Warnings
Our last dataset in this package is &quot;al092017_09_ww_wlin&quot;. These are the current watches and warnings in effect. This is a spatial lines dataframe that needs shp_to_df. Again, we use geom_path instead of geom_line. And we want to group our paths by the variable TCWW.
bp +
geom_path(data = shp_to_df(gis$al092017_019_ww_wwlin),
aes(x = long, y = lat, group = group, color = TCWW), size = 1)
The paths won't follow our coastlines exactly but you get the idea. The abbreviations don't really give much information, either. Convert TCWW to factor and provide better labels for your legend.
ww_wlin % pull(Date))
## [1] &quot;2017-08-25 03:00:00 UTC&quot;
esurge
Now, let's view all esurge products for date only (exlude time).
gis_prob_storm_surge(key = key,
products = list(&quot;esurge&quot; = seq(10, 50, by = 10)),
datetime = strftime(d, &quot;%Y%m%d&quot;, tz = &quot;UTC&quot;))
## [1] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge10_2017082500.zip&quot;
## [2] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge10_2017082506.zip&quot;
## [3] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge10_2017082512.zip&quot;
## [4] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge10_2017082518.zip&quot;
## [5] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge20_2017082500.zip&quot;
## [6] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge20_2017082506.zip&quot;
## [7] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge20_2017082512.zip&quot;
## [8] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge20_2017082518.zip&quot;
## [9] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge30_2017082500.zip&quot;
## [10] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge30_2017082506.zip&quot;
## [11] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge30_2017082512.zip&quot;
## [12] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge30_2017082518.zip&quot;
## [13] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge40_2017082500.zip&quot;
## [14] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge40_2017082506.zip&quot;
## [15] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge40_2017082512.zip&quot;
## [16] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge40_2017082518.zip&quot;
## [17] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge50_2017082500.zip&quot;
## [18] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge50_2017082506.zip&quot;
## [19] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge50_2017082512.zip&quot;
## [20] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge50_2017082518.zip&quot;
That's still quite a bit. We can filter it to more by adding hour to the datetime parameter.
gis_prob_storm_surge(key = key,
products = list(&quot;esurge&quot; = seq(10, 50, by = 10)),
datetime = strftime(d, &quot;%Y%m%d%H&quot;, tz = &quot;UTC&quot;))
This call will give you an error:
Error: No data available for requested storm/advisory
But, this isn't entirely correct. When an advisory package is issued it contains information for the release time. Some of the GIS datasets are based on the release time -3 hours. So, we need to subtract 3 hours from d.
Note: There is an additional value that, as of the latest release is not extracted, records the position of the cyclone three hours prior. (As I understand it from the NHC, this is due to the time it takes to collect and prepare the data.) Per Issue #102, these values will be added for release 0.2.1. Therefore, instead of subtracting three hours from the Date variable, you can simply use the PrevPosDate value for this function.
Let's try it again with the math:
gis_prob_storm_surge(key = key,
products = list(&quot;esurge&quot; = seq(10, 50, by = 10)),
datetime = strftime(d - 60 * 60 * 3, &quot;%Y%m%d%H&quot;,
tz = &quot;UTC&quot;))
## [1] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge10_2017082500.zip&quot;
## [2] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge20_2017082500.zip&quot;
## [3] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge30_2017082500.zip&quot;
## [4] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge40_2017082500.zip&quot;
## [5] &quot;http://www.nhc.noaa.gov/gis/storm_surge/al092017_esurge50_2017082500.zip&quot;
As I don't want to get all of these datasets, I'll limit my esurge to show surge values with at least a 50% chance of being exceeded:
gis %
gis_download()
## OGR data source with driver: ESRI Shapefile
## Source: &quot;/var/folders/gs/4khph0xs0436gmd2gdnwsg080000gn/T//Rtmp1wYg2x&quot;, layer: &quot;al092017_2017082500_e50&quot;
## with 97 features
## It has 2 fields
This will bring us a spatial polygon dataframe.
df