Deep Learning with R
This post is an excerpt from Chapter 5 of François Chollet’s and J.J. Allaire’s book, Deep Learning with R (Manning Publications). Use the code fccallaire for a 42% discount on the book at manning.com.
Time Series Forecasting with Recurrent Neural Networks
In this section, we’ll review three advanced techniques for improving the performance and generalization power of recurrent neural networks. By the end of the section, you’ll know most of what there is to know about using recurrent networks with Keras. We’ll demonstrate all three concepts on a temperature-forecasting problem, where you have access to a time series of data points coming from sensors installed on the roof of a building, such as temperature, air pressure, and humidity, which you use to predict what the temperature will be 24 hours after the last data point. This is a fairly challenging problem that exemplifies many common difficulties encountered when working with time series.
We’ll cover the following techniques:
Recurrent dropout — This is a specific, built-in way to use dropout to fight overfitting in recurrent layers.
Stacking recurrent layers — This increases the representational power of the network (at the cost of higher computational loads).
Bidirectional recurrent layers — These present the same information to a recurrent network in different ways, increasing accuracy and mitigating forgetting issues.
A temperature-forecasting problem
Until now, the only sequence data we’ve covered has been text data, such as the IMDB dataset and the Reuters dataset. But sequence data is found in many more problems than just language processing. In all the examples in this section, you’ll play with a weather timeseries dataset recorded at the Weather Station at the Max Planck Institute for Biogeochemistry in Jena, Germany.
In this dataset, 14 different quantities (such air temperature, atmospheric pressure, humidity, wind direction, and so on) were recorded every 10 minutes, over several years. The original data goes back to 2003, but this example is limited to data from 2009–2016. This dataset is perfect for learning to work with numerical time series. You’ll use it to build a model that takes as input some data from the recent past (a few days’ worth of data points) and predicts the air temperature 24 hours in the future.
Download and uncompress the data as follows:
dir.create(&quot;~/Downloads/jena_climate&quot;, recursive = TRUE)
download.file(
&quot;https://s3.amazonaws.com/keras-datasets/jena_climate_2009_2016.csv.zip&quot;,
&quot;~/Downloads/jena_climate/jena_climate_2009_2016.csv.zip&quot;
)
unzip(
&quot;~/Downloads/jena_climate/jena_climate_2009_2016.csv.zip&quot;,
exdir = &quot;~/Downloads/jena_climate&quot;
)
Let’s look at the data.
library(tibble)
library(readr)
data_dir