<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/deep-learning-for-time-series-forecasting-predicting-sunspot-frequency-with-keras/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://tensorflow.rstudio.com/blog/sunspots-lstm.html"> TensorFlow for R</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<h2>Forecasting sunspots with deep learning</h2>
<p>In this post we will examine making time series predictions using the <a rel="nofollow" href="https://stat.ethz.ch/R-manual/R-devel/library/datasets/html/sunspot.month.html" target="_blank">sunspots</a> dataset that ships with base R. Sunspots are dark spots on the sun, associated with lower temperature. Here’s an image from NASA showing the solar phenomenon.</p>
<figure><img src="https://i0.wp.com/tensorflow.rstudio.com/blog/images/sunspots/sunspot_nasa.jpg?w=50%25&ssl=1" data-recalc-dims="1"><figcaption>
Source: <a rel="nofollow" href="https://www.nasa.gov/content/goddard/largest-sunspot-of-solar-cycle" target="_blank">https://www.nasa.gov/content/goddard/largest-sunspot-of-solar-cycle</a><br>
</figcaption></figure>
<p> </p>
<p>We’re using the monthly version of the dataset, <code>sunspots.month</code> (there is a yearly version, too). It contains 265 years worth of data (from 1749 through 2013) on the number of sunspots per month.</p>
<div>
<img src="https://i2.wp.com/tensorflow.rstudio.com/blog/images/sunspots/sunspots_full.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p>Forecasting this dataset is challenging because of high short term variability as well as long-term irregularities evident in the cycles. For example, maximum amplitudes reached by the low frequency cycle differ a lot, as does the number of high frequency cycle steps needed to reach that maximum low frequency cycle height.</p>
<p>Our post will focus on two dominant aspects: how to apply deep learning to time series forecasting, and how to properly apply cross validation in this domain. For the latter, we will use the <a rel="nofollow" href="https://cran.r-project.org/package=rsample" target="_blank">rsample</a> package that allows to do resampling on time series data. As to the former, our goal is not to reach utmost performance but to show the general course of action when using recurrent neural networks to model this kind of data.</p>
</div>
<div>
<h2>Recurrent neural networks</h2>
<p>When our data has a sequential structure, it is recurrent neural networks (RNNs) we use to model it.</p>
<p>As of today, among RNNs, the best established architectures are the GRU (Gated Recurrent Unit) and the LSTM (Long Short Term Memory). For today, let’s not zoom in on what makes them special, but on what they have in common with the most stripped-down RNN: the basic recurrence structure.</p>
<p>In contrast to the prototype of a neural network, often called Multilayer Perceptron (MLP), the RNN has a state that is carried on over time. This is nicely seen in this diagram from <a rel="nofollow" href="http://www.deeplearningbook.org/" target="_blank">Goodfellow et al.</a>, a.k.a. the “bible of deep learning”:</p>
<div>
<img alt="Source: http://www.deeplearningbook.org" src="https://i1.wp.com/tensorflow.rstudio.com/blog/images/sunspots/rnn.png?w=456&ssl=1" data-recalc-dims="1">
<p>Source: <a rel="nofollow" href="http://www.deeplearningbook.org/" target="_blank">http://www.deeplearningbook.org</a></p>
</div>
<p>At each time, the state is a combination of the current input and the previous hidden state. This is reminiscent of autoregressive models, but with neural networks, there has to be some point where we halt the dependence.</p>
<p>That’s because in order to determine the weights, we keep calculating how our loss changes as the input changes. Now if the input we have to consider, at an arbitrary timestep, ranges back indefinitely – then we will not be able to calculate all those gradients. In practice, then, our hidden state will, at every iteration, be carried forward through a fixed number of steps.</p>
<p>We’ll come back to that as soon as we’ve loaded and pre-processed the data.</p>
</div>
<div>
<h2>Setup, pre-processing, and exploration</h2>
<div>
<h3>Libraries</h3>
<p>Here, first, are the libraries needed for this tutorial.</p>
<div>
<pre><code><span># Core Tidyverse</span>
<span>library</span>(tidyverse)
<span>library</span>(glue)
<span>library</span>(forcats)

<span># Time Series</span>
<span>library</span>(timetk)
<span>library</span>(tidyquant)
<span>library</span>(tibbletime)

<span># Visualization</span>
<span>library</span>(cowplot)

<span># Preprocessing</span>
<span>library</span>(recipes)

<span># Sampling / Accuracy</span>
<span>library</span>(rsample)
<span>library</span>(yardstick) 

<span># Modeling</span>
<span>library</span>(keras)
<span>library</span>(tfruns)</code></pre>
</div>
<p>If you have not previously run Keras in R, you will need to install Keras using the <code><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/install_keras.html" target="_blank">install_keras()</a></code> function.</p>
<div>
<pre><code><span># Install Keras if you have not installed before</span>
<span><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/install_keras.html" target="_blank">install_keras</a></span>()</code></pre>
</div>
</div>
<div>
<h3>Data</h3>
<p><code>sunspot.month</code> is a <code>ts</code> class (not tidy), so we’ll convert to a tidy data set using the <code><a rel="nofollow" href="http://www.rdocumentation.org/packages/timetk/topics/tk_tbl?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">tk_tbl()</a></code> function from <code>timetk</code>. We use this instead of <code>as.tibble()</code> from <code>tibble</code> to automatically preserve the time series index as a <code>zoo</code> <code>yearmon</code> index. Last, we’ll convert the <code>zoo</code> index to date using <code><a rel="nofollow" href="http://lubridate.tidyverse.org/reference/as_date.html" target="_blank">lubridate::as_date()</a></code> (loaded with <code>tidyquant</code>) and then change to a <code>tbl_time</code> object to make time series operations easier.</p>
<div>
<pre><code>sun_spots &lt;-<span> </span>datasets<span>::</span>sunspot.month <span>%>%</span>
<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/timetk/topics/tk_tbl?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">tk_tbl</a></span>() <span>%>%</span>
<span> </span><span>mutate</span>(<span>index =</span> <span>as_date</span>(index)) <span>%>%</span>
<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/tibbletime/topics/tbl_time?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">as_tbl_time</a></span>(<span>index =</span> index)

sun_spots</code></pre>
</div>
<pre><code># A time tibble: 3,177 x 2
# Index: index
   index      value
   </code></pre></div></div>