<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/smooth-package-for-r-common-ground-part-i-prediction-intervals/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://forecasting.svetunkov.ru/en/2017/06/11/smooth-package-for-r-prediction-intervals/"> R – Modern Forecasting</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>We have spent previous six posts discussing basics of
<pre>es()</pre>
<p> function (underlying models and their implementation). Now it is time to move forward. Starting from this post we will discuss common parameters, shared by all the forecasting functions implemented in smooth. This means that the topics that we discuss are not only applicable to
<pre>es()</pre>
<p>, but also to
<pre>ssarima()</pre>
<p>,
<pre>ces()</pre>
<p>,
<pre>ges()</pre>
<p> and
<pre>sma()</pre>
<p>. However, taking that we have only discussed ETS so far, we will use
<pre>es()</pre>
<p> in our examples for now.</p>
<p>And I would like to start this series of general posts from the topic of prediction intervals.</p>
<h3>Prediction intervals for smooth functions</h3>
<p>One of the features of
<pre>smooth</pre>
<p> functions is their ability to produce different types of prediction intervals. Parametric prediction intervals (triggered by
<pre>intervals=&quot;p&quot;</pre>
<p>,
<pre>intervals=&quot;parametric&quot;</pre>
<p> or
<pre>intervals=TRUE</pre>
<p>) are derived analytically only for <a rel="nofollow" href="http://forecasting.svetunkov.ru/en/2016/11/02/smooth-package-for-r-es-function-part-ii-pure-additive-models/" target="_blank">pure additive</a> and <a rel="nofollow" href="http://forecasting.svetunkov.ru/en/2016/11/18/smooth-package-for-r-es-function-part-iii-multiplicative-models/" target="_blank">pure multiplicative</a> models and are based on the state-space model discussed in previous posts. In the current
<pre>smooth</pre>
<p> version (v2.0.0) only
<pre>es()</pre>
<p> function has multiplicative components, all the other functions are based on additive models. This makes
<pre>es()</pre>
<p> “special”. While constructing intervals for pure models (either additive or multiplicative) is relatively easy to do, the mixed models cause pain in the arse (one of the reasons why I don’t like them). So in case of <a rel="nofollow" href="http://forecasting.svetunkov.ru/en/2017/01/24/smooth-package-for-r-es-function-part-iv-model-selection-and-combination-of-forecasts/" target="_blank">mixed ETS models</a>, we have to use several tricks.</p>
<p>If the model has multiplicative error, non-multiplicative other components (trend, seasonality) and low variance of the error (smaller than 0.1), then the intervals can be approximated by similar models with additive error term. For example, the intervals for ETS(M,A,N) can be approximated with intervals of ETS(A,A,N), when the variance is low, because the distribution of errors in both models will be similar. In all the other cases we use simulations for prediction intervals construction (via
<pre>sim.es()</pre>
<p> function). In this case the data is generated with preset parameters (including variance) and contains \(h\) observations. This process is repeated 10,000 times, resulting in 10,000 possible trajectories. After that the necessary quantiles of these trajectories for each step ahead are taken using
<pre>quantile()</pre>
<p> function from
<pre>stats</pre>
<p> package and returned as prediction intervals. This cannot be considered as a pure parametric approach, but it is the closest we have.</p>
<pre>smooth</pre>
<p> functions also introduce semiparametric and nonparametric prediction intervals. Both of them are based on multiple steps ahead (also sometimes called as “trace”) forecast errors. These are obtained via producing forecasts for horizon 1 to \(h\) from each observation of time series. As a result a matrix with \(h\) columns and \(T-h\) rows is produced. In case of semi-parametric intervals (called using
<pre>intervals=&quot;sp&quot;</pre>
<p> or
<pre>intervals=&quot;semiparametric&quot;</pre>
<p>), variances of forecast errors for each horizon are calculated and then used in order to extract quantiles of either normal or log-normal distribution (depending on error type). This way we cover possible violation of assumptions of homoscedasticity and no autocorrelation in residuals, but we still assume that each separate observation has some parametric distribution.</p>
<p>In case of non-parametric prediction intervals (defined in R via
<pre>intervals=&quot;np&quot;</pre>
<p> or
<pre>intervals=&quot;nonparametric&quot;</pre>
<p>), we loosen assumptions further, dropping part about distribution of residuals. In this case quantile regressions are used as proposed by <a rel="nofollow" href="https://www.jstor.org/stable/2634872?seq=1#page_scan_tab_contents" target="_blank">Taylor and Bunn, 1999</a>. However we use a different form of regression model than the authors do:<br>
\begin{equation} \label{eq:ssTaylorPIs}<br>
\hat{e}_{j} = a_0 j ^ {a_{1}},<br>
\end{equation}<br>
where \(j = 1, .., h\) is forecast horizon. This function has an important advantage over the proposed by the authors second order polynomial: it does not have extremum (turning point) for \(j>0\), which means that the intervals won’t behave strangely after several observations ahead. Using polynomials for intervals sometimes leads to weird bounds (for example, expanding and then shrinking). On the other hand, power function allows producing wide variety of forecast trajectories, which correspond to differently increasing or decreasing bounds of prediction intervals (depending on values of \(a_0\) and \(a_1\)), without producing any ridiculous trajectories.</p>
<p>The main problem with nonparametric intervals produced by
<pre>smooth</pre>
<p> is caused by quantile regressions, which do not behave well on small samples. In order to produce correct 0.95 quantile, we need to have at least 20 observations, and if we want 0.99 quantile, then the sample must contain at least 100. In the cases, when there is not enough observations, the produced intervals can be inaccurate and may not correspond to the nominal level values.</p>
<p>As a small note, if a user produces only one-step-ahead forecast, then semiparametric interval will correspond to parametric one (because only the variance of the one-step-ahead error is used), and the nonparametric interval is constructed using
<pre>quantile()</pre>
<p> function from
<pre>stats</pre>
<p> package.</p>
<p>Finally, the width of prediction intervals is regulated by parameter
<pre>level</pre>
<p>, which can be written either as a fraction number (
<pre>level=0.95</pre>
<p>) or as an integer number, less than 100 (
<pre>level=95</pre>
<p>). I personally prefer former, but the latter is needed for the consistency with
<pre>forecast</pre>
<p> package functions. By default all the
<pre>smooth</pre>
<p> functions produce 95% prediction intervals.</p>
<p>There are some other features of prediction interval construction for specific intermittent models and cumulative forecasts, but they will be covered in upcoming posts.</p>
<h3>Examples in R</h3>
<p>We will use a time series N1241 as an example and we will estimate model ETS(A,Ad,N). Here’s how we do that:</p>
<pre>ourModel1 &lt;- es(M3$N1241$x, &quot;AAdN&quot;, h=8, holdout=TRUE, intervals=&quot;p&quot;)
ourModel2 &lt;- es(M3$N1241$x, &quot;AAdN&quot;, h=8, holdout=TRUE, intervals=&quot;sp&quot;)
ourModel3 &lt;- es(M3$N1241$x, &quot;AAdN&quot;, h=8, holdout=TRUE, intervals=&quot;np&quot;)</pre>
<p>
The resulting graphs demonstrate some differences in prediction intervals widths and shapes:</p>
<div class="wp-caption"><a rel="nofollow" href="https://i2.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2016/10/N1241-AAdN-PI-parametric.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i2.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2016/10/N1241-AAdN-PI-parametric-300x175.png?resize=300%2C175" alt="Series N1241 from M3, es() forecast, parametric prediction intervals" class="wp-image-960" data-recalc-dims="1"></a>
<p class="wp-caption-text">Series N1241 from M3, es() forecast, parametric prediction intervals</p>
</div>
<div class="wp-caption"><a rel="nofollow" href="https://i2.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2016/10/N1241-AAdN-PI-semiparametric.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i0.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2016/10/N1241-AAdN-PI-semiparametric-300x175.png?resize=300%2C175" alt="Series N1241 from M3, es() forecast, semi-parametric prediction intervals" class="wp-image-959" data-recalc-dims="1"></a>
<p class="wp-caption-text">Series N1241 from M3, es() forecast, semiparametric prediction intervals</p>
</div>
<div class="wp-caption"><a rel="nofollow" href="https://i1.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2016/10/N1241-AAdN-PI-nonparametric.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i1.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2016/10/N1241-AAdN-PI-nonparametric-300x175.png?resize=300%2C175" alt="Series N1241 from M3, es() forecast, non-parametric prediction intervals" class="wp-image-958" data-recalc-dims="1"></a>
<p class="wp-caption-text">Series N1241 from M3, es() forecast, nonparametric prediction intervals</p>
</div>
<p>All of them cover actual values in the holdout, because the intervals are very wide. It is not obvious, which of them is the most appropriate for this task. So we can calculate the spread of intervals and see, which of them is on average wider:</p>
<pre>mean(ourModel1$upper-ourModel1$lower)
mean(ourModel2$upper-ourModel2$lower)
mean(ourModel3$upper-ourModel3$lower)</pre>
<p>
Which results in:</p>
<pre>950.4171
955.0831
850.614</pre>
<p>
In this specific example, the non-parametric interval appeared to be the narrowest, which is good, taking that it adequately covered values in the holdout sample. However, this doesn’t mean that it is in general superior to the other methods. Selection of the appropriate intervals should be done based on the general understanding of the violated assumptions. If we didn’t know the actual values in the holdout sample, then we could make a decision based on the analysis of the in-sample residuals in order to get a clue about the violation of any assumptions. This can be done, for example, this way:</p>
<pre>forecast::tsdisplay(ourModel1$residuals)

hist(ourModel1$residuals)

qqnorm(ourModel3$residuals)
qqline(ourModel3$residuals)</pre>
<p><div class="wp-caption"><a rel="nofollow" href="https://i1.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2017/05/N1241-residuals-plot.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i0.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2017/05/N1241-residuals-plot-300x175.png?resize=300%2C175" alt="" class="wp-image-1254" data-recalc-dims="1"></a>
<p class="wp-caption-text">Linear plot and correlation functions of the residuals of the ETS(A,Ad,N) model</p>
</div>
<div class="wp-caption"><a rel="nofollow" href="https://i0.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2017/05/N1241-residuals-histogram.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i1.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2017/05/N1241-residuals-histogram-300x175.png?resize=300%2C175" alt="" class="wp-image-1248" data-recalc-dims="1"></a>
<p class="wp-caption-text">Histogram of the residuals of the ETS(A,Ad,N) model</p>
</div>
<div class="wp-caption"><a rel="nofollow" href="https://i1.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2017/05/N1241-residuals-qqplot.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i1.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2017/05/N1241-residuals-qqplot-300x175.png?resize=300%2C175" alt="" class="wp-image-1250" data-recalc-dims="1"></a>
<p class="wp-caption-text">Q-Q plot of the residuals of the ETS(A,Ad,N) model</p>
</div>
<p>The first plot shows how residuals change over time and how the autocorrelation and partial autocorrelation functions look for this time series. There is no obvious autocorrelation and no obvious heteroscedasticity in the residuals. This means that we can assume that these conditions are not violated in the model, so there is no need to use semiparametric prediction intervals. However, the second and the third graphs demonstrate that the residuals are not normally distributed (as assumed by the model ETS(A,Ad,N)). This means that parametric prediction intervals may be wrong for this time series. All of this motivates the usage of nonparametric prediction intervals for the series N1241.</p>
<p>That’s it for today.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://forecasting.svetunkov.ru/en/2017/06/11/smooth-package-for-r-prediction-intervals/"> R – Modern Forecasting</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...

	<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/lLP-BIm9PD0" height="1"></a></div>