<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/smooth-functions-in-2017/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://forecasting.svetunkov.ru/en/2018/01/01/smooth-functions-in-2017/"> R – Modern Forecasting</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Over the year 2017 the </p>
<pre>smooth</pre>
<p> package has grown from v1.6.0 to v2.3.1. Now it is much more mature and has more downloads. It even now has its own hex (thanks to <a rel="nofollow" href="http://www.bath.ac.uk/management/faculty/fotios-petropoulos.html" target="_blank">Fotios Petropoulos</a>):</p>
<p><a rel="nofollow" href="https://i0.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2018/01/smooth.png" target="_blank"><img sizes="(max-width: 249px) 100vw, 249px" src="https://i2.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2018/01/smooth-249x300.png?resize=249%2C300" alt="" width="249" class="wp-image-1534" data-recalc-dims="1" height="300"></a></p>
<p>A lot of changes happened in 2017, and it is hard to mention all of them, but the major ones are:</p>
<ul>
<li>Introduction of
<pre>ves()</pre>
<p> function (Vector Exponential Smoothing);
<li>Decrease of computation time of
<pre>es()</pre>
<p>, </p>
<pre>ssarima()</pre>
<p> and other functions;
<li>More flexibility in
<pre>es()</pre>
<p> function;
<li>Development of model selection mechanism for exogenous variables (this will be covered in one of the next posts in a couple of weeks);</li>
<li>Development of intermittent data model for
<pre>es()</pre>
<p>, </p>
<pre>ssarima()</pre>
<p>, </p>
<pre>ces()</pre>
<p> and </p>
<pre>ges()</pre>
<p> (will also be covered in one of the posts in 2018);
<li>Introduction of
<pre>auto.ges()</pre>
<p> function, which allows selecting the most appropriate GES model for the data (once again, a post on that is planned);
<li>Switch to
<pre>roxygen2</pre>
<p> package for the maintenance purposes, which simplified my life substantially.

<p>Not all of these things sound exciting, but any house is built brick by brick and some of those bricks are pretty fundamental. Now the new year has started, and I don’t have any intentions of stopping the development of </p>
<pre>smooth</pre>
<p> package. The 2018 will bring more interesting things and features, mainly driven by my research in multivariate models. At the moment I have the following list of features to implement:</p>
<ul>
<li>Intermittent VES model. This is driven by one of the topics I’m working together with <a rel="nofollow" href="http://www.lancaster.ac.uk/lums/people/john-boylan" target="_blank">John Boylan</a> and <a rel="nofollow" href="https://www.linkedin.com/in/stephankolassa/" target="_blank">Stephan Kolassa</a>;</li>
<li>Exogenous variables in VES model (also connected with that topic);</li>
<li>Model selection mechanism in
<pre>ves()</pre>
<p> models;
<li>Conditional intervals for VES model (currently only the version with independent intervals is implemented);</li>
<li>State-space VARMA, which is needed for my research with <a rel="nofollow" href="https://www.hse.ru/staff/victoria.grigoreva" target="_blank">Victoria Grigorieva</a> and Yana Salihova;</li>
<li>
<pre>sim.iss()</pre>
<p> function, which should allow generating the occurrence part of intermittent data. This should be helpful for another research I plan to conduct together with <a rel="nofollow" href="http://www.lancaster.ac.uk/lums/people/john-boylan" target="_blank">John Boylan</a> and <a rel="nofollow" href="https://www.intechopen.com/profiles/51772/patricia-ramos" target="_blank">Patricia Ramos</a>;
<li>Vector SMA – the idea proposed by Mikhail Artukhov, this is yet another research topic;</li>
<li>…And a lot of other cool things and features.</li>

<p>But before starting the work in those areas, I’ve decided to measure the performance of the most recent </p>
<pre>smooth</pre>
<p> functions (accuracy and computation time) and report it here. I used </p>
<pre>ets()</pre>
<p> and </p>
<pre>auto.arima()</pre>
<p> functions from </p>
<pre>forecast</pre>
<p> package as benchmarks. I have done this experiment in serial and included the following models:</p>
<ul>
<li>ES – the default exponential smoothing, ETS(Z,Z,Z) model produced by
<pre>es()</pre>
<p> function from </p>
<pre>smooth</pre>
<p>;
<li>ESa – the same ETS model, but without multiplicative trends. This was needed in order to make a proper comparison with
<pre>ets()</pre>
<p> function, which neglects the multiplicative trends by default. The call for this model is </p>
<pre>es(datasets[[i]],&quot;ZXZ&quot;)</pre>
<p>. I report two versions of this thing: one from </p>
<pre>smooth</pre>
<p> v2.3.0 from <a rel="nofollow" href="http://cran.r-project.org/package=smooth" target="_blank">CRAN</a> and the other from </p>
<pre>smooth</pre>
<p> v2.3.1 available on <a rel="nofollow" href="https://github.com/config-i1/smooth" target="_blank">GitHub</a>. The latter has the advanced branch and bound mechanism for these types of models, while the former uses the brute force in model selection;
<li>SSARIMA – State-space ARIMA implemented in
<pre>auto.ssarima()</pre>
<p> function from </p>
<pre>smooth</pre>
<p>;
<li>CES – Complex Exponential Smoothing via
<pre>auto.ces()</pre>
<p> function from </p>
<pre>smooth</pre>
<p>;
<li>GES – Generalised Exponential Smoothing via
<pre>auto.ges()</pre>
<p> function from </p>
<pre>smooth</pre>
</li>
<li>ETS – ETS model implemented in
<pre>ets()</pre>
<p> function from </p>
<pre>forecast</pre>
<p> package;
<li>ETSm – ETS model with multiplicative trends, done via
<pre>ets(dataset[[i]]$x, allow.multiplicative.trend=TRUE)</pre>
<p>. This makes the function comparable with the default </p>
<pre>es()</pre>
<p>;
<li>ARIMA – ARIMA model constructed using
<pre>auto.arima()</pre>
<p> fucntion.

<p>I used five error measures, some of which were discussed in the <a rel="nofollow" href="http://forecasting.svetunkov.ru/en/2017/07/29/naughty-apes-and-the-quest-for-the-holy-grail/" target="_blank">post about APEs</a>, and aggregated them over the dataset using mean and median values. Here’s the list:</p>
<ol>
<li>MASE – Mean Absolute Scaled Error;</li>
<li>sMAE – scaled Mean Absolute Error;</li>
<li>RelMAE – Relative Mean Absolute Error (the benchmark in this case is a simple Naive);</li>
<li>sMSE – scaled Mean Squared Error (MSE divided by the squared average actuals in-sample);</li>
<li>sCE – scaled Cumulative Error (just the sum of errors divided by the average actuals in-sample);</li>
</ol>
<p>The following code was used in the experiment, so feel free to use it for whatever purposes:</p>
<div>
<div>
<div><span></span>Large chunk of R code here</div>

</div>
</div>
<p>All the </p>
<pre>smooth</pre>
<p> functions can be applied to the data of class “Mcomp” directly: the in-sample data will be used for the fit of the model, the forecast of the needed length will be produced and then the accuracy of the forecast will be measured – everything in one go. Applying </p>
<pre>forecast</pre>
<p> functions is slightly trickier, and in order to have the same measurements I used </p>
<pre>Accuracy()</pre>
<p> function from </p>
<pre>smooth</pre>
<p> v2.3.1 (it is unavailable in earlier versions of the package).</p>
<p>I ended up having 5315 forecasts and I calculated those error measures for the whole dataset and for each of the following categories: yearly, quarterly, monthly and other data. You can find 6 tables with the results at the end of this page (the best value is in <span><strong>green bold</strong></span>). Things to notice are:</p>
<ol>
<li>ES performs similar to ETS, when they go through the similar models. e.g. ESa sometimes outperforms ETS and vice versa, but the difference is usually not significant (see, for example, Table 1. But this can also be observed in the other tables);</li>
<li>ETS with multiplicative trends performs worse than the one without them. This holds for both
<pre>es()</pre>
<p> and </p>
<pre>ets()</pre>
<p>;
<li>SSARIMA performs (in terms of accuracy) similar to ARIMA on monthly and quarterly data, but fails on yearly data;</li>
<li>CES and GES sometimes outperform other models, but this is not consistent;</li>
<li>Branch and bound mechanism of
<pre>es()</pre>
<p> is efficient and improves the performance of the function several times (29 minutes for the calculation of ESa v2.3.1 on the whole dataset vs 91 minutes of ESa v2.3.0) – see Table 6 for the details;
<li>The three fastest functions in the comparison are:
<pre>ets()</pre>
<p> with the default parameters values, </p>
<pre>es()</pre>
<p> without multiplicative trends and </p>
<pre>ces()</pre>
</li>

<p>Finally, I’ve conducted <a rel="nofollow" href="https://en.wikipedia.org/wiki/Nemenyi_test" target="_blank">Nemenyi test</a> using </p>
<pre>nemenyi()</pre>
<p> function from <a rel="nofollow" href="https://github.com/trnnick/TStools" target="_blank">TStools package</a> for RelMAE and for the computation time on the whole dataset in order to see if the differences between the models are on average statistically significant and how they rank (sort of average temperature in the hospital metrics). Keep in mind that this test compares the ranks of the contestants, but does not measure the distances between them. Here are the graphs produced by the function:</p>
<div class="wp-caption"><a rel="nofollow" href="https://i0.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2018/01/nemenyi-overall.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i0.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2018/01/nemenyi-overall-300x214.png?resize=300%2C214" alt="" width="300" class="wp-image-1518" data-recalc-dims="1" height="214"></a>
<p class="wp-caption-text">Overall accuracy of models</p>
</div>
<p>The graph above tells us that ETS, ES, ESa, ARIMA and ETSm overall perform similar to each other – they are in one group of models. The other models also perform very close to these, but probably slightly less accurate. In order to see if the difference between the models is indeed significant, we would need to conduct the similar experiment on a larger dataset (<a rel="nofollow" href="https://www.m4.unic.ac.cy/" target="_blank">M4</a> maybe?).</p>
<div class="wp-caption"><a rel="nofollow" href="https://i2.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2018/01/nemenyi-time.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i0.wp.com/forecasting.svetunkov.ru/wp-content/uploads/2018/01/nemenyi-time-300x214.png?resize=300%2C214" alt="" width="300" class="wp-image-1519" data-recalc-dims="1" height="214"></a>
<p class="wp-caption-text">Computation time ranking</p>
</div>
<p>This graph tells us that </p>
<pre>ets()</pre>
<p> is significantly faster than all the other functions and that ESa, ES and ARIMA have similar performance. Once again, thing to keep in mind is that 0.0001 second is faster than 0.0002 seconds, which means that the former would have rank 1 and the latter would have rank 2. This explains, why ESa is overall faster than ETS (see Table 6), but the Nemenyi test tells us the opposite.</p>
<p>Concluding this New Year experiment, I would say that </p>
<pre>smooth</pre>
<p> package is a good package and can now be efficiently used for automatic forecasting. But I’m biased, so don’t believe my words and give it a try yourselves ;).</p>
<p>Happy New Year!</p>
<table>
<caption><strong>Table 1. Overall results</strong></caption>
<thead>
<tr>
<td></td>
<td colspan="5">Mean</td>
<td colspan="5">Median</td>
</tr>
<tr>
<td></td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
</tr>
</thead>
<tbody>
<tr>
<td>ES</td>
<td align="right">236.8</td>
<td align="right">25.7</td>
<td align="right">74.7</td>
<td align="right">167.2</td>
<td align="right">-9.9</td>
<td align="right">136.2</td>
<td align="right">13.2</td>
<td align="right">89.1</td>
<td align="right">2.5</td>
<td align="right">-12.9</td>
</tr>
<tr>
<td>ESa</td>
<td align="right"><span><strong>224.8</strong></span></td>
<td align="right">23.4</td>
<td align="right">73.4</td>
<td align="right">30.0</td>
<td align="right">-37.3</td>
<td align="right">133.4</td>
<td align="right">12.9</td>
<td align="right">88.0</td>
<td align="right">2.4</td>
<td align="right">-17.2</td>
</tr>
<tr>
<td>SSARIMA</td>
<td align="right">235.9</td>
<td align="right">25.2</td>
<td align="right">76.2</td>
<td align="right">46.7</td>
<td align="right">-15.0</td>
<td align="right">136.2</td>
<td align="right">13.4</td>
<td align="right">84.7</td>
<td align="right">2.6</td>
<td align="right">-7.7</td>
</tr>
<tr>
<td>CES</td>
<td align="right">233.7</td>
<td align="right">25.5</td>
<td align="right">75.5</td>
<td align="right">52.8</td>
<td align="right"><span><strong>8.4</strong></span></td>
<td align="right">136.7</td>
<td align="right">13.5</td>
<td align="right">81.3</td>
<td align="right">2.6</td>
<td align="right"><span><strong>2.1</strong></span></td>
</tr>
<tr>
<td>GES</td>
<td align="right">244.1</td>
<td align="right">27.6</td>
<td align="right">75.8</td>
<td align="right">95.3</td>
<td align="right">38.7</td>
<td align="right">135.2</td>
<td align="right">13.3</td>
<td align="right"><span><strong>79.9</strong></span></td>
<td align="right">2.6</td>
<td align="right">2.2</td>
</tr>
<tr>
<td>ETS</td>
<td align="right">227.0</td>
<td align="right"><span><strong>23.2</strong></span></td>
<td align="right"><span><strong>73.2</strong></span></td>
<td align="right"><span><strong>28.7</strong></span></td>
<td align="right">-28.3</td>
<td align="right"><span><strong>131.2</strong></span></td>
<td align="right"><span><strong>12.8</strong></span></td>
<td align="right">85.2</td>
<td align="right"><span><strong>2.3</strong></span></td>
<td align="right">-13.3</td>
</tr>
<tr>
<td>ETSm</td>
<td align="right">241.6</td>
<td align="right">25.9</td>
<td align="right">75.0</td>
<td align="right">53.8</td>
<td align="right">8.9</td>
<td align="right">133.7</td>
<td align="right">13.1</td>
<td align="right">85.8</td>
<td align="right">2.5</td>
<td align="right">-8.7</td>
</tr>
<tr>
<td>ARIMA</td>
<td align="right">232.6</td>
<td align="right">24.7</td>
<td align="right">74.6</td>
<td align="right">38.2</td>
<td align="right">-11.2</td>
<td align="right">132.5</td>
<td align="right">13.3</td>
<td align="right">82.4</td>
<td align="right">2.5</td>
<td align="right">-7.4</td>
</tr>
</tbody>
</table>
<table>
<caption><strong>Table 2. Yearly data</strong></caption>
<thead>
<tr>
<td></td>
<td colspan="5">Mean</td>
<td colspan="5">Median</td>
</tr>
<tr>
<td></td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
</tr>
</thead>
<tbody>
<tr>
<td>ES</td>
<td align="right">332.4</td>
<td align="right">45.1</td>
<td align="right">89.9</td>
<td align="right">598.9</td>
<td align="right">25.2</td>
<td align="right">211.1</td>
<td align="right">21.2</td>
<td align="right">100.0</td>
<td align="right">5.8</td>
<td align="right">-17.0</td>
</tr>
<tr>
<td>ESa</td>
<td align="right"><span><strong>300.7</strong></span></td>
<td align="right"><span><strong>37.7</strong></span></td>
<td align="right"><span><strong>86.2</strong></span></td>
<td align="right">78.1</td>
<td align="right">-34.6</td>
<td align="right">206.9</td>
<td align="right">20.4</td>
<td align="right">100.0</td>
<td align="right">5.4</td>
<td align="right">-25.2</td>
</tr>
<tr>
<td>SSARIMA</td>
<td align="right">327.6</td>
<td align="right">42.8</td>
<td align="right">90.9</td>
<td align="right">141.9</td>
<td align="right"><span><strong>-1.0</strong></span></td>
<td align="right">214.1</td>
<td align="right">21.0</td>
<td align="right">100.0</td>
<td align="right">5.9</td>
<td align="right">-23.2</td>
</tr>
<tr>
<td>CES</td>
<td align="right">323.3</td>
<td align="right">43.8</td>
<td align="right">89.2</td>
<td align="right">163.3</td>
<td align="right">45.5</td>
<td align="right"><span><strong>203.5</strong></span></td>
<td align="right"><span><strong>19.8</strong></span></td>
<td align="right">94.0</td>
<td align="right"><span><strong>4.9</strong></span></td>
<td align="right">2.1</td>
</tr>
<tr>
<td>GES</td>
<td align="right">346.2</td>
<td align="right">50.4</td>
<td align="right">90.2</td>
<td align="right">325.3</td>
<td align="right">90.5</td>
<td align="right">208.1</td>
<td align="right">20.0</td>
<td align="right"><span><strong>91.8</strong></span></td>
<td align="right">5.3</td>
<td align="right"><span><strong>0.2</strong></span></td>
</tr>
<tr>
<td>ETS</td>
<td align="right">304.3</td>
<td align="right"><span><strong>37.7</strong></span></td>
<td align="right">87.2</td>
<td align="right"><span><strong>74.7</strong></span></td>
<td align="right">-47.1</td>
<td align="right">210.8</td>
<td align="right">20.8</td>
<td align="right">100.0</td>
<td align="right">5.9</td>
<td align="right">-26.6</td>
</tr>
<tr>
<td>ETSm</td>
<td align="right">340.0</td>
<td align="right">45.3</td>
<td align="right">91.7</td>
<td align="right">150.4</td>
<td align="right">18.9</td>
<td align="right">216.2</td>
<td align="right">21.4</td>
<td align="right">100.0</td>
<td align="right">6.2</td>
<td align="right">-16.9</td>
</tr>
<tr>
<td>ARIMA</td>
<td align="right">312.1</td>
<td align="right">41.0</td>
<td align="right">86.9</td>
<td align="right">107.7</td>
<td align="right">-5.7</td>
<td align="right">207.6</td>
<td align="right">20.8</td>
<td align="right">100.0</td>
<td align="right">5.7</td>
<td align="right">-20.4</td>
</tr>
</tbody>
</table>
<table>
<caption><strong>Table 3. Quarterly data</strong></caption>
<thead>
<tr>
<td></td>
<td colspan="5">Mean</td>
<td colspan="5">Median</td>
</tr>
<tr>
<td></td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
</tr>
</thead>
<tbody>
<tr>
<td>ES</td>
<td align="right">211.2</td>
<td align="right">20.2</td>
<td align="right">70.9</td>
<td align="right">18.6</td>
<td align="right">-27.7</td>
<td align="right">129.8</td>
<td align="right">11.4</td>
<td align="right">80.4</td>
<td align="right">1.8</td>
<td align="right">-10.9</td>
</tr>
<tr>
<td>ESa</td>
<td align="right"><span><strong>202.6</strong></span></td>
<td align="right">19.6</td>
<td align="right">69.6</td>
<td align="right">14.5</td>
<td align="right">-44.0</td>
<td align="right">128.3</td>
<td align="right">10.8</td>
<td align="right">78.8</td>
<td align="right">1.6</td>
<td align="right">-15.4</td>
</tr>
<tr>
<td>SSARIMA</td>
<td align="right">209.2</td>
<td align="right">20.3</td>
<td align="right">71.5</td>
<td align="right">15.8</td>
<td align="right">-37.9</td>
<td align="right">133.0</td>
<td align="right">11.7</td>
<td align="right">82.0</td>
<td align="right">2.0</td>
<td align="right">-7.8</td>
</tr>
<tr>
<td>CES</td>
<td align="right">209.8</td>
<td align="right">19.7</td>
<td align="right">71.0</td>
<td align="right"><span><strong>14.1</strong></span></td>
<td align="right">-7.1</td>
<td align="right">127.4</td>
<td align="right">11.9</td>
<td align="right">77.0</td>
<td align="right">1.9</td>
<td align="right">4.4</td>
</tr>
<tr>
<td>GES</td>
<td align="right">213.9</td>
<td align="right">20.9</td>
<td align="right">70.7</td>
<td align="right">18.3</td>
<td align="right"><span><strong>2.2</strong></span></td>
<td align="right">126.2</td>
<td align="right">11.9</td>
<td align="right"><span><strong>76.0</strong></span></td>
<td align="right">2.0</td>
<td align="right">4.7</td>
</tr>
<tr>
<td>ETS</td>
<td align="right">207.3</td>
<td align="right"><span><strong>19.3</strong></span></td>
<td align="right"><span><strong>68.3</strong></span></td>
<td align="right"><span><strong>14.1</strong></span></td>
<td align="right">-36.9</td>
<td align="right"><span><strong>121.3</strong></span></td>
<td align="right"><span><strong>10.4</strong></span></td>
<td align="right">77.4</td>
<td align="right"><span><strong>1.5</strong></span></td>
<td align="right">-11.6</td>
</tr>
<tr>
<td>ETSm</td>
<td align="right">214.9</td>
<td align="right">20.0</td>
<td align="right">69.9</td>
<td align="right">14.9</td>
<td align="right">-18.1</td>
<td align="right">126.7</td>
<td align="right">11.3</td>
<td align="right">78.1</td>
<td align="right">1.7</td>
<td align="right">-6.4</td>
</tr>
<tr>
<td>ARIMA</td>
<td align="right">209.2</td>
<td align="right">20.2</td>
<td align="right">69.7</td>
<td align="right">15.7</td>
<td align="right">-23.3</td>
<td align="right">124.3</td>
<td align="right">11.5</td>
<td align="right">77.6</td>
<td align="right">1.8</td>
<td align="right"><span><strong>-3.5</strong></span></td>
</tr>
</tbody>
</table>
<table>
<caption><strong>Table 4. Monthly data</strong></caption>
<thead>
<tr>
<td></td>
<td colspan="5">Mean</td>
<td colspan="5">Median</td>
</tr>
<tr>
<td></td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
</tr>
</thead>
<tbody>
<tr>
<td>ES</td>
<td align="right">202.2</td>
<td align="right">19.6</td>
<td align="right">70.9</td>
<td align="right">23.9</td>
<td align="right">-20.5</td>
<td align="right">106.2</td>
<td align="right">12.1</td>
<td align="right">79.5</td>
<td align="right">2.2</td>
<td align="right">-19.0</td>
</tr>
<tr>
<td>ESa</td>
<td align="right"><span><strong>198.3</strong></span></td>
<td align="right">18.9</td>
<td align="right">70.4</td>
<td align="right">14.2</td>
<td align="right">-38.3</td>
<td align="right"><span><strong>105.5</strong></span></td>
<td align="right">12.1</td>
<td align="right">78.3</td>
<td align="right"><span><strong>2.1</strong></span></td>
<td align="right">-21.4</td>
</tr>
<tr>
<td>SSARIMA</td>
<td align="right">203.3</td>
<td align="right">19.8</td>
<td align="right">72.9</td>
<td align="right">14.8</td>
<td align="right">-11.5</td>
<td align="right">110.6</td>
<td align="right">12.7</td>
<td align="right">78.9</td>
<td align="right">2.3</td>
<td align="right"><span><strong>-1.6</strong></span></td>
</tr>
<tr>
<td>CES</td>
<td align="right">199.0</td>
<td align="right">20.1</td>
<td align="right">72.1</td>
<td align="right">17.1</td>
<td align="right"><span><strong>-3.6</strong></span></td>
<td align="right">112.3</td>
<td align="right">12.7</td>
<td align="right">78.1</td>
<td align="right">2.4</td>
<td align="right">-3.6</td>
</tr>
<tr>
<td>GES</td>
<td align="right">207.4</td>
<td align="right">20.4</td>
<td align="right">72.6</td>
<td align="right">18.2</td>
<td align="right">32.6</td>
<td align="right">111.4</td>
<td align="right">12.6</td>
<td align="right"><span><strong>76.4</strong></span></td>
<td align="right">2.3</td>
<td align="right">-2.1</td>
</tr>
<tr>
<td>ETS</td>
<td align="right">198.4</td>
<td align="right"><span><strong>18.8</strong></span></td>
<td align="right"><span><strong>70.3</strong></span></td>
<td align="right"><span><strong>13.5</strong></span></td>
<td align="right">-15.4</td>
<td align="right">105.8</td>
<td align="right"><span><strong>12.0</strong></span></td>
<td align="right">76.8</td>
<td align="right">2.2</td>
<td align="right">-11.3</td>
</tr>
<tr>
<td>ETSm</td>
<td align="right">206.4</td>
<td align="right">20.0</td>
<td align="right">71.2</td>
<td align="right">26.2</td>
<td align="right">18.9</td>
<td align="right">106.6</td>
<td align="right">12.1</td>
<td align="right">77.8</td>
<td align="right">2.2</td>
<td align="right">-9.8</td>
</tr>
<tr>
<td>ARIMA</td>
<td align="right">205.4</td>
<td align="right">19.7</td>
<td align="right">72.8</td>
<td align="right">15.2</td>
<td align="right">-8.6</td>
<td align="right">111.0</td>
<td align="right">12.5</td>
<td align="right">78.3</td>
<td align="right">2.3</td>
<td align="right">-7.0</td>
</tr>
</tbody>
</table>
<table>
<caption><strong>Table 5. Other data</strong></caption>
<thead>
<tr>
<td></td>
<td colspan="5">Mean</td>
<td colspan="5">Median</td>
</tr>
<tr>
<td></td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
<td align="right">MASE</td>
<td align="right">sMAE</td>
<td align="right">RelMAE</td>
<td align="right">sMSE</td>
<td align="right">sCE</td>
</tr>
</thead>
<tbody>
<tr>
<td>ES</td>
<td align="right"><span><strong>181.0</strong></span></td>
<td align="right"><span><strong>3.7</strong></span></td>
<td align="right"><span><strong>55.7</strong></span></td>
<td align="right">0.8</td>
<td align="right">8.9</td>
<td align="right">144.5</td>
<td align="right"><span><strong>1.7</strong></span></td>
<td align="right">81.8</td>
<td align="right"><span><strong>0.0</strong></span></td>
<td align="right">1.3</td>
</tr>
<tr>
<td>ESa</td>
<td align="right">181.9</td>
<td align="right"><span><strong>3.7</strong></span></td>
<td align="right">56.8</td>
<td align="right">0.8</td>
<td align="right">7.5</td>
<td align="right">152.4</td>
<td align="right"><span><strong>1.7</strong></span></td>
<td align="right">82.5</td>
<td align="right"><span><strong>0.0</strong></span></td>
<td align="right"><span><strong>0.3</strong></span></td>
</tr>
<tr>
<td>SSARIMA</td>
<td align="right">192.9</td>
<td align="right">3.9</td>
<td align="right">59.9</td>
<td align="right">0.9</td>
<td align="right">10.7</td>
<td align="right">149.3</td>
<td align="right">1.8</td>
<td align="right">71.4</td>
<td align="right"><span><strong>0.0</strong></span></td>
<td align="right">2.7</td>
</tr>
<tr>
<td>CES</td>
<td align="right">212.3</td>
<td align="right">3.9</td>
<td align="right">64.6</td>
<td align="right"><span><strong>0.7</strong></span></td>
<td align="right">11.8</td>
<td align="right">171.0</td>
<td align="right">2.1</td>
<td align="right">67.2</td>
<td align="right">0.1</td>
<td align="right">7.2</td>
</tr>
<tr>
<td>GES</td>
<td align="right">202.5</td>
<td align="right">3.9</td>
<td align="right">61.8</td>
<td align="right">0.8</td>
<td align="right">12.2</td>
<td align="right">154.0</td>
<td align="right">1.8</td>
<td align="right">67.1</td>
<td align="right"><span><strong>0.0</strong></span></td>
<td align="right">5.4</td>
</tr>
<tr>
<td>ETS</td>
<td align="right">182.1</td>
<td align="right"><span><strong>3.7</strong></span></td>
<td align="right">57.5</td>
<td align="right">0.8</td>
<td align="right">7.0</td>
<td align="right">149.0</td>
<td align="right"><span><strong>1.7</strong></span></td>
<td align="right">82.1</td>
<td align="right"><span><strong>0.0</strong></span></td>
<td align="right"><span><strong>0.3</strong></span></td>
</tr>
<tr>
<td>ETSm</td>
<td align="right">182.2</td>
<td align="right"><span><strong>3.7</strong></span></td>
<td align="right">56.6</td>
<td align="right">0.8</td>
<td align="right">8.2</td>
<td align="right">149.0</td>
<td align="right"><span><strong>1.7</strong></span></td>
<td align="right">78.4</td>
<td align="right"><span><strong>0.0</strong></span></td>
<td align="right">1.0</td>
</tr>
<tr>
<td>ARIMA</td>
<td align="right">183.2</td>
<td align="right"><span><strong>3.7</strong></span></td>
<td align="right">56.5</td>
<td align="right"><span><strong>0.7</strong></span></td>
<td align="right"><span><strong>6.0</strong></span></td>
<td align="right"><span><strong>140.7</strong></span></td>
<td align="right"><span><strong>1.7</strong></span></td>
<td align="right"><span><strong>62.9</strong></span></td>
<td align="right"><span><strong>0.0</strong></span></td>
<td align="right">-0.6</td>
</tr>
</tbody>
</table>
<table>
<caption><strong>Table 6. Time elapsed in minutes</strong></caption>
<thead>
<tr>
<td align="right">ES</td>
<td align="right">ESa 2.3.0</td>
<td align="right">ESa 2.3.1</td>
<td align="right">SSARIMA</td>
<td align="right">CES</td>
<td align="right">GES</td>
<td align="right">ETS</td>
<td align="right">ETSm</td>
<td align="right">ARIMA</td>
</tr>
</thead>
<tbody>
<tr>
<td align="right">39.0</td>
<td align="right">91.6</td>
<td align="right"><span><strong>29.3</strong></span></td>
<td align="right">135.4</td>
<td align="right">33.1</td>
<td align="right">1206.3</td>
<td align="right">31.0</td>
<td align="right">52.1</td>
<td align="right">78.6</td>
</tr>
</tbody>
</table>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://forecasting.svetunkov.ru/en/2018/01/01/smooth-functions-in-2017/"> R – Modern Forecasting</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/bi7jjXcjKXg" height="1"></a></div></ol></ul></ul></ul>