<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/how-to-add-trend-lines-in-r-using-plotly/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://www.displayr.com/how-to-add-trend-lines-in-r-using-plotly/?utm_medium=Feed&utm_source=Syndication"> R – Displayr</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2>1. Global trend lines</h2>
<p>One of the simplest methods to identify trends is to fit a ordinary least squares regression model to the data. The model most people are familiar with is the linear model, but you can add other polynomial terms for extra flexibility. In practice, avoid polynomials of degrees larger than three because they are less stable.</p>
<p><img sizes="(max-width: 1003px) 100vw, 1003px" src="https://i2.wp.com/www.displayr.com/wp-content/uploads/2018/07/plotly-trendlines-global.png?w=450&ssl=1" alt="Global trendlines with plotly" class="wp-image-5658" data-recalc-dims="1"></p>
<p>Below, we use the <span>EuStockMarkets</span> data set (available in R data sets) to construct linear, quadratic and cubic trend lines.</p>
<pre></pre>
<pre>
data(&quot;EuStockMarkets&quot;)
xx = EuStockMarkets[,1]
x.info = attr(xx, &quot;tsp&quot;)
tt = seq(from=x.info[1], to = x.info[2], by=1/x.info[3])
data.fmt = list(color=rgb(0.8,0.8,0.8,0.8), width=4)
line.fmt = list(dash=&quot;solid&quot;, width = 1.5, color=NULL)

ti = 1:length(xx)
m1 = lm(xx~ti)
m2 = lm(xx~ti+I(ti^2))
m3 = lm(xx~ti+I(ti^2)+I(ti^3))

require(plotly)
p.glob = plot_ly(x=tt, y=xx, type=&quot;scatter&quot;, mode=&quot;lines&quot;, line=data.fmt, name=&quot;Data&quot;)
p.glob = add_lines(p.glob, x=tt, y=predict(m1), line=line.fmt, name=&quot;Linear&quot;)
p.glob = add_lines(p.glob, x=tt, y=predict(m2), line=line.fmt, name=&quot;Quadratic&quot;)
p.glob = add_lines(p.glob, x=tt, y=predict(m3), line=line.fmt, name=&quot;Cubic&quot;)
p.glob = layout(p.glob, title = &quot;Global smoothers&quot;)
print(p.glob)
</pre>
<h2>2. Local smoothers</h2>
<p>Global models assume that the time series follows a single trend. For many data sets, however, we would want to relax this assumption. In the following section, we demonstrate the use of local smoothers using the <span>Nile</span> data set. It contains measurements of the annual river flow of the Nile over 100 years and is less regular than the <span>EuStockMarkets</span> data set.</p>
<h3>i. Moving averages</h3>
<p>The moving average (also known as running mean) method consists of taking the mean of a fixed number of nearby points.  Even with this simple method we see that the question of how to choose the neighborhood is crucial for local smoothers. Increasing the bandwidth from 5 to 20 suggests that there is a gradual decrease in annual river flow from 1890 to 1905 instead of a sharp decrease at around 1900.</p>
<p>Many packages include functions to compute the running mean such as <span>caTools::runmean</span> and <span>forecast::ma</span>, which may have additional features, but <span>filter</span> in the base <span>stats</span> package can be used to compute moving averages without installing additional packages.</p>
<p><img sizes="(max-width: 1016px) 100vw, 1016px" src="https://i0.wp.com/www.displayr.com/wp-content/uploads/2018/07/plotly-trendlines-movingaverage.png?w=450&ssl=1" alt="Moving averages in plotly" class="wp-image-5659" data-recalc-dims="1"></p>
<pre>
data(&quot;Nile&quot;)
xx = Nile
x.info = attr(xx, &quot;tsp&quot;)
tt = seq(from=x.info[1], to = x.info[2], by=1/x.info[3])

rmean20 = stats::filter(xx, rep(1/20, 20), side=2)
rmean5 = stats::filter(xx, rep(1/5, 5), side=2)

require(plotly)
p.rm = plot_ly(x=tt, y=xx, type=&quot;scatter&quot;, mode=&quot;lines&quot;, line=data.fmt, name=&quot;Data&quot;)
p.rm = add_lines(p.rm, x=tt, y=rmean20, line=line.fmt, name=&quot;Bandwidth = 20&quot;)
p.rm = add_lines(p.rm, x=tt, y=rmean5, line=line.fmt, name=&quot;Bandwidth = 5&quot;)
p.rm = layout(p.rm, title = &quot;Running mean&quot;)
print(p.rm)
</pre>
<h3>ii. Running line smoothers</h3>
<p>The running line smoother reduces the bias by fitting a linear regression in a local neighborhood of the target value. A popular algorithm using the running line smoother is Friedman’s super smoother <span>supsmu</span>, which by default uses cross-validation to find the best span.</p>
<p><img sizes="(max-width: 1021px) 100vw, 1021px" src="https://i2.wp.com/www.displayr.com/wp-content/uploads/2018/07/plotly-trendlines-runningline.png?w=450&ssl=1" alt="Running line smoothers in plotly" class="wp-image-5660" data-recalc-dims="1"></p>
<pre>
rlcv = supsmu(tt, xx)
rlst = supsmu(tt, xx, span = 0.05)
rllt = supsmu(tt, xx, span = 0.75)

p.rl = plot_ly(x=tt, y=xx, type=&quot;scatter&quot;, mode=&quot;lines&quot;, line = data.fmt, name=&quot;Data&quot;)
p.rl = add_lines(p.rl, x=tt, y=rllt$y, line=line.fmt, name=&quot;Span = 0.75&quot;)
p.rl = add_lines(p.rl, x=tt, y=rlst$y, line=line.fmt, name=&quot;Span = 0.05&quot;)
p.rl = add_lines(p.rl, x=tt, y=rlcv$y, line=line.fmt, name=&quot;Cross-validated span&quot;)
p.rl = layout(p.rl, title = &quot;Running line smoothers&quot;)
print(p.rl)
</pre>
<h3>iii. Kernel smoothers</h3>
<p>An alternative approach to specifying a neighborhood is to decrease weights further away from the target value. These estimates are much smoother than the results from either the running mean or running line smoothers.</p>
<p><img sizes="(max-width: 993px) 100vw, 993px" src="https://i2.wp.com/www.displayr.com/wp-content/uploads/2018/07/plotly-trendlines-kernel.png?w=450&ssl=1" alt="Kernel smoothers in plotly" class="wp-image-5661" data-recalc-dims="1"></p>
<pre>
ks1 = ksmooth(tt, xx, &quot;normal&quot;, 20, x.points=tt)
ks2 = ksmooth(tt, xx, &quot;normal&quot;, 5, x.points=tt)

p.ks = plot_ly(x=tt, y=xx, type=&quot;scatter&quot;, mode=&quot;lines&quot;, line=data.fmt, name=&quot;Data&quot;)
p.ks = add_lines(p.ks, x=ks1$x, y=ks1$y, line=line.fmt, name=&quot;Bandwidth = 20&quot;)
p.ks = add_lines(p.ks, x=ks1$x, y=ks2$y, line=line.fmt, name=&quot;Bandwidth = 5&quot;)
p.ks = layout(p.ks, title = &quot;Kernel smoother&quot;)\
print(p.ks)
</pre>
<h3>iv. Smoothing splines</h3>
<p>Splines consist of a piece-wise polynomial with pieces defined by a sequence of knots where the pieces join smoothly. A smoothing splines is estimated by minimizing a criterion containing a penalty for both goodness of fit, and smoothness. The trade-off between the two is controlled by the smoothing parameter <span>lambda</span>, which is typically chosen by cross-validation.</p>
<p><img sizes="(max-width: 838px) 100vw, 838px" src="https://i2.wp.com/www.displayr.com/wp-content/uploads/2018/07/plotly-trendlines-gam.png?w=450&ssl=1" alt="Smoothing splines trend lines in plotly" class="wp-image-5662" data-recalc-dims="1"></p>
<p>In the base package, <span>smooth.spline</span> can be used to compute splines, but it is more common to use the <span>GAM</span> function in <span>mgcv</span>. Both functions use cross-validation to choose the default smoothing parameter; but as seen in the chart above, the results vary between implementations. Another advantage to using <span>GAM</span> is that it allows estimation of confidence intervals.</p>
<pre>
require(mgcv)
sp.base = smooth.spline(tt, xx)
sp.cr = gam(xx~s(tt, bs=&quot;cr&quot;))
sp.gam = gam(xx~s(tt))
sp.pred = predict(sp.gam, type=&quot;response&quot;, se.fit=TRUE)
sp.df = data.frame(x=sp.gam$model[,2], y=sp.pred$fit,
                    lb=as.numeric(sp.pred$fit - (1.96 * sp.pred$se.fit)),
                    ub=as.numeric(sp.pred$fit + (1.96 * sp.pred$se.fit)))
sp.df = sp.df[order(sp.df$x),]

pp = plot_ly(x=tt, y=xx, type=&quot;scatter&quot;, mode=&quot;lines&quot;, line=data.fmt, name=&quot;Data&quot;)
pp = add_lines(pp, x=tt, y=sp.pred$fit, name=&quot;GAM&quot;, line=list(color=&quot;#366092&quot;, width=2))
pp = add_ribbons(pp, x=sp.df$x, ymin=sp.df$lb, ymax=sp.df$ub, name=&quot;GAM 95% CI&quot;, line=list(color=&quot;#366092&quot;, opacity=0.4, width=0))
pp = add_lines(pp, x=tt, y=predict(sp.base)$y, name=&quot;smooth.spline&quot;, line=list(color=&quot;orange&quot;, width=2))
pp = layout(pp, title=&quot;Smoothing splines&quot;)
print(pp)
</pre>
<h3>v. LOESS</h3>
<p>LOESS (Locally Estimated Scatterplot Smoother) combines local regression with kernels by using locally weighted polynomial regression (by default, quadratic regression with tri-cubic weights). It also allows estimation of approximate confidence intervals. However, it is important to note that unlike <span>supsmu</span>, <span>smooth.spline</span> or <span>gam</span>, <span>loess</span> does not use cross-validation. By default, the <span>span</span> is set to 0.75; that is, the estimated smooth at each target value consists of a local regression constructed using 75% of the data points closest to the target value. This span is fairly large and results in estimated values that are smoother than those from other methods.</p>
<p><img sizes="(max-width: 1008px) 100vw, 1008px" src="https://i2.wp.com/www.displayr.com/wp-content/uploads/2018/07/plotly-trendlines-loess.png?w=450&ssl=1" alt="LOESS trend line with different spans in plotly" class="wp-image-5663" data-recalc-dims="1"></p>
<pre>
ll.rough = loess(xx~tt, span=0.1)
ll.smooth = loess(xx~tt, span=0.75)

p.ll = plot_ly(x=tt, y=xx, type=&quot;scatter&quot;, mode=&quot;lines&quot;, line=data.fmt, name=&quot;Data&quot;)
p.ll = add_lines(p.ll, x=tt, y=predict(ll.smooth), line=line.fmt, name=&quot;Span = 0.75&quot;)
p.ll = add_lines(p.ll, x=tt, y=predict(ll.rough), line=line.fmt, name=&quot;Span = 0.10&quot;)
p.ll = layout(p.ll, title=&quot;LOESS&quot;)
print(p.ll)
</pre>
<p><img sizes="(max-width: 836px) 100vw, 836px" src="https://i2.wp.com/www.displayr.com/wp-content/uploads/2018/07/plotly-trendlines-loessCI.png?w=450&ssl=1" alt="LOESS trend line with confidence intervals in plotly" class="wp-image-5664" data-recalc-dims="1"></p>
<pre>
ll.smooth = loess(xx~tt, span=0.75)
ll.pred = predict(ll.smooth, se = TRUE)
ll.df = data.frame(x=ll.smooth$x, fit=ll.pred$fit,
lb = ll.pred$fit - (1.96 * ll.pred$se),
ub = ll.pred$fit + (1.96 * ll.pred$se))
ll.df = ll.df[order(ll.df$tt),]

p.llci = plot_ly(x=tt, y=xx, type=&quot;scatter&quot;, mode=&quot;lines&quot;, line=data.fmt, name=&quot;Data&quot;)
p.llci = add_lines(p.llci, x=tt, y=ll.pred$fit, name=&quot;Mean&quot;, line=list(color=&quot;#366092&quot;, width=2))
p.llci = add_ribbons(p.llci, x=ll.df$tt, ymin=ll.df$lb, ymax=ll.df$ub, name=&quot;95% CI&quot;, line=list(opacity=0.4, width=0, color=&quot;#366092&quot;))
p.llci = layout(p.llci, title = &quot;LOESS with confidence intervals&quot;)
print(p.llci)
</pre>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://www.displayr.com/how-to-add-trend-lines-in-r-using-plotly/?utm_medium=Feed&utm_source=Syndication"> R – Displayr</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/D_N0UNh8ab8" height="1"></a></div>