<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/generalized-additive-models/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://datascienceplus.com/generalized-additive-models/"> R Programming – DataScience+</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>This is also a flexible and smooth technique which captures the Non linearities in the data and helps us to fit Non linear Models.In this article I am going to discuss the implementation of GAMs in R using the <code>'gam'</code> package .Simply saying GAMs are just a Generalized version of Linear Models in which the Predictors \(X_i\) depend Linearly or Non linearly on some Smooth Non Linear functions like Splines , Polynomials or Step functions etc. </p>
<p>The Regression Function \(F(x) \) gets modified in Generalized Additive Models , and only due to this transformation the GAMs are better in terms of Generalization to random unseen data , fits the data very smoothly and flexibly without adding Complexities or much variance to the Model most of the times. </p>
<p>The basic idea in Splines is that we are going to fit Smooth Non linear Functions on a bunch of Predictors \(X_i\) . Additive in the name means we are going to fit and retain the additivity of the Linear Models.</p>
<p>The Regression Equation becomes:<br>
$$f(x) \ = y_i \ = \alpha \ + f_1(x_{i1}) \ + f_2(x_{i2}) \ + …. f_p(x_{ip}) \ + \epsilon_i$$</p>
<p>where the functions \(f_1,f_2,f_3,….f_p \) are different Non Linear Functions on variables \(X_p\) .</p>
<p>Let’s begin with its Implementation in R —<br>
We will use the <code>gam()</code> function in R to fit a GAM.</p>
<pre>
#requiring the Package 
require(gam)

#ISLR package contains the 'Wage' Dataset
require(ISLR)
attach(Wage) #Mid-Atlantic Wage Data

?Wage # To search more on the dataset
?gam() # To search on the gam function 

gam1&lt;-gam(wage~s(age,df=6)+s(year,df=6)+education ,data = Wage)
#in the above function s() is the shorthand for fitting smoothing splines 
#in gam() function

summary(gam1)
<em>## ## Call: gam(formula = wage ~ s(age, df = 6) + s(year, df = 6) + education, ## data = Wage)
## Deviance Residuals:
## Min 1Q Median 3Q Max ## -119.89 -19.73 -3.28 14.27 214.45 ## ## (Dispersion Parameter for gaussian family taken to be 1235.516)
## ## Null Deviance: 5222086 on 2999 degrees of freedom
## Residual Deviance: 3685543 on 2983 degrees of freedom
## AIC: 29890.31 ## ## Number of Local Scoring Iterations: 2 ## ## Anova for Parametric Effects
## Df Sum Sq Mean Sq F value Pr(>F) ## s(age, df = 6) 1 200717 200717 162.456 &lt; 2.2e-16 ***
## s(year, df = 6) 1 22090 22090 17.879 2.425e-05 ***
## education 4 1069323 267331 216.372 &lt; 2.2e-16 ***
## Residuals 2983 3685543 1236 ## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## ## Anova for Nonparametric Effects
## Npar Df Npar F Pr(F) ## (Intercept) ## s(age, df = 6) 5 26.2089 &lt;2e-16 ***
## s(year, df = 6) 5 1.0144 0.4074 ## education ## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
</em></pre>
<p>Now in the above code we are fitting a GAM which is Non linear in ‘age’ and ‘year’ with 6 degrees of freedom because they are fitted using Smoothing Splines , whereas it is Linear in Terms of variable ‘education’.</p>
<h3>Plotting the Model</h3>
<pre>
#Plotting the Model
par(mfrow=c(1,3)) #to partition the Plotting Window
plot(gam1,se = TRUE) 
#se stands for standard error Bands</pre>
<p>Gives this plot:<br>
<a rel="nofollow" href="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/gam1.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/gam1-490x350.png?w=450&ssl=1" alt="" class="wp-image-7419" data-recalc-dims="1"></a></p>
<p>The above image has 3 different plots for each variable included in the Model.The X-axis contains the variable values itself and the Y-axis contains the Response values i.e the Salaries.<br>
From the plots and their shapes we can see that Salary first increases with ‘age’ then decreases after around 60.For variable ‘year’ the Salaries tend to increase , and it seems that there is a decrease in salary at around year 2007 or 2008. And for the Categorical variable ‘education’ , Salary also increases monotonically. The curvy shapes for the variables ‘age’ and ‘year’ is due to the Smoothing splines which models the Non linearities in the data.The dotted Lines around the main curve lines are the <code>Standard Error</code> Bands. </p>
<p>Hence this is a very effective way of fitting Non linear functions on several variables and producing the plots for each and study the effect on the Response.</p>
<h2>Logistic Regression using GAM</h2>
<p>We can also fit a Logistic Regression Model using GAMs for predicting the Probabilities of the Binary Response values. We will use the identity <code>I()</code> function to convert the Response to a Binary variable.</p>
<pre>
#logistic Regression Model
logitgam1&lt;-gam(I(wage > 250) ~ s(age,df=4) + s(year,df=4) + education ,data=Wage,family=binomial)
plot(logitgam1,se=T)</pre>
<p>In this Logistic Regression Model we are trying to find the conditional probability for the Wage variable which can take 2 values either, \( P(wage>250 \ | \ X_i) \) and \( P(wage&lt;250 \ | \ X_i) \).<br>
Gives this plot:<br>
<a rel="nofollow" href="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/1.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/1-490x350.png?w=450&ssl=1" alt="" class="wp-image-7423" data-recalc-dims="1"></a></p>
<p><a rel="nofollow" href="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/2.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/2-490x350.png?w=450&ssl=1" alt="" class="wp-image-7424" data-recalc-dims="1"></a></p>
<p><a rel="nofollow" href="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/3.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/3-490x350.png?w=450&ssl=1" alt="" class="wp-image-7425" data-recalc-dims="1"></a></p>
<p>The above Plots are the same as the first Model,difference is that the Y-axis will now be the Logit \( log\frac{P(X)}{(1-P(X))} \) of the Probability values , and we now fit using 4 degrees of freedom for the variables ‘age’ and ‘year’ and again linear in terms of ‘education’ variable.</p>
<p>In the above Plot for ‘Year’ variable we can see that the error bands are quiet wide and broad.This might be an indication that our Non linear function fitted for ‘Year’ variable is not significant.</p>
<h3>Now we can also check if we need Non linear terms for Year using Anova Test</h3>
<p>We are now going to fit another model which in Linear in variable ‘year’.</p>
<pre>
#fitting the Additive Regression Model which is linear in Year
logitgam2&lt;-gam(I(wage >250) ~ s(age,df=4)+ year + education , data =Wage, family = binomial)
plot(logitgam2)</pre>
<p>Gives this plot:<br>
<a rel="nofollow" href="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/1-1.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/1-1-490x350.png?w=450&ssl=1" alt="" class="wp-image-7439" data-recalc-dims="1"></a></p>
<p><a rel="nofollow" href="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/2-1.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/2-1-490x350.png?w=450&ssl=1" alt="" class="wp-image-7440" data-recalc-dims="1"></a></p>
<p><a rel="nofollow" href="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/3-1.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/3-1-490x350.png?w=450&ssl=1" alt="" class="wp-image-7441" data-recalc-dims="1"></a></p>
<p>Now for this Model,we don’t fit a Non linear function on ‘year’ variable , and it is simply a linear function in nature.As we can analyze from the plot above for ‘year’ , it is linear i.e a straight line (a polynomial of degree 1).</p>
<p>Now we will use <code>anova()</code> function in R for checking the goodness of fit for the above models, one which is Non Linear in Year and another which is Linear in Year.</p>
<pre>
#anova() function to test the goodness of fit and choose the best Model
#Using Chi-squared Non parametric Test due to Binary Classification Problem and categorical Target

anova(logitgam1,logitgam2,test = &quot;Chisq&quot;)
<em>## Analysis of Deviance Table
## ## Model 1: I(wage > 250) ~ s(age, df = 4) + s(year, df = 4) + education
## Model 2: I(wage > 250) ~ s(age, df = 4) + year + education
## Resid. Df Resid. Dev Df Deviance Pr(>Chi)
## 1 2987 602.87 ## 2 2990 603.78 -3 -0.90498 0.8242
</em></pre>
<p>The above results indicate that Model 2 i.e the one which is linear in terms of ‘year’ variable is significant and much better.Hence this indicates that we don’t need a GAM which fits a Non linear function for variable ‘year’.</p>
<h2>Another way of Fitting a GAM</h2>
<p>Now we can also fit a Generalized Additive Model using the <code>lm()</code> function in R,which stands for linear Model.And then we can fit Non linear functions on different variables \(X_i\) using the <code>ns()</code> or <code>bs()</code> function which stands for natural splines and cubic splines and add them to the Regression Model.</p>
<pre>
lm1&lt;-lm(wage ~ ns(age,df=4) + ns(year,df=4)+ education , data  = Wage)
#ns() is function used to fit a Natural Cubic Spline
lm1

#Now plotting the Model

plot.gam(lm1,se=T)
<em>## ## Call:
## lm(formula = wage ~ ns(age, df = 4) + ns(year, df = 4) + education, ## data = Wage)
## ## Coefficients:
## (Intercept) ns(age, df = 4)1 ## 43.976 46.541 ## ns(age, df = 4)2 ns(age, df = 4)3 ## 29.070 63.853 ## ns(age, df = 4)4 ns(year, df = 4)1 ## 10.881 8.417 ## ns(year, df = 4)2 ns(year, df = 4)3 ## 3.596 8.000 ## ns(year, df = 4)4 education2. HS Grad ## 6.701 10.870 ## education3. Some College education4. College Grad ## 23.354 38.112 ## education5. Advanced Degree ## 62.517
</em>
</pre>
<p>Gives this plot:<br>
<a rel="nofollow" href="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/1-2.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/1-2-490x350.png?w=450&ssl=1" alt="" class="wp-image-7446" data-recalc-dims="1"></a></p>
<p><a rel="nofollow" href="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/2-2.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/2-2-490x350.png?w=450&ssl=1" alt="" class="wp-image-7447" data-recalc-dims="1"></a></p>
<p><a rel="nofollow" href="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/07/3png.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/07/3png-490x350.png?w=450&ssl=1" alt="" class="wp-image-7448" data-recalc-dims="1"></a></p>
<p>Hence as the plot shows that the output of <code>lm()</code> function is also similar and same.It does not makes a difference if we use gam() or lm() to fit Generalized Additive Models.Both produce exactly same results.</p>
<h2>Conclusion</h2>
<p>Generalized Additive Models are a very nice and effective way of fitting Non linear Models which are smooth and flexible.Best part is that they lead to interpretable Models. We can easily mix terms in GAMs,some linear and some Non Linear terms and then compare those Models using the <code>anova()</code> function which performs a Anova test for goodness of fit.The non linear terms on Predictors \(X_i\) can be anything from smoothing splines , natural cubic splines to polynomial functions or step functions etc. GAMs are additive in nature , which means there are no interaction terms in the Model.</p>
<p>Thanks a lot for reading the article,and make sure to like and share it.Cheers!</p>
<ul></ul>
<p><strong>
<p>Related Post</p>
<p>
<ol>
<li><a rel="nofollow" href="https://datascienceplus.com/second-step-with-non-linear-regression-adding-predictors/" title="Permanent Link toSecond step with non-linear regression: adding predictors" target="_blank">Second step with non-linear regression: adding predictors</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/weather-forecast-with-regression-models-part-4/" title="Permanent Link toWeather forecast with regression models – part 4" target="_blank">Weather forecast with regression models – part 4</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/weather-forecast-with-regression-models-part-3/" title="Permanent Link toWeather forecast with regression models – part 3" target="_blank">Weather forecast with regression models – part 3</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/weather-forecast-with-regression-models-part-2/" title="Permanent Link toWeather forecast with regression models – part 2" target="_blank">Weather forecast with regression models – part 2</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/weather-forecast-with-regression-models-part-1/" title="Permanent Link toWeather forecast with regression models – part 1" target="_blank">Weather forecast with regression models – part 1</a></li>
</ol></strong>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://datascienceplus.com/generalized-additive-models/"> R Programming – DataScience+</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/KmGeYcEH240" height="1"></a></div>