<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/machine-learning-linear-regression-full-example-boston-housing/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://feedproxy.google.com/~r/DataRValue/~3/ZyOwQfki3QQ/machine-learning-linear-regression-full.html"> Data R Value</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><span><span lang="en">It is important to mention that the present posts began as a personal way of practicing R programming and machine learning. <span>Subsequently feedback from the community, urged me to continue performing these exercises and sharing them.</span><span> </span></span></span><br><span><span lang="en"><span>The bibliography and corresponding authors are cited at all times and it is a way of honoring them and giving them the credit they deserve for their work.</span></span></span></p>
<p><span><span lang="en"><span>We will develop a linear regression</span></span></span><span><span lang="en"><span><span><span lang="en"><span> example, </span></span></span></span></span></span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span>including simple linear regression, multiple linear regression, linear regression with term interaction, linear regression with higher order terms, and finally with a transformation.</span></span></span></span></span> The exercise was originally published in <b><i>“An Introduction to Statistical Learning. With applications in R” </i>by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. Springer 2015</b>.</span></span></span></p>
<p><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span>The example we will develop is about predicting the median house value on the Boston Housing dataset.</span></span></span></span></span></span></span></span> </span></span></span><br><span><span lang="en"><span><br></span></span></span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>We will carry out the exercise verbatim as published in the aforementioned reference</span></span></span></span></span></span><span lang="en"><span> and o</span></span></span></span></span></span><span lang="en"><span>nly with slight changes in the coding style.</span></span></span></p>
<p><span><span lang="en"><span>For more details on the models, algorithms and parameters interpretation, it is recommended to check the aforementioned reference or any other bibliography of your choice.</span></span></span></p>
<p><span><span lang="en"><span>
<p><b>### install and load required packages</b><br>#install.packages(“ISLR”)<br>#install.packages(“car”)</p>
<p>library(ISLR)<br>library(car)<br> <br><span><span lang="en"><span><b>### MASS library contains the Boston dataset </b><br>library(MASS)
<p><b>### explore dataset</b><br>#fix(Boston)<br>str(Boston)<br>summary(Boston)<br>names(Boston)</p>
<p><b>### we will seek to predict target: medv (median house value)<br>### using 13 predictors
<p>### we will start with $medv as target and $lstat as predictor<br>### create the model<br>lm.fit &lt;-lm(medv~lstat, data=Boston)<br><b><br>### explore the model parameters</b><br>lm.fit<br>summary(lm.fit)<br>names(lm.fit)<br>coef(lm.fit)<br><b><br>### explore confidence interval for the coefficient estimates</b><br>confint(lm.fit)</p></b>
<p><b>### use predict() to produce confidence intervals and prediction<br>### intervals for the prediction of $medv for a given value of<br>### $lstat</b><br>predict (lm.fit , data.frame(lstat = c(5 ,10 ,15) ),<br>          interval = “confidence”)<br>predict (lm.fit , data.frame(lstat = c(5 ,10 ,15) ),<br>          interval = “prediction”)<br> <br><span><span lang="en"><span><b>### 95% confidence interval associated with $lstat=10 is<br>### (24.47, 25.63)<br>### 95% prediction interval is (12.828, 37.28)<br>### center around a predicted value of 25.05 for $medv when<br>### $lstat = 10
<p>### now plot $medv and lstat along with least squares regression<br>### line<br>plot(Boston$lstat , Boston$medv, main=”Boston Housing Data”,<br>      xlab=”Percent of households with low<br>     socioeconomic status”, ylab= “Median house value”)</p></b>
<p>abline(lm.fit, col = “red”, lwd = 3)<br><span><span lang="en"><span><br></span></span></span>
<div><a target="_blank" rel="nofollow" href="https://i2.wp.com/2.bp.blogspot.com/-unPiJifdOP0/WRX_5-HLzkI/AAAAAAAAC-8/_k5kCdsV-_MVRVKybFGhpOgxa-wDY8pSQCLcB/s1600/housing_1.png?ssl=1"><img src="https://i2.wp.com/2.bp.blogspot.com/-unPiJifdOP0/WRX_5-HLzkI/AAAAAAAAC-8/_k5kCdsV-_MVRVKybFGhpOgxa-wDY8pSQCLcB/s400/housing_1.png?resize=400%2C265&ssl=1" border="0" data-recalc-dims="1"></a></div></span>
<p><span><span lang="en"><span></span></span></span><br><span><span lang="en"><span></span></span></span><br><span><span lang="en"><span><br><b>### explore some diagnostic plots</b><br>par(mfrow = c(2, 2))<br>plot(lm.fit)
<p>plot(predict(lm.fit), residuals(lm.fit))<br>plot(predict(lm.fit), rstudent(lm.fit))
<div><a target="_blank" rel="nofollow" href="https://i1.wp.com/3.bp.blogspot.com/-ihmZgswShfM/WRYABxpagfI/AAAAAAAAC_A/FL6g3zYnUx41Tq7ZlFSv8TU9ICsLdW8eQCLcB/s1600/housing_2.png?ssl=1"><img src="https://i1.wp.com/3.bp.blogspot.com/-ihmZgswShfM/WRYABxpagfI/AAAAAAAAC_A/FL6g3zYnUx41Tq7ZlFSv8TU9ICsLdW8eQCLcB/s400/housing_2.png?resize=400%2C356&ssl=1" border="0" data-recalc-dims="1"></a></div></span>
<p><span><span lang="en"><span><br><b> </b></span></span></span><br><span><span lang="en"><span><b>### leverage statistics hatvalues()</b><br>plot(hatvalues(lm.fit))<br>which.max(hatvalues(lm.fit))</span></span></span></p>
<div><a target="_blank" rel="nofollow" href="https://i1.wp.com/1.bp.blogspot.com/-8c1vhQ9ZUnY/WRYAKuEedJI/AAAAAAAAC_E/vIp2Lgecre8-3n5sO9E6IxVbTGxkM045ACLcB/s1600/housing_3.png?ssl=1"><img src="https://i0.wp.com/1.bp.blogspot.com/-8c1vhQ9ZUnY/WRYAKuEedJI/AAAAAAAAC_E/vIp2Lgecre8-3n5sO9E6IxVbTGxkM045ACLcB/s400/housing_3.png?resize=400%2C356&ssl=1" border="0" data-recalc-dims="1"></a></div>
<p><span><span lang="en"><span><b>### multiple linear regression<br>### two predictors</b><br>lm.fit &lt;-lm(medv ~ lstat + age , data = Boston)<br>summary(lm.fit)
<p><b>### all 13 predictors</b><br>lm.fit &lt;-lm(medv ~. , data = Boston)<br>summary(lm.fit)</p>
<p><b>### R^2</b><br>summary(lm.fit)$r.sq<br><b>### RSE</b><br>summary(lm.fit)$sigma<br><b><br>### use vif() to compute variance inflation factors<br>### vif is part of car library</b><br>vif(lm.fit)<br><b><br>### in the last regression $age has a high p-value<br>### run regression using all predictors but one: $age</b><br>lm.fit1 &lt;-lm(medv ~. -age, data = Boston)<br>summary(lm.fit1)</p>
<p><b>### interaction terms</b><br>summary(lm(medv ~ lstat*age , data = Boston ))<br><b><br>### non-linear transformation of the predictors</b><br>lm.fit2 &lt;-lm(medv ~lstat + I(lstat ^2), data = Boston)<br>summary(lm.fit2)</p>
<p><b>### a near cero p-value associated with the quadratic term<br>### leads to an improved model</b></p>
<p><b>### use anova() to further quantify the extent to which quadratic<br>### fit is superior to the linear fit</b><br>lm.fit &lt;-lm(medv ~ lstat, data = Boston)<br>anova(lm.fit, lm.fit2)</p>
<p><b>### null hypothesis is that the two models fit data equally well<br>### alternative hypothesis is that the full model is superior<br>### F-statistic is 135 and p-value ~ 0 is clear evidence </b><br><span><span lang="en"><span><b>### that model containing the predictors lstat and lstat^2<br>### is far superior<br>### we saw in the above plots evidence of nonlinearity</b>
<p><b><br>### now use higher order polynomials with poly()</b><br>lm.fit5 = lm(medv ~ poly(lstat, 5), data = Boston)<br>summary(lm.fit5)<br><b><br>### additional polynomial terms leads to an improvement in the<br>### model fit
<p>### now try a log transformation for $rm<br>summary (lm(medv ~ log(rm) , data = Boston ) )</p></b>
<p><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>You can get the example in:</span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><a target="_blank" rel="nofollow" href="https://github.com/pakinja/Data-R-Value/tree/master/MachineLearning_LinearRegression_BostonHousing">https://github.com/pakinja/Data-R-Value</a></span></span></span></span></span></span></span></span><br><img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/DataRValue/~4/ZyOwQfki3QQ"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://feedproxy.google.com/~r/DataRValue/~3/ZyOwQfki3QQ/machine-learning-linear-regression-full.html"> Data R Value</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/BSm1mJgu0mA"></a></div></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>