<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/ordinary-least-squares-ols-linear-regression-in-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://sw23993.wordpress.com/2017/07/04/ordinary-least-squares-ols-linear-regression-in-r/"> Environmental Science and Data Analytics</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Ordinary Least Squares (OLS) linear regression is a statistical technique used for the analysis and modelling of linear relationships between a response variable and one or more predictor variables. If the relationship between two variables appears to be linear, then a straight line can be fit to the data in order to model the relationship. The linear equation (or equation for a straight line) for a bivariate regression takes the following form:</p>
<h2><em>y = mx + c</em></h2>
<p>where <em>y</em> is the response (dependent) variable, <em>m</em> is the gradient (slope),<em> x</em> is the predictor (independent) variable, and <em>c</em> is the intercept. The modelling application of OLS linear regression allows one to predict the value of the response variable for varying inputs of the predictor variable given the slope and intercept coefficients of the line of best fit.</p>
<p>The line of best fit is calculated in R using the lm() function which outputs the slope and intercept coefficients. The slope and intercept can also be calculated from five summary statistics: the standard deviations of <em>x</em> and <em>y</em>, the means of <em>x</em> and <em>y</em>, and the <em><strong>Pearson correlation coefficient</strong> </em>between <em>x</em> and <em>y</em> variables.</p>
<pre>slope &lt;- cor(x, y) * (sd(y) / sd(x))
intercept &lt;- mean(y) - (slope * mean(x))</pre>
<p>The scatterplot is the best way to assess linearity between two numeric variables. From a scatterplot, the strength, direction and form of the relationship can be identified. To carry out a linear regression in R, one needs only the data they are working with and the lm() and predict() base R functions. In this brief tutorial, two packages are used which are not part of base R. They are <em><strong>dplyr</strong></em> and <em><strong>ggplot2</strong></em>.</p>
<p>The built-in mtcars dataset in R is used to visualise the bivariate relationship between fuel efficiency (mpg) and engine displacement (disp).</p>
<pre>library(dplyr)
library(ggplot2)

mtcars %>%
 ggplot(aes(x = disp, y = mpg)) +
 geom_point(colour = &quot;red&quot;)
</pre>
<p> </p>
<p><img data-orig-file="https://sw23993.files.wordpress.com/2017/07/rplot.png?w=456" data-orig-size="592,377" data-medium-file="https://sw23993.files.wordpress.com/2017/07/rplot.png?w=456?w=300" data-attachment-id="1847" data-image-meta="{" data-image-description="" src="https://sw23993.files.wordpress.com/2017/07/rplot.png?w=456" data-permalink="https://sw23993.wordpress.com/2017/07/04/ordinary-least-squares-ols-linear-regression-in-r/rplot-8/" data-large-file="https://sw23993.files.wordpress.com/2017/07/rplot.png?w=456?w=592" alt="Rplot" data-image-title="Rplot" sizes="(max-width: 592px) 100vw, 592px" data-comments-opened="1" class="wp-image-1847" data-recalc-dims="1"></p>
<p>Upon visual inspection, the relationship appears to be linear, has a negative direction, and looks to be moderately strong. The strength of the relationship can be quantified using the Pearson correlation coefficient.</p>
<pre>cor(mtcars$disp, mtcars$mpg)
[1] -0.8475514</pre>
<p>This is a strong negative correlation. Note that<em><strong> correlation does not imply causation</strong></em>. It just indicates whether a mutual relationship, causal or not, exists between variables.</p>
<p>If the relationship is non-linear, a common approach in linear regression modelling is to <strong><em>transform</em> </strong>the response and predictor variable in order to coerce the relationship to one that is more linear. Common transformations include natural and base ten logarithmic, square root, cube root and inverse transformations. The mpg and disp relationship is already linear but it can be strengthened using a square root transformation.</p>
<pre>mtcars %>%
 ggplot(aes(x = sqrt(disp), y = sqrt(mpg))) +
 geom_point(colour = &quot;red&quot;) 

cor(sqrt(mtcars$disp), sqrt(mtcars$mpg))
[1] -0.8929046</pre>
<p> </p>
<p><img data-orig-file="https://sw23993.files.wordpress.com/2017/07/rplot01.png?w=456" data-orig-size="592,377" data-medium-file="https://sw23993.files.wordpress.com/2017/07/rplot01.png?w=456?w=300" data-attachment-id="1862" data-image-meta="{" data-image-description="" src="https://sw23993.files.wordpress.com/2017/07/rplot01.png?w=456" data-permalink="https://sw23993.wordpress.com/2017/07/04/ordinary-least-squares-ols-linear-regression-in-r/rplot01-6/" data-large-file="https://sw23993.files.wordpress.com/2017/07/rplot01.png?w=456?w=592" alt="Rplot01" data-image-title="Rplot01" sizes="(max-width: 592px) 100vw, 592px" data-comments-opened="1" class="wp-image-1862" data-recalc-dims="1"></p>
<p>The next step is to determine whether the relationship is <em><strong>statistically significant</strong> </em>and not just some random occurrence. This is done by investigating the variance of the data points about the fitted line. If the data fit well to the line, then the relationship is likely to be a real effect. The goodness of fit can be quantified using the <strong><em>root mean squared error</em></strong> (RMSE) and <strong><em>R-squared</em></strong> metrics. The RMSE represents the variance of the model errors and is an absolute measure of fit which has units identical to the response variable.<em> R</em>-squared is simply the Pearson correlation coefficient squared and represents variance explained in the response variable by the predictor variable.</p>
<p>The number of data points is also important and influences the <em><strong>p-value</strong></em> of the model. A rule of thumb for OLS linear regression is that at least 20 data points are required for a valid model. The <em>p</em>-value is the probability of there being no relationship (the null hypothesis) between the variables.</p>
<p>An OLS linear model is now fit to the transformed data.</p>
<pre>mtcars %>%
 ggplot(aes(x = sqrt(disp), y = sqrt(mpg))) +
 geom_point(colour = &quot;red&quot;) +
 geom_smooth(method = &quot;lm&quot;, fill = NA)</pre>
<p> </p>
<p><img data-orig-file="https://sw23993.files.wordpress.com/2017/07/rplot02.png?w=456" data-orig-size="592,377" data-medium-file="https://sw23993.files.wordpress.com/2017/07/rplot02.png?w=456?w=300" data-attachment-id="1886" data-image-meta="{" data-image-description="" src="https://sw23993.files.wordpress.com/2017/07/rplot02.png?w=456" data-permalink="https://sw23993.wordpress.com/2017/07/04/ordinary-least-squares-ols-linear-regression-in-r/rplot02-7/" data-large-file="https://sw23993.files.wordpress.com/2017/07/rplot02.png?w=456?w=592" alt="Rplot02" data-image-title="Rplot02" sizes="(max-width: 592px) 100vw, 592px" data-comments-opened="1" class="wp-image-1886" data-recalc-dims="1"></p>
<p>The model object can be created as follows.</p>
<pre>lmodel &lt;- lm(sqrt(mpg) ~ sqrt(disp), data = mtcars)</pre>
<p>The slope and the intercept can be obtained.</p>
<pre>lmodel$coefficients

(Intercept) sqrt(disp) 
 6.5192052 -0.1424601</pre>
<p>And the model summary contains the important statistical information.</p>
<pre>summary(lmodel)

Call:
lm(formula = sqrt(mpg) ~ sqrt(disp), data = mtcars)

Residuals:
 Min 1Q Median 3Q Max 
-0.45591 -0.21505 -0.07875 0.16790 0.71178

Coefficients:
 Estimate Std. Error t value Pr(>|t|) 
(Intercept) 6.51921 0.19921 32.73 &lt; 2e-16 ***
sqrt(disp) -0.14246 0.01312 -10.86 6.44e-12 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.3026 on 30 degrees of freedom
Multiple R-squared: 0.7973, Adjusted R-squared: 0.7905 
F-statistic: 118 on 1 and 30 DF, p-value: 6.443e-12</pre>
<p>The <em>p</em>-value of 6.443e-12 indicates a statistically significant relationship at the<em> p</em>&lt;0.001 cut-off level. The multiple <em>R</em>-squared value (<em>R</em>-squared) of 0.7973 gives the variance explained and can be used as a measure of predictive power (in the absence of overfitting). The RMSE is also included in the output (Residual standard error) where it has a value of 0.3026.</p>
<p>The take home message from the output is that for every unit increase in the square root of engine displacement there is a -0.14246 decrease in the square root of fuel efficiency (mpg). Therefore, fuel efficiency decreases with increasing engine displacement.</p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sw23993.wordpress.com/1802/" target="_blank"><img border="0" alt="" src="http://feeds.wordpress.com/1.0/comments/sw23993.wordpress.com/1802/"></a> <img border="0" alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" data-recalc-dims="1"></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://sw23993.wordpress.com/2017/07/04/ordinary-least-squares-ols-linear-regression-in-r/"> Environmental Science and Data Analytics</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/ZhQzf6e4o14" height="1"></a></div>