<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-6-linear-regression-model-in-r-eda-and-normality-tests/"></div></div><div><a data-url="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-6-linear-regression-model-in-r-eda-and-normality-tests/" data-counturl="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-6-linear-regression-model-in-r-eda-and-normality-tests/" data-count="horizontal" data-text="Introduction to R for Data Science :: Session 6 [Linear Regression Model in R  + EDA, and Normality Tests]" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.exactness.net/post/145521402750"> The Exactness of Mind</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Welcome to Introduction to R for Data Science Session 6: Linear Regression + EDA, and Normality tests [<i>Linear Regression in R: Exploratory Data Analysis, assumptions of the simple linear model, correlation, and visualization. Predictions from the linear model. Confidence Intervals and Residuals. Inspecting the basic linear model. Infulential cases and the Influence Plot.<i>]</i></i></p>
<p>The course is co-organized by <a target="_blank" href="http://www.datascience.rs/">Data Science Serbia</a> and <a target="_blank" href="http://en.startit.rs/">Startit</a>. You will find all course material (R scripts, data sets, SlideShare presentations, readings) on these pages.</p>
<p>Check out the <a target="_blank" href="http://www.exactness.net/post/143651799365/introduction-to-r-for-data-science">Course Overview</a> to acess the learning material presented thus far.</p>
<p><a target="_blank" href="http://www.datascience.rs/kurs-r-za-data-science/">Data Science Serbia Course Pages [in Serbian]</a></p>
<p><a target="_blank" href="http://startit.rs/uvod-u-r-za-data-science/">Startit Course Pages [in Serbian]</a></p>
<h2><b>Lecturers</b></h2>
<ul>
<li><a target="_blank" href="https://rs.linkedin.com/in/kovacbranko">dipl. ing Branko Kovač</a>, Data Analyst at <a target="_blank" href="http://cube.rs/">CUBE</a>, Data Science Mentor at <a target="_blank" href="https://www.springboard.com/">Springboard</a>, Data Science Serbia</li>
<li><a target="_blank" href="http://www.exactness.net/">Goran S. Milovanović, Phd</a>, <a target="_blank" href="http://www.diplomacy.edu/">DataScientist@DiploFoundation</a>, Data Science Serbia</li>
</ul>
<h2><b>Summary of Session 6, 02. June 2016 :: Linear Regression + EDA, and Normality tests.</b></h2>
<p><i>Linear Regression + EDA and Normality tests. Linear Regression in R: Exploratory Data Analysis, assumptions of the simple linear model, correlation, and visualization. Predictions from the linear model. Confidence Intervals and Residuals. Inspecting the basic linear model. Infulential cases and the Influence Plot.</i></p>
<ul>
<li><a target="_blank" href="http://www.exactness.net/post/145521402750#slides">Session 6 SlideShare</a></li>
<li><a target="_blank" href="http://www.exactness.net/post/145521402750#script">Session 6 R Script</a></li>
<li><a target="_blank" href="http://www.exactness.net/post/145521402750#readings">Readings for Session 7</a></li>
</ul>
<h2><b>Intro to R for Data Science SlideShare :: Session 6</b></h2>
<p><iframe height="485" width="595" allowfullscreen src="http://www.slideshare.net/slideshow/embed_code/key/N5vaVbT7FcdDKB"> </iframe>
<div> <strong> <a title="Introduction to R for Data Science :: Session 6 [Linear Regression in R]" target="_blank" href="http://www.slideshare.net/Krule/introduction-to-r-for-data-science-session-5-linear-regression-in-r">Introduction to R for Data Science :: Session 6 [Linear Regression in R] </a> </strong> from <strong><a target="_blank" href="http://www.slideshare.net/Krule">Goran S. Milovanovic</a></strong> </div>
<h2><b>R script :: Session 6</b></h2>
<pre>########################################################
# Introduction to R for Data Science
# SESSION 6 :: 2 June, 2016
# Simple Linear Regression in R
# Data Science Community Serbia + Startit
# :: Goran S. Milovanović and Branko Kovač ::
########################################################
 
# clear
<a target="_blank" href="http://inside-r.org/r-doc/base/rm">rm</a>(<a target="_blank" href="http://inside-r.org/r-doc/base/list">list</a>=<a target="_blank" href="http://inside-r.org/r-doc/base/ls">ls</a>())
 
#### read data
<a target="_blank" href="http://inside-r.org/r-doc/base/library">library</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets">datasets</a>)
<a target="_blank" href="http://inside-r.org/r-doc/utils/data">data</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>)
 
### iris data set description:
# <a target="_blank" href="https://stat.ethz.ch/R-manual/R-devel/library/iriss/html/iris.html">https://stat.ethz.ch/R-manual/R-devel/library/iriss/html/iris.html</a>
 
### Exploratory Data Analysis (EDA)
<a target="_blank" href="http://inside-r.org/r-doc/utils/str">str</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>)
<a target="_blank" href="http://inside-r.org/r-doc/base/summary">summary</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>)
 
### EDA plots
# plot layout: 2 x 2
<a target="_blank" href="http://inside-r.org/r-doc/graphics/par">par</a>(mfcol = <a target="_blank" href="http://inside-r.org/r-doc/base/c">c</a>(2,2))
# boxplot iris$Sepal.Length
<a target="_blank" href="http://inside-r.org/r-doc/graphics/boxplot">boxplot</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length,
        horizontal = TRUE, 
        xlab=&quot;Sepal Length&quot;)
# histogram: iris$Sepal.Length
<a target="_blank" href="http://inside-r.org/r-doc/graphics/hist">hist</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length, 
     main=&quot;&quot;, 
     xlab=&quot;Sepal.Length&quot;, 
     <a target="_blank" href="http://inside-r.org/packages/cran/prob">prob</a>=T)
# overlay iris$Sepal.Length density function over the empirical distribution
<a target="_blank" href="http://inside-r.org/r-doc/graphics/lines">lines</a>(<a target="_blank" href="http://inside-r.org/r-doc/stats/density">density</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length),
      lty=&quot;dashed&quot;, 
      lwd=2.5, 
      <a target="_blank" href="http://inside-r.org/r-doc/base/col">col</a>=&quot;red&quot;)
# boxplot iris$Petal.Length
<a target="_blank" href="http://inside-r.org/r-doc/graphics/boxplot">boxplot</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length,
        horizontal = TRUE, 
        xlab=&quot;Petal Length&quot;)
# histogram: iris$Petal.Length,
<a target="_blank" href="http://inside-r.org/r-doc/graphics/hist">hist</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length,
     main=&quot;&quot;, 
     xlab=&quot;Petal Length&quot;, 
     <a target="_blank" href="http://inside-r.org/packages/cran/prob">prob</a>=T)
# overlay iris$Petal.Length density function over the empirical distribution
<a target="_blank" href="http://inside-r.org/r-doc/graphics/lines">lines</a>(<a target="_blank" href="http://inside-r.org/r-doc/stats/density">density</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length),
      lty=&quot;dashed&quot;, 
      lwd=2.5, 
      <a target="_blank" href="http://inside-r.org/r-doc/base/col">col</a>=&quot;red&quot;)</pre>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="451" data-orig-width="500"><img alt="image" src="http://i2.wp.com/65.media.tumblr.com/44eea9c4f2e12bd8465a50d57b7be04e/tumblr_inline_o8d7c5HL1I1qa0hyw_540.png?resize=456%2C411" data-orig-height="451" data-recalc-dims="1" data-orig-width="500"></figure>
<pre># NOTE: Boxplot &quot;fences&quot; and outlier detection
# Boxplot in R recognizes as outliers those data points that are found beyond OUTTER fences
# Source: <a target="_blank" href="http://www.itl.nist.gov/div898/handbook/prc/section1/prc16.htm">http://www.itl.nist.gov/div898/handbook/prc/section1/prc16.htm</a>
# Q3 = 75 percentile, Q1 = 25 percentile
# IQ = Q3 - Q1; Interquartile range
# lower inner fence: Q1 - 1.5*IQ
# upper inner fence: Q3 + 1.5*IQ
# lower outer fence: Q1 - 3*IQ
# upper outer fence: Q3 + 3*IQ 
# A point beyond an inner fence on either side is considered a mild outlier
# A point beyond an outer fence is considered an extreme outlier
 
# plot variable density in general: Sepal Width
# plot layout
<a target="_blank" href="http://inside-r.org/r-doc/graphics/par">par</a>(mfcol = <a target="_blank" href="http://inside-r.org/r-doc/base/c">c</a>(1,2))
# NOTE: this is kernel density estimation in R. You are not testing any distribution yet.
PLengthDensity &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/density">density</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length)
<a target="_blank" href="http://inside-r.org/r-doc/graphics/plot">plot</a>(PLengthDensity, main=&quot;Petal Length Probability Density&quot;, 
     xlab = &quot;Petal Length&quot;,cex.main = .7)
<a target="_blank" href="http://inside-r.org/r-doc/graphics/polygon">polygon</a>(PLengthDensity, 
        <a target="_blank" href="http://inside-r.org/r-doc/base/col">col</a>=&quot;red&quot;, border=&quot;black&quot;) 
 
### Step 1: Pearson Correlation
## check whether the assumption of normality holds: Sepal Widht
# NOTE: check the documentation of the ks.test()
# URL: <a target="_blank" href="https://stat.ethz.ch/R-manual/R-devel/library/stats/html/ks.test.html">https://stat.ethz.ch/R-manual/R-devel/library/stats/html/ks.test.html</a>
ksPLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/ks.test">ks.test</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length,
                   &quot;pnorm&quot;,
                   <a target="_blank" href="http://inside-r.org/r-doc/base/mean">mean</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length),
                   <a target="_blank" href="http://inside-r.org/r-doc/stats/sd">sd</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length),
                   alternative = &quot;two.sided&quot;,
                   exact = NULL)
ksPLength # According to KS, Petal.Length is not Gaussian
# NOTE: If a single-sample test is used, the parameters specified in ... 
# must be pre-specified and not estimated from the data. {TRUE}
# Thus what we're doing here is not acceptable. Remember how many times have you seen
# the KS test applied in this manner? Imagine...
 
# NOTE: Gaussian Density in R
# All R distribution and density functions are similar
# dnorm(x, mean = 0, sd = 1, log = FALSE) for density (PDF)
# pnorm(q, mean = 0, sd = 1, lower.tail = TRUE, log.p = FALSE) for distribution function (CDF)
# NOTE: lower.tail = FALSE --> P[X>x], survivor (ie. decumulative probability) function
# qnorm(p, mean = 0, sd = 1, lower.tail = TRUE, log.p = FALSE) (quantiles)
# rnorm(n, mean = 0, sd = 1) # random deviates
 
# plot variable density in general: Sepal Length
SLengthDensity &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/density">density</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length)
<a target="_blank" href="http://inside-r.org/r-doc/graphics/plot">plot</a>(SLengthDensity, main=&quot;Sepal Length Probability Density&quot;, 
     xlab = &quot;Sepal Length&quot;, cex.main = .7)
<a target="_blank" href="http://inside-r.org/r-doc/graphics/polygon">polygon</a>(SLengthDensity, 
        <a target="_blank" href="http://inside-r.org/r-doc/base/col">col</a>=&quot;cadetblue&quot;, border=&quot;black&quot;)</pre>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="319" data-orig-width="500"><img alt="image" src="http://i1.wp.com/67.media.tumblr.com/5f918d19301584539bc66820e8fc0b94/tumblr_inline_o8d7c5vOME1qa0hyw_540.png?resize=456%2C291" data-orig-height="319" data-recalc-dims="1" data-orig-width="500"></figure>
<pre>## check whether the assumption of normality holds: Sepal Length
ksSLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/ks.test">ks.test</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length,
                   &quot;pnorm&quot;,
                   <a target="_blank" href="http://inside-r.org/r-doc/base/mean">mean</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length),
                   <a target="_blank" href="http://inside-r.org/r-doc/stats/sd">sd</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length),
                   alternative = &quot;two.sided&quot;,
                   exact = NULL)
ksSLength # KS test says this is a normal distribution
 
## AGAIN: check whether the assumption of normality holds: Petal.Lenght
# Shapiro-Wilk Test :: better for small samples + for Normal distribution only!
# NOTE: The most powerful statistical test given for a given alpha and compared
# to other tests similar in purpose!
swPLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/shapiro.test">shapiro.test</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length)
swPLength
# Null hypothesis: the data are normally distributed...
swPLength$statistic
swPLength$p # ... - while SW test says they are not normally distributed.
## Again: check whether the assumption of normality holds: Sepal Length
swSLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/shapiro.test">shapiro.test</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length)
swSLength
# Null hypothesis: the data are normally distributed...
swSLength$statistic
swSLength$p # ... - and to SW test they do not seem to to be.
 
# Take-home message: good luck with normality tests
 
# more plots to check the normality assumption
theoNormSLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/qnorm">qnorm</a>(<a target="_blank" href="http://inside-r.org/r-doc/stats/ppoints">ppoints</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length),
                       <a target="_blank" href="http://inside-r.org/r-doc/base/mean">mean</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length),
                       <a target="_blank" href="http://inside-r.org/r-doc/stats/sd">sd</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length))
# ?ppoints - produces evenly spaced probability points
<a target="_blank" href="http://inside-r.org/r-doc/stats/qqplot">qqplot</a>(theoNormSLength,<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length,
       main = &quot;Q-Q Plot: Sepal Length&quot;,
       xlab = &quot;Theoretical Quantiles&quot;,
       ylab = &quot;Empirical Quantiles&quot;,
       xlim = <a target="_blank" href="http://inside-r.org/r-doc/base/c">c</a>(<a target="_blank" href="http://inside-r.org/r-doc/base/min">min</a>(theoNormSLength),<a target="_blank" href="http://inside-r.org/r-doc/base/max">max</a>(theoNormSLength)),
       ylim = <a target="_blank" href="http://inside-r.org/r-doc/base/c">c</a>(<a target="_blank" href="http://inside-r.org/r-doc/base/min">min</a>(theoNormSLength),<a target="_blank" href="http://inside-r.org/r-doc/base/max">max</a>(theoNormSLength)),
       cex.main = .7
       )
<a target="_blank" href="http://inside-r.org/r-doc/graphics/abline">abline</a>(0,1,<a target="_blank" href="http://inside-r.org/r-doc/base/col">col</a>=&quot;red&quot;)</pre>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="328" data-orig-width="500"><img alt="image" src="http://i2.wp.com/66.media.tumblr.com/b0d0b8d9d7f298080884ee1481a5a6b4/tumblr_inline_o8d7c6mxyg1qa0hyw_540.png?resize=456%2C299" data-orig-height="328" data-recalc-dims="1" data-orig-width="500"></figure>
<pre># ... and more plots to check the normality assumption
theoNormPLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/qnorm">qnorm</a>(<a target="_blank" href="http://inside-r.org/r-doc/stats/ppoints">ppoints</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length),
                       <a target="_blank" href="http://inside-r.org/r-doc/base/mean">mean</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length), 
                       <a target="_blank" href="http://inside-r.org/r-doc/stats/sd">sd</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length))
<a target="_blank" href="http://inside-r.org/r-doc/stats/qqplot">qqplot</a>(theoNormPLength,<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length,
       main = &quot;Q-Q Plot: Petal Length&quot;,
       xlab = &quot;Theoretical Quantiles&quot;,
       ylab = &quot;Empirical Quantiles&quot;,
       xlim = <a target="_blank" href="http://inside-r.org/r-doc/base/c">c</a>(<a target="_blank" href="http://inside-r.org/r-doc/base/min">min</a>(theoNormPLength),<a target="_blank" href="http://inside-r.org/r-doc/base/max">max</a>(theoNormPLength)),
       ylim = <a target="_blank" href="http://inside-r.org/r-doc/base/c">c</a>(<a target="_blank" href="http://inside-r.org/r-doc/base/min">min</a>(theoNormPLength),<a target="_blank" href="http://inside-r.org/r-doc/base/max">max</a>(theoNormPLength)),
       cex.main = .7
)
<a target="_blank" href="http://inside-r.org/r-doc/graphics/abline">abline</a>(0,1,<a target="_blank" href="http://inside-r.org/r-doc/base/col">col</a>=&quot;cadetblue&quot;)</pre>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<pre>## Pearson correlation in R {base}
cor1 &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/cor">cor</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length, <a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length, 
            method=&quot;pearson&quot;)
cor1
<a target="_blank" href="http://inside-r.org/r-doc/graphics/par">par</a>(mfcol = <a target="_blank" href="http://inside-r.org/r-doc/base/c">c</a>(1,1))
<a target="_blank" href="http://inside-r.org/r-doc/graphics/plot">plot</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length, <a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length,
     main = &quot;Sepal Length vs Petal Length&quot;,
     xlab = &quot;Sepal Length&quot;, ylab = &quot;Petal Length&quot;)
 
## Correlation matrix and treatment of missing data
dSet &lt;- <a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>
# Remove one discrete variable
dSet$Species &lt;- NULL
# introduce NA in dSet$Sepal.Length[5]
dSet$Sepal.Length[5] &lt;- NA
 
# Pairwise and Listwise Deletion:
cor1a &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/cor">cor</a>(dSet,use=&quot;complete.obs&quot;) # listwise deletion
cor1a
cor1a &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/cor">cor</a>(dSet,use=&quot;pairwise.complete.obs&quot;) # pairwise deletion
cor1a
cor1a &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/cor">cor</a>(dSet,use=&quot;all.obs&quot;) # all observations - error
cor1a
 
# A fact about R: cor{base} does not test for statistical significance. True.
<a target="_blank" href="http://inside-r.org/r-doc/base/library">library</a>(<a target="_blank" href="http://inside-r.org/packages/cran/Hmisc">Hmisc</a>)
cor2 &lt;- rcorr(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length, 
              <a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length, 
              type=&quot;pearson&quot;)
cor2$r # correlations
cor2$r[1,2] # that's what you need, right
cor2$P # significant at
cor2$n # num. observations
# NOTE: rcorr uses Pairwise deletion!
 
# Correlation matrix
cor2a &lt;- rcorr(<a target="_blank" href="http://inside-r.org/r-doc/base/as.matrix">as.matrix</a>(dSet),
               type=&quot;pearson&quot;) # NOTE: as.matrix
# select significant at alpha == .05
w &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/which">which</a>(!(cor2a$P&lt;.05),arr.ind = T)
cor2a$r[w] &lt;- NA
cor2a$P # compare w.
cor2a$r
 
# But the assumption of normality seems to fail for one variable!
# Use Spearman's rho (essentially, Pearson's R over ranks):
cor2a &lt;- rcorr(<a target="_blank" href="http://inside-r.org/r-doc/base/as.matrix">as.matrix</a>(dSet),
               type=&quot;spearman&quot;) # NOTE: as.matrix
cor2a
 
# Linear Regression: lm()
# Predicting: Petal Length from Sepal Length
reg &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/lm">lm</a>(Petal.Length ~ Sepal.Length, <a target="_blank" href="http://inside-r.org/r-doc/utils/data">data</a>=<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>) 
<a target="_blank" href="http://inside-r.org/r-doc/base/class">class</a>(reg)
<a target="_blank" href="http://inside-r.org/r-doc/base/summary">summary</a>(reg)
coefsReg &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/coefficients">coefficients</a>(reg)
coefsReg
slopeReg &lt;- coefsReg[2]
interceptReg &lt;- coefsReg[1]
 
# Prediction from this model
newSLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/data.frame">data.frame</a>(Sepal.Length = <a target="_blank" href="http://inside-r.org/r-doc/stats/runif">runif</a>(100,
                                         <a target="_blank" href="http://inside-r.org/r-doc/base/min">min</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length), 
                                         <a target="_blank" href="http://inside-r.org/r-doc/base/max">max</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length))
                         ) # watch the variable names in the new data.frame!
predictPLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/predict">predict</a>(reg, newSLength)
predictPLength
 
# Results, more thoroughly
regSum &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/summary">summary</a>(reg)
regSum
 
# Coefficient of determination
regSum$r.squared
 
# Residuals
regSum$residuals
 
# Coefficients
regSum$coefficients
 
# F-statistic
regSum$fstatistic
 
# Confidence Intervals
<a target="_blank" href="http://inside-r.org/r-doc/stats/confint">confint</a>(reg, level=.95) # 95% CI
<a target="_blank" href="http://inside-r.org/r-doc/stats/confint">confint</a>(reg, level=.99) # 99% CI
# 95% CI for slope only
<a target="_blank" href="http://inside-r.org/r-doc/stats/confint">confint</a>(reg, &quot;price&quot;, level=.95)
 
# Standardized regression coefficients {QuantPsych}
<a target="_blank" href="http://inside-r.org/r-doc/base/library">library</a>(<a target="_blank" href="http://inside-r.org/packages/cran/QuantPsyc">QuantPsyc</a>)
lm.beta(reg)
 
# Reminder: standardized regression coefficients are...
# What you would obtain upon performing linear regression over standardized variables
# z-score in R
zSLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/scale">scale</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length, center = T, <a target="_blank" href="http://inside-r.org/r-doc/base/scale">scale</a> = T); # computes z-score
zPLength &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/scale">scale</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length, center = T, <a target="_blank" href="http://inside-r.org/r-doc/base/scale">scale</a> = T); # again; ?scale
# new dSet w. standardized variables
dSet &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/data.frame">data.frame</a>(Sepal.Length &lt;- zSLength,
                   Petal.Length &lt;- zPLength);
# Linear Regression w. lm() over standardized variables
reg1 &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/lm">lm</a>(Petal.Length ~ Sepal.Length, <a target="_blank" href="http://inside-r.org/r-doc/utils/data">data</a>=dSet) 
<a target="_blank" href="http://inside-r.org/r-doc/base/summary">summary</a>(reg1)
# compare
<a target="_blank" href="http://inside-r.org/r-doc/stats/coefficients">coefficients</a>(reg1)[2] # beta from reg1
lm.beta(reg) # standardized beta w. QuantPscy lm.beta from reg
 
# plots w. {base} and {ggplot2}
<a target="_blank" href="http://inside-r.org/r-doc/base/library">library</a>(<a target="_blank" href="http://inside-r.org/packages/cran/ggplot2">ggplot2</a>)
 
# Predictor vs Criterion {base}
<a target="_blank" href="http://inside-r.org/r-doc/graphics/plot">plot</a>(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Sepal.Length, <a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$Petal.Length,
     main = &quot;Petal Length vs Sepal Length&quot;,
     xlab = &quot;Sepal Length&quot;,
     ylab = &quot;Petal Length&quot;
     )
<a target="_blank" href="http://inside-r.org/r-doc/graphics/abline">abline</a>(reg,<a target="_blank" href="http://inside-r.org/r-doc/base/col">col</a>=&quot;red&quot;)</pre>
<figure data-orig-height="394" data-orig-width="500"><img alt="image" src="http://i0.wp.com/67.media.tumblr.com/ac4cd0b9236201372206b534900c193f/tumblr_inline_o8d7c9Aoqg1qa0hyw_540.png?resize=456%2C359" data-orig-height="394" data-recalc-dims="1" data-orig-width="500"></figure>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<pre># Predictor vs Criterion {ggplot2}
<a target="_blank" href="http://inside-r.org/packages/cran/ggplot">ggplot</a>(<a target="_blank" href="http://inside-r.org/r-doc/utils/data">data</a> = <a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>,
       aes(x = Sepal.Length, y = Petal.Length)) +
  geom_point(size = 2, colour = &quot;black&quot;) +
  geom_point(size = 1, colour = &quot;white&quot;) +
  geom_smooth(aes(colour = &quot;red&quot;),
              method='lm') +
  ggtitle(&quot;Sepal Length vs Petal Length&quot;) +
  xlab(&quot;Sepal Length&quot;) + ylab(&quot;Petal Length&quot;) +
  theme(legend.position = &quot;none&quot;)</pre>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="424" data-orig-width="500"><img alt="image" src="http://i1.wp.com/67.media.tumblr.com/73d44fd4f0dcb6ae77c8d35a361a31a3/tumblr_inline_o8d9ixLKNZ1qa0hyw_540.png?resize=456%2C387" data-orig-height="424" data-recalc-dims="1" data-orig-width="500"></figure>
<p># check normality of residuals</p>
<pre>resStReg &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/rstandard">rstandard</a>(reg) # get *standardized* residuals from reg
<a target="_blank" href="http://inside-r.org/r-doc/stats/qqnorm">qqnorm</a>(resStReg)
<a target="_blank" href="http://inside-r.org/r-doc/stats/qqline">qqline</a>(resStReg, <a target="_blank" href="http://inside-r.org/r-doc/base/col">col</a>=&quot;red&quot;)</pre>
<figure data-orig-height="424" data-orig-width="500"><img alt="image" src="http://i0.wp.com/67.media.tumblr.com/be38150811ebaeee5ffd85a7ee7274a3/tumblr_inline_o8d7c7MiJA1qa0hyw_540.png?resize=456%2C387" data-orig-height="424" data-recalc-dims="1" data-orig-width="500"></figure>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<pre># Predicted vs. residuals {ggplot2}
predReg &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/predict">predict</a>(reg) # get predictions from reg
resReg &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/residuals">residuals</a>(reg) # get residuals from reg
# resStReg &lt;- rstandard(reg) # get residuals from reg
plotFrame &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/data.frame">data.frame</a>(predicted = predReg,
                        residual = resReg);
<a target="_blank" href="http://inside-r.org/packages/cran/ggplot">ggplot</a>(<a target="_blank" href="http://inside-r.org/r-doc/utils/data">data</a> = plotFrame,
       aes(x = predicted, y = residual)) +
  geom_point(size = 2, colour = &quot;black&quot;) +
  geom_point(size = 1, colour = &quot;white&quot;) +
  geom_smooth(aes(colour = &quot;blue&quot;),
              method='lm',
              se=F) +
  ggtitle(&quot;Predicted vs Residual Lengths&quot;) +
  xlab(&quot;Predicted Lengths&quot;) + ylab(&quot;Residual&quot;) +
  theme(legend.position = &quot;none&quot;)</pre>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="424" data-orig-width="500"><img alt="image" src="http://i2.wp.com/66.media.tumblr.com/3fdc33de43189eb684ae6d33aa6dd079/tumblr_inline_o8d7cbrZLF1qa0hyw_540.png?resize=456%2C387" data-orig-height="424" data-recalc-dims="1" data-orig-width="500"></figure>
<pre>## Detect influential cases
infReg &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/as.data.frame">as.data.frame</a>(<a target="_blank" href="http://inside-r.org/r-doc/stats/influence.measures">influence.measures</a>(reg)$infmat)
 
# Cook's Distance: Cook and Weisberg (1982):
# values greater than 1 are troublesome
wCook &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/which">which</a>(infReg$cook.d>1) # we're fine here
 
# Average Leverage = (k+1)/n, k - num. of predictors, n - num. observations
# Also termed: hat values, range: 0 - 1
# see: <a target="_blank" href="https://en.wikipedia.org/wiki/Leverage_%28statistics%29">https://en.wikipedia.org/wiki/Leverage_%28statistics%29</a>
# Various criteria (twice the leverage, three times the average...)
wLev &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/which">which</a>(infReg$hat>2*(2/length(<a target="_blank" href="http://inside-r.org/r-doc/datasets/iris">iris</a>$price))) # we seem to be fine here too...
 
## Influence plot
infReg &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/as.data.frame">as.data.frame</a>(<a target="_blank" href="http://inside-r.org/r-doc/stats/influence.measures">influence.measures</a>(reg)$infmat)
plotFrame &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/data.frame">data.frame</a>(residual = resStReg,
                        leverage = infReg$hat,
                        cookD = infReg$cook.d)
<a target="_blank" href="http://inside-r.org/packages/cran/ggplot">ggplot</a>(plotFrame,
       aes(y = residual,
           x = leverage)) +
  geom_point(size = plotFrame$cookD*100, <a target="_blank" href="http://inside-r.org/packages/cran/shape">shape</a> = 1) +
  ggtitle(&quot;Influence Plot\nSize of the circle corresponds to Cook's distance&quot;) +
  theme(plot.title = element_text(size=8, face=&quot;bold&quot;)) +
  ylab(&quot;Standardized Residual&quot;) + xlab(&quot;Leverage&quot;)</pre>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="424" data-orig-width="500"><img alt="image" src="http://i0.wp.com/66.media.tumblr.com/d4a450348aaf7aad00290f27d0ed5752/tumblr_inline_o8d9khpsAZ1qa0hyw_540.png?resize=456%2C387" data-orig-height="424" data-recalc-dims="1" data-orig-width="500"></figure>
<pre># {broom} tools to access the linear regression model
<a target="_blank" href="http://inside-r.org/r-doc/base/library">library</a>(broom)
tidy(reg)
glance(reg)
augment(reg)
# deviance is another measure of goodness-of-fit - used w. MLE
# more on deviance in Session 8 on Logistic Regression</pre>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<h2><b>Readings :: Session 7: Multiple Regression [9. June, 2016, @Startit.rs, 19h CET]</b></h2>
<ul>
<li>
<a target="_blank" href="http://www.r-tutor.com/elementary-statistics/multiple-linear-regression"><i>R Tutorial</i>, by Chi Yau: <b>Chapter: Multiple Linear Regression</b></a>
</li>
<li>
More readings will be announced via the course mailing list.
</li>
</ul>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-6-linear-regression-model-in-r-eda-and-normality-tests/"></div></div><div><a data-url="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-6-linear-regression-model-in-r-eda-and-normality-tests/" data-counturl="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-6-linear-regression-model-in-r-eda-and-normality-tests/" data-count="vertical" data-text="Introduction to R for Data Science :: Session 6 [Linear Regression Model in R  + EDA, and Normality Tests]" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.exactness.net/post/145521402750"> The Exactness of Mind</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/S0XIGnYLRWg"></a></div>