<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/how-to-go-about-interpreting-regression-cofficients/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://itsalocke.com/interpreting-regression-cofficients/"> R – Locke Data</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div></div>
<p>Following my post about <a target="_blank" rel="nofollow" href="http://itsalocke.com/logistic-regressions-r/">logistic regressions</a>, Ryan got in touch about one bit of building logistic regressions models that I didn’t cover in much detail – interpreting regression coefficients. This post will hopefully help Ryan (and others) out.</p>
<div class="twitter-tweet">
<blockquote data-lang="en" class="twitter-tweet">
<p dir="ltr" lang="en"><a target="_blank" rel="nofollow" href="https://twitter.com/SteffLocke">@SteffLocke</a> This was so helpful. Thank you! I'd love to see more about interpreting the glm coefficients.</p>
<p>— Ryan (@RyanEs) <a target="_blank" rel="nofollow" href="https://twitter.com/RyanEs/status/855499658398883840">April 21, 2017</a></p>
</blockquote>
</div>
<h2>What is a coefficient?</h2>
<p>Coefficients are what a line of best fit model produces. A line of best fit (aka regression) model usually consist of an intercept (where the line starts) and the gradients (or slope) for the line for one or more variables.</p>
<p>When we perform a linear regression in R, it’ll output the model and the coefficients.</p>
<pre><code>Call:
lm(formula = Sepal.Width ~ Sepal.Length + Petal.Width + Species, data = iris)

Coefficients:
(Intercept) Sepal.Length
1.9309 0.2730
Petal.Width Speciesversicolor
0.5307 -1.4850
Speciesvirginica
-1.8305
</code></pre>
<p>Each value represents the straight line relationship between each variable and the predicted variable. For every one unit of change for each input variable, the coefficient tells us how much we should change our predicted variable.</p>
<figure class="wp-caption"><img sizes="(max-width: 750px) 100vw, 750px" alt="Interpreting coefficients in regression" class="wp-image-62201" src="https://i2.wp.com/itsalocke.com/wp-content/uploads/2017/05/coefficients-2-1024x576.png?w=450" data-recalc-dims="1"><br>
<figcaption class="wp-caption-text">Interpreting coefficients in regression</figcaption>
</figure>
<h2>Interpreting coefficients</h2>
<p>When we read the list of coefficients, here is how we interpret them:</p>
<ul>
<li>The intercept is the starting point – so if you knew no other information it would be the best guess.</li>
<li>Each coefficient multiplies the corresponding column to refine the prediction from the estimate. It tells us how much one unit in each column shifts the prediction.</li>
<li>When you use a categorical variable, in R the intercept represents the default position for a given value in the categorical column. Every other value then gets a modifier to the base prediction.</li>
</ul>
<p>So for our linear regression of our iris data, we can say that:</p>
<ul>
<li>For setosas, with no other information, we expect a sepal width of 1.93cm</li>
<li>For versicolors, with no other information, we expect a sepal width of 0.54cm (1.93cm + -1.49cm)</li>
<li>For virginicas, with no other information, we expect a sepal width of 0.10cm (1.93cm + -1.83cm)</li>
<li>For every 1cm of sepal length, we increase our estimated sepal width by 0.27cm</li>
<li>For every 1cm of petal width, we increase our estimated sepal width by 0.53cm</li>
</ul>
<h2>Scaled variables</h2>
<p>When we use z-score scaling, the coefficients represent the impact of one standard deviation’s worth of change. As such, to determine the impact of one unit of change in the underlying variable you would divide the coefficient by the standard deviation of the underlying variable.</p>
<figure class="wp-caption"><img sizes="(max-width: 750px) 100vw, 750px" alt="How a variable gets scaled via a z-score" class="wp-image-62184" src="https://i2.wp.com/itsalocke.com/wp-content/uploads/2017/05/scaledvariable-1024x576.png?w=450" data-recalc-dims="1"><br>
<figcaption class="wp-caption-text">How a variable gets scaled via a z-score</figcaption>
</figure>
<h2>Logistic regression</h2>
<p>With a logistic regression, the outcome value is the logit, or log of the odds of an event happening. Any sum of values from the regression that is greater than 0 would represent classifying it as the thing we’re trying to predict. Interpreting logits is rather more complicated because we’re dealing with logarithms and odds.</p>
<blockquote><p>
Not all of the maths will be explained here. Check out <a target="_blank" rel="nofollow" href="http://www.win-vector.com/blog/2011/09/the-simpler-derivation-of-logistic-regression/">Win-vectors derivation of logistic regression</a> for a more proof-oriented read.
</p>
</blockquote>
<h3>How do odds work?</h3>
<p>An odds ratio is the probability of something happening versus it not happening i.e. <img title="frac{p}{1-p}" alt="frac{p}{1-p}" class="latex" src="http://s0.wp.com/latex.php?latex=%5Cfrac%7Bp%7D%7B1-p%7D&bg=ffffff&%23038;fg=000&%23038;s=0"></p>
<p>Odds can be expressed in nicer ratios, e.g. 2:1, where the sum of the two numbers represents the denominator for working out each probability. So it becomes (2/3):(1/3) and the odds would be <img title="frac{frac{2}{3}}{frac{1}{3}}" alt="frac{frac{2}{3}}{frac{1}{3}}" class="latex" src="http://s0.wp.com/latex.php?latex=%5Cfrac%7B%5Cfrac%7B2%7D%7B3%7D%7D%7B%5Cfrac%7B1%7D%7B3%7D%7D&bg=ffffff&%23038;fg=000&%23038;s=0"> or 2.</p>
<p>You convert from an odds ratio back to a probability by doing <img title="frac{o}{1+o}" alt="frac{o}{1+o}" class="latex" src="http://s0.wp.com/latex.php?latex=%5Cfrac%7Bo%7D%7B1%2Bo%7D&bg=ffffff&%23038;fg=000&%23038;s=0">.</p>
<blockquote><p>
Confirm it in R: <code>p=0.5; o=1; all.equal(p/(1-p), o); all.equal(o/(1+o), p)</code>
</p>
</blockquote>
<p>To combine two odds we multiply them. <img title="frac{p_1}{1-p_1} frac{p_2}{1-p_2}" alt="frac{p_1}{1-p_1} frac{p_2}{1-p_2}" class="latex" src="http://s0.wp.com/latex.php?latex=%5Cfrac%7Bp_1%7D%7B1-p_1%7D+%5Cfrac%7Bp_2%7D%7B1-p_2%7D&bg=ffffff&%23038;fg=000&%23038;s=0"></p>
<blockquote><p>
Confirm it in R: <code>p=0.875; oA=2; oB=4; oO=oA*oB; all.equal(p/(1-p), oO, (oA*oB)/(1+(oA*oB)))</code>
</p>
</blockquote>
<h3>Why do we do the logarithm?</h3>
<p>We do the logarithm in big part because it gives us a <img title="[-infty,+infty]" alt="[-infty,+infty]" class="latex" src="http://s0.wp.com/latex.php?latex=%5B-%5Cinfty%2C%2B%5Cinfty%5D&bg=ffffff&%23038;fg=000&%23038;s=0"> value to predict, but we can only do it because of a fundamental property of logarithms, namely, that adding to logarithms is equivalent to taking the logarithm of the multiplied values i.e. <img title="ln(a) + ln(b) = ln(a*b)" alt="ln(a) + ln(b) = ln(a*b)" class="latex" src="http://s0.wp.com/latex.php?latex=ln%28a%29+%2B+ln%28b%29+%3D+ln%28a%2Ab%29&bg=ffffff&%23038;fg=000&%23038;s=0"></p>
<blockquote><p>
Confirm it in R: <code>all.equal( log(2) + log(3) , log(2*3) )</code>
</p>
</blockquote>
<p>This means that when we sum the logit components in a logistic regression, we’re effectively taking the log of all the multiplied odds values.</p>
<h3>So how do I understand the impact?</h3>
<p>Firstly, it’s a logarithmic scale where a logit of 0 is 1:1 odds.</p>
<table>
<thead>
<tr>
<th align="right">logit</th>
<th align="right">odds</th>
<th align="left">oddsratio</th>
</tr>
</thead>
<tbody>
<tr>
<td align="right">-1.6094379</td>
<td align="right">0.2</td>
<td align="left">1:5</td>
</tr>
<tr>
<td align="right">-0.9162907</td>
<td align="right">0.4</td>
<td align="left">2:5</td>
</tr>
<tr>
<td align="right">-0.5108256</td>
<td align="right">0.6</td>
<td align="left">3:5</td>
</tr>
<tr>
<td align="right">-0.2231436</td>
<td align="right">0.8</td>
<td align="left">4:5</td>
</tr>
<tr>
<td align="right">0.0000000</td>
<td align="right">1.0</td>
<td align="left">5:5</td>
</tr>
<tr>
<td align="right">0.1823216</td>
<td align="right">1.2</td>
<td align="left">6:5</td>
</tr>
<tr>
<td align="right">0.3364722</td>
<td align="right">1.4</td>
<td align="left">7:5</td>
</tr>
<tr>
<td align="right">0.4700036</td>
<td align="right">1.6</td>
<td align="left">8:5</td>
</tr>
<tr>
<td align="right">0.5877867</td>
<td align="right">1.8</td>
<td align="left">9:5</td>
</tr>
<tr>
<td align="right">0.6931472</td>
<td align="right">2.0</td>
<td align="left">10:5</td>
</tr>
</tbody>
</table>
<p>If I had a coefficient of 0.69 for a variable in my model, it would be equivalent to every unit of change in that variable doubling the likelihood from the other variables.</p>
<p>If I had a coefficient of -1.6 for a variable in my model, it would be equivalent to every unit of change in that variable reducing the likelihood from the other variables by 5/6.</p>
<p>I made an R package, <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=optiRum"><code>optiRum</code></a>, that has helper functions for translating between probabilities, odds, and logits. Hopefully, it’ll help you out when needing to move between different measures and making them interpretable.</p>
<figure class="wp-caption"><img sizes="(max-width: 750px) 100vw, 750px" alt="Movement between probability, odds, and logit in logistic regression" class="wp-image-62185" src="https://i1.wp.com/itsalocke.com/wp-content/uploads/2017/05/transformation-1024x576.png?w=450" data-recalc-dims="1"><br>
<figcaption class="wp-caption-text">Movement between probability, odds, and logit in logistic regression</figcaption>
</figure>
<h2>Wrap up</h2>
<p>Hopefully, this has helped you become more comfortable interpreting regression coefficients. If you’d like to learn more about working with logistic regressions, check out my recent <a target="_blank" rel="nofollow" href="https://itsalocke.com/logistic-regressions-r/">logistic regressions (in R) post</a>. If you have any questions or comments, please add them below!</p>
<p>The post <a target="_blank" rel="nofollow" href="http://itsalocke.com/interpreting-regression-cofficients/">How to go about interpreting regression cofficients</a> appeared first on <a target="_blank" rel="nofollow" href="http://itsalocke.com/">Locke Data</a>. Locke Data are a data science consultancy aimed at helping organisations get ready and get started with data science.</p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://itsalocke.com/interpreting-regression-cofficients/"> R – Locke Data</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/r6eQu42S844"></a></div>