<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/assessing-the-accuracy-of-our-models-r-squared-adjusted-r-squared-rmse-mae-aic/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://r-video-tutorial.blogspot.com/2017/07/assessing-accuracy-of-our-models-r.html"> R tutorial for Spatial Statistics</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2>Assessing the accuracy of our model</h2>
<div>
<div>There are several ways to check the accuracy of our models, some are printed directly in R within the summary output, others are just as easy to calculate with specific functions.</div>
<div></div>
<div></div>
<h3>R-Squared</h3>
</div>
<div>
<div>This is probably the most commonly used statistics and allows us to understand the percentage of variance in the target variable explained by the model. It can be computed as a ratio of the regression sum of squares and the total sum of squares. This is one of the standard measures of accuracy that R prints out through the function summary for linear models and ANOVAs.</div>
<div></div>
<div></div>
<h3>Adjusted R-Squared</h3>
</div>
<div>
<div>This is a form of R-squared that is adjusted for the number of terms in the model. It can be computed as follows:</div>
<div><a rel="nofollow" href="https://i0.wp.com/2.bp.blogspot.com/-Qp_42uVuVOU/WVOkk-uaD9I/AAAAAAAAXQk/VDJTT2NxqUkwd16SKsZurq9aT59A24_OgCLcBGAs/s1600/Eq10.png?ssl=1" target="_blank"><img border="0" data-original-height="129" src="https://i0.wp.com/2.bp.blogspot.com/-Qp_42uVuVOU/WVOkk-uaD9I/AAAAAAAAXQk/VDJTT2NxqUkwd16SKsZurq9aT59A24_OgCLcBGAs/s640/Eq10.png?resize=450%2C129&ssl=1" data-recalc-dims="1" data-original-width="450"></a></div>
<div></div>
<div>Where R2 is the R-squared of the model, n is the sample size and p is the number of terms (or predictors) in the model. This index is extremely useful to determine possible overfitting in the model. This happens particularly when the sample size is small, in such cases if we fill the model with predictors we may end up increasing the R-squared simply because the model starts adapting to the noise in the data and not properly describing the data. </div>
<div></div>
<div></div>
<h3>Root Mean Squared Deviation or Root Mean Squared Error</h3>
</div>
<div>
<div>The previous indexes measure the amount of variance in the target variable that can be explained by our model. This is a good indication but in some cases we are more interested in quantifying the error in the same measuring unit of the variable. In such cases we need to compute indexes that average the residuals of the model. The problem is the residuals are both positive and negative and their distribution should be fairly symmetrical. This means that their average will always be zero. So we need to find other indexes to quantify the average residuals, for example by averaging the squared residuals:</div>
<div><a rel="nofollow" href="https://i0.wp.com/3.bp.blogspot.com/-1N_pvQGyNqs/WVOk1b-TmZI/AAAAAAAAXQo/IA_EdL3QRt89SRDKyVTWL39uHY2Ik1nBQCLcBGAs/s1600/Eq11.png?ssl=1" target="_blank"><img border="0" data-original-height="172" src="https://i2.wp.com/3.bp.blogspot.com/-1N_pvQGyNqs/WVOk1b-TmZI/AAAAAAAAXQo/IA_EdL3QRt89SRDKyVTWL39uHY2Ik1nBQCLcBGAs/s640/Eq11.png?resize=450%2C172&ssl=1" data-recalc-dims="1" data-original-width="450"></a></div>
<div></div>
<div>This is the square root of the mean the squared residuals, with <span></span> being the estimated value at point t, <span></span> being the observed value in t and <span></span> being the sample size. The RMSE has the same measuring unit of the variable y.</div>
<div></div>
<div></div>
<h3>Mean Squared Deviation or Mean Squared Error</h3>
</div>
<div>This is simply the numerator of the previous equation, but it is not used often. The issue with both the RMSE and the MSE is that since they square the residuals they tend to be more affected by large residuals. This means that even if our model explains the large majority of the variation in the data very well, with few exceptions; these exceptions will inflate the value of RMSE.</div>
<div></div>
<div></div>
<div></div>
<h3>Mean Absolute Deviation or Mean Absolute Error</h3>
<div>
<div>To solve the problem with large residuals we can use the mean absolute error, where we average the absolute value of the residuals:</div>
<div><a rel="nofollow" href="https://i1.wp.com/1.bp.blogspot.com/-5pkJOaONGjM/WVOlLlRQneI/AAAAAAAAXQs/-KXevZgMNoIKnM6Tx9PUCL9L6nPTfoYPgCLcBGAs/s1600/Eq12.png?ssl=1" target="_blank"><img border="0" data-original-height="124" src="https://i2.wp.com/1.bp.blogspot.com/-5pkJOaONGjM/WVOlLlRQneI/AAAAAAAAXQs/-KXevZgMNoIKnM6Tx9PUCL9L6nPTfoYPgCLcBGAs/s640/Eq12.png?resize=450%2C124&ssl=1" data-recalc-dims="1" data-original-width="450"></a></div>
<div>This index is more robust against large residuals. Since RMSE is still widely used, even though its problems are well known, it is always better calculate and present both in a research paper.</div>
<div></div>
<div></div>
<div></div>
<h3>Akaike Information Criterion</h3>
</div>
<div>
<div>This index is another popular index we have used along the text to compare different models. It is very popular because it corrects the RMSE for the number of predictors in the model, thus allowing to account for overfitting. It can be simply computed as follows:</div>
<div></div>
<div><a rel="nofollow" href="https://i1.wp.com/1.bp.blogspot.com/-ppqORBZ87as/WVOlcbQ4keI/AAAAAAAAXQw/EIM1N146Zi85fLz5j-_U6URtliWbTTnLACLcBGAs/s1600/Eq13.png?ssl=1" target="_blank"><img border="0" data-original-height="81" src="https://i1.wp.com/1.bp.blogspot.com/-ppqORBZ87as/WVOlcbQ4keI/AAAAAAAAXQw/EIM1N146Zi85fLz5j-_U6URtliWbTTnLACLcBGAs/s640/Eq13.png?resize=450%2C81&ssl=1" data-recalc-dims="1" data-original-width="450"></a></div>
<div></div>
<div>Where again p is the number of terms in the model.</div>
</div>
<div></div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://r-video-tutorial.blogspot.com/2017/07/assessing-accuracy-of-our-models-r.html"> R tutorial for Spatial Statistics</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/_QLIZKC6Cgw" height="1"></a></div>