<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/lasso-regression-in-r-exercises/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.r-exercises.com/2017/06/12/lasso-regression-in-r-exercises/?utm_source=rss&utm_medium=rss&utm_campaign=lasso-regression-in-r-exercises"> R-exercises</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a rel="nofollow" href="https://i0.wp.com/www.r-exercises.com/wp-content/uploads/2017/05/Rplot2_ex4.jpeg" target="_blank"><img alt="" src="https://i1.wp.com/www.r-exercises.com/wp-content/uploads/2017/06/Rplotset5.2.jpeg?resize=150%2C150" class="wp-image-5754" data-recalc-dims="1"></a></p>
<p>Lease Absolute Shrinkage and Selection Operator (LASSO) performs regularization and variable selection on a given model. Depending on the size of the penalty term, LASSO shrinks less relevant predictors to (possibly) zero. Thus, it enables us to consider a more parsimonious model. In this exercise set we will use the <code>glmnet</code> package (package description: <a rel="nofollow" href="https://cran.r-project.org/web/packages/glmnet/glmnet.pdf" target="_blank">here</a>) to implement LASSO regression in R.</p>
<p>Answers to the exercises are available <a rel="nofollow" href="http://r-exercises.com/2017/06/12/lasso-regression-in-r-solutions/" target="_blank">here</a>.</p>
<p><strong>Exercise 1</strong><br>
Load the <code>lars</code> package and the <code>diabetes</code> dataset (Efron, Hastie, Johnstone and Tibshirani (2003) “Least Angle Regression” Annals of Statistics). This has patient level data on the progression of diabetes. Next, load the <code>glmnet</code> package that will be used to implement LASSO.</p>
<p><strong>Exercise 2</strong><br>
The dataset has three matrices <code>x</code>, <code>x2</code> and <code>y</code>. While <code>x</code> has a smaller set of independent variables, <code>x2</code> contains the full set with quadratic and interaction terms. <code>y</code> is the dependent variable which is a quantitative measure of the progression of diabetes.<br>
It is a good idea to visually inspect the relationship of each of the predictors with the dependent variable. Generate separate scatterplots with the line of best fit for all the predictors in <code>x</code> with <code>y</code> on the vertical axis. Use a loop to automate the process.</p>
<p><strong>Exercise 3</strong><br>
Regress <code>y</code> on the predictors in <code>x</code> using OLS. We will use this result as benchmark for comparison.</p>
<p><strong>Exercise 4</strong><br>
Use the <code>glmnet</code> function to plot the path of each of x’s variable coefficients against the L1 norm of the beta vector. This graph indicates at which stage each coefficient shrinks to zero.</p>
<div>
<aside><img src="http://www.r-exercises.com/2017/06/12/lasso-regression-in-r-exercises/?utm_source=rss&utm_medium=rss&utm_campaign=lasso-regression-in-r-exercises"></aside>
<div><strong>Learn more</strong> about the glmnet package in the online course <a rel="nofollow" href="http://www.r-exercises.com/product/regression-machine-learning-with-r/" target="_blank">Regression Machine Learning with R</a>. In this course you will learn how to:
<ul>
<li>Avoid model over-fitting using cross-validation for optimal parameter selection</li>
<li>Explore maximum margin methods such as best penalty of error term support vector machines with linear and non-linear kernels.</li>
<li>And much more</li>
</ul>
</div>
</div>
<p><strong>Exercise 5</strong><br>
Use the <code>cv.glmnet</code> function to get the cross validation curve and the value of lambda that minimizes the mean cross validation error.</p>
<p><strong>Exercise 6</strong><br>
Using the minimum value of lambda from the previous exercise, get the estimated beta matrix. Note that some coefficients have been shrunk to zero. This indicates which predictors are important in explaining the variation in y.</p>
<p><strong>Exercise 7</strong><br>
To get a more parsimonious model we can use a higher value of lambda that is within one standard error of the minimum. Use this value of lambda to get the beta coefficients. Note that more coefficients are now shrunk to zero.</p>
<p><strong>Exercise 8</strong><br>
As mentioned earlier, <code>x2</code> contains a wider variety of predictors. Using OLS, regress <code>y</code> on <code>x2</code> and evaluate results.</p>
<p><strong>Exercise 9</strong><br>
Repeat exercise-4 for the new model.</p>
<p><strong>Exercise 10</strong><br>
Repeat exercises 5 and 6 for the new model and see which coefficients are shrunk to zero. This is an effective way to narrow down on important predictors when there are many candidates.</p>
<div>
<h3>Related exercise sets:</h3>
<ol>
<li><a rel="nofollow" href="http://www.r-exercises.com/2017/05/31/evaluate-your-model-with-r-exercises/" title="Evaluate your model with R Exercises" target="_blank">Evaluate your model with R Exercises </a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/2017/03/11/multiple-regression-part-3-diagnostics/" title="Multiple Regression (Part 3) Diagnostics" target="_blank">Multiple Regression (Part 3) Diagnostics </a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/2017/04/02/data-science-for-doctors-inferential-statistics-part4/" title="Data science for Doctors: Inferential Statistics Exercises(Part-4)" target="_blank">Data science for Doctors: Inferential Statistics Exercises(Part-4) </a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/start-here-to-learn-r/" target="_blank">Explore all our (>1000) R exercises</a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/r-courses/" target="_blank">Find an R course using our R Course Finder</a> directory</li>
</ol>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.r-exercises.com/2017/06/12/lasso-regression-in-r-exercises/?utm_source=rss&utm_medium=rss&utm_campaign=lasso-regression-in-r-exercises"> R-exercises</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...

	<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/Zpmv5PX8HZk" height="1"></a></div>