<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/glmnetutils-quality-of-life-enhancements-for-elastic-net-regression-with-glmnet/"></div></div><div><a data-url="https://www.r-bloggers.com/glmnetutils-quality-of-life-enhancements-for-elastic-net-regression-with-glmnet/" data-counturl="https://www.r-bloggers.com/glmnetutils-quality-of-life-enhancements-for-elastic-net-regression-with-glmnet/" data-count="horizontal" data-text="glmnetUtils: quality of life enhancements for elastic net regression with glmnet" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2016/11/glmnetutils.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>The <a target="_blank" rel="nofollow" href="https://github.com/hong-revo/glmnetUtils">glmnetUtils package</a> provides a collection of tools to streamline the process of fitting <a title="Elastic net regularization" target="_blank" rel="nofollow" href="http://en.wikipedia.org/wiki/Elastic_net_regularization">elastic net</a> models with <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=glmnet">glmnet</a>. I wrote the package after a couple of projects where I found myself writing the same boilerplate code to convert a data frame into a predictor matrix and a response vector. In addition to providing a formula interface, it also has a function (<code>cvAlpha.glmnet</code>) to do crossvalidation for both elastic net parameters α and λ, as well as some utility functions.</p>
<h2>The formula interface</h2>
<p>The interface that glmnetUtils provides is very much the same as for most modelling functions in R. To fit a model, you provide a formula and data frame. You can also provide any arguments that glmnet will accept. Here is a simple example:</p>
<div>
<pre>mtcarsMod &lt;- glmnet(mpg ~ cyl + disp + hp, data=mtcars)

<span>## Call:</span>
<span>## glmnet.formula(formula = mpg ~ cyl + disp + hp, data = mtcars)</span>
<span>## </span>
<span>## Model fitting options:</span>
<span>## Sparse model matrix: FALSE</span>
<span>## Use model.frame: FALSE</span>
<span>## Alpha: 1</span>
<span>## Lambda summary:</span>
<span>## Min. 1st Qu. Median Mean 3rd Qu. Max. </span>
<span>## 0.03326 0.11690 0.41000 1.02800 1.44100 5.05500</span>
</pre>
</div>
<p>Under the hood, glmnetUtils creates a model matrix and response vector, and passes them to the glmnet package to do the actual model fitting. Prediction also works as you'd expect: just pass a data frame containing the new observations, along with any arguments that <code>predict.glmnet</code> needs.</p>
<div>
<pre><span># least squares regression: get predictions for lambda=1</span>
predict(mtcarsMod, newdata=mtcars, s=1)
</pre>
</div>
<h2>Building the model matrix</h2>
<p>You may have noticed the options &quot;use model.frame&quot; and &quot;sparse model matrix&quot; in the printed output above. glmnetUtils includes a couple of options to improve performance, especially on wide datasets and/or have many categorical (factor) variables.</p>
<p>The standard R method for creating a model matrix out of a data frame uses the <code>model.frame</code> function, which has a major disadvantage when it comes to wide data. It generates a <em>terms</em> object, which specifies how the original columns of data relate to the columns in the model matrix. This involves creating and storing a (roughly) square matrix of size <em>p</em> × <em>p</em>, where <em>p</em> is the number of variables in the model. When <em>p</em> > 10000, which isn't uncommon these days, the terms object can exceed a gigabyte in size. Even if there is enough memory to store the object, processing it can be very slow.</p>
<p>Another issue with the standard approach is the treatment of factors. Normally, <code>model.matrix</code> will turn an <em>N</em>-level factor into an indicator matrix with <em>N</em>−1 columns, with one column being dropped. This is necessary for unregularised models as fit with <code>lm</code> and <code>glm</code>, since the full set of <em>N</em> columns is linearly dependent. However, this may not be appropriate for a regularised model as fit with glmnet. The regularisation procedure shrinks the coefficients towards zero, which forces the estimated differences from the baseline to be smaller. But this only makes sense if the baseline level was chosen beforehand, or is otherwise meaningful as a default; otherwise it is effectively making the levels more similar to an arbitrarily chosen level.</p>
<p>To deal with these problems, glmnetUtils by default will avoid using <code>model.frame</code>, instead building up the model matrix term-by-term. This avoids the memory cost of creating a terms object, and can be much faster than the standard approach. It will also include one column in the model matrix for all levels in a factor; that is, no baseline level is assumed. In this situation, the coefficients represent differences from the overall mean response, and shrinking them to zero <em>is</em> meaningful (usually). Machine learners may also recognise this as <a target="_blank" rel="nofollow" href="http://stackoverflow.com/questions/17469835/one-hot-encoding-for-machine-learning">one-hot encoding</a>.</p>
<p>glmnetUtils can also generate a <em>sparse</em> model matrix, using the <code>sparse.model.matrix</code> function provided in the Matrix package. This works exactly the same as a regular model matrix, but takes up significantly less memory if many of its entries are zero. A scenario where this is the case would be where many of the predictors are factors, each with a large number of levels.</p>
<h2>Crossvalidation for α</h2>
<p>One piece missing from the standard glmnet package is a way of choosing α, the elastic net mixing parameter, similar to how <code>cv.glmnet</code> chooses λ, the shrinkage parameter. To fix this, glmnetUtils provides the <code>cvAlpha.glmnet</code> function, which uses crossvalidation to examine the impact on the model of changing α and λ. The interface is the same as for the other functions:</p>
<div>
<pre><span># Leukemia dataset from Trevor Hastie's website:</span>
<span># http://web.stanford.edu/~hastie/glmnet/glmnetData/Leukemia.RData</span>
load(<span>&quot;~/Leukemia.rdata&quot;</span>)
leuk &lt;- do.call(data.frame, Leukemia)

cvAlpha.glmnet(y ~ ., data=leuk, family=<span>&quot;binomial&quot;</span>)

<span>## Call:</span>
<span>## cvAlpha.glmnet.formula(formula = y ~ ., data = leuk, family = &quot;binomial&quot;)</span>
<span>## </span>
<span>## Model fitting options:</span>
<span>## Sparse model matrix: FALSE</span>
<span>## Use model.frame: FALSE</span>
<span>## Alpha values: 0 0.001 0.008 0.027 0.064 0.125 0.216 0.343 0.512 0.729 1</span>
<span>## Number of crossvalidation folds for lambda: 10</span>
</pre>
</div>
<p><code>cvAlpha.glmnet</code> uses the algorithm described in the help for <code>cv.glmnet</code>, which is to fix the distribution of observations across folds and then call <code>cv.glmnet</code> in a loop with different values of α. Optionally, you can parallelise this outer loop, by setting the <code>outerParallel</code> argument to a non-NULL value. Currently, glmnetUtils supports the following methods of parallelisation:</p>
<ul>
<li>Via <code>parLapply</code> in the parallel package. To use this, set <code>outerParallel</code> to a valid cluster object created by<code>makeCluster</code>.</li>
<li>Via <code>rxExec</code> as supplied by <a target="_blank" rel="nofollow" href="https://www.microsoft.com/en-au/cloud-platform/r-server">Microsoft R Server’s</a> RevoScaleR package. To use this, set <code>outerParallel</code> to a valid compute context created by <code>RxComputeContext</code>, or a character string specifying such a context.</li>
</ul>
<h2>Conclusion</h2>
<p>The glmnetUtils package is a way to improve quality of life for users of glmnet. As with many R packages, it’s always under development; you can get the latest version from my <a target="_blank" rel="nofollow" href="https://github.com/Hong-Revo/glmnetUtils">GitHub repo</a>. The easiest way to install it is via <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=devtools">devtools</a>:</p>
<div>
<pre>library(devtools)
install_github(<span>&quot;hong-revo/glmnetUtils&quot;</span>)
</pre>
</div>
<p>A more detailed version of this post can also be found at the package vignette. If you find a bug, or if you want to suggest improvements to the package, please feel free to contact me at <a target="_blank" rel="nofollow" href="mailto:hongooi@microsoft.com">hongooi@microsoft.com</a>.</p>
</div>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/glmnetutils-quality-of-life-enhancements-for-elastic-net-regression-with-glmnet/"></div></div><div><a data-url="https://www.r-bloggers.com/glmnetutils-quality-of-life-enhancements-for-elastic-net-regression-with-glmnet/" data-counturl="https://www.r-bloggers.com/glmnetutils-quality-of-life-enhancements-for-elastic-net-regression-with-glmnet/" data-count="vertical" data-text="glmnetUtils: quality of life enhancements for elastic net regression with glmnet" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2016/11/glmnetutils.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/fEedJonL954"></a></div>