<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/risk-models-with-generalized-pls/"></div></div><div><a data-url="http://www.r-bloggers.com/risk-models-with-generalized-pls/" data-counturl="http://www.r-bloggers.com/risk-models-with-generalized-pls/" data-count="horizontal" data-text="Risk Models with Generalized PLS" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://statcompute.wordpress.com/2016/06/12/risk-models-with-generalized-pls/"> S+/R – Yet Another Blog in Statistical Computing</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>While developing risk models with hundreds of potential variables, we often run into the situation that risk characteristics or macro-economic indicators are highly correlated, namely multicollinearity. In such cases, we might have to drop variables with high VIFs or employ “variable shrinkage” methods, e.g. lasso or ridge, to suppress variables with colinearity. </p>
<p>Feature extraction approaches based on PCA and PLS have been widely discussed but are rarely used in real-world applications due to concerns around model interpretability and implementation. In the example below, it is shown that there shouldn’t any hurdle in the model implementation, e.g. score, given that coefficients can be extracted from a GPLS model in the similar way from a GLM model. In addition, compared with GLM with 8 variables, GPLS with only 5 components is able to provide a comparable performance in the hold-out testing data. </p>
<p><strong>R Code</strong></p>
<pre>
library(gpls)
library(pROC)

df1 &lt;- read.csv(&quot;credit_count.txt&quot;)
df2 &lt;- df1[df1$CARDHLDR == 1, -c(1, 10, 11, 12, 13)]
set.seed(2016)
n &lt;- nrow(df2)
sample &lt;- sample(seq(n), size = n / 2, replace = FALSE)
train &lt;- df2[sample, ]
test &lt;- df2[-sample, ]

m1 &lt;- glm(DEFAULT ~ ., data = train, family = &quot;binomial&quot;)
cat(&quot;\n### ROC OF GLM PREDICTION WITH TRAINING DATA ###\n&quot;)
print(roc(train$DEFAULT, predict(m1, newdata = train, type = &quot;response&quot;)))
cat(&quot;\n### ROC OF GLM PREDICTION WITH TESTING DATA ###\n&quot;)
print(roc(test$DEFAULT, predict(m1, newdata = test, type = &quot;response&quot;)))

m2 &lt;- gpls(DEFAULT ~ ., data = train, family = &quot;binomial&quot;, K.prov = 5)
cat(&quot;\n### ROC OF GPLS PREDICTION WITH TRAINING DATA ###\n&quot;)
print(roc(train$DEFAULT, predict(m2, newdata = train)$predicted[, 1]))
cat(&quot;\n### ROC OF GPLS PREDICTION WITH TESTING DATA ###\n&quot;)
print(roc(test$DEFAULT, predict(m2, newdata = test)$predicted[, 1]))

cat(&quot;\n### COEFFICIENTS COMPARISON BETWEEN GLM AND GPLS ###\n&quot;)
print(data.frame(glm = m1$coefficients, gpls = m2$coefficients))
</pre>
<p><strong>Output</strong></p>
<pre>
### ROC OF GLM PREDICTION WITH TRAINING DATA ###

Call:
roc.default(response = train$DEFAULT, predictor = predict(m1,     newdata = train, type = &quot;response&quot;))

Data: predict(m1, newdata = train, type = &quot;response&quot;) in 4753 controls (train$DEFAULT 0) &lt; 496 cases (train$DEFAULT 1).
Area under the curve: 0.6641

### ROC OF GLM PREDICTION WITH TESTING DATA ###

Call:
roc.default(response = test$DEFAULT, predictor = predict(m1,     newdata = test, type = &quot;response&quot;))

Data: predict(m1, newdata = test, type = &quot;response&quot;) in 4750 controls (test$DEFAULT 0) &lt; 500 cases (test$DEFAULT 1).
Area under the curve: 0.6537

### ROC OF GPLS PREDICTION WITH TRAINING DATA ###

Call:
roc.default(response = train$DEFAULT, predictor = predict(m2,     newdata = train)$predicted[, 1])

Data: predict(m2, newdata = train)$predicted[, 1] in 4753 controls (train$DEFAULT 0) &lt; 496 cases (train$DEFAULT 1).
Area under the curve: 0.6627

### ROC OF GPLS PREDICTION WITH TESTING DATA ###

Call:
roc.default(response = test$DEFAULT, predictor = predict(m2,     newdata = test)$predicted[, 1])

Data: predict(m2, newdata = test)$predicted[, 1] in 4750 controls (test$DEFAULT 0) &lt; 500 cases (test$DEFAULT 1).
Area under the curve: 0.6542

### COEFFICIENTS COMPARISON BETWEEN GLM AND GPLS ###
                      glm          gpls
(Intercept) -0.1940785071 -0.1954618828
AGE         -0.0122709412 -0.0147883358
ACADMOS      0.0005302022  0.0003671781
ADEPCNT      0.1090667092  0.1352491711
MAJORDRG     0.0757313171  0.0813835741
MINORDRG     0.2621574192  0.2547176301
OWNRENT     -0.2803919685 -0.1032119571
INCOME      -0.0004222914 -0.0004531543
LOGSPEND    -0.1688395555 -0.1525963363
</pre>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/statcompute.wordpress.com/1643/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/statcompute.wordpress.com/1643/" border="0"></a> <img height="1" alt="" width="1" src="https://pixel.wp.com/b.gif?host=statcompute.wordpress.com&blog=961631&%23038;post=1643&%23038;subd=statcompute&%23038;ref=&%23038;feed=1" border="0"></p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/risk-models-with-generalized-pls/"></div></div><div><a data-url="http://www.r-bloggers.com/risk-models-with-generalized-pls/" data-counturl="http://www.r-bloggers.com/risk-models-with-generalized-pls/" data-count="vertical" data-text="Risk Models with Generalized PLS" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://statcompute.wordpress.com/2016/06/12/risk-models-with-generalized-pls/"> S+/R – Yet Another Blog in Statistical Computing</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/f8hHpFccGIQ"></a></div>