<span><span lang="en">It is important to mention that the present posts series began as a personal way of practicing R programming and machine learning. <span>Subsequently feedback from the community, urged me to continue performing these exercises and sharing them.</span><span> </span></span></span><br><span><span lang="en"><span>The bibliography and corresponding authors are cited at all times and this posts series is a way of honoring and giving them the credit they deserve for their work.</span></span></span><br><br><span><span lang="en"><span>We will develop a </span></span></span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span>logistic regression example.</span></span></span></span></span> The exercise was originally published in <i>&quot;An Introduction to Statistical Learning. With applications in R&quot; </i>by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. Springer 2015.</span></span></span><br><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span>The example we will develop is about predicting when the market value will rise (UP) or fall (Down).</span></span></span></span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>We will carry out the exercise verbatim as published in the aforementioned reference</span></span></span></span></span></span><span lang="en"><span> and o</span></span></span></span></span></span><span lang="en"><span>nly with slight changes in the coding style.</span></span></span><br><br><span><span lang="en"><span>For more details on the models, algorithms and parameters interpretation, it is recommended to check the aforementioned reference or any other bibliography of your choice.</span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>### &quot;An Introduction to Statistical Learning.<br>### With applications in R&quot; by Gareth James,<br>### Daniela Witten, Trevor Hastie and Robert Tibshirani.<br>### Springer 2015.<br><br><br>### install and load required packages<br>library(ISLR)<br>library(psych)<br><br>### explore the dataset<br>names(Smarket)<br>dim(Smarket)<br>summary(Smarket)<br><br>### correlation matrix<br>cor(Smarket[,-9])</span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>### correlations between th lag variables and today<br>### returns are close to zero<br>### the only substantial correlation is between $Year<br>### and $Volume<br>plot(Smarket$Volume, main= &quot;Stock Market Data&quot;,<br>     ylab = &quot;Volume&quot;)</span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><div><a href="https://3.bp.blogspot.com/-WbjKSFQQ6XA/WRilhIicHWI/AAAAAAAAC_g/_cLgEuWoDKkV8x9trD77LVgQpF3nS5gowCLcB/s1600/smd_1.png"></a></div><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><br> </span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>### scatterplots, distributions and correlations<br>pairs.panels(Smarket)</span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><br><div><a href="https://4.bp.blogspot.com/-fOqfF-34xZs/WRilo6w7dOI/AAAAAAAAC_k/XcLE0q6vaWkzPR2S2SLM0fUlHsrQdueHQCLcB/s1600/smd_2.png"></a></div><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><br></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><br>### fit a logistic regression model to predict $Direction<br>### using $Lag1 through $Lag5 and $Volume<br>### glm(): generalized linear model function<br>### family=binomial => logistic regression<br>glm.fit &lt;- glm(Direction~Lag1+Lag2+Lag3+Lag4+Lag5+Volume,<br>               data = Smarket, family = binomial)<br>summary(glm.fit)<br><br>### the smallest p_value is associated with Lag1<br>### the negative coefficient for this predictor suggests<br>### that if the market had a positive return yesterday,<br>### then it is less likely to go up today<br>### at a value of 0.15, the p-value is still relatively large,<br>### and so there is no clear evidence of a real association<br>### between $Lag1 and $Direction<br><br>### explore fitted model coefficients<br>coef(glm.fit)<br>summary(glm.fit)$coef<br>summary(glm.fit)$coef[ ,4]<br><br>### predict the probability that the market will go up,<br>### given values of the predictors<br>glm.probs &lt;- predict(glm.fit, type=&quot;response&quot;)<br>glm.probs[1:10]<br>contrasts(Smarket$Direction)<br><br>### these values correspond to the probability of the market<br>### going up, rather than down, because the contrasts()<br>### function indicates that R has created a dummy variable with<br>### a 1 for Up<br><br>### create a vector of class predictions based on whether the<br>### predicted probability of a market increase is greater than<br>### or less than 0.5<br>glm.pred &lt;- rep (&quot;Down&quot;, 1250)<br>glm.pred[glm.probs > .5] &lt;- &quot;Up&quot;<br><br>### confusion matrix in order to determine how many observations<br>### were correctly or incorrectly classified<br>table(glm.pred, Smarket$Direction)<br>mean(glm.pred == Smarket$Direction)<br><br>### model correctly predicted that the market would go up on 507<br>### days and that it would go down on 145 days, for a total of<br>### 507 + 145 = 652 correct predictions<br>### ogistic regression correctly predicted the movement of the<br>### market 52.2 % of the time<br><br>### to better assess the accuracy of the logistic regression model<br>### in this setting, we can fit the model using part of the data,<br>### and then examine how well it predicts the held out data<br>train &lt;- (Smarket$Year &lt; 2005)<br>Smarket.2005 &lt;- Smarket[!train, ]<br>dim(Smarket.2005)<br><br>Direction.2005 &lt;- Smarket$Direction[!train]<br><br>glm.fit &lt;- glm(Direction~Lag1+Lag2+Lag3+Lag4+Lag5+Volume,<br>               data = Smarket, family = binomial, subset = train)<br>glm.probs &lt;- predict(glm.fit, Smarket.2005, type = &quot;response&quot;)<br><br>### compute the predictions for 2005 and compare them to the</span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>### actual movements of the market over that time period<br>glm.pred &lt;- rep(&quot;Down&quot;, 252)<br>glm.pred[glm.probs > 0.5] &lt;- &quot;Up&quot;<br>table(glm.pred, Direction.2005)<br><br>mean(glm.pred == Direction.2005)<br>mean(glm.pred != Direction.2005)<br><br>### not generally expect to be able to use previous days returns ### to predict future market performance<br><br>### refit the logistic regression using just $Lag1 and $Lag2,</span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>### which seemed to have the highest predictive power in the</span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>### original logistic regression model<br>glm.fit &lt;- glm(Direction ~ Lag1 + Lag2 , data = Smarket,<br>               family = binomial, subset = train)<br>glm.probs &lt;- predict(glm.fit, Smarket.2005 , type = &quot;response&quot;)<br>glm.pred &lt;- rep(&quot;Down&quot;, 252)<br>glm.pred[glm.probs > 0.5] &lt;- &quot;Up&quot;<br>table(glm.pred, Direction.2005)<br>mean(glm.pred == Direction.2005)<br><br>### results appear to be a little better: 56%<br><br>### if we want to predict the returns associated with particular<br>### values of $Lag1 and $Lag2<br>predict(glm.fit, newdata = data.frame(Lag1 = c (1.2 ,1.5),<br>           Lag2 = c(1.1, -0.8)) , type = &quot;response&quot;)</span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><br><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>You can get the example in:</span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><a target="_blank" href="https://github.com/pakinja/Data-R-Value/tree/master/MachineLearning_Classification_StockMarketData">https://github.com/pakinja/Data-R-Value</a> </span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span> </span></span></span></span></span></span></span></span></span></span></span></span></span></span>