<span><span lang="en">It is important to mention that the present posts series began as a personal way of practicing R programming and machine learning. <span>Subsequently feedback from the community, urged me to continue performing these exercises and sharing them.</span><span> </span></span></span><span><span lang="en"><span>The bibliography and corresponding authors are cited at all times and this posts series is a way of honoring and giving them the credit they deserve for their work.</span></span></span><br> <span><br></span><span><span lang="en"><span>We will develop a linear discriminant analysis</span></span><span lang="en"><span><span lang="en"><span><span lang="en"><span> example.</span></span></span></span> The exercise was originally published in <i>&quot;An Introduction to Statistical Learning. With applications in R&quot; </i>by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. Springer 2015.</span></span></span><span></span><br><span><br></span> <span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>The example we will develop is about c</span></span></span></span></span></span></span></span></span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>lassifying</span></span> when the market value will rise (UP) or fall (Down).</span></span></span></span></span></span></span></span></span><span><br></span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span> </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>We will carry out the exercise verbatim as published in the aforementioned reference</span></span></span></span></span></span><span lang="en"><span> and o</span></span></span></span></span></span><span lang="en"><span>nly with slight changes in the coding style.</span></span><br><br><span lang="en"><span>For more details on the models, algorithms and parameters interpretation, it is recommended to check the aforementioned reference or any other bibliography of your choice.</span></span></span></span></span></span></span></span></span></span></span><br><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span>### &quot;An Introduction to Statistical Learning.<br>### with applications in R&quot; by Gareth James,<br>### Daniela Witten, Trevor Hastie and Robert Tibshirani.<br>### Springer 2015.<br><br><br>### install and load required packages<br>library(ISLR)<br>library(psych)<br>library(MASS)<br><br>### perform linear discriminant analysis LDA on the stock<br>### market data<br>train &lt;- (Smarket$Year &lt; 2005)<br>Smarket.2005 &lt;- Smarket[!train, ]<br>Direction.2005 &lt;- Smarket$Direction[!train]<br><br>lda.fit &lt;- lda(Direction ~ Lag1 + Lag2 , data = Smarket , subset = train )<br>lda.fit</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><div><a href="https://2.bp.blogspot.com/-huw0v4iZRe0/WRnLwxn1DMI/AAAAAAAADAA/TQk8FkvaivEc2DTXrySn3Uynr8fc67DhgCLcB/s1600/LDA_1.png"></a></div><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span><br><br>### LDA indicates that 49.2% of training observations<br>### correspond to days during wich the market went down<br><br>### group means suggest that there is a tendency for the<br>### previous 2 days returns to be negative on days when the<br>### market increases, and a tendency for the previous days<br>### returns to be positive on days when the market declines<br><br>### coefficients of linear discriminants output provides the<br>### linear combination of Lag1 and Lag2 that are used to form<br>### the LDA decision rule<br><br>### if (−0.642 * Lag1 − 0.514 * Lag2) is large, then the LDA<br>### classifier will predict a market increase, and if it is<br>### small, then the LDA classifier will predict a market decline<br><br>### plot() function produces plots of the linear discriminants,<br>### obtained by computing (−0.642 * Lag1 − 0.514 * Lag2) for<br>### each of the training observations<br>plot (lda.fit)</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><div><a href="https://1.bp.blogspot.com/-VBXsaNwX_Wk/WRnL37NALJI/AAAAAAAADAE/wc1XlgKV0AU-uYf6Df3mjABV0k4OTH14ACLcB/s1600/LDA_2.png"></a></div><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span><br>### predictions</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span>### the LDA and logistic regression predictions are almost identical</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> <br>lda.pred &lt;- predict(lda.fit, Smarket.2005)<br>names(lda.pred)<br><br>lda.class &lt;- lda.pred$class<br>table(lda.class, Direction.2005)<br>mean(lda.class == Direction.2005)<br><br>### apply a 50% threshold to the posterior probabilities allows<br>### us to recreate the predictions in lda.pred$class<br>sum(lda.pred$posterior [ ,1] >= 0.5)<br>sum(lda.pred$posterior [ ,1] &lt; 0.5)<br><br>### posterior probability output by the model corresponds to<br>### the probability that the market will decrease<br>lda.pred$posterior[1:20 ,1]<br>lda.class[1:20]<br><br>### use a posterior probability threshold other than 50 % in order<br>### to make predictions<br><br>### suppose that we wish to predict a market decrease only if we<br>### are very certain that the market will indeed decrease on that<br>### day-say, if the posterior probability is at least 90%<br>sum(lda.pred$posterior[ ,1] > 0.9)<br><br>### No days in 2005 meet that threshold! In fact, the greatest<br>### posterior probability of decrease in all of 2005 was 52.02%</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>You can get the example in:</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><a target="_blank" href="https://github.com/pakinja/Data-R-Value/tree/master/MachineLearning_Classification_StockMarketData_LDA">https://github.com/pakinja/Data-R-Value</a> </span></span></span></span></span></span> </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span>  </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>