<span><span lang="en">It is important to mention that the present posts series began as a personal way of practicing R programming and machine learning. <span>Subsequently feedback from the community, urged me to continue performing these exercises and sharing them.</span><span> </span></span><span lang="en"><span>The bibliography and corresponding authors are cited at all times and this posts series is a way of honoring and giving them the credit they deserve for their work.</span></span></span><br><span><br></span><span><span lang="en"><span>We will develop a quadratic discriminant analysis and a K nearest neighbors</span></span><span lang="en"><span><span lang="en"><span><span lang="en"><span> examples.</span></span></span></span> The examples was originally published in <i>&quot;An Introduction to Statistical Learning. With applications in R&quot; </i>by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. Springer 2015.</span></span></span><br><br><div><a href="https://1.bp.blogspot.com/-_wt6CYdUTG4/WRsZSIHobbI/AAAAAAAADBY/R1hvwqYkp0MGB6VShlF3oOgxv7tWNoHhgCLcB/s1600/ml2.jpg"></a></div><div></div><br><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>The example we will develop is about c</span></span></span></span></span></span></span></span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>lassifying</span></span> when the market value will rise (UP) or fall (Down).</span></span></span></span></span></span></span></span><br> </span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>We will carry out the exercise verbatim as published in the aforementioned reference</span></span></span></span></span></span><span lang="en"><span> and o</span></span></span></span></span></span><span lang="en"><span>nly with slight changes in the coding style.</span></span><br><br><span lang="en"><span>For more details on the models, algorithms and parameters interpretation, it is recommended to check the aforementioned reference or any other bibliography of your choice.</span></span></span></span></span></span></span></span></span></span></span><br><br><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>### &quot;An Introduction to Statistical Learning.<br>### With applications in R&quot; by Gareth James,<br>### Daniela Witten, Trevor Hastie and Robert Tibshirani.<br>### Springer 2015.<br><br><br>### install and load required packages<br>library(ISLR)<br>library(psych)<br>library(MASS)<br>library(class)<br><br>### split the dataset into train and test sets<br>train &lt;- (Smarket$Year &lt; 2005)<br>Smarket.2005 &lt;- Smarket[!train, ]<br>Direction.2005 &lt;- Smarket$Direction[!train]<br><br>### Quadratic Discriminant Analysis<br><br>### perform quadratic discriminant analysis QDA on the stock<br>### market data<br>qda.fit &lt;- qda(Direction ~ Lag1 + Lag2, data = Smarket, subset = train)<br><br>### the output does not contain the coefficients of the linear discriminants,<br>### because the QDA classifier involves a quadratic, rather than a linear,<br>### function of the predictors<br>qda.fit<br><br>### predictions<br>qda.class &lt;- predict(qda.fit, Smarket.2005)$class<br>table(qda.class, Direction.2005)<br>mean(qda.class == Direction.2005)<br><br>### QDA predictions are accurate almost 60 % of the time,<br>### even though the 2005 data was not used to fit the model<br>### quite impressive for stock market data, which is known to<br>### be quite hard to model accurately<br><br>### K-Nearest Neighbors<br><br>### split the dataset into train and test sets<br>train.X &lt;- cbind(Smarket$Lag1, Smarket$Lag2)[train, ]<br>test.X &lt;- cbind(Smarket$Lag1, Smarket$Lag2)[!train, ]<br>train.Direction &lt;- Smarket$Direction[train]<br><br>### use knn() function can be used to predict the market’s movement<br>### for the dates in 2005<br>set.seed(1)<br>knn.pred &lt;- knn(train.X, test.X, train.Direction, k = 1)<br>table(knn.pred, Direction.2005)                   <br>(83+43)/252                   <br><br>### the results using K = 1 are not very good, since only 50 % of the<br>### observations are correctly predicted<br><br>### try k=3<br>knn.pred &lt;- knn(train.X, test.X, train.Direction, k = 3)<br>table(knn.pred, Direction.2005)<br>mean(knn.pred == Direction.2005)<br><br>### the results have improved slightly. But increasing K further</span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>### turns out to provide no further improvements. It appears that for ### this data, QDA provides the best results of the methods that we</span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span>### have examined so far.</span> </span></span></span></span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>You can get the example in:</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><a href="https://github.com/pakinja/Data-R-Value/tree/master/MachineLearning_Classification_StockMarketData_QDA_KNN" target="_blank">https://github.com/pakinja/Data-R-Value</a> </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><br>