<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/machine-learning-stock-market-data-part-3-quadratic-discriminant-analysis-and-knn/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://feedproxy.google.com/~r/DataRValue/~3/xvh5w6127wM/machine-learning-stock-market-data-part_16.html"> Data R Value</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><span><span lang="en">It is important to mention that the present posts series began as a personal way of practicing R programming and machine learning. <span>Subsequently feedback from the community, urged me to continue performing these exercises and sharing them.</span><span> </span></span><span lang="en"><span>The bibliography and corresponding authors are cited at all times and this posts series is a way of honoring and giving them the credit they deserve for their work.</span></span></span><br><span><br></span><span><span lang="en"><span>We will develop a quadratic discriminant analysis and a K nearest neighbors</span></span><span lang="en"><span><span lang="en"><span><span lang="en"><span> examples.</span></span></span></span> The examples was originally published in <b><i>“An Introduction to Statistical Learning. With applications in R” </i>by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. Springer 2015</b>.</span></span></span></p>
<div><a rel="nofollow" href="https://i1.wp.com/1.bp.blogspot.com/-_wt6CYdUTG4/WRsZSIHobbI/AAAAAAAADBY/R1hvwqYkp0MGB6VShlF3oOgxv7tWNoHhgCLcB/s1600/ml2.jpg?ssl=1" target="_blank"><img border="0" src="https://i0.wp.com/1.bp.blogspot.com/-_wt6CYdUTG4/WRsZSIHobbI/AAAAAAAADBY/R1hvwqYkp0MGB6VShlF3oOgxv7tWNoHhgCLcB/s320/ml2.jpg?resize=320%2C218&ssl=1" data-recalc-dims="1"></a></div>
<div></div>
<p><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>The example we will develop is about c</span></span></span></span></span></span></span></span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>lassifying</span></span> when the market value will rise (UP) or fall (Down).</span></span></span></span></span></span></span></span><br> </span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>We will carry out the exercise verbatim as published in the aforementioned reference</span></span></span></span></span></span><span lang="en"><span> and o</span></span></span></span></span></span><span lang="en"><span>nly with slight changes in the coding style.</span></span>
<p><span lang="en"><span>For more details on the models, algorithms and parameters interpretation, it is recommended to check the aforementioned reference or any other bibliography of your choice.</span></span></p></span>
<p><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><b>### “An Introduction to Statistical Learning.<br>### With applications in R” by Gareth James,<br>### Daniela Witten, Trevor Hastie and Robert Tibshirani.<br>### Springer 2015.
<p>### install and load required packages<br>library(ISLR)<br>library(psych)<br>library(MASS)<br>library(class)</p></b>
<p><b>### split the dataset into train and test sets</b><br>train &lt;-(Smarket$Year &lt; 2005)<br>Smarket.2005 &lt;-Smarket[!train, ]<br>Direction.2005 &lt;-Smarket$Direction[!train]<br><b><br>### Quadratic Discriminant Analysis
<p>### perform quadratic discriminant analysis QDA on the stock<br>### market data<br>qda.fit &lt;-qda(Direction ~ Lag1 + Lag2, data = Smarket, subset = train)</p></b>
<p><b>### the output does not contain the coefficients of the linear discriminants,<br>### because the QDA classifier involves a quadratic, rather than a linear,<br>### function of the predictors</b><br>qda.fit</p>
<p><b>### predictions</b><br>qda.class &lt;-predict(qda.fit, Smarket.2005)$class<br>table(qda.class, Direction.2005)<br>mean(qda.class == Direction.2005)</p>
<p><b>### QDA predictions are accurate almost 60 % of the time,<br>### even though the 2005 data was not used to fit the model<br>### quite impressive for stock market data, which is known to<br>### be quite hard to model accurately
<p>### K-Nearest Neighbors</p>
<p>### split the dataset into train and test sets<br>train.X &lt;-cbind(Smarket$Lag1, Smarket$Lag2)[train, ]<br>test.X &lt;-cbind(Smarket$Lag1, Smarket$Lag2)[!train, ]<br>train.Direction &lt;-Smarket$Direction[train]<br><b><br>### use knn() function can be used to predict the market’s movement<br>### for the dates in 2005</b><br>set.seed(1)<br>knn.pred &lt;-knn(train.X, test.X, train.Direction, k = 1)<br>table(knn.pred, Direction.2005)                   <br>(83+43)/252                   </p></b>
<p><b>### the results using K = 1 are not very good, since only 50 % of the<br>### observations are correctly predicted
<p>### try k=3<br>knn.pred &lt;-knn(train.X, test.X, train.Direction, k = 3)<br>table(knn.pred, Direction.2005)<br>mean(knn.pred == Direction.2005)</p></b>
<p><b>### the results have improved slightly. But increasing K further</b><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><b>### turns out to provide no further improvements. It appears that for ### this data, QDA provides the best results of the methods that we</b></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span><b>### have examined so far.</b></span> </span></span></span></span></span></span></span></span></span></span></span></p></span>
<p><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>You can get the example in:</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><span lang="en"><span><a rel="nofollow" href="https://github.com/pakinja/Data-R-Value/tree/master/MachineLearning_Classification_StockMarketData_QDA_KNN" target="_blank">https://github.com/pakinja/Data-R-Value</a> </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></p>
<p><img width="1" alt="" src="http://feeds.feedburner.com/~r/DataRValue/~4/xvh5w6127wM" height="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://feedproxy.google.com/~r/DataRValue/~3/xvh5w6127wM/machine-learning-stock-market-data-part_16.html"> Data R Value</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/R7cmyIyz6Jo" height="1"></a></div></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>