<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/sp500-dashboard/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://nycdatascience.com/blog/student-works/sp500-dashboard/"> R – NYC Data Science Academy Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><img sizes="(max-width: 1024px) 100vw, 1024px" src="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/business-2904768_1920-1024x678.jpg?w=450&ssl=1" alt="S&amp;P500 Dashboard" class="wp-post-image" data-recalc-dims="1"></p>
<p>This is my Shiny Dashboard for stocks listed on S&amp;P500. Using R shiny to give a visualized general insight of market and using ARIMA model to analyze risk for a single stock.</p>
<hr>
<h1><span><strong>Background</strong></span></h1>
<p>When I worked as a trader. I found that it was really inconvenient to do the daily market research because the information on the mainstream website could be confusing and overwhelming sometime. As a result, it was difficult to find a single website that can give me all the information that I prefer to use. So, I usually had to check many different websites at the same time to extract the information that I was interested in. Even though I successfully finished gathering all information together, I might not be able to get a big picture of the whole market because there were so many noises in the information I gathered. So, I made this dashboard which can help me to extract information and do the basic fundamental analysis for me.</p>
<h1><span><strong>Introduction</strong></span></h1>
<p>This is a dashboard for the S&amp;P 500 stock market to give the user a general insight of stock market. And it is based on the  R studio Shiny package.  The main dataset comes from <a rel="nofollow" href="https://www.kaggle.com/camnugent/sandp500" target="_blank">Kaggle</a>. What’s more, I use <a rel="nofollow" href="https://www.alphavantage.co/" target="_blank">Alpha Vantage</a> API to get the price movement of sector ETFs listed on SPDR. In order to process the dataset, I use dplyr and tidyr. Besides, in order to visualize market major trend, I also use ggplot2, plotly, word cloud, and googlevis. Then, I use ARIMA model to do price risk analysis. If you are interested in, this is the link to my <a rel="nofollow" href="https://github.com/Zheeee/Shiny_SP500_Dashboard" target="_blank">Github</a>, and this is the output of my <a rel="nofollow" href="https://zheyang.shinyapps.io/ShinyProject/" target="_blank">ShinyApp.IO</a>.</p>
<h1><span><strong>App Detail</strong></span></h1>
<h2><span><b>Word Cloud</b></span></h2>
<p><a rel="nofollow" href="https://i2.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.17.45-PM.png" target="_blank"><img sizes="(max-width: 423px) 100vw, 423px" src="https://i1.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.17.45-PM-300x188.png?resize=423%2C265" alt="" width="423" class="wp-image-27966" data-recalc-dims="1" height="265"></a></p>
<p>Based on web scraping and word cloud visualization, this page can give users the hot words that were discussed on any date that has Nasdaq historical news page. Users only need to copy the URL of Nasdaq news page and paste in the URL input. The word cloud will change automatically.</p>
<h2><span><strong>Main tab</strong></span></h2>
<p><a rel="nofollow" href="https://i2.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.29.48-PM.png" target="_blank"><img sizes="(max-width: 423px) 100vw, 423px" src="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.29.48-PM-300x188.png?resize=423%2C265" alt="" width="423" class="wp-image-27969" data-recalc-dims="1" height="265"></a></p>
<p>There are four plots on this page. All of them are based on the sector user choose. Following will introduce the details of each plot.</p>
<p><span><strong>1.</strong></span><strong>Treemap</strong></p>
<table width="450">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.38.00-PM.png" target="_blank"><img sizes="(max-width: 234px) 100vw, 234px" src="https://i1.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.38.00-PM-300x185.png?resize=234%2C144" alt="" width="234" class="wp-image-27971" data-recalc-dims="1" height="144"></a></td>
<td><a rel="nofollow" href="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.39.05-PM.png" target="_blank"><img sizes="(max-width: 243px) 100vw, 243px" src="https://i2.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.39.05-PM-300x181.png?resize=243%2C147" alt="" width="243" class="wp-image-27972" data-recalc-dims="1" height="147"></a></td>
</tr>
</tbody>
</table>
<p> </p>
<p>Users can choose the date they are interested in by using the date selector in the sidebar.The size of each node is based on the volume of the ETF or stock. And the color means the gain or loss of each ETF or stock. Users can use click on ETF nodes to zoom in and check the stocks that are listed in the selected sector.</p>
<p><strong>2.Barplot and Heatmap</strong></p>
<table width="450">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i2.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.44.38-PM.png" target="_blank"><img sizes="(max-width: 297px) 100vw, 297px" src="https://i1.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.44.38-PM-295x300.png?resize=297%2C302" alt="" width="297" class="wp-image-27974" data-recalc-dims="1" height="302"></a></td>
<td><a rel="nofollow" href="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.45.24-PM.png" target="_blank"><img sizes="(max-width: 295px) 100vw, 295px" src="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.45.24-PM-294x300.png?resize=295%2C301" alt="" width="295" class="wp-image-27975" data-recalc-dims="1" height="301"></a></td>
</tr>
</tbody>
</table>
<p> </p>
<p>Users can choose the indicator and the sector they are interested in and plot on the bar plot. Both of sector and indicator can be chosen in the sidebar. At the same time, the correlation between each stock and the SPY will be plot as a heat map.</p>
<p><strong>3.Price movement</strong></p>
<p><a rel="nofollow" href="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.50.29-PM.png" target="_blank"><img sizes="(max-width: 646px) 100vw, 646px" src="https://i2.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.50.29-PM-300x97.png?w=450" alt="" class="wp-image-27976" data-recalc-dims="1"></a></p>
<p>Users can choose the stock they are interested in and plot them as candlestick on these two plots. Users can also select the two SMA they prefer to use.</p>
<h2>Time Series Analysis</h2>
<table width="450">
<tbody>
<tr>
<td><img sizes="(max-width: 300px) 100vw, 300px" src="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.59.13-PM-300x188.png?resize=300%2C188" alt="" width="300" class="wp-image-27984" data-recalc-dims="1" height="188"></td>
<td><img sizes="(max-width: 300px) 100vw, 300px" src="https://i2.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.59.16-PM-300x188.png?resize=300%2C188" alt="" width="300" class="wp-image-27983" data-recalc-dims="1" height="188"></td>
</tr>
<tr>
<td><a rel="nofollow" href="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.59.17-PM.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i2.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.59.17-PM-300x188.png?resize=300%2C188" alt="" width="300" class="wp-image-27982" data-recalc-dims="1" height="188"></a></td>
<td><a rel="nofollow" href="https://i0.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.59.19-PM.png" target="_blank"><img sizes="(max-width: 300px) 100vw, 300px" src="https://i1.wp.com/nycdatascience.com/blog/wp-content/uploads/2018/02/Screen-Shot-2018-02-03-at-10.59.19-PM-300x188.png?resize=300%2C188" alt="" width="300" class="wp-image-27981" data-recalc-dims="1" height="188"></a></td>
</tr>
</tbody>
</table>
<p>Time series analysis for the stock users chose. Using three parameters of ARIMA model to make visualized model test process. I include basic tests for ARIMA model. They are Box-test and Shapiro-test for the residuals of the model. By default, users can use 200 points to construct the model and use the rest 50 points to test model. The result of this accuracy test is displayed by using MAPE, MPE etc indicators. Users can choose which they prefer.</p>
<h2>Future Improvement</h2>
<p>I will make the word cloud can be controlled by using the date in the sidebar after I find a good website which has API or URL that can be parsed by only using a date. I also will include the natural language analysis for the news analysis. For the time analysis part, I will put the result of the optimized model by using auto.arima in R package.</p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://nycdatascience.com/blog/student-works/sp500-dashboard/"> R – NYC Data Science Academy Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/0ZtGgAwuuHU" height="1"></a></div>