<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/tidy-time-series-analysis-part-3-the-rolling-correlation/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.business-science.io/timeseries-analysis/2017/07/30/tidy-timeseries-analysis-pt-3.html"> business-science.io - Articles</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>In the third part in a series on <strong>Tidy Time Series Analysis</strong>, we’ll use the <code>runCor</code> function from <code>TTR</code> to investigate <strong>rolling (dynamic) correlations</strong>. We’ll again use <code>tidyquant</code> to investigate CRAN downloads. This time we’ll also get some help from the <code>corrr</code> package to investigate correlations over specific timespans, and the <code>cowplot</code> package for multi-plot visualizations. We’ll end by reviewing the changes in rolling correlations to show how to <strong>detect events and shifts in trend</strong>. If you like what you read, please <a rel="nofollow" href="http://www.business-science.io/timeseries-analysis/2017/07/30/tidy-timeseries-analysis-pt-3.html#social" target="_blank">follow us on social media</a> to stay up on the latest <a rel="nofollow" href="http://www.business-science.io/timeseries-analysis/2017/07/30/tidy-timeseries-analysis-pt-3.html#contact" target="_blank">Business Science</a> news, events and information! As always, we are interested in both expanding our <em>network of data scientists</em> and seeking <em>new clients interested in applying data science to business and finance</em>. If interested, <a rel="nofollow" href="http://www.business-science.io/contact.html" target="_blank">contact us</a>.</p>
<p>If you haven’t checked out the previous two tidy time series posts, you may want to review them to get up to speed.</p>
<ul>
<li><a rel="nofollow" href="http://www.business-science.io/timeseries-analysis/2017/07/02/tidy-timeseries-analysis.html" target="_blank">Part 1: Tidy Period Apply</a></li>
<li><a rel="nofollow" href="http://www.business-science.io/timeseries-analysis/2017/07/23/tidy-timeseries-analysis-pt-2.html" target="_blank">Part 2: Tidy Rolling Functions</a></li>
</ul>
<p>An example of the visualization we can create using the <code>runCor</code> function with <code>tq_mutate_xy()</code> in combination with the <code>corrr</code> and <code>cowplot</code> packages:</p>
<p><img alt="tidyquant correlation over time" src="https://i1.wp.com/www.business-science.io/assets/tidy-timeseries-part3.png?w=456" data-recalc-dims="1"></p>
<h1>Libraries Needed</h1>
<p>We’ll need to load four libraries today.</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>tidyquant</span><span>)</span><span> </span><span># Loads tidyverse, tidyquant, financial pkgs, xts/zoo
</span><span>library</span><span>(</span><span>cranlogs</span><span>)</span><span> </span><span># For inspecting package downloads over time
</span><span>library</span><span>(</span><span>corrr</span><span>)</span><span> </span><span># Tidy correlation tables and correlation plotting
</span><span>library</span><span>(</span><span>cowplot</span><span>)</span><span> </span><span>#</span><span> </span><span>Multiple</span><span> </span><span>plots</span><span> </span><span>with</span><span> </span><span>plot_grid</span><span>()</span></code></pre>
</figure>
<h1>CRAN tidyverse Downloads</h1>
<p>We’ll be using the same “tidyverse” dataset as the last two posts. The script below gets the package downloads for the first half of 2017.</p>
<figure>
<pre><code data-lang="r"><span># tidyverse packages (see my laptop stickers from first post) ;)
</span><span>pkgs</span><span> </span><span>&lt;-</span><span> </span><span>c</span><span>(</span><span>
</span><span>&quot;tidyr&quot;</span><span>,</span><span> </span><span>&quot;lubridate&quot;</span><span>,</span><span> </span><span>&quot;dplyr&quot;</span><span>,</span><span> </span><span>&quot;broom&quot;</span><span>,</span><span> </span><span>&quot;tidyquant&quot;</span><span>,</span><span> </span><span>&quot;ggplot2&quot;</span><span>,</span><span> </span><span>&quot;purrr&quot;</span><span>,</span><span> </span><span>&quot;stringr&quot;</span><span>,</span><span> </span><span>&quot;knitr&quot;</span><span>
</span><span>)</span><span>
</span><span># Get the downloads for the individual packages
</span><span>tidyverse_downloads</span><span> </span><span>&lt;-</span><span> </span><span>cran_downloads</span><span>(</span><span>
</span><span>packages</span><span> </span><span>=</span><span> </span><span>pkgs</span><span>,</span><span> </span><span>from</span><span> </span><span>=</span><span> </span><span>&quot;2017-01-01&quot;</span><span>,</span><span> </span><span>to</span><span> </span><span>=</span><span> </span><span>&quot;2017-06-30&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>tibble</span><span>::</span><span>as_tibble</span><span>()</span><span> </span><span>%>%</span><span>
</span><span>group_by</span><span>(</span><span>package</span><span>)</span><span>
</span><span># Visualize the package downloads
</span><span>tidyverse_downloads</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>date</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>count</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>package</span><span>))</span><span> </span><span>+</span><span>
</span><span># Data
</span><span> </span><span>geom_point</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>)</span><span> </span><span>+</span><span>
</span><span>facet_wrap</span><span>(</span><span>~</span><span> </span><span>package</span><span>,</span><span> </span><span>ncol</span><span> </span><span>=</span><span> </span><span>3</span><span>,</span><span> </span><span>scale</span><span> </span><span>=</span><span> </span><span>&quot;free_y&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span># Aesthetics
</span><span> </span><span>labs</span><span>(</span><span>title</span><span> </span><span>=</span><span> </span><span>&quot;tidyverse packages: Daily downloads&quot;</span><span>,</span><span> </span><span>x</span><span> </span><span>=</span><span> </span><span>&quot;&quot;</span><span>,</span><span>
</span><span>subtitle</span><span> </span><span>=</span><span> </span><span>&quot;2017-01-01 through 2017-06-30&quot;</span><span>,</span><span>
</span><span>caption</span><span> </span><span>=</span><span> </span><span>&quot;Downloads data courtesy of cranlogs package&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>scale_color_tq</span><span>()</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span> </span><span>+</span><span>
</span><span>theme</span><span>(</span><span>legend.position</span><span>=</span><span>&quot;none&quot;</span><span>)</span></code></pre>
</figure>
<p><img alt="plot of chunk unnamed-chunk-2" src="https://i1.wp.com/www.business-science.io/figure/source/2017-7-30-tidy-timeseries-analysis-pt-3/unnamed-chunk-2-1.png?w=456" data-recalc-dims="1"></p>
<p>We’ll also investigate correlations to the “broader market” meaning the <strong>total CRAN dowloads over time</strong>. To do this, we need to get the total downloads using <code>cran_downloads()</code> and leaving the <code>package</code> argument <code>NULL</code>, which is the default.</p>
<figure>
<pre><code data-lang="r"><span># Get data for total CRAN downloads
</span><span>all_downloads</span><span> </span><span>&lt;-</span><span> </span><span>cran_downloads</span><span>(</span><span>from</span><span> </span><span>=</span><span> </span><span>&quot;2017-01-01&quot;</span><span>,</span><span> </span><span>to</span><span> </span><span>=</span><span> </span><span>&quot;2017-06-30&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>tibble</span><span>::</span><span>as_tibble</span><span>()</span><span>
</span><span># Visualize the downloads
</span><span>all_downloads</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>date</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>count</span><span>))</span><span> </span><span>+</span><span>
</span><span># Data
</span><span> </span><span>geom_point</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>palette_light</span><span>()[[</span><span>1</span><span>]],</span><span> </span><span>size</span><span> </span><span>=</span><span> </span><span>2</span><span>)</span><span> </span><span>+</span><span>
</span><span># Aesthetics
</span><span> </span><span>labs</span><span>(</span><span>title</span><span> </span><span>=</span><span> </span><span>&quot;Total CRAN Packages: Daily downloads&quot;</span><span>,</span><span> </span><span>x</span><span> </span><span>=</span><span> </span><span>&quot;&quot;</span><span>,</span><span>
</span><span>subtitle</span><span> </span><span>=</span><span> </span><span>&quot;2017-01-01 through 2017-06-30&quot;</span><span>,</span><span>
</span><span>caption</span><span> </span><span>=</span><span> </span><span>&quot;Downloads data courtesy of cranlogs package&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>scale_y_continuous</span><span>(</span><span>labels</span><span> </span><span>=</span><span> </span><span>scales</span><span>::</span><span>comma</span><span>)</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span> </span><span>+</span><span>
</span><span>theme</span><span>(</span><span>legend.position</span><span>=</span><span>&quot;none&quot;</span><span>)</span></code></pre>
</figure>
<p><img alt="plot of chunk unnamed-chunk-3" src="https://i1.wp.com/www.business-science.io/figure/source/2017-7-30-tidy-timeseries-analysis-pt-3/unnamed-chunk-3-1.png?w=456" data-recalc-dims="1"></p>
<h1>Rolling Correlations</h1>
<p>Correlations in time series are very useful because <strong>if a relationship exists, you can actually model/predict/forecast using the correlation</strong>. However, there’s one issue: <strong>a correlation is NOT static</strong>! It changes over time. Even the best models can be rendered useless during periods when correlation is low.</p>
<p>One of the most important calculations in time series analysis is the <strong>rolling correlation</strong>. Rolling correlations are simply applying a correlation between two time series (say sales of product x and product y) as a rolling window calculation.</p>
<p><img alt="Rolling Correlation Example" src="https://i0.wp.com/www.business-science.io/assets/rolling-corr-explanation.png?w=456" data-recalc-dims="1"></p>
<p>One major benefit of a rolling correlation is that we can visualize the change in correlation over time. The sample data (above) is charted (below). As shown, there’s a relatively high correlation between Sales of Product X and Y until a big shift in December. The question is, “What happened in December?” Just being able to ask this question can be critical to an organization.</p>
<p><img alt="plot of chunk unnamed-chunk-4" src="https://i0.wp.com/www.business-science.io/figure/source/2017-7-30-tidy-timeseries-analysis-pt-3/unnamed-chunk-4-1.png?w=456" data-recalc-dims="1"></p>
<p>In addition to visualizations, the rolling correlation is great for a number of reasons. First, <strong>changes in correlation can signal events</strong> that have occurred causing two correlated time series to deviate from each other. Second, when modeling, <strong>timespans of low correlation can help in determining whether or not to trust a forecast model</strong>. Third, you can <strong>detect shifts in trend as time series</strong> become more or less correlated over time.</p>
<h1>Time Series Functions</h1>
<p>The <code>xts</code>, <code>zoo</code>, and <code>TTR</code> packages have some great functions that enable working with time series. Today, we’ll take a look at the <code>runCor()</code> function from the <code>TTR</code> package. You can see which <code>TTR</code> functions are integrated into <code>tidyquant</code> package below:</p>
<figure>
<pre><code data-lang="r"><span># &quot;run&quot; functions from TTR
</span><span>tq_mutate_fun_options</span><span>()</span><span>$</span><span>TTR</span><span> </span><span>%>%</span><span>
</span><span>stringr</span><span>::</span><span>str_subset</span><span>(</span><span>&quot;^run&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">##  [1] &quot;runCor&quot;         &quot;runCov&quot;         &quot;runMAD&quot;        
##  [4] &quot;runMax&quot;         &quot;runMean&quot;        &quot;runMedian&quot;     
##  [7] &quot;runMin&quot;         &quot;runPercentRank&quot; &quot;runSD&quot;         
## [10] &quot;runSum&quot;         &quot;runVar&quot;</code></pre>
</figure>
<h1>Tidy Implementation of Time Series Functions</h1>
<p>We’ll use the <code>tq_mutate_xy()</code> function to apply time series functions in a “tidy” way. Similar to <code>tq_mutate()</code> used in the <a rel="nofollow" href="http://www.business-science.io/timeseries-analysis/2017/07/23/tidy-timeseries-analysis-pt-2.html" target="_blank">last post</a>, the <code>tq_mutate_xy()</code> function always adds columns to the existing data frame (rather than returning a new data frame like <code>tq_transmute()</code>). It’s well suited for tasks that result in column-wise dimension changes (not row-wise such as periodicity changes, use <code>tq_transmute</code> for those!).</p>
<p>Most running statistic functions only take one data argument, <code>x</code>. In these cases you can use <code>tq_mutate()</code>, which has an argument, <code>select</code>. See how <code>runSD</code> only takes <code>x</code>.</p>
<figure>
<pre><code data-lang="r"><span># If first arg is x (and no y) --> us tq_mutate()
</span><span>args</span><span>(</span><span>runSD</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## function (x, n = 10, sample = TRUE, cumulative = FALSE) 
## NULL</code></pre>
</figure>
<p>However, functions like <code>runCor</code> and <code>runCov</code> are setup to take in two data arguments, <code>x</code> and <code>y</code>. In these cases, use <code>tq_mutate_xy()</code>, which takes two arguments, <code>x</code> and <code>y</code> (as opposed to <code>select</code> from <code>tq_mutate()</code>). This makes it well suited for functions that have the first two arguments being <code>x</code> and <code>y</code>. See how <code>runCor</code> has two arguments <code>x</code> and <code>y</code>.</p>
<figure>
<pre><code data-lang="r"><span># If first two arguments are x and y --> use tq_mutate_xy()
</span><span>args</span><span>(</span><span>runCor</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## function (x, y, n = 10, use = &quot;all.obs&quot;, sample = TRUE, cumulative = FALSE) 
## NULL</code></pre>
</figure>
<h1>Static Correlations</h1>
<p>Before we jump into rolling correlations, let’s examine the static correlations of our package downloads. This gives us an idea of how in sync the various packages are with each other over the entire timespan.</p>
<p>We’ll use the <code>correlate()</code> and <code>shave()</code> functions from the <code>corrr</code> package to output a tidy correlation table. We’ll hone in on the last column “all_cran”, which measures the correlation between individual packages and the broader market (i.e. total CRAN downloads).</p>
<figure>
<pre><code data-lang="r"><span># Correlation table
</span><span>tidyverse_static_correlations</span><span> </span><span>&lt;-</span><span> </span><span>tidyverse_downloads</span><span> </span><span>%>%</span><span>
</span><span># Data wrangling
</span><span> </span><span>spread</span><span>(</span><span>key</span><span> </span><span>=</span><span> </span><span>package</span><span>,</span><span> </span><span>value</span><span> </span><span>=</span><span> </span><span>count</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>left_join</span><span>(</span><span>all_downloads</span><span>,</span><span> </span><span>by</span><span> </span><span>=</span><span> </span><span>&quot;date&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>rename</span><span>(</span><span>all_cran</span><span> </span><span>=</span><span> </span><span>count</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>-</span><span>date</span><span>)</span><span> </span><span>%>%</span><span>
</span><span># Correlation and formating
</span><span> </span><span>correlate</span><span>()</span><span> </span><span># Pretty printing
</span><span>tidyverse_static_correlations</span><span> </span><span>%>%</span><span>
</span><span>shave</span><span>(</span><span>upper</span><span> </span><span>=</span><span> </span><span>F</span><span>)</span></code></pre>
</figure>
<table>
<thead>
<tr>
<th>rowname</th>
<th>broom</th>
<th>dplyr</th>
<th>ggplot2</th>
<th>knitr</th>
<th>lubridate</th>
<th>purrr</th>
<th>stringr</th>
<th>tidyquant</th>
<th>tidyr</th>
<th>all_cran</th>
</tr>
</thead>
<tbody>
<tr>
<td>broom</td>
<td> </td>
<td>0.63</td>
<td>0.78</td>
<td>0.67</td>
<td>0.52</td>
<td>0.40</td>
<td>0.81</td>
<td>0.17</td>
<td>0.53</td>
<td>0.74</td>
</tr>
<tr>
<td>dplyr</td>
<td> </td>
<td> </td>
<td>0.73</td>
<td>0.71</td>
<td>0.59</td>
<td>0.58</td>
<td>0.71</td>
<td>0.14</td>
<td>0.64</td>
<td>0.76</td>
</tr>
<tr>
<td>ggplot2</td>
<td> </td>
<td> </td>
<td> </td>
<td>0.91</td>
<td>0.82</td>
<td>0.67</td>
<td>0.91</td>
<td>0.20</td>
<td>0.82</td>
<td>0.94</td>
</tr>
<tr>
<td>knitr</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>0.72</td>
<td>0.74</td>
<td>0.88</td>
<td>0.21</td>
<td>0.89</td>
<td>0.92</td>
</tr>
<tr>
<td>lubridate</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>0.79</td>
<td>0.72</td>
<td>0.29</td>
<td>0.73</td>
<td>0.82</td>
</tr>
<tr>
<td>purrr</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>0.66</td>
<td>0.35</td>
<td>0.82</td>
<td>0.80</td>
</tr>
<tr>
<td>stringr</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>0.23</td>
<td>0.81</td>
<td>0.91</td>
</tr>
<tr>
<td>tidyquant</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>0.26</td>
<td>0.31</td>
</tr>
<tr>
<td>tidyr</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>0.87</td>
</tr>
<tr>
<td>all_cran</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
</tbody>
</table>
<p>The correlation table is nice, but the outliers don’t exactly jump out. For instance, it’s difficult to see that <code>tidyquant</code> is low compared to the other packages withing the “all_cran” column.</p>
<p>Fortunately, the <code>corrr</code> package has a nice visualization called a <code>network_plot()</code>. It helps to identify strength of correlation. Similar to a “kmeans” analysis, we are looking for association by distance (or in this case by correlation). <strong>How well the packages correlate with each other is akin to how associated they are with each other.</strong> The network plot shows us exactly this association!</p>
<figure>
<pre><code data-lang="r"><span># Network plot
</span><span>gg_all</span><span> </span><span>&lt;-</span><span> </span><span>tidyverse_static_correlations</span><span> </span><span>%>%</span><span>
</span><span>network_plot</span><span>(</span><span>colours</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>palette_light</span><span>()[[</span><span>2</span><span>]],</span><span> </span><span>&quot;white&quot;</span><span>,</span><span> </span><span>palette_light</span><span>()[[</span><span>4</span><span>]]),</span><span> </span><span>legend</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>
</span><span>title</span><span> </span><span>=</span><span> </span><span>&quot;Correlations of tidyverse Package Downloads to Total CRAN Downloads&quot;</span><span>,</span><span>
</span><span>subtitle</span><span> </span><span>=</span><span> </span><span>&quot;Looking at January through June, tidyquant is a clear outlier&quot;</span><span>
</span><span>)</span><span> </span><span>+</span><span>
</span><span>expand_limits</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>-0.75</span><span>,</span><span> </span><span>0.25</span><span>),</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>-0.4</span><span>,</span><span> </span><span>0.4</span><span>))</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span> </span><span>+</span><span>
</span><span>theme</span><span>(</span><span>legend.position</span><span> </span><span>=</span><span> </span><span>&quot;bottom&quot;</span><span>)</span><span>
</span><span>gg_all</span></code></pre>
</figure>
<p><img alt="plot of chunk unnamed-chunk-10" src="https://i2.wp.com/www.business-science.io/figure/source/2017-7-30-tidy-timeseries-analysis-pt-3/unnamed-chunk-10-1.png?w=456" data-recalc-dims="1"></p>
<p>We can see that <code>tidyquant</code> has a very low correlation to “all_cran” and the rest of the “tidyverse” packages. This would lead us to believe that <code>tidyquant</code> is trending abnormally with respect to the rest, and thus is possibly not as associated as we think. Is this really the case?</p>
<h1>Rolling Correlations</h1>
<p>Let’s see what happens when we incorporate time using a rolling correlation. The script below uses the <code>runCor</code> function from the <code>TTR</code> package. We apply it using <code>tq_mutate_xy()</code>, which is useful for applying functions such has <code>runCor</code> that have both an <code>x</code> and <code>y</code> input.</p>
<figure>
<pre><code data-lang="r"><span># Get rolling correlations
</span><span>tidyverse_rolling_corr</span><span> </span><span>&lt;-</span><span> </span><span>tidyverse_downloads</span><span> </span><span>%>%</span><span>
</span><span># Data wrangling
</span><span> </span><span>left_join</span><span>(</span><span>all_downloads</span><span>,</span><span> </span><span>by</span><span> </span><span>=</span><span> </span><span>&quot;date&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>date</span><span>,</span><span> </span><span>package</span><span>,</span><span> </span><span>count.x</span><span>,</span><span> </span><span>count.y</span><span>)</span><span> </span><span>%>%</span><span>
</span><span># Mutation
</span><span> </span><span>tq_mutate_xy</span><span>(</span><span>
</span><span>x</span><span> </span><span>=</span><span> </span><span>count.x</span><span>,</span><span>
</span><span>y</span><span> </span><span>=</span><span> </span><span>count.y</span><span>,</span><span>
</span><span>mutate_fun</span><span> </span><span>=</span><span> </span><span>runCor</span><span>,</span><span> </span><span># runCor args
</span><span> </span><span>n</span><span> </span><span>=</span><span> </span><span>30</span><span>,</span><span>
</span><span>use</span><span> </span><span>=</span><span> </span><span>&quot;pairwise.complete.obs&quot;</span><span>,</span><span>
</span><span># tq_mutate args
</span><span> </span><span>col_rename</span><span> </span><span>=</span><span> </span><span>&quot;rolling_corr&quot;</span><span>
</span><span>)</span><span>
</span><span># Join static correlations with rolling correlations
</span><span>tidyverse_static_correlations</span><span> </span><span>&lt;-</span><span> </span><span>tidyverse_static_correlations</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>rowname</span><span>,</span><span> </span><span>all_cran</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>rename</span><span>(</span><span>package</span><span> </span><span>=</span><span> </span><span>rowname</span><span>)</span><span>
</span><span>tidyverse_rolling_corr</span><span> </span><span>&lt;-</span><span> </span><span>tidyverse_rolling_corr</span><span> </span><span>%>%</span><span>
</span><span>left_join</span><span>(</span><span>tidyverse_static_correlations</span><span>,</span><span> </span><span>by</span><span> </span><span>=</span><span> </span><span>&quot;package&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>rename</span><span>(</span><span>static_corr</span><span> </span><span>=</span><span> </span><span>all_cran</span><span>)</span><span>
</span><span># Plot
</span><span>tidyverse_rolling_corr</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>date</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>package</span><span>))</span><span> </span><span>+</span><span>
</span><span># Data
</span><span> </span><span>geom_line</span><span>(</span><span>aes</span><span>(</span><span>y</span><span> </span><span>=</span><span> </span><span>static_corr</span><span>),</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>&quot;red&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>geom_point</span><span>(</span><span>aes</span><span>(</span><span>y</span><span> </span><span>=</span><span> </span><span>rolling_corr</span><span>),</span><span> </span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>)</span><span> </span><span>+</span><span>
</span><span>facet_wrap</span><span>(</span><span>~</span><span> </span><span>package</span><span>,</span><span> </span><span>ncol</span><span> </span><span>=</span><span> </span><span>3</span><span>,</span><span> </span><span>scales</span><span> </span><span>=</span><span> </span><span>&quot;free_y&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span># Aesthetics
</span><span> </span><span>scale_color_tq</span><span>()</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>
</span><span>title</span><span> </span><span>=</span><span> </span><span>&quot;tidyverse: 30-Day Rolling Download Correlations, Package vs Total CRAN&quot;</span><span>,</span><span>
</span><span>subtitle</span><span> </span><span>=</span><span> </span><span>&quot;Relationships are dynamic vs static correlation (red line)&quot;</span><span>,</span><span>
</span><span>x</span><span> </span><span>=</span><span> </span><span>&quot;&quot;</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>&quot;Correlation&quot;</span><span>
</span><span>)</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span> </span><span>+</span><span>
</span><span>theme</span><span>(</span><span>legend.position</span><span>=</span><span>&quot;none&quot;</span><span>)</span></code></pre>
</figure>
<p><img alt="plot of chunk unnamed-chunk-11" src="https://i1.wp.com/www.business-science.io/figure/source/2017-7-30-tidy-timeseries-analysis-pt-3/unnamed-chunk-11-1.png?w=456" data-recalc-dims="1"></p>
<p>The rolling correlation shows the dynamic nature of the relationship. If we just went by the static correlation over the full timespan (red line), we’d be misled about the dynamic nature of these time series. Further, we can see that most packages are highly correlated with the broader market (total CRAN downloads) with the exception of various periods where the correlations dropped. The <strong>drops could indicate events or changes in user behavior</strong> that resulted in shocks to the download patterns.</p>
<p>Focusing on the main outlier <code>tidyquant</code>, we can see that once April hit <code>tidyquant</code> is trending closer to a 0.60 correlation meaning that the 0.31 relationship (red line) is likely too low going forward.</p>
<p>Last, we can redraw the network plot from April through June to investigate the shift in relationship. We can use the <code>cowplot</code> package to plot two ggplots (or corrr network plots) side-by-side.</p>
<figure>
<pre><code data-lang="r"><span># Redrawing Network Plot from April through June
</span><span>gg_subset</span><span> </span><span>&lt;-</span><span> </span><span>tidyverse_downloads</span><span> </span><span>%>%</span><span>
</span><span># Filter by date >= April 1, 2017
</span><span> </span><span>filter</span><span>(</span><span>date</span><span> </span><span>>=</span><span> </span><span>ymd</span><span>(</span><span>&quot;2017-04-01&quot;</span><span>))</span><span> </span><span>%>%</span><span>
</span><span># Data wrangling
</span><span> </span><span>spread</span><span>(</span><span>key</span><span> </span><span>=</span><span> </span><span>package</span><span>,</span><span> </span><span>value</span><span> </span><span>=</span><span> </span><span>count</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>left_join</span><span>(</span><span>all_downloads</span><span>,</span><span> </span><span>by</span><span> </span><span>=</span><span> </span><span>&quot;date&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>rename</span><span>(</span><span>all_cran</span><span> </span><span>=</span><span> </span><span>count</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>-</span><span>date</span><span>)</span><span> </span><span>%>%</span><span>
</span><span># Correlation and formating
</span><span> </span><span>correlate</span><span>()</span><span> </span><span>%>%</span><span>
</span><span># Network Plot
</span><span> </span><span>network_plot</span><span>(</span><span>colours</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>palette_light</span><span>()[[</span><span>2</span><span>]],</span><span> </span><span>&quot;white&quot;</span><span>,</span><span> </span><span>palette_light</span><span>()[[</span><span>4</span><span>]]),</span><span> </span><span>legend</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>
</span><span>title</span><span> </span><span>=</span><span> </span><span>&quot;April through June (Last 3 Months)&quot;</span><span>,</span><span>
</span><span>subtitle</span><span> </span><span>=</span><span> </span><span>&quot;tidyquant correlation is increasing&quot;</span><span>
</span><span>)</span><span> </span><span>+</span><span>
</span><span>expand_limits</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>-0.75</span><span>,</span><span> </span><span>0.25</span><span>),</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>-0.4</span><span>,</span><span> </span><span>0.4</span><span>))</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span> </span><span>+</span><span>
</span><span>theme</span><span>(</span><span>legend.position</span><span> </span><span>=</span><span> </span><span>&quot;bottom&quot;</span><span>)</span><span>
</span><span># Modify the January through June network plot (previous plot)
</span><span>gg_all</span><span> </span><span>&lt;-</span><span> </span><span>gg_all</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>
</span><span>title</span><span> </span><span>=</span><span> </span><span>&quot;January through June (Last 6 months)&quot;</span><span>,</span><span>
</span><span>subtitle</span><span> </span><span>=</span><span> </span><span>&quot;tidyquant is an outlier&quot;</span><span>
</span><span>)</span><span>
</span><span># Format cowplot
</span><span>cow_net_plots</span><span> </span><span>&lt;-</span><span> </span><span>plot_grid</span><span>(</span><span>gg_all</span><span>,</span><span> </span><span>gg_subset</span><span>,</span><span> </span><span>ncol</span><span> </span><span>=</span><span> </span><span>2</span><span>)</span><span>
</span><span>title</span><span> </span><span>&lt;-</span><span> </span><span>ggdraw</span><span>()</span><span> </span><span>+</span><span> </span><span>draw_label</span><span>(</span><span>label</span><span> </span><span>=</span><span> </span><span>'tidyquant is getting &quot;tidy&quot;-er'</span><span>,</span><span>
</span><span>fontface</span><span> </span><span>=</span><span> </span><span>'bold'</span><span>,</span><span> </span><span>size</span><span> </span><span>=</span><span> </span><span>18</span><span>)</span><span>
</span><span>cow_out</span><span> </span><span>&lt;-</span><span> </span><span>plot_grid</span><span>(</span><span>title</span><span>,</span><span> </span><span>cow_net_plots</span><span>,</span><span> </span><span>ncol</span><span>=</span><span>1</span><span>,</span><span> </span><span>rel_heights</span><span>=</span><span>c</span><span>(</span><span>0.1</span><span>,</span><span> </span><span>1</span><span>))</span><span>
</span><span>cow_out</span></code></pre>
</figure>
<p><img alt="plot of chunk unnamed-chunk-12" src="https://i0.wp.com/www.business-science.io/figure/source/2017-7-30-tidy-timeseries-analysis-pt-3/unnamed-chunk-12-1.png?w=456" data-recalc-dims="1"></p>
<h1>Conclusions</h1>
<p>The <code>tq_mutate_xy()</code> function from <code>tidyquant</code> enables efficient and “tidy” application of <code>TTR::runCor()</code> and other functions with x and y arguments. The <code>corrr</code> package is useful for computing the correlations and visualizing relationships, and it fits nicely into the “tidy” framework. The <code>cowplot</code> package helps with arranging multiple ggplots to create compeling stories. In this case, it appears that <strong><code>tidyquant</code> is becoming “tidy”-er</strong>, not to be confused with the package <code>tidyr</code>. <img alt="😉" src="https://s.w.org/images/core/emoji/2/72x72/1f609.png" class="wp-smiley"></p>
<h1>About Business Science <a></a></h1>
<p>We have a full suite of data science services to <em>supercharge</em> your organizations financial and business performance! For example, our experienced data scientists reduced a manufacturer’s sales forecasting error by 50%, which led to improved personnel planning, material purchasing and inventory management.</p>
<p>How do we do it? <strong>With team-based data science</strong>: Using our network of data science consultants with expertise in Marketing, Forecasting, Finance and more, we pull together the <em>right team</em> to get <em>custom projects</em> done <em>on time</em>, <em>within budget</em>, and of the <em>highest quality</em>. Learn about our <a rel="nofollow" href="http://www.business-science.io/services.html" target="_blank">data science services</a> or <a rel="nofollow" href="http://www.business-science.io/contact.html" target="_blank">contact us</a>!</p>
<p>We are growing! Let us know if you are interested in joining our <strong>network of data scientist consultants</strong>. If you have expertise in Marketing Analytics, Data Science for Business, Financial Analytics, Forecasting or data science in general, we’d love to talk. <a rel="nofollow" href="http://www.business-science.io/contact.html" target="_blank">Contact us</a>!</p>
<h1>Follow Business Science on Social Media <a></a></h1>
<ul>
<li>Connect with <a rel="nofollow" href="https://twitter.com/bizScienc" target="_blank">@bizScienc</a> on <a rel="nofollow" href="https://twitter.com/bizScienc" target="_blank">twitter</a>!</li>
<li>Like us on <a rel="nofollow" href="https://www.facebook.com/Business-Science-LLC-754699134699054/" target="_blank">Facebook</a>!!!</li>
<li>Follow us on <a rel="nofollow" href="https://www.linkedin.com/company/business.science" target="_blank">LinkedIn</a>!</li>
<li>Sign up for <a rel="nofollow" href="http://www.business-science.io/" target="_blank">our insights blog</a> to stay updated!</li>
<li>If you like our software, <a rel="nofollow" href="https://github.com/business-science" target="_blank">star our GitHub packages</a> <img alt="🙂" src="https://s.w.org/images/core/emoji/2/72x72/1f642.png" class="wp-smiley"></li>
</ul>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.business-science.io/timeseries-analysis/2017/07/30/tidy-timeseries-analysis-pt-3.html"> business-science.io - Articles</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/1QhbeGkdRrI" height="1"></a></div>