<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/data-science-for-business-time-series-forecasting-part-2-forecasting-with-timekit/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://shiring.github.io/forecasting/2017/06/09/retail_forcasting_part2"> Shirin's playgRound</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>In <a rel="nofollow" href="https://shiring.github.io/forecasting/2017/05/28/retail_forcasting_part1" target="_blank">my last post</a>, I prepared and visually explored time series data.</p>
<p>Now, I will use this data to test the <a rel="nofollow" href="https://cran.r-project.org/web/packages/timekit/index.html" target="_blank"><strong>timekit</strong> package</a> for time series forecasting with machine learning.</p>
<p></p>
<h2>Forecasting</h2>
<p>In time series forecasting, we use models to predict future time points based on past observations.</p>
<p>As mentioned in <strong>timekit</strong>’s vignette, “as with most machine learning applications, the prediction is only as good as the patterns in the data. Forecasting using this approach may not be suitable when patterns are not present or when the future is highly uncertain (i.e. past is not a suitable predictor of future performance).”</p>
<p>And while this is certainly true, we don’t always have data with a strong regular pattern. And, I would argue, data that has very obvious patterns doesn’t need a complicated model to generate forecasts – we can already guess the future curve just by looking at it. So, if we think of use-cases for businesses, who want to predict e.g. product sales, forecasting models are especially relevant in cases where we can’t make predictions manually or based on experience.</p>
<p>The packages I am using are <strong>timekit</strong> for forecasting, <strong>tidyverse</strong> for data wrangling and visualization, <strong>caret</strong> for additional modeling functions, <strong>tidyquant</strong> for its ggplot theme, <strong>broom</strong> and <strong>modelr</strong> for (tidy) modeling.</p>
<div>
<pre><code><span>library</span><span>(</span><span>tidyverse</span><span>)</span><span>
</span><span>library</span><span>(</span><span>caret</span><span>)</span><span>
</span><span>library</span><span>(</span><span>tidyquant</span><span>)</span><span>
</span><span>library</span><span>(</span><span>broom</span><span>)</span><span>
</span><span>library</span><span>(</span><span>timekit</span><span>)</span><span>
</span><span>library</span><span>(</span><span>modelr</span><span>)</span><span>
</span><span>options</span><span>(</span><span>na.action</span><span> </span><span>=</span><span> </span><span>na.warn</span><span>)</span><span>
</span></code></pre>
</div>
<p></p>
<h3>Training and test data</h3>
<p>My input data is the tibble <code>retail_p_day</code>, that was created in <a rel="nofollow" href="https://shiring.github.io/forecasting/2017/05/28/retail_forcasting_part1" target="_blank">my last post</a>.</p>
<p>I am splitting this dataset into training (all data points before/on Nov. 1st 2011) and test samples (all data points after Nov. 1st 2011).</p>
<div>
<pre><code><span>retail_p_day</span><span> </span><span>&lt;-</span><span> </span><span>retail_p_day</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>model</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>day</span><span> </span><span>&lt;=</span><span> </span><span>&quot;2011-11-01&quot;</span><span>,</span><span> </span><span>&quot;train&quot;</span><span>,</span><span> </span><span>&quot;test&quot;</span><span>))</span><span>
</span><span>colnames</span><span>(</span><span>retail_p_day</span><span>)[</span><span>grep</span><span>(</span><span>&quot;^[0-9]+&quot;</span><span>,</span><span> </span><span>colnames</span><span>(</span><span>retail_p_day</span><span>))]</span><span> </span><span>&lt;-</span><span> </span><span>paste0</span><span>(</span><span>&quot;P_&quot;</span><span>,</span><span> </span><span>colnames</span><span>(</span><span>retail_p_day</span><span>)[</span><span>grep</span><span>(</span><span>&quot;^[0-9]+&quot;</span><span>,</span><span> </span><span>colnames</span><span>(</span><span>retail_p_day</span><span>))])</span><span>
</span></code></pre>
</div>
<p>Here, I am testing out <strong>timekit</strong>’s functions with the net income per day as response variable. Because the time series in our data set is relatively short and doesn’t cover multiple years, this forecast will only be able to capture recurring variation in days and weeks. Variations like increased sales before holidays, etc. would need additional data from several years to be accurately forecast.</p>
<p>As we can see in the plot below, the net income shows variation between days.</p>
<div>
<pre><code><span>retail_p_day</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>day</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>sum_income</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>model</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_point</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>)</span><span> </span><span>+</span><span>
</span><span>geom_line</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>)</span><span> </span><span>+</span><span>
</span><span>scale_color_manual</span><span>(</span><span>values</span><span> </span><span>=</span><span> </span><span>palette_light</span><span>())</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span>
</span></code></pre>
</div>
<p><img alt="" src="https://i0.wp.com/shiring.github.io/forecasting/2017/06/09/retail_forcasting_part2_files/figure-markdown_github/unnamed-chunk-4-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p></p>
<h3>Augmenting the time series signature</h3>
<p>With <strong>timekit</strong>, we can do forecasting with only a time series signature (a series of dates and times) and a corresponding response variable. If we had additional features that could be forecast independently, we could also introduce these into the model, but here, I will only work with the minimal data set.</p>
<p>A central function of <strong>timekit</strong> is <code>tk_augment_timeseries_signature()</code>, which adds a number of features based on the properties of our time series signature:</p>
<ul>
<li>index: The index value that was decomposed</li>
<li>index.num: The numeric value of the index in seconds. The base is “1970-01-01 00:00:00”.</li>
<li>diff: The difference in seconds from the previous numeric index value.</li>
<li>year: The year component of the index.</li>
<li>half: The half component of the index.</li>
<li>quarter: The quarter component of the index.</li>
<li>month: The month component of the index with base 1.</li>
<li>month.xts: The month component of the index with base 0, which is what xts implements.</li>
<li>month.lbl: The month label as an ordered factor beginning with January and ending with December.</li>
<li>day: The day component of the index.</li>
<li>hour: The hour component of the index.</li>
<li>minute: The minute component of the index.</li>
<li>second: The second component of the index.</li>
<li>hour12: The hour component on a 12 hour scale.</li>
<li>am.pm: Morning (AM) = 1, Afternoon (PM) = 2.</li>
<li>wday: The day of the week with base 1. Sunday = 1 and Saturday = 7.</li>
<li>wday.xts: The day of the week with base 0, which is what xts implements. Sunday = 0 and Saturday = 6.</li>
<li>wday.lbl: The day of the week label as an ordered factor begining with Sunday and ending with Saturday.</li>
<li>mday: The day of the month.</li>
<li>qday: The day of the quarter.</li>
<li>yday: The day of the year.</li>
<li>mweek: The week of the month.</li>
<li>week: The week number of the year (Sunday start).</li>
<li>week.iso: The ISO week number of the year (Monday start).</li>
<li>week2: The modulus for bi-weekly frequency.</li>
<li>week3: The modulus for tri-weekly frequency.</li>
<li>week4: The modulus for quad-weekly frequency.</li>
<li>mday7: The integer division of day of the month by seven, which returns the first, second, third, … instance the day has appeared in the month. Values begin at 1. For example, the first Saturday in the month has mday7 = 1. The second has mday7 = 2.</li>
</ul>
<p>Because we have missing data for the first column of diff, I am removing this row. We need to keep in mind too, that we have an irregular time series, because we never have data on Saturdays. This will affect the modeling and results and we need to account for this later on! Alternatively, it might make sense to compare the results when setting all NAs/Saturdays to 0, assuming that no information means that there was no income on a given day. Or we could impute missing values. Which strategy most accurately represents your data needs to be decided based on a good understanding of the business and how the data was collected.</p>
<div>
<pre><code><span>retail_p_day_aug</span><span> </span><span>&lt;-</span><span> </span><span>retail_p_day</span><span> </span><span>%>%</span><span>
</span><span>rename</span><span>(</span><span>date</span><span> </span><span>=</span><span> </span><span>day</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>model</span><span>,</span><span> </span><span>date</span><span>,</span><span> </span><span>sum_income</span><span>)</span><span> </span><span>%>%</span><span> </span><span>tk_augment_timeseries_signature</span><span>()</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>-</span><span>contains</span><span>(</span><span>&quot;month&quot;</span><span>))</span><span>
</span><span>retail_p_day_aug</span><span> </span><span>&lt;-</span><span> </span><span>retail_p_day_aug</span><span>[</span><span>complete.cases</span><span>(</span><span>retail_p_day_aug</span><span>),</span><span> </span><span>]</span><span>
</span></code></pre>
</div>
<p></p>
<h3>Preprocessing</h3>
<p>Not all of these augmented features will be informative for our model. For example, we don’t have information about time of day, so features like hour, minute, second, etc. will be irrelevant here.</p>
<p>Let’s look at column variation for all numeric feature and remove those with a variance of 0.</p>
<div>
<pre><code><span>library</span><span>(</span><span>matrixStats</span><span>)</span><span>
</span><span>(</span><span>var</span><span> </span><span>&lt;-</span><span> </span><span>data.frame</span><span>(</span><span>colnames</span><span> </span><span>=</span><span> </span><span>colnames</span><span>(</span><span>retail_p_day_aug</span><span>[,</span><span> </span><span>sapply</span><span>(</span><span>retail_p_day_aug</span><span>,</span><span> </span><span>is.numeric</span><span>)]),</span><span>
</span><span>colvars</span><span> </span><span>=</span><span> </span><span>colVars</span><span>(</span><span>as.matrix</span><span>(</span><span>retail_p_day_aug</span><span>[,</span><span> </span><span>sapply</span><span>(</span><span>retail_p_day_aug</span><span>,</span><span> </span><span>is.numeric</span><span>)])))</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>colvars</span><span> </span><span>==</span><span> </span><span>0</span><span>))</span><span>
</span></code></pre>
</div>
<div>
<pre><code>##   colnames colvars
## 1     hour       0
## 2   minute       0
## 3   second       0
## 4   hour12       0
## 5    am.pm       0
</code></pre>
</div>
<div>
<pre><code><span>retail_p_day_aug</span><span> </span><span>&lt;-</span><span> </span><span>select</span><span>(</span><span>retail_p_day_aug</span><span>,</span><span> </span><span>-</span><span>one_of</span><span>(</span><span>as.character</span><span>(</span><span>var</span><span>$</span><span>colnames</span><span>)))</span><span>
</span></code></pre>
</div>
<p>Next, we want to remove highly correlated features. By plotting them, we can get an idea about which cutoff to set.</p>
<div>
<pre><code><span>library</span><span>(</span><span>ggcorrplot</span><span>)</span><span>
</span><span>cor</span><span> </span><span>&lt;-</span><span> </span><span>cor</span><span>(</span><span>retail_p_day_aug</span><span>[,</span><span> </span><span>sapply</span><span>(</span><span>retail_p_day_aug</span><span>,</span><span> </span><span>is.numeric</span><span>)])</span><span>
</span><span>p.cor</span><span> </span><span>&lt;-</span><span> </span><span>cor_pmat</span><span>(</span><span>retail_p_day_aug</span><span>[,</span><span> </span><span>sapply</span><span>(</span><span>retail_p_day_aug</span><span>,</span><span> </span><span>is.numeric</span><span>)])</span><span>
</span><span>ggcorrplot</span><span>(</span><span>cor</span><span>,</span><span> </span><span>type</span><span> </span><span>=</span><span> </span><span>&quot;upper&quot;</span><span>,</span><span> </span><span>outline.col</span><span> </span><span>=</span><span> </span><span>&quot;white&quot;</span><span>,</span><span> </span><span>hc.order</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>,</span><span> </span><span>p.mat</span><span> </span><span>=</span><span> </span><span>p.cor</span><span>,</span><span>
</span><span>colors</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>palette_light</span><span>()[</span><span>1</span><span>],</span><span> </span><span>&quot;white&quot;</span><span>,</span><span> </span><span>palette_light</span><span>()[</span><span>2</span><span>]))</span><span>
</span></code></pre>
</div>
<p><img alt="" src="https://i1.wp.com/shiring.github.io/forecasting/2017/06/09/retail_forcasting_part2_files/figure-markdown_github/unnamed-chunk-7-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>I am going to choose a cutoff of 0.9 for removing features:</p>
<div>
<pre><code><span>cor_cut</span><span> </span><span>&lt;-</span><span> </span><span>findCorrelation</span><span>(</span><span>cor</span><span>,</span><span> </span><span>cutoff</span><span>=</span><span>0.9</span><span>)</span><span> </span><span>retail_p_day_aug</span><span> </span><span>&lt;-</span><span> </span><span>select</span><span>(</span><span>retail_p_day_aug</span><span>,</span><span> </span><span>-</span><span>one_of</span><span>(</span><span>colnames</span><span>(</span><span>cor</span><span>)[</span><span>cor_cut</span><span>]))</span><span>
</span></code></pre>
</div>
<p></p>
<p>Now, I can split the data into training and test sets:</p>
<div>
<pre><code><span>train</span><span> </span><span>&lt;-</span><span> </span><span>filter</span><span>(</span><span>retail_p_day_aug</span><span>,</span><span> </span><span>model</span><span> </span><span>==</span><span> </span><span>&quot;train&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>-</span><span>model</span><span>)</span><span>
</span><span>test</span><span> </span><span>&lt;-</span><span> </span><span>filter</span><span>(</span><span>retail_p_day_aug</span><span>,</span><span> </span><span>model</span><span> </span><span>==</span><span> </span><span>&quot;test&quot;</span><span>)</span><span>
</span></code></pre>
</div>
<p></p>
<h3>Modeling</h3>
<p>To model the time series of the response variable <code>sum_income</code>, I am using a <a rel="nofollow" href="https://en.wikipedia.org/wiki/Generalized_linear_model" target="_blank">generalized linear model</a>. We could try all kinds of different models and modeling parameters, but for this test I am keeping it simple.</p>
<div>
<pre><code><span>fit_lm</span><span> </span><span>&lt;-</span><span> </span><span>glm</span><span>(</span><span>sum_income</span><span> </span><span>~</span><span> </span><span>.</span><span>,</span><span> </span><span>data</span><span> </span><span>=</span><span> </span><span>train</span><span>)</span><span>
</span></code></pre>
</div>
<p></p>
<p>We can examine our model e.g. by visualizing:</p>
<div>
<pre><code><span>tidy</span><span>(</span><span>fit_lm</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>gather</span><span>(</span><span>x</span><span>,</span><span> </span><span>y</span><span>,</span><span> </span><span>estimate</span><span>:</span><span>p.value</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>term</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>y</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>x</span><span>,</span><span> </span><span>fill</span><span> </span><span>=</span><span> </span><span>x</span><span>))</span><span> </span><span>+</span><span>
</span><span>facet_wrap</span><span>(</span><span>~</span><span> </span><span>x</span><span>,</span><span> </span><span>scales</span><span> </span><span>=</span><span> </span><span>&quot;free&quot;</span><span>,</span><span> </span><span>ncol</span><span> </span><span>=</span><span> </span><span>4</span><span>)</span><span> </span><span>+</span><span>
</span><span>geom_bar</span><span>(</span><span>stat</span><span> </span><span>=</span><span> </span><span>&quot;identity&quot;</span><span>,</span><span> </span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.8</span><span>)</span><span> </span><span>+</span><span>
</span><span>scale_color_manual</span><span>(</span><span>values</span><span> </span><span>=</span><span> </span><span>palette_light</span><span>())</span><span> </span><span>+</span><span>
</span><span>scale_fill_manual</span><span>(</span><span>values</span><span> </span><span>=</span><span> </span><span>palette_light</span><span>())</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span> </span><span>+</span><span>
</span><span>theme</span><span>(</span><span>axis.text.x</span><span> </span><span>=</span><span> </span><span>element_text</span><span>(</span><span>angle</span><span> </span><span>=</span><span> </span><span>45</span><span>,</span><span> </span><span>vjust</span><span> </span><span>=</span><span> </span><span>1</span><span>,</span><span> </span><span>hjust</span><span> </span><span>=</span><span> </span><span>1</span><span>))</span><span>
</span></code></pre>
</div>
<p><img alt="" src="https://i0.wp.com/shiring.github.io/forecasting/2017/06/09/retail_forcasting_part2_files/figure-markdown_github/unnamed-chunk-11-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<div>
<pre><code><span>augment</span><span>(</span><span>fit_lm</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>date</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>.resid</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_hline</span><span>(</span><span>yintercept</span><span> </span><span>=</span><span> </span><span>0</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>&quot;red&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>geom_point</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>palette_light</span><span>()[[</span><span>1</span><span>]])</span><span> </span><span>+</span><span>
</span><span>geom_smooth</span><span>()</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span>
</span></code></pre>
</div>
<p><img alt="" src="https://i1.wp.com/shiring.github.io/forecasting/2017/06/09/retail_forcasting_part2_files/figure-markdown_github/unnamed-chunk-12-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p></p>
<p>With this model, we can now add predictions and residuals for the test data…</p>
<div>
<pre><code><span>pred_test</span><span> </span><span>&lt;-</span><span> </span><span>test</span><span> </span><span>%>%</span><span>
</span><span>add_predictions</span><span>(</span><span>fit_lm</span><span>,</span><span> </span><span>&quot;pred_lm&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>add_residuals</span><span>(</span><span>fit_lm</span><span>,</span><span> </span><span>&quot;resid_lm&quot;</span><span>)</span><span>
</span></code></pre>
</div>
<p>… and visualize the residuals.</p>
<div>
<pre><code><span>pred_test</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>date</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>resid_lm</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_hline</span><span>(</span><span>yintercept</span><span> </span><span>=</span><span> </span><span>0</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>&quot;red&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>geom_point</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>palette_light</span><span>()[[</span><span>1</span><span>]])</span><span> </span><span>+</span><span>
</span><span>geom_smooth</span><span>()</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span>
</span></code></pre>
</div>
<p><img alt="" src="https://i1.wp.com/shiring.github.io/forecasting/2017/06/09/retail_forcasting_part2_files/figure-markdown_github/unnamed-chunk-14-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>We can also compare the predicted with the actual sum income in the test set.</p>
<div>
<pre><code><span>pred_test</span><span> </span><span>%>%</span><span>
</span><span>gather</span><span>(</span><span>x</span><span>,</span><span> </span><span>y</span><span>,</span><span> </span><span>sum_income</span><span>,</span><span> </span><span>pred_lm</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>date</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>y</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>x</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_point</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>)</span><span> </span><span>+</span><span>
</span><span>geom_line</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>)</span><span> </span><span>+</span><span>
</span><span>scale_color_manual</span><span>(</span><span>values</span><span> </span><span>=</span><span> </span><span>palette_light</span><span>())</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span>
</span></code></pre>
</div>
<p><img alt="" src="https://i2.wp.com/shiring.github.io/forecasting/2017/06/09/retail_forcasting_part2_files/figure-markdown_github/unnamed-chunk-15-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p></p>
<h3>Forecasting</h3>
<p>Once we are satisfied with our model’s performance on the test set, we can use it to forecast future events. To create future time points for modeling, we need to extract the time index (the date column <code>day</code> in our data frame).</p>
<div>
<pre><code><span># Extract index
</span><span>idx</span><span> </span><span>&lt;-</span><span> </span><span>retail_p_day</span><span> </span><span>%>%</span><span>
</span><span>tk_index</span><span>()</span><span>
</span></code></pre>
</div>
<p>From this index we can generate the future time series.</p>
<p>Here, we need to beware of a couple of things. Most importantly, we need to account for the irregularity of our data: We never have data for Saturdays and we have a few random missing values in between, as can be seen in the <code>diff</code> column of <code>retail_p_day_aug</code> (1 day difference == 86400 seconds).</p>
<div>
<pre><code><span>retail_p_day_aug</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>date</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>diff</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_point</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>,</span><span> </span><span>aes</span><span>(</span><span>color</span><span> </span><span>=</span><span> </span><span>as.factor</span><span>(</span><span>diff</span><span>)))</span><span> </span><span>+</span><span>
</span><span>geom_line</span><span>(</span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>)</span><span> </span><span>+</span><span>
</span><span>scale_color_manual</span><span>(</span><span>values</span><span> </span><span>=</span><span> </span><span>palette_light</span><span>())</span><span> </span><span>+</span><span>
</span><span>theme_tq</span><span>()</span><span>
</span></code></pre>
</div>
<p><img alt="" src="https://i2.wp.com/shiring.github.io/forecasting/2017/06/09/retail_forcasting_part2_files/figure-markdown_github/unnamed-chunk-19-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>What dates are these? Let’s filter for dates with more than 1 day between the last recorded day that are not Sundays (as Saturdays are always off-days).</p>
<div>
<pre><code><span>retail_p_day_aug</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>date</span><span>,</span><span> </span><span>wday.lbl</span><span>,</span><span> </span><span>diff</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>wday.lbl</span><span> </span><span>!=</span><span> </span><span>&quot;Sunday&quot;</span><span> </span><span>&amp;</span><span> </span><span>diff</span><span> </span><span>></span><span> </span><span>86400</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>days_missing</span><span> </span><span>=</span><span> </span><span>diff</span><span> </span><span>/</span><span> </span><span>86400</span><span> </span><span>-1</span><span>)</span><span>
</span></code></pre>
</div>
<div>
<pre><code>## # A tibble: 5 x 4
##         date wday.lbl    diff days_missing
##       </code></pre></div>