<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/padr-version-0-4-0-now-on-cran/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://edwinth.github.io/blog/padr-v04/"> That’s so Random</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I am happy to share that the latest version of <code>padr</code> just hit CRAN. This new version comprises bug fixes, performance improvements and new functions for formatting datetime variables. But above all, it introduces the custom paradigm that enables you to do asymmetric analysis.</p>
<h2>Improvements to existing functions</h2>
<p><code>thicken</code> used to get slowish when the data frame got large. Several adjustments resulted in a considerable performance gain. It is now approximately 6 times faster than in version 0.3.0.</p>
<p>Both <code>pad</code> and <code>thicken</code> used to break with noninformative errors when the datetime variable contains missing values. Both functions allow for missing values now. <code>thicken</code> will leave the record containing the missing datetime value where it is and will enter a missing value in the column added to the data frame. <code>pad</code> will move all the records with missing values in the datetime variable to the end of the data frame and will further ignore them for padding. Both functions throw a warning when the datetime variable has missing values.</p>
<h2>New functions</h2>
<p><code>span_date</code> and <code>span_time</code> are wrappers around <code>seq.Date</code> and <code>seq.POSIXt</code> and were previously described in <a rel="nofollow" href="https://edwinth.github.io/blog/span_date_time/" target="_blank">this blog post</a>.<br>
Two new functions are introduced to reformat datetime variable. These can be especially useful for showing the data in a plot or a graph. <code>center_interval</code> shifts the datetime point from the beginning of the interval to the (approximate) center. Especially bar, line and dot plots will reflect the data in a more accurate way after centering.</p>
<div>
<div>
<pre><code><span>library</span><span>(</span><span>tidyverse</span><span>)</span><span>
</span><span>library</span><span>(</span><span>padr</span><span>)</span><span>
</span><span>jan_first</span><span> </span><span>&lt;-</span><span> </span><span>emergency</span><span> </span><span>%>%</span><span> </span><span>filter</span><span>(</span><span>as.Date</span><span>(</span><span>time_stamp</span><span>,</span><span> </span><span>tz</span><span> </span><span>=</span><span> </span><span>&quot;EST&quot;</span><span>)</span><span> </span><span>==</span><span> </span><span>as.Date</span><span>(</span><span>&quot;2016-01-01&quot;</span><span>))</span><span> </span><span>%>%</span><span> </span><span>thicken</span><span>(</span><span>&quot;3 hour&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>count</span><span>(</span><span>time_stamp_3_hour</span><span>)</span><span> </span><span>ggplot</span><span>(</span><span>jan_first</span><span>,</span><span> </span><span>aes</span><span>(</span><span>time_stamp_3_hour</span><span>,</span><span> </span><span>n</span><span>))</span><span> </span><span>+</span><span> </span><span>geom_bar</span><span>(</span><span>stat</span><span> </span><span>=</span><span> </span><span>&quot;identity&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="plot of chunk unnamed-chunk-1" src="https://i1.wp.com/edwinth.github.io/figure/source/2017-11-17-padr-v04/unnamed-chunk-1-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<div>
<div>
<pre><code><span>jan_first</span><span> </span><span>%>%</span><span> </span><span>mutate</span><span>(</span><span>ts_center</span><span> </span><span>=</span><span> </span><span>center_interval</span><span>(</span><span>time_stamp_3_hour</span><span>))</span><span> </span><span>%>%</span><span> </span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>ts_center</span><span>,</span><span> </span><span>n</span><span>))</span><span> </span><span>+</span><span> </span><span>geom_bar</span><span>(</span><span>stat</span><span> </span><span>=</span><span> </span><span>&quot;identity&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="plot of chunk unnamed-chunk-2" src="https://i0.wp.com/edwinth.github.io/figure/source/2017-11-17-padr-v04/unnamed-chunk-2-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Secondly, there is the <code>format_interval</code> function, that will create a character from the interval using <code>strftime</code> on both the beginning and the end of the datetime points of each interval.</p>
<div>
<div>
<pre><code><span>jan_first</span><span> </span><span>%>%</span><span> </span><span>mutate</span><span>(</span><span>hours</span><span> </span><span>=</span><span> </span><span>format_interval</span><span>(</span><span>time_stamp_3_hour</span><span>,</span><span>
</span><span>start_format</span><span> </span><span>=</span><span> </span><span>&quot;%b%d %H&quot;</span><span>,</span><span>
</span><span>end_format</span><span> </span><span>=</span><span> </span><span>&quot;%H&quot;</span><span>,</span><span>
</span><span>sep</span><span> </span><span>=</span><span> </span><span>&quot;-&quot;</span><span>))</span><span> </span><span>%>%</span><span> </span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>hours</span><span>,</span><span> </span><span>n</span><span>))</span><span> </span><span>+</span><span> </span><span>geom_bar</span><span>(</span><span>stat</span><span> </span><span>=</span><span> </span><span>&quot;identity&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="plot of chunk unnamed-chunk-3" src="https://i2.wp.com/edwinth.github.io/figure/source/2017-11-17-padr-v04/unnamed-chunk-3-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>When using the interval “week”, one might want to start the weeks on different day than Sunday. In <code>thicken</code> the start_val should than be specified with a day of the desired weekday. Finding this day by hand is a bit tedious, thats why <code>closest_weekday</code> is introduced.</p>
<div>
<div>
<pre><code><span>thicken</span><span>(</span><span>coffee</span><span>,</span><span> </span><span>&quot;week&quot;</span><span>,</span><span> </span><span>start_val</span><span> </span><span>=</span><span> </span><span>closest_weekday</span><span>(</span><span>coffee</span><span>$</span><span>time_stamp</span><span>,</span><span> </span><span>6</span><span>))</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>##            time_stamp amount time_stamp_week
## 1 2016-07-07 09:11:21   3.14      2016-07-02
## 2 2016-07-07 09:46:48   2.98      2016-07-02
## 3 2016-07-09 13:25:17   4.11      2016-07-09
## 4 2016-07-10 10:45:11   3.14      2016-07-09
</code></pre>
</div>
</div>
<h2>The custom suite</h2>
<p><code>thicken</code> and <code>pad</code> are highly automated, because they assume all datetime points to be equally spaced. Internally they span vectors of the desired interval. However, it might be useful to have asymmetrical periods between datetime points. Especially when there are periods in which the number of observations is consistently lower, such as nightly hours or weekend days. The functions <code>thicken_cust</code> and <code>pad_cust</code> work like the original functions. However, the user has to provide his own spanning vector to whch the observations are mapped.</p>
<p>Lets do an analysis of vehicle accidents in the <code>emergency</code>. We want to distinguish between morning rush hour (7-10), daytime (10-16), evening rush hour (16-19) and nighttime (19-7). This is the full analysis.</p>
<div>
<div>
<pre><code><span>accidents_span</span><span> </span><span>&lt;-</span><span>
</span><span>span_around</span><span>(</span><span>emergency</span><span>$</span><span>time_stamp</span><span>,</span><span> </span><span>&quot;hour&quot;</span><span>,</span><span> </span><span>start_shift</span><span> </span><span>=</span><span> </span><span>&quot;2 hour&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>subset_span</span><span>(</span><span>list</span><span>(</span><span>hour</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>7</span><span>,</span><span> </span><span>10</span><span>,</span><span> </span><span>16</span><span>,</span><span> </span><span>19</span><span>)))</span><span>
</span><span>emergency</span><span> </span><span>%>%</span><span> </span><span>filter</span><span>(</span><span>title</span><span> </span><span>==</span><span> </span><span>&quot;Traffic: VEHICLE ACCIDENT -&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>thicken_cust</span><span>(</span><span>accidents_span</span><span>,</span><span> </span><span>&quot;hour&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>count</span><span>(</span><span>hour</span><span>)</span><span> </span><span>%>%</span><span> </span><span>pad_cust</span><span>(</span><span>accidents_span</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>fill_by_value</span><span>()</span><span> </span><span>%>%</span><span> </span><span>mutate</span><span>(</span><span>day</span><span> </span><span>=</span><span> </span><span>as.Date</span><span>(</span><span>hour</span><span>,</span><span> </span><span>tz</span><span> </span><span>=</span><span> </span><span>&quot;EST&quot;</span><span>),</span><span>
</span><span>period</span><span> </span><span>=</span><span> </span><span>format_interval</span><span>(</span><span>hour</span><span>,</span><span> </span><span>start_format</span><span> </span><span>=</span><span> </span><span>&quot;%H&quot;</span><span>,</span><span> </span><span>sep</span><span> </span><span>=</span><span> </span><span>&quot;-&quot;</span><span>))</span><span> </span><span>%>%</span><span> </span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>day</span><span>,</span><span> </span><span>n</span><span>,</span><span> </span><span>col</span><span> </span><span>=</span><span> </span><span>period</span><span>))</span><span> </span><span>+</span><span> </span><span>geom_line</span><span>()</span><span> </span><span>+</span><span>
</span><span>facet_wrap</span><span>(</span><span>~</span><span>period</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="plot of chunk unnamed-chunk-5" src="https://i2.wp.com/edwinth.github.io/figure/source/2017-11-17-padr-v04/unnamed-chunk-5-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>The helper functions <code>span_around</code> and <code>subset_span</code> are used for building the spanning vector. The first takes the original datetime variable and spans a variable of the requested interval around it. This saves you the manual trouble of finding the min and the max of the variable and determine which points are respectively before and after them to build a variable of the interval. <code>subset_span</code>, subsequently, will only leave the datetime points in the input that meet the criteria given in a list. In the example these are the hours 7, 10, 16, and 19. In total there are eight different datetime parts you can subset on.</p>
<p>The remainder of the analysis is greatly similar to a regular <code>padr</code> analysis. Instead of an interval to which to <code>thicken</code> and <code>pad</code> to, you use the asymmetrically spaced <code>accidents_span</code> variable in both <code>thicken_cust</code> and <code>pad_cust</code>. <code>thicken_cust</code> will then map each observation to a datetime point in the spanned vector. <code>pad_cust</code> will insert rows for each of the observations that are in the spanned vector, but not in the datetime variable.</p>
<h2>Thanks</h2>
<p>This release completes the initial plans I had for <code>padr</code>. For the next release I plan on refactoring and further increasing performance. Do you find anything still missing from <code>padr</code>? Did you find a bug or an inconsistency? Please notify by sending an email or file an issue on <a rel="nofollow" href="https://github.com/EdwinTh/padr" target="_blank">the github page</a>.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://edwinth.github.io/blog/padr-v04/"> That’s so Random</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/onNoH9xulco" height="1"></a></div>