<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/here-is-the-new-padr/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://edwinth.github.io/blog/padr-v0-3-0/"> That’s so Random</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I am very happy to announce v0.3.0 of the <code>padr</code> package, <a rel="nofollow" href="https://edwinth.github.io/blog/padr-intro/" target="_blank">which was introduced in January</a>. As requested by many, you are now able to use intervals of which the unit is different from 1. In earlier version the eight interval values only allowed for a single unit (e.g. year, day, hour). Now you can use any time period that is accepted by <code>seq.Date</code> or <code>seq.POSIXt</code> (e.g. 2 months, 6 hours, 5 minutes) in both <code>thicken</code> and <code>pad</code>. <code>get_interval</code> does test for both the interval and the unit of the interval of the datetime variable from now on.</p>
<div>
<pre><code><span>library</span><span>(</span><span>padr</span><span>)</span><span>
</span><span>library</span><span>(</span><span>tidyverse</span><span>)</span><span>
</span><span>get_interval</span><span>(</span><span>as.Date</span><span>(</span><span>c</span><span>(</span><span>&quot;2017-01-01&quot;</span><span>,</span><span> </span><span>&quot;2017-03-01&quot;</span><span>)))</span><span>
</span></code></pre>
</div>
<div>
<pre><code>## [1] &quot;2 month&quot;
</code></pre>
</div>
<p>With this new definition of the interval the possibilities of both <code>thicken</code> and <code>pad</code> are expanded. See the following analysis, where the new functionality is demonstrated by aggregating to daypart:</p>
<div>
<pre><code><span>emergency</span><span> </span><span>%>%</span><span> </span><span>filter</span><span>(</span><span>title</span><span> </span><span>==</span><span> </span><span>&quot;EMS: CARDIAC EMERGENCY&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>thicken</span><span>(</span><span>interval</span><span> </span><span>=</span><span> </span><span>&quot;6 hours&quot;</span><span>,</span><span> </span><span>colname</span><span> </span><span>=</span><span> </span><span>&quot;daypart&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>count</span><span>(</span><span>daypart</span><span>)</span><span> </span><span>%>%</span><span> </span><span>pad</span><span>()</span><span> </span><span>%>%</span><span> </span><span>fill_by_value</span><span>(</span><span>n</span><span>)</span><span> </span><span>%>%</span><span> </span><span>mutate</span><span>(</span><span>start_daypart</span><span> </span><span>=</span><span> </span><span>lubridate</span><span>::</span><span>hour</span><span>(</span><span>daypart</span><span>)</span><span> </span><span>%>%</span><span> </span><span>as.factor</span><span>())</span><span> </span><span>%>%</span><span> </span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>n</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_density</span><span>(</span><span>aes</span><span>(</span><span>fill</span><span> </span><span>=</span><span> </span><span>start_daypart</span><span>))</span><span> </span><span>+</span><span>
</span><span>facet_wrap</span><span>(</span><span>~</span><span>start_daypart</span><span>)</span><span>
</span></code></pre>
</div>
<div>
<pre><code>## pad applied on the interval: 6 hour
</code></pre>
</div>
<p><img alt="plot of chunk unnamed-chunk-2" src="https://i0.wp.com/edwinth.github.io/figure/source/2017-05-16-padr-v0-3-0/unnamed-chunk-2-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>The addition of unit specification to the interval made it unfortunately impossible to make v0.3.0 fully backwords compatible. The two main functions are affected in the the following way.</p>
<ul>
<li>
<p>In <code>thicken</code> the <code>interval</code> argument now has to be specified. In earlier versions it was optional. When it was not specified, the added variable was one interval level higher than that of the input datetime variable. With the widening of the interval definition, there is not longer a natural step up.</p>
</li>
<li>
<p>When the <code>interval</code> argument is not specified in <code>pad</code>, <code>get_interval</code> is applied on the datetime variable. Since this function now returns the units additionally to the interval, padding might now be done at a different interval. In earlier versions, the following dataframe would have 7 rows after padding because the interval of <code>dt</code> used to be “month”. Now <code>pad</code> would return only 4 rows, because the new interval is “2 month”.</p>
</li>
</ul>
<div>
<pre><code><span>example_df</span><span> </span><span>&lt;-</span><span> </span><span>data.frame</span><span>(</span><span>dt</span><span> </span><span>=</span><span> </span><span>as.Date</span><span>(</span><span>c</span><span>(</span><span>&quot;2017-01-01&quot;</span><span>,</span><span> </span><span>&quot;2017-03-01&quot;</span><span>,</span><span> </span><span>&quot;2017-07-01&quot;</span><span>)),</span><span>
</span><span>y</span><span> </span><span>=</span><span> </span><span>1</span><span>:</span><span>3</span><span>)</span><span>
</span><span>pad</span><span>(</span><span>example_df</span><span>)</span><span>
</span></code></pre>
</div>
<div>
<pre><code>## pad applied on the interval: 2 month
</code></pre>
</div>
<div>
<pre><code>##           dt  y
## 1 2017-01-01  1
## 2 2017-03-01  2
## 3 2017-05-01 NA
## 4 2017-07-01  3
</code></pre>
</div>
<p>One should thus be a little more careful that there is no higher unit within the interval that explains the data as well. To reduce the risk of padding at the wrong unit, <code>pad</code> now always prints the interval at which the padding occured.</p>
<h2>Reimplementation of <code>pad</code></h2>
<p>The second significant change in this version is the reimplementatiion of <code>pad</code>. Performance was poor when <code>pad</code> was applied on more than a handfull of groups. By leveraging <code>dplyr</code> this is now greatly improved.</p>
<p>Besides, functionality is slightly adjusted as well:</p>
<ul>
<li>
<p>When the <code>interval</code> argument is not specified and the <code>group</code> argument is specified, the interval is now determined on all observations instead of within each group separately. With the new definition of the interval it would far more likely that we get deviating intervals for small groups unintendedly. To prevent this kind of interval overfitting, groups are no longer taking into account when determining the interval.</p>
</li>
<li>
<p>For better integration with <code>dplyr</code>, it is now possible to specifiy the grouping variable(s) for <code>pad</code> with <code>dplyr::group_by</code>. The following two are interchangeable:</p>
</li>
</ul>
<div>
<pre><code><span>x</span><span> </span><span>&lt;-</span><span> </span><span>emergency</span><span> </span><span>%>%</span><span> </span><span>thicken</span><span>(</span><span>&quot;day&quot;</span><span>,</span><span> </span><span>&quot;d&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>count</span><span>(</span><span>title</span><span>,</span><span> </span><span>d</span><span>)</span><span>
</span><span>x</span><span> </span><span>%>%</span><span> </span><span>pad</span><span>(</span><span>group</span><span> </span><span>=</span><span> </span><span>&quot;title&quot;</span><span>)</span><span>
</span><span>x</span><span> </span><span>%>%</span><span> </span><span>group_by</span><span>(</span><span>title</span><span>)</span><span> </span><span>%>%</span><span> </span><span>pad</span><span>()</span><span>
</span></code></pre>
</div>
<ul>
<li>
<p>Moreover, both <code>pad</code> and <code>thicken</code> now maintain <code>dplyr</code> grouping. The grouping of their outputs is equal to the grouping of the input.</p>
</li>
<li>
<p><code>pad</code> has gained the <code>break_above</code> argument. This is a number in millions. Before starting to pad, the function makes an estimate of the number of returned rows. If the number of rows is above the number specified in <code>break_above</code> the function will break. This is a safety net for the situation where the interval of the datetime variable is more granualar than the user thinks it is. For instance, when forgetting to apply <code>thicken</code> first.</p>
</li>
<li>
<p>For determining the interval in <code>pad</code> the <code>start_val</code> and/or the <code>end_val</code> are taken into account, if specified. They are concatenated to the datetime variable before the interval is determined.</p>
</li>
</ul>
<h2>Other changes</h2>
<ul>
<li>
<p>The new function <code>pad_int</code> pads an integer column, instead of a datetime column. Helpful for datasets where time is specified as just the years, but also for interger id variabels that are incomplete.</p>
</li>
<li>
<p>The <code>fill_</code> used to require specification of all the column names that had to filled. This is annoying when many columns had to be filled. The functions no longer break when no variable names are specified, but instead fill all columns in the data frame.</p>
</li>
<li>
<p>In <code>thicken</code> observations before the <code>start_val</code> are now removed from the dataset. They used to be all mapped to the <code>start_val</code>. The behavior of <code>thicken</code> is thereby consistent with the behavior of <code>pad</code>.</p>
</li>
</ul>
<h2>Bug fixes</h2>
<ul>
<li>
<p>When the <code>end_val</code> is specified in <code>pad</code>, it would mistakenly update the <code>start_val</code> with its value. This resulted in the return of only the last line of the padded data.frame, instead of the full padded data.frame. (#13)</p>
</li>
<li>
<p>When dt_var has NULL as timezone, <code>to_posix</code> (helper of <code>round_thicken</code>, which itself is a helper of <code>thicken</code>) used to break, and thereby <code>thicken</code> itself broke. This is fixed. (#14)</p>
</li>
<li>
<p>In <code>pad</code> with grouping, the function will no longer breaks if for one of the groups the <code>start_val</code> is behind its last observation, or the <code>end_val</code> is before its first observation. Group is omitted and warning is thrown. If all groups are omitted, function breaks with an informative error. The same goes of course when there is no grouping. (#24)</p>
</li>
<li>
<p>Both <code>pad</code> and <code>thicken</code> now throw informative errors when the start_val or end_val (<code>pad</code> only) are of the wrong class.</p>
</li>
</ul>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://edwinth.github.io/blog/padr-v0-3-0/"> That’s so Random</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/Y_d2SXmjmAs" height="1"></a></div>