<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/introducing-padr/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://edwinth.github.io/blog/padr-intro/"> That’s so Random</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I am happy to introduce the <code>padr</code> package, which is now available on CRAN. If you frequently work with data containing a timestamp, especially automatically created data, you might find this package helpful. It solves two problems that you can be confronted with when preparing datetime data for analysis. First, data is often recorded on too low a level for your analysis. For instance the timestamp records the moment up to the second, where you want to do the analysis on an hourly level. Second, when no events toke place there are typically no data records. This is sensible from a storage perspective, but often unhelpful for analyzing the data. When calculating a moving average for example, you want missing observations to have the value 0. You don’t want them to be lacking from your set.</p>
<h2>Aggregate data with <code>thicken</code></h2>
<p>When your data is on too low a level, you need to aggregate it to a higher level. <code>thicken</code> creates this higher level. Lets say we want to graph the expenses at a coffee place and we have a data set containing a timestamp and the amount spent.</p>
<div>
<pre><code><span>library</span><span>(</span><span>padr</span><span>)</span><span>
</span><span>library</span><span>(</span><span>dplyr</span><span>)</span><span>
</span><span>coffee</span><span> </span><span>%>%</span><span> </span><span>thicken</span><span>(</span><span>interval</span><span> </span><span>=</span><span> </span><span>&quot;day&quot;</span><span>)</span><span>
</span></code></pre>
</div>
<div>
<pre><code>##            time_stamp amount time_stamp_day
## 1 2016-07-07 09:11:21   3.14     2016-07-07
## 2 2016-07-07 09:46:48   2.98     2016-07-07
## 3 2016-07-09 13:25:17   4.11     2016-07-09
## 4 2016-07-10 10:45:11   3.14     2016-07-10
</code></pre>
</div>
<p>So we fed <code>thicken</code> the data frame and it did a few things for us. First, it figured out what the datetime variable in the dataset was. Then it noticed the <em>interval</em> of the dataset, which is “second” in this case. Next, it mapped each timestamp to the corresponding “day”, which is the desired <em>interval</em> of the output. Finally, it added the mapping as a new column to our data frame. We can now easily aggregate to this new variable, for instance using the <code>dplyr</code> functions <code>group_by</code> and <code>summarise</code>.</p>
<div>
<pre><code><span>coffee_day</span><span> </span><span>&lt;-</span><span> </span><span>coffee</span><span> </span><span>%>%</span><span> </span><span>thicken</span><span>(</span><span>interval</span><span> </span><span>=</span><span> </span><span>&quot;day&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>group_by</span><span>(</span><span>time_stamp_day</span><span>)</span><span> </span><span>%>%</span><span> </span><span>summarise</span><span>(</span><span>total</span><span> </span><span>=</span><span> </span><span>sum</span><span>(</span><span>amount</span><span>))</span><span>
</span><span>coffee_day</span><span>
</span></code></pre>
</div>
<div>
<pre><code>## # A tibble: 3 &lt;U+00D7> 2
##   time_stamp_day total
##           &lt;date> &lt;dbl>
## 1     2016-07-07  6.12
## 2     2016-07-09  4.11
## 3     2016-07-10  3.14
</code></pre>
</div>
<h2>Fill gaps with <code>pad</code></h2>
<p>When we would make a line graph of <code>coffee_day</code>, it would skip over 2016-07-08, when there was no visit. This is not what we wanted. Since the expenses at this day were 0, we would like a record here indicating there were no puchases. We now invoke the <code>pad</code> function to insert this record.</p>
<div>
<pre><code><span>coffee_full</span><span> </span><span>&lt;-</span><span> </span><span>coffee_day</span><span> </span><span>%>%</span><span> </span><span>pad</span><span>
</span><span>coffee_full</span><span>
</span></code></pre>
</div>
<div>
<pre><code>## # A tibble: 4 &lt;U+00D7> 2
##   time_stamp_day total
##           &lt;date> &lt;dbl>
## 1     2016-07-07  6.12
## 2     2016-07-08    NA
## 3     2016-07-09  4.11
## 4     2016-07-10  3.14
</code></pre>
</div>
<p>Like <code>thicken</code>, the function figured out what the datetime variable was, and that it has the interval “day”. It then noticed that an observation is missing from this interval, and it created a record for it. The other variable, <em>total</em>, got a missing value inserted for this record.</p>
<h2>Use <code>fill_</code> for missing values</h2>
<p>So <code>pad</code> will always leave us with a data frame with missing values for the inserted records. How we want to fill this missings depends on the data at hand. Sometimes you would like to carry the last value forward, you can then use <code>tidyr::fill</code>. In this case we would like to fill the NA with 0, for which we use <code>fill_by_value</code>. Other functions available in <code>padr</code> are <code>fill_by_function</code> and <code>fill_by_prevalent</code>.</p>
<div>
<pre><code><span>library</span><span>(</span><span>ggplot2</span><span>)</span><span>
</span><span>coffee_full</span><span> </span><span>%>%</span><span> </span><span>fill_by_value</span><span>(</span><span>total</span><span>)</span><span> </span><span>%>%</span><span> </span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>time_stamp_day</span><span>,</span><span> </span><span>total</span><span>))</span><span> </span><span>+</span><span> </span><span>geom_line</span><span>()</span><span>
</span></code></pre>
</div>
<p><img alt="Coffee plot" src="https://i2.wp.com/edwinth.github.io/images/2017-01-18/coffee_plot.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>So here we have our plot, and here we have our quick overview of <code>padr</code>. A more thorough introduction can be found in <code>vignette(&quot;padr&quot;)</code>. Here I also get into more detail on the meaning of “interval” within the package. Feedback is very welcome, either send me an email, open an issue on github, or do a pull request. The code is maintained <a target="_blank" rel="nofollow" href="https://github.com/edwinth/padr">here</a>. Finally, I’d like to thank Barret Schloerke for his contributions at making this package CRAN ready.</p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://edwinth.github.io/blog/padr-intro/"> That’s so Random</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/FIZyhkQsXY8"></a></div>