<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/creating-integer64-and-nanotime-vectors-in-c/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://gallery.rcpp.org//articles/creating-integer64-and-nanotime-vectors/"> Rcpp Gallery</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h3>Motivation: More Precise Timestamps</h3>
<p>R has excellent facilities for dealing with both dates and datetime objects.<br>
For datetime objects, the <code>POSIXt</code> time type can be mapped to <code>POSIXct</code> and<br>
its representation of fractional seconds since the January 1, 1970 “epoch” as<br>
well as to the broken-out list representation in <code>POSIXlt</code>. Many add-on<br>
packages use these facilities.</p>
<p><code>POSIXct</code> uses a double to provide 53 bits of resolution. That is generally<br>
good enough for timestamps down to just above a microsecond, and has served<br>
the R community rather well.</p>
<p>But increasingly, time increments are measure in nanoseconds. Other languages uses a (signed)<br>
64-bit integer to represent (integer) nanoseconds since the epoch. A bit over a year I realized<br>
that we have this in R too—by combining the <code>integer64</code> type in the<br>
<a rel="nofollow" href="https://cran.r-project.org/package=bit64" target="_blank">bit64</a> package by Jens Oehlschlaegel with the<br>
<a rel="nofollow" href="https://github.com/google/cctz" target="_blank">CCTZ</a>-based parser and formatter in my<br>
<a rel="nofollow" href="http://dirk.eddelbuettel.com/code/rcpp.cctz.html" target="_blank">RcppCCTZ</a> package. And thus the<br>
<a rel="nofollow" href="http://dirk.eddelbuettel.com/code/nanotime.html" target="_blank">nanotime</a> package was created.</p>
<p><a rel="nofollow" href="https://gitlab.com/lsilvest" target="_blank">Leonardo Silvestri</a> then significantly enhanced<br>
<a rel="nofollow" href="http://dirk.eddelbuettel.com/code/nanotime.html" target="_blank">nanotime</a> by redoing it as an S4 class.</p>
<p>A simple example:</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>nanotime</span><span>)</span><span>
</span><span>n</span><span> </span><span>&lt;-</span><span> </span><span>nanotime</span><span>(</span><span>42</span><span>)</span><span>
</span><span>n</span></code></pre>
</figure>
<pre>
[1] &quot;1970-01-01T00:00:00.000000042+00:00&quot;
</pre>
<p>Here we used a single element with value 42, and created a <code>nanotime</code> vector from it—which is<br>
taken to me 42 nanoseconds since the epoch, or basically almost at January 1, 1970.</p>
<h3>Step 1: Large Integer Types</h3>
<p>So more recently I had a need to <em>efficiently</em> generate such integer vector from <code>int64_t</code> data.<br>
Both <a rel="nofollow" href="https://gitlab.com/lsilvest" target="_blank">Leonardo</a> and <a rel="nofollow" href="https://github.com/dcdillon" target="_blank">Dan</a> helped with<br>
initial discussion and tests. One can either use a <code>reinterpret_cast&lt;></code> or a straight <code>memcpy</code> as<br>
the key trick in <a rel="nofollow" href="https://cran.r-project.org/package=bit64" target="_blank">bit64</a> is to use the underlying 64-bit<br>
<code>double</code>. So we have the space, we just need to ensure we copy <em>the bits</em> rather than their values.<br>
This leads to the following function to create an <code>integer64</code> vector for use in R at the C++ level:</p>
<figure>
<pre><code data-lang="cpp"><span>#include &lt;Rcpp.h>
</span>                                        
<span>Rcpp</span><span>::</span><span>NumericVector</span> <span>makeInt64</span><span>(</span><span>std</span><span>::</span><span>vector</span><span></span><span>int64_t</span><span>></span> <span>v</span><span>)</span> <span>{</span>
    <span>size_t</span> <span>len</span> <span>=</span> <span>v</span><span>.</span><span>size</span><span>();</span>
    <span>Rcpp</span><span>::</span><span>NumericVector</span> <span>n</span><span>(</span><span>len</span><span>);</span>         <span>// storage vehicle we return them in
</span>
    <span>// transfers values 'keeping bits' but changing type
</span>    <span>// using reinterpret_cast would get us a warning
</span>    <span>std</span><span>::</span><span>memcpy</span><span>(</span><span>&amp;</span><span>(</span><span>n</span><span>[</span><span>0</span><span>]),</span> <span>&amp;</span><span>(</span><span>v</span><span>[</span><span>0</span><span>]),</span> <span>len</span> <span>*</span> <span>sizeof</span><span>(</span><span>double</span><span>));</span>

    <span>n</span><span>.</span><span>attr</span><span>(</span><span>&quot;class&quot;</span><span>)</span> <span>=</span> <span>&quot;integer64&quot;</span><span>;</span>
    <span>return</span> <span>n</span><span>;</span>
<span>}</span></code></pre>
</figure>
<p>This uses the standard trick of setting a <code>class</code> attribute to set an S3 class. Now the values in<br>
<code>v</code> will return to R (exactly how is treated below), and R will treat the vector as <code>integer64</code><br>
object (provided the <a rel="nofollow" href="https://cran.r-project.org/package=bit64" target="_blank">bit64</a> package has been loaded).</p>
<h3>Step 2: Nanotime</h3>
<p>A <code>nanotime</code> vector is creating using an internal <code>integer64</code> vector. So the previous functions<br>
almost gets us there. But we need to set the S4 type correctly. So that needed some extra work.<br>
The following function does it:</p>
<figure>
<pre><code data-lang="cpp"><span>#include &lt;Rcpp.h>
</span>
<span>Rcpp</span><span>::</span><span>S4</span> <span>makeNanotime</span><span>(</span><span>std</span><span>::</span><span>vector</span><span></span><span>int64_t</span><span>></span> <span>v</span><span>)</span> <span>{</span>
    <span>size_t</span> <span>len</span> <span>=</span> <span>v</span><span>.</span><span>size</span><span>();</span>
    <span>Rcpp</span><span>::</span><span>NumericVector</span> <span>n</span><span>(</span><span>len</span><span>);</span>         <span>// storage vehicle we return them in
</span>
    <span>// transfers values 'keeping bits' but changing type
</span>    <span>// using reinterpret_cast would get us a warning
</span>    <span>std</span><span>::</span><span>memcpy</span><span>(</span><span>&amp;</span><span>(</span><span>n</span><span>[</span><span>0</span><span>]),</span> <span>&amp;</span><span>(</span><span>v</span><span>[</span><span>0</span><span>]),</span> <span>len</span> <span>*</span> <span>sizeof</span><span>(</span><span>double</span><span>));</span>

    <span>// do what needs to be done for the S4-ness: class, and .S3Class
</span>    <span>// this was based on careful reading of .Internal(inspect(nanotime(c(0,1))))
</span>    <span>Rcpp</span><span>::</span><span>CharacterVector</span> <span>cl</span> <span>=</span> <span>Rcpp</span><span>::</span><span>CharacterVector</span><span>::</span><span>create</span><span>(</span><span>&quot;nanotime&quot;</span><span>);</span>
    <span>cl</span><span>.</span><span>attr</span><span>(</span><span>&quot;package&quot;</span><span>)</span> <span>=</span> <span>&quot;nanotime&quot;</span><span>;</span>
    <span>n</span><span>.</span><span>attr</span><span>(</span><span>&quot;.S3Class&quot;</span><span>)</span> <span>=</span> <span>&quot;integer64&quot;</span><span>;</span>
    <span>n</span><span>.</span><span>attr</span><span>(</span><span>&quot;class&quot;</span><span>)</span> <span>=</span> <span>cl</span><span>;</span>
    <span>SET_S4_OBJECT</span><span>(</span><span>n</span><span>);</span>

    <span>return</span> <span>Rcpp</span><span>::</span><span>S4</span><span>(</span><span>n</span><span>);</span>
<span>}</span></code></pre>
</figure>
<p>This creates a <code>nanotime</code> vector as a proper S4 object.</p>
<h3>Step 3: Returning them R via data.table</h3>
<p>The astute reader will have noticed that neither function had an <code>Rcpp::export</code> tag. This is<br>
because of the function argument: <code>int64_t</code> is not representable natively by R, which is why we<br>
need a workaround. <a rel="nofollow" href="https://github.com/mattdowle" target="_blank">Matt Dowle</a> has been very helpful in providing<br>
excellent support for <code>nanotime</code> in <a rel="nofollow" href="https://github.com/Rdatatable/data.table/wiki" target="_blank">data.table</a><br>
(even after we, ahem, borked it by switching from S3 to S4). This support was of course relatively<br>
straightforward because <a rel="nofollow" href="https://github.com/Rdatatable/data.table/wiki" target="_blank">data.table</a> already had<br>
support for the underlying <code>integer64</code>, and we had the additional formatters etc.</p>
<figure>
<pre><code data-lang="cpp"><span>#include &lt;Rcpp.h>
</span>
<span>// Enable C++11 via this plugin (Rcpp 0.10.3 or later)
// [[Rcpp::plugins(&quot;cpp11&quot;)]]
</span>
<span>Rcpp</span><span>::</span><span>NumericVector</span> <span>makeInt64</span><span>(</span><span>std</span><span>::</span><span>vector</span><span></span><span>int64_t</span><span>></span> <span>v</span><span>)</span> <span>{</span>
    <span>size_t</span> <span>len</span> <span>=</span> <span>v</span><span>.</span><span>size</span><span>();</span>
    <span>Rcpp</span><span>::</span><span>NumericVector</span> <span>n</span><span>(</span><span>len</span><span>);</span>         <span>// storage vehicle we return them in
</span>
    <span>// transfers values 'keeping bits' but changing type
</span>    <span>// using reinterpret_cast would get us a warning
</span>    <span>std</span><span>::</span><span>memcpy</span><span>(</span><span>&amp;</span><span>(</span><span>n</span><span>[</span><span>0</span><span>]),</span> <span>&amp;</span><span>(</span><span>v</span><span>[</span><span>0</span><span>]),</span> <span>len</span> <span>*</span> <span>sizeof</span><span>(</span><span>double</span><span>));</span>

    <span>n</span><span>.</span><span>attr</span><span>(</span><span>&quot;class&quot;</span><span>)</span> <span>=</span> <span>&quot;integer64&quot;</span><span>;</span>
    <span>return</span> <span>n</span><span>;</span>
<span>}</span>

<span>Rcpp</span><span>::</span><span>S4</span> <span>makeNanotime</span><span>(</span><span>std</span><span>::</span><span>vector</span><span></span><span>int64_t</span><span>></span> <span>v</span><span>)</span> <span>{</span>
    <span>size_t</span> <span>len</span> <span>=</span> <span>v</span><span>.</span><span>size</span><span>();</span>
    <span>Rcpp</span><span>::</span><span>NumericVector</span> <span>n</span><span>(</span><span>len</span><span>);</span>         <span>// storage vehicle we return them in
</span>
    <span>// transfers values 'keeping bits' but changing type
</span>    <span>// using reinterpret_cast would get us a warning
</span>    <span>std</span><span>::</span><span>memcpy</span><span>(</span><span>&amp;</span><span>(</span><span>n</span><span>[</span><span>0</span><span>]),</span> <span>&amp;</span><span>(</span><span>v</span><span>[</span><span>0</span><span>]),</span> <span>len</span> <span>*</span> <span>sizeof</span><span>(</span><span>double</span><span>));</span>

    <span>// do what needs to be done for the S4-ness: class, and .S3Class
</span>    <span>// this was based on careful reading of .Internal(inspect(nanotime(c(0,1))))
</span>    <span>Rcpp</span><span>::</span><span>CharacterVector</span> <span>cl</span> <span>=</span> <span>Rcpp</span><span>::</span><span>CharacterVector</span><span>::</span><span>create</span><span>(</span><span>&quot;nanotime&quot;</span><span>);</span>
    <span>cl</span><span>.</span><span>attr</span><span>(</span><span>&quot;package&quot;</span><span>)</span> <span>=</span> <span>&quot;nanotime&quot;</span><span>;</span>
    <span>n</span><span>.</span><span>attr</span><span>(</span><span>&quot;.S3Class&quot;</span><span>)</span> <span>=</span> <span>&quot;integer64&quot;</span><span>;</span>
    <span>n</span><span>.</span><span>attr</span><span>(</span><span>&quot;class&quot;</span><span>)</span> <span>=</span> <span>cl</span><span>;</span>
    <span>SET_S4_OBJECT</span><span>(</span><span>n</span><span>);</span>

    <span>return</span> <span>Rcpp</span><span>::</span><span>S4</span><span>(</span><span>n</span><span>);</span>
<span>}</span>

<span>// [[Rcpp::export]]
</span><span>Rcpp</span><span>::</span><span>DataFrame</span> <span>getDT</span><span>()</span> <span>{</span>
    <span>std</span><span>::</span><span>vector</span><span></span><span>int64_t</span><span>></span> <span>d</span> <span>=</span> <span>{</span> <span>1L</span><span>,</span> <span>1000L</span><span>,</span> <span>1000000L</span><span>,</span> <span>1000000000L</span> <span>};</span>
    <span>std</span><span>::</span><span>vector</span><span></span><span>int64_t</span><span>></span> <span>ns</span> <span>=</span> <span>{</span> <span>1510442294123456789L</span><span>,</span> <span>1510442295123456789L</span><span>,</span> 
        <span>1510442296123456789L</span><span>,</span> <span>1510442297123456789L</span> <span>};</span>
        
    <span>Rcpp</span><span>::</span><span>DataFrame</span> <span>df</span> <span>=</span> <span>Rcpp</span><span>::</span><span>DataFrame</span><span>::</span><span>create</span><span>(</span><span>Rcpp</span><span>::</span><span>Named</span><span>(</span><span>&quot;int64s&quot;</span><span>)</span> <span>=</span> <span>makeInt64</span><span>(</span><span>d</span><span>),</span>
                                                 <span>Rcpp</span><span>::</span><span>Named</span><span>(</span><span>&quot;nanos&quot;</span><span>)</span> <span>=</span> <span>makeNanotime</span><span>(</span><span>ns</span><span>));</span>
    <span>df</span><span>.</span><span>attr</span><span>(</span><span>&quot;class&quot;</span><span>)</span> <span>=</span> <span>Rcpp</span><span>::</span><span>CharacterVector</span><span>::</span><span>create</span><span>(</span><span>&quot;data.table&quot;</span><span>,</span> <span>&quot;data.frame&quot;</span><span>);</span>
    <span>return</span><span>(</span><span>df</span><span>);</span>
<span>}</span></code></pre>
</figure>
<h3>Example</h3>
<p>The following example shows the output from the preceding function:</p>
<figure>
<pre><code data-lang="r"><span>suppressMessages</span><span>(</span><span>library</span><span>(</span><span>&quot;data.table&quot;</span><span>))</span><span>
</span><span>dt</span><span> </span><span>&lt;-</span><span> </span><span>getDT</span><span>()</span><span>
</span><span>print</span><span>(</span><span>dt</span><span>)</span></code></pre>
</figure>
<pre>
       int64s                               nanos
1:          1 2017-11-11T23:18:14.123456789+00:00
2:       1000 2017-11-11T23:18:15.123456789+00:00
3:    1000000 2017-11-11T23:18:16.123456789+00:00
4: 1000000000 2017-11-11T23:18:17.123456789+00:00
</pre>
<figure>
<pre><code data-lang="r"><span>dt</span><span>[[</span><span>1</span><span>]]</span></code></pre>
</figure>
<pre>
integer64
[1] 1          1000       1000000    1000000000
</pre>
<figure>
<pre><code data-lang="r"><span>dt</span><span>[[</span><span>2</span><span>]]</span></code></pre>
</figure>
<pre>
[1] &quot;2017-11-11T23:18:14.123456789+00:00&quot;
[2] &quot;2017-11-11T23:18:15.123456789+00:00&quot;
[3] &quot;2017-11-11T23:18:16.123456789+00:00&quot;
[4] &quot;2017-11-11T23:18:17.123456789+00:00&quot;
</pre>
<figure>
<pre><code data-lang="r"><span>diff</span><span>(</span><span>dt</span><span>[[</span><span>2</span><span>]])</span><span> </span><span># here 1e9 nanoseconds between them</span></code></pre>
</figure>
<pre>
integer64
[1] 1000000000 1000000000 1000000000
</pre>
<p><img width="1" alt="" src="http://feeds.feedburner.com/~r/rcpp-gallery/~4/mAu5RjB2tO0" height="1"></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://gallery.rcpp.org//articles/creating-integer64-and-nanotime-vectors/"> Rcpp Gallery</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/tRkCjvPRK2k" height="1"></a></div>