<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/fst-fast-serialization-of-r-data-frames/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2017/02/fst-fast-serialization-of-r-data-frames.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>If you want to get data out of R and into another application or system, simply copying the data as it resides in memory generally isn't an option. Instead you have to <em>serialize</em> the data (into a file, usually), which the other application can then <em>deserialize</em> to recreate the original data. R has several options to serialize data frames:</p>
<ul>
<li>You can serialize (export) data to comma-separated files (CSVs), which can be imported by just about any application. R has several packages to read and write CSVs, including <code><a target="_blank" rel="nofollow" href="https://www.rdocumentation.org/packages/data.table/versions/1.10.4/topics/fwrite">fwrite</a></code> and <code><a target="_blank" rel="nofollow" href="https://www.rdocumentation.org/packages/data.table/versions/1.10.4/topics/fread">fread</a></code> from the data.table package. The downside is that as an ASCII format, CSVs are inefficient, particularly for numeric data.</li>
<li>The base R function <code><a target="_blank" rel="nofollow" href="https://www.rdocumentation.org/packages/base/versions/3.3.2/topics/readRDS">saveRDS</a></code> (and its deserialization counterpart, <code>readRDS</code>) can write any R object to a file. This is fairly efficient binary representation of the data, but not many applications can read RDS files. </li>
<li>The <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/05/feather-package.html">feather package</a> provides the functions <code><a target="_blank" rel="nofollow" href="https://www.rdocumentation.org/packages/feather/versions/0.3.1/topics/read_feather">read_feather</a></code> and <code><a target="_blank" rel="nofollow" href="https://www.rdocumentation.org/packages/feather/versions/0.3.1/topics/write_feather">write_feather</a></code>, an efficient binary format based on the open <a target="_blank" rel="nofollow" href="http://arrow.apache.org/">Apache Arrow framework</a>.</li>
</ul>
<p>And now there's a new package to add to the list: the <a target="_blank" rel="nofollow" href="http://www.fstpackage.org/">fst package</a>. Like the <a target="_blank" rel="nofollow" href="https://github.com/Rdatatable/data.table/wiki">data.table</a> package (the fast data.frame replacement for R), the primary focus of the fst package is speed. The chart below compares the speed of reading and writing data to/from CSV files (with fwrite/fread), feather, fts, and the native R RDS format. The vertical axis is throughput in megabytes per second — more is better. As you can see, fst outperforms the other options for both reading (orange) and writing (green). </p>
<p><a target="_blank" rel="nofollow" href="http://www.fstpackage.org/"><img title="Fst benchmarks" alt="Fst benchmarks" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b8d25bba6e970c-800wi" border="0"></a></p>
<p>The fst package achieves these impressive benchmarks thanks to the magic of compression. Even in this modern age of fast, solid-state storage, it's still (usually) faster to spend time using the CPU to compress the data first, rather than simply writing a larger file to disk. (The same applies to de-compression, and because that's an easier task than compression, there are even more performance gains to be had when reading.) The benefits are dependent on the data itself though: data without a lot of repetition (or, in the worst case, truly random numerical data) won't see performance gains like this.</p>
<p>Nonetheless, fst looks like it will be a useful package for applications that need to export data from R and into another R session as quickly as possible. (The fst format isn't supported by any systems other than R, as far as I know.) The package is still in its early days — the authors warn that the file format is likely to change in the future — but it likely has a place in high-performance R applications that rely on data transfer.</p>
<p><a target="_blank" rel="nofollow" href="http://www.fstpackage.org/">fst package</a>: Lightning Fast Serialization of Data Frames for R</p>
</div>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2017/02/fst-fast-serialization-of-r-data-frames.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/dTJjgptsrLQ"></a></div>