<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/feather-fast-interoperable-data-importexport-for-r/"></div></div><div><a data-url="http://www.r-bloggers.com/feather-fast-interoperable-data-importexport-for-r/" data-counturl="http://www.r-bloggers.com/feather-fast-interoperable-data-importexport-for-r/" data-count="horizontal" data-text="Feather: fast, interoperable data import/export for R" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2016/05/feather-package.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>Unlike most other statistical software packages, R doesn't have a native data file format. You can certainly import and export data in any number of formats, but there's no native &quot;R data file format&quot;. The closest equivalent is the <code>saveRDS</code>/<code>loadRDS</code> function pair, which allows you to serialize an R object to a file and then load it back into a later R session. But these files don't hew to a standardized format (it's essentially a dump of R in-memory representation of the object), and so you can't read the data with any software other than R.</p>
<p>The goal of the <a target="_blank" href="https://blog.rstudio.org/2016/03/29/feather/">feather project</a>, a collaboration of Wes McKinney and Hadley Wickham, is to create a standard data file format that can be used for data exchange by and between R, Python, and any other software that implements its <a target="_blank" href="https://github.com/wesm/feather">open-source format</a>. Data are stored in a computer-native binary format, which makes the files small (a 10-digit integer takes just 4 bytes, instead of the 10 ASCII characters required by a CSV file), and fast to read and write (no need to convert numbers to text and back again). Another reason why feather is fast is that it's a <em>column-oriented</em> file format, which matches R's internal representation of data. (In fact, feather is based on the <a target="_blank" href="http://arrow.apache.org/">Apache Arrow framework</a> for working with columnar data stores.) When reading or writing traditional data files with R, it must spend signfican time translating the data from column format to row format and back again; with feather the entire second step in the process below is eliminated.</p>
<p> </p>
<div><a target="_blank" href="http://a6.typepad.com/6a0105360ba1c6970c01b7c8616f0e970b-pi"><img title="Columns" alt="Columns" src="http://a6.typepad.com/6a0105360ba1c6970c01b7c8616f0e970b-800wi" border="0"></a>
<div>The three-step process of importing/exporting traditional row-based data formats with R. Diagram credit: Hadley Wickham.</div>
</div>
<p> </p>
<p> </p>
<p>For users of R 3.3.0 and later, the <a target="_blank" href="https://cran.r-project.org/web/packages/feather/index.html">feather package</a> is now available on CRAN. (Users of older versions of R can install <a target="_blank" href="https://github.com/wesm/feather/tree/master/R">feather from GitHub</a>.) With feather installed, you can read and write R data frames to feather files using simple functions:</p>
<p><code>write_feather(mtcars. &quot;mtcars.feather&quot;)</code></p>
<p><code>mtcars2 &lt;- read_feather(&quot;mtcars.feather&quot;)</code></p>
<p>Better yet, the <span>mtcars.feather</span> file can easily be read into Python, using its <a target="_blank" href="https://pypi.python.org/pypi/feather-format">feather-format</a> package. This example uses the small built-in <span>mtcars</span> data frame, but you should see a significant performance impact when working with larger data. Eduardo Ariño de la Rubia <a target="_blank" href="https://blog.dominodatalab.com/the-r-data-i-o-shootout/">performed some benchmarking of feather</a>, and found it to be significantly faster for ingesting data than other popular R functions. The chart below compares using feather, the <a target="_blank" href="https://github.com/Rdatatable/data.table/wiki">data.table package</a>, and <code>loadRDS</code> to import  508Mb file of 8.5 million rows and 7 columns:</p>
<p><a target="_blank" href="https://blog.dominodatalab.com/the-r-data-i-o-shootout/"><img title="Featherbenchmark" alt="Featherbenchmark" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01b7c86170e4970b-400wi"></a></p>
<p>Feather wasn't the fastest function benchmarked for <em>writing</em> data — data.table's <code>fwrite</code> function generally performed a bit better — but given that you typically read a file more often than writing it, the speedups should be very noticable in day-to-day data science activites.</p>
<p>For more on the feather package, check out its announcement from the RStudio blog linked below. </p>
<p>RStudio blog: <a target="_blank" href="https://blog.rstudio.org/2016/03/29/feather/">Feather: A Fast On-Disk Format for Data Frames for R and Python, powered by Apache Arrow</a></p>
</div>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/feather-fast-interoperable-data-importexport-for-r/"></div></div><div><a data-url="http://www.r-bloggers.com/feather-fast-interoperable-data-importexport-for-r/" data-counturl="http://www.r-bloggers.com/feather-fast-interoperable-data-importexport-for-r/" data-count="vertical" data-text="Feather: fast, interoperable data import/export for R" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2016/05/feather-package.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/0e8yAhIeopk"></a></div>