<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/big-news-vtreat-1-2-0-is-available-on-cran-and-it-is-now-big-data-capable/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.win-vector.com/blog/2018/06/big-news-vtreat-1-2-0-is-available-on-cran-and-it-is-now-big-data-capable/"> R – Win-Vector Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>We here at <a rel="nofollow" href="http://www.win-vector.com/" target="_blank">Win-Vector LLC</a> have some really big news we would please like the <a rel="nofollow" href="https://www.r-project.org/" target="_blank"><code>R</code></a>-community’s help sharing.</p>
<p><a rel="nofollow" href="https://winvector.github.io/vtreat/" target="_blank"><code>vtreat</code></a> <a rel="nofollow" href="https://cran.r-project.org/package=vtreat" target="_blank">version 1.2.0 is now available on CRAN</a>, and this version of <code>vtreat</code> can now implement its data cleaning and preparation steps on databases and big data systems such as <a rel="nofollow" href="https://spark.apache.org/" target="_blank"><code>Apache Spark</code></a>.</p>
<p><a rel="nofollow" href="https://winvector.github.io/vtreat/" target="_blank"><code>vtreat</code></a> is a very complete and <a rel="nofollow" href="https://arxiv.org/abs/1611.09477" target="_blank">rigorous</a> tool for preparing messy real world data for supervised machine-learning tasks. It implements a technique we call “safe y-aware processing” <a rel="nofollow" href="https://winvector.github.io/vtreat/articles/vtreatCrossFrames.html" target="_blank">using cross-validation or stacking techniques</a>. It is very easy to use: you show it some data and it designs a data transform for you.</p>
<p>Thanks to the <a rel="nofollow" href="https://cran.r-project.org/package=rquery" target="_blank"><code>rquery</code></a> package, this data preparation transform can now be directly applied to databases, or big data systems such as <a rel="nofollow" href="https://www.postgresql.org/" target="_blank"><code>PostgreSQL</code></a>, <a rel="nofollow" href="https://aws.amazon.com/redshift/" target="_blank"><code>Amazon RedShift</code></a>, <a rel="nofollow" href="https://spark.apache.org/" target="_blank"><code>Apache Spark</code></a>, or <a rel="nofollow" href="https://cloud.google.com/bigquery/" target="_blank"><code>Google BigQuery</code></a>. Or, thanks to the <a rel="nofollow" href="https://github.com/Rdatatable/data.table/wiki" target="_blank"><code>data.table</code></a> and <a rel="nofollow" href="https://cran.r-project.org/package=rqdatatable" target="_blank"><code>rqdatatable</code></a> packages, even fast large in-memory transforms are possible.</p>
<p>We have some basic examples of the new <code>vtreat</code> capabilities <a rel="nofollow" href="https://github.com/WinVector/vtreat/blob/master/extras/vtreatOnSpark.md" target="_blank">here</a> and <a rel="nofollow" href="https://github.com/WinVector/vtreat/blob/master/extras/rquery_vtreat.md" target="_blank">here</a>.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.win-vector.com/blog/2018/06/big-news-vtreat-1-2-0-is-available-on-cran-and-it-is-now-big-data-capable/"> R – Win-Vector Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/suA87RPy4I4" height="1"></a></div>