<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/vtreat-0-5-27-released-on-cran/"></div></div><div><a data-url="https://www.r-bloggers.com/vtreat-0-5-27-released-on-cran/" data-counturl="https://www.r-bloggers.com/vtreat-0-5-27-released-on-cran/" data-count="horizontal" data-text="vtreat 0.5.27 released on CRAN" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.win-vector.com/blog/2016/08/vtreat-0-5-27-released-on-cran/"> R – Win-Vector Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a target="_blank" href="http://www.win-vector.com/">Win-Vector LLC</a>, <a target="_blank" href="https://ninazumel.com/">Nina Zumel</a> and <a target="_blank" href="http://johnmount.com/">I</a> are pleased to announce that <a target="_blank" href="https://cran.r-project.org/package=vtreat">‘vtreat’ version 0.5.27 has been released on CRAN</a>. </p>
<blockquote><p><code>vtreat</code> is a data.frame processor/conditioner that prepares real-world data for predictive modeling in a statistically sound manner.</p>
<p><a target="_blank" href="http://winvector.github.io/vtreat/vtreat.html">(from the package documentation)</a>

</blockquote>
<p>Very roughly <code>vtreat</code> accepts an arbitrary “from the wild” data frame (with different column types, <code>NA</code>s, <code>NaN</code>s and so forth) and returns a transformation that reliably and repeatably converts similar data frames to numeric (matrix-like) frames (all independent variables numeric free of <code>NA</code>, <code>NaN</code>s, infinities, and so on) ready for predictive modeling. This is a systematic way to work with high-cardinality character and factor variables (which are incompatible with some machine learning implementations such as <a target="_blank" href="https://cran.r-project.org/package=randomForest">random forest</a>, and also bring in a danger of statistical over-fitting) and leaves the analyst more time to incorporate domain specific data preparation (as <code>vtreat</code> tries to handle as much of the common stuff as practical). For more of an overall description please see <a target="_blank" href="http://www.win-vector.com/blog/2016/07/vtreat-version-0-5-26-released-on-cran/">here</a>.</p>
<p>We suggest any users please update (and you will want to re-run any “design” steps instead of mixing “design” and “prepare” from two different versions of <code>vtreat</code>).</p>
<p>For what is new in version 0.5.27 please read on.<span></span></p>
<p>
<p><code>vtreat</code> 0.5.27 is a maintenance release. User visible improvements include.</p>
<ul>
<li>Switching `catB` encodings to a logit scale (instead of the previous log scale).</li>
<li>Increasing the degree of parallelism by separately parallelizing the level pruning steps (using the methods outlined <a target="_blank" href="http://www.win-vector.com/blog/2016/08/can-you-nest-parallel-operations-in-r/">here</a>).</li>
<li>Changing the default for <code>catScaling</code> to <code>FALSE</code>. We still think working logistic link-space is a great idea for classification problems, we are just not fully satisfied that un-regularized logistic regressions are the best way to get there (largely due to issues of separation and quasi-separation). In the meantime we think working in an expectation space is the safer (and now default) alternative.</li>
<li>Falling back to <code>stats::chisq.test()</code> instead of insisting on <code>stats::fisher.test()</code> for large counts. This calculation is used for level pruning and only relevant if <code>rareSig &lt; 1</code> (the default is <code>1</code>). We caution that setting <code>rareSig &lt; 1</code> remains a fairly expensive setting. We are trying to make significance estimation much more transparent, for example we now return how many extra degrees of freedom are hidden by categorical variable re-encodings in a new score frame column called <code>extraModelDegrees</code> (found in <code>designTreatments*()$scoreFrame</code>).</li>
</ul>
<p>The idea is having data preparation as a re-usable library lets us research, document, optimize, and fine tune a lot more details than would make sense on any one analysis project. The main design difference from other data preparation packages is we emphasize <a target="_blank" href="https://github.com/WinVector/Examples/blob/master/PCR/YAwarePCA_BARUG.pdf">“y-aware”</a> (or outcome aware) processing (using the training outcome to generate useful re-encodings of the data).</p>
<p>We have pre-rendered a lot of the package documentation, examples, and tutorials <a target="_blank" href="http://winvector.github.io/vtreat/">here</a>.</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/vtreat-0-5-27-released-on-cran/"></div></div><div><a data-url="https://www.r-bloggers.com/vtreat-0-5-27-released-on-cran/" data-counturl="https://www.r-bloggers.com/vtreat-0-5-27-released-on-cran/" data-count="vertical" data-text="vtreat 0.5.27 released on CRAN" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.win-vector.com/blog/2016/08/vtreat-0-5-27-released-on-cran/"> R – Win-Vector Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/FnXqN05FKAY"></a></div>