<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/modeling-muti-category-outcomes-with-vtreat/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.win-vector.com/blog/2018/10/modeling-muti-category-outcomes-with-vtreat/"> R – Win-Vector Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a rel="nofollow" href="https://github.com/WinVector/vtreat" target="_blank"><code>vtreat</code></a> is a powerful <a rel="nofollow" href="https://www.r-project.org/" target="_blank"><code>R</code></a> package for preparing messy real-world data for machine learning. We have further extended the package with a number of features including <a rel="nofollow" href="https://cran.r-project.org/package=rquery" target="_blank">rquery</a>/<a rel="nofollow" href="https://cran.r-project.org/package=rqdatatable" target="_blank">rqdatatable</a> integration (allowing vtreat application at scale on Apache Spark or data.table!).</p>
<p>In addition <a rel="nofollow" href="https://github.com/WinVector/vtreat" target="_blank"><code>vtreat</code></a> and can now effectively prepare data for multi-class classification or multinomial modeling.</p>
<p><span></span></p>
<p>
<p>The two functions needed (<a rel="nofollow" href="https://winvector.github.io/vtreat/reference/mkCrossFrameMExperiment.html" target="_blank"><code>mkCrossFrameMExperiment()</code></a> and the <code>S3</code> method <a rel="nofollow" href="https://winvector.github.io/vtreat/reference/prepare.multinomial_plan.html" target="_blank"><code>prepare.multinomial_plan()</code></a>) are now part of <code>vtreat</code>.</p>
<p>Let’s work a specific example: trying to model multi-class <code>y</code> as a function of <code>x1</code> and <code>x2</code>.</p>
<div>
<pre><code><span>library</span>(<span>&quot;vtreat&quot;</span>)</code></pre>
</div>
<div>
<pre><code><span># create example data</span>
<span>set.seed</span>(<span>326346</span>)
sym_bonuses &lt;-<span> </span><span>rnorm</span>(<span>3</span>)
<span>names</span>(sym_bonuses) &lt;-<span> </span><span>c</span>(<span>&quot;a&quot;</span>, <span>&quot;b&quot;</span>, <span>&quot;c&quot;</span>)
sym_bonuses3 &lt;-<span> </span><span>rnorm</span>(<span>3</span>)
<span>names</span>(sym_bonuses3) &lt;-<span> </span><span>as.character</span>(<span>seq_len</span>(<span>length</span>(sym_bonuses3)))
n_row &lt;-<span> </span><span>1000</span>
d &lt;-<span> </span><span>data.frame</span>(
  <span>x1 =</span> <span>rnorm</span>(n_row),
  <span>x2 =</span> <span>sample</span>(<span>names</span>(sym_bonuses), n_row, <span>replace =</span> <span>TRUE</span>),
  <span>x3 =</span> <span>sample</span>(<span>names</span>(sym_bonuses3), n_row, <span>replace =</span> <span>TRUE</span>),
  <span>y =</span> <span>&quot;NoInfo&quot;</span>,
  <span>stringsAsFactors =</span> <span>FALSE</span>)
d<span>$</span>y[sym_bonuses[d<span>$</span>x2] <span>></span><span> </span>
<span> </span><span>pmax</span>(d<span>$</span>x1, sym_bonuses3[d<span>$</span>x3], <span>runif</span>(n_row))] &lt;-<span> &quot;Large1&quot;</span>
d<span>$</span>y[sym_bonuses3[d<span>$</span>x3] <span>></span><span> </span>
<span> </span><span>pmax</span>(sym_bonuses[d<span>$</span>x2], d<span>$</span>x1, <span>runif</span>(n_row))] &lt;-<span> &quot;Large2&quot;</span>

knitr<span>::</span><span>kable</span>(<span>head</span>(d))</code></pre>
</div>
<table>
<thead>
<tr>
<th align="right">x1</th>
<th align="left">x2</th>
<th align="left">x3</th>
<th align="left">y</th>
</tr>
</thead>
<tbody>
<tr>
<td align="right">0.8178292</td>
<td align="left">b</td>
<td align="left">3</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="right">0.5867139</td>
<td align="left">b</td>
<td align="left">3</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="right">-0.6711920</td>
<td align="left">a</td>
<td align="left">3</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="right">0.1033166</td>
<td align="left">c</td>
<td align="left">2</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="right">-0.3182176</td>
<td align="left">b</td>
<td align="left">1</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="right">-0.5914308</td>
<td align="left">c</td>
<td align="left">2</td>
<td align="left">NoInfo</td>
</tr>
</tbody>
</table>
<p>We define the problem controls and use <code>mkCrossFrameMExperiment()</code> to build both a cross-frame and a treatment plan.</p>
<div>
<pre><code><span># define problem</span>
vars &lt;-<span> </span><span>c</span>(<span>&quot;x1&quot;</span>, <span>&quot;x2&quot;</span>, <span>&quot;x3&quot;</span>)
y_name &lt;-<span> &quot;y&quot;</span>

<span># build the multi-class cross frame and treatments</span>
cfe_m &lt;-<span> </span><span>mkCrossFrameMExperiment</span>(d, vars, y_name)</code></pre>
</div>
<p>The cross-frame is the entity safest for training on (unless you have made separate data split for the treatment design step). It uses cross-validation to reduce nested model bias. Some notes on this issue are available <a rel="nofollow" href="https://winvector.github.io/vtreat/articles/vtreatCrossFrames.html" target="_blank">here</a>, and <a rel="nofollow" href="https://github.com/WinVector/vtreat/blob/master/extras/vtreat.pdf" target="_blank">here</a>.</p>
<div>
<pre><code><span># look at the data we would train models on</span>
<span>str</span>(cfe_m<span>$</span>cross_frame)</code></pre>
</div>
<pre><code>## 'data.frame':    1000 obs. of  16 variables:
##  $ x1_clean      : num  0.818 0.587 -0.671 0.103 -0.318 ...
##  $ x2_catP       : num  0.313 0.313 0.325 0.362 0.313 0.362 0.362 0.325 0.313 0.325 ...
##  $ x3_catP       : num  0.333 0.333 0.333 0.347 0.32 0.347 0.333 0.347 0.333 0.347 ...
##  $ x2_lev_x_a    : num  0 0 1 0 0 0 0 1 0 1 ...
##  $ x2_lev_x_b    : num  1 1 0 0 1 0 0 0 1 0 ...
##  $ x2_lev_x_c    : num  0 0 0 1 0 1 1 0 0 0 ...
##  $ x3_lev_x_1    : num  0 0 0 0 1 0 0 0 0 0 ...
##  $ x3_lev_x_2    : num  0 0 0 1 0 1 0 1 0 1 ...
##  $ x3_lev_x_3    : num  1 1 1 0 0 0 1 0 1 0 ...
##  $ Large1_x2_catB: num  -11.23 -11.2 1.25 -11.41 -11.27 ...
##  $ Large1_x3_catB: num  -11.356 -11.239 -11.239 0.379 0.431 ...
##  $ Large2_x2_catB: num  0.0862 0.1446 -0.0243 -0.1268 0.0862 ...
##  $ Large2_x3_catB: num  4.98 6.09 4.69 -3.11 -13.86 ...
##  $ NoInfo_x2_catB: num  -0.0537 0.1084 -0.2827 0.2859 0.1084 ...
##  $ NoInfo_x3_catB: num  -4.82 -5.24 -4.83 2.13 2.53 ...
##  $ y             : chr  &quot;Large2&quot; &quot;Large2&quot; &quot;Large2&quot; &quot;NoInfo&quot; ...</code></pre>
<p><code>prepare()</code> can apply the designed treatments to new data. Here we are simulating new data by re-using our design data.</p>
<div>
<pre><code><span># pretend original data is new data to be treated</span>
<span># NA out top row to show processing</span>
<span>for</span>(vi <span>in</span> vars) {
  d[[vi]][[<span>1</span>]] &lt;-<span> </span><span>NA</span>
}
<span>str</span>(<span>prepare</span>(cfe_m<span>$</span>treat_m, d))</code></pre>
</div>
<pre><code>## 'data.frame':    1000 obs. of  16 variables:
##  $ x1_clean      : num  0.0205 0.5867 -0.6712 0.1033 -0.3182 ...
##  $ x2_catP       : num  0.0005 0.313 0.325 0.362 0.313 0.362 0.362 0.325 0.313 0.325 ...
##  $ x3_catP       : num  0.0005 0.333 0.333 0.347 0.32 0.347 0.333 0.347 0.333 0.347 ...
##  $ x2_lev_x_a    : num  0 0 1 0 0 0 0 1 0 1 ...
##  $ x2_lev_x_b    : num  0 1 0 0 1 0 0 0 1 0 ...
##  $ x2_lev_x_c    : num  0 0 0 1 0 1 1 0 0 0 ...
##  $ x3_lev_x_1    : num  0 0 0 0 1 0 0 0 0 0 ...
##  $ x3_lev_x_2    : num  0 0 0 1 0 1 0 1 0 1 ...
##  $ x3_lev_x_3    : num  0 1 1 0 0 0 1 0 1 0 ...
##  $ Large1_x2_catB: num  0 -11.6 1.2 -11.8 -11.6 ...
##  $ Large1_x3_catB: num  0 -11.702 -11.702 0.411 0.436 ...
##  $ Large2_x2_catB: num  0 0.133 -0.0215 -0.0999 0.133 ...
##  $ Large2_x3_catB: num  0 5.1 5.1 -3.54 -14.29 ...
##  $ NoInfo_x2_catB: num  0 0.0206 -0.2829 0.2536 0.0206 ...
##  $ NoInfo_x3_catB: num  0 -4.95 -4.95 2.11 2.34 ...
##  $ y             : chr  &quot;Large2&quot; &quot;Large2&quot; &quot;Large2&quot; &quot;NoInfo&quot; ...</code></pre>
<p>We can easily estimate per-outcome variable importance and per-variable variable importance.</p>
<div>
<pre><code>knitr<span>::</span><span>kable</span>(
  cfe_m<span>$</span>score_frame[, 
                    <span>c</span>(<span>&quot;varName&quot;</span>, <span>&quot;rsq&quot;</span>, <span>&quot;sig&quot;</span>, <span>&quot;outcome_level&quot;</span>), 
                    <span>drop =</span> <span>FALSE</span>])</code></pre>
</div>
<table>
<thead>
<tr>
<th align="left">varName</th>
<th align="right">rsq</th>
<th align="right">sig</th>
<th align="left">outcome_level</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">x1_clean</td>
<td align="right">0.0558908</td>
<td align="right">0.0000382</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">x2_catP</td>
<td align="right">0.0275238</td>
<td align="right">0.0038536</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">x2_lev_x_a</td>
<td align="right">0.2680953</td>
<td align="right">0.0000000</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">x2_lev_x_b</td>
<td align="right">0.0885021</td>
<td align="right">0.0000002</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">x2_lev_x_c</td>
<td align="right">0.1060407</td>
<td align="right">0.0000000</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">x3_catP</td>
<td align="right">0.0000346</td>
<td align="right">0.9183445</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">x3_lev_x_1</td>
<td align="right">0.0141504</td>
<td align="right">0.0382554</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">x3_lev_x_2</td>
<td align="right">0.0140364</td>
<td align="right">0.0390420</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">x3_lev_x_3</td>
<td align="right">0.0955004</td>
<td align="right">0.0000001</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">x1_clean</td>
<td align="right">0.0015382</td>
<td align="right">0.1615618</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">x2_catP</td>
<td align="right">0.0013055</td>
<td align="right">0.1971725</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">x2_lev_x_a</td>
<td align="right">0.0000387</td>
<td align="right">0.8242956</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">x2_lev_x_b</td>
<td align="right">0.0014571</td>
<td align="right">0.1730603</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">x2_lev_x_c</td>
<td align="right">0.0009604</td>
<td align="right">0.2686774</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">x3_catP</td>
<td align="right">0.0007725</td>
<td align="right">0.3211959</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">x3_lev_x_1</td>
<td align="right">0.2602002</td>
<td align="right">0.0000000</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">x3_lev_x_2</td>
<td align="right">0.2483708</td>
<td align="right">0.0000000</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">x3_lev_x_3</td>
<td align="right">0.9197595</td>
<td align="right">0.0000000</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">x1_clean</td>
<td align="right">0.0064771</td>
<td align="right">0.0034947</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">x2_catP</td>
<td align="right">0.0040540</td>
<td align="right">0.0208595</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">x2_lev_x_a</td>
<td align="right">0.0071709</td>
<td align="right">0.0021196</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">x2_lev_x_b</td>
<td align="right">0.0000340</td>
<td align="right">0.8323647</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">x2_lev_x_c</td>
<td align="right">0.0060493</td>
<td align="right">0.0047665</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">x3_catP</td>
<td align="right">0.0006576</td>
<td align="right">0.3520950</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">x3_lev_x_1</td>
<td align="right">0.1838759</td>
<td align="right">0.0000000</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">x3_lev_x_2</td>
<td align="right">0.1857824</td>
<td align="right">0.0000000</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">x3_lev_x_3</td>
<td align="right">0.7372570</td>
<td align="right">0.0000000</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">Large1_x2_catB</td>
<td align="right">0.2675964</td>
<td align="right">0.0000000</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">Large1_x3_catB</td>
<td align="right">0.0946910</td>
<td align="right">0.0000001</td>
<td align="left">Large1</td>
</tr>
<tr>
<td align="left">Large2_x2_catB</td>
<td align="right">0.0000291</td>
<td align="right">0.8472707</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">Large2_x3_catB</td>
<td align="right">0.9239860</td>
<td align="right">0.0000000</td>
<td align="left">Large2</td>
</tr>
<tr>
<td align="left">NoInfo_x2_catB</td>
<td align="right">0.0068238</td>
<td align="right">0.0027207</td>
<td align="left">NoInfo</td>
</tr>
<tr>
<td align="left">NoInfo_x3_catB</td>
<td align="right">0.7326682</td>
<td align="right">0.0000000</td>
<td align="left">NoInfo</td>
</tr>
</tbody>
</table>
<p>One can relate these per-target and per-treatment performances back to original columns by aggregating.</p>
<div>
<pre><code><span>tapply</span>(cfe_m<span>$</span>score_frame<span>$</span>rsq, 
       cfe_m<span>$</span>score_frame<span>$</span>origName, 
       max)</code></pre>
</div>
<pre><code>##         x1         x2         x3 
## 0.05589076 0.26809534 0.92398602</code></pre>
<div>
<pre><code><span>tapply</span>(cfe_m<span>$</span>score_frame<span>$</span>sig, 
       cfe_m<span>$</span>score_frame<span>$</span>origName, 
       min)</code></pre>
</div>
<pre><code>##            x1            x2            x3 
##  3.819834e-05  1.892838e-19 5.746904e-258</code></pre>
<p>Obvious issues include: computing variable importance, and blow up and co-dependency of produced columns. These we leave for the next modeling step to deal with (this is our philosophy with most issues that involve joint distributions of variables).</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.win-vector.com/blog/2018/10/modeling-muti-category-outcomes-with-vtreat/"> R – Win-Vector Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/qQJWERs_Fag" height="1"></a></div>