<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/combining-automatically-factor-levels-in-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://freakonometrics.hypotheses.org/55451"> R-english – Freakonometrics</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Each time we face real applications in an applied econometrics course, we have to deal with categorial variables. And the same question arise, from students : how can we combine automatically factor levels ? Is there a simple R function ?</p>
<p>I did upload a few blog posts, over the pas years. But so far, nothing satistfying. Let me write down a few lines about what could be done. And if some wants to write a nice R function, that would be awesome. To illustrate the idea, consider the following (simulated dataset)</p>
<div><table><tr><td><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
</pre></td><td><pre>n<span>=</span><span>200</span>
<span>set.<span>seed</span></span><span>(</span><span>1</span><span>)</span>
x1<span>=</span><span>runif</span><span>(</span>n<span>)</span>
x2<span>=</span><span>runif</span><span>(</span>n<span>)</span>
y<span>=</span><span>1</span><span>+</span><span>2</span><span>*</span>x1<span>-</span>x2<span>+</span><span>rnorm</span><span>(</span>n,<span>0</span>,.2<span>)</span>
LB<span>=</span><span>sample</span><span>(</span><span>LETTERS</span><span>[</span><span>1</span><span>:</span><span>10</span><span>]</span><span>)</span>
b<span>=</span><span>data.<span>frame</span></span><span>(</span>y<span>=</span>y,x1<span>=</span>x1,
             x2<span>=</span><span>cut</span><span>(</span>x2,breaks<span>=</span>
             <span>c</span><span>(</span><span>-</span><span>1</span>,.05,.1,.2,.35,.4,.55,.65,.8,.9,<span>2</span><span>)</span>,
             <span>labels</span><span>=</span>LB<span>)</span><span>)</span>
<span>str</span><span>(</span>b<span>)</span>
<span>'data.frame'</span><span>:</span>	<span>200</span> obs. <span>of</span>  <span>3</span> variables<span>:</span>
 $ y <span>:</span> num  <span>1.345</span> <span>1.863</span> <span>1.946</span> <span>2.481</span> <span>0.765</span> ...
 $ x1<span>:</span> num  <span>0.266</span> <span>0.372</span> <span>0.573</span> <span>0.908</span> <span>0.202</span> ...
 $ x2<span>:</span> Factor w<span>/</span> <span>10</span> <span>levels</span> <span>&quot;I&quot;</span>,<span>&quot;A&quot;</span>,<span>&quot;H&quot;</span>,<span>&quot;F&quot;</span>,..<span>:</span> <span>4</span> <span>4</span> <span>6</span> <span>4</span> <span>3</span> <span>6</span> <span>7</span> <span>3</span> <span>4</span> <span>8</span> ...
<span>table</span><span>(</span>b$x2<span>)</span><span>[</span><span>LETTERS</span><span>[</span><span>1</span><span>:</span><span>10</span><span>]</span><span>]</span>
 
 A  B  <span>C</span>  <span>D</span>  E  <span>F</span>  G  H  <span>I</span>  J 
<span>11</span> <span>12</span> <span>23</span> <span>34</span> <span>23</span> <span>36</span> <span>12</span> <span>32</span>  <span>3</span> <span>14</span></pre></td></tr></table></div>
<p>There is one (continuous) dependent variable <img alt="y" title="y" src="https://s0.wp.com/latex.php?latex=y&bg=ffffff&fg=000&s=0" class="latex">, one continuous covariable <img alt="x_1" title="x_1" src="https://s0.wp.com/latex.php?latex=x_1&bg=ffffff&fg=000&s=0" class="latex"> and one categorical variable <img alt="x_2" title="x_2" src="https://s0.wp.com/latex.php?latex=x_2&bg=ffffff&fg=000&s=0" class="latex">, with here ten levels. We can plot the data using</p>
<div><table><tr><td><pre>1
2
</pre></td><td><pre><span>plot</span><span>(</span>b$x1,y,<span>col</span><span>=</span><span>&quot;white&quot;</span>,xlim<span>=</span><span>c</span><span>(</span><span>0</span>,<span>1.1</span><span>)</span><span>)</span>
<span>text</span><span>(</span>b$x1,y,<span>as.<span>character</span></span><span>(</span>b$x2<span>)</span>,cex<span>=</span>.5<span>)</span></pre></td></tr></table></div>
<p><img src="https://i2.wp.com/freakonometrics.hypotheses.org/files/2018/10/Capture-d%E2%80%99e%CC%81cran-2018-10-06-a%CC%80-06.13.56.png?resize=362%2C338&ssl=1" alt="" width="362" data-recalc-dims="1" height="338"></p>
<p>The output of a linear regression yield the following predictions</p>
<div><table><tr><td><pre>1
2
3
4
5
</pre></td><td><pre><span>for</span><span>(</span>i <span>in</span> <span>1</span><span>:</span><span>10</span><span>)</span><span>{</span>
p<span>=</span><span>function</span><span>(</span>x<span>)</span> <span>predict</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span>,newdata<span>=</span><span>data.<span>frame</span></span><span>(</span>x1<span>=</span>x,x2<span>=</span><span>LETTERS</span><span>[</span>i<span>]</span><span>)</span><span>)</span>
u<span>=</span><span>seq</span><span>(</span><span>-</span><span>1</span>,<span>1.065</span>,<span>by</span><span>=</span>.01<span>)</span>
v<span>=</span><span>Vectorize</span><span>(</span>p<span>)</span><span>(</span>u<span>)</span>
<span>lines</span><span>(</span>u,v<span>)</span><span>}</span></pre></td></tr></table></div>
<p><img src="https://i2.wp.com/freakonometrics.hypotheses.org/files/2018/10/Capture-d%E2%80%99e%CC%81cran-2018-10-06-a%CC%80-06.14.18.png?resize=362%2C340&ssl=1" alt="" width="362" data-recalc-dims="1" height="340"></p>
<p>the slope for <img alt="x_1" title="x_1" src="https://s0.wp.com/latex.php?latex=x_1&bg=ffffff&fg=000&s=0" class="latex"> is the same, we simply add a different constant for each level. As we can see, some levels are very very close, so it seems legitimate to combine them into one single category. Here is the output of the linear regression,</p>
<div><table><tr><td><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
</pre></td><td><pre><span>summary</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>
Coefficients<span>:</span>
             Estimate Std. <span>Error</span> <span>t</span> value Pr<span>(</span><span>&amp;</span>gt<span>;|</span>t<span>|</span><span>)</span>    
<span>(</span>Intercept<span>)</span>  <span>0.843802</span>   <span>0.119655</span>   <span>7.052</span> 3.23e<span>-</span>11 <span>***</span>
x1           <span>1.992878</span>   <span>0.053838</span>  <span>37.016</span>  <span>&amp;</span>lt<span>;</span> 2e<span>-</span>16 <span>***</span>
x2A          <span>0.055500</span>   <span>0.131173</span>   <span>0.423</span>   <span>0.6727</span>    
x2H          <span>0.009293</span>   <span>0.121626</span>   <span>0.076</span>   <span>0.9392</span>    
x2F         <span>-</span><span>0.177002</span>   <span>0.121020</span>  <span>-</span><span>1.463</span>   <span>0.1452</span>    
x2B         <span>-</span><span>0.218152</span>   <span>0.130192</span>  <span>-</span><span>1.676</span>   <span>0.0955</span> .  
<span>x2D</span>         <span>-</span><span>0.206970</span>   <span>0.121294</span>  <span>-</span><span>1.706</span>   <span>0.0896</span> .  
<span>x2G</span>         <span>-</span><span>0.407417</span>   <span>0.129999</span>  <span>-</span><span>3.134</span>   <span>0.0020</span> <span>**</span> 
x2C         <span>-</span><span>0.526708</span>   <span>0.123690</span>  <span>-</span><span>4.258</span> 3.24e<span>-</span>05 <span>***</span>
x2J         <span>-</span><span>0.664281</span>   <span>0.128126</span>  <span>-</span><span>5.185</span> 5.54e<span>-</span>07 <span>***</span>
x2E         <span>-</span><span>0.816454</span>   <span>0.123625</span>  <span>-</span><span>6.604</span> 3.94e<span>-</span>10 <span>***</span>
<span>---</span>
Signif. <span>codes</span><span>:</span>  <span>0</span> ‘<span>***</span>’ <span>0.001</span> ‘<span>**</span>’ <span>0.01</span> ‘<span>*</span>’ <span>0.05</span> ‘.’ <span>0.1</span> ‘ ’ <span>1</span>
 
Residual standard error<span>:</span> <span>0.2014</span> on <span>189</span> degrees of freedom
Multiple R<span>-</span>squared<span>:</span>  <span>0.8995</span>,	Adjusted R<span>-</span>squared<span>:</span>  <span>0.8942</span> 
F<span>-</span>statistic<span>:</span> <span>169.1</span> on <span>10</span> and <span>189</span> DF,  p<span>-</span>value<span>:</span> <span>&amp;</span>lt<span>;</span> 2.2e<span>-</span>16
<span>AIC</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>
<span>[</span><span>1</span><span>]</span> <span>-</span><span>60.74443</span>
BIC<span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>
<span>[</span><span>1</span><span>]</span> <span>-</span><span>21.16463</span></pre></td></tr></table></div>
<p>Here the reference category is “I”. And it looks like we could actually combine that category with several others. One strategy here would be to select all categories that seem to be not significantly different, and to run a (multiple) test</p>
<div><table><tr><td><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
</pre></td><td><pre><span>library</span><span>(</span>car<span>)</span>
linearHypothesis<span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span>, <span>c</span><span>(</span><span>&quot;x2A = 0&quot;</span>, <span>&quot;x2H = 0&quot;</span>, <span>&quot;x2F = 0&quot;</span><span>)</span><span>)</span>
 
Hypothesis<span>:</span>
x2A <span>=</span> <span>0</span>
x2H <span>=</span> <span>0</span>
x2F <span>=</span> <span>0</span>
 
Model <span>1</span><span>:</span> restricted model
Model <span>2</span><span>:</span> y ~ x1 <span>+</span> x2
 
  Res.<span>Df</span>    RSS Df Sum of Sq      <span>F</span> Pr<span>(</span><span>&amp;</span>gt<span>;</span>F<span>)</span>    
<span>1</span>    <span>192</span> <span>8.4651</span>                               
<span>2</span>    <span>189</span> <span>7.6654</span>  <span>3</span>   <span>0.79971</span> <span>6.5726</span>  3e<span>-</span>04 <span>***</span>
<span>---</span>
Signif. <span>codes</span><span>:</span>  <span>0</span> ‘<span>***</span>’ <span>0.001</span> ‘<span>**</span>’ <span>0.01</span> ‘<span>*</span>’ <span>0.05</span> ‘.’ <span>0.1</span> ‘ ’ <span>1</span></pre></td></tr></table></div>
<p>It seems that we can combine those four categories together.</p>
<p>Here, we can see what’s going on when we change the reference category (actually, loop on all categories)</p>
<div><table><tr><td><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
</pre></td><td><pre>P<span>=</span><span>matrix</span><span>(</span>NA,<span>nlevels</span><span>(</span>b$x2<span>)</span>,<span>nlevels</span><span>(</span>b$x2<span>)</span><span>)</span>
<span>colnames</span><span>(</span>P<span>)</span><span>=</span><span>rownames</span><span>(</span>P<span>)</span><span>=</span><span>LETTERS</span><span>[</span><span>1</span><span>:</span><span>10</span><span>]</span>
<span>plot</span><span>(</span><span>1</span><span>:</span><span>nlevels</span><span>(</span>b$x2<span>)</span>,<span>1</span><span>:</span><span>nlevels</span><span>(</span>b$x2<span>)</span>,<span>col</span><span>=</span><span>&quot;white&quot;</span>,xlab<span>=</span><span>&quot;&quot;</span>,ylab<span>=</span><span>&quot;&quot;</span>,axes<span>=</span><span>F</span>,xlim<span>=</span><span>c</span><span>(</span><span>0</span>,<span>10.5</span><span>)</span>,
     ylim<span>=</span><span>c</span><span>(</span><span>0</span>,<span>10.5</span><span>)</span><span>)</span>
<span>text</span><span>(</span><span>1</span><span>:</span><span>10</span>,<span>0</span>,<span>LETTERS</span><span>[</span><span>1</span><span>:</span><span>10</span><span>]</span><span>)</span>
<span>text</span><span>(</span><span>0</span>,<span>1</span><span>:</span><span>10</span>,<span>LETTERS</span><span>[</span><span>1</span><span>:</span><span>10</span><span>]</span><span>)</span>
<span>for</span><span>(</span>i <span>in</span> <span>1</span><span>:</span><span>nlevels</span><span>(</span>b$x2<span>)</span><span>)</span><span>{</span>
<span>#levels(b$x2)=LETTERS[1:10]</span>
b$x2<span>=</span><span>relevel</span><span>(</span>b$x2,<span>LETTERS</span><span>[</span>i<span>]</span><span>)</span>
p<span>=</span><span>summary</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>$coefficients<span>[</span><span>-</span><span>(</span><span>1</span><span>:</span><span>2</span><span>)</span>,<span>4</span><span>]</span>
<span>names</span><span>(</span>p<span>)</span><span>=</span><span>substr</span><span>(</span><span>names</span><span>(</span>p<span>)</span>,<span>3</span>,<span>3</span><span>)</span>
P<span>[</span><span>LETTERS</span><span>[</span>i<span>]</span>,<span>names</span><span>(</span>p<span>)</span><span>]</span><span>=</span>p
p<span>=</span>P<span>[</span><span>LETTERS</span><span>[</span>i<span>]</span>,<span>]</span>
idx<span>=</span><span>which</span><span>(</span>p<span>&amp;</span>gt<span>;</span>.05<span>)</span>
<span>points</span><span>(</span><span>(</span><span>(</span><span>1</span><span>:</span><span>10</span><span>)</span><span>)</span><span>[</span>idx<span>]</span>,<span>rep</span><span>(</span>i,<span>length</span><span>(</span>idx<span>)</span><span>)</span>,pch<span>=</span><span>1</span>,cex<span>=</span><span>2</span><span>)</span>
idx<span>=</span><span>which</span><span>(</span>p<span>&amp;</span>gt<span>;</span>.1<span>)</span>
<span>points</span><span>(</span><span>(</span><span>(</span><span>1</span><span>:</span><span>10</span><span>)</span><span>)</span><span>[</span>idx<span>]</span>,<span>rep</span><span>(</span>i,<span>length</span><span>(</span>idx<span>)</span><span>)</span>,pch<span>=</span><span>19</span>,cex<span>=</span><span>2</span><span>)</span><span>}</span></pre></td></tr></table></div>
<p><img src="https://i1.wp.com/freakonometrics.hypotheses.org/files/2018/10/Capture-d%E2%80%99e%CC%81cran-2018-10-06-a%CC%80-06.15.06.png?resize=367%2C347&ssl=1" alt="" width="367" data-recalc-dims="1" height="347"></p>
<p>We are glad to see that it is symmetric : if “H” should be combined with “I”, “I” should also be combined with “H”.</p>
<p>Here black points are related with the 10% <img alt="p" title="p" src="https://s0.wp.com/latex.php?latex=p&bg=ffffff&fg=000&s=0" class="latex">-value, and white points the 5% <img alt="p" title="p" src="https://s0.wp.com/latex.php?latex=p&bg=ffffff&fg=000&s=0" class="latex">-value. This graph is actually hard to read… And actually, this reminds us of  <a rel="nofollow" href="https://visionscarto.net/la-semiologie-graphique-a-50-ans" target="_blank">Bertin (1967)</a>.</p>
<p><img alt="" src="https://i1.wp.com/freakonometrics.hypotheses.org/files/2018/10/Permuted-matrix-display-from-Bertin-1967-This-figure-was-devised-to-illustrate-the.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Here, we can predefine manually some ordering (we will see below how it might be automatised)</p>
<div><table><tr><td><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
</pre></td><td><pre>LETTERSord<span>=</span><span>c</span><span>(</span><span>&quot;I&quot;</span>,<span>&quot;A&quot;</span>,<span>&quot;H&quot;</span>,<span>&quot;F&quot;</span>,<span>&quot;B&quot;</span>,<span>&quot;D&quot;</span>,<span>&quot;G&quot;</span>,<span>&quot;C&quot;</span>,<span>&quot;J&quot;</span>,<span>&quot;E&quot;</span><span>)</span>
P<span>=</span><span>matrix</span><span>(</span>NA,<span>nlevels</span><span>(</span>b$x2<span>)</span>,<span>nlevels</span><span>(</span>b$x2<span>)</span><span>)</span>
<span>colnames</span><span>(</span>P<span>)</span><span>=</span><span>rownames</span><span>(</span>P<span>)</span><span>=</span>LETTERSord
<span>plot</span><span>(</span><span>1</span><span>:</span><span>nlevels</span><span>(</span>b$x2<span>)</span>,<span>1</span><span>:</span><span>nlevels</span><span>(</span>b$x2<span>)</span>,<span>col</span><span>=</span><span>&quot;white&quot;</span>,xlab<span>=</span><span>&quot;&quot;</span>,ylab<span>=</span><span>&quot;&quot;</span>,axes<span>=</span><span>F</span>,xlim<span>=</span><span>c</span><span>(</span><span>0</span>,<span>10.5</span><span>)</span>,
     ylim<span>=</span><span>c</span><span>(</span><span>0</span>,<span>10.5</span><span>)</span><span>)</span>
ct<span>=</span><span>c</span><span>(</span><span>3</span>,<span>3</span>,<span>2</span>,<span>1</span>,<span>1</span><span>)</span>
<span>abline</span><span>(</span>v<span>=</span>.5<span>+</span><span>c</span><span>(</span><span>0</span>,<span>cumsum</span><span>(</span>ct<span>)</span><span>)</span>,lty<span>=</span><span>2</span><span>)</span>
<span>abline</span><span>(</span>h<span>=</span>.5<span>+</span><span>c</span><span>(</span><span>0</span>,<span>cumsum</span><span>(</span>ct<span>)</span><span>)</span>,lty<span>=</span><span>2</span><span>)</span>
<span>text</span><span>(</span><span>1</span><span>:</span><span>10</span>,<span>0</span>,LETTERSord<span>)</span>
<span>text</span><span>(</span><span>0</span>,<span>1</span><span>:</span><span>10</span>,LETTERSord<span>)</span>
<span>for</span><span>(</span>i <span>in</span> <span>1</span><span>:</span><span>nlevels</span><span>(</span>b$x2<span>)</span><span>)</span><span>{</span>
  <span>#levels(b$x2)=LETTERS[1:10]</span>
  b$x2<span>=</span><span>relevel</span><span>(</span>b$x2,LETTERSord<span>[</span>i<span>]</span><span>)</span>
  p<span>=</span><span>summary</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>$coefficients<span>[</span><span>-</span><span>(</span><span>1</span><span>:</span><span>2</span><span>)</span>,<span>4</span><span>]</span>
  <span>names</span><span>(</span>p<span>)</span><span>=</span><span>substr</span><span>(</span><span>names</span><span>(</span>p<span>)</span>,<span>3</span>,<span>3</span><span>)</span>
  P<span>[</span>LETTERSord<span>[</span>i<span>]</span>,<span>names</span><span>(</span>p<span>)</span><span>]</span><span>=</span>p
  p<span>=</span>P<span>[</span>LETTERSord<span>[</span>i<span>]</span>,<span>]</span>
  idx<span>=</span><span>which</span><span>(</span>p<span>&amp;</span>gt<span>;</span>.05<span>)</span>
  <span>points</span><span>(</span><span>(</span><span>(</span><span>1</span><span>:</span><span>10</span><span>)</span><span>)</span><span>[</span>idx<span>]</span>,<span>rep</span><span>(</span>i,<span>length</span><span>(</span>idx<span>)</span><span>)</span>,pch<span>=</span><span>1</span>,cex<span>=</span><span>2</span><span>)</span>
  idx<span>=</span><span>which</span><span>(</span>p<span>&amp;</span>gt<span>;</span>.1<span>)</span>
  <span>points</span><span>(</span><span>(</span><span>(</span><span>1</span><span>:</span><span>10</span><span>)</span><span>)</span><span>[</span>idx<span>]</span>,<span>rep</span><span>(</span>i,<span>length</span><span>(</span>idx<span>)</span><span>)</span>,pch<span>=</span><span>19</span>,cex<span>=</span><span>2</span><span>)</span>
<span>}</span></pre></td></tr></table></div>
<p>Here we get the following</p>
<p><img src="https://i2.wp.com/freakonometrics.hypotheses.org/files/2018/10/Capture-d%E2%80%99e%CC%81cran-2018-10-06-a%CC%80-06.14.57.png?resize=358%2C326&ssl=1" alt="" width="358" data-recalc-dims="1" height="326"></p>
<p>It looks like we have our combined categories…</p>
<p>Actually, it is possible to use another strategy. We start from some level, say “A”. Then, we merge it with all non-significantly different levels. If “B” is not one of them, we use it as the new reference. Etc.</p>
<div><table><tr><td><pre>1
2
3
4
5
6
7
8
9
</pre></td><td><pre><span>for</span><span>(</span>i <span>in</span> <span>1</span><span>:</span><span>nlevels</span><span>(</span>b$x2<span>)</span><span>)</span><span>{</span>
  <span>if</span><span>(</span><span>LETTERS</span><span>[</span>i<span>]</span><span>%</span>in<span>%</span><span>levels</span><span>(</span>b$x2<span>)</span><span>)</span><span>{</span>
  b$x2<span>=</span><span>relevel</span><span>(</span>b$x2,<span>LETTERS</span><span>[</span>i<span>]</span><span>)</span>
  p<span>=</span><span>summary</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>$coefficients<span>[</span><span>-</span><span>(</span><span>1</span><span>:</span><span>2</span><span>)</span>,<span>4</span><span>]</span>
  <span>names</span><span>(</span>p<span>)</span><span>=</span><span>substr</span><span>(</span><span>names</span><span>(</span>p<span>)</span>,<span>3</span>,<span>nchar</span><span>(</span>p<span>)</span><span>)</span>
  idx<span>=</span><span>which</span><span>(</span>p<span>&amp;</span>gt<span>;</span>.05<span>)</span>
  mix<span>=</span><span>c</span><span>(</span><span>LETTERS</span><span>[</span>i<span>]</span>,<span>names</span><span>(</span>p<span>)</span><span>[</span>idx<span>]</span><span>)</span>
  b$x2<span>=</span>recode<span>(</span>b$x2, <span>paste</span><span>(</span><span>&quot;c('&quot;</span>,<span>paste</span><span>(</span>mix,collapse <span>=</span> <span>&quot;','&quot;</span><span>)</span>,<span>&quot;')='&quot;</span>,<span>paste</span><span>(</span>mix,collapse <span>=</span> <span>&quot;+&quot;</span><span>)</span>,<span>&quot;'&quot;</span>,sep<span>=</span><span>&quot;&quot;</span><span>)</span><span>)</span>
<span>}</span><span>}</span></pre></td></tr></table></div>
<p>The final categories are</p>
<div><table><tr><td><pre>1
2
3
4
</pre></td><td><pre><span>table</span><span>(</span>b$x2<span>)</span>
 
A<span>+</span><span>I</span><span>+</span>H B<span>+</span><span>D</span><span>+</span><span>F</span>   <span>C</span><span>+</span>G     E     J 
   <span>46</span>    <span>82</span>    <span>35</span>    <span>23</span>    <span>14</span></pre></td></tr></table></div>
<p>with the following regression output</p>
<div><table><tr><td><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
</pre></td><td><pre><span>summary</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>
 
Coefficients<span>:</span>
            Estimate Std. <span>Error</span> <span>t</span> value Pr<span>(</span><span>&amp;</span>gt<span>;|</span>t<span>|</span><span>)</span>    
<span>(</span>Intercept<span>)</span>  <span>0.86407</span>    <span>0.03950</span>  <span>21.877</span>  <span>&amp;</span>lt<span>;</span> 2e<span>-</span>16 <span>***</span>
x1           <span>1.99180</span>    <span>0.05323</span>  <span>37.417</span>  <span>&amp;</span>lt<span>;</span> 2e<span>-</span>16 <span>***</span>
x2B<span>+</span><span>D</span><span>+</span><span>F</span>     <span>-</span><span>0.21517</span>    <span>0.03699</span>  <span>-</span><span>5.817</span> 2.44e<span>-</span>08 <span>***</span>
x2C<span>+</span>G       <span>-</span><span>0.50545</span>    <span>0.04528</span> <span>-</span><span>11.164</span>  <span>&amp;</span>lt<span>;</span> 2e<span>-</span>16 <span>***</span>
x2E         <span>-</span><span>0.83617</span>    <span>0.05128</span> <span>-</span><span>16.305</span>  <span>&amp;</span>lt<span>;</span> 2e<span>-</span>16 <span>***</span>
x2J         <span>-</span><span>0.68398</span>    <span>0.06131</span> <span>-</span><span>11.156</span>  <span>&amp;</span>lt<span>;</span> 2e<span>-</span>16 <span>***</span>
<span>---</span>
Signif. <span>codes</span><span>:</span>  <span>0</span> ‘<span>***</span>’ <span>0.001</span> ‘<span>**</span>’ <span>0.01</span> ‘<span>*</span>’ <span>0.05</span> ‘.’ <span>0.1</span> ‘ ’ <span>1</span>
 
Residual standard error<span>:</span> <span>0.2008</span> on <span>194</span> degrees of freedom
Multiple R<span>-</span>squared<span>:</span>  <span>0.8975</span>,	Adjusted R<span>-</span>squared<span>:</span>  <span>0.8948</span> 
F<span>-</span>statistic<span>:</span> <span>339.6</span> on <span>5</span> and <span>194</span> DF,  p<span>-</span>value<span>:</span> <span>&amp;</span>lt<span>;</span> 2.2e<span>-</span>16
<span>AIC</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>
<span>[</span><span>1</span><span>]</span> <span>-</span><span>66.76939</span>
BIC<span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>
<span>[</span><span>1</span><span>]</span> <span>-</span><span>43.68117</span></pre></td></tr></table></div>
<p>Which is consistent with the group we got before. But actually, if we change the order, we can get different combinations. For instance, if we go from “J” to “A”, instead of “A” to “J”, we obtain</p>
<div><table><tr><td><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td><pre><span>for</span><span>(</span>i <span>in</span> <span>nlevels</span><span>(</span>b$x2<span>)</span><span>:</span><span>1</span><span>)</span><span>{</span>
  <span>#levels(b$x2)=LETTERS[1:10]</span>
  <span>if</span><span>(</span><span>LETTERS</span><span>[</span>i<span>]</span><span>%</span>in<span>%</span><span>levels</span><span>(</span>b$x2<span>)</span><span>)</span><span>{</span>
  b$x2<span>=</span><span>relevel</span><span>(</span>b$x2,<span>LETTERS</span><span>[</span>i<span>]</span><span>)</span>
  p<span>=</span><span>summary</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>$coefficients<span>[</span><span>-</span><span>(</span><span>1</span><span>:</span><span>2</span><span>)</span>,<span>4</span><span>]</span>
  <span>names</span><span>(</span>p<span>)</span><span>=</span><span>substr</span><span>(</span><span>names</span><span>(</span>p<span>)</span>,<span>3</span>,<span>nchar</span><span>(</span>p<span>)</span><span>)</span>
  idx<span>=</span><span>which</span><span>(</span>p<span>&amp;</span>gt<span>;</span>.05<span>)</span>
  mix<span>=</span><span>c</span><span>(</span><span>LETTERS</span><span>[</span>i<span>]</span>,<span>names</span><span>(</span>p<span>)</span><span>[</span>idx<span>]</span><span>)</span>
  b$x2<span>=</span>recode<span>(</span>b$x2, <span>paste</span><span>(</span><span>&quot;c('&quot;</span>,<span>paste</span><span>(</span>mix,collapse <span>=</span> <span>&quot;','&quot;</span><span>)</span>,<span>&quot;')='&quot;</span>,<span>paste</span><span>(</span>mix,collapse <span>=</span> <span>&quot;+&quot;</span><span>)</span>,<span>&quot;'&quot;</span>,sep<span>=</span><span>&quot;&quot;</span><span>)</span><span>)</span>
<span>}</span><span>}</span>
<span>table</span><span>(</span>b$x2<span>)</span>
 
          E         G<span>+</span><span>C</span> <span>I</span><span>+</span>A<span>+</span>B<span>+</span><span>D</span><span>+</span><span>F</span><span>+</span>H           J 
         <span>23</span>          <span>35</span>         <span>128</span>          <span>14</span></pre></td></tr></table></div>
<p>with different information criteria here</p>
<div><table><tr><td><pre>1
2
3
4
</pre></td><td><pre><span>AIC</span><span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>
<span>[</span><span>1</span><span>]</span> <span>-</span><span>36.61665</span>
BIC<span>(</span><span>lm</span><span>(</span>y~x1<span>+</span>x2,<span>data</span><span>=</span>b<span>)</span><span>)</span>
<span>[</span><span>1</span><span>]</span> <span>-</span><span>16.82675</span></pre></td></tr></table></div>
<p>I guess it would be necessary to run randomly the order we go through the levels. Last, but not least, one can use regression trees (even if it not <em>per se</em> in the syllabus of the course). The problem is that there is another explanatory variable that might interphere. So I would suggest (1) to fit a linear model <img alt="y=\beta_0+\beta_1x_1+u_i" title="y=\beta_0+\beta_1x_1+u_i" src="https://s0.wp.com/latex.php?latex=y%3D%5Cbeta_0%2B%5Cbeta_1x_1%2Bu_i&bg=ffffff&fg=000&s=0" class="latex">, to calculate the residuals, <img alt="\widehat{u}_i" title="\widehat{u}_i" src="https://s0.wp.com/latex.php?latex=%5Cwidehat%7Bu%7D_i&bg=ffffff&fg=000&s=0" class="latex"> (2) to run a regression tree, to explain <img alt="\widehat{u}_i" title="\widehat{u}_i" src="https://s0.wp.com/latex.php?latex=%5Cwidehat%7Bu%7D_i&bg=ffffff&fg=000&s=0" class="latex"> with categorical variable <img alt="x_2" title="x_2" src="https://s0.wp.com/latex.php?latex=x_2&bg=ffffff&fg=000&s=0" class="latex"> (I did explain how trees are build when the explanatory variable is a categorical one in a <a rel="nofollow" href="https://freakonometrics.hypotheses.org/20736" target="_blank">previous post</a>)</p>
<div><table><tr><td><pre>1
2
3
4
5
</pre></td><td><pre><span>library</span><span>(</span>rpart<span>)</span>
<span>library</span><span>(</span>rpart.<span>plot</span><span>)</span>
b$e<span>=</span><span>residuals</span><span>(</span><span>lm</span><span>(</span>y~x1,<span>data</span><span>=</span>b<span>)</span><span>)</span>
arbre<span>=</span>rpart<span>(</span>e~x2,<span>data</span><span>=</span>b<span>)</span>
prp<span>(</span>arbre,type<span>=</span><span>2</span>,extra<span>=</span><span>1</span><span>)</span></pre></td></tr></table></div>
<p><img alt="" src="https://i0.wp.com/freakonometrics.hypotheses.org/files/2018/10/Capture-d%E2%80%99e%CC%81cran-2018-10-06-a%CC%80-06.14.32.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Observe that the leaves have the same groups as the one we got.</p>
<div><table><tr><td><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
</pre></td><td><pre>arbre
n<span>=</span> <span>200</span> 
 
node<span>)</span>, <span>split</span>, n, <span>deviance</span>, yval
      <span>*</span> denotes terminal node
 
<span>1</span><span>)</span> root <span>200</span> <span>22.563500</span>  7.771561e<span>-</span>18  
  <span>2</span><span>)</span> x2<span>=</span>G,<span>C</span>,J,E <span>72</span>  <span>4.441495</span> <span>-</span>3.232525e<span>-</span>01  
    <span>4</span><span>)</span> x2<span>=</span>J,E <span>37</span>  <span>1.553520</span> <span>-</span>4.578492e<span>-</span>01 <span>*</span>
    <span>5</span><span>)</span> x2<span>=</span>G,<span>C</span> <span>35</span>  <span>1.509068</span> <span>-</span>1.809646e<span>-</span>01 <span>*</span>
  <span>3</span><span>)</span> x2<span>=</span><span>I</span>,A,H,<span>F</span>,B,<span>D</span> <span>128</span>  <span>6.366628</span>  1.818295e<span>-</span>01  
    <span>6</span><span>)</span> x2<span>=</span><span>F</span>,B,<span>D</span> <span>82</span>  <span>2.983381</span>  1.048246e<span>-</span>01 <span>*</span>
    <span>7</span><span>)</span> x2<span>=</span><span>I</span>,A,H <span>46</span>  <span>2.030229</span>  3.190993e<span>-</span>01 <span>*</span></pre></td></tr></table></div>
<p>I guess that it should be possible to put all that in an R function, to suggest combinations of level that might improve the regression.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://freakonometrics.hypotheses.org/55451"> R-english – Freakonometrics</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/UNKgEej27vE" height="1"></a></div>