<span><span lang="en"><span>We will develop a </span></span></span><span><span lang="en"><span><span><span lang="en"><span>forecasting example</span></span></span> using model trees and regression trees algorithms. The exercise was originally published in <i>&quot;Machine Learning in R&quot; </i>by Brett Lantz, PACKT publishing 2015 (open source community experience destilled).</span></span></span><br><br><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span>The example we will develop is about predicting wine quality.</span></span></span></span></span></span></span></span><br><br><div><a href="https://3.bp.blogspot.com/-rjPuZ2Bw9HU/WRCD_SF1wfI/AAAAAAAAC8E/fX9flvfgEJMTdFIbLbPX0wUheycA_7HQwCLcB/s1600/Different-Types-of-Wine-v2.jpg"></a></div><div></div><div><span><span>Image from Wine Folly</span></span></div><br><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>We will carry out the exercise verbatim as published in the aforementioned reference</span></span></span></span></span></span><span lang="en"><span>.</span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span>For more details on the </span></span></span></span></span></span></span></span></span></span></span></span></span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>model trees and regression trees</span></span></span> algorithms it is recommended to check the aforementioned reference or </span></span></span></span></span><span><span lang="en"><span><span lang="en"><span><span lang="en"><span>any other bibliography of your choice</span></span>.</span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><br></span></span></span></span></span></span></span></span><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><br>### install an load required packages<br>#install.packages(&quot;rpart&quot;)<br>#install.packages(&quot;rpart.plot&quot;)<br><br>library(rpart)<br>library(rpart.plot)<br>library(RWeka)<br><br>### read and explore the data<br>wine &lt;- read.csv(&quot;whitewines.csv&quot;)<br>str(wine)<br><br>### examine the distribution of the outcome variable<br>hist(wine$quality, main= &quot;Wine Quality&quot;, col= &quot;red&quot;)</span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span></span></span></span></span></span></span></span></span><br><div><a href="https://2.bp.blogspot.com/-5vZ2YzDjAC8/WRCEZxn3mYI/AAAAAAAAC8I/oLrr1av2_P0WhCmVW3cN9Dp2WJMtVFtowCLcB/s1600/wine_1.png"></a></div><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>### examine output for outliers or other potential data problems<br>summary(wine)<br><br>### split the data into training and testing datasets<br>### 75% - 25%<br>wine_train &lt;- wine[1:3750, ]<br>wine_test &lt;- wine[3751:4898, ]<br><br>### training a model on the data<br><br>### training a regression tree model<br>m.rpart &lt;- rpart(quality ~ ., data = wine_train)<br><br>### explore basic information about the tree<br>### for each node in the tree, the number of examples reaching<br>### the decision point is listed<br>m.rpart<br><br>### more detailed summary of tree's fit<br>summary(m.rpart)<br><br>### visualazing the decision tree<br>rpart.plot(m.rpart, digits = 3)</span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span></span></span></span></span></span></span></span></span><br><div><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span></span></span></span></span></span></span></span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span> </span></span></span></span></span></span></span></span><a href="https://2.bp.blogspot.com/-d_Mer08-0Vg/WRCEw6q6G4I/AAAAAAAAC8M/pprvlgryjKQIrHe6E_656vkxFUufeRpqQCLcB/s1600/wine_2.png"></a></div><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>rpart.plot(m.rpart, digits = 4, fallen.leaves = TRUE,<br>           type = 3, extra = 101)</span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span></span></span></span></span></span></span></span></span><br><div><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span> </span></span></span></span></span></span></span></span><a href="https://2.bp.blogspot.com/-43vN_vPu1jQ/WRCE7sM6LnI/AAAAAAAAC8Q/UiLhbbeL47sONWKR4YimC90xmOr4NcCDgCLcB/s1600/wine_3.png"></a></div><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>### evaluating model performance<br>p.rpart &lt;- predict(m.rpart, wine_test)<br><br>### model is not correctly identifying the extreme cases<br>summary(p.rpart)<br>summary(wine_test$quality)<br><br>### correlation between actual and predicted quality<br>cor(p.rpart, wine_test$quality)<br><br>### Measuring performance with the mean absolute error<br><br>MAE &lt;- function(actual, predicted) {<br>  mean(abs(actual - predicted))<br>}<br><br>### the MAE for our predictions<br>MAE(p.rpart, wine_test$quality)<br><br>### the mean quality rating in the training data<br>mean(wine_train$quality)<br><br>### If we predicted the value 5.87 for every wine sample,<br>### we would have a mean absolute error of only about 0.67:<br>MAE(5.87, wine_test$quality)<br><br>### improving model performance?<br><br>### model tree<br>### model tree improves on regression trees by replacing the<br>### leaf nodes with regression models<br>### M5 algorithm (M5-prime)<br>m.m5p &lt;- M5P(quality ~ ., data = wine_train)<br><br>### examine the tree and linear models<br>m.m5p<br>summary(m.m5p)<br><br>### predicted values and performance<br>p.m5p &lt;- predict(m.m5p, wine_test)<br><br>summary(p.m5p)<br><br>cor(p.m5p, wine_test$quality)<br><br>MAE(wine_test$quality, p.m5p)</span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span></span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>###</span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>You can get the example and the dataset in:</span></span></span></span></span></span></span></span><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span><a target="_blank" href="https://github.com/pakinja/Data-R-Value/tree/master/MachineLearning_RegressionTrees_ModelTrees">https://github.com/pakinja/Data-R-Value</a> </span></span></span></span></span></span></span></span><br><br>