<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/rxneuralnet-vs-xgboost-vs-h2o/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://tomaztsql.wordpress.com/2017/02/20/rxneuralnet-vs-xgboost-vs-h2o/"> R – TomazTsql</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Recently, I did a session at local user group in <a target="_blank" rel="nofollow" href="https://www.visitljubljana.com/en/visitors/">Ljubljana, Slovenija</a>, where I introduced the new algorithms that are available with <a target="_blank" rel="nofollow" href="https://msdn.microsoft.com/en-us/library/mt790482.aspx">MicrosoftML</a> package for <a target="_blank" rel="nofollow" href="https://msdn.microsoft.com/en-us/microsoft-r/microsoft-r-getting-started">Microsoft R Server</a> 9.0.3.</p>
<p>For dataset, I have used two from (still currently) running sessions from <a target="_blank" rel="nofollow" href="https://www.kaggle.com/">Kaggle</a>. In the last part, I did image detection and prediction of MNIST dataset and compared the performance and accuracy between.</p>
<p>MNIST Handwritten digit database is available <a target="_blank" rel="nofollow" href="http://yann.lecun.com/exdb/mnist/">here</a>.</p>
<p><img sizes="(max-width: 212px) 100vw, 212px" data-image-description="" data-orig-file="https://i1.wp.com/tomaztsql.files.wordpress.com/2017/02/picture2.png?w=456&amp;ssl=1" data-orig-size="212,100" data-image-title="picture2" class="wp-image-3721" data-comments-opened="1" data-recalc-dims="1" data-attachment-id="3721" data-permalink="https://tomaztsql.wordpress.com/2017/02/20/rxneuralnet-vs-xgboost-vs-h2o/picture2/#main" alt="picture2" data-medium-file="https://i1.wp.com/tomaztsql.files.wordpress.com/2017/02/picture2.png?w=456&amp;ssl=1?w=212" data-large-file="https://i1.wp.com/tomaztsql.files.wordpress.com/2017/02/picture2.png?w=456&amp;ssl=1?w=212" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i1.wp.com/tomaztsql.files.wordpress.com/2017/02/picture2.png?w=456&ssl=1"></p>
<p>Starting off with rxNeuralNet, we have to build a NET# model or Neural network to work it’s way.</p>
<p>Model for Neural network:</p>
<pre>const { T = true; F = false; }

input Picture [28, 28];

hidden C1 [5 * 13^2]
from Picture convolve {
InputShape  = [28, 28];
UpperPad    = [ 1,  1];
KernelShape = [ 5,  5];
Stride      = [ 2,  2];
MapCount = 5;
}

hidden C2 [50, 5, 5]
from C1 convolve {
InputShape  = [ 5, 13, 13];
KernelShape = [ 1,  5,  5];
Stride      = [ 1,  2,  2];
Sharing     = [ F,  T,  T];
MapCount = 10;
}

hidden H3 [100]
from C2 all;

// Output layer definition.
output Result [10]
from H3 all;</pre>
<p>Once we have this, we can work out with rxNeuralNet algorithm:</p>
<pre>model_DNN_GPU &lt;- rxNeuralNet(label ~.
      ,data = dataTrain
      ,type = &quot;multi&quot;
      ,numIterations = 10
      ,normalize = &quot;no&quot;
      #,acceleration = &quot;gpu&quot; #enable this if you have CUDA driver
      ,miniBatchSize = 64 #set to 1 else set to 64 if you have CUDA driver problem 
      ,netDefinition = netDefinition
      ,optimizer = sgd(learningRate = 0.1, lRateRedRatio = 0.9, lRateRedFreq = 10)
)</pre>
<p>Then do the prediction and calculate accuracy matrix:</p>
<pre>DNN_GPU_score &lt;- rxPredict(model_DNN_GPU, dataTest, extraVarsToWrite = &quot;label&quot;)
sum(Score_DNN$Label == DNN_GPU_score$PredictedLabel)/dim(DNN_GPU_score)[1]</pre>
<p>Accuracy for this model is:</p>
<pre>[1] 0.9789</pre>
<p> </p>
<p>When working with H2O package, the following code was executed to get same paramethers for Neural network:</p>
<pre>model_h20 &lt;- h2o.deeplearning(x = 2:785
                     ,y = 1   # label for label
                     ,training_frame = train_h2o
                     ,activation = &quot;RectifierWithDropout&quot;
                     ,input_dropout_ratio = 0.2 # % of inputs dropout
                     ,hidden_dropout_ratios = c(0.5,0.5) # % for nodes dropout
                     ,balance_classes = TRUE 
                     ,hidden = c(50,100,100) 
                     ,momentum_stable = 0.99
                     ,nesterov_accelerated_gradient = T # use it for speed
                     ,epochs = 15)</pre>
<p>When results of test dataset against the learned model is executed:</p>
<pre>h2o.confusionMatrix(model_h20)
100-(416/9978)*100</pre>
<p>the  result is confusion matrix for accuracy of predicted values with value of:</p>
<pre># [1] 95.83083</pre>
<p> </p>
<p>For comparison, I have added xgBoost (eXtrem Gradient Boosting), but this time, I will not focus on this one.</p>
<p>Time comparison against the packages (in seconds), from left to right are: H20, MicrosoftML with GPU acceleration, MicrosoftML without GPU acceleration and xgBoost.</p>
<p><img sizes="(max-width: 678px) 100vw, 678px" data-image-description="" data-orig-file="https://tomaztsql.files.wordpress.com/2017/02/picture1.png" data-orig-size="1114,865" data-image-title="picture1" data-comments-opened="1" class="wp-image-3723" data-recalc-dims="1" data-attachment-id="3723" data-permalink="https://tomaztsql.wordpress.com/2017/02/20/rxneuralnet-vs-xgboost-vs-h2o/picture1/#main" alt="picture1" data-medium-file="https://tomaztsql.files.wordpress.com/2017/02/picture1.png?w=300" data-large-file="https://tomaztsql.files.wordpress.com/2017/02/picture1.png?w=700" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i0.wp.com/tomaztsql.files.wordpress.com/2017/02/picture1.png?resize=456%2C354&ssl=1"></p>
<p>As for the accuracy of the trained model, here are results (based on my tests):</p>
<p>MicrosoftML – Neural Network – 97,8%</p>
<p>H20 – Deep Learning – 95,3 %</p>
<p>xgBoost – 94,9 %</p>
<p> </p>
<p>As always, code and dataset are available at <a target="_blank" rel="nofollow" href="https://github.com/tomaztk/SLODUG2017">GitHub</a>.</p>
<p>Happy R-ing <img alt="🙂" class="wp-smiley" src="https://i0.wp.com/s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f642.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p> </p>
<p> </p>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/tomaztsql.wordpress.com/3705/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/tomaztsql.wordpress.com/3705/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://tomaztsql.wordpress.com/2017/02/20/rxneuralnet-vs-xgboost-vs-h2o/"> R – TomazTsql</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/mJrJ-FStm0s"></a></div>