<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/finally-you-can-plot-h2o-decision-trees-in-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://novyden.blogspot.com/2018/12/finally-you-can-plot-h2o-decision-trees.html"> novyden</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><span><span>Creating and plotting decision trees (</span></span><span><span><span><span>like one below)</span></span> for the models created in H2O will be main objective of this post:</span></span></p>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i2.wp.com/4.bp.blogspot.com/-kbVVCS7q2gY/XBVO21lZJII/AAAAAAABKDQ/Qk3TXQW-mJwQof3tBO-0ABaBOVy5OHP4wCLcBGAs/s1600/Titanic-Decision-tree-h2o.png?ssl=1" target="_blank"><img border="0" data-original-height="414" src="https://i2.wp.com/4.bp.blogspot.com/-kbVVCS7q2gY/XBVO21lZJII/AAAAAAABKDQ/Qk3TXQW-mJwQof3tBO-0ABaBOVy5OHP4wCLcBGAs/s1600/Titanic-Decision-tree-h2o.png?resize=450%2C414&ssl=1" data-recalc-dims="1" data-original-width="450"></a></td>
</tr>
<tr>
<td><span><span>Figure 1. Decision Tree Visualization in R</span></span></td>
</tr>
</tbody>
</table>
<p></p>
<h3><span><span><span><span><span><span><span><span>Decision Trees with H<sub>2</sub>O</span></span></span></span></span></span></span></span></h3>
<p><span><span>With release <a rel="nofollow" href="http://docs.h2o.ai/h2o/latest-stable/h2o-docs/downloading.html" target="_blank">3.22.0.1</a> H<sub>2</sub>O-3 (a.k.a. open source </span></span><span><span><span><span>H<sub>2</sub>O or simply </span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>) added to its family of tree-based algorithms (which already included <a rel="nofollow" href="http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/drf.html" target="_blank">DRF</a>, <a rel="nofollow" href="http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/gbm.html" target="_blank">GBM</a>, and <a rel="nofollow" href="http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/xgboost.html" target="_blank">XGBoost</a>) support for one more: <a rel="nofollow" href="https://www.h2o.ai/blog/anomaly-detection-with-isolation-forests-using-h2o/" target="_blank">Isolation Forest (random forest for unsupervised anomaly detection)</a>. There were no simple way to visualize model trees until recently except following clunky (albeit reliable) method of <a rel="nofollow" href="https://dzone.com/articles/visualizing-h2o-gbm-and-random-forest-mojo-models" target="_blank">creating a MOJO object and running combination of Java and dot commands</a>. </span></span><br><span><span><br></span></span><span><span>That changed in</span></span><span><span> 3.22.0.1 too with introduction of unified Tree API to work with any of the tree-based algorithms above. Data scientists are now able to utilize powerful visualization tools in R (or Python) without resorting to </span></span><span><span><span><span>producing intermediate </span></span></span></span><span><span><span><span><span><span><span><span>artifacts</span></span></span></span> like MOJO and running external utilities. Please read <a rel="nofollow" href="https://dzone.com/articles/inspecting-decision-trees-in-h2o" target="_blank">this article by Pavel Pscheidl</a> who did superb job of explaining </span></span></span></span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> Tree API and S4 classes in R before coming back to take it a step further to visualize trees.</span></span></span></span></p>
<h3><span><span><span><span>The Workflow: from Data to Decision Tree </span></span></span></span></h3>
<p><span><span><span><span>Whether you are still here or came back after reading Pavel’s excellent post let’s set goal straight: create single decision tree model in </span></span></span></span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> and visualize its tree graph. With </span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> there is always a choice between using Python or R – </span></span></span></span>the choice for R here will become clear when discussing its graphical and analytical capabilities later.</span></span></span></span></p>
<p><span><span><span><span><a rel="nofollow" href="https://en.wikipedia.org/wiki/Predictive_analytics#Classification_and_regression_trees_(CART)" target="_blank">CART models</a> operate on labeled data (classification and regression) and offer arguably unmatched model interpretability by means of analyzing a tree graph. In data science there is never single way to solve given problem so let’s define end-to-end logical workflow from “raw” data to visualized decision tree</span></span></span></span><span><span><span><span>:</span></span></span></span></p>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i1.wp.com/2.bp.blogspot.com/-0t4_oMSFuxQ/XB6RqAYqwYI/AAAAAAABKG8/01_iBrzS4EkNTw8-a1f7StTnh13uQVYOACLcBGAs/s1600/h2o-tree-visual-flow-abstract.png?ssl=1" target="_blank"><img border="0" data-original-height="168" src="https://i1.wp.com/2.bp.blogspot.com/-0t4_oMSFuxQ/XB6RqAYqwYI/AAAAAAABKG8/01_iBrzS4EkNTw8-a1f7StTnh13uQVYOACLcBGAs/s1600/h2o-tree-visual-flow-abstract.png?resize=450%2C168&ssl=1" data-recalc-dims="1" data-original-width="450"></a></td>
</tr>
<tr>
<td><span><span>Figure 2. Workflow of tasks in this post</span></span></td>
</tr>
</tbody>
</table>
<p></p>
<div></div>
<p><span><span><span><span>One may argue that allocating tasks between </span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> and R</span></span></span></span></span></span></span></span> could be different but let’s follow outlined plan for this post. Next diagram adds implementation specifics to each step:</span></span></span></span></p>
<ul>
<li><span><span><span><span>R package <span>data.table</span> for data munging</span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> grid for hyper-parameter search</span></span></span></span></span></span></span></span></span></span></span></span> </span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> GBM for modeling single decision tree algorithm</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> Tree API for tree model representation</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>R package <span>data.tree</span> for visualization  </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> </span></span></span></span></li>
</ul>
<div></div>
<div></div>
<table cellspacing="0" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i0.wp.com/4.bp.blogspot.com/-EpP0S3m6Tdc/XB6SXziwrwI/AAAAAAABKHE/xdPQqI4IJlYMzaPS2GFkVkXyW4-dK3AIwCLcBGAs/s1600/h2o-tree-visual-flow.png?ssl=1" target="_blank"><img border="0" data-original-height="166" src="https://i0.wp.com/4.bp.blogspot.com/-EpP0S3m6Tdc/XB6SXziwrwI/AAAAAAABKHE/xdPQqI4IJlYMzaPS2GFkVkXyW4-dK3AIwCLcBGAs/s1600/h2o-tree-visual-flow.png?resize=450%2C166&ssl=1" data-recalc-dims="1" data-original-width="450"></a></td>
</tr>
<tr>
<td><span><span>Figure 3. Workflow of tasks in this post with implementation details</span></span></td>
</tr>
</tbody>
</table>
<p><span><span><span><span><br></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>Discussion of this workflow continues for the rest of this post.</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></p>
<h3><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>Titanic Dataset</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></h3>
<p><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>The famous <a rel="nofollow" href="https://www.kaggle.com/c/titanic" target="_blank">Titanic dataset</a> contains information about the fate of passengers of the <a rel="nofollow" href="https://en.wikipedia.org/wiki/RMS_Titanic" target="_blank">RMS Titanic</a> that sank after colliding with an iceberg. It regularly serves as toy data for blog exercises like this.</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><br></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> public S3 bucket holds the Titanic dataset readly available and using package </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>data.table</span></span></span></span></span> makes it fast one-liner </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>to load into R</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>:</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span><span><span><br></span></span></span></span><br><span><span><span><span></span></span></span></span> </p>
<div><code data-gist-hide-footer="true" data-gist-id="149fb361fdfe933cd6317601e8a7107c" data-gist-file="load-titanic-data-from-S3.R"></code></div>
<p></p>
<h3><span><span><span><span>Data Engineering</span></span></span></span></h3>
<p><span><span><span><span>Passenger features from the Titanic dataset are discussed at length online, e.g. see <a rel="nofollow" href="https://towardsdatascience.com/predicting-the-survival-of-titanic-passengers-30870ccc7e8" target="_blank">Predicting the Survival of Titanic Passengers</a> and <a rel="nofollow" href="https://www.kaggle.com/thilakshasilva/predicting-titanic-survival-using-five-algorithms" target="_blank">Predicting Titanic Survival using Five Algorithms</a>. To summarize the following features were selected and engineered for decision tree model:</span></span></span></span></p>
<ul>
<li><span><span><span><span><i>survived</i> indicates if passenger survived the wreck</span></span></span></span></li>
<li><span><span><span><span><i>boat</i> and <i>body</i> leak survival outcome and were dropped completely before modeling</span></span></span></span></li>
<li><span><span><span><span><i>name</i> and <i>cabin</i> are too noisy as they are and only used to derive new features</span></span></span></span></li>
<li><span><span><span><span><i>title</i> is parsed from <i>name</i></span></span></span></span></li>
<li><span><span><span><span><i>cabin_type</i> is parsed from <i>cabin</i></span></span></span></span></li>
<li><span><span><span><span><i>family_size</i> and <i>family_type</i> are derived from combination of count features <i>sibsp</i> (siblings+spouse) and <i>parch</i> (parents+children)</span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><i>ticket</i> and <i>home.dest</i> are dropped to preserve simplicity of the model</span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span>missing values in <i>age</i> and <i>fare</i> are imputed using target encoding (mean) over grouping by <i>survived</i>, <i>sex</i>, and <i>embarked</i> columns. </span></span></span></span></span></span></span></span></li>
</ul>
<p><span><span><span><span><span><span><span><span>Data load and data munging steps above are implemented in R using <i>data.table</i>:</span></span></span></span></span></span></span></span><br><span><span><span><span><span><span><span><span><br></span></span></span></span></span></span></span></span></p>
<div><code data-gist-hide-footer="true" data-gist-id="149fb361fdfe933cd6317601e8a7107c" data-gist-file="decision-tree-visual-with-H2O.R"></code></div>
<p></p>
<h3><span><span><span><span><span><span><span><span>Starting with H<sub>2</sub>O</span></span></span></span></span></span></span></span></h3>
<p><span><span><span><span><span><span><span><span>Creating models with </span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> requires running a server process (remote or local) and a client (package <span>h2o</span> in R available from CRAN) where the latter connects and sends commands to the former. The Tree API was introduced with release </span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>3.22.0.1</span></span></span></span></span></span></span></span> (10/26/2018) but due to CRAN policies <span>h2o</span> package usually lags several versions behind (on the time of this writing CRAN hosted version 3.20.0.8). There are two ways to work around this:</span></span></span></span></span></span></span></span></p>
<ol>
<li><span><span><span><span><span><span><span><span>Install and run package available from CRAN and use <span>strict_version_check=FALSE</span> inside <span>h2o.connect()</span> to communicate with newer version running on server</span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span>Or install the latest version of <span>h2o</span> available from </span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span></span></span></span></span></span></span></span></span> repository either to connect to remote server or to both connect and run server locally.</span></span></span></span></span></span></span></span></li>
</ol>
<p><span><span><span><span><span><span><span><span>Tree API is available only with 2d option because it requires access to new classes and functions in </span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>h2o</span></span></span></span></span></span></span></span></span> package (remember, I asked you read Pavel’s blog). Below code from the official </span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span></span></span></span></span></span></span></span></span> download page shows how to download and install the latest version of the</span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span> package: </span></span></span></span></span></span></span></span></p>
<div><code data-gist-hide-footer="true" data-gist-id="149fb361fdfe933cd6317601e8a7107c" data-gist-file="install-latest-h2o-package.R"></code></div>
<p></p>
<h3><span><span><span><span><span><span><span><span>Building Decision Tree with H<sub>2</sub>O</span></span></span></span></span></span></span></span></h3>
<p><span><span><span><span><span><span><span><span>While </span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H2O offers </span></span></span></span></span></span></span></span>no dedicated single decision tree algorithm <a rel="nofollow" href="https://0xdata.atlassian.net/browse/PUBDEV-4324" target="_blank">there two approaches using superseding models</a>:</span></span></span></span></span></span></span></span></p>
<ul>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/drf.html" target="_blank">Distributed Random Forest (DRF)</a> function </span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>h2o.randomForest()</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> with arguments</span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><br>ntrees = 1</span></span></span></span></span><span><span><span><span><span><span><br>mtries = number of features <span>(would be determined dynamically at runtime)</span></span></span></span></span></span></span><span><span><span><span><span><span><br>sample_rate = 1<br>min_rows = 1</span></span></span></span></span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/gbm.html" target="_blank">Gradient Boosting Machine (GBM)</a> function </span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>h2o.gbm()</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> with arguments</span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><br>ntrees = 1</span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><br>min_rows = 1</span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><br>sample_rate = 1<br>col_sample_rate = 1</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> </span></span></span></span></span></span></span></span></li>
</ul>
<p><span><span>Choosing GBM option requires one less line of code (no need to calculate number of features to set </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>mtries</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>) so it was used for this post. Otherwise both ways result in the same decision tree with the steps below fully reproducible using </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>h2o.randomForest()</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> instead of </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>h2o.gbm()</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>.</span></span></p>
<h3><span><span>Decision Tree Depth</span></span></h3>
<p><span><span><span>When building single decision tree models maximum tree depth stands as t</span></span><span><span><span><span>he most important parameter to pick</span></span></span>. Shallow trees tend to underfit by failing to capture important relationships in data producing similar trees despite varying training data (error due to high bias). On the other hand trees grown too deep overfit by reacting to noise and slight changes in data (error due to high variance). </span><span>Tuning </span><span><span><span><span><span><span>H<sub>2</sub>O </span></span></span></span></span>model’s parameter </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>max_depth</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span> that limits decision tree depth aims at balancing the effects of bias and variance</span></span><span><span>. In R using </span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span></span></span> to split data and to tune the model, then visualizing results with </span></span><span><span><span><span><span><span><span><span><span><span><span>ggplot</span></span></span></span></span></span></span></span></span> to look for right value unfolds like this</span><span>:</span><span>    </span></span></p>
<ol>
<li><span><span>split Titanic data into training and validation sets</span></span></li>
<li><span><span>define grid search object with parameter </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>max_depth</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> </span></li>
<li><span><span>launch grid search on GBM models and grid object to obtain AUC values (model performance) </span></span></li>
<li><span><span>plot grid model AUC’es vs. </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>max_depth</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span> values to <span>determine “inflection point” where AUC growth stops or saturates (see plot below)</span></span></span></li>
<li><span><span><span>register tree depth value at <a rel="nofollow" href="https://en.wikipedia.org/wiki/Inflection_point" target="_blank">inflection point</a> to use in the final model</span></span></span></li>
</ol>
<p><span><span>Code below implements these steps:</span></span></p>
<div><code data-gist-hide-footer="true" data-gist-id="149fb361fdfe933cd6317601e8a7107c" data-gist-file="grid-search-and-1tree-model.R"></code></div>
<p><span><span><br></span></span><span><span>and produces chart that points to inflection point for maximum tree depth at 5:</span></span></p>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i0.wp.com/1.bp.blogspot.com/-y_02-0_dKEQ/XAgMU5LY-LI/AAAAAAABJ44/iOonYjqyXfMCQeBuD85dGftRNScS8PXAACEwYBhgL/s1600/decision-tree-grid-max-depth.png?ssl=1" target="_blank"><img border="0" data-original-height="555" src="https://i0.wp.com/1.bp.blogspot.com/-y_02-0_dKEQ/XAgMU5LY-LI/AAAAAAABJ44/iOonYjqyXfMCQeBuD85dGftRNScS8PXAACEwYBhgL/s1600/decision-tree-grid-max-depth.png?resize=450%2C555&ssl=1" data-recalc-dims="1" data-original-width="450"></a></td>
</tr>
<tr>
<td><span>Figure 4. Visualization of AUC vs. maximum tree depth hyper-parameter trend </span><br><span>extracted from the H2O grid object after running grid search in H2O. </span><br><span>Marked inflection point indicates when increasing maximum tree depth </span><br><span>no longer improves model performance on validation set</span></td>
</tr>
</tbody>
</table>
<h3><span><span> </span></span></h3>
<h3><span><span>Creating Decision Tree</span></span></h3>
<p><span><span>As evident from the Figure 3 optimal decision tree depth is 5. The code below constructs single decision tree model in </span></span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span></span></span></span> and then retrieves tree representation from a GBM model with Tree API function </span></span><span><span><span><span><span><span><span><code>h2o.getModelTree()</code></span></span></span></span></span>, which creates an instance of S4 class </span></span><span><span><span><span><span><code>H2OTree</code></span></span></span> and assigns to variable </span></span><span><span><span><span><span><span><span><code>titanicH2oTree</code></span></span></span></span></span>:</span></span></p>
<div><code data-gist-hide-footer="true" data-gist-id="149fb361fdfe933cd6317601e8a7107c" data-gist-file="creaate-decision-tree-and-get-tree.R"></code></div>
<p><span><span>At this point all action moved back inside R with its unparalleled access to analytical and visualization tools. So before navigating and plotting a decision tree – final goal for this post – let’s have brief intro to networks in R.</span></span><br><span><span><br></span></span></p>
<h3><span></span><span><span>Overview of Network Analysis in R</span></span></h3>
<p><span><span>R offers arguably the richest functionality when it comes to analyzing and visualizing network (graph, tree) objects. Before taking on the task of conquering it spend time visiting a couple of comprehensive articles describing vast landscape of tools and approaches available: <a rel="nofollow" href="http://kateto.net/network-visualization" target="_blank">Static and dynamic network visualization with R</a> by Katya Ognyanova and <a rel="nofollow" href="https://www.jessesadler.com/post/network-analysis-with-r/" target="_blank">Introduction to Network Analysis with R</a> by Jesse Sadler.</span></span><br><span><span><br></span></span><span><span>To summarize there are two commonly used packages to manage and analyze networks in R: </span></span><span><span><span><span><span><span><span><code>network</code></span></span></span></span></span> (part of statnet family) and </span></span><span><span><span><span><span><span><span><code>igraph</code></span></span></span></span></span> (family in itself). Each package implements namesake classes to represent network structures so there is significant overlap between the two and  they mask each other’s functions. Preferred approach is picking only one of two: <a rel="nofollow" href="https://www.jessesadler.com/post/network-analysis-with-r/" target="_blank">it appears</a> that </span></span><span><span><span><span><span><span><span><span><span><code>igraph</code></span></span></span></span></span></span></span> is more common for general-purpose applications while </span></span><span><span><span><span><span><span><span><span><span><code>network</code></span></span></span></span></span></span></span> is preferred for social network and statistical analysis (my subjective assessment). And while researching these packages </span></span><span><span>do not forget about package </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>intergraph</code></span></span></span></span></span></span></span></span></span></span></span></span></span> that seamlessly transforms objects between</span></span><span><span><span><span> </span></span><span><span><span><span><span><span><span><span><span><code>network</code></span></span></span></span></span></span></span> and </span></span><span><span><span><span><span><span><span><span><span><span><span><code>igraph</code></span></span></span></span></span></span></span></span></span> </span></span> classes. (And this analysis stopped short of expanding into universe of R packages hosted on <a rel="nofollow" href="http://bioconductor.org/" target="_blank">Bioconductor</a>).</span></span></p>
<p><span><span>When it comes to visualizing networks choices quickly proliferate. Both </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>network</code></span></span></span></span></span></span></span> and </span></span><span><span><span><span><span><span><span><span><span><span><span><code>igraph</code></span></span></span></span></span></span></span></span></span> offer graphical functions that use R base plotting system but it doesn’t stop here. Following packages specialize in advanced visualizations for at least one or both of the classes:</span></span></span></span></span></span></p>
<ul>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><code>ggraph</code></span></span></span></span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><code>ggnet2</code></span></span></span></span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><code>ggnetwork</code></span></span></span></span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><code>visNetwork</code></span></span></span></span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><code>DiagrammeR </code></span></span></span></span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><span><span><span><code>networkD3 </code></span></span></span></span></span></span></span></span></span></span></span></span></span></li>
</ul>
<p><span><span><span><span><span><span><span><span><span><span><span><span><span><code><span>Finally, there is package</span></code></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><code><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>data.tree</code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> designed specifically to create and analyze trees in R. It fits the bill of representing and visualizing decision trees perfectly, so it became a tool of choice for this post. Still, visualizing </span></code></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><code><span><span><span>H2O</span></span> model trees could be fully reproduced with any of network and visualization packages mentioned above. </span> </code></span></span></span></span></span></span></span></span></span></span></span></span></span><br><span><span><span><span><span><span><span><span><span><span><span><span><span></span></span></span></span></span></span></span></span></span></span></span></span></span></p>
<h3><span><span><span><span><span><span><span><span><span><span><span><span><span>Visualizing </span></span></span></span></span>H<sub>2</sub>O<span><span><span><span><span> Trees</span></span></span></span></span></span></span></span></span></span></span></span></span></h3>
<p><span><span><span><span><span><span><span><span><span><span><span><span><span>In the last step a decision tree for the model created by GBM</span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span> moved from </span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span> cluster</span></span></span></span></span></span></span></span></span></span></span></span></span> memory to </span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>H2OTree</code><span><span> </span></span></span>object in R by means of Tree API. Still, specific to </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>the </span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>H2OTree</code><span><span> </span></span></span>object now contains necessary details about decision tree, but not in the format understood by R packages such as</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>data.tree.</code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></p>
<p><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>To fill this gap function</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code> createDataTree(H2OTree)</code></span></span></span></span></span></span></span></span><span><span> created that traverses a tree and translates it from </span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code><span><span><span><span><span><span><span><span><code>H2OTree</code></span></span></span></span></span></span></span></span></code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>into</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span> </span></span><span><span><span><span><span><span><span><span><span><code>data.tree</code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><span> accumulating information about decision tree splits and predictions into node and edge attributes of a tree:</span></span></p>
<div><code data-gist-hide-footer="true" data-gist-id="149fb361fdfe933cd6317601e8a7107c" data-gist-file="map-h2otree-to-datatree.R"></code></div>
<p><span><span>Finally everything lined up and ready for the final step of plotting decision tree:</span></span></p>
<ul>
<li><span><span>single decision tree model created in H2O</span></span></li>
<li><span><span>its structure made available in R</span></span></li>
<li><span><span>and translated to specialized </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>data.tree</code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> for network analysis.</span></span></li>
</ul>
<p><span><span>Styling and plotting </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>data.tree</code></span></span></span></span></span></span></span></span></span></span></span></span></span> objects is built around rich functionality of the </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>DiagrammerR</code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> package. For anything that goes beyond simple plotting read documentation <a rel="nofollow" href="http://rich-iannone.github.io/DiagrammeR/docs.html" target="_blank">here</a> but also remember that for plotting </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>data.tree</code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> takes advantage of:</span></span></p>
<ul>
<li><span><span>hierarchical nature of tree structures</span></span></li>
<li><span><span><a rel="nofollow" href="https://graphviz.gitlab.io/_pages/doc/info/attrs.html" target="_blank">GraphViz attributes</a> to style graph, node and edge properties</span></span></li>
<li><span><span>and dynamic callback functions (in this example </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><code>GetEdgeLabel(node), GetNodeShape(node), GetFontName(node)</code></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>) to customize tree’s feel and look</span></span><span><span> </span></span></li>
</ul>
<p><span><span>The following code will produce this moderately customized decision tree for our </span></span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span><span>H<sub>2</sub>O</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span> model: </span></span></p>
<div><code data-gist-hide-footer="true" data-gist-id="149fb361fdfe933cd6317601e8a7107c" data-gist-file="plot-h2o-decision-tree.R"></code></div>
<p></p>
<div></div>
<div></div>
<table cellspacing="0" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i1.wp.com/2.bp.blogspot.com/-VOZWCpehg0I/XB8m_DIfDMI/AAAAAAABKIw/Ww8DHvuanlIjSzU3w6EQR3R0fJ6xyLF0ACLcBGAs/s1600/decision-tree-titanic-final.png?ssl=1" target="_blank"><img border="0" src="https://i2.wp.com/2.bp.blogspot.com/-VOZWCpehg0I/XB8m_DIfDMI/AAAAAAABKIw/Ww8DHvuanlIjSzU3w6EQR3R0fJ6xyLF0ACLcBGAs/s640/decision-tree-titanic-final.png?resize=450%2C1000&ssl=1" data-original-width="450" width="450" data-original-height="1000" data-recalc-dims="1"></a></td>
</tr>
<tr>
<td><span><span>Figure 5. H2O Decision Tree for Titanic Model Visualized with data.tree in R</span></span></td>
</tr>
</tbody>
</table>
<p><span><span>     </span></span></p>
<h3><span><span>References</span></span></h3>
<ul>
<li><a rel="nofollow" href="https://www.h2o.ai/blog/anomaly-detection-with-isolation-forests-using-h2o/" target="_blank"><span><span>Anomaly Detection with Isolation Forests using H2O</span></span></a></li>
<li><span><span><a rel="nofollow" href="https://github.com/h2oai/h2o-3/blob/master/Changes.md#xia-32201---10262018" target="_blank">Changes in H2O Xia (3.22.0.1) – 10/26/2018</a></span> </span></li>
<li><span><span><span><span><span><span><a rel="nofollow" href="https://dzone.com/articles/visualizing-h2o-gbm-and-random-forest-mojo-models" target="_blank">Visualizing H2O GBM and Random Forest MOJO Model Trees</a></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><a rel="nofollow" href="https://dzone.com/articles/inspecting-decision-trees-in-h2o" target="_blank">Inspecting Decision Trees in H2O</a> </span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><a rel="nofollow" href="https://en.wikipedia.org/wiki/Predictive_analytics#Classification_and_regression_trees_(CART)" target="_blank">Classification and Regression Trees</a> </span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><a rel="nofollow" href="https://towardsdatascience.com/predicting-the-survival-of-titanic-passengers-30870ccc7e8" target="_blank">Predicting the Survival of Titanic Passengers</a> </span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><a rel="nofollow" href="https://www.kaggle.com/thilakshasilva/predicting-titanic-survival-using-five-algorithms" target="_blank">Predicting Titanic Survival using Five Algorithms</a></span></span></span></span> </span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/drf.html" target="_blank">Distributed Random Forest (DRF)</a></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/gbm.html" target="_blank">Gradient Boosting Machine (GBM)</a></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="https://en.wikipedia.org/wiki/Inflection_point" target="_blank">Inflection Point</a></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><a rel="nofollow" href="http://kateto.net/network-visualization" target="_blank">Static and dynamic network visualization with R</a> by Katya Ognyanova</span></span></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><span><span><a rel="nofollow" href="https://www.jessesadler.com/post/network-analysis-with-r/" target="_blank">Introduction to Network Analysis with R</a> by Jesse Sadler</span></span> </span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="https://cran.r-project.org/web/views/gR.html" target="_blank">CRAN Graphical Models in R Task View</a></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="http://bioconductor.org/packages/release/BiocViews.html#___GraphAndNetwork" target="_blank">Bioconductor GraphAndNetwork packages</a> </span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="https://cran.r-project.org/web/packages/data.tree/vignettes/data.tree.html" target="_blank">Introduction to data.tree</a> </span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="https://github.com/rich-iannone/DiagrammeR" target="_blank">DiagrammeR package on github</a></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="https://graphviz.gitlab.io/_pages/doc/info/attrs.html" target="_blank">Node, Edge, and Graph attributes for Graphviz tools</a></span></span></span></span></span></span></span></span></li>
<li><span><span><span><span><span><span><span><span><a rel="nofollow" href="https://gist.github.com/grigory93/149fb361fdfe933cd6317601e8a7107c" target="_blank">Public GitHub gist with source code</a>  </span></span></span></span></span></span></span></span></li>
</ul>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://novyden.blogspot.com/2018/12/finally-you-can-plot-h2o-decision-trees.html"> novyden</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/bax0EH7NnRI" height="1"></a></div>