<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/too-much-parallelism-is-as-bad/"></div></div><div><a data-url="http://www.r-bloggers.com/too-much-parallelism-is-as-bad/" data-counturl="http://www.r-bloggers.com/too-much-parallelism-is-as-bad/" data-count="horizontal" data-text="Too Much Parallelism is as Bad" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.quintuitive.com/2016/05/08/much-parallelism-bad/"> R – Quintuitive</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>The other day I run a machine learning backtest on a new data set. Once I got through the LDA and QDA initial run, I decided to try <a target="_blank" href="https://cran.r-project.org/web/packages/xgboost/">xgboost</a>. The first thing I observed was a really bad performance. The results from the following debugging session were quite surprising to me.<br>
<span></span><br>
I have been using the same framework for a few years now. I think there are some examples outlining the approach even on this blog, but I am lazy to dig them out now. Without going into further details, let me outline my “stack”:</p>
<div align="center">
<a target="_blank" href="http://i0.wp.com/www.quintuitive.com/wp-content/uploads/2016/05/stack-1.png"><img sizes="(max-width: 415px) 100vw, 415px" srcset="http://i0.wp.com/www.quintuitive.com/wp-content/uploads/2016/05/stack-1.png?resize=415%2C320 415w, http://www.quintuitive.com/wp-content/uploads/2016/05/stack-1-240x185.png 240w" alt="stack" class="wp-image-2938" src="http://i0.wp.com/www.quintuitive.com/wp-content/uploads/2016/05/stack-1.png?resize=415%2C320" data-recalc-dims="1"></a>
</div>
<p>As I mentioned, I have been using this stack for a few years now, and during this time, I have seen some really slow models. Two factors got me suspicious in this case:</p>
<ul>
<li>I was using a new method – yeah, my first attempt with xgboost.</li>
<li>The data set was rather small and simple.</li>
</ul>
<p>What I found out was that there was too much parallelization happening. Somehow, all these threads and process were getting messed up, and, although there was progress, it was glacially slow.</p>
<p>Looking at the stack – the parallization is not that obvious. Certainly I was using multiple processes via the parallel package, but what else – I was seeing a lot more threads running. The culprit in this case was the default parallelization in xgboost. Nowadays apparently every layer is trying to exploit multiple cores, thus, that wasn’t surprising, just something new to me.</p>
<p>The fix ended up being quite simple – call caret’s <em>train</em> with <em>nthread=1</em>, which in turn is passed to <em>xgb.train</em> and solves the problem.</p>
<p>Looking at the stack above, I realized that, potentially, there might be other similar issues. For instance, Microsoft’s R Open provides some multi-threaded improvements via the <a target="_blank" href="https://software.intel.com/en-us/intel-mkl/?cid=sem43700010399172817&intel_term=intel+mkl&%23038;gclid=CjwKEAjwpLa5BRCTwcXS6_rpvC4SJACTDQMMQrhCw4ThIDU-3myUZLBumUi_NovTseKdycwCid_K5xoCDUTw_wcB&%23038;gclsrc=aw.ds">Intel’s MKL library</a>. In my case, that was not causing any observable problems, but in case it is – the threading can be disabled via:</p>
<pre>
setMKLthreads(1)
</pre>
<p>Now everything is up and running, and I am looking forward to the output.</p>
<p>The post <a target="_blank" rel="nofollow" href="http://www.quintuitive.com/2016/05/08/much-parallelism-bad/">Too Much Parallelism is as Bad</a> appeared first on <a target="_blank" rel="nofollow" href="http://www.quintuitive.com/">Quintuitive</a>.</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/too-much-parallelism-is-as-bad/"></div></div><div><a data-url="http://www.r-bloggers.com/too-much-parallelism-is-as-bad/" data-counturl="http://www.r-bloggers.com/too-much-parallelism-is-as-bad/" data-count="vertical" data-text="Too Much Parallelism is as Bad" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.quintuitive.com/2016/05/08/much-parallelism-bad/"> R – Quintuitive</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/0-wmWJWx3kg"></a></div>