<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/a-few-thoughts-on-the-existing-code-parallelization/"></div></div><div><a data-url="https://www.r-bloggers.com/a-few-thoughts-on-the-existing-code-parallelization/" data-counturl="https://www.r-bloggers.com/a-few-thoughts-on-the-existing-code-parallelization/" data-count="horizontal" data-text="A few thoughts on the existing code parallelization" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.vesnam.com/Rblog/existing-code-parallelization-yes-or-no/"> Fun with R</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>A few weeks ago I worked on some old code parallelization. The whole process made me think about how efficient parallelization of the existing code in R can really be and what should be considered efficient. There is a lot of information about parallelization in R online (thanks to everybody who put time to write tutorials and overviews on that topic, they were all helpful in their own way!), but here are a few of my thoughts. </p>
<p>Ideally, parallelization would consist of adding a few lines of code that would distribute old procedures across processors and, thus, make them run in parallel. However, this would require that the existing code is written in the portable manners that allows such a simple addition. In R, this would most likely consist of using <em>pply</em> functions instead of loops, minimizing (or avoiding) using <em>for</em> or other loops, avoiding nested loops, and “packing” independent chunks of code into individual functions that would return results in standardize format, e.g., data frame, vector, or a list. Hence, to achieve this seamless parallelization one would either have to specifically write a code with idea of portability in mind or, in general, be an eager supporter (and implementer) of good R programming practices. </p>
<p>Unfortunately, every now and then there are circumstances in which having a working version of a program that performs as expected and runs reasonably fast on the given data set trumps code optimization and (some of the) good practices. And in the most of such cases, once the project is done, the code gets archived without any additional optimization. But what happens when the old code needs to be reused and it does not scale well with the new data sets? If one is lucky (although we can debate whether or not the luck has anything to do with it), pretty efficient parallelization can be achieved by adjusting one or two procedures to make them suitable for distributed processing would also be an acceptable option. But often more effort is required. And that is when one comes to the point when it has to be decided how much effort is really worth putting into existing code parallelization vs (re)writing the parallel version of the code itself?</p>
<p>Clearly, there is no universal answer. But there are at least three things to consider when weighting that decision: 1) who is the original code author (it is probably easier to modify your own code than somebody else’s), 2) when was the code written (how much one remembers about the code/project and are there new libraries available now that can improve the code), and 3) programming practices used in coding, code length, quality of code commenting, and the formatting style used. Generally, building on with your own code is always easier, not only because you are more familiar with your coding style, but also because you are familiar with the problem the code is trying to solve. In that case, even if you decide to rewrite the code, you may still be able to reuse large portions of the original code (just make sure that if your goal is a clean and optimized code, you will reuse only code that fits those criteria). Working with somebody else’s code could be tricky, especially if there are no standardized programming practices defined and code is not well documented. The most important thing here is to be sure to understand what each line of the code intends to do. And of course, in both cases, the new code needs to be tested and the results need to be compared to those obtained by the old code.</p>
<p>Now, let’s talk about some more practical parallelization issues.</p>
<p>R does not support parallelization by default. But as with many other topics, there is a number of great libraries to use to enable parallelization of your code. <a target="_blank" href="https://cran.r-project.org/web/views/HighPerformanceComputing.html">Cran r-project</a> provides a great summary list of libraries for different types of parallel computing: implicit, explicit, hadoop, grid computing, etc. When thinking about how to parallelize the code it is very important to thing to think about where will the code run, as it impacts the way in which parallelization has to be addressed in the code. Thus, the implementation of parallel code that runs locally on a single multiprocessor computer or on a cluster of computers is probably differ, was well as implementation for unix vs windows system.</p>
<p>Probably the easiest way to illustrate this is using <em>registerDoParallel</em> function from the <a target="_blank" href="https://cran.r-project.org/web/packages/doParallel/">doParallel</a> package. This package provides a parallel backend for the <a target="_blank" href="https://cran.r-project.org/web/packages/foreach/index.html">foreach</a> package and represents a merger of <a target="_blank" href="https://cran.r-project.org/web/packages/doMC/index.html">doMC</a> and <a target="_blank" href="https://cran.r-project.org/web/packages/doSNOW/index.html">doSNOW</a> packages. By default, doParallel uses multicore functionality on Unix-like systems and snow functionality on Windows, as multicore supports multiple workers only on those operating systems that support the fork system call. </p>
<p>Let’s say we have an <em>N</em>-core processor (on a local machine) and we want to use <em>N-1</em> cores to run our program. </p>
<p>On Unix-based systems, we could do it as:<br>
<code>cl &lt;- makeCluster(detectCores() - 1)</code><br>
<code>registerDoParallel(cl)</code><br>
<code>the rest of code including foreach</code><br>
<code>#stopCluster(cl)</code></p>
<p>Where <em>cl</em> represents cluster size, or in this case a number of workers that will be used to execute the program.</p>
<p>Alternatively, we could specify the number of cores we want to use directly as:<br>
<code>registerDoParallel(cores = (detectCores() - 1))</code><br>
<code></code>