<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/parallel-computing-exercises-snow-and-rmpi-part-3/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.r-exercises.com/2017/08/03/parallel-computing-exercises-snow-and-rmpi-part-3/?utm_source=rss&utm_medium=rss&utm_campaign=parallel-computing-exercises-snow-and-rmpi-part-3"> R-exercises</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a rel="nofollow" href="https://i0.wp.com/www.r-exercises.com/wp-content/uploads/2017/07/parallel-computing-3.jpg" target="_blank"><img alt="" src="https://i0.wp.com/www.r-exercises.com/wp-content/uploads/2017/07/parallel-computing-3.jpg?resize=265%2C190" class="wp-image-7213" data-recalc-dims="1"></a><br>
The <code>foreach</code> statement, which was introduced in the previous set of exercises of this series, can work with various parallel backends. This set allows to train in working with backends provided by the <code>snow</code> and <code>Rmpi</code> packages (on a single machine with multiple CPUs). The name of the former package stands for “Simple Network of Workstations”. It can employ various parallelization techniques; socket clustering is used here. The latter one is an R’s wrapper for the MPI (Message-Passing Interface), which is another paralellization technique.<br>
The set also demonstrates that inter-process communication overhead has to be taken into account when preparing to use parallelization. If short tasks are run in parallel the overhead can offset the gains in performance from using multiple CPUs, and in some cases execution can get even slower. For parallelization to be useful, tasks that are run in parallel have to be long enough.<br>
The exercises are based on an example of using bootstrapping to estimate the sampling distribution of linear regression coefficients. The regression is run multiple times on different sets of data derived from an original sample. The size of each derived data set is equal to the size of the original sample, which is possible because the sets are produced by random sampling with replacement. The original sample is taken from the <code>InstEval</code> data set, which comes with the <code>lme4</code> package, and represents lecture/instructor evaluations by students at the ETH. The estimated distribution is not analyzed in the exercises.<br>
The exercises require the packages <code>foreach</code>, <code>snow</code>, <code>doSNOW</code>, <code>Rmpi</code>, and <code>doMPI</code> to be installed.<br>
IMPORTANT NOTE: the <code>Rmpi</code> package depends on an MPI software, which has to be installed on the machine separately. The software can be the following:</p>
<ul>
<li>Windows: either the <code>Microsoft MPI</code>, or <code>Open MPI</code> library (the former one can be installed as an ordinary application).</li>
<li>OS X/macOS: the <code>Open MPI</code> library (available through <code>Homebrew</code>).</li>
<li>Linux: the <code>Open MPI</code> library (look for packages named <code>  libopenmpi</code> (or <code>openmpi</code>, <code>lib64openmpi</code>, or similar), as well as <code>libopenmpi-dev</code> (or <code>libopenmpi-devel</code>, or similar) in your distribution’s repository).</li>
</ul>
<p>The zipped data set can be downloaded <a rel="nofollow" href="http://www.r-exercises.com/wp-content/uploads/2017/07/InstEval_reduced.zip" target="_blank">here</a>. For other parts of the series follow the tag <a rel="nofollow" href="http://www.r-exercises.com/tags/parallel-computing" target="_blank">parallel computing</a>.<br>
Answers to the exercises are available <a rel="nofollow" href="http://www.r-exercises.com/2017/08/03/parallel-computing-exercises-snow-and-rmpi-part-3-solutions" target="_blank">here</a>.</p>
<p><strong>Exercise 1</strong><br>
Load the data set, and assign it to the <code>data_large</code> variable.</p>
<p><strong>Exercise 2</strong><br>
Create a smaller data set that will be used to compare how parallel computing performance depends on the size of the task. Use the <code>sample</code> function to obtain a random subset from the loaded data. Its size has to be 10% of the size of the original dataset (in terms of rows). Assign the subset to the <code>data_small</code> variable.<br>
For reproducibility, set the seed to <code>1234</code>.<br>
Print the number of rows in the <code>data_large</code> and <code>data_small</code> data sets.</p>
<p><strong>Exercise 3</strong><br>
Write a function that will be used as a task in parallel computing. The function has to take a data set as an input, and do the following:</p>
<ol>
<li>Resample the data, i.e. obtain a new sample of data based on the input data set. The number of rows in the new sample has to be equal to the one in the input data set (use the <code>sample</code> function as in the previous exercise, but change parameters to allow for resampling with replacement).</li>
<li>Run a linear regression on the resampled data. Use <code>y</code> as the dependent variable, and the others as independent variables (this can be done by using the formula <code>y ~ .</code> as an argument to the <code>lm</code> function).</li>
<li>Return a vector of coefficients of the linear regression.</li>
</ol>
<div>
<aside><img src="http://www.r-exercises.com/2017/08/03/parallel-computing-exercises-snow-and-rmpi-part-3/?utm_source=rss&utm_medium=rss&utm_campaign=parallel-computing-exercises-snow-and-rmpi-part-3"></aside>
<div><strong>Learn more</strong> about optimizing your workflow in the online course <a rel="nofollow" href="http://www.r-exercises.com/product/getting-started-with-r-for-data-science/" target="_blank">Getting Started with R for Data Science</a>. In this course you will learn how to:
<ul>
<li>efficiently organize your workflow to get the best performance of your entire project</li>
<li>get a full introduction to using R for a data science project</li>
<li>And much more</li>
</ul>
</div>
</div>
<p><strong>Exercise 4</strong><br>
Let’s test how much time it takes to run the task multiple times sequentially (not in parallel). Use the <code>foreach</code> statement with the <code>%do%</code> operator (as discussed in the previous set of exercises of this series) to run it:</p>
<ul>
<li>10 times with the <code>data_large</code> data set, and</li>
<li>100 times with the <code>data_small</code> data set.</li>
</ul>
<p>Use the <code>rbind</code> function as an argument to <code>foreach</code> to combine the results.<br>
In both cases, measure how much time is spent on execution of the task (with the <code>system.time</code> function). Theoretically, the length of time spent should be roughly the same because the total number of rows processed is equal (it is 100,000 rows: 10,000 rows 10 times in the first case, and 1,000 rows 100 times in the second case), and the row length is the same. But is this the case in practice?</p>
<p><strong>Exercise 5</strong><br>
Now we’ll prepare to run the task in parallel using 2 CPU cores. First, we’ll use a parallel computing backend for the <code>foreach</code> statement from the <code>snow</code> package. This requires to steps:</p>
<ol>
<li>Make a cluster for parallel execution using the <code>makeCluster</code> function from the <code>snow</code> package. Pass two arguments to this function: the size of the cluster (i.e. the number of CPU cores that will be used in computations), and the type of the cluster (<code>&quot;SOCK&quot;</code> in this case).</li>
<li>Register the cluster with the <code>registerDoSNOW</code> function from the <code>doSNOW</code> package (which provides a <code>foreach</code> parallel adapter for the <code>'snow'</code> package).</li>
</ol>
<p><strong>Exercise 6</strong><br>
Run the task 10 times with the large data set in parallel using the <code>foreach</code> statement with the <code>%dopar%</code> operator (as discussed in the previous set of exercises of this series). Measure the time spent on execution with the <code>system.time</code> function.<br>
When done, use the <code>stopCluster</code> function from the <code>snow</code> package to stop the cluster.<br>
Is the length of execution time smaller comparing to the one measured in Exercise 4?</p>
<p><strong>Exercise 7</strong><br>
Repeat the steps listed in Exercise 5 and Exercise 6 to run the task 100 times using the small data set.<br>
What is the change in the execution time?</p>
<p><strong>Exercise 8</strong><br>
Next, we’ll use another parallel backend for the <code>foreach</code> function: the one that is provided by the <code>Rmpi</code> package (R’s wrapper to Message-Passing Interface), and accessible through an adapter from the <code>doMPI</code> package. From the user perspective, it differs from the <code>snow</code>-based backend in the following ways:</p>
<ul>
<li>as mentioned above, additional software has to be installed for this backend to work (either (a) the <code>openmpi</code> library, available for Windows, macOS, and Linux, or (b) the <code>Microsoft MPI</code> library, which is available for Windows,</li>
<li>when an <code>mpi</code> cluster is created, it immediately starts using CPUs as much as it can,</li>
<li>when the work is complete, the <code>mpi</code> execution environment has to be terminated; if terminated, it can’t be relaunched without restarting the R session (if you try to create an <code>mpi</code> cluster after the environment was terminated, the session will be aborted, which may result in a loss of data; see Exercise 10 for more details).</li>
</ul>
<p>In this exercise, we’ll create an <code>mpi</code> execution environment to run the task using 2 CPU cores. This requires actions similar to the ones performed in Exercise 5: </p>
<ol>
<li>Make a cluster for parallel execution using the <code>startMPIcluster</code> function from the <code>doMPI</code> package. This function can take just one argument, which is the number of CPU cores to be used in computations.</li>
<li>Register the cluster with the <code>registerDoMPI</code> function from the <code>doMPI</code> package.</li>
</ol>
<p>After creating a cluster, you may check whether the CPU usage on your machine increased using Resource Monitor (Windows), Activity Monitor (macOS), <code>top</code> or <code>htop</code> commands (Linux), or other tools.</p>
<p><strong>Exercise 9</strong><br>
Stop the cluster created in the previous exercise with the <code>closeCluster</code> command from the <code>doMPI</code> package. The CPU usage should fall immediately.</p>
<p><strong>Exercise 10</strong><br>
Create an <code>mpi</code> cluster again, and use it as a backend for the <code>foreach</code> statement to run the task defined above:</p>
<ul>
<li>10 times with the <code>data_large</code> data set, and</li>
<li>100 times with the <code>data_small</code> data set.</li>
</ul>
<p>In both cases, start a cluster before running the task, and stop it afterwards. Measure how much time is spent on execution of the task. How the time compares to the execution time with the <code>snow</code> cluster (found in Exercises 6 and 7)?<br>
When done working with the clusters, terminate the <code>mpi</code> execution environment with the <code>mpi.finalize</code> function. Note that this function always returns 1.<br>
Important! As mentioned above, if you intend to create an <code>mpi</code> cluster again after the environment was terminated you have to restart the R session, otherwise the current session will be aborted, which may result in a loss of data. In RStudio, an R session can be relaunched from the <code>Session</code> menu (relaunching the session this way does not affect the data, you’ll only need to reload libraries). In other cases, you may have to quit and restart R.</p>
<div>
<h3>Related exercise sets:</h3>
<ol>
<li><a rel="nofollow" href="http://www.r-exercises.com/2017/07/13/parallel-computing-exercises-foreach-and-doparallel-part-2/" title="Parallel Computing Exercises: Foreach and DoParallel (Part-2)" target="_blank">Parallel Computing Exercises: Foreach and DoParallel (Part-2) </a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/2017/07/06/parallel-computing-exercises-snowfall-part-1/" title="Parallel Computing Exercises: Snowfall (Part-1)" target="_blank">Parallel Computing Exercises: Snowfall (Part-1) </a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/2016/10/07/creating-sample-datasets-exercises/" title="Creating Sample Datasets – Exercises" target="_blank">Creating Sample Datasets – Exercises </a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/start-here-to-learn-r/" target="_blank">Explore all our (>1000) R exercises</a></li>
<li><a rel="nofollow" href="http://www.r-exercises.com/r-courses/" target="_blank">Find an R course using our R Course Finder</a> directory</li>
</ol>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.r-exercises.com/2017/08/03/parallel-computing-exercises-snow-and-rmpi-part-3/?utm_source=rss&utm_medium=rss&utm_campaign=parallel-computing-exercises-snow-and-rmpi-part-3"> R-exercises</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/1QRs1-pq_bo" height="1"></a></div>