<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/analyze-pull-requests-and-travis-builds-using-rperform/"></div></div><div><a data-url="https://www.r-bloggers.com/analyze-pull-requests-and-travis-builds-using-rperform/" data-counturl="https://www.r-bloggers.com/analyze-pull-requests-and-travis-builds-using-rperform/" data-count="horizontal" data-text="Analyze pull requests and Travis builds using Rperform" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://techandmortals.wordpress.com/2016/09/05/analyze-pull-requests-and-travis-builds-using-rperform/"> R – Tech and Mortals</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<blockquote>
<div><em>Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.</em></div>
<div><em>– Martin Golding</em></div>
</blockquote>
<p>In previous posts, I had discussed how <a target="_blank" href="http://github.com/analyticalmonk/rperform/">Rperform</a> can be used to <a target="_blank" href="https://techandmortals.wordpress.com/2016/07/08/obtaining-package-performance-data-using-rperform/">obtain</a> and <a target="_blank" href="https://techandmortals.wordpress.com/2016/06/28/visualizing-package-performance-using-rperform-and-grammar-of-graphics/">visualize</a> package performance data. However, real-world software development is a collaborative process. Thus, automating performance testing for your package is not only a good idea, it’s a critical one; testing projects locally might not be good enough. This post will cover usage of Rperform with Travis CI for automated performance testing. <strong>More importantly, we will be able to assess performance impact of incoming PRs</strong>.</p>
<h3>Background</h3>
<p><b><a target="_blank" href="http://travis-ci.org/">Travis CI</a></b> is a hosted, distributed <a title="Continuous integration" target="_blank" href="https://en.wikipedia.org/wiki/Continuous_integration">continuous integration</a> service used to build and test software projects hosted on <a title="GitHub" target="_blank" href="https://en.wikipedia.org/wiki/GitHub">GitHub</a>. <strong>Open source projects may be tested at no charge via travis-ci.org.</strong> Private projects may be tested at the same location on a fee basis.<br>
The travis system provides support for many major programming languages, including <a target="_blank" href="https://docs.travis-ci.com/user/languages/r/">R</a>. Detailed documentation is available on their website, including a <a target="_blank" href="https://docs.travis-ci.com/user/for-beginners">guide for complete beginners</a> to continuous integration and a <a target="_blank" href="https://docs.travis-ci.com/user/getting-started">tutorial</a> for using Travis CI with a github hosted repository.</p>
<p><strong><a target="_blank" href="https://github.com/analyticalmonk/rperform">Rperform</a></strong> is a package that allows R developers to track and visualize quantitative performance metrics of their code. It was initiated as a <a target="_blank" href="https://www.google-melange.com/archive/gsoc/2015/orgs/rproject/projects/akashtndn.html">GSoC 2015 project</a> and was also a part of <a target="_blank" href="https://summerofcode.withgoogle.com/projects/?sp-search=akashtndn#5195867359281152">GSoC 2016 </a>under the organization, <a target="_blank" href="https://www.r-project.org/">R Project for Statistical Computing</a>.<br>
It focuses on providing changes in a package’s performance metrics, related to runtime and memory, over different git versions and across git branches. Rperform can be integrated with Travis CI to do performance testing during Travis builds by making changes to the repo’s .travis.yml file. <strong>It can prove to be particularly useful while measuring the possible changes which can be introduced by a pull request (PR).</strong><br>
More information about the package, including introductory tutorials, can be found on its <a target="_blank" href="https://github.com/analyticalmonk/rperform/wiki">Github Wiki</a>.</p>
<h3>Overview</h3>
<p>When code is pushed into a repository using Travis CI, a <a target="_blank" href="https://en.wikipedia.org/wiki/Build_automation">build</a> for the project gets triggered on the Travis system. A build gets triggered in the event of an incoming pull request (PR) as well. All that needs to be done for Travis to work is to include a configuration file in the github repo, and enable it for the repository you want to build on your Travis profile page.</p>
<p>
<figure data-shortcode="caption" class="wp-caption"><img alt="travis_steps" class="wp-image-1071" src="https://i0.wp.com/techandmortals.files.wordpress.com/2016/09/travis_steps.png?w=456&ssl=1" data-recalc-dims="1"><br>
<figcaption class="wp-caption-text">Image from <a target="_blank" href="https://travis-ci.org/">https://travis-ci.org</a></figcaption>
</figure>
<p>By adding a few lines to the Travis configuration file, Rperform can be set up to run each time a build gets triggered. The results for each build are saved as a webpage in a subdirectory named ‘<em>Rperform</em>‘ on the repo’s gh-pages branch. An example of such a webpage can be found <a target="_blank" href="http://akashtandon.com/PRDemo_Rperform/Rperform">here</a>.</p>
<h3>Setup</h3>
<h4>Dummy package</h4>
<p>We will use a dummy package to understand how Rperform can be integrated with Travis CI builds. This package, <a target="_blank" href="https://github.com/analyticalmonk/PRDemo_Rperform">PRDemo_Rperform</a>, is hosted on Github and is already set up to run Rperform with Travis. Before being configured for Rperform, the .travis.yml file <a target="_blank" href="https://github.com/analyticalmonk/PRDemo_Rperform/blob/3fb574425ba46be4399152f304c30d41b378b392/.travis.yml">looked like this</a>, as opposed to <a target="_blank" href="https://github.com/analyticalmonk/PRDemo_Rperform/blob/3fb574425ba46be4399152f304c30d41b378b392/.travis.yml">how it looks now</a>.<br>
<strong>Rest of the section explains the steps that were followed to configure the repo.</strong></p>
<p><em>Note: Steps provided in this section can also be found as a <a target="_blank" href="https://github.com/analyticalmonk/Rperform/wiki/Tutorial:-Using-Rperform-with-Travis-CI">follow-along tutorial</a> on Rperform’s wiki.</em></p>
<h4>Step 1: Generation of an encrypted access token</h4>
<p>You will need to generate an access token that will be used to allow Travis to push a webpage containing Rperform’s results to your GitHub repo’s gh-page branch. This will be some really long alphanumeric string. You can generate one by going to <code>settings -> personal access tokens -> generate new token</code>. <strong>Since we are working with a public repo, selecting only the <em>public_repo</em> option under scope would do.</strong></p>
<p>You will also need to install the travis ruby framework.</p>
<p><code>gem install travis</code></p>
<p>After installing, navigate to your git repo for the project you want to enable automatic pushing of content for, and then login to travis-ci and secure the GitHub token.</p>
<p><code>travis login</code><br>
<code>travis encrypt GH_TOKEN=&quot;yourgithubtoken&quot;</code></p>
<p>This will generate output that should be copied to your .travis.yml file in a subsequent step. We will refer to this output as <em>yoursecurestring</em>. Essentially we have created an environment variable GH_TOKEN with your actual GitHub token, that is encrypted on the travis-ci servers. <strong>So this way you don’t expose your actual GitHub token to anyone who looks at your .travis.yml file!</strong></p>
<h4>Step 2: Creation of gh-pages branch</h4>
<ul>
<li>
<p>Create and checkout into an orphan gh-pages branch in the RperformTestPackage directory. <em>This step is required if a gh-pages branch doesn’t already exist for the repo. Otherwise, simply checking out into the gh-pages branch will do.</em></p>
</li>
</ul>
<pre><code>cd RperformTestPackage
git checkout --orphan gh-pages
git rm -rf .
</code></pre>
<ul>
<li>Make an Rperform directory and create an empty <em>index.html</em> file inside it.</li>
</ul>
<pre><code>mkdir Rperform
touch Rperform/index.html
</code></pre>
<ul>
<li>Commit and push the changes. Then, checkout into the master branch.</li>
</ul>
<pre><code>git add Rperform/index.html
git commit -m &quot;Added Rperform directory in gh-pages branch&quot;
git push origin gh-pages
git checkout master
</code></pre>
<p> </p>
<h4>Step 3: Updating the .travis.yml file</h4>
<ul>
<li>
<p>For a sample .travis.yml file configured to work with Rperform, look <a target="_blank" href="https://github.com/analyticalmonk/Rperform/blob/master/SampleFiles_TravisPR/sample_travis.yml">here</a>. We use this sample file as a reference.</p>
</li>
<li>
<p>Add the package names given under <em>r_packages</em> and <em>r_github_packages</em> in the sample file to your .travis.yml file under the respective sections.</p>
</li>
</ul>
<pre><code>r_packages:
  - microbenchmark
  - roxygen2
  - rmarkdown

r_github_packages:
  - analyticalmonk/Rperform
</code></pre>
<ul>
<li>Add the command given under <em>before_script</em> in the sample file to your .travis.yml file.</li>
</ul>
<pre><code>before_script:
- travis_wait 30 source `Rscript -e &quot;cat(find.package(\&quot;Rperform\&quot;))&quot;`/push_gh_pages.sh
</code></pre>
<ul>
<li>Next, we need to configure our environment variables.</li>
</ul>
<pre><code>env:
  global:
  - secure: &quot;ENTER_YOUR_ENCRYPTED_GITHUB_ACCESS_TOKEN_HERE&quot;
  - USER_EMAIL=&quot;INSERT_A_VALID_EMAIL_ID_HERE&quot;
  - USER_NAME=&quot;INSERT_USERNAME_HERE&quot;
  - PR_COMMAND=&quot;Rperform::plot_PR_webpage('./INSERT_PATH_TO_TEST_DIR/INSERT_TEST_NAME_HERE', metric = 'time')&quot;
  - RPERFORM_COMMAND=&quot;Rperform::plot_webpage(test_directory = './INSERT_PATH_TO_TEST_DIR/', metric = 'time')&quot;
</code></pre>
<p>Your <em>encrypted_github_access_token</em> is nothing but the <em>yoursecurestring</em> generated in Step 1.<br>
USER_EMAIL and USER_NAME aren’t required but put them in anyway if you want.<br>
RPERFORM_COMMAND and PR_COMMAND are the Rperform commands run when a regular build and Pull Request build are triggered respectively. We will focus on regular builds. More on PR builds in another tutorial.</p>
<p><em>When configured for PRDemo_Rperform, these environment variables look like this:</em></p>
<pre><code>env:
  global:
  - secure: &quot;<span>p17aMAh2hcyzK59ExIh8wCirdFvZM/BdoOavFomi4oIF4vIN51XDEoIQgySHcL2dEr24BJapIJTrFnfTOzc/6dyfEnF5HJssn1ilBM5jCWy2jY5fQDm0VZqqIHH8bFKRuoC2v6phP3wgKVK8YBAqpbpM338v0RsxAVNjtw0Rf1UO3h4RQMxn+BSvGiB7RKM+7Q24qpmzFxUNvDPT+R67Tguz92ohyoFj4fC9tWYU+qJuznGiciOTtI3wkfaD/zxIt+LjfpxKUJlNzD0Vq4jAmzrO8C5F5YSXGiwtrPoJhnUoQRvDVH4RChQvybCQ9CjzuMSp36DS/5FdDtve5GrmFcW2chSL/DC4/6ZEQLnpc8bU5WkfsTwt5eEvxesSBilGvihg2WwQLXJMeZUf0LSJKm0a4wJCRxqb9kkq0bRHZnv/CP2XZ9VD4rzCIS7zBYk3CI5UR2NYKFHNTy4mv7jeKFgWvSJFCiERW9BIdS0hUK00y/6g1lItdiPbUwllcORzzuqXHO+YD54JqzhWpIzCIlXZzdXBeaZTPoZy04iNeQREVZBR/YCEpL7ilQpZQ07yxV05IZZXiM/B4llBdgZPhUTH/anxCO6Y4x84r6zYZ/ph4+QzO4W3AC+p3wi/6B076+kHUr/0tuO1Q0hrJACGAE8LOMGG04Lc/eBWFReTido=</span>&quot;
  - USER_EMAIL=&quot;analyticalmonk@github.com&quot;
  - USER_NAME=&quot;analyticalmonk&quot;
  - <span>PR_COMMAND=&quot;Rperform::plot_PR_webpage(test_path = './tests/testthat/testTime.R', metric = 'time')&quot;</span>
  - RPERFORM_COMMAND=&quot;Rperform::plot_webpage(test_directory = './tests/testthat', metric = 'time')&quot;
</code></pre>
<ul>
<li>Add and commit the changes.</li>
</ul>
<pre><code>git add .
git commit -m &quot;Configured Travis CI to run Rperform&quot;
</code></pre>
<h3></h3>
<h3>How it works?</h3>
<p>As explained in a <a target="_blank" href="https://techandmortals.wordpress.com/2016/07/08/obtaining-package-performance-data-using-rperform/">previous post</a>, <a target="_blank" href="https://github.com/analyticalmonk/Rperform">Rperform</a> can be used for performance testing by analyzing a package’s test files. In PRDemo_Rperform’s case, there are <a target="_blank" href="https://github.com/analyticalmonk/PRDemo_Rperform/tree/master/R">two types of functions</a> which need to be analyzed. We do so by running Rperform commands, written into the .travis.yml file as environment variables in the previous section, on the <a target="_blank" href="https://github.com/analyticalmonk/PRDemo_Rperform/tree/master/tests/testthat">packages’s test files</a>.</p>
<p>The performance test results generated (as webpages) for the PRDemo_Rperform can be found here: <a target="_blank" href="https://github.com/analyticalmonk/PRDemo_Rperform/tree/gh-pages/Rperform">https://github.com/analyticalmonk/PRDemo_Rperform/tree/gh-pages/Rperform</a>. Latest build results can be directly viewed here: <a target="_blank" href="https://analyticalmonk.github.io/PRDemo_Rperform/Rperform">https://analyticalmonk.github.io/PRDemo_Rperform/Rperform</a>.</p>
<p><em>Note: The shell script responsible for pushing generated webpages to the repo can be found <a target="_blank" href="https://github.com/analyticalmonk/Rperform/blob/master/inst/push_gh_pages.sh">here</a>.</em></p>
<h4>Pull Requests</h4>
<p>Provided that a PR is started by a contributor (someone with access to the repository), a performance webpage will be generated for it as well. Two PRs were started for PRDemo_Rperform, as seen <a target="_blank" href="https://github.com/analyticalmonk/PRDemo_Rperform/pulls">here</a>. One of them was by the repo owner and other by a contributor. <strong>Through the webpages generated for both <a target="_blank" href="https://github.com/analyticalmonk/PRDemo_Rperform/blob/gh-pages/Rperform/index_buildnum4.html">PR#1</a> and <a target="_blank" href="https://github.com/analyticalmonk/PRDemo_Rperform/blob/gh-pages/Rperform/index_buildnum7.html">PR#2</a>, we can assess the potential performance impact of merging them.</strong></p>
<p>For example, PR#2 slows down the justTimeIt2() function.</p>
<p>
<figure data-shortcode="caption" class="wp-caption"><img alt="PRDemo_RperformPR" class="wp-image-1178" src="https://i1.wp.com/techandmortals.files.wordpress.com/2016/09/prdemo_rperformpr.png?w=456&ssl=1" data-recalc-dims="1"><br>
<figcaption class="wp-caption-text">After the merge, the test for justTimeIt2() takes ~0.8 seconds as opposed to 0.4 previously.</figcaption>
</figure>
<p><em>Tip: To see the webpage for yourself, copy the contents of the HTML file into a real-time HTML editor such as <a target="_blank" href="http://htmledit.squarefree.com/">this one</a>.</em></p>
<h3>Road Ahead</h3>
<p>Work needs to be done on creating a UI. Also, there are minor functions yet to implemented for the case of PR builds. But the package, even in its current state, should prove to be pretty useful. If you are an R package developer, please <a target="_blank" href="http://github.com/analyticalmonk/rperform/">try out Rperform</a> on your code and provide feedback if possible.</p>
<p><strong>Note:<br>
If you want to get in touch, drop me a <a target="_blank" href="mailto:akashtndn.acm@gmail.com">mail</a>, or hit me up on <a target="_blank" href="http://twitter.com/AkashTandon">Twitter</a>, <a target="_blank" href="http://github.com/analyticalmonk">Github</a> or <a target="_blank" href="https://www.quora.com/profile/Akash-Tandon-1">Quora</a>.</strong><br>
<strong> If any problem arises, please <a target="_blank" href="http://github.com/analyticalmonk/rperform/issues">open an issue</a> on Github.<br>
You can find out more about me on my <a target="_blank" href="http://akashtandon.com/">personal webpage</a>.<br>
</strong></p>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/techandmortals.wordpress.com/976/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/techandmortals.wordpress.com/976/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/analyze-pull-requests-and-travis-builds-using-rperform/"></div></div><div><a data-url="https://www.r-bloggers.com/analyze-pull-requests-and-travis-builds-using-rperform/" data-counturl="https://www.r-bloggers.com/analyze-pull-requests-and-travis-builds-using-rperform/" data-count="vertical" data-text="Analyze pull requests and Travis builds using Rperform" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://techandmortals.wordpress.com/2016/09/05/analyze-pull-requests-and-travis-builds-using-rperform/"> R – Tech and Mortals</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/M3jnkjqUvuo"></a></div>