<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/new-version-of-pqr-with-major-speed-improvements/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://radfordneal.wordpress.com/2018/11/25/new-version-of-pqr-with-major-speed-improvements/"> R Programming – Radford Neal's blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<table>
<tbody>
<tr>
<td valign="top"><a rel="nofollow" href="https://radfordneal.files.wordpress.com/2013/12/logolg.png" target="_blank"><img src="https://radfordneal.files.wordpress.com/2013/12/logolg.png?w=256&h=200&fit=256%2C200&resize=256%2C200" alt="" width="256" align="top" data-recalc-dims="1" height="200"></a></td>
<td valign="top">I’ve released pqR-2018-11-18, a new version of my variant implementation of R.  You can install it on Linux, Windows, or Mac as described at <a rel="nofollow" href="http://www.pqr-project.org/" target="_blank">pqR-project.org</a>. Installation must currently be from source, similarly to source installs of R Core versions of R.</td>
</tr>
</tbody>
</table>
<p>This version has some major speed improvements, as well as some new features. I’ll details some of these improvements in future posts. Here, I’ll just mention a few things to show the flavour of the improvements in this release, and why you might be interested in pqR as an alternative to the R Core implementation.</p>
<h3>Performance improvements</h3>
<p>One landmark reached in this release is that it is no longer advisable to use the byte-code compiler in pqR. The speed of direct interpretation of R code has now been improved to the point where it is about as fast at executing simple scalar code as the byte-code interpreter. Eliminating the byte-code compiler simplifies the overall implementation, and avoids possible semantic differences between interpreted and byte-compiled code. It is also important for pqR because some pqR optimizations and some new pqR features are not implemented in byte-code. For example, only the interpreter does optimizations such as deferring vector operations so that they may automatically be merged with other operations or be done in parallel when multiple cores are available.</p>
<p>Some vector operations have been substantially sped up compared to the previous release of pqR-2017-06-09. The improvement compared to R-3.5.1 can be even greater. Here is an example of replacing a subset of vector elements, benchmarked on an Intel “Skylake” processor, with both pqR-2018-11-18 and R-3.5.1 compiled from source with gcc 8.2.0 at optimization level -O3:</p>
<p>Here’s R-3.5.1:</p>
<pre>> a &lt;- numeric(20000)
> system.time(for (i in 1:100000) a[2:19999] &lt;- 3.1)
   user  system elapsed 
  4.211   0.148   4.360 
</pre>
<p>And here’s pqR-2018-11-18:</p>
<pre>> a &lt;- numeric(20000)
> system.time(for (i in 1:100000) a[2:19999] &lt;- 3.1)
   user  system elapsed 
  0.256   0.000   0.257 
</pre>
<p>So the current R Core implementation is 17 times slower than pqR for this replacement operation.<br>
</p>
<p>The advantage of pqR isn’t always this large, but many vector operations are sped up by smaller but still significant factors. An example:</p>
<p>With R-3.5.1:</p>
<pre>> a &lt;- seq(0,1,length=2000); b &lt;- seq(1,0,length=2000)
> system.time (for (i in 1:100000) {
+       d &lt;- abs(a-b); r &lt;- sum (d>0.4 &amp; d&lt;0.7) })
   user  system elapsed 
  1.215   0.015   1.231 
</pre>
<p>With pqR-2018-11-18:</p>
<pre>> a &lt;- seq(0,1,length=2000); b &lt;- seq(1,0,length=2000)
> system.time (for (i in 1:100000) {
+       d &lt;- abs(a-b); r &lt;- sum (d>0.4 &amp; d&lt;0.7) })
   user  system elapsed 
  0.654   0.008   0.662 
</pre>
<p>So for this example, pqR is almost twice as fast.</p>
<p>For some operations, pqR’s implementation has lower asymptotic time complexity, and so can be enormously faster. An example is the following convenient coding pattern that R programmers are currently warned to avoid:</p>
<p>With R-3.5.1:</p>
<pre>> n &lt;- 200000; a &lt;- numeric(0);
> system.time (for (i in 1:n) a &lt;- c(a,(i+1)^2))
   user  system elapsed 
 30.387   0.223  30.612 
</pre>
<p>With pqR-2018-11-18:</p>
<pre>> n &lt;- 200000; a &lt;- numeric(0);
> system.time (for (i in 1:n) a &lt;- c(a,(i+1)^2))
   user  system elapsed 
  0.040   0.004   0.045 
</pre>
<p>In R-3.5.1, extending a vector one element at a time with “c” takes time growing as n<sup>2</sup>, as a new vector is allocated when each element is appended. With the latest version of pqR, the time grows only as n log n. In this example, that leads to pqR being 680 times faster, but the ratio could be made arbitrarily large by increasing n.</p>
<p>It’s still faster in pqR to preallocate a vector of length n, but only by about a factor of three, which would often be tolerable when writing one-off code if using “c” is more convenient.</p>
<h3>New features</h3>
<p>The latest version of pqR has some new features. As for earlier pqR versions, some new features are aimed at addressing <a rel="nofollow" href="https://radfordneal.wordpress.com/2016/06/25/fixing-rs-design-flaws-in-a-new-version-of-pqr" target="_blank">design flaws</a> in R that lead to unreliable code, and others are aimed at making R more convenient for programming and scripting.</p>
<p>One new convenience feature is that the paste and paste0 operations can now be written with new <tt>!!</tt> and <tt>!</tt> operators. For example,</p>
<pre>> city &lt;- &quot;Toronto&quot;; province &lt;- &quot;Ontario&quot;
> city ! &quot;,&quot; !! province
[1] &quot;Toronto, Ontario&quot;
</pre>
<p>The <tt>!!</tt> operator pastes strings together with space separation; the <tt>!</tt> operator pastes with no separation. Of course, <tt>!</tt> retains its meaning of “not” when used as a unary operator; there is no ambiguity.</p>
<h3>What next?</h3>
<p>I’ll be writing some more blog posts regarding improvements in pqR-2018-11-18, and regarding some improvements in earlier pqR versions that I haven’t already blogged about. Of course, you can read about these now in the <a rel="nofollow" href="http://pqr-project.org/NEWS.txt" target="_blank">pqR NEWS file</a>.</p>
<p>The main disadvantage of pqR is that it is not fully compatible with the current R Core version. It is a fork of R-2.15.0, with many, but not all, later changes incorporated. This affects <a rel="nofollow" href="https://github.com/radfordneal/pqR/wiki/Packages-tested-with-pqR" target="_blank">what packages will work with pqR</a>.</p>
<p>Addressing this compatibility issue is one thing that needs to be done going forward. I’ll discuss this and other plans — notably implementing automatic differentiation — in another future blog post.</p>
<p>I’m open to other people getting involved in this project. Of course, you can contribute now by trying out pqR and reporting any problems in the comments here or at the <a rel="nofollow" href="https://github.com/radfordneal/pqR/issues" target="_blank">pqR issues page</a>. Performance comparisons, especially on real-world applications, are also welcome.</p>
<p>Finally, for the paranoid, here are the shasum values for the compressed and uncompressed tar files that you can download from <a rel="nofollow" href="http://www.pqr-project.org/" target="_blank">pqR-project.org</a>:</p>
<pre>89216dc76be23b3928c26561acc155b6e5ad32f3  pqR-2018-11-18.tar.gz
f0ee8a37198b7e078fa1aec7dd5cda762f1a7799  pqR-2018-11-18.tar
</pre>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://radfordneal.wordpress.com/2018/11/25/new-version-of-pqr-with-major-speed-improvements/"> R Programming – Radford Neal's blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/-tMNE7E99ts" height="1"></a></div>