<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/crul-an-http-client/"></div></div><div><a data-url="https://www.r-bloggers.com/crul-an-http-client/" data-counturl="https://www.r-bloggers.com/crul-an-http-client/" data-count="horizontal" data-text="crul – an HTTP client" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://ropensci.org/blog/technotes/2016/11/09/crul-release"> rOpenSci Blog - R</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>A new package <a target="_blank" rel="nofollow" href="https://cran.rstudio.com/web/packages/crul">crul</a> is<br>
on CRAN. <code>crul</code> is another HTTP client for R, but is relatively simplified<br>
compared to <a target="_blank" rel="nofollow" href="https://github.com/hadley/httr">httr</a>, and is being built<br>
to link closely with <a target="_blank" rel="nofollow" href="https://github.com/ropenscilabs/webmockr">webmockr</a> and <a target="_blank" rel="nofollow" href="https://github.com/ropenscilabs/vcr">vcr</a>. <code>webmockr</code> and<br>
<code>vcr</code> are packages ported from Ruby's <a target="_blank" rel="nofollow" href="https://github.com/bblimke/webmock">webmock</a><br>
and <a target="_blank" rel="nofollow" href="https://github.com/vcr/vcr">vcr</a>, respectively.<br>
They both make mocking HTTP requests really easy.</p>
<p>A major use case for mocking HTTP requests is for unit tests. Nearly all the<br>
packages I work on personally make HTTP<br>
requests in their test suites, so I wanted to make it really easy to<br>
mock HTTP requests. You don't have to use mocking in test suites of course.</p>
<p><code>crul</code> is not meant to replace other HTTP R libraries, but rather to make it<br>
easy to integrate mocking.</p>
<h2>Installation</h2>
<div>
<pre><code data-lang="r">install.packages<span>(</span><span>&quot;crul&quot;</span><span>)</span>
</code></pre>
</div>
<p>If binaries aren't available, try from source:<br>
<code>install.packages(&quot;crul&quot;, type = &quot;source&quot;)</code> or from GitHub:<br>
<code>devtools::install_github(&quot;ropenscilabs/crul&quot;)</code></p>
<div>
<pre><code data-lang="r"><span>library</span><span>(</span>crul<span>)</span>
</code></pre>
</div>
<h2>The client</h2>
<p><code>HttpClient</code> is where to start</p>
<div>
<pre><code data-lang="r"><span>(</span>x <span>&lt;-</span> HttpClient<span>$</span>new<span>(</span>
  url <span>=</span> <span>&quot;https://httpbin.org&quot;</span><span>,</span>
  opts <span>=</span> <span>list</span><span>(</span>
    timeout <span>=</span> <span>1</span>
  <span>),</span>
  headers <span>=</span> <span>list</span><span>(</span>
    a <span>=</span> <span>&quot;hello world&quot;</span>
  <span>)</span>
<span>))</span>
<span>#> &lt;crul connection></span>
<span>#> url: https://httpbin.org</span>
<span>#> options:</span>
<span>#> timeout: 1</span>
<span>#> headers:</span>
<span>#> a: hello world</span>
</code></pre>
</div>
<p>Makes a R6 class, that has all the bits and bobs you'd expect for doing HTTP<br>
requests. When it prints, it gives any defaults you've set. As you update<br>
the object you can see what's been set</p>
<div>
<pre><code data-lang="r">x<span>$</span>opts
<span>#> $timeout</span>
<span>#> [1] 1</span>
</code></pre>
</div>
<div>
<pre><code data-lang="r">x<span>$</span>headers
<span>#> $a</span>
<span>#> [1] &quot;hello world&quot;</span>
</code></pre>
</div>
<h2>Make a request</h2>
<p>The client object created above has http methods that you can call,<br>
and pass paths to, as well as query parameters, body values, and any other<br>
curl options.</p>
<p>Here, we'll do a <strong>GET</strong> request on the route <code>/get</code> on our base url<br>
<code>https://httpbin.org</code> (the full url is then <code>https://httpbin.org/get</code>)</p>
<div>
<pre><code data-lang="r">res <span>&lt;-</span> x<span>$</span><span>get</span><span>(</span><span>&quot;get&quot;</span><span>)</span>
</code></pre>
</div>
<p>The response from a http request is another R6 class <code>HttpResponse</code>, which<br>
has slots for the outputs of the request, and some functions to deal with<br>
the response:</p>
<p>Status code</p>
<div>
<pre><code data-lang="r">res<span>$</span>status_code
<span>#> [1] 200</span>
</code></pre>
</div>
<p>Status code with the message and explanation</p>
<div>
<pre><code data-lang="r">res<span>$</span>status_http<span>()</span>
<span>#> &lt;Status code: 200></span>
<span>#> Message: OK</span>
<span>#> Explanation: Request fulfilled, document follows</span>
</code></pre>
</div>
<p>The content</p>
<div>
<pre><code data-lang="r">res<span>$</span>content
<span>#> [1] 7b 0a 20 20 22 61 72 67 73 22 3a 20 7b 7d 2c 20 0a 20 20 22 68 65 61</span>
<span>#> [24] 64 65 72 73 22 3a 20 7b 0a 20 20 20 20 22 41 22 3a 20 22 68 65 6c 6c</span>
<span>#> [47] 6f 20 77 6f 72 6c 64 22 2c 20 0a 20 20 20 20 22 41 63 63 65 70 74 22</span>
<span>#> [70] 3a 20 22 2a 2f 2a 22 2c 20 0a 20 20 20 20 22 41 63 63 65 70 74 2d 45</span>
<span>#> [93] 6e 63 6f 64 69 6e 67 22 3a 20 22 67 7a 69 70 2c 20 64 65 66 6c 61 74</span>
<span>#> [116] 65 22 2c 20 0a 20 20 20 20 22 48 6f 73 74 22 3a 20 22 68 74 74 70 62</span>
<span>#> [139] 69 6e 2e 6f 72 67 22 2c 20 0a 20 20 20 20 22 55 73 65 72 2d 41 67 65</span>
<span>#> [162] 6e 74 22 3a 20 22 6c 69 62 63 75 72 6c 2f 37 2e 34 39 2e 31 20 72 2d</span>
<span>#> [185] 63 75 72 6c 2f 32 2e 32 20 63 72 75 6c 2f 30 2e 31 2e 30 22 0a 20 20</span>
<span>#> [208] 7d 2c 20 0a 20 20 22 6f 72 69 67 69 6e 22 3a 20 22 31 35 37 2e 31 33</span>
<span>#> [231] 30 2e 31 37 39 2e 38 36 22 2c 20 0a 20 20 22 75 72 6c 22 3a 20 22 68</span>
<span>#> [254] 74 74 70 73 3a 2f 2f 68 74 74 70 62 69 6e 2e 6f 72 67 2f 67 65 74 22</span>
<span>#> [277] 0a 7d 0a</span>
</code></pre>
</div>
<p>HTTP method</p>
<div>
<pre><code data-lang="r">res<span>$</span>method
<span>#> [1] &quot;get&quot;</span>
</code></pre>
</div>
<p>Request headers</p>
<div>
<pre><code data-lang="r">res<span>$</span>request_headers
<span>#> $a</span>
<span>#> [1] &quot;hello world&quot;</span>
</code></pre>
</div>
<p>Response headers</p>
<div>
<pre><code data-lang="r">res<span>$</span>response_headers
<span>#> [1] &quot;HTTP/1.1 200 OK&quot;</span>
<span>#> [2] &quot;Server: nginx&quot;</span>
<span>#> [3] &quot;Date: Wed, 09 Nov 2016 19:25:14 GMT&quot;</span>
<span>#> [4] &quot;Content-Type: application/json&quot;</span>
<span>#> [5] &quot;Content-Length: 279&quot;</span>
<span>#> [6] &quot;Connection: keep-alive&quot;</span>
<span>#> [7] &quot;Access-Control-Allow-Origin: *&quot;</span>
<span>#> [8] &quot;Access-Control-Allow-Credentials: true&quot;</span>
</code></pre>
</div>
<p>And you can parse the content with a provided function:</p>
<div>
<pre><code data-lang="r">res<span>$</span><span>parse</span><span>()</span>
<span>#> [1] &quot;{\n \&quot;args\&quot;: {}, \n \&quot;headers\&quot;: {\n \&quot;A\&quot;: \&quot;hello world\&quot;, \n \&quot;Accept\&quot;: \&quot;*/*\&quot;, \n \&quot;Accept-Encoding\&quot;: \&quot;gzip, deflate\&quot;, \n \&quot;Host\&quot;: \&quot;httpbin.org\&quot;, \n \&quot;User-Agent\&quot;: \&quot;libcurl/7.49.1 r-curl/2.2 crul/0.1.0\&quot;\n }, \n \&quot;origin\&quot;: \&quot;157.130.179.86\&quot;, \n \&quot;url\&quot;: \&quot;https://httpbin.org/get\&quot;\n}\n&quot;</span>
</code></pre>
</div>
<p>parse the JSON</p>
<div>
<pre><code data-lang="r">jsonlite<span>::</span>fromJSON<span>(</span>res<span>$</span><span>parse</span><span>())</span>
<span>#> $args</span>
<span>#> named list()</span>
<span>#></span>
<span>#> $headers</span>
<span>#> $headers$A</span>
<span>#> [1] &quot;hello world&quot;</span>
<span>#></span>
<span>#> $headers$Accept</span>
<span>#> [1] &quot;*/*&quot;</span>
<span>#></span>
<span>#> $headers$`Accept-Encoding`</span>
<span>#> [1] &quot;gzip, deflate&quot;</span>
<span>#></span>
<span>#> $headers$Host</span>
<span>#> [1] &quot;httpbin.org&quot;</span>
<span>#></span>
<span>#> $headers$`User-Agent`</span>
<span>#> [1] &quot;libcurl/7.49.1 r-curl/2.2 crul/0.1.0&quot;</span>
<span>#></span>
<span>#></span>
<span>#> $origin</span>
<span>#> [1] &quot;157.130.179.86&quot;</span>
<span>#></span>
<span>#> $url</span>
<span>#> [1] &quot;https://httpbin.org/get&quot;</span>
</code></pre>
</div>
<h2>Future work</h2>
<h3>Mocking</h3>
<p><code>crul</code> doesn't currently have mocking ability, but I am working right now on<br>
getting the first version of <a target="_blank" rel="nofollow" href="https://github.com/ropenscilabs/webmockr">webmockr</a> on<br>
CRAN. It will work together with <code>crul</code> so that when you use <code>crul</code> you can<br>
choose to turn on mocking, specify which patterns to match for mocking,<br>
and much more. <code>webmockr</code> is built with plugin system in mind, so that we<br>
could make a plugin for <code>httr</code>, <code>RCurl</code>, or any other http R library.</p>
<p>I actually started <code>vcr</code> first, but realized that I needed to go back and<br>
build <code>webmockr</code> first. So once <code>webmockr</code> is up on CRAN, I'll work on getting<br>
<code>vcr</code> on CRAN as well.</p>
<h3>Errors</h3>
<p>Another package I've been working on, <a target="_blank" rel="nofollow" href="https://github.com/sckott/fauxpas">fauxpas</a>, is meant to be a<br>
general purpose HTTP errors package, and work with any HTTP R library.<br>
Other similar languages like Ruby and Python have a better error story,<br>
and I thought I'd try it out for R. I'll work on incorporating <code>fauxpas</code><br>
into <code>crul</code> as well, though probably in Suggests in case users don't want<br>
to use it.</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/crul-an-http-client/"></div></div><div><a data-url="https://www.r-bloggers.com/crul-an-http-client/" data-counturl="https://www.r-bloggers.com/crul-an-http-client/" data-count="vertical" data-text="crul – an HTTP client" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://ropensci.org/blog/technotes/2016/11/09/crul-release"> rOpenSci Blog - R</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/1MHQL_R6-Ns"></a></div>