<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/asynchronous-api-calls-with-postlightmercury/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.56n.dk/asynchronous-api-calls-with-postlightmercury/"> Renglish – 56north | Skræddersyet dataanalyse</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<h1></h1>
</div>
<p>In this post I’ll tell you about a new package I’ve built and also take you under the hood to show you a really cool thing thats going on: asynchronous API calls.</p>
<div>
<h2>The package: postlightmercury</h2>
<p>I created a package called <a rel="nofollow" href="https://cran.r-project.org/web/packages/postlightmercury/" target="_blank"><code>postlightmercury</code></a> which is now on cran. The package is a wrapper for the <a rel="nofollow" href="https://mercury.postlight.com/" target="_blank">Mercury web parser by Postlight</a>.</p>
<p>Basically you sign up for free, get an API key and with that you can send it urls that it then parses for you. This is actually pretty clever if you are scraping a lot of different websites and you don’t want to write a web parser for each and every one of them.</p>
<div>
<h3>Here is how the package works</h3>
<div>
<h4>Installation</h4>
<p>Since the package is on cran, it’s very straight forward to install it:</p>
<pre><code><span>install.packages</span><span>(</span><span>&quot;postlightmercury&quot;</span><span>)</span></code></pre>
</div>
<div>
<h4>Load libraries</h4>
<p>We’ll need the <a rel="nofollow" href="https://cran.r-project.org/web/packages/postlightmercury/" target="_blank"><code>postlightmercury</code></a>, <a rel="nofollow" href="https://cran.r-project.org/web/packages/dplyr/" target="_blank"><code>dplyr</code></a> and <a rel="nofollow" href="https://cran.r-project.org/web/packages/stringr/" target="_blank"><code>stringr</code></a> libraries:</p>
<pre><code><span>library</span><span>(</span><span>postlightmercury</span><span>)</span>
<span>library</span><span>(</span><span>dplyr</span><span>)</span>
<span>library</span><span>(</span><span>stringr</span><span>)</span></code></pre>
</div>
<div>
<h4>Get an API key</h4>
<p>Before you can use the package you need to get an API key from Postlight. Get yours here: <a rel="nofollow" href="https://mercury.postlight.com/web-parser/" target="_blank">https://mercury.postlight.com/web-parser/</a> Replace the XXXX’s below with your new API key.</p>
</div>
<div>
<h4>Parse an URL</h4>
<p>We will use this extremely sad story from BBC that Gangnam Style is no longer the #1 most viewed video on YouTube. Sad to see a masterpiece like that get dethroned <img alt="🙁" src="https://i1.wp.com/s.w.org/images/core/emoji/2/72x72/1f641.png?w=456&ssl=1" class="wp-smiley" data-recalc-dims="1"></p>
<pre><code><span># Then run the code below replacing the X's wih your api key:</span>
<span>parsed_url</span> <span>&lt;-</span> <span>web_parser</span><span>(</span><span>page_urls</span> <span>=</span> <span>&quot;http://www.bbc.co.uk/news/entertainment-arts-40566816&quot;</span>, <span>api_key</span> <span>=</span> <span>XXXXXXXXXXXXXXXXXXXXXXX</span><span>)</span></code></pre>
<p>As you can see below the result is a tibble (data frame) with 14 different variables:</p>
<pre><code><span>glimpse</span><span>(</span><span>parsed_url</span><span>)</span></code></pre>
<pre><code>## Observations: 1
## Variables: 14
## $ title          </code></pre></div></div></div>