<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/package-update-longurl-0-3-0-is-hitting-cran-mirrors/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rud.is/b/2016/12/18/package-update-longurl-0-3-0-is-hitting-cran-mirrors/"> R – rud.is</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>The <a target="_blank" rel="nofollow" href="https://github.com/hrbrmstr/longurl"><code>longurl</code></a> package has been updated to version 0.3.0 as a result of a bug report noting that the URL expansion API it was using went pay-for-use. Since this was the second time a short URL expansion service either went belly-up or had breaking changes the package is now completely client-side-based and a very thin, highly-focused wrapper around the <code>httr::HEAD()</code> function.</p>
<h3>Why <code>longurl</code>?</h3>
<p>On the D&amp;D alignment scale, short links are <a target="_blank" rel="nofollow" href="https://en.wikipedia.org/wiki/Alignment_(Dungeons_%26_Dragons)#Chaotic_evil">chaotic evil</a>. [<em>Full-disclosure: I use shortened links all the time, so the pot is definitely kettle-calling here</em>]. Ostensibly, they are for making it easier to show memorable links on tiny, glowing rectangles or printed prose but they are mostly used to directly track you and mask other tracking parameters that the target site is using to keep tabs on you. Furthermore, short URLs are also used by those with even more malicious intent than greedy startups or mega-corporations. </p>
<p>In retrospect, giving a third-party API service access to URLs you are interested in expanding just exacerbated the tracking problem, but many of these third-party URL expansion services do use some temporal caching of results, so they can be a bit faster than doing this in a non-caching package (but, there’s nothing stopping you putting caching code around it if you are using it in a “production” capacity).</p>
<h3>How does the updated package work without a URL expansion API?</h3>
<p>By default, <code>httr</code> “verb” requests use the <code>curl</code> package and that is a wrapper for <code>libcurl</code>. The <code>httr</code> verb calls set the <em>“please follow all HTTP status 3xx redirects that are found in responses”</em> option (this is the <code>libcurl</code> <code>CURLOPT_FOLLOWLOCATION</code> equivalent option). There are other options that can be set to help configure minutae around how redirect following works. So, just by calling <code>httr::HEAD(some_url)</code> you get built-in short URL expansion (if what you passed in was a short URL or a URL with a redirect). </p>
<p>Take, for example, this innocent link: <code>http://lnk.direct/zFu</code>. We can see what goes on under the covers by passing in the <code>verbose()</code> option to an <code>httr::HEAD()</code> call:</p>
<pre><code>httr::HEAD(&quot;http://lnk.direct/zFu&quot;, verbose())

## -> HEAD /zFu HTTP/1.1
## -> Host: lnk.direct
## -> User-Agent: libcurl/7.51.0 r-curl/2.3 httr/1.2.1
## -> Accept-Encoding: gzip, deflate
## -> Cookie: shorturl=4e0aql3p49rat1c8kqcrmv4gn2
## -> Accept: application/json, text/xml, application/xml, */*
## -> 
## &lt;- HTTP/1.1 301 Moved Permanently
## &lt;- Server: nginx/1.0.15
## &lt;- Date: Sun, 18 Dec 2016 19:03:48 GMT
## &lt;- Content-Type: text/html; charset=UTF-8
## &lt;- Connection: keep-alive
## &lt;- X-Powered-By: PHP/5.6.20
## &lt;- Expires: Thu, 19 Nov 1981 08:52:00 GMT
## &lt;- Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
## &lt;- Pragma: no-cache
## &lt;- Location: http://ow.ly/Ko70307eKmI
## &lt;- 
## -> HEAD /Ko70307eKmI HTTP/1.1
## -> Host: ow.ly
## -> User-Agent: libcurl/7.51.0 r-curl/2.3 httr/1.2.1
## -> Accept-Encoding: gzip, deflate
## -> Accept: application/json, text/xml, application/xml, */*
## -> 
## &lt;- HTTP/1.1 301 Moved Permanently
## &lt;- Content-Length: 0
## &lt;- Location: http://bit.ly/2gZq7qG
## &lt;- Connection: close
## &lt;- 
## -> HEAD /2gZq7qG HTTP/1.1
## -> Host: bit.ly
## -> User-Agent: libcurl/7.51.0 r-curl/2.3 httr/1.2.1
## -> Accept-Encoding: gzip, deflate
## -> Accept: application/json, text/xml, application/xml, */*
## -> 
## &lt;- HTTP/1.1 301 Moved Permanently
## &lt;- Server: nginx
## &lt;- Date: Sun, 18 Dec 2016 19:04:36 GMT
## &lt;- Content-Type: text/html; charset=utf-8
## &lt;- Content-Length: 127
## &lt;- Connection: keep-alive
## &lt;- Cache-Control: private, max-age=90
## &lt;- Location: http://example.com/IT_IS_A_SURPRISE
## &lt;- 
## -> HEAD /IT_IS_A_SURPRISE HTTP/1.1
## -> Host: example.com
## -> User-Agent: libcurl/7.51.0 r-curl/2.3 httr/1.2.1
## -> Accept-Encoding: gzip, deflate
## -> Cookie: _csrf/link=g3iBgezgD_OYN0vOh8yI930E1O9ZAKLr4uHmVioxwwQ; mc=null; dmvk=5856d9e39e747; ts=475630; v1st=03AE3C5AD67E224DEA304AEB56361C9F
## -> Accept: application/json, text/xml, application/xml, */*
## -> 
## &lt;- HTTP/1.1 200 OK
## ...
## &lt;- </code></pre>
<p>We can reduce the clutter and see that it follows multiple redirects from multiple URL shorteners:</p>
<p><center><img src="https://rud.is/dl/HEADs.svg"></center></p>
<p>Here’s what the output of a request to <code>longurl::expand_urls()</code> returns:</p>
<pre><code>longurl::expand_urls(&quot;http://lnk.direct/zFu&quot;)
## # A tibble: 1 × 3
##                orig_url                        expanded_url status_code
##                   &lt;chr>                               &lt;chr>       &lt;int>
## 1 http://lnk.direct/zFu http://example.com/IT_IS_A_SURPRISE         200</code></pre>
<p><em>NOTE: the link does actually go <strong>somewhere</strong>, and somewhere not malicious, political or preachy (a rarity in general in this post-POTUS-election world of ours).</em></p>
<h3>What else is different?</h3>
<p>The <code>longurl::expand_urls()</code> function returns a <code>tbl_df</code> and now includes the HTTP status code of the final, resolved link. You can also pass in a custom HTTP referrer since many (<em>many</em>) sites will change behavior depending on the referrer.</p>
<h3>What’s next?</h3>
<p>This bug-fix release had to go out fairly quickly since the package was essentially broken. With the new foundation being built on client-side machinations, future enhancements will be to pull more features (in the machine learning sense) out of the <code>curl</code> or <code>httr</code> requests (I may switch directly to using <code>curl</code> if I need more granular data) and include some basic visualizations for both request trees (mostly likely using the <code>DiagrammeR</code> and <code>ggplot2</code> packages). I may try to add a caching layer, but I believe that’s more of a situation-specific feature folks should add on their own, so I may just add a “check hook” capability that will add an extra function call to a cache checking function of your choosing.</p>
<p>If you have a feature request, please add it to the <a target="_blank" rel="nofollow" href="https://github.com/hrbrmstr/longurl">github repo</a>. </p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://rud.is/b/2016/12/18/package-update-longurl-0-3-0-is-hitting-cran-mirrors/"> R – rud.is</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/Jx7AUIXO6N4"></a></div>