<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/r%e2%81%b6-scraping-images-to-pdfs/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rud.is/b/2017/06/05/r%E2%81%B6-scraping-images-to-pdfs/"> R – rud.is</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I’ve been doing intermittent prep work for a follow-up to an earlier <a rel="nofollow" href="https://rud.is/b/2017/03/19/exploring-2017-retail-store-closings-with-r/" target="_blank">post on store closings</a> and came across <a rel="nofollow" href="http://money.cnn.com/interactive/technology/radio-shack-closure-list/index.html" target="_blank">this CNN Money</a> “article” on it. Said “article” is a deliberately obfuscated or lazily crafted series of GIF images that contain all the Radio Shack impending store closings. It’s the most comprehensive list I’ve found, but the format is terrible and there’s no easy, in-browser way to download them all.</p>
<p>CNN has ToS that prevent automated data gathering from CNN-proper. But, they used Adobe Document Cloud for these images which has no similar restrictions from a quick glance at their ToS. That means <em>you</em> get an R⁶ post on how to grab the individual 38 images and combine them into one PDF. I did this all with the hopes of OCRing the text, which has not panned out too well since the image quality and font was likely deliberately set to make it hard to do precisely what I’m trying to do.</p>
<p>If you work through the example, you’ll get a feel for:</p>
<ul>
<li>using <code>sprintf()</code> to take a template and build a vector of URLs</li>
<li>use <code>dplyr</code> progress bars</li>
<li>customize <code>httr</code> verb options to ensure you can get to content</li>
<li>use <code>purrr</code> to iterate through a process of turning raw image bytes into image content (via <code>magick</code>) and turn a list of images into a PDF</li>
</ul>
<pre><code>library(httr)
library(magick)
library(tidyverse)

url_template &lt;- &quot;https://assets.documentcloud.org/documents/1657793/pages/radioshack-convert-p%s-large.gif&quot;

pb &lt;- progress_estimated(38)

sprintf(url_template, 1:38) %>% 
  map(~{
    pb$tick()$print()
    GET(url = .x, 
        add_headers(
          accept = &quot;image/webp,image/apng,image/*,*/*;q=0.8&quot;, 
          referer = &quot;http://money.cnn.com/interactive/technology/radio-shack-closure-list/index.html&quot;, 
          authority = &quot;assets.documentcloud.org&quot;))    
  }) -> store_list_pages

map(store_list_pages, content) %>% 
  map(image_read) %>% 
  reduce(image_join) %>% 
  image_write(&quot;combined_pages.pdf&quot;, format = &quot;pdf&quot;)</code></pre>
<p>I figured out the Document Cloud links and necessary <code>httr::GET()</code> options by using Chrome Developer Tools and my <a rel="nofollow" href="https://github.com/hrbrmstr/curlconverter" target="_blank"><code>curlconverter</code></a> package.</p>
<p>If any academic-y folks have a <strike>test subject</strike>summer intern with a free hour and would be willing to have them transcribe this list and stick it on GitHub, you’d have my eternal thanks.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://rud.is/b/2017/06/05/r%E2%81%B6-scraping-images-to-pdfs/"> R – rud.is</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...

	<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/ENfyX3FzIOA" height="1"></a></div>