<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/oreilly-animals-in-trouble-conservation-status-of-book-covers/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://masalmon.eu/2018/08/25/oreilly/"> Posts on MaÃ«lle's R blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>What can a kaka, a kakapo, an European rabbit and a grey heron have in<br>
common? Well, they might co-habit in the bookshelf of an R user, since<br>
they're all animals on the covers of popular R books: <a rel="nofollow" href="http://r-pkgs.had.co.nz/" target="_blank">&quot;R<br>
Packagesâ€�</a>, <a rel="nofollow" href="http://r4ds.had.co.nz/" target="_blank">&quot;R for Data<br>
Scienceâ€�</a>, <a rel="nofollow" href="https://www.tidytextmining.com/" target="_blank">&quot;Text mining with<br>
Râ€�</a> and <a rel="nofollow" href="https://csgillespie.github.io/efficientR/" target="_blank">&quot;Efficient R<br>
programmingâ€�</a>, respectively.<br>
Their publisher, O'Reilly, has now based its brand on covers featuring<br>
beautiful gravures of animals.</p>
<p>Recently, while wondering what the name of R for Data Science bird was<br>
again (I thought it was a kea!), I was thrilled to find <a rel="nofollow" href="https://www.oreilly.com/animals.csp" target="_blank">the whole<br>
O'Reilly menagerie</a>, i.e.Â a list of<br>
books and corresponding animals! The website also features a link to <a rel="nofollow" href="https://www.oreilly.com/ideas/a-short-history-of-the-oreilly-animals" target="_blank">&quot;A<br>
short history of the O'Reilly<br>
animalsâ€�</a><br>
that was an amazing read. In it was noted that &quot;The animals are in<br>
trouble.â€�, with a few examples of endangered species. It inspired me to<br>
actually try and assess the conservation status of O'Reilly animals<br>
using responsible webscraping, taxonomic name resolving and IUCN Redlist<br>
API queryingâ€¦</p>

<h1>Scraping the menagerie: an utter delight!</h1>
<p>I had a great time webscraping the menagerie, not only thanks to my now<br>
reasonable experience doing such things, but also thanks to</p>
<ul>
<li>
<p>my using the <a rel="nofollow" href="https://github.com/dmi3kno/polite" target="_blank">wonderful <code>polite</code> package for<br>
webscraping</a>, that makes me feel<br>
so good about myself. Read more about this package <a rel="nofollow" href="https://masalmon.eu/2018/07/31/alldatascience/" target="_blank">in my previous<br>
post</a>.</p>
</li>
<li>
<p>the webpage having really good structured html with specific<br>
classes.</p>
</li>
</ul>
<p>The menagerie is divided into pages of 20 books, so I mapped over all<br>
possible offsets up to the number of animals indicated on the website,<br>
1227.</p>
<pre><code>library(&quot;magrittr&quot;)
home_url &lt;- &quot;https://www.oreilly.com/animals.csp&quot;

session &lt;- polite::bow(home_url,
                       user_agent = &quot;MaÃ«lle Salmon https://masalmon.eu/&quot;)

get_twenty &lt;- function(offset, session){
  # offset parameter to get all books 20 by 20
  params &lt;- glue::glue(&quot;?x-o={offset}&quot;)
  
  # scraping with content parameter
  # cf https://github.com/dmi3kno/polite/issues/6
  # https://www.oreilly.com/animals.csp?x-o=720 was problematic
  # (German characters)
  page &lt;- polite::scrape(session, params = params, 
                         content = &quot;text/html;charset=iso-8859-1&quot;)
  # get all animal rows
  rows &lt;- rvest::xml_nodes(page,
                           xpath = &quot;//div[@class='animal-row']&quot;)
  # extract book titles
  rows %>%
    rvest::xml_nodes(xpath = &quot;a[@class='book']&quot;) %>%
    rvest::xml_nodes(xpath = &quot;h1[@class='book-title']&quot;) %>%
    rvest::html_text() -> book_titles
  
  rows %>%
    rvest::xml_nodes(xpath = &quot;h2[@class='animal-name']&quot;) %>%
    rvest::html_text() -> animal_names
  
  tibble::tibble(book = book_titles,
                 animal = animal_names)
  
}

no_animals &lt;- 1227 # by hand!

offsets &lt;- (0:floor(no_animals/20))*20

purrr::map_df(offsets, get_twenty, session = session) %>%
  readr::write_csv(&quot;oreilly_animals.csv&quot;)
  
</code></pre>
<p>I got 1134 rows, each corresponding to a book, with animals potentially<br>
repeated.</p>
<pre><code>animals
</code></pre>
<pre><code>## # A tibble: 1,134 x 2
##    book                            animal                                 
##    </code></pre>