<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/what-have-these-birds-been-studied-for-querying-science-outputs-with-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://ropensci.org/blog/2018/09/11/birds-science/"> rOpenSci - open tools for open science</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>In the <a rel="nofollow" href="https://ropensci.org/blog/2018/08/21/birds-radolfzell/" target="_blank">second post of the series where we obtained data from<br>
eBird</a> we<br>
determined what birds were observed in the county of Constance, and we<br>
complemented this knowledge with some taxonomic and trait information in<br>
<a rel="nofollow" href="https://ropensci.org/blog/2018/09/04/birds-taxo-traits/" target="_blank">the fourth post of the<br>
series</a>. Now,<br>
we could be curious about the occurrence of these birds in <em>scientific<br>
work</em>. In this post, we will query the scientific literature and an open<br>
scientific data repository for species names: what have these birds been<br>
studied for? Read on if you want to learn how to use R packages allowing<br>
to do so!</p>
<h3>Getting a list of 50 species from occurrence data</h3>
<p>For more details about the following code, refer to the <a rel="nofollow" href="https://ropensci.org/blog/2018/08/21/birds-radolfzell/" target="_blank">previous post<br>
of the series</a>.<br>
The single difference is our adding a step to keep only data for the<br>
most recent years.</p>
<pre><code># polygon for filtering
landkreis_konstanz &lt;- osmdata::getbb(&quot;Landkreis Konstanz&quot;,
                             format_out = &quot;sf_polygon&quot;)
crs &lt;- sf::st_crs(landkreis_konstanz)

# get and filter data
f_out_ebd &lt;- &quot;ebird/ebd_lk_konstanz.txt&quot;

library(&quot;magrittr&quot;)

ebd &lt;- auk::read_ebd(f_out_ebd) %>%
  sf::st_as_sf(coords = c(&quot;longitude&quot;, &quot;latitude&quot;), 
                crs = crs) 

in_indices &lt;- sf::st_within(ebd, landkreis_konstanz)

ebd &lt;- dplyr::filter(ebd, lengths(in_indices) > 0)

ebd &lt;- as.data.frame(ebd)

ebd &lt;- dplyr::filter(ebd, approved, lubridate::year(observation_date) > 2010)
</code></pre>
<p>For the sake of simplicity, we shall only use the 50 species observed<br>
the most often.</p>
<pre><code>species &lt;- ebd %>%
  dplyr::count(common_name, sort = TRUE) %>%
  head(n = 50) %>%
  dplyr::pull(common_name)
</code></pre>
<p>The species are Carrion Crow, Eurasian Blackbird, Mallard, Eurasian<br>
Coot, Great Tit, Great Crested Grebe, Mute Swan, Great Cormorant,<br>
Eurasian Blue Tit, Gray Heron, Black-headed Gull, Common Chaffinch,<br>
Common Chiffchaff, Tufted Duck, European Starling, White Wagtail,<br>
European Robin, Little Grebe, Common Wood-Pigeon, Red-crested Pochard,<br>
Ruddy Shelduck, Graylag Goose, Red Kite, Common Buzzard, Eurasian<br>
Blackcap, Great Spotted Woodpecker, Eurasian Magpie, Gadwall, Common<br>
Pochard, Eurasian Nuthatch, Green-winged Teal, House Sparrow, Eurasian<br>
Jay, Yellow-legged Gull, Yellowhammer, Eurasian Green Woodpecker, Eared<br>
Grebe, Eurasian Reed Warbler, Barn Swallow, Northern Shoveler, Eurasian<br>
Moorhen, Black Redstart, Great Egret, White Stork, Eurasian Wren,<br>
Long-tailed Tit, Common House-Martin, Eurasian Kestrel, European<br>
Goldfinch and European Greenfinch<br>
<a rel="nofollow" href="https://twitter.com/LucyStats/status/1031938964796657665?s=19" target="_blank">(<code>glue::glue_collapse(species, sep = &quot;, &quot;, last = &quot; and &quot;)</code>)</a>.</p>
<h3>Querying the scientific literature</h3>
<p>Just like rOpenSci has a taxonomic toolbelt<br>
(<a rel="nofollow" href="https://github.com/ropensci/taxize" target="_blank"><code>taxize</code></a>) and a species<br>
occurrence data toolbelt (<a rel="nofollow" href="https://github.com/ropensci/spocc" target="_blank"><code>spocc</code></a>),<br>
it has a super package for querying the scientific literature:<br>
<a rel="nofollow" href="https://github.com/ropensci/fulltext" target="_blank"><code>fulltext</code></a>! This package<br>
supports search for “PLOS via the rplos package, Crossref via the<br>
rcrossref package, Entrez via the rentrez package, arXiv via the aRxiv<br>
package, and BMC, Biorxiv, EuroPMC, and Scopus via internal helper<br>
functions”.</p>
<p>We shall use <code>fulltext</code> to retrieve the titles and abstracts of<br>
scientific articles mentioning each species, and will use <code>tidytext</code> to<br>
compute the most prevalent words in these works.</p>
<p>We first define a function retrieving the titles and abstracts of works<br>
obtained as result when querying one species name.</p>
<p>We use <code>dplyr::bind_rows</code> because we want all results for one species at<br>
once, while <code>fulltext</code> returns a list of data.frames with one data.frame<br>
by data source.</p>
<pre><code>.get_papers &lt;- function(species){
  species %>%
    tolower() %>%
    fulltext::ft_search() %>%
    fulltext::ft_get() %>%
    fulltext::ft_collect() %>%
    fulltext::ft_chunks(c(&quot;title&quot;, &quot;abstract&quot;)) %>%
    fulltext::ft_tabularize() %>%
    dplyr::bind_rows()
}

.get_papers(species[1]) %>%
  dplyr::pull(title)
</code></pre>
<pre><code>##  [1] &quot;Great spotted cuckoo nestlings have no antipredatory effect on magpie or carrion crow host nests in southern Spain&quot;                                
##  [2] &quot;Donor-Control of Scavenging Food Webs at the Land-Ocean Interface&quot;                                                                                 
##  [3] &quot;Formal comment to Soler et al.: Great spotted cuckoo nestlings have no antipredatory effect on magpie or carrion crow host nests in southern Spain&quot;
##  [4] &quot;Socially Driven Consistent Behavioural Differences during Development in Common Ravens and Carrion Crows&quot;                                          
##  [5] &quot;Behavioral Responses to Inequity in Reward Distribution and Working Effort in Crows and Ravens&quot;                                                    
##  [6] &quot;Early Duplication of a Single MHC IIB Locus Prior to the Passerine Radiations&quot;                                                                     
##  [7] &quot;Investigating the impact of media on demand for wildlife: A case study of Harry Potter and the UK trade in owls&quot;                                   
##  [8] &quot;New Caledonian Crows Rapidly Solve a Collaborative Problem without Cooperative Cognition&quot;                                                          
##  [9] &quot;Nest Predation Deviates from Nest Predator Abundance in an Ecologically Trapped Bird&quot;                                                              
## [10] &quot;Dietary Compositions and Their Seasonal Shifts in Japanese Resident Birds, Estimated from the Analysis of Volunteer Monitoring Data&quot;
</code></pre>
<p>If we were working on a scientific study, we’d add a few more filters,<br>
e.g. having the species mentioned in the abstract, and not only<br>
somewhere in the paper which is probably the way the different<br>
literature search providers define a match. But we’re not, so we can<br>
keep our query quite free! My favourite paper involving the Carrion Crow<br>
is <a rel="nofollow" href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0182368" target="_blank">“Investigating the impact of media on demand for wildlife: A case<br>
study of Harry Potter and the UK trade in<br>
owls”</a><br>
because it’s a fun and important scientific question, and is supported<br>
by open data (by the way you can access CITES trade data (international<br>
trade in endangered species) in R using<br>
<a rel="nofollow" href="https://github.com/ecohealthalliance/cites/" target="_blank"><code>cites</code></a> and CITES<br>
Speciesplus database using<br>
<a rel="nofollow" href="https://ibartomeus.github.io/rcites/" target="_blank"><code>rcites</code></a>).</p>
<p>We then apply this function to all 50 species and keep each article only<br>
once.</p>
<pre><code>get_papers &lt;- ratelimitr::limit_rate(.get_papers,
                                     rate = ratelimitr::rate(1, 2))

all_papers &lt;- purrr::map_df(species, get_papers)

nrow(all_papers)
</code></pre>
<pre><code>## [1] 522
</code></pre>
<pre><code>all_papers &lt;- unique(all_papers)

nrow(all_papers)
</code></pre>
<pre><code>## [1] 378
</code></pre>
<p>Now, we get the most common words from titles and abstracts. For that we<br>
first append the title to the abstract which is a quick hack.</p>
<pre><code>library(&quot;tidytext&quot;)
library(&quot;rcorpora&quot;)

stopwords &lt;- corpora(&quot;words/stopwords/en&quot;)$stopWords

all_papers %>%
  dplyr::group_by(title, abstract) %>%
  dplyr::summarise(text = paste(title, abstract)) %>%
  dplyr::ungroup() %>%
  unnest_tokens(word, text) %>%
  dplyr::filter(!word %in% stopwords) %>%
  dplyr::count(word, sort = TRUE) -> words
</code></pre>
<p>So, what are the most common words in these papers?</p>
<pre><code>head(words, n = 10) 
</code></pre>
<pre><code>##           word   n
## 1      species 754
## 2        birds 514
## 3        virus 270
## 4        avian 268
## 5         bird 262
## 6        study 243
## 7     breeding 231
## 8         wild 227
## 9  populations 217
## 10  population 213
</code></pre>
<p>Not too surprising, and obviously less entertaining than looking at<br>
individual species’ results. Maybe a wordcloud can give us a better idea<br>
of the wide area of topics of studies involving our 50 most frequent<br>
bird species. We use the <a rel="nofollow" href="https://cran.r-project.org/web/packages/wordcloud/index.html" target="_blank"><code>wordcloud</code><br>
package</a>.</p>
<pre><code>library(&quot;wordcloud&quot;)

with(words, wordcloud(word, n, max.words = 100))
</code></pre>
<p><img alt="wordcloud of titles and abstracts of scientific
papers" src="https://i1.wp.com/ropensci.org/img/blog-images/2018-09-11-birds-science/wordcloud-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>We see that topics include ecological words such as “foraging” but also<br>
epidemiological questions since “influenza” and “h5n1” come up. Now, how<br>
informative as this wordcloud can be, it’s a bit ugly, so we’ll prettify<br>
it using the <a rel="nofollow" href="https://github.com/Lchiffon/wordcloud2" target="_blank"><code>wordcloud2</code><br>
package</a> instead, and the<br>
silhouette of a bird <a rel="nofollow" href="http://phylopic.org/image/6209c9be-060e-4d7f-bc74-a75f3ccf4629/" target="_blank">from<br>
Phylopic</a>.</p>
<pre><code>bird &lt;- words %>%
  head(n = 100) %>%
  wordcloud2::wordcloud2(figPath = &quot;bird.png&quot;, 
                       color = &quot;black&quot;, size = 1.5)
# https://www.r-graph-gallery.com/196-the-wordcloud2-library/
htmlwidgets::saveWidget(bird,
                        &quot;tmp.html&quot;,
                        selfcontained = F)
</code></pre>
<p>I wasn’t able to <code>webshot</code> the resulting html despite increasing the<br>
<code>delay</code> parameter so I screenshot it by hand!</p>
<pre><code>magick::image_read(&quot;screenshot.png&quot;)
</code></pre>
<p><img alt="wordcloud shaped as a bird" src="https://i0.wp.com/ropensci.org/img/blog-images/2018-09-11-birds-science/wordcloud2-1.png?w=450&ssl=1" data-recalc-dims="1"></p>
<p>
wordcloud shaped as a bird
</p>

<p>The result is a bit kitsch, doesn’t include the word “species”, one<br>
needs to know it’s the silhouette of a bird to recognize it, and we’d<br>
need to work a bit on not reshaping the silhouette, but it’s fun as it<br>
is.</p>
<h3>Querying scientific open data</h3>
<p>There are quite a few scientific open data repositories out there, among<br>
which the giant <a rel="nofollow" href="https://www.dataone.org/" target="_blank">DataONE</a> that has an API<br>
interfaced with an R package. We shall use it to perform a search<br>
similar to the previous section, but looking at the data indexed on<br>
DataONE. Since DataONE specializes in ecological and environmental data,<br>
we expect to find rather ecological data.</p>
<p>We first define a function to retrieve metadata of datasets for one<br>
species. It looks the species names in the abstract.</p>
<pre><code>.get_meta &lt;- function(species){
  
  cn &lt;- dataone::CNode(&quot;PROD&quot;)
  search &lt;- list(q = glue::glue(&quot;abstract:{species}&quot;),
                        fl = &quot;id,title,abstract&quot;,
                        sort = &quot;dateUploaded+desc&quot;)
  
  result &lt;- dataone::query(cn, solrQuery = search,
                           as=&quot;data.frame&quot;)
  
  if(nrow(result) == 0){
    NULL
  }else{
    # otherwise one line by version
  result &lt;- unique(result)
  
  tibble::tibble(species = species,
                 title = result$title,
                 abstract = result$abstract)
  }
}
</code></pre>
<p>Note that DataONE searching could be more precise: one can choose to<br>
search from a given data source only for instance. See the <a rel="nofollow" href="https://github.com/DataONEorg/rdataone/blob/master/vignettes/searching-dataone.Rmd" target="_blank">searching<br>
DataONE<br>
vignette</a>.</p>
<pre><code>get_meta &lt;- ratelimitr::limit_rate(.get_meta,
                                     rate = ratelimitr::rate(1, 2))

all_meta &lt;- purrr::map_df(species, get_meta)

nrow(all_meta)
</code></pre>
<pre><code>## [1] 266
</code></pre>
<pre><code>length(unique(all_meta$species))
</code></pre>
<pre><code>## [1] 35
</code></pre>
<p>35 species are represented.</p>
<pre><code>all_meta &lt;- unique(all_meta[,c(&quot;title&quot;, &quot;abstract&quot;)])

nrow(all_meta)
</code></pre>
<pre><code>## [1] 104
</code></pre>
<p>We then extract the most common words.</p>
<pre><code>all_meta %>%
  dplyr::group_by(title, abstract) %>%
  dplyr::summarise(text = paste(title, abstract)) %>%
  dplyr::ungroup() %>%
  unnest_tokens(word, text) %>%
  dplyr::filter(!word %in% stopwords) %>%
  dplyr::count(word, sort = TRUE) -> data_words

head(data_words, n = 10)
</code></pre>
<pre><code>## # A tibble: 10 x 2
##    word           n
##    </code></pre>