<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/pubchunks-extract-parts-of-scholarly-xml-articles/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://ropensci.org/technotes/2018/10/16/pubchunks/"> rOpenSci - open tools for open science</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a rel="nofollow" href="https://github.com/ropensci/pubchunks" target="_blank">pubchunks</a> is a package grown out of the <a rel="nofollow" href="https://github.com/ropensci/fulltext" target="_blank">fulltext</a> package. <code>fulltext</code><br>
provides a single interface to many sources of full text scholarly articles. As<br>
part of the user flow in <code>fulltext</code> there is an extraction step where <code>fulltext::chunks()</code><br>
pulls parts of articles out of XML format article files.</p>
<p>As part of making <code>fulltext</code> more maintainable and focused on simply fetching articles,<br>
and realizing that pulling out bits of structured XML files is a more general problem,<br>
we broke out <code>pubchunks</code> into a separate package. <code>fulltext::ft_chunks()</code> and<br>
<code>fulltext::ft_tabularize()</code> will eventually be removed and we’ll point users to<br>
<code>pubchunks</code>.</p>
<p>The goal of <code>pubchunks</code> is to fetch sections out of XML format scholarly articles.<br>
Users do not need to know about XML and all of its warts. They only need to know<br>
where their files or XML strings are and what sections they want of<br>
each article. Then the user can combine these sections and do whatever they wish<br>
downstream; for example, analysis of the text structure or a meta-analysis combining<br>
p-values or other data.</p>
<p>The other major format, and more common format, that articles come in is PDF. However,<br>
PDF has no structure other than perhaps separate pages, so it’s not really possible<br>
to easily extract specific sections of an article. Some publishers provide absolutely<br>
no XML versions (cough, Wiley) while others that do a good job of this are almost<br>
entirely paywalled (cough, Elsevier). There are some open access publishers that<br>
do provide XML (PLOS, Pensoft, Hindawi) – so you have the best of both worlds with<br>
those publishers.</p>
<p><code>pubchunks</code> is still in early days of development, so we’d love any feedback.</p>
<ul>
<li>pubchunks on GitHub: <a rel="nofollow" href="https://github.com/ropensci/pubchunks" target="_blank">https://github.com/ropensci/pubchunks</a></li>
<li>pubchunks on CRAN: <a rel="nofollow" href="https://cran.rstudio.com/web/packages/pubchunks/" target="_blank">https://cran.rstudio.com/web/packages/pubchunks/</a></li>
</ul>
<h2>Functions in pubchunks</h2>
<p>All exported functions are prefixed with <code>pub</code> to help reduce namespace conflicts.</p>
<p>The two main functions are:</p>
<ul>
<li><code>pub_chunks()</code>: fetch XML sections</li>
<li><code>pub_tabularize()</code>: coerce output of <code>pub_chunks()</code> into data.frame’s</li>
</ul>
<p>Other functions that you may run in to:</p>
<ul>
<li><code>pub_guess_publisher()</code>: guess publisher from XML file or character string</li>
<li><code>pub_sections()</code>: sections <code>pubchunks</code> knows how to handle</li>
<li><code>pub_providers()</code>: providers (i.e., publishers) <code>pubchunks</code> knows how to handle explicitly</li>
</ul>
<p></p>
<h2>How it works</h2>
<p>When using <code>pub_chunks()</code> we first figure out what publisher the XML comes from. We do this<br>
beacause journals from the same publisher often/usually follow the same format/structure, so<br>
we can be relatively confident of rules for pulling out certain sections.</p>
<p>Once we have the publisher, we go through each section of the article the user reqeusts, and<br>
use the publisher specific <a rel="nofollow" href="https://en.wikipedia.org/wiki/XPath" target="_blank">XPATH</a> (an XML query language)<br>
that we’ve written to extract that section from the XML.</p>
<p>The ouput is a named list, where names are the sections; and the output is an S3 class with<br>
a print method to make it more easily digestable.</p>
<p></p>
<h2>Installation</h2>
<p>The first version of the package has just hit CRAN, so not all binaries are available as of this<br>
date.</p>
<pre><code>install.packages(&quot;pubchunks&quot;)
</code></pre>
<p>You may have to install the dev version</p>
<pre><code>remotes::install_github(&quot;ropensci/pubchunks&quot;)
</code></pre>
<p>After sending to CRAN a few days back I noticed a number of things that needed fixing/improving,<br>
so you can also try out those fixes:</p>
<pre><code>remotes::install_github(&quot;ropensci/pubchunks@fix-pub_chunks&quot;)
</code></pre>
<p>Load the package</p>
<pre><code>library(pubchunks)
</code></pre>
<p></p>
<h2>Pub chunks</h2>
<p><code>pub_chunks</code> accepts a file path, XML as character string, <code>xml_document</code> object, or a list of<br>
those three things.</p>
<p>We currently support the following publishers, though not all sections below are allowed<br>
in every publisher:</p>
<ul>
<li>elife</li>
<li>plos</li>
<li>elsevier</li>
<li>hindawi</li>
<li>pensoft</li>
<li>peerj</li>
<li>copernicus</li>
<li>frontiers</li>
<li>f1000research</li>
</ul>
<p>We currently support the following sections, not all of which are supported, or make sense, for<br>
each publisher:</p>
<ul>
<li>front – Publisher, journal and article metadata elements</li>
<li>body – Body of the article</li>
<li>back – Back of the article, acknowledgments, author contributions, references</li>
<li>title – Article title</li>
<li>doi – Article DOI</li>
<li>categories – Publisher’s categories, if any</li>
<li>authors – Authors</li>
<li>aff – Affiliation (includes author names)</li>
<li>keywords – Keywords</li>
<li>abstract – Article abstract</li>
<li>executive_summary – Article executive summary</li>
<li>refs – References</li>
<li>refs_dois – References DOIs – if available</li>
<li>publisher – Publisher name</li>
<li>journal_meta – Journal metadata</li>
<li>article_meta – Article metadata</li>
<li>acknowledgments – Acknowledgments</li>
<li>permissions – Article permissions</li>
<li>history – Dates, recieved, published, accepted, etc.</li>
</ul>
<p>Get an example XML file from the package</p>
<pre><code>x &lt;- system.file(&quot;examples/pensoft_1.xml&quot;, package = &quot;pubchunks&quot;)
</code></pre>
<p>Pass to <code>pub_chunks</code> and state which section(s) you want</p>
<pre><code>pub_chunks(x, sections = &quot;abstract&quot;)
#> </code></pre>