<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/automatically-importing-publications-from-bibtex-to-a-hugo-academic-blog/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://lbusettspatialr.blogspot.com/2018/03/automatically-importing-publications.html"> Spatial Processing in R</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<h2>The problem</h2>
<div>I am in the process of migrating my (rather ugly) <a rel="nofollow" href="https://www.blogger.com/blogger.g?blogID=4683863749820351779#allposts" target="_blank">small blog</a> from “Bloggers” to `blogdown` and, as several others, I choose to use the <a rel="nofollow" href="https://github.com/gcushen/hugo-academic" target="_blank">hugo-academic theme</a> due to its good looks, simplicity, and “focus” towards researchers.</div>
<div>One nice feature of <code>hugo-academic</code> is that it includes out-of-the-box a “Publications” section, allowing researchers to easily create a list of their publication as a section of the website.</div>
<div><span>Unfortunately</span>, in order to populate that list, users have to manually create one different <code>.md</code> file for each publication, by cutting and pasting several different info (e.g., title, authors, etc.) in a “simple” template like <a rel="nofollow" href="https://github.com/gcushen/hugo-academic/blob/master/archetypes/publication.md" target="_blank">this one</a>.</div>
<div><span>THIS IS BORING!</span><br><span><br></span><span></span><br><a name="more"></a></div>
<div>Since I was not in the mood of doing that, and no automatic solutions could be found (well, there appears to be a <code>python</code> one, but we are speaking <code>R</code>, here…), I decided to try and develop some script to automatically create the required <code>md</code> files starting from a <code>BibTex</code> list of my publications. Here are the results of that effort.</div>
</div>
<div>
<h2>A possible solution</h2>
<div>
<h3>Preparing the <code>BibTex</code> file</h3>
<div>To automatically create the publications <code>md</code> files, all you need is a (properly formatted) <code>BibTex</code> file. Since I did not have one ready, I created mine by exporting my publications list from <a rel="nofollow" href="https://www.scopus.com/authid/detail.uri?authorId=23003461400" target="_blank">Scopus</a>, but you could use any valid BibTeX file.</div>
<div><span>One important thing, though, is that you have to be sure that the file is saved with UTF-8 encoding</span>. If you are not sure, you can open it in RStudio (or any decent text editor), and then re-save it with a forced encoding (in RStudio, you can use <code>File-->Save with Encoding</code>)</div>
</div>
<div>
<h3>Creating an import script</h3>
<div>Now, you need a script that reads the <code>BibTex</code> entries and use the data to populate one different <code>md</code>file for each publication. Below you can find my attempt at this.</div>
<pre><code><span>bibtex_2academic &lt;-<span>function</span>(bibfile,<br> outfold,<br> abstract = <span>FALSE</span>, <br> overwrite = <span>FALSE</span>) {<br> <br> <span>require</span>(RefManageR)<br> <span>require</span>(dplyr)<br> <span>require</span>(stringr)<br> <span>require</span>(anytime)<br> <br> <span># Import the bibtex file and convert to data.frame</span><br> mypubs &lt;-ReadBib(bibfile, check = <span>&quot;warn&quot;</span>, .Encoding = <span>&quot;UTF-8&quot;</span>) %>%<br> as.data.frame()<br> <br> <span># assign &quot;categories&quot; to the different types of publications</span><br> mypubs &lt;-mypubs %>%<br> dplyr::mutate(<br> pubtype = dplyr::case_when(document_type == <span>&quot;Article&quot;</span> ~ <span>&quot;2&quot;</span>,<br> document_type == <span>&quot;Article in Press&quot;</span> ~ <span>&quot;2&quot;</span>,<br> document_type == <span>&quot;InProceedings&quot;</span> ~ <span>&quot;1&quot;</span>,<br> document_type == <span>&quot;Proceedings&quot;</span> ~ <span>&quot;1&quot;</span>,<br> document_type == <span>&quot;Conference&quot;</span> ~ <span>&quot;1&quot;</span>,<br> document_type == <span>&quot;Conference Paper&quot;</span> ~ <span>&quot;1&quot;</span>,<br> document_type == <span>&quot;MastersThesis&quot;</span> ~ <span>&quot;3&quot;</span>,<br> document_type == <span>&quot;PhdThesis&quot;</span> ~ <span>&quot;3&quot;</span>,<br> document_type == <span>&quot;Manual&quot;</span> ~ <span>&quot;4&quot;</span>,<br> document_type == <span>&quot;TechReport&quot;</span> ~ <span>&quot;4&quot;</span>,<br> document_type == <span>&quot;Book&quot;</span> ~ <span>&quot;5&quot;</span>,<br> document_type == <span>&quot;InCollection&quot;</span> ~ <span>&quot;6&quot;</span>,<br> document_type == <span>&quot;InBook&quot;</span> ~ <span>&quot;6&quot;</span>,<br> document_type == <span>&quot;Misc&quot;</span> ~ <span>&quot;0&quot;</span>,<br> <span>TRUE</span> ~ <span>&quot;0&quot;</span>))<br> <br> <span># create a function which populates the md template based on the info</span><br> <span># about a publication</span><br> create_md &lt;-<span>function</span>(x) {<br> <br> <span># define a date and create filename by appending date and start of title</span><br> <span>if</span> (!is.na(x[[<span>&quot;year&quot;</span>]])) {<br> x[[<span>&quot;date&quot;</span>]] &lt;-paste0(x[[<span>&quot;year&quot;</span>]], <span>&quot;-01-01&quot;</span>)<br> } <span>else</span> {<br> x[[<span>&quot;date&quot;</span>]] &lt;-<span>&quot;2999-01-01&quot;</span><br> }<br> <br> filename &lt;-paste(x[[<span>&quot;date&quot;</span>]], x[[<span>&quot;title&quot;</span>]] %>%<br> str_replace_all(fixed(<span>&quot; &quot;</span>), <span>&quot;_&quot;</span>) %>%<br> str_remove_all(fixed(<span>&quot;:&quot;</span>)) %>%<br> str_sub(<span>1</span>, <span>20</span>) %>%<br> paste0(<span>&quot;.md&quot;</span>), sep = <span>&quot;_&quot;</span>)<br> <span># start writing</span><br> <span>if</span> (!file.exists(file.path(outfold, filename)) | overwrite) {<br> fileConn &lt;-file.path(outfold, filename)<br> write(<span>&quot;+++&quot;</span>, fileConn)<br> <br> <span># Title and date</span><br> write(paste0(<span>&quot;title = \&quot;&quot;</span>, x[[<span>&quot;title&quot;</span>]], <span>&quot;\&quot;&quot;</span>), fileConn, append = <span>T</span>)<br> write(paste0(<span>&quot;date = \&quot;&quot;</span>, anydate(x[[<span>&quot;date&quot;</span>]]), <span>&quot;\&quot;&quot;</span>), fileConn, append = <span>T</span>)<br> <br> <span># Authors. Comma separated list, e.g. `[&quot;Bob Smith&quot;, &quot;David Jones&quot;]`.</span><br> auth_hugo &lt;-str_replace_all(x[<span>&quot;author&quot;</span>], <span>&quot; and &quot;</span>, <span>&quot;\&quot;, \&quot;&quot;</span>)<br> auth_hugo &lt;-stringi::stri_trans_general(auth_hugo, <span>&quot;latin-ascii&quot;</span>)<br> write(paste0(<span>&quot;authors = [\&quot;&quot;</span>, auth_hugo,<span>&quot;\&quot;]&quot;</span>), fileConn, append = <span>T</span>)<br> <br> <span># Publication type. Legend:</span><br> <span># 0 = Uncategorized, 1 = Conference paper, 2 = Journal article</span><br> <span># 3 = Manuscript, 4 = Report, 5 = Book, 6 = Book section</span><br> write(paste0(<span>&quot;publication_types = [\&quot;&quot;</span>, x[[<span>&quot;pubtype&quot;</span>]],<span>&quot;\&quot;]&quot;</span>), <br> fileConn, append = <span>T</span>)<br> <br> <span># Publication details: journal, volume, issue, page numbers and doi link</span><br> publication &lt;-x[[<span>&quot;journal&quot;</span>]]<br> <span>if</span> (!is.na(x[[<span>&quot;volume&quot;</span>]])) publication &lt;-paste0(publication, <br> <span>&quot;, (&quot;</span>, x[[<span>&quot;volume&quot;</span>]], <span>&quot;)&quot;</span>)<br> <span>if</span> (!is.na(x[[<span>&quot;number&quot;</span>]])) publication &lt;-paste0(publication,<br> <span>&quot;, &quot;</span>, x[[<span>&quot;number&quot;</span>]])<br> <span>if</span> (!is.na(x[[<span>&quot;pages&quot;</span>]])) publication &lt;-paste0(publication,<br> <span>&quot;, _pp. &quot;</span>, x[[<span>&quot;pages&quot;</span>]], <span>&quot;_&quot;</span>)<br> <span>if</span> (!is.na(x[[<span>&quot;doi&quot;</span>]])) publication &lt;-paste0(publication,<br> <span>&quot;, &quot;</span>, paste0(<span>&quot;https://doi.org/&quot;</span>, <br> x[[<span>&quot;doi&quot;</span>]]))<br> <br> write(paste0(<span>&quot;publication = \&quot;&quot;</span>, publication,<span>&quot;\&quot;&quot;</span>), fileConn, append = <span>T</span>)<br> write(paste0(<span>&quot;publication_short = \&quot;&quot;</span>, publication,<span>&quot;\&quot;&quot;</span>),fileConn, append = <span>T</span>)<br> <br> <span># Abstract and optional shortened version.</span><br> <span>if</span> (abstract) {<br> write(paste0(<span>&quot;abstract = \&quot;&quot;</span>, x[[<span>&quot;abstract&quot;</span>]],<span>&quot;\&quot;&quot;</span>), fileConn, append = <span>T</span>)<br> } <span>else</span> {<br> write(<span>&quot;abstract = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> }<br> write(paste0(<span>&quot;abstract_short = \&quot;&quot;</span>,<span>&quot;\&quot;&quot;</span>), fileConn, append = <span>T</span>)<br> <br> <span># other possible fields are kept empty. They can be customized later by </span><br> <span># editing the created md</span><br> <br> write(<span>&quot;image_preview = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;selected = false&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;projects = []&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;tags = []&quot;</span>, fileConn, append = <span>T</span>)<br> <span>#links</span><br> write(<span>&quot;url_pdf = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;url_preprint = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;url_code = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;url_dataset = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;url_project = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;url_slides = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;url_video = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;url_poster = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;url_source = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> <span>#other stuff</span><br> write(<span>&quot;math = true&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;highlight = true&quot;</span>, fileConn, append = <span>T</span>)<br> <span># Featured image</span><br> write(<span>&quot;[header]&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;image = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> write(<span>&quot;caption = \&quot;\&quot;&quot;</span>, fileConn, append = <span>T</span>)<br> <br> write(<span>&quot;+++&quot;</span>, fileConn, append = <span>T</span>)<br> }<br> }<br> <span># apply the &quot;create_md&quot; function over the publications list to generate</span><br> <span># the different &quot;md&quot; files.</span><br> <br> apply(mypubs, FUN = <span>function</span>(x) create_md(x), MARGIN = <span>1</span>)<br>}</span></code></pre>
<div>Nothing fancy, here: I just use the <a rel="nofollow" href="https://cran.r-project.org/web/packages/RefManageR/index.html" target="_blank"><code>RefManageR</code></a> package to read the BibTeX file, and then cycle over publications to create files properly formatted for <code>hugo-academic</code> use.</div>
</div>
<div>
<h3>Running the script</h3>
<div>All is left is to run the script:</div>
<pre><code><span>my_bibfile &lt;-<span>&quot;/path/to/mybibtex.bib&quot;</span><br>out_fold &lt;-<span>&quot;/path/to/myoutfolder&quot;</span><br><br>bibtex_2academic(bibffile = my_bibfile, <br> outfold = out_fold, <br> abstract = <span>FALSE</span>)</span></code></pre>
<ul>
<li>
<div>The <code>outfold</code> argument allows specifying where the generated <code>md</code> files will be saved. Though in the end they will have to be moved to folder <code>content/publication</code> you may want to save them at first in a different folder to be able to check them before trying to deploy.</div>
</li>
<li>
<div>The <code>abstract</code> argument specifies whether to include the abstract in the <code>md</code> or not.</div>
</li>
</ul>
<div>Running the script will give you files like this one:</div>
<pre><code><span><span>+++</span><br><span>title</span> <span>=</span> <span>&quot;Estimating canopy water content of poplar plantation from MIVIS data&quot;</span><br><span>date</span> <span>=</span> <span>&quot;2006-01-01&quot;</span><br><span>authors</span> <span>=</span> <span>[&quot;R.</span> <span>Colombo&quot;,</span> <span>&quot;L. Busetto&quot;</span><span>,</span> <span>&quot;A. Marchesi&quot;</span><span>,</span> <span>&quot;M. Meroni&quot;</span><span>,</span> <span>&quot;C. Giardino&quot;</span><span>]</span><br><span>publication_types</span> <span>=</span> <span>[&quot;1&quot;]</span><br><span>publication</span> <span>=</span> <span>&quot;AIP Conference Proceedings, (852), _pp. 242-249_, https://doi.org/10.1063/1.2349350&quot;</span><br><span>publication_short</span> <span>=</span> <span>&quot;&quot;</span><br><span>abstract_short</span> <span>=</span> <span>&quot;&quot;</span><br><span>image_preview</span> <span>=</span> <span>&quot;&quot;</span><br><span>selected</span> <span>=</span> <span>false</span><br><span>projects</span> <span>=</span> <span>[]</span><br><span>tags</span> <span>=</span> <span>[]</span><br><span>url_pdf</span> <span>=</span> <span>&quot;&quot;</span><br><span>url_preprint</span> <span>=</span> <span>&quot;&quot;</span><br><span>url_code</span> <span>=</span> <span>&quot;&quot;</span><br><span>url_dataset</span> <span>=</span> <span>&quot;&quot;</span><br><span>url_project</span> <span>=</span> <span>&quot;&quot;</span><br><span>url_slides</span> <span>=</span> <span>&quot;&quot;</span><br><span>url_video</span> <span>=</span> <span>&quot;&quot;</span><br><span>url_poster</span> <span>=</span> <span>&quot;&quot;</span><br><span>url_source</span> <span>=</span> <span>&quot;&quot;</span><br><span>math</span> <span>=</span> <span>false</span><br><span>highlight</span> <span>=</span> <span>true</span><br><span>[header]</span><br><span>image</span> <span>=</span> <span>&quot;&quot;</span><br><span>caption</span> <span>=</span> <span>&quot;&quot;</span><br><span>+++</span></span></code></pre>
<div>, where I tweaked a bit the hugo-academic format to include bibliographic info such as volume, number, pages and doi link. The files can then be further customized to include, for example, links to pdfs, images, etcetera.</div>
<div>After moving all the <code>md</code> files to <code>content/publication</code>, the publications section of your <code>hugo-academic</code>site will be auto-populated, and should look more or less like this:</div>
<div><img src="https://i2.wp.com/lbusett.netlify.com/img/publications/publications.png?w=456&ssl=1" data-recalc-dims="1"></div>
<div>You can have a look at the final results on my (under construction) website <a rel="nofollow" href="https://lbusett.netlify.com/publication/" target="_blank">here</a>. I think it’s quite nice!</div>
</div>
</div>
<div>
<h2>Final Notes</h2>
<ul>
<li>
<div>My importing script is quite “quick and dirty”. It <span>does not attempt to deal with special characters, and even substitutes accented letters with “bare” letters to avoid rendering problems</span>. If someone more knowledgeable about encoding issues wants to try and improve it, I put it in <a rel="nofollow" href="https://gist.github.com/lbusett/da7b1fba4345e03192a450226a17636e" target="_blank">this gist</a></div>
</li>
<li>
<div>It may happen that your site will stop rendering when you put the new <code>md</code> files in <code>content/publication</code>. If so, the reason is probably that you have some “strange” formatting in some of the files. Usual suspects would be unproperly recognized <span>accents</span>, <span>math formulas</span> or other special characters in the <span>authors</span> and <span>abstract</span> fields. You will have to look into each file and remove any offending areas. (It happened to me a lot before properly saving to UTF-8)</div>
</li>
</ul>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://lbusettspatialr.blogspot.com/2018/03/automatically-importing-publications.html"> Spatial Processing in R</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/oj7uhA39xNM" height="1"></a></div>