<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/the-nhmrcdata-package-nhmrc-funding-outcomes-data-made-tidy/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://nsaunders.wordpress.com/2017/03/09/the-nhmrcdata-package-nhmrc-funding-outcomes-data-made-tidy/"> R – What You're Doing Is Rather Desperate</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Do you like R? Information about Australian biomedical research funding outcomes? Tidy data? If the answers to those questions are “yes”, then you may also like <a target="_blank" rel="nofollow" href="https://github.com/neilfws/politics/tree/master/nhmrcData">nhmrcData</a>, a collection of datasets derived from funding statistics provided by the Australian National Health &amp; Medical Research Council. It’s also my first R package (more correctly, R data package).</p>
<p>Read on for the details.<br>
<span></span></p>
<p><strong>1. Installation</strong></p>
<p>The package is hosted at Github and is in a subdirectory of a top-level repository, so it can be installed using the <code>devtools</code> package, then loaded in the usual way:</p>
<pre>
devtools::install_github(&quot;neilfws/politics&quot;, subdir = &quot;nhmrcData&quot;,
                          build_vignettes = TRUE)

library(nhmrcData)
</pre>
<p>There are currently 14 datasets in the package. No need to type <code>data()</code>; they are “lazy-loaded”, so just start typing “nhmrc” then hit Tab to see their names. The datasets are (somewhat) documented and briefly described in the repository README so for this post, I’ll focus on just four examples: those related to gender.</p>
<p><strong>2. Examples</strong><br>
Example code is (or will be) in <a target="_blank" rel="nofollow" href="https://github.com/neilfws/politics/blob/master/nhmrcData/vignettes/nhmrcData.Rmd">the package vignette</a> (I’m a bit over copy-pasting code to WordPress). Here are the results.</p>
<p><strong>2.1 nhmrcOutcomesGenderBRA</strong><br>
<a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/?attachment_id=4760"><img sizes="(max-width: 672px) 100vw, 672px" data-image-description="" data-orig-file="https://i1.wp.com/nsaunders.files.wordpress.com/2017/03/nhmrcgenderbra-plot-1.png?w=456&amp;ssl=1" data-orig-size="672,576" data-image-title="nhmrcgenderbra-plot-1" data-comments-opened="1" class="wp-image-4760" data-recalc-dims="1" data-attachment-id="4760" data-permalink="https://nsaunders.wordpress.com/2017/03/09/the-nhmrcdata-package-nhmrc-funding-outcomes-data-made-tidy/nhmrcgenderbra-plot-1/" alt="" data-medium-file="https://i1.wp.com/nsaunders.files.wordpress.com/2017/03/nhmrcgenderbra-plot-1.png?w=456&amp;ssl=1?w=300" data-large-file="https://i1.wp.com/nsaunders.files.wordpress.com/2017/03/nhmrcgenderbra-plot-1.png?w=456&amp;ssl=1?w=672" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i1.wp.com/nsaunders.files.wordpress.com/2017/03/nhmrcgenderbra-plot-1.png?w=456&ssl=1"></a>The <code>nhmrcOutcomesGenderBRA</code> dataset contains information on funding outcomes by gender, fellowship scheme and broad research area for 2013 – 2015. Here’s an attempt to capture all of that in an area plot.</p>
<p>Mmm, look at those success rates. My qualitative first impression is that there are more women applicants in several categories, without a corresponding increase in funded proposals.</p>
<p><br></p>
<p><strong>2.2 nhmrcOutcomesGenderFellows</strong><br>
<a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/?attachment_id=4763"><img sizes="(max-width: 672px) 100vw, 672px" data-image-description="" data-orig-file="https://i1.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderfellows-plot-1.png?w=456&amp;ssl=1" data-orig-size="672,768" data-image-title="outcomesgenderfellows-plot-1" data-comments-opened="1" class="wp-image-4763" data-recalc-dims="1" data-attachment-id="4763" data-permalink="https://nsaunders.wordpress.com/2017/03/09/the-nhmrcdata-package-nhmrc-funding-outcomes-data-made-tidy/outcomesgenderfellows-plot-1/" alt="" data-medium-file="https://i1.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderfellows-plot-1.png?w=456&amp;ssl=1?w=263" data-large-file="https://i1.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderfellows-plot-1.png?w=456&amp;ssl=1?w=672" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i1.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderfellows-plot-1.png?w=456&ssl=1"></a>The <code>nhmrcOutcomesGenderFellows</code> dataset also summarises outcomes by fellowship scheme, this time broken down into levels.</p>
<p>What’s interesting here is the difference in numbers by gender after the career development stage.</p>
<p><br></p>
<p><strong>2.3 nhmrcOutcomesGenderPartTime</strong><br>
<a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/?attachment_id=4765"><img sizes="(max-width: 672px) 100vw, 672px" data-image-description="" data-orig-file="https://i2.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderpt-plot-1.png?w=456&amp;ssl=1" data-orig-size="672,480" data-image-title="outcomesgenderpt-plot-1" data-comments-opened="1" class="wp-image-4765" data-recalc-dims="1" data-attachment-id="4765" data-permalink="https://nsaunders.wordpress.com/2017/03/09/the-nhmrcdata-package-nhmrc-funding-outcomes-data-made-tidy/outcomesgenderpt-plot-1/" alt="" data-medium-file="https://i2.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderpt-plot-1.png?w=456&amp;ssl=1?w=300" data-large-file="https://i2.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderpt-plot-1.png?w=456&amp;ssl=1?w=672" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i2.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderpt-plot-1.png?w=456&ssl=1"></a>The <code>nhmrcOutcomesGenderPartTime</code> dataset looks at the relatively-small number of part-time fellowship applications.</p>
<p>I tried to create this chart in the style of a “population pyramid”, to contrast directly the numbers by gender. Looks rather like women could use more part-time opportunities earlier in their careers.</p>
<p><br></p>
<p><strong>2.4 nhmrcOutcomesGenderScheme</strong><br>
<a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/?attachment_id=4766"><img sizes="(max-width: 672px) 100vw, 672px" data-image-description="" data-orig-file="https://i0.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderscheme-plot-1.png?w=456&amp;ssl=1" data-orig-size="672,480" data-image-title="outcomesgenderscheme-plot-1" data-comments-opened="1" class="wp-image-4766" data-recalc-dims="1" data-attachment-id="4766" data-permalink="https://nsaunders.wordpress.com/2017/03/09/the-nhmrcdata-package-nhmrc-funding-outcomes-data-made-tidy/outcomesgenderscheme-plot-1/" alt="" data-medium-file="https://i0.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderscheme-plot-1.png?w=456&amp;ssl=1?w=300" data-large-file="https://i0.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderscheme-plot-1.png?w=456&amp;ssl=1?w=672" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i0.wp.com/nsaunders.files.wordpress.com/2017/03/outcomesgenderscheme-plot-1.png?w=456&ssl=1"></a>Finally, the <code>nhmrcOutcomesGenderScheme</code> dataset: yet another summary by fellowship scheme but this time, broken down into 29 categories, such as “Australian Biomedical (Peter Doherty)”.</p>
<p>That’s the category I chose to display here and again, it indicates that more women are trying as hard or harder, yet this is not reflected in funding rates.</p>
<p>Of course, these charts are exploratory qualitative observations and it would be useful to apply some statistical tests where appropriate.</p>
<p><strong>3. Thoughts on publicly-available data and tidyness</strong><br>
It’s great when organisations make their data available, but the next step is making it <em>usable</em>. Excel files is a start – but why are they always so dreadfully-formatted? The answer of course is that people cannot decide whether Excel is a database or a presentation tool (it is neither), so we end up with the worst of all worlds. PDF as a data source is just plain wrong, end of story. For this project I used the excellent <a target="_blank" rel="nofollow" href="https://github.com/ropensci/tabulizer">tabulizer</a> package to extract PDF tables into R matrices, followed by a lot of customised, manual cleaning which was impossible to automate or to make reproducible. The best that I could do was to dump the results as CSV files in the package <code>data-raw</code> directory.</p>
<p>Given that organisations struggle to get even the basics of data correct, I suppose hoping for some consideration of tidyness is wildly optimistic. So many spreadsheets with a header row of years (2000, 2001, 2002…) as opposed to a column named <code>year</code>. The tidy philosophy certainly makes manipulation and visualisation using R much easier, though I note that the occasional “spread” to wide format is still required for certain calculations. For example, funding outcomes are often expressed as total applications and successful applications. When those values are stored in the same column, it’s easier to calculate the number of unsuccessful applications (total – successful) by spreading to two columns, then gathering back again.</p>
<p><strong>4. Thoughts on package writing</strong><br>
I’ve long wanted to write an R package but like many people, I kept putting it into the “too hard just now” basket. This is no longer the case thanks to some excellent resources. The first of these of course is RStudio itself, which includes several tools to facilitate package writing, building, testing and documentation. For an overview, <a target="_blank" rel="nofollow" href="https://support.rstudio.com/hc/en-us/articles/200486488-Developing-Packages-with-RStudio">start here</a>. There are also many excellent online guides and tutorials. One such resource (which includes links to others) is Karl Broman’s <a target="_blank" rel="nofollow" href="http://kbroman.org/pkg_primer/">R package primer</a>.</p>
<p>Time will tell how easy it will be to update the package with new NHMRC data. In the meantime, I hope some of you find this useful and welcome your constructive feedback.</p>
<p>Filed under: <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/category/statistics/r/">R</a>, <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/category/statistics/">statistics</a> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/nsaunders.wordpress.com/4710/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/nsaunders.wordpress.com/4710/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://nsaunders.wordpress.com/2017/03/09/the-nhmrcdata-package-nhmrc-funding-outcomes-data-made-tidy/"> R – What You're Doing Is Rather Desperate</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/S2mB514IOoM"></a></div>