<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/reporting-in-a-repeatable-parameterised-transparent-way/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://blog.ouseful.info/2017/02/23/reporting-in-a-repeatable-parameterised-transparent-way/"> Rstats – OUseful.Info, the blog…</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Earlier this week, I spent a day chatting to folk from the House of Commons Library as a part of a bit of temporary day-a-week-or-so bit of work I’m doing with the Parliamentary Digital Service.</p>
<p>During one of the conversations on matters loosely geodata-related with Carl Baker, Carl mentioned an <a target="_blank" rel="nofollow" href="http://www.content.digital.nhs.uk/catalogue/PUB23139">NHS Digital data set describing the number of people on a GP Practice list who live within a particular LSOA (Lower Super Output Area)</a>. There are possible GP practice closures on the Island at the moment, so I thought this might be an interesting dataset to play with in that respect.</p>
<p>Another thing Carl is involved with is producing a regularly updated briefing on <a target="_blank" rel="nofollow" href="http://researchbriefings.parliament.uk/ResearchBriefing/Summary/SN06964">Accident and Emergency Statistics</a>. Excel and QGIS templates do much of the work in producing the updated documents, so much of the data wrangling side of the report generation is automated using those tools. Supporting regular updating of briefings, as well as answering specific, <em>ad hoc</em> questions from MPs, producing debate briefings and other current topic briefings, seems to be an important Library activity.</p>
<p>As I’ve been looking for opportunities to compare different automation routes using things like Jupyter notebooks and RMarkdown, I thought I’d have a play with the GP list/LSOA data, showing how we might be able to use each of those two routes to generate maps showing the geographical distribution, across LSOAs at least, for GP practices on the Isle of Wight. This demonstrates several things, including: data ingest; filtering according to practice codes accessed from another dataset; importing a geoJSON shapefile; generating a choropleth map using the shapefile matched to the GP list LSOA codes.</p>
<p>The first thing I tried was using a python/pandas Jupyter notebook to create a choropleth map for a particular practice using the <code>folium</code> library. This didn’t take long to do at all – I’ve previously built an NHS admin database that lets me find practice codes associated with a particular CCG, such as the Isle of Wight CCG, as well as a notebook that generates a choropleth over LSOA boundaries, so it was simply a case of copying and pasting old bits of code and adding in the new dataset.You can see a <a target="_blank" rel="nofollow" href="https://psychemedia.github.io/parlihacks/GP%2BPractice%2BList%2Bby%2BLSOA.html">rendered example of the notebook here</a> (<a target="_blank" rel="nofollow" href="https://github.com/psychemedia/parlihacks/blob/master/notebooks/GP%20Practice%20List%20by%20LSOA.ipynb">download</a>).</p>
<p>One thing you might notice from the rendered notebook is that I actually “widgetised” it, allowing users of the live notebook to select a particular practice and render the associated map.</p>
<p>Whilst I find the Jupyter notebooks to provide a really friendly and accommodating environment for pulling together a recipe such as this, the report generation workflows are arguably still somewhat behind the workflows supported by RStudio and the <code>knitr</code> tools.</p>
<p>So what does an RStudio workflow have to offer? Using Rmarkdown (Rmd) we can combine text, code and code outputs in much the same way as we can in a Jupyter notebook, but with slightly more control over the presentation of the output.</p>
<p><img sizes="(max-width: 643px) 100vw, 643px" data-image-description="" data-orig-file="https://i2.wp.com/ouseful.files.wordpress.com/2017/02/dropbox_parlidata_rdemos_-_rstudio.png?resize=456%2C387&amp;ssl=1" data-orig-size="1642,1392" data-image-title="__dropbox_parlidata_rdemos_-_rstudio" class="wp-image-15917" data-comments-opened="1" data-recalc-dims="1" data-attachment-id="15917" data-permalink="https://blog.ouseful.info/2017/02/23/reporting-in-a-repeatable-parameterised-transparent-way/__dropbox_parlidata_rdemos_-_rstudio/" alt="__dropbox_parlidata_rdemos_-_rstudio" data-medium-file="https://i2.wp.com/ouseful.files.wordpress.com/2017/02/dropbox_parlidata_rdemos_-_rstudio.png?resize=456%2C387&amp;ssl=1?w=300" data-large-file="https://i2.wp.com/ouseful.files.wordpress.com/2017/02/dropbox_parlidata_rdemos_-_rstudio.png?resize=456%2C387&amp;ssl=1?w=830" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i2.wp.com/ouseful.files.wordpress.com/2017/02/dropbox_parlidata_rdemos_-_rstudio.png?resize=456%2C387&ssl=1"></p>
<p>For example, from a <a target="_blank" rel="nofollow" href="https://github.com/psychemedia/parlihacks/blob/master/Rmd/gplsoa.Rmd">single Rmd file</a> we can knit an <a target="_blank" rel="nofollow" href="https://psychemedia.github.io/parlihacks/gplsoa.html">output HTML file</a> that incorporates an interactive <em>leaflet</em> map, or a <a target="_blank" rel="nofollow" href="https://psychemedia.github.io/parlihacks/gplsoa.html">static PDF document</a>.</p>
<p>It’s also possible to use a <a target="_blank" rel="nofollow" href="http://rmarkdown.rstudio.com/developer_parameterized_reports.html">parameterised report generation</a> workflow to generate separate reports for each practice. For example, applying <a target="_blank" rel="nofollow" href="https://github.com/psychemedia/parlihacks/blob/master/Rmd/gplsoaMultiReport.Rmd">this parameterised report generation script</a> to a <a target="_blank" rel="nofollow" href="https://github.com/psychemedia/parlihacks/blob/master/Rmd/gplsoa_base.Rmd">generic base template report</a> will generate a <a target="_blank" rel="nofollow" href="https://psychemedia.github.io/parlihacks/iwgplsoadocs/">set of PDF reports on a per practice basis for each practice on the Isle of Wight</a>.</p>
<p>The <a target="_blank" rel="nofollow" href="https://bookdown.org/yihui/bookdown/"><code>bookdown</code></a> package, which I haven’t played with yet, also looks promising for its ability to generate a single output document from a set of source documents. (I have a question in about <a target="_blank" rel="nofollow" href="https://github.com/rstudio/bookdown/issues/346">the extent to which <code>bookdown</code> supports partially paramterised compound document creation</a>).</p>
<p>Having started thinking about comparisons between Excel, Jupyter and RStudio workflows, possible next steps are:</p>
<ul>
<li>to look for sensible ways of comparing the workflow associated with each,</li>
<li>the ramp-up skills required, and blockers (including cultural blockers) associated with getting started with new tools such as Jupyter or RStudio, and</li>
<li>the various ways in which each tool/workflow supports: transparency; maintainability; extendibility; correctness; reuse; integration with other tools; ease and speed of use.</li>
</ul>
<p>It would also be interesting to explore how much time and effort would actually be involved in trying to port a legacy Excel report generating template to Rmd or ipynb, and what sorts of issue would be likely to arise, and what benefits Excel offers compared to Jupyter and RStudio workflows.</p>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/15912/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/15912/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://blog.ouseful.info/2017/02/23/reporting-in-a-repeatable-parameterised-transparent-way/"> Rstats – OUseful.Info, the blog…</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/lNq2HsEQo1A"></a></div>