<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/let-it-flow-let-it-flow-let-it-flow/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://johnmackintosh.com/2017-12-21-flow/"> HighlandR</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>	Animating dot plots to demonstrate patient flow – </p>
<h2>Dots..</h2>
<p>This is not the blog post I’d originally intended to write. But I’m glad – because this one is so much better.</p>
<p>Some background. I’m one of the few Scotland based members of <a rel="nofollow" href="https://www.aphanalysts.org/" target="_blank">AphA</a> – the Association of Professional Healthcare Analysts. They’ve had a couple of events recently that I was keeping my eye on via Twitter and it became apparent that a session demonstrating R had made some waves – in a good way.</p>
<p>I’d been having a wee exchange with <a rel="nofollow" href="https://twitter.com/kurtstat" target="_blank">Neil Pettinger</a> regarding R and took the opportunity to ask permission to use one of his Excel files. This featured a dot plot chart that demonstrated patient flow.<br>
I wanted to show an alternative way of creating the plot using R.</p>
<p>Neil is an “information training consultant” – at least that’s what it says on his <a rel="nofollow" href="http://www.kurtosis.co.uk/index.htm" target="_blank">website</a> but for me that is shorthand for guru of statistics, information and analysis with a huge breadth of experience working in and alongside analytical teams in the UK NHS. There can’t be many boards/ trusts within the NHS that Neil has not worked with at some point.</p>
<p>Neil pointed out that instead of the 2014 version I was referring to, there was a new updated version, and he kindly forwarded me that instead, along with a guide on how to produce the plot using Excel.</p>
<p>The plot in question shows admissions / discharges and transfers in/out on a fictional “good day” in a fictional hospital. I’m not going to try and explain Neil’s philosophy and vision regarding patient flow – (he sent me a brilliant presentation featuring works of art &amp; sculpture -which if you attend one of his training sessions you may be fortunate to see) but suffice to say this plot is merely a starting point.</p>
<p>Here it is:</p>
<p><img alt="2017-12-20-Neil-Original-red-green-grey.jpg" src="https://i2.wp.com/johnmackintosh.com/img/2017-12-20-Neil-Original-red-green-grey.jpg?w=456&ssl=1" data-recalc-dims="1"></p>
<p>We have red dots for admissions (extra work/resource required), green for discharges (reduction in workload/ resource requirements).</p>
<p>Transfers (moving from one area/ward within the hospital to another) are represented by grey dots.</p>
<p>Finally, the above are split out across 3 “staging posts” – Accident &amp; Emergency (A&amp;E), Assessment areas and Wards.<br>
For example, if a patient comes to A&amp;E, gets sent for assessment, and is then sent to a ward for further care, then they have 5 moves recorded in the dataset( into A&amp;E, out of A&amp;E, into the Assessment area, out of the Assessment area, and then in to the ward).</p>
<p>If you want to follow along (go on) then you should head over to Neil’s site, download the <a rel="nofollow" href="http://www.kurtosis.co.uk/technique/RedGreenGreyDots.xlsx" target="_blank">excel file</a> and take a look at the “how to” guide on the same <a rel="nofollow" href="http://www.kurtosis.co.uk/technique/red_green_grey.html" target="_blank">page</a>. Existing R users are already likely to be shuddering at all the manual manipulation required.</p>
<p>For the first attempt, I followed Neil’s approach pretty closely, resulting in a lot of code to sort and group, although ggplot2 made the actual plotting much simpler.<br>
I shared my very first attempt ( produce with barely any ggplot2 code) which was quite good, but there were a few issues – the ins/ outs being coloured blue instead of grey, and overplotting of several points.</p>
<p>Here’s the code:</p>
<div>
<div>
<pre><code><span># Set limits for plotting</span><span>
</span><span>lims</span><span> </span><span>&lt;-</span><span> </span><span>as.POSIXct</span><span>(</span><span>strptime</span><span>(</span><span>c</span><span>(</span><span>&quot;2014-09-03 00:00&quot;</span><span>,</span><span>&quot;2014-09-03 24:00&quot;</span><span>)</span><span>
</span><span>,</span><span> </span><span>format</span><span> </span><span>=</span><span> </span><span>&quot;%Y-%m-%d %H:%M&quot;</span><span>))</span><span> </span><span>ggplot</span><span>(</span><span>plot_data</span><span>,</span><span>aes</span><span>(</span><span>Movement15</span><span>,</span><span>Movement_15_SEQNO</span><span>,</span><span> </span><span>colour</span><span>=</span><span>Movement_Type</span><span>))</span><span>+</span><span>
</span><span>geom_point</span><span>()</span><span>+</span><span>
</span><span>facet_grid</span><span>(</span><span>Staging_Post</span><span>~</span><span>.</span><span>)</span><span>+</span><span>
</span><span>scale_x_datetime</span><span>(</span><span>date_labels</span><span>=</span><span>&quot;%H:%M&quot;</span><span>,</span><span>date_breaks</span><span> </span><span>=</span><span> </span><span>&quot;3 hours&quot;</span><span>,</span><span>
</span><span>limits</span><span> </span><span>=</span><span> </span><span>lims</span><span>,</span><span> </span><span>timezone</span><span> </span><span>=</span><span> </span><span>&quot;CET&quot;</span><span>,</span><span>expand</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>0</span><span>,</span><span>0</span><span>))</span><span>+</span><span>
</span><span>theme_minimal</span><span>()</span><span>+</span><span>
</span><span>theme</span><span>(</span><span>legend.position</span><span>=</span><span>&quot;bottom&quot;</span><span>)</span><span>+</span><span>
</span><span>theme</span><span>(</span><span>panel.grid.minor</span><span>=</span><span>element_blank</span><span>(),</span><span>
</span><span>panel.grid.major</span><span>=</span><span>element_blank</span><span>())</span><span>
</span></code></pre>
</div>
</div>
<p>And here’s what you get</p>
<p><img alt="2017-12-20-rough-draft.png" src="https://i0.wp.com/johnmackintosh.com/img/2017-12-20-rough-draft.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>If you try and run that code as is – it won’t work, because we need to do some data prep first..</p>
<p>The next day, I came back to the task with a view to coming up with a “slicker” process.<br>
I realised the bulk of the prep work required in Excel was actually straightforward in SQL.<br>
If you don’t know SQL, that’s OK – because this is R, and we have dplyr.</p>
<p>Here’s how to do it:</p>
<div>
<div>
<pre><code><span>data</span><span> </span><span>&lt;-</span><span> </span><span>read_xlsx</span><span>(</span><span>&quot;RedGreenGreyDots.xlsx&quot;</span><span>,</span><span> </span><span>sheet</span><span> </span><span>=</span><span> </span><span>1</span><span>)</span><span> </span><span>#read raw data from Excel</span><span>
</span><span>plot_data</span><span> </span><span>&lt;-</span><span> </span><span>data</span><span> </span><span>%>%</span><span> </span><span>mutate</span><span>(</span><span>Movement15</span><span> </span><span>=</span><span> </span><span>lubridate</span><span>::</span><span>floor_date</span><span>(</span><span>MovementDateTime</span><span>,</span><span>&quot;15 minutes&quot;</span><span>))</span><span> </span><span>%>%</span><span> </span><span>group_by</span><span>(</span><span>IN_OUT</span><span>,</span><span> </span><span>Movement_Type</span><span>,</span><span>Staging_Post</span><span>,</span><span>Movement15</span><span>)</span><span> </span><span>%>%</span><span> </span><span>mutate</span><span>(</span><span>counter</span><span> </span><span>=</span><span> </span><span>case_when</span><span> </span><span>(</span><span>
</span><span>IN_OUT</span><span> </span><span>==</span><span> </span><span>'IN'</span><span> </span><span>~</span><span> </span><span>1</span><span>,</span><span>
</span><span>IN_OUT</span><span> </span><span>==</span><span> </span><span>'OUT'</span><span> </span><span>~</span><span> </span><span>-1</span><span>))</span><span> </span><span>%>%</span><span> </span><span>mutate</span><span>(</span><span>Movement_15_SEQNO</span><span> </span><span>=</span><span>cumsum</span><span>(</span><span>counter</span><span>))</span><span> </span><span>%>%</span><span> </span><span>ungroup</span><span>()</span><span> </span><span># Change &quot;Tranfer In&quot; or &quot;Transfer Out&quot; to &quot;Transfer&quot;</span><span>
</span><span>plot_data</span><span>$</span><span>Movement_Type</span><span> </span><span>&lt;-</span><span> </span><span>gsub</span><span>(</span><span>&quot;Transfer.*&quot;</span><span>,</span><span>&quot;Transfer&quot;</span><span>,</span><span>x</span><span>=</span><span>plot_data</span><span>$</span><span>Movement_Type</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p>NOW you can try running the ggplot2 code above and you should get a very similar looking plot.</p>
<p>If you’re new to R / dplyr and wondering what the heck’s going on:</p>
<ul>
<li>
<p>First we read the data in using the <em>readxlsx</em> function from the <strong>readxl</strong> package</p>
</li>
<li>
<p>(Not shown) check the structure of the dataframe and make sure the data types all check out (always an especially good idea when importing from Excel)</p>
</li>
<li>
<p>use <strong>dplyr</strong>’s <em>mutate</em> function to create a new variable to group all the movements into 15 minute intervals, which is a piece of cake with <strong>lubridate</strong>’s <em>floordate</em> function.</p>
</li>
<li>
<p>Next we group the data by the IN_OUT, Movement_Type, Staging_Post and our new Movement15 variable</p>
</li>
<li>
<p>We then create another new column, this time to create a counter field, with a value of 1 when the IN_OUT column = “IN” (so that these points appear above the x axis horizon) and -1 when the value is “OUT” (so the points display below the horizon)</p>
</li>
<li>
<p>We create yet another variable, giving the cumulative sum of the counter field, so that we have a dots that stack upon each other at each 15 minute interval ( rather than just having one point representing the maximum / minimum values at each stage)</p>
</li>
</ul>
<p>The last thing to do is to tidy up the Movement_Type field – we don’t want 4 movement types on our final plot, so we just change the values to “Transfer” – they are colour coded grey regardless of whether they are a transfer in or transfer out.</p>
<h3>Plots..</h3>
<p>Here is how the plot looks once we work some ggplot2 magic:</p>
<div>
<div>
<pre><code><span>ggplot</span><span>(</span><span>plot_data</span><span>,</span><span>aes</span><span>(</span><span>Movement15</span><span>,</span><span>Movement_15_SEQNO</span><span>,</span><span> </span><span>colour</span><span>=</span><span>Movement_Type</span><span>))</span><span>+</span><span>
</span><span>geom_jitter</span><span>(</span><span>width</span><span>=</span><span>0.10</span><span>)</span><span>+</span><span>
</span><span>scale_colour_manual</span><span>(</span><span>values</span><span>=</span><span>c</span><span>(</span><span>&quot;#D7100D&quot;</span><span>,</span><span>&quot;#40B578&quot;</span><span>,</span><span>&quot;grey60&quot;</span><span>))</span><span>+</span><span>
</span><span>facet_grid</span><span>(</span><span>Staging_Post</span><span>~</span><span>.</span><span>,</span><span>switch</span><span> </span><span>=</span><span> </span><span>&quot;y&quot;</span><span>)</span><span>+</span><span>
</span><span>scale_x_datetime</span><span>(</span><span>date_labels</span><span>=</span><span>&quot;%H:%M&quot;</span><span>,</span><span>date_breaks</span><span> </span><span>=</span><span> </span><span>&quot;3 hours&quot;</span><span>,</span><span>
</span><span>limits</span><span> </span><span>=</span><span> </span><span>lims</span><span>,</span><span>
</span><span>timezone</span><span> </span><span>=</span><span> </span><span>&quot;CET&quot;</span><span>,</span><span>
</span><span>expand</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>0</span><span>,</span><span>0</span><span>))</span><span>+</span><span>
</span><span>ggtitle</span><span>(</span><span>label</span><span> </span><span>=</span><span> </span><span>&quot;Anytown General Hospital | Wednesday 3rd September 2014 00:00 to 23:59\n&quot;</span><span>,</span><span>
</span><span>subtitle</span><span>=</span><span>&quot;A&amp;E AND INPATIENT ARRIVALS, DEPARTURES AND TRANSFERS&quot;</span><span>)</span><span>+</span><span>
</span><span>labs</span><span>(</span><span>x</span><span>=</span><span> </span><span>NULL</span><span>,</span><span> </span><span>y</span><span>=</span><span> </span><span>NULL</span><span>)</span><span>+</span><span>
</span><span>theme_ipsum</span><span>(</span><span>base_family</span><span> </span><span>=</span><span> </span><span>&quot;Arial Narrow&quot;</span><span>)</span><span>+</span><span>
</span><span>theme</span><span>(</span><span>axis.text.y</span><span>=</span><span>element_blank</span><span>(),</span><span>
</span><span>axis.ticks.y</span><span>=</span><span>element_blank</span><span>())</span><span>+</span><span>
</span><span>theme</span><span>(</span><span>axis.text.x</span><span>=</span><span>element_text</span><span>(</span><span>size</span><span>=</span><span>7</span><span>))</span><span> </span><span>+</span><span>
</span><span>theme</span><span>(</span><span>axis.ticks.x</span><span>=</span><span>element_blank</span><span>())</span><span>+</span><span>
</span><span>theme</span><span>(</span><span>legend.position</span><span>=</span><span>&quot;bottom&quot;</span><span>)</span><span>+</span><span>
</span><span>theme</span><span>(</span><span>panel.grid.minor</span><span>=</span><span>element_blank</span><span>(),</span><span>
</span><span>panel.grid.major</span><span>=</span><span>element_blank</span><span>())</span><span>+</span><span>
</span><span>theme</span><span>(</span><span>strip.text.y</span><span> </span><span>=</span><span> </span><span>element_text</span><span>(</span><span>angle</span><span> </span><span>=</span><span> </span><span>0</span><span>))</span><span>+</span><span>
</span><span>guides</span><span>(</span><span>color</span><span>=</span><span>guide_legend</span><span>(</span><span>&quot;Movement Type&quot;</span><span>))</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="2017-12-19-minimal-row-of-dots.png" src="https://i0.wp.com/johnmackintosh.com/img/2017-12-19-minimal-row-of-dots.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Not bad. Being really picky – I haven’t replicated the colour coding behind each of the facet labels that Neil had on his original plot (A&amp;E should have a yellow background for instance). Similarly, the clever colour coding of the title text on the original chart does away with the need for a legend.<br>
This is kind of annoying because its very easy to do this in Excel but a real faff to emulate in ggplot2 (probably for good reason). I have found a couple of solutions to resolve both problems, but they involve a lot of mucking about with grid settings and I’m not sure its worth the aggro.</p>
<p>A key change to the original ggplot2 code, apart from manually specifying the colours, is that I’m using geom_jitter instead of geom_point. This helps get rid of over-plotting issues when 2 different Movement Types occur at exactly the same 15 minute interval.</p>
<p>During our initial discussion, Neil asked if it was possible to animate plots in R. I reassured him it was, but had to be honest and told him it was not something I had experience of. There was a jokey aside along the lines of “ lets get the static plots working, we can make movies later”.</p>
<p>Well, here is the animated version of the above plot:</p>
<p><img alt="2017-12-19-row-of-dots-output.gif" src="https://i0.wp.com/johnmackintosh.com/img/2017-12-19-row-of-dots-output.gif?w=456&ssl=1" data-recalc-dims="1"></p>
<p>This was produced using very similar ggplot2 code to the static version.<br>
The key difference was the use of the gganimate package, and the use of the “frame” and cumulative arguments, which basically instruct ggplot2 to make a plot showing the cumulative picture of movements from midnight up to each 15 minute segment.<br>
For example, the first image shows midnight to quarter past, the second midnight to half past, then midnight to quarter to midnight, then midnight to 1 AM, and so on.<br>
The resulting plots are then magically stitched together and saved to the desired output – in this case as a gif.</p>
<p>There are a couple of ways of setting the animation options for the timescale (interval- basically how fast you want the plot to cycle through from start to finish).</p>
<p>You can simply tag on an “ interval = x” argument when saving the plot, or you can use the animation package, and set the options, plot size etc up front.<br>
This is the approach I used, as you’ll see in the accompanying code, but either should work – you can uncomment /comment out the relevant lines in the code and try both. There’s a link to the <a rel="nofollow" href="https://github.com/johnmackintosh/RowOfDots" target="_blank">repo</a> at the top of the post.</p>
<h3>..&amp; Dr Cox</h3>
<p>To finish up – I let Neil have a sneak peak at this post and he came up with the seasonally inspired title.<br>
As it’s that time of year, and this is likely to be my last post before then – seasons greetings to all / both my readers, hope Santa is good to you, stay safe and well.</p>
<p><img alt="" src="https://i2.wp.com/media.giphy.com/media/l0MYLFtDZFcIli98I/giphy.gif?w=456&ssl=1" data-recalc-dims="1"></p>
<p><img alt="" src="https://i2.wp.com/media.giphy.com/media/3o7TKCQGGFKQNAHs3e/giphy.gif?w=456&ssl=1" data-recalc-dims="1"></p>
<p>All the best,</p>
<p>John</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://johnmackintosh.com/2017-12-21-flow/"> HighlandR</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/XF8yJ8ciwN4" height="1"></a></div>