<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/experimenting-with-sankey-diagrams-in-r-and-python/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/"> Rstats – OUseful.Info, the blog…</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>A couple of days ago, I spotted a post by Oli Hawkins on <a target="_blank" rel="nofollow" href="http://olihawkins.com/2017/03/1">Visualising migration between the countries of the UK</a> which linked to a Sankey diagram demo of <a target="_blank" rel="nofollow" href="http://olihawkins.com/visualisation/8">Internal migration flows in the UK</a>.</p>
<p><a target="_blank" rel="nofollow" href="http://olihawkins.com/visualisation/8"><img sizes="(max-width: 555px) 100vw, 555px" data-image-description="" data-orig-file="https://ouseful.files.wordpress.com/2017/03/oli_hawkins__internal_migration_flows_in_the_uk.png?w=450#038;h=416" data-orig-size="1700,1276" data-image-title="Oli_Hawkins__Internal_migration_flows_in_the_UK" class="wp-image-16011" data-comments-opened="1" data-recalc-dims="1" data-attachment-id="16011" data-permalink="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/oli_hawkins__internal_migration_flows_in_the_uk/" alt="" data-medium-file="https://ouseful.files.wordpress.com/2017/03/oli_hawkins__internal_migration_flows_in_the_uk.png?w=450#038;h=416?w=300" data-large-file="https://ouseful.files.wordpress.com/2017/03/oli_hawkins__internal_migration_flows_in_the_uk.png?w=450#038;h=416?w=830" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://ouseful.files.wordpress.com/2017/03/oli_hawkins__internal_migration_flows_in_the_uk.png?w=450#038;h=416"></a></p>
<p>One of the things that interests me about the Jupyter and RStudio centred reproducible research ecosystems is their support for libraries that generate interactive HTML/javascript outputs (charts, maps, etc) from a computational data analysis context such as R, or python/pandas, so it was only natural (?!) that I though I should see how easy it would be to generate something similar from a code context.</p>
<p>In an R context, there are several libraries available that support the generation of Sankey diagrams, including <a target="_blank" rel="nofollow" href="https://www.rdocumentation.org/packages/googleVis/versions/0.6.0/topics/gvisSankey"><code>googleVis</code></a> (which wraps <a target="_blank" rel="nofollow" href="https://developers.google.com/chart/interactive/docs/gallery/sankey">Google Chart tools</a>), and a couple of packages that wrap <a target="_blank" rel="nofollow" href="https://bost.ocks.org/mike/sankey/">d3.js</a> – an original <a target="_blank" rel="nofollow" href="https://github.com/timelyportfolio/rCharts_d3_sankey">rCharts Sankey diagram demo by @timelyporfolio</a>, and a more recent HTMLWidgets demo (<a target="_blank" rel="nofollow" href="https://github.com/fbreitwieser/sankeyD3">sankeyD3</a>).</p>
<p>Here’s an example of the evolution of my Sankey diagram in R using <code>googleVis</code> – the <em>Rmd</em> code is <a target="_blank" rel="nofollow" href="https://github.com/psychemedia/parlihacks/blob/master/Rmd/sankeyFlow.Rmd">here</a> and a version of the <code>knitr</code>ed HTML output is <a target="_blank" rel="nofollow" href="https://psychemedia.github.io/parlihacks/sankeyFlow.html">here</a>.</p>
<p>The original data comprised a matrix relating population flows between English regions, Wales, Scotland and Northern Ireland. The simplest rendering of the data using the <code>googleViz</code> Sankey diagram generator produces an output that uses default colours to label the nodes.</p>
<p><img sizes="(max-width: 508px) 100vw, 508px" data-image-description="" data-orig-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html.png?w=450#038;h=571" data-orig-size="1028,1156" data-image-title="__Dropbox_parlidata_rdemos_sankeyFlow_html" data-comments-opened="1" class="wp-image-16010" data-recalc-dims="1" data-attachment-id="16010" data-permalink="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/__dropbox_parlidata_rdemos_sankeyflow_html/" alt="" data-medium-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html.png?w=450#038;h=571?w=267" data-large-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html.png?w=450#038;h=571?w=830" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html.png?w=450#038;h=571"></p>
<p>Using the country code indicator at the start of each region/country identifier, we can generate a mapping from country to a country colour that can then be used to identify the country associated with each node.</p>
<p><img sizes="(max-width: 511px) 100vw, 511px" data-image-description="" data-orig-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html2.png?w=450#038;h=468" data-orig-size="894,820" data-image-title="__Dropbox_parlidata_rdemos_sankeyFlow_html2" data-comments-opened="1" class="wp-image-16009" data-recalc-dims="1" data-attachment-id="16009" data-permalink="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/__dropbox_parlidata_rdemos_sankeyflow_html2/" alt="" data-medium-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html2.png?w=450#038;h=468?w=300" data-large-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html2.png?w=450#038;h=468?w=830" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html2.png?w=450#038;h=468"></p>
<p>One of the settings for the diagram allows the source (or target) node colour to determine the edge colour. We can also play with the values we use as node labels:</p>
<p><img sizes="(max-width: 464px) 100vw, 464px" data-image-description="" data-orig-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html3.png?w=450#038;h=456" data-orig-size="820,806" data-image-title="__Dropbox_parlidata_rdemos_sankeyFlow_html3" data-comments-opened="1" class="wp-image-16008" data-recalc-dims="1" data-attachment-id="16008" data-permalink="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/__dropbox_parlidata_rdemos_sankeyflow_html3/" alt="" data-medium-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html3.png?w=450#038;h=456?w=300" data-large-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html3.png?w=450#038;h=456?w=820" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html3.png?w=450#038;h=456"></p>
<p>If we exclude edges relating to flow between regions of the same country, we get a diagram that is more reminiscent of Oli’s orignal (country level) demo. Note also that the charts that are generated are interactive – in this case, we see a popup that describes the flow along one particular edge.</p>
<p><img sizes="(max-width: 490px) 100vw, 490px" data-image-description="" data-orig-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html4.png?w=450#038;h=455" data-orig-size="874,812" data-image-title="__Dropbox_parlidata_rdemos_sankeyFlow_html4" data-comments-opened="1" class="wp-image-16007" data-recalc-dims="1" data-attachment-id="16007" data-permalink="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/__dropbox_parlidata_rdemos_sankeyflow_html4/" alt="" data-medium-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html4.png?w=450#038;h=455?w=300" data-large-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html4.png?w=450#038;h=455?w=830" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html4.png?w=450#038;h=455"></p>
<p>If we associate a country with each region, we can group the data and sum the flow values to produce country level flows. Charting this produces a chart similar to the original inspiration.</p>
<p><img sizes="(max-width: 474px) 100vw, 474px" data-image-description="" data-orig-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html5.png?w=450#038;h=436" data-orig-size="880,808" data-image-title="__Dropbox_parlidata_rdemos_sankeyFlow_html5" data-comments-opened="1" class="wp-image-16006" data-recalc-dims="1" data-attachment-id="16006" data-permalink="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/__dropbox_parlidata_rdemos_sankeyflow_html5/" alt="" data-medium-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html5.png?w=450#038;h=436?w=300" data-large-file="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html5.png?w=450#038;h=436?w=830" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://ouseful.files.wordpress.com/2017/03/dropbox_parlidata_rdemos_sankeyflow_html5.png?w=450#038;h=436"></p>
<p>As well as providing the code for generating each of the above Sankey diagrams, the Rmd file linked above also includes demonstrations for generating basic Sankey diagrams for the original dataset using the rCharts and htmlwidgets R libraries.</p>
<p>In order to provide a point of comparison, I also generated a python/pandas workflow using Jupyter notebooks and the <a target="_blank" rel="nofollow" href="https://github.com/ricklupton/ipysankeywidget"><code>ipysankey</code> widget</a>. (In fact, I generated the full workflow through the different chart versions first in <em>pandas</em> – I find it an easier language to think in than R! – and then used that workflow as a crib for the R version…)</p>
<p>The original notebook is <a target="_blank" rel="nofollow" href="https://github.com/psychemedia/parlihacks/blob/master/notebooks/MigrantFlow.ipynb">here</a> and an example of the HTML version of it <a target="_blank" rel="nofollow" href="https://psychemedia.github.io/parlihacks/MigrantFlow.html">here</a>. Note that I tried to save a rasterisation of the widgets but they don’t seem to have turned out that well…</p>
<p>The original (default) diagram looks like this:</p>
<p><img sizes="(max-width: 504px) 100vw, 504px" data-image-description="" data-orig-file="https://ouseful.files.wordpress.com/2017/03/migrantflow.png?w=450#038;h=380" data-orig-size="1512,1140" data-image-title="MigrantFlow" data-comments-opened="1" class="wp-image-16005" data-recalc-dims="1" data-attachment-id="16005" data-permalink="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/migrantflow/" alt="" data-medium-file="https://ouseful.files.wordpress.com/2017/03/migrantflow.png?w=450#038;h=380?w=300" data-large-file="https://ouseful.files.wordpress.com/2017/03/migrantflow.png?w=450#038;h=380?w=830" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://ouseful.files.wordpress.com/2017/03/migrantflow.png?w=450#038;h=380"></p>
<p>and the final version, after a bit of data wrangling, looks like this:</p>
<p><img sizes="(max-width: 469px) 100vw, 469px" data-image-description="" data-orig-file="https://ouseful.files.wordpress.com/2017/03/migrantflow2.png?w=450#038;h=438" data-orig-size="1212,1132" data-image-title="MigrantFlow2" data-comments-opened="1" class="wp-image-16004" data-recalc-dims="1" data-attachment-id="16004" data-permalink="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/migrantflow2/" alt="" data-medium-file="https://ouseful.files.wordpress.com/2017/03/migrantflow2.png?w=450#038;h=438?w=300" data-large-file="https://ouseful.files.wordpress.com/2017/03/migrantflow2.png?w=450#038;h=438?w=830" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://ouseful.files.wordpress.com/2017/03/migrantflow2.png?w=450#038;h=438"></p>
<p>Once again, all the code is provided in the notebook.</p>
<p>One of the nice things about all these packages is that they produce outputs than can be reused/embedded elsewhere, or that can be used as a first automatically produced draft of code that can be tweaked by hand. I’ll have more to say about <em>that</em> in a future post…</p>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/16002/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/16002/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://blog.ouseful.info/2017/03/17/experimenting-with-sankey-diagrams-in-r-and-python/"> Rstats – OUseful.Info, the blog…</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/E_kCazK4c3o"></a></div>