<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/re-creating-a-voronoi-style-map-with-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/"> r on Tony ElHabr</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h1>Introduction</h1>
<p>I’ve written some “tutorial”-like content recently—see<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/texas-high-schools-academics-fuzzy-join" target="_blank">here</a>,<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/split-apply-combine-machine-learning-r" target="_blank">here</a>, and<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/nested-json-to-tidy-data-frame-r" target="_blank">here</a>—but<br>
I’ve been lacking on ideas for “original” content since then. With that said,<br>
I thought it would to try to re-create something with <code>R</code>. (Not too long ago<br>
I saw that<br>
<a rel="nofollow" href="https://www.andrewheiss.com/" target="_blank">Andrew Heiss</a> did something akin to this with<br>
<a rel="nofollow" href="https://www.andrewheiss.com/blog/2017/08/10/exploring-minards-1812-plot-with-ggplot2/" target="_blank">Charles Minard’s well-known visualization of Napoleon’s 1812</a>.)</p>
<p>The focus of my re-creation here is the price contour map shown on the<br>
front page of the website for the <a rel="nofollow" href="http://http//www.ercot.com/" target="_blank">Electric Reliability Council of Texas</a>,<br>
the independent system operator of electric power flow<br>
for about 90 percent of Texas residents, as well as the employer of yours truly).</p>
<p><img alt="" src="https://i2.wp.com/tonyelhabr.rbind.io/post/recreate-voronoi-map-r/rtmSpp-20181115-071500.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Without going too much into the details of what kind of information<br>
and insight the map is supposed to provide <sup><a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fn:1" target="_blank">1</a></sup>, it’s sufficient for the<br>
reader to simply understand the following:</p>
<ul>
<li>“cooler” colors (i.e. blue and green)<br>
generally indicate regions of electric energy stability—where electric power<br>
generation is sufficiently meeting electric demand at a given point<br>
in time;</li>
<li>and “warmer” colors (i.e. orange and red) generally represent the<br>
opposite—regions where there is some “stress” at a given point in time<br>
due to either the existence or threat of power imbalance.</li>
</ul>
<p>Anyways, aside from my familiarity with the map (just from visiting<br>
the website so often), I thought that it would<br>
be interesting to try to re-create it because it presents some “challenges”<br>
that I had not really tackled before (even in my previous endeavors<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/texas-high-schools-bubble-grid" target="_blank">with {ggplot2} and maps</a>)—namely,<br>
(1) manipulating <a rel="nofollow" href="https://developers.google.com/kml/documentation/" target="_blank">KML data</a><br>
and (2) creating <a rel="nofollow" href="https://en.wikipedia.org/wiki/Voronoi_diagram" target="_blank">Voronoi</a><br>
shapes. <sup><a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fn:2" target="_blank">2</a></sup></p>
<h1>Re-creation Process</h1>
<p>Unfortunately, the code needed to re-produce the map is not exactly trivial,<br>
so I’m going to leave details-hungry readers<br>
<a rel="nofollow" href="https://github.com/tonyelhabr/voronoi-contour" target="_blank">a link to the repository</a><br>
to explore the techniques that I used on their own time. <sup><a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fn:3" target="_blank">3</a></sup> To summarize the process,<br>
here’s what I did:</p>
<ol>
<li>
<p>I downloaded the price data CSVs for a specific settlement<br>
interval—7:15 AM interval on November 15, 2018—from public links<br>
on ERCOT’s website. <sup><a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fn:4" target="_blank">4</a></sup></p>
<ul>
<li>The “Settlement Point Prices at Resource Nodes, Hubs and Load Zones” link<br>
in the “Real-Time Prices Reports” section at <a rel="nofollow" href="http://www.ercot.com/mktinfo" target="_blank">http://www.ercot.com/mktinfo</a>.</li>
<li>The “DAM Settlement Point Prices” link in the “DAM Results Reports” section<br>
at <a rel="nofollow" href="http://www.ercot.com/mktinfo" target="_blank">http://www.ercot.com/mktinfo</a>.</li>
</ul>
<p>Additionally, I downloaded the “static” KML data upon which the map is built.</p>
<ul>
<li><a rel="nofollow" href="http://www.ercot.com/content/cdr/static/texas_counties.kml" target="_blank">http://www.ercot.com/content/cdr/static/texas_counties.kml</a></li>
<li><a rel="nofollow" href="http://www.ercot.com/content/cdr/static/ercot_boundary.kml" target="_blank">http://www.ercot.com/content/cdr/static/ercot_boundary.kml</a></li>
</ul>
<p>Finally, I downloaded “dynamic” KML data representing the price points<br>
themselves (for both the Real-Time Market (RTM) and the Day-Ahead Market (DAM).</p>
<ul>
<li><a rel="nofollow" href="http://www.ercot.com/content/cdr/contours/rtmSppPoints.kml" target="_blank">http://www.ercot.com/content/cdr/contours/rtmSppPoints.kml</a></li>
<li><a rel="nofollow" href="http://www.ercot.com/content/cdr/contours/damSpp7Points.kml" target="_blank">http://www.ercot.com/content/cdr/contours/damSpp7Points.kml</a></li>
</ul>
<p>I should Note that with this step and the others that follow,<br>
I repeated the actions for both data sets—those representing the<br>
RTM and DAM prices and points.</p>
</li>
<li>
<p>With the KML files for the price points, I extracted the<br>
longitude and latitude value pairs from the points defined in the KML<br>
using the <a rel="nofollow" href="https://github.com/briatte/tidykml" target="_blank"><code>{tidykml}</code> package</a>.<br>
Additionally, I converted the legend (embedded in the KML) into a vector<br>
to be used for the final recreated plot(s) themselves. <sup><a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fn:5" target="_blank">5</a></sup></p>
</li>
<li>
<p>With the CSVs with the price data, I did a bit of “data munging” to convert<br>
the data for the two markets into an equivalent, easy-to-work-with format, and<br>
I joined this with the coordinate data from the previous step.</p>
</li>
<li>
<p>With the KML files representing the map’s static data, I created<br>
a <code>SpatialPolygonsDataFrame</code> encompassing both the Texas state and county boundaries,<br>
as well as the ERCOT region boundaries (which is separate).</p>
<p>This was probably the most difficult part for me to figure out. Nonetheless,<br>
with the help of the R community’s packages geared towards working with spatial<br>
data—most notably, the <a rel="nofollow" href="https://cran.r-project.org/web/packages/sp/index.html" target="_blank"><code>{sp}</code></a><br>
and <a rel="nofollow" href="https://cran.r-project.org/web/packages/sf/index.html" target="_blank"><code>{sf}</code></a> packages—as well as some great<br>
documentation and tutorials—I found<br>
<a rel="nofollow" href="https://mgimond.github.io/Spatial/interpolation-in-r.html" target="_blank">this one</a><br>
particularly helpful, although one may consider its techniques “outdated” since it doesn’t<br>
utilize <code>{sf}</code> for interpolation. <sup><a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fn:6" target="_blank">6</a></sup></p>
</li>
<li>
<p>Next was the “heart” of the process—the interpolation to create the Voronoi<br>
shapes. <sup><a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fn:7" target="_blank">7</a></sup> This may be the part that is most interesting to the reader,<br>
since a similar process could be applied to one’s own project. <sup><a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fn:8" target="_blank">8</a></sup></p>
</li>
<li>
<p>Finally, I used <code>{ggplot2}</code> to create the visualizations themselves.<br>
Notably, one must be careful with the order of calls to<br>
<code>geom_polygon()</code>, <code>geom_sf()</code>, and <code>coord_sf()</code> in order to get<br>
the ordering of layers correct.</p>
</li>
</ol>
<p>As for other details about the implementation, this process provided a perfect<br>
example of how/why functions can be so useful—it was really nice to just call<br>
a custom function twice to repeat each step for<br>
the two data sets, which certainly saved me some time and effort.</p>
<h1>The Maps</h1>
<p>So what does the final product look like? Below, I’ve placed my re-creations<br>
and the actual image side by side (with the help of the<br>
<a rel="nofollow" href="https://cran.r-project.org/web/packages/magick/index.html" target="_blank"><code>{magick}</code> package</a>).</p>
<p><img alt="" src="https://i2.wp.com/tonyelhabr.rbind.io/post/recreate-voronoi-map-r/viz_pnt_rtm_compare.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p><img alt="" src="https://i2.wp.com/tonyelhabr.rbind.io/post/recreate-voronoi-map-r/viz_pnt_dam_compare.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Not bad, right? OK, so the sizing isn’t perfect, but I would say it’s close enough.</p>
<p>To take things one step further, I tried “subtracting” the values between the two<br>
re-created maps. There is some value to doing this—to visualize the discrepancies<br>
between prices in the two markets (which, hypothetically, should be equal).</p>
<p><img alt="" src="https://i1.wp.com/tonyelhabr.rbind.io/post/recreate-voronoi-map-r/viz_pnt_rtm2dam.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Because this map displays information that is inherently different than those<br>
of the stand-alone RTM and DAM maps, I used a different color<br>
palette—specifically, the “E” palette that comes with the<br>
awesome <a rel="nofollow" href="https://cran.r-project.org/web/packages/viridis/index.html" target="_blank"><code>{viridis}</code> package</a>,<br>
which is conveniently made available in<br>
<a rel="nofollow" href="https://www.tidyverse.org/articles/2018/07/ggplot2-3-0-0/" target="_blank"><code>{ggplot2}</code> ever since the v3 release</a>.</p>
<h1>Final Thoughts</h1>
<p>I really enjoyed this challenge to myself. I can certainly see myself doing something<br>
similar in the future, perhaps with a focus with which I’m less familiar (to give<br>
myself an even greater challenge). <sup><a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fn:9" target="_blank">9</a></sup></p>
<hr>
<div>
<hr>
<ol>
<li><a rel="nofollow" href="http://www.ercot.com/help/popup/lmpContourMapHelp" target="_blank">Check out the pop-up help menu</a> for a bit more information about the map itself and <a rel="nofollow" href="https://www.e-education.psu.edu/ebf483/node/647" target="_blank">this link</a> for more information about Locational Marginal Prices (LMPs).<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fnref:1" target="_blank"><sup>^</sup></a></li>
<li>If you’ve never heard of <a rel="nofollow" href="https://en.wikipedia.org/wiki/Voronoi_diagram" target="_blank">Voronoi diagrams</a>, I would highly suggest reading a bit about them—the math behind them is fairly straightforward, and the results are quite elegant.<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fnref:2" target="_blank"><sup>^</sup></a></li>
<li>I wouldn’t normally use the <code>if ([... exists]) then [do ...] else [...]</code> technique heavily like I do in the “main” script in the repository (because I don’t think it’s exactly a “best practice” and I’d rather use functions to achieve the same thing), but this idiom fit what I was trying to achieve at the time.<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fnref:3" target="_blank"><sup>^</sup></a></li>
<li>Note that the map updates every 5 minutes for Real-Time Market (RTM) Locational Marginal Prices (LMPs) and every 15 minutes for RTM Settlement Point Prices (SPPs). Additionally, the Day-Ahead Market (DAM) LMP and SPP maps are update once per day, with filters allowing user to browse prices at a specified hour.<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fnref:4" target="_blank"><sup>^</sup></a></li>
<li>There was some serous “hard-coding” needed to do this.<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fnref:5" target="_blank"><sup>^</sup></a></li>
<li>The <a rel="nofollow" href="https://cran.r-project.org/web/packages/tidyverse/index.html" target="_blank"><code>{tidyverse}</code> set of packages</a>, as well as <a rel="nofollow" href="https://cran.r-project.org/web/packages/rvest/index.html" target="_blank"><code>{rvest}</code></a> and <a rel="nofollow" href="https://cran.r-project.org/web/packages/xml2/index.html" target="_blank"><code>{xml2}</code></a> were also instrumental in this process.<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fnref:6" target="_blank"><sup>^</sup></a></li>
<li>As with the previous step, <code>R</code>’s spatial packages were extremely helpful here.<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fnref:7" target="_blank"><sup>^</sup></a></li>
<li>The rest of the steps may be too project-specific to be “generalizable”.<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fnref:8" target="_blank"><sup>^</sup></a></li>
<li>Just brainstorming, I really enjoyed <a rel="nofollow" href="https://fivethirtyeight.com/features/how-beto-orourke-shifted-the-map-in-texas/" target="_blank">Kirk Goldsberry’s piece post-2018-midterm elections</a> (mostly for the infomrative and “clean” visuals). The maps are practically begging to be re-created!<br>
<a rel="nofollow" href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/#fnref:9" target="_blank"><sup>^</sup></a></li>
</ol>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://tonyelhabr.rbind.io/post/recreate-voronoi-map-r/"> r on Tony ElHabr</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/-Wj1GFhPetI" height="1"></a></div>