<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/plotting-background-data-for-groups-with-ggplot2/"></div></div><div><a data-url="https://www.r-bloggers.com/plotting-background-data-for-groups-with-ggplot2/" data-counturl="https://www.r-bloggers.com/plotting-background-data-for-groups-with-ggplot2/" data-count="horizontal" data-text="Plotting background data for groups with ggplot2" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://drsimonj.svbtle.com/plotting-background-data-for-groups-with-ggplot2"> blogR</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><a target="_blank" href="https://twitter.com/coolbutuseless/status/762956810680610816">This tweet</a> by <a target="_blank" href="https://twitter.com/coolbutuseless">mikefc</a> alerted me to a mind-blowingly simple but amazing trick using the <a target="_blank" href="https://cran.r-project.org/web/packages/ggplot2/index.html">ggplot2</a> package: to visualise data for different groups in a facetted plot with all of the data plotted in the background. Here’s an example that we’ll learn to make in this post so you know what I’m talking about:</p>
<p><a target="_blank" href="https://i0.wp.com/svbtleusercontent.com/ofzisqe5egsi2w.png?ssl=1"><img alt="init-example-1.png" src="https://i0.wp.com/svbtleusercontent.com/ofzisqe5egsi2w_small.png?w=456&ssl=1" data-recalc-dims="1"></a></p>
<h2><a target="_blank" href="http://drsimonj.svbtle.com/plotting-background-data-for-groups-with-ggplot2#credit-where-credit39s-due_2"> </a>Credit where credit’s due</h2>
<p>Before continuing, I’d be remiss for not mentioning that the origin of this ingenious suggestion is <a target="_blank" href="https://twitter.com/hadleywickham">Hadley Wickham</a>. The tip comes in his latest ggplot book, for which hardcopies are available online at places like <a target="_blank" href="https://www.amazon.com/dp/0387981403/ref=cm_sw_su_dp?tag=ggplot2-20">Amazon</a>, and the code and text behind it are freely available on Hadley’s Github at <a target="_blank" href="https://github.com/hadley/ggplot2-book">this repository</a>.</p>
<h2><a target="_blank" href="http://drsimonj.svbtle.com/plotting-background-data-for-groups-with-ggplot2#some-motivating-examples_2"> </a>Some motivating examples</h2>
<p>Let’s start with some examples that explain just why I’m so excited about this trick. Consider wanting to plot the results shown in the example above. That is, for the <code>iris</code> data set (that comes with R), we want to plot a histogram of the <code>Sepal.Width</code> variable, but separately for each flower <code>Species</code>. Let’s start by creating the histogram of all the data:</p>
<pre><code>library(ggplot2)
ggplot(iris, aes(x = Sepal.Width)) +
  geom_histogram()
</code></pre>
<p><a target="_blank" href="https://i1.wp.com/svbtleusercontent.com/dgojyid0pt1xw.png?ssl=1"><img alt="sepal_all-1.png" src="https://i0.wp.com/svbtleusercontent.com/dgojyid0pt1xw_small.png?w=456&ssl=1" data-recalc-dims="1"></a></p>
<p>So how can we present this separately for each of the flower <code>Species</code>? Well, here are two possibilities…</p>
<p>Coloured stacking via <code>fill</code>:</p>
<pre><code>ggplot(iris, aes(x = Sepal.Width, fill = Species)) +
  geom_histogram()
</code></pre>
<p><a target="_blank" href="https://i2.wp.com/svbtleusercontent.com/auhyxg8jnxis9w.png?ssl=1"><img alt="sepal_stack-1.png" src="https://i1.wp.com/svbtleusercontent.com/auhyxg8jnxis9w_small.png?w=456&ssl=1" data-recalc-dims="1"></a></p>
<p>Or separating the results into panels with <code>facet_wrap()</code>:</p>
<pre><code>ggplot(iris, aes(x = Sepal.Width)) +
  geom_histogram() +
  facet_wrap(~ Species)
</code></pre>
<p><a target="_blank" href="https://i2.wp.com/svbtleusercontent.com/x9yb1gtwrwuajq.png?ssl=1"><img alt="sepal_facet-1.png" src="https://i2.wp.com/svbtleusercontent.com/x9yb1gtwrwuajq_small.png?w=456&ssl=1" data-recalc-dims="1"></a></p>
<p>There’s certainly nothing wrong with these options, but there’s always room for improvement.</p>
<h2><a target="_blank" href="http://drsimonj.svbtle.com/plotting-background-data-for-groups-with-ggplot2#general-idea-for-adding-background-data_2"> </a>General idea for adding background data</h2>
<p>As the first example showed, a nice improvement (and the focus of the post) is to add the complete data set in the background of each facet panel. This way, it’s easy to interpret the data for any group relative to all other groups in the data set.</p>
<p>The following pseudo code describes a generic approach for implementing this trick:</p>
<pre><code>ggplot(data = full_data_frame, aes(...)) +
  geom_*(data = data_frame_without_grouping_var, colour/fill = &quot;neutral_color&quot;) +
  geom_*() +
  facet_wrap(~ grouping_var)
</code></pre>
<p>This pseudo code will become clearer as we try some examples. For the moment, the important thing to recognise is that the first geom layer we’re plotting uses data without the grouping variable, and will appear in some neutral colour. On top of this, we add the same sort of geom, but with the group data highlighted (e.g., with different colours). Finally, we facet the plot based on our grouping variable.</p>
<p>Let’s try it out with some examples.</p>
<h2><a target="_blank" href="http://drsimonj.svbtle.com/plotting-background-data-for-groups-with-ggplot2#histogram-example_2"> </a>Histogram example</h2>
<p>We’ll start by creating something similar to the example shown at the beginning.</p>
<pre><code>d &lt;- iris        # Full data set
d_bg &lt;- d[, -5]  # Background Data - full without the 5th column (Species)

ggplot(d, aes(x = Sepal.Width)) +
  geom_histogram(data = d_bg, fill = &quot;grey&quot;) +
  geom_histogram() +
  facet_wrap(~ Species)
</code></pre>
<p><a target="_blank" href="https://i2.wp.com/svbtleusercontent.com/ytxoe3ztfdcclg.png?ssl=1"><img alt="uni_1-1.png" src="https://i1.wp.com/svbtleusercontent.com/ytxoe3ztfdcclg_small.png?w=456&ssl=1" data-recalc-dims="1"></a></p>
<p>The above plot is a good start. It’s already easy to see how the distribution of each <code>Species</code> fits in with the complete data set.</p>
<p>So how does this work? The first important point is that we have a full data set that includes the variable by which to group our data (<code>d</code> with <code>Species</code>), and a background data set without this variable (<code>d_bg</code> without <code>Species</code>).</p>
<p>The <code>ggplot()</code> command sets up a general canvas with our full data set.</p>
<p>We then plot a <code>geom_histogram()</code> using the background data (<code>d_bg</code>) and <code>fill</code> it grey so as to give it a neutral appearance. It makes use of the <code>aes()</code> command within <code>ggplot()</code>, thus plotting the data we want.</p>
<p>On top of this, we plot another <code>geom_histogram()</code>. Because we haven’t specified a data set, it will use <code>d</code>, which is specified in <code>ggplot()</code>. This time, it will plot in the default colour, black.</p>
<p>Finally, we <code>facet_wrap()</code> the plot by <code>Species</code>. Because <code>Species</code> is not in the background data set, the output of our first <code>geom_histogram()</code> is replicated across all the panels. But because it is in the full data set used to plot the second <code>geom_histogram()</code> layer, this data gets split up.</p>
<p>Below is the code that makes this all a bit prettier by adding the colours and visual touch-ups that appear in the first example.</p>
<pre><code>d &lt;- iris        # Full data set
d_bg &lt;- d[, -5]  # Background Data - full without the 5th column (Species)

ggplot(d, aes(x = Sepal.Width, fill = Species)) +
  geom_histogram(data = d_bg, fill = &quot;grey&quot;, alpha = .5) +
  geom_histogram(colour = &quot;black&quot;) +
  facet_wrap(~ Species) +
  guides(fill = FALSE) +  # to remove the legend
  theme_bw()              # for clean look overall
</code></pre>
<p><a target="_blank" href="https://i0.wp.com/svbtleusercontent.com/pueunwz0vqsfiq.png?ssl=1"><img alt="uni-1.png" src="https://i2.wp.com/svbtleusercontent.com/pueunwz0vqsfiq_small.png?w=456&ssl=1" data-recalc-dims="1"></a></p>
<p>The code used to produce this is similar to the earlier pseudo code but with a few tweaks to improve the overall look of the plot. For visual improvements:</p>
<ul>
<li> The background data is made transparent with <code>alpha = .5</code>.</li>
<li> The second <code>geom_histogram()</code> layer makes use of the <code>fill = Species</code> in <code>ggplot()</code> to color each group differently. This adds a legend which is later removed with <code>guides(fill = FALSE)</code>.</li>
<li> The coloured bars are then outlined with <code>colour = &quot;black&quot;</code></li>
<li> Clean overall look with <code>theme_bw()</code></li>
</ul>
<h2><a target="_blank" href="http://drsimonj.svbtle.com/plotting-background-data-for-groups-with-ggplot2#points-example_2"> </a>Points example</h2>
<p>Let’s take a look at another example with the same data set.</p>
<p>Here it is following the pseudo code without any special tweaking:</p>
<pre><code>ggplot(d, aes(x = Sepal.Width, y = Sepal.Length)) +
  geom_point(data = d_bg, colour = &quot;grey&quot;) +
  geom_point() + 
  facet_wrap(~ Species)
</code></pre>
<p><a target="_blank" href="https://i1.wp.com/svbtleusercontent.com/uraskblx1lvvqq.png?ssl=1"><img alt="biv1-1.png" src="https://i2.wp.com/svbtleusercontent.com/uraskblx1lvvqq_small.png?w=456&ssl=1" data-recalc-dims="1"></a></p>
<p>And here it is with some visual improvements:</p>
<pre><code>ggplot(d, aes(x = Sepal.Width, y = Sepal.Length, colour = Species)) +
  geom_point(data = d_bg, colour = &quot;grey&quot;, alpha = .2) +
  geom_point() + 
  facet_wrap(~ Species) +
  guides(colour = FALSE) +
  theme_bw()
</code></pre>
<p><a target="_blank" href="https://i1.wp.com/svbtleusercontent.com/lnecveenipjq2w.png?ssl=1"><img alt="biv-1.png" src="https://i1.wp.com/svbtleusercontent.com/lnecveenipjq2w_small.png?w=456&ssl=1" data-recalc-dims="1"></a></p>
<p>This example plots the points of <code>Sepal.Width</code> against <code>Sepal.Length</code> separately for each <code>Species</code>. The logic is the same as above:</p>
<ul>
<li> Background data without the grouping variable (<code>Species</code>) plotted first in a neutral colour.</li>
<li> Full data set plotted on top of this.</li>
<li> The plot is facetted by the grouping variable, which only appears in the full data set.</li>
</ul>
<h2><a target="_blank" href="http://drsimonj.svbtle.com/plotting-background-data-for-groups-with-ggplot2#map-example_2"> </a>Map example</h2>
<p>Here’s one final example I’d like to share with you:</p>
<pre><code>library(nycflights13)
library(dplyr)

usa_map &lt;- map_data(&quot;usa&quot;)

airports &lt;- read.csv(&quot;https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat&quot;, stringsAsFactors = FALSE, header = FALSE)
airports &lt;- airports[, c(5, 7, 8)]
names(airports) &lt;- c(&quot;code&quot;, &quot;lat&quot;, &quot;long&quot;)
orig &lt;- airports %>% dplyr::rename(origin = code, long_o = long, lat_o = lat)
dest &lt;- airports %>% dplyr::rename(dest = code, long_d = long, lat_d = lat)

d &lt;- flights %>%
       left_join(orig) %>% 
       left_join(dest) %>% 
       filter(carrier %in% c(&quot;AS&quot;, &quot;F9&quot;, &quot;OO&quot;, &quot;YV&quot;, &quot;VX&quot;, &quot;FL&quot;))

d_bg &lt;- d %>% select(-carrier)

ggplot(d) +
  geom_polygon(data = usa_map, aes(long, lat, group = region)) +
  geom_segment(data = d_bg, colour = &quot;grey&quot;, alpha = .7,
               aes(x = long_o, y = lat_o,
                   xend = long_d, yend = lat_d)) +
  geom_segment(aes(x = long_o, y = lat_o,
                   xend = long_d, yend = lat_d,
                   colour = carrier)) +
  facet_wrap(~ carrier) +
  guides(colour = FALSE) +
  theme_bw()
</code></pre>
<p><a target="_blank" href="https://i1.wp.com/svbtleusercontent.com/lr7j9ptc0yge1a.png?ssl=1"><img alt="map-1.png" src="https://i2.wp.com/svbtleusercontent.com/lr7j9ptc0yge1a_small.png?w=456&ssl=1" data-recalc-dims="1"></a></p>
<p>This plot presents the flight paths for six airline carriers that fly in/out of New York (data from the nycflights13 package). All flight paths appear in the background of each plot, and the flight paths of the relevant carrier are highlighted.</p>
<p>The code that comes before <code>ggplot()</code> does all the data prep (which you can do with the appropriate packages installed). If you run the code yourself, you’ll find that you end up with a full data set, <code>d</code>, and a background data set without the flight carriers, <code>d_bg</code>. The plot itself is a bit different because we need to place the map in the background, but it’s the same logic as presented earlier.</p>
<h2><a target="_blank" href="http://drsimonj.svbtle.com/plotting-background-data-for-groups-with-ggplot2#sign-off_2"> </a>Sign off</h2>
<p>Thanks for reading and I hope this was useful for you.</p>
<p>For updates of recent blog posts, follow <a target="_blank" href="https://twitter.com/drsimonj">@drsimonj</a> on Twitter, or email me at <a target="_blank" href="mailto:drsimonjackson@gmail.com">drsimonjackson@gmail.com</a> to get in touch.</p>
<p>If you’d like the code that produced this blog, check out the <a target="_blank" href="https://github.com/drsimonj/blogR">blogR GitHub repository</a>.</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/plotting-background-data-for-groups-with-ggplot2/"></div></div><div><a data-url="https://www.r-bloggers.com/plotting-background-data-for-groups-with-ggplot2/" data-counturl="https://www.r-bloggers.com/plotting-background-data-for-groups-with-ggplot2/" data-count="vertical" data-text="Plotting background data for groups with ggplot2" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://drsimonj.svbtle.com/plotting-background-data-for-groups-with-ggplot2"> blogR</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/ZPMGnnTYnOY"></a></div>