Provided different probability distributions, animate independent sample distributions to demonstrate mode variability. There is a thorough code description with some subtle tips and tricks.
Prologue
Not so long time ago I encountered the following task: given several groups of samples (one group - several samples from one distribution) make a visual presentation of sample mode variability in different groups (how sample mode changes between different samples). I decided to do this by animating these distributions with respect to their mode. The whole process can be summarised as follows:
For every sample:
Count its values. Sample values are considered to a certain degree of precision, e.g. to the third digit, so talking about “counting values” has reasonable background.
Divide counts by maximum count. The output is named “mode share distribution” (made up term, didn’t find any present one). For every unique value in sample the outcome is a number between 0 (value is totally absent) and 1 (value is a strict mode). Here value is considered to be “mode” if its “mode share” is above some threshold, say 0.9. For strict thresholds 1 is used.
For every set of group samples (first samples within groups, second samples, and so on) plot their “mode share distribution”. This is done by plotting several “mode share distributions” at once, in non-overlapping ridgeline-like fashion.
Animate between plots. This simultaneously shows a mode variability in different groups. As just changing pictures didn’t seem like a challenge, I decided to make smooth transitions between plots. Packages tweenr (CRAN version) and gganimate (original one by David Robinson, not a soon-to-be-revolutional one by Thomas Lin Pedersen) provide core functionality for this.
Methods I ended up using (some of them were discovered during solving this task) contain surprisingly many subtle base R and tidyverse tricks. This motivated me to write a post about my solution. It heavily uses core tidyverse packages.
For educational purposes, I will slightly change the task: provided different probability distributions (by random generation R functions like rnorm), animate independent sample distributions to demonstrate mode variability.
Overview
The main point of this post is to provide a thorough code description with some highlighted tips and tricks. Considering this, the most information will be provided in code comments with some tricks described in text. I can see that provided solutions might be slightly overcomplicated but they seem to demonstrate stable behavior. Please note that this post, probably, won’t be that helpful for beginners in R, and some confident knowledge of R and tidyverse tools is needed.
Most of the code will be hidden under spoilers (text which appears after clicking on its summary, usually beginning with “Code for …”) to shrink initial post size, but I really encourage you to read them. However, only looking at plots and animations is, of course, completely fine :) They are considered to be a valuable by-product.
This post is organized as follows:
Data generation describes the process of creating new samples from given random generators.
Static plots describes the format of plotting data and a process of building a plot. There are also some example plots.
Animation describes the logic behind transition between different plots and the process of creating animations. There are also some example animations.
Bonus accidental art has an example of “accidental art” which was produced unintentionally but has an interesting look.
Conclusion lists some lessons learned and some of my favorite tricks from this post.
We will need the following setup:
Code for setup
# Used tidyverse packages
suppressPackageStartupMessages(library(dplyr))
library(tidyr)
library(purrr)
library(ggplot2)
# Using current CRAN version 0.1.5
library(tweenr)
# Using current github 'dgrtwo/gganimate' version 0.1.0.9000 (commit bf82002)
library(gganimate)
theme_set(theme_bw())
animation::ani.options(interval = 1/24)
# Very important step for reproducibility
set.seed(20180614)
# Data about distributions of interest and sample size
r_funs %
mutate(distr = factor(distr, levels = c(&quot;runif&quot;, &quot;rnorm&quot;, &quot;rbeta&quot;))) %__%
get_plot_data()
## # A tibble: 297 x 7
## distr x y yend isMode colour size
## ## 1 runif 0 1 1.28 FALSE black 1
## 2 runif 0.01 1 1.39 FALSE black 1
## 3 runif 0.02 1 1.51 FALSE black 1
## 4 runif 0.03 1 1.51 FALSE black 1
## 5 runif 0.04 1 1.45 FALSE black 1
## # ... with 292 more rows
Tips and tricks:
While plotting several distributions organized vertically, one can compute the base y coordinate for them with as.integer(as.factor(distr)). Using factor distr column is a way to control vertical order of distributions.
Creating plots
Code for creating static plots
# Adds segment layers. May be used in both static and animated plots.
# In static plots should be preceded by `ggplot(data)` in which `data` should
# have column `.frame` with all 1.
# In animated plots should be preceded by
# `ggplot(data_tweened, aes(frame = .frame))`.
# Returns a list with results of `ggplot2` constructor functions. Elements will
# be used sequentially to build plot. This list should be added to `ggplot()`
# call with `+` function (like other `ggplot2` functionality).
add_layers % filter(!isMode), mapping = common_aes
),
geom_segment(data = . %__% filter(isMode), mapping = common_aes),
# Explicitly label distributions.
geom_label(
data = . %__%
# Compute all possible labels in case of factor `distr`
distinct(.frame, distr) %__%
complete(.frame, distr) %__%
# Position label on the left side and in the middle of the group plot
mutate(x = -Inf, y = as.integer(as.factor(distr)) + 0.5),
mapping = aes(x = x, y = y, label = distr),
size = 5, hjust = -0.1
),
# Tweak axes labels
scale_x_continuous(name = NULL),
scale_y_continuous(
name = NULL, labels = NULL, breaks = NULL, minor_breaks = NULL
),
scale_colour_identity(),
scale_size_identity(),
# Manually fix plot range for x axis to (0, 1).
coord_cartesian(xlim = c(0, 1)),
# Add common subtitle to describe plot. Wrap `labs()` in `list()` as
# `labs()` itself returns a list which will be appended in a wrong way.
list(labs(
subtitle = paste0(
c(&quot;Rows represent counts of samples from different distributions.&quot;,
&quot;Actual counts are normalized by maximum count to plot mode share.&quot;,
&quot;Modes (in red) are values with mode share above some threshold.&quot;),
collapse = &quot;\n&quot;
)
))
)
}
# Wrapper for constructing static plot
make_static_plot %
mutate(.frame = 1) %__%
ggplot() +
add_layers() +
labs(title = title)
}
Function make_static_plot() expects plot data (as output of get_plot_data()) and title of the plot. It returns a ggplot object with many segments and zooming into (0, 1) in x-axis.
Tips and tricks:
For effective code reuse while “ggplotting”, one can create custom function custom_fun() which should return a list with results of ggplot2 constructor functions. Usually, they are geoms but can be scales, theme elements and so on. This function can then be used as function layer with ggplot(data) + custom_fun().
Value of data argument in layer function can be a function! This one I discovered during solving this task and was pleasantly surprised by this functionality. A function will be applied to present data initialized in ggplot() call. It is very useful when one needs to plot object for one subset of data over the other. A handy way to create those functions are with . %__% ... notation which creates a functional sequence (one of the features of magrittr pipe). Here this trick is used to plot segments for mode values over the other segments.
Example plots
distr_data %__%
get_plot_data() %__%
make_static_plot(
title = &quot;Mode share distributions for samples rounded to 2 digits&quot;
)
distr_data %__%
get_plot_data(mode_thres = 0.8) %__%
make_static_plot(
title = paste0(
c(&quot;Mode share distributions for samples rounded to 2 digits.&quot;,
&quot;Relaxed definition of mode&quot;),
collapse = &quot; &quot;
)
)
distr_data %__%
mutate(distr = factor(distr, levels = c(&quot;rnorm&quot;, &quot;runif&quot;, &quot;rbeta&quot;))) %__%
get_plot_data() %__%
make_static_plot(
title = &quot;Control order of distributions with factor `distr` column&quot;
)
Animations
The general idea behind animations is to take a list of state plotting data and compute transitions between them: multiple valid plotting data for transition states between consecutive state plotting data. All “transition plotting data” will be plotted consecutively. The output sequence of plots will be animated with relatively high speed to produce smooth animation.
Transitions
Code for creating transitions
# Analogous to `findInterval()` but:
# - Returns not the index of &quot;left&quot; `vec` element of interval but index of
# nearest element between two.
# - `vec` can be unordered.
find_nearest_ind %
# Convert `distr` to factor with common levels to ensure animation stability
map(~ mutate(., distr = factor(distr, levels = sort(names(r_funs))))) %__%
# Remove distributions in tables
map2(names(r_funs), ~ filter(.x, as.character(distr) != .y)) %__%
# The same animation process
map(. %__% mutate(value = round(value, 2))) %__%
map(get_plot_data) %__%
make_animation(
title = &quot;Automated animation for different distribution sets&quot;
)
# Evolution of modes for different mode thresholds
map(
seq(from = 1, to = 0, by = -0.05),
~ get_plot_data(sample_data[[1]] %__% mutate(value = round(value, 2)), .x)
) %__%
make_animation(
title = &quot;Evolution of modes for different mode thresholds (from 1 to 0)&quot;,
nframes = 6
)
# Mode variability of one sample with different roundings
map2(sample_data[1], 3:1, ~ mutate(.x, value = round(value, .y))) %__%
map(get_plot_data) %__%
make_animation(
title = &quot;Mode variability for different roundings of one sample&quot;,
tweenlength = 3, statelength = 1
)
Bonus accidental art
This piece of “accidental art” I created unintentionally when supplied tweened plot data for different roundings to make_static_plot():
map2(sample_data[1], 3:1, ~ mutate(.x, value = round(value, .y))) %__%
map(get_plot_data) %__%
tween_cycled() %__%
make_static_plot(title = &quot;Accidental art&quot;)
Conclusion
Sometimes a rather simple task can lead to pretty complicated solutions (if one feels like accepting a challenge).
Visualization of mode variability for different distributions can be done by creating animation with transitions between independent sample distributions (highlighting modes).
Along the journey of solving small tasks, several tips and tricks were found:
purrr::pmap_dfr() helps create a data frame output after iterating in parallel over multiple vectors. Shorthand formula notation for function should be used with caution when one of the input vectors contain functions to be applied.
While plotting several distributions organized vertically, one can compute the base y coordinate for them with as.integer(as.factor(distr)) (distr being name of the distribution stored as character or factor).
For efficient “ggplotting” one can create custom function custom_fun() which should return a list with results of ggplot2 constructor functions. It can then be used as function layer with ggplot(data) + custom_fun().
Value of data argument in ggplot2 layer function can itself be a function! It will be applied to present data initialized in ggplot() call. It is very useful when one needs to plot objects for one subset of data over the other. A handy way to create those functions are with . %__% ... notation which creates a functional sequence (one of the features of magrittr pipe).
Index of the nearest reference point can be found by a slight modification of findInterval() (used with all.inside = TRUE) with adding 1 when right end of interval is closer than left one.
Using full_join() can be very helpful in situation with different sets of groups. It enables later repairing of unmatched data.
One can change resolution (for example, to 100 ppi) of output animation in gganimate by supplying ani.dev = function(...) {png(res = 100, ...)}.
sessionInfo()
sessionInfo()
## R version 3.4.4 (2018-03-15)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 16.04.4 LTS
## ## Matrix products: default
## BLAS: /usr/lib/openblas-base/libblas.so.3
## LAPACK: /usr/lib/libopenblasp-r0.2.18.so
## ## locale:
## [1] LC_CTYPE=ru_UA.UTF-8 LC_NUMERIC=C ## [3] LC_TIME=ru_UA.UTF-8 LC_COLLATE=ru_UA.UTF-8 ## [5] LC_MONETARY=ru_UA.UTF-8 LC_MESSAGES=ru_UA.UTF-8 ## [7] LC_PAPER=ru_UA.UTF-8 LC_NAME=C ## [9] LC_ADDRESS=C LC_TELEPHONE=C ## [11] LC_MEASUREMENT=ru_UA.UTF-8 LC_IDENTIFICATION=C ## ## attached base packages:
## [1] methods stats graphics grDevices utils datasets base ## ## other attached packages:
## [1] bindrcpp_0.2.2 gganimate_0.1.0.9000 tweenr_0.1.5 ## [4] ggplot2_2.2.1 purrr_0.2.5 tidyr_0.8.1 ## [7] dplyr_0.7.5 ## ## loaded via a namespace (and not attached):
## [1] Rcpp_0.12.17 pillar_1.2.3 compiler_3.4.4 plyr_1.8.4 ## [5] bindr_0.1.1 tools_3.4.4 digest_0.6.15 evaluate_0.10.1 ## [9] tibble_1.4.2 gtable_0.2.0 pkgconfig_2.0.1 rlang_0.2.1 ## [13] cli_1.0.0 yaml_2.1.19 blogdown_0.6 xfun_0.1 ## [17] stringr_1.3.1 knitr_1.20 rprojroot_1.3-2 grid_3.4.4 ## [21] tidyselect_0.2.4 glue_1.2.0 R6_2.2.2 rmarkdown_1.9 ## [25] animation_2.5 bookdown_0.7 magrittr_1.5 backports_1.1.2 ## [29] scales_0.5.0 htmltools_0.3.6 assertthat_0.2.0 colorspace_1.3-2
## [33] labeling_0.3 utf8_1.1.4 stringi_1.2.2 lazyeval_0.2.1 ## [37] munsell_0.4.3 crayon_1.3.4