Why is this about trucks?
Last month, at the R/Pharma conference that took place on the Harvard Campus, I presented bioWARP, a large Shiny application containing more than 500,000 lines of code. Although several other Shiny apps were presented at the conference, I noticed that none of them came close to being as big as bioWARP. And I asked myself, why?
I concluded that most people just don’t need to build them that big! So now, I would like to explain why we needed such a large app and how we went about building it.
To give you an idea of the scale I am talking about, an automotive metaphor might be useful. A typical Shiny app I see in my daily work has about 50 or even fewer interaction items. Let’s imagine this as a car. With less than 50 interactions, think of a small car like a Mini Cooper. Compared to these applications, with more than 500 interactions, bioWARP is a truck, maybe even a “monster” truck. So why do my customers want to drive trucks when everyone else is driving cars?
Images by Paul V and DaveR
Why do we need a truck?
Building software often starts with checking the user requirements. So when we started the development of our statistical web application, we did that, too. Asking a lot of people inside our department, we noticed that the list of requirements was huge:
Main user requirements
Pretty design that works universally
Interactive elements
Mathematical correctness of all results
Main application features
Session logging
Standardized PDF reports of all results
Ability to restore sessions
Harmonize it with other software applications
Everything has to be tested
Help pages
More requirements came then from all the analysis people perform on daily basis. They wanted to have some tasks integrated into our app:
Mathematical tasks
Linear regression app
Descriptive statistics app
Homogeneity test app
T-Test app
Bootstrap simulation app
Sensitivity/Specificity app
Linearity app
Clustering app
BoxPlotting app
Additionally, it was required to write the whole application in R, as all our mathematical packages are written in R. So we decided to do it all with shiny because it already covers two of the three main user requirements: being pretty and being interactive.
How did we build the truck?
Modularity + Standardization
Inside our department, we were running some large-scale desktop applications already. When it came to testing, we always noticed that testing takes forever. If one single piece of software gathers data, calculates statistics, provides plot outputs, and renders PDF reports, this is a huge truck and you can just test it by driving it a thousand miles and see if it still works. The idea we came up with was building our truck out of Lego bricks. Each Lego brick can be tested on its own. If a Lego wheel runs, the truck will run. The wheel holder part is universal, and if we change the size of the wheels, we can still run the truck, assuming each wheel was tested. This is called modularity. There exist different solutions in R and shiny that can be combined to make things modular:
Shiny Modules
Object orientation
R-packages
Clever name-spacing
As Shiny modules did not exist when we started, we chose options 2 and 3.
As an example, I’ll compare two simple Shiny apps representing two cars here. One is written using object orientation; one is a simple Shiny application. The image below illustrates that the renderPlot function in a standard Shiny app includes a plot, in this case using the hist function. So whenever you add a new plot, its function has to be called inside.
In the object-oriented app, the renderPlot function calls the shinyElement method of a generic plot object we created called AnyPlot. The first advantage is that the plot can easily be exchanged. (Please look into the code if you wonder if this really is so.) To describe that advantage, you can imagine a normal car, built of car parts. Our car is really a a Lego car, using even smaller standardized parts (Lego bricks) to construct each part of the car. So instead of the grille made of one piece of steel, we constructed it of many little grey Lego bricks. Changing the grille for an update of the car does not require us to reconstruct the whole front; just use green bricks instead of grey bricks, for example, since they should have the same shape.
By going into the code of the two applications, you see there is a straight-forward disadvantage of object orientation: there is much more code. We have to define what a Lego brick is and what features it shall have.
Object oriented shiny app
library(methods)
library(rlang)
setGeneric(&quot;plotElement&quot;,where = parent.frame(),def = function(object){standardGeneric(&quot;plotElement&quot;)})
setGeneric(&quot;shinyElement&quot;,where = parent.frame(),def = function(object){standardGeneric(&quot;shinyElement&quot;)})
setClass(&quot;AnyPlot&quot;, representation(plot_element = &quot;call&quot;))
setClass(&quot;HistPlot&quot;, representation(color=&quot;character&quot;,obs=&quot;numeric&quot;), contains = &quot;AnyPlot&quot;)
AnyPlot