<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/how-to-build-a-shiny-truck/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rviews.rstudio.com/2018/09/04/how-to-build-shiny-trucks-not-shiny-cars/"> R Views</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<h2>Why is this about trucks?</h2>
<p>Last month, at the <a rel="nofollow" href="https://www.rinpharma.com/" target="_blank">R/Pharma</a> conference that took place on the Harvard Campus, I presented bioWARP, a large <a rel="nofollow" href="https://shiny.rstudio.com/" target="_blank">Shiny</a> application containing more than 500,000 lines of code. Although several other Shiny apps were presented at the conference, I noticed that none of them came close to being as big as bioWARP. And I asked myself, why?</p>
<p>I concluded that most people just don’t need to build them that big! So now, I would like to explain why we needed such a large app and how we went about building it.</p>
<div align="justify">
<p>To give you an idea of the scale I am talking about, an automotive metaphor might be useful. A typical Shiny app I see in my daily work has about 50 or even fewer interaction items. Let’s imagine this as a car. With less than 50 interactions, think of a small car like a Mini Cooper. Compared to these applications, with more than 500 interactions, bioWARP is a truck, maybe even a “monster” truck. So why do my customers want to drive trucks when everyone else is driving cars?</p>
<p>Images by <a rel="nofollow" href="https://flic.kr/p/B4TwtZ" target="_blank">Paul V</a> and <a rel="nofollow" href="https://flic.kr/p/q33yzD" target="_blank">DaveR</a></p>
</div>
<div>
<a rel="nofollow" data-flickr-embed="true" href="https://www.flickr.com/photos/85090026@N06/23016489623/in/photolist-B4TwtZ-9AAG2n-ATVCE2-8H7BXs-bWEepB-b5oGq6-9ScLZx-nsQVXP-SwRfJn-p4pBSb-s8qk1A-RdvAzc-UTegWT-V5ik7W-nFDVyu-U4hhS5-S1YzQ4-TNJKtE-VMa34Q-dbBA7T-dxY87H-AUKEtM-BSURfW-R8PE1f-TxSqCy-Rn42kf-UWcGq3-ni1aoM-b5tFPc-cHPh5h-boLdCo-dm5ESK-9RCSui-aWp9Z8-8C8Ng3-9A1yQq-azvuBD-SsHkof-Tvu42W-RnbZWN-qHYHjN-of5Sks-BhCWbB-pqEKus-8CFqHD-aSwrqB-gcnXM8-dD4kxS-eNo35G-RuQVQa" title="Peterbilt Truck" target="_blank"><img alt="Peterbilt Truck" src="https://i0.wp.com/farm1.staticflickr.com/743/23016489623_736b8a0fe9_k.jpg?w=47%25&ssl=1" data-recalc-dims="1"></a><br>
<br>
<a rel="nofollow" data-flickr-embed="true" href="https://www.flickr.com/photos/paulsimpsonphotography/13976701169/in/photolist-ni5fjn-hEgG2w-dTbDzB-abjfn3-3taEfy-dLxzEr-5yi6v6-jsLLQY-83tFh7-n2RGZ-q33yzD-ajnH6q-ndAoYs-ejZRJ2-FWSHe-dT9v4V-8JBRm6-8JBMHn-ag2k7S-ggMQTe-8JBRUa-nuN77T-dVSEWm-poHbs9-ERMU7x-51keup-e87ZL9-6FMLjc-sbUX5C-eahQCz-cdeN9f-5YJau2-nGsU7-E2HYa-9YM6rR-5RfA6G-6FMQFi-5qSaux-9tMCaH-aTH7rc-5bk6Eq-6ndgG5-2Vj84C-h1SGnS-5gsVwN-m7CnAF-6UeYKd-32NNds-fzmfft-8kMdQ2" title="Red Mini Cooper" target="_blank"><img alt="Red Mini Cooper" src="https://i2.wp.com/farm3.staticflickr.com/2902/13976701169_cb0c28b2b3_z.jpg?w=49%25&ssl=1" data-recalc-dims="1"></a><br>

</div>
<div>
</div>
</div>
<div>
<h2>Why do we need a truck?</h2>
<p>Building software often starts with checking the user requirements. So when we started the development of our statistical web application, we did that, too. Asking a lot of people inside our department, we noticed that the list of requirements was huge:</p>
<p><strong>Main user requirements</strong></p>
<ul>
<li>Pretty design that works universally</li>
<li>Interactive elements</li>
<li>Mathematical correctness of all results</li>
</ul>
<p><strong>Main application features</strong></p>
<ul>
<li>Session logging</li>
<li>Standardized PDF reports of all results</li>
<li>Ability to restore sessions</li>
<li>Harmonize it with other software applications</li>
<li>Everything has to be tested</li>
<li>Help pages</li>
</ul>
<p>More requirements came then from all the analysis people perform on daily basis. They wanted to have some tasks integrated into our app:</p>
<p><strong>Mathematical tasks</strong></p>
<ul>
<li>Linear regression app</li>
<li>Descriptive statistics app</li>
<li>Homogeneity test app</li>
<li>T-Test app</li>
<li>Bootstrap simulation app</li>
<li>Sensitivity/Specificity app</li>
<li>Linearity app</li>
<li>Clustering app</li>
<li>BoxPlotting app</li>
</ul>
<p>Additionally, it was required to write the whole application in <strong>R</strong>, as all our mathematical packages are written in <strong>R</strong>. So we decided to do it all with <a rel="nofollow" href="https://shiny.rstudio.com/" target="_blank"><code>shiny</code></a> because it already covers two of the three main user requirements: being pretty and being interactive.</p>
</div>
<div>
<h2>How did we build the truck?</h2>
<div>
<h3>Modularity + Standardization</h3>
<p>Inside our department, we were running some large-scale desktop applications already. When it came to testing, we always noticed that testing takes forever. If one single piece of software gathers data, calculates statistics, provides plot outputs, and renders PDF reports, this is a huge truck and you can just test it by driving it a thousand miles and see if it still works. The idea we came up with was building our truck out of Lego bricks. Each Lego brick can be tested on its own. If a Lego wheel runs, the truck will run. The wheel holder part is universal, and if we change the size of the wheels, we can still run the truck, assuming each wheel was tested. This is called modularity. There exist different solutions in R and <code>shiny</code> that can be combined to make things modular:</p>
<ol>
<li>Shiny Modules</li>
<li>Object orientation</li>
<li>R-packages</li>
<li>Clever name-spacing</li>
</ol>
<p>As Shiny modules did not exist when we started, we chose options 2 and 3.</p>
<p>As an example, I’ll compare two simple Shiny apps representing two cars here. One is written using object orientation; one is a simple Shiny application. The image below illustrates that the <code>renderPlot</code> function in a standard Shiny app includes a plot, in this case using the <code>hist</code> function. So whenever you add a new plot, its function has to be called inside.</p>
<p>In the object-oriented app, the <code>renderPlot</code> function calls the <code>shinyElement</code> method of a generic plot object we created called <code>AnyPlot</code>. The first advantage is that the plot can easily be exchanged. (Please look into the code if you wonder if this really is so.) To describe that advantage, you can imagine a normal car, built of car parts. Our car is really a a Lego car, using even smaller <strong>standardized</strong> parts (Lego bricks) to construct each part of the car. So instead of the grille made of one piece of steel, we constructed it of many little grey Lego bricks. Changing the grille for an update of the car does not require us to reconstruct the whole front; just use green bricks instead of grey bricks, for example, since they should have the same shape.</p>
<p>By going into the code of the two applications, you see there is a straight-forward disadvantage of object orientation: there is much more code. We have to define what a Lego brick is and what features it shall have.</p>
<div>
<img src="https://i1.wp.com/rviews.rstudio.com/post/2018-08-28-wolf-trucks_files/apps.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p><a rel="nofollow" href="https://github.com/zappingseb/biowarptruck/blob/master/example_apps/app_object.R" target="_blank">Object oriented shiny app</a></p>
<pre><code>library(methods)
library(rlang)


setGeneric(&quot;plotElement&quot;,where = parent.frame(),def = function(object){standardGeneric(&quot;plotElement&quot;)})
setGeneric(&quot;shinyElement&quot;,where = parent.frame(),def = function(object){standardGeneric(&quot;shinyElement&quot;)})

setClass(&quot;AnyPlot&quot;, representation(plot_element = &quot;call&quot;))
setClass(&quot;HistPlot&quot;, representation(color=&quot;character&quot;,obs=&quot;numeric&quot;), contains = &quot;AnyPlot&quot;)

AnyPlot &lt;- function(plot_element=expr(plot(1,1))){
  new(&quot;AnyPlot&quot;,
      plot_element = plot_element
  )
}

HistPlot &lt;- function(color=&quot;darkgrey&quot;,obs=100){
  new(&quot;HistPlot&quot;,
      plot_element = expr(hist(rnorm(!!obs), col = !!color, border = 'white')),
      color = color,
      obs = obs
      )
}

#' Method to plot a Plot element
setMethod(&quot;plotElement&quot;,signature = &quot;AnyPlot&quot;,definition = function(object){
  eval(object@plot_element)
})
#' Method to render a Plot Element
setMethod(&quot;shinyElement&quot;,signature = &quot;AnyPlot&quot;,definition = function(object){
  renderPlot(plotElement(object))
})



server &lt;- function(input, output, session) {

  # Create a reactive to create the Report object
  report_obj &lt;- reactive(HistPlot(obs=input$obs))

  # Check for change of the slider to change the plots
  observeEvent(input$obs,{
    output$renderedPDF &lt;- renderText(&quot;&quot;)
    output$renderPlot &lt;-  shinyElement(  report_obj() )
  } )

}

# Simple shiny App containing the standard histogram + PDF render and Download button
ui &lt;- fluidPage(
  sidebarLayout(
    sidebarPanel(
      sliderInput(
        &quot;obs&quot;,
        &quot;Number of observations:&quot;, min = 10, max = 500, value = 100)
    ),
    mainPanel(
      plotOutput(&quot;renderPlot&quot;)
    )
  )
)
shinyApp(ui = ui, server = server)</code></pre>
<p><a rel="nofollow" href="https://github.com/zappingseb/biowarptruck/blob/master/example_apps/app_stiff.R" target="_blank">Standard shiny app</a></p>
<pre><code>server &lt;- function(input, output) {
  # Output Gray Histogram
  output$distPlot &lt;- renderPlot({
    hist(rnorm(input$obs), col = 'darkgray', border = 'white')
  })

}

# Simple shiny App containing the standard histogram + PDF render and Download button
ui &lt;- fluidPage(
  sidebarLayout(
    sidebarPanel(
      sliderInput(
            &quot;obs&quot;,
            &quot;Number of observations:&quot;, min = 10, max = 500, value = 100)
    ),
    mainPanel(
      plotOutput(&quot;distPlot&quot;)
    )
  )
)
shinyApp(ui = ui, server = server)</code></pre>
<p>But an advantage of the object orientation is that you can now output the plot in a lot of different formats. We solved this by introducing methods called <code>pdfElement</code>, <code>logElement</code>, or <code>archiveElement</code>. To get a deeper look, you can check out some examples stored on GitHub <a rel="nofollow" href="https://github.com/zappingseb/biowarptruck/tree/master/example_apps" target="_blank">here</a>. These show differences between object-oriented and standard <a rel="nofollow" href="https://shiny.rstudio.com/" target="_blank"><code>shiny</code></a> apps. You can see that duplicated code is reduced in object-oriented apps, and while the code of the <a rel="nofollow" href="https://shiny.rstudio.com/" target="_blank"><code>shiny</code></a> app itself does not change for object-oriented apps, the code constructing the objects shown on the page changes. For the standard apps, the <a rel="nofollow" href="https://shiny.rstudio.com/" target="_blank"><code>shiny</code></a> code itself also changes every time an element is updated.</p>
<p>The main advantage of this approach is that you can keep your <a rel="nofollow" href="https://shiny.rstudio.com/" target="_blank">shiny</a> app exactly the same whatever it calculates or whatever it reports. Inside our department, this meant that whenever somebody wants a different plot inside an app, we do not have to touch our main app again. Whenever somebody wanted to change just the linear regression app, we did not have to touch other apps. The look and feel, the logging, and the PDF report stays exactly the same. Those three features shall never be touched unless an update is needed to their own functionality.</p>
</div>
<div>
<h3>Packaging</h3>
<p>As you know, we did not build a singular app; we had to build many for the different mathematical analyses. So we decided that for each app we will construct a separate R package. This means we had to define one Class that defines what an app will look like in a <em>core</em> package. This can be seen as fitting into the Lego theme. So our app would be Lego City, where you have trucks and cars. Other apps may be more advanced and range inside Lego Technic.</p>
<p>Now each contributor to our <code>shiny</code> app builds a package that contains a child of our <em>core</em> class. We called this class <strong>Module</strong>, and we have a lot of <strong>Module</strong> packages. This is not a <a rel="nofollow" href="https://shiny.rstudio.com/articles/modules.html" target="_blank">Shiny module</a>, but it’s modular. Our app now allows the bringing together a lot of those modules and making it bigger and bigger and bigger. It get’s more <em>HP</em> and I wouldn’t call it a car anymore. Yeah, we have a truck! Made of Lego bricks!</p>
<p><a rel="nofollow" data-flickr-embed="true" href="https://www.flickr.com/photos/157267479@N02/27368344058/in/photostream/" title="truck peterbilt" target="_blank"><img alt="truck peterbilt" src="https://i2.wp.com/farm1.staticflickr.com/807/27368344058_4d23c92951_o.png?w=100%25&ssl=1" data-recalc-dims="1"></a><br>
</p>
<p><a rel="nofollow" href="https://www.flickr.com/photos/157267479@N02/27368344058/in/photostream/" target="_blank">Image by Barney Sharman</a></p>
<p>The modularization and packaging now enables fast testing. Why? Each package can be tested using basic <a rel="nofollow" href="https://github.com/r-lib/testthat" target="_blank">testthat</a> functions. So first we tested our <em>core</em> application package, which allows adding building blocks. Afterwards, we tested each single package on its own. Finally, the whole application is tested. Our truck is ready to roll. Upon updates, we do not have to test the whole truck again. If we want to have larger tires, we just update the tire package, but not the <em>core</em> package or any other packages.</p>
</div>
<div>
<h3>Config files</h3>
<p>The truck is made of bricks – actuallyn the same bricks we used to build the car, just many more of them. Now the hard part is putting them all together and not losing track.</p>
<p>We are dealing with many the different <strong>Modules</strong> that we were writing, each of which comes in its own package. The main issue we had was that we wanted all apps to be deeply tested. During development, of course, not all apps were tested right away, so we had to give them a tag (tested yes/no). Additionally, some apps required help pages while others don’t. Some apps came with example data sets; some don’t. Some apps had a nice title in them already; for some it shall be easy to configure. For each <strong>Module</strong>, we’ll also have to source <code>js</code> and <code>css</code> files, which we allowed to be added for each app. The folder from which to source them shall be chosen by the app author. We wanted to provide as much flexibility as possible while keeping our standards for Lego bricks (look and feel, logging, plotting, and reporting). A simple example for such an app can be found on GitHub <a rel="nofollow" href="https://github.com/zappingseb/biowarptruck/tree/master/example_packaged" target="_blank">here</a>.</p>
<p>We came up with the idea of config <code>XML</code> files, which contain all the information needed to tell what needs to be set for each <strong>Module</strong>. An example <code>XML</code> document is given below, which you can read as the LEGO manual. These small configurations allow managing the apps. We also build an <code>XML</code> document that allows the apps to use features of what we call <em>core</em> package. This <code>XML</code> file is rather difficult to set up, but imagine it tells which plot shall be logged, which input shall be used, and which plots shall go into the PDF report. It allows fast development while sticking to standards.</p>
<div>
</div>
<div>
<a rel="nofollow" href="http://lego.brickinstructions.com/" target="_blank"><img alt="drawing" src="https://i0.wp.com/i.imgur.com/nPzkkYd.png?w=456&ssl=1" data-recalc-dims="1"> <br>from LegoBrickinstructions.com<br></a>
</div>
<div>
<pre><code></code></pre></div></div></div>