<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/4x3-r-hackathoning-the-finishers-guide/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://staff.math.su.se/hoehle/blog/2016/12/12/hackinthedark.html"> Theory meets practice...</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2>Abstract</h2>
<p>We present experiences from organizing a small R hackathon aimed at advancing knowledge and documentation of the R package surveillance. The hackathon was piggybacked on the ESCAIDE2016 conference visited by current and potential package users in the area of infectious disease epidemiology. The output of the hackathon is available at <a target="_blank" rel="nofollow" href="https://surveillancer.github.io/tutorials/">https://surveillancer.github.io/tutorials/</a>.</p>
<p><center><br>
<img src="https://i2.wp.com/staff.math.su.se/hoehle/blog/figure/source/2016-12-12-hackinthedark/ears-1.png?w=456" data-recalc-dims="1"><br>
</center></p>
<p> <a target="_blank" rel="nofollow" href="http://creativecommons.org/licenses/by-sa/4.0/"><img alt="Creative Commons License" src="https://i1.wp.com/i.creativecommons.org/l/by-sa/4.0/88x31.png?w=456&ssl=1" data-recalc-dims="1"></a> This work is licensed under a <a target="_blank" rel="nofollow" href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>. The markdown+Rknitr source code of this blog is available under a <a target="_blank" rel="nofollow" href="https://www.gnu.org/licenses/gpl-3.0.html">GNU General Public License (GPL v3)</a> license from <a target="_blank" rel="nofollow" href="https://github.com/hoehleatsu/hoehleatsu.github.io"><img alt="github" src="https://i2.wp.com/staff.math.su.se/hoehle/blog/images/GitHub-Mark-32px.png?w=456" data-recalc-dims="1"></a>.</p>
<h2>Introduction</h2>
<p>A <strong><a target="_blank" rel="nofollow" href="https://en.wikipedia.org/wiki/Hackathon">hackathon</a></strong> is a extreme-programming sprint-like event where people involved in software development (and beyond) meet for a short period of time with the purpose of collaborative programming, typically in <strong>open-source</strong>. The word hackathon is a merger of <strong>hack</strong> and <strong>marathon</strong>, where <a target="_blank" rel="nofollow" href="http://www.dictionary.com/browse/hack"><em>hacking</em></a> is to be understood as the skillful modification of computer programs (and not the malicious circumvention of security measures). Lots of good guides have been written on <a target="_blank" rel="nofollow" href="https://hackathon.guide/">how to run a successful Hackathon</a>. In the area of <strong>infectious disease epidemiology</strong>, which has been the main area of motivation for our statistical developments and implementations, very successful events (<a target="_blank" rel="nofollow" href="https://sites.google.com/site/hackoutwiki/home">hackout</a>, <a target="_blank" rel="nofollow" href="https://sites.google.com/site/hackout2/">hackout2</a>, <a target="_blank" rel="nofollow" href="http://hackout3.ropensci.org/">hackout3</a>) have previously been organized. At a much smaller scale we wanted to ignite some of the energy and enthusiasm such an event spawns.</p>
<p>As a consequence, this blog post gathers our experiences from organising a small <strong>4×3 hackathon</strong> (4 people, 3 days) for the surveillance R-package in connection with the <a target="_blank" rel="nofollow" href="http://ecdc.europa.eu/en/escaide/Pages/ESCAIDE.aspx">ESCAIDE2016</a> conference. Our hope is that these might be useful for others – possibly working in very different contexts.</p>
<h2>Organizing and Running the Hackathon</h2>
<p>We report on a number of practical &quot;?&quot; and &quot;!&quot; below.</p>
<h3>Why?</h3>
<p>Over the last several years we worked on a package for the visualization, modelling and monitoring of surveillance time series. As most of us are busy with other tasks now, it felt like a good idea to all meet in person to work a little on the package and network with potential users in order to increase awareness of the package. The 10’th European Scientific Conference on Applied Infectious Disease Epidemiology (<a target="_blank" rel="nofollow" href="http://ecdc.europa.eu/en/escaide/Pages/ESCAIDE.aspx">ESCAIDE2016</a>) organized by the <a target="_blank" rel="nofollow" href="http://ecdc.europa.eu/">European Centre for Disease Prevention and Control</a> (ECDC) in Stockholm 28-30 Nov 2016 in Stockholm Sweden with its about 600 participants, felt like the right place to be.</p>
<h3>A Cool Name?</h3>
<p>Stockholm is placed on the 59th parallel north, hence, during end of November daylight is limited to approximately <a target="_blank" rel="nofollow" href="http://www.timeanddate.com/sun/sweden/stockholm">6:30 hours</a>. In other words: Perfect hacking conditions. To honour this <strong>Hack in the Dark</strong> became our internal handle for the hackathon.</p>
<h3>Who?</h3>
<p>Hackathons come in all sizes. We decided on a mini four person hackathon of experienced R users, who all knew the package well: two former Ph.D. students who had used the package as the implementational repository for their methodological developments (one of them now being the package maintainer), a former power-user of the package and the package creator. It might have been good to involve new persons in order to expand awareness of the package and increase diversity of the hackathonians, but we decided to go for the small team in order to maximize efficiency.</p>
<h3>Venue?</h3>
<p>Piggybacking on an existing conference held in a large conference centre meant we did not have to worry about WLAN, food and seating. Especially when only being 4 persons.</p>
<h3>What to do?</h3>
<p>We started about 8 weeks before the hackathon to brainstorm using <a target="_blank" rel="nofollow" href="https://slack.com/">slack</a> as a messaging system. We then created a priority matrix in <a target="_blank" rel="nofollow" href="http://docs.google.com/">google docs</a> allowing each of the participants to prioritize the ideas. This gave us an initial idea of what we wanted to do. Unfortunately, most of us then got pretty busy with other activities so we never managed to revisit the matrix until a couple of days before the start of the hackathon. Instead, we recapitulated matters in an &quot;indian buffet process&quot; meeting in Stockholm at the night before the hackathon:</p>
<ol type="1">
<li>Write i/o tutorials explaining how to get data into the package and then use package functions for visualization on them</li>
<li>Use open European data for the tutorials – the theme of ESCAIDE2016 was after all: <strong>Data for action</strong></li>
<li>Make a <a target="_blank" rel="nofollow" href="https://shiny.rstudio.com/">shiny app</a> to visualize the effect of various parameters choices for the surveillance algorithms implemented in the package. The best choice of configuration has been a recurrent user question throughout the years.</li>
<li>Demo twitter surveillance by monitoring the conference hashtag <code>#ESCAIDE2016</code>.</li>
</ol>
<h3>Source Code Management?</h3>
<p>We chose to create an organization <strong>surveillancer</strong> on <a target="_blank" rel="nofollow" href="http://www.github.com/">github</a>, which we then all joined with our individual github accounts. All new projects were then conducted by initiating new repositories. The <code>surveillance</code> package itself is still developed on <a target="_blank" rel="nofollow" href="http://www.r-forge.r-project.org/">R-Forge</a> using <code>svn</code>, but since we knew most of our work would be <em>using</em> the surveillance package rather than <em>developing</em> the package, we decided to keep the existing infrastructure for the package and instead develop the planned tutorials and visualizations in a new github project. This worked ok, but switching between <code>svn</code> and <code>git</code> for commits on different projects was not always helpful: <code>git commit -a</code> is a useful friend if you don’t know the <a target="_blank" rel="nofollow" href="https://githowto.com/staging_changes">git staging area</a>…</p>
<h3>Project Output Format?</h3>
<p>We decided to create R package structure and use Hadley Wickham’s new <a target="_blank" rel="nofollow" href="https://hadley.github.io/pkgdown/">pkgdown</a> package to create website containing all hackathon output. The tutorials were then created as vignettes. An immediate advantage of this approach was that all vignette code was then directly available for the interested user.</p>
<h3>Demo, demo, demo!</h3>
<p>Inspired by the extreme programming paradigm, and because we wanted to interact with the conference, we decided to demo at least once a day by posting hackathon output on twitter. Besides the outgoing publicity we also frequently demo’ed internally in order to get input and suggestions. This worked pretty well – there is nothing as motivating and interactive as getting constructive input and suggestions from your table neighbour!</p>
<p>Interaction with the other conference participants, on the other hand, was moderate. We showed parts and pieces to interested people, but in hindsight we should have aimed for a poster presentation or a related activity in order to generate more real-life awareness of the hackathon outside the virtual world of twitter.</p>
<h2>Summing Up</h2>
<p>The three days of hackathon passed quickly, but we managed to get the four formulated outputs done.</p>
<h2>Reflections</h2>
<p>Intense software sprints are hard work, thus, it was natural that towards the end of the hackathon the concentration decreased slightly. However, phases of intense coding are perfectly supplemented by listening to scientific talks, talking to former colleagues or share the passion of R with others working in the field. In particular it was nice to exchange ideas with <a target="_blank" rel="nofollow" href="http://www.imperial.ac.uk/people/t.jombart">Thibaut Jombart</a> who’s <a target="_blank" rel="nofollow" href="http://www.repidemicsconsortium.org/">R Epidemics Consortium (RECON)</a> project hopefully is able to bundle the R initiatives in infectious disease epidemiology a little more. Besides the availability of software, the training aspect of new users (e.g. ongoing epidemiologists) is also crucial. Finally, the aftermath of the hackathon is as important as the pre-event planning: One or more dedicated coordinators have to ensure that loose ends are wrapped up. Here, the participants’ enthusiasm declines quickly as other activities become higher priorities. Again, it’s valuable to have a clear goal of what needs to be done (e.g. a blog entry…).</p>
<p>Altogether, code sessions such as a <a target="_blank" rel="nofollow" href="http://staff.math.su.se/hoehle/blog/2016/08/04/outbreakEnd.html">reproducibility session</a> or a <strong>toolbox session</strong> could be components for spicing up scientific conferences. For example the toolbox event could consist of a set of interested people, who meet up on conference day one to implement a particular method useful in practice and then demo it on the last day. Of course this all takes time away from other conference content and might be more work than dozing of in the plenary sessions…</p>
<p>No matter what, focus of a hackathon should also be on social aspects. It also proves wise not to ignore fresh air &amp; sunlight completely. Actually, to our surprise, the 6:30 hours of daylight were actually quite sunny in Stockholm!</p>
<h2>Visit the Hack in the Dark Output</h2>
<p>The output of the hackathon can be found here:</p>
<p><center><br>
<a target="_blank" rel="nofollow" href="https://surveillancer.github.io/tutorials/">https://surveillancer.github.io/tutorials/</a><br>
</center></p>
<p><p>In order to run the accompanying code (available from github by clicking on the &quot;fork me on github&quot; icons), version 1.12.3 of the <code>surveillance</code> package is needed. Currently, this development version of the package is not on CRAN, but can be installed from R-Forge as</p>
<div>
<pre><code><span>install.packages</span>(<span>&quot;surveillance&quot;</span>, <span>repos=</span><span>&quot;http://R-Forge.R-project.org&quot;</span>)</code></pre>
</div>
<p>As an appetizer to actually check out the <a target="_blank" rel="nofollow" href="https://surveillancer.github.io/tutorials/">site</a> or the <a target="_blank" rel="nofollow" href="https://github.com/surveillancer/tutorials">code</a>, here are two of our tweets demoing the output:</p>
<blockquote data-lang="en" class="twitter-tweet">
<p dir="ltr" lang="en">
First draft output of the <a target="_blank" rel="nofollow" href="https://twitter.com/hashtag/ESCAIDE2016?src=hash">#ESCAIDE2016</a> surveillance <a target="_blank" rel="nofollow" href="https://twitter.com/hashtag/rstats?src=hash">#rstats</a> hackathon: Visualizing <a target="_blank" rel="nofollow" href="https://twitter.com/hashtag/opendata?src=hash">#opendata</a> by <a target="_blank" rel="nofollow" href="https://twitter.com/ECDC_EU"><span data-cites="ECDC_EU">(<span data-reference-id="ECDC_EU"><strong>???</strong></span>)</span></a> on Salmonella Agona. <a target="_blank" rel="nofollow" href="https://twitter.com/hashtag/data4action?src=hash">#data4action</a> <a target="_blank" rel="nofollow" href="https://t.co/8ApaDNF07L">pic.twitter.com/8ApaDNF07L</a>
</p>
<p>— Michael Höhle (@m_hoehle) <a target="_blank" rel="nofollow" href="https://twitter.com/m_hoehle/status/803270577150631937">November 28, 2016</a>
</p></blockquote>
<p></p>
<blockquote data-lang="en" class="twitter-tweet">
<p dir="ltr" lang="en">
Interactive illustration of monitoring algorithms for infectious disease surveillance <a target="_blank" rel="nofollow" href="https://twitter.com/hashtag/escaide2016?src=hash">#escaide2016</a> <a target="_blank" rel="nofollow" href="https://twitter.com/hashtag/rstats?src=hash">#rstats</a> <a target="_blank" rel="nofollow" href="https://t.co/qizcqqMbEJ">https://t.co/qizcqqMbEJ</a> <a target="_blank" rel="nofollow" href="https://t.co/LyZujDLsF2">pic.twitter.com/LyZujDLsF2</a>
</p>
<p>— Dirk Schumacher (@dirk_sch) <a target="_blank" rel="nofollow" href="https://twitter.com/dirk_sch/status/803573405660286976">November 29, 2016</a>
</p></blockquote>
<p></p>
<h2>The Future</h2>
<p>We wish you all the best for your hackathon event. Put software on the scientific agenda!</p>
<h2>Acknowledgments</h2>
<p>Thanks to <a target="_blank" rel="nofollow" href="http://www.masalmon.eu/">Maëlle Salmon</a>, <a target="_blank" rel="nofollow" href="https://www.dirk-schumacher.net/">Dirk Schumacher</a> and <a target="_blank" rel="nofollow" href="http://www.imbe.med.uni-erlangen.de/cms/sebastian_meyer.html">Sebastian Meyer</a> for all their great work and the creative atmosphere during the hackathon! The event was implicitly supported by the Swedish Research Council as part of the project Statistical Modelling, Monitoring and Predictive Analytics against Infectious Disease Outbreaks (grant number 2015-05182_VR).</p>
<div>
</div>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://staff.math.su.se/hoehle/blog/2016/12/12/hackinthedark.html"> Theory meets practice...</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/1E8wbD-li2U"></a></div>