<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/voice-control-your-shiny-apps/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://www.jumpingrivers.com/blog/voice-control-your-shiny-apps/"> r – Jumping Rivers</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I love R and I love <a rel="nofollow" href="https://shiny.rstudio.com/gallery/" target="_blank">Shiny</a>. One of the things I really like about shiny is the ease with which you can incorporate other Javascript based tools and libraries. By my own admission, my <a rel="nofollow" href="https://en.wikipedia.org/wiki/JavaScript" target="_blank">JavaScript</a> skills are definitely lacking but there are so many cool libraries out there which can really make visualisation and interaction with displayed content come alive. One such library I came across about a while ago now is <a rel="nofollow" href="https://www.talater.com/annyang/" target="_blank">annyang</a>.</p>
<h2>What is annyang?</h2>
<blockquote>
<p>“annyang is a tiny javascript library that lets your visitors control your site with voice commands. annyang supports multiple languages, has no dependencies, weighs just 2kb and is free to use.”</p>
</blockquote>
<p>Annyang works with all browsers that implement the speech recognition interface of the Web Speech API and makes it extremely easy to get started. Here is some JavaScript that you can include inside raw HTML to add a voice command that listens for the phrase <strong>Jumping Rivers</strong> and takes you to our <a rel="nofollow" href="https://www.jumpingrivers.com/" target="_blank">home page</a>.</p>
<pre><code>

</code></pre>
<h3>More complex commands</h3>
<p>In addition to explicit phrases like the above, annyang can also understand named variables, optional phrases and multi-word text (splats). I refer you to the guides on the <a rel="nofollow" href="https://www.talater.com/annyang/" target="_blank">annyang page</a> for further info.</p>
<h2>Include it in Shiny</h2>
<p><strong>Note: This will not function in the RStudio browser if you run the code yourself open the app in another browser, preferably Chrome. You will likely have to give permission to the browser to use the microphone too.</strong></p>
<p>Shiny makes it easy to include snippets of Javascript code in your apps through the HTML <code>tags</code>. We could include the above command into a Shiny app with the following code</p>
<pre><code>## Simple Shiny App - just say &quot;Jumping Rivers&quot;
library(&quot;shiny&quot;)
ui = fluidPage(
  singleton(
    tags$head(
      tags$script(src = &quot;//cdnjs.cloudflare.com/ajax/libs/annyang/2.6.0/annyang.min.js&quot;),
      tags$script(HTML(&quot;
        if(annyang) {
          var commands = {
            'Jumping Rivers' : function() {
              window.open('https://www.jumpingrivers.com', '_blank');
            } 
          };
          annyang.addCommands(commands);
          annyang.start();
        }&quot;
      ))
    )
  )
)
server = function(input, output, session) {}
shinyApp(ui, server)
</code></pre>
<p>It’s potentially easier to keep your JavaScript file separate and include it using <code>includeScript()</code> in your ui.R. So we have two files:</p>
<ul>
<li>
<p><strong>voice.js</strong></p>
<pre><code>if(annyang){
  var commands = {
    'Jumping Rivers': function() {
      window.open('https://www.jumpingrivers.com','_blank');
    } 
  };
  annyang.addCommands(commands);
  annyang.start();
}
</code></pre>
</li>
<li>
<p><strong>app.R</strong></p>
<pre><code>library(&quot;shiny&quot;)
ui = fluidPage(
  singleton(
    tags$head(
      tags$script(src = &quot;//cdnjs.cloudflare.com/ajax/libs/annyang/2.6.0/annyang.min.js&quot;),
      includeScript(&quot;voice.js&quot;)
    )
  )
)
server = function(input, output, session) {}
shinyApp(ui, server)
</code></pre>
</li>
</ul>
<p>See the <a rel="nofollow" href="https://jumpingrivers.shinyapps.io/speech/" target="_blank">Shiny App</a> in action.</p>
<h3>What about changing R variables?</h3>
<p>Now I think that by itself is kind of cool. But it would be much more useful if we could use this to interact with R objects. Fortunately Shiny makes this possible too. The Javascript <code>Shiny.onInputChange('var', value)</code> function can be used to set a <code>value</code> on a reactive under <code>input$var</code>.</p>
<p>As a simple example, let’s create a counter that increments when I say <strong>“Count”</strong>. Our <code>voice.js</code> file would be</p>
<pre><code>if(annyang) {
  var value = 0;
  var commands = {
    'count': function() {
      value += 1;
      Shiny.onInputChange('counter', value);
    }
  };
  annyang.addCommands(commands);
  annyang.start();
}
</code></pre>
<p>and a Shiny app that displays the counter might be</p>
<pre><code>library(&quot;shiny&quot;)
ui = fluidPage(
  singleton(
    tags$head(
      tags$script(src = &quot;//cdnjs.cloudflare.com/ajax/libs/annyang/2.6.0/annyang.min.js&quot;),
      includeScript(&quot;voice.js&quot;)
    )
  ),
  textOutput(&quot;out&quot;)
)
server = function(input, output, session) output$out = renderText(input$counter)
shinyApp(ui, server)
</code></pre>
<p>Check out the end result <a rel="nofollow" href="https://jumpingrivers.shinyapps.io/Speech2/" target="_blank">app</a>.</p>
<h2>But I don’t know any Javascript and don’t want to know</h2>
<p>The above is all well and good if you are happy to write the JavaScript code for each of your apps. But if you really don’t want to or don’t feel comfortable doing that then we could define an R function which registers a keyword to start listening (think Alexa), takes all the spoken word after the keyword and stores the string as an R object. This way all processing of the result can be done in the comfort of the R language. One such implementation of this using the annyang splat is</p>
<pre><code># I have used the keyword jarvis as default because I like iron man
# default value is an empty string
# necessary to choose an &quot;inputId&quot; to bind the value to
voiceInput = function(inputId, keyword = &quot;jarvis&quot;, value = &quot;&quot;) {
  tagList(
    singleton(
      tags$head(
        tags$script(src = &quot;//cdnjs.cloudflare.com/ajax/libs/annyang/2.6.0/annyang.min.js&quot;),
        tags$script(HTML(&quot;annyang.start({ autoRestart: true, continuous: true});&quot;)))),
        tags$head(tags$script(HTML(
          glue::glue(&quot;var </code></pre>