<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/a-shiny-app-to-create-sentimental-tweets-based-on-project-gutenberg-books/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://fronkonstin.com/2017/10/24/a-shiny-app-to-create-sentimental-tweets-based-on-project-gutenberg-books/"> R – Fronkonstin</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div class="twitter-share"><a rel="nofollow" href="https://twitter.com/intent/tweet?url=https%3A%2F%2Ffronkonstin.com%2F2017%2F10%2F24%2Fa-shiny-app-to-create-sentimental-tweets-based-on-project-gutenberg-books%2F&hashtags=rstats%20%23NLP&%23038;via=aschinchon" class="twitter-share-button" target="_blank">Tweet</a></div>
<blockquote>
<p>There was something about them that made me uneasy, some longing and at the same time some deadly fear – Dracula (Stoker, Bram)</p>
</blockquote>
<p>Twitter is a very good source of inspiration. Some days ago I came across with this:</p>
<div class="twitter-tweet">
<blockquote class="twitter-tweet" data-lang="en">
<p lang="es" dir="ltr">Mi presentación sobre análisis de textos con R para el primer meetup de <a rel="nofollow" href="https://twitter.com/RLadiesSantiago?ref_src=twsrc%5Etfw" target="_blank">@RLadiesSantiago</a>: <a rel="nofollow" href="https://t.co/KPU89vyeND" target="_blank">https://t.co/KPU89vyeND</a> <a rel="nofollow" href="https://twitter.com/hashtag/rladies?src=hash&ref_src=twsrc%5Etfw" target="_blank">#rladies</a> <a rel="nofollow" href="https://twitter.com/hashtag/rstatsES?src=hash&ref_src=twsrc%5Etfw" target="_blank">#rstatsES</a></p>
<p>— Riva Quiroga (@rivaquiroga) <a rel="nofollow" href="https://twitter.com/rivaquiroga/status/912149538495959042?ref_src=twsrc%5Etfw" target="_blank">September 25, 2017</a></p>
</blockquote>
</div>
<p>The tweet refers to a presentation (in Spanish) available <a rel="nofollow" href="https://t.co/KPU89vyeND" target="_blank">here</a>, which is a very concise and well illustrated document about the <em>state-of-the-art</em> of text mining in R. I discovered there several libraries that I will try to use in the future. In this experiment I have used one of them: the <code>syuzhet</code> package. As can be read in the documentation:</p>
<p><em>this package extracts sentiment and sentiment-derived plot arcs from text using three sentiment dictionaries conveniently packaged for consumption by R users. Implemented dictionaries include <code>syuzhet</code> (default) developed in the Nebraska Literary Lab, <code>afinn</code> developed by Finn Arup Nielsen, <code>bing</code> developed by Minqing Hu and Bing Liu, and <code>nrc</code> developed by Mohammad, Saif M. and Turney, Peter D.</em></p>
<p>You can find a complete explanation of the package in <a rel="nofollow" href="https://cran.r-project.org/web/packages/syuzhet/vignettes/syuzhet-vignette.html" target="_blank">its vignette</a>. A very interesting application of these techniques is the <em>Sentiment Graph</em> of a book, which represents how sentiment changes over time. This is the Sentiment Graph of <em>Romeo and Juliet</em>, by William Shakespeare, taken from <a rel="nofollow" href="http://projectalexandria.net/" target="_blank">Project Alexandria</a>:</p>
<p><img data-orig-file="https://i1.wp.com/fronkonstin.com/wp-content/uploads/2017/10/RomeoAndJuliet.png?fit=480%2C480&amp;ssl=1" data-orig-size="480,480" data-medium-file="https://i1.wp.com/fronkonstin.com/wp-content/uploads/2017/10/RomeoAndJuliet.png?fit=300%2C300&amp;ssl=1" data-attachment-id="4281" data-image-meta="{" data-image-description="" src="https://i1.wp.com/fronkonstin.com/wp-content/uploads/2017/10/RomeoAndJuliet.png?resize=480%2C480&ssl=1" data-permalink="https://fronkonstin.com/2017/10/24/a-shiny-app-to-create-sentimental-tweets-based-on-project-gutenberg-books/romeoandjuliet/" data-large-file="https://i1.wp.com/fronkonstin.com/wp-content/uploads/2017/10/RomeoAndJuliet.png?fit=480%2C480&amp;ssl=1" alt="" data-image-title="RomeoAndJuliet" sizes="(max-width: 480px) 100vw, 480px" data-comments-opened="1" class="wp-image-4281" data-recalc-dims="1"></p>
<p><em>Darkest</em> sentiments can be seen at the end of the book, where the tragedy reaches its highest level. It is also nice to see how sentiments are cyclical. This graphs can be very useful for people who just want to read happy endings books (my sister is one of those).</p>
<p>Inspired by this analysis, I have done another experiment in which I download a book from Project Gutenberg and measure sentiment of all its sentences. Based on this measurement, I filter top 5% (positive or negative sentiment) sentences to build tweets. I have done a Shiny app where all these steps are explained. The app is available <a rel="nofollow" href="http://91.134.134.147:3838/gutenberg" target="_blank">here</a>.</p>
<p>From a technical point of view I used <a rel="nofollow" href="https://shiny.rstudio.com/articles/selectize.html" target="_blank"><code>selectize</code></a> JavaScript library to filter books in a flexible way. I customized as well the appearance with CSS bootstrap from <a rel="nofollow" href="https://bootswatch.com/" target="_blank">Bootswatch</a> as explained <a rel="nofollow" href="https://shiny.rstudio.com/articles/css.html" target="_blank">here</a>.</p>
<p>This is the code of the experiment.</p>
<p><code>UI.R</code>:</p>
<pre>
library(shiny)

fluidPage(theme = &quot;bootstrap.css&quot;,

  titlePanel(h1(&quot;Sentimental Tweets from Project Gutenberg Books&quot;, align=&quot;center&quot;),
             windowTitle=&quot;Tweets from Project Gutenberg&quot;),
  sidebarLayout(
      sidebarPanel(

        selectInput(
          'book', 'Choose a book:', 
          multiple=FALSE,
          selectize = TRUE,
          choices=c(&quot;Enter some words of title or author&quot; = &quot;&quot;, gutenberg_works$searchstr)
          ),
        
        radioButtons(inputId = &quot;sent&quot;,
             label = &quot;Choose sentiment:&quot;,
             choices = c(&quot;Dark&quot;=&quot;1&quot;, &quot;Bright&quot;=&quot;20&quot;),
             selected=&quot;1&quot;,
             inline=TRUE),
        
        radioButtons(inputId = &quot;meth&quot;,
                    label = &quot;Choose a method to measure sentiment:&quot;,
                    choices = c(&quot;syuzhet&quot;, &quot;bing&quot;, &quot;afinn&quot;, &quot;nrc&quot;),
                    selected=&quot;syuzhet&quot;,
                    inline=TRUE),
        
        radioButtons(inputId = &quot;char&quot;,
                     label = &quot;Number of characters (max):&quot;,
                     choices = list(&quot;140&quot;, &quot;280&quot;),
                     inline=TRUE),

        checkboxInput(inputId = &quot;auth&quot;,
                      label = &quot;Add author&quot;,
                      value=FALSE),
        
        checkboxInput(inputId = &quot;titl&quot;,
                      label = &quot;Add title&quot;,
                      value=FALSE),
        
        checkboxInput(inputId = &quot;post&quot;,
                      label=&quot;Add link to post (thanks!)&quot;,
                      value=TRUE),
        
        textInput(inputId = &quot;adds&quot;,
                  label=&quot;Something else?&quot;,
                  placeholder=&quot;Maybe a #hastag?&quot;),
        
        actionButton('do','Go!', 
                     class=&quot;btn btn-success action-button&quot;, 
                     css.class=&quot;btn btn-success&quot;)
  ),
  

  
  mainPanel(
     tags$br(),
     p(&quot;First of all, choose a book entering some keywords of its 
        title or author and doing dropdown navigation. Books are 
        downloaded from Project Gutenberg. You can browse the complete 
        catalog&quot;, tags$a(href = &quot;https://www.gutenberg.org/catalog/&quot;, &quot;here.&quot;)),

     p(&quot;After that, choose the sentiment of tweets you want to generate. 
        There are four possible methods than can return slightly different results. 
        All of them assess the sentiment of each word of a sentence and sum up the 
        result to give a scoring for it. The more negative is this scoring, 
        the&quot;, em(&quot;darker&quot;) ,&quot;is the sentiment. The more positive, the &quot;, em(&quot;brighter.&quot;), 
        &quot; You can find a nice explanation of these techniques&quot;,
        tags$a(href = &quot;http://www.matthewjockers.net/2017/01/12/resurrecting/&quot;, &quot;here.&quot;)),
        
        p(&quot;Next parameters are easy: you can add the title and author of the book where 
          sentence is extracted as well as a link to my blog and any other string you want. 
          Clicking on the lower button you will get after some seconds a tweet below. 
          Click as many times you want until you like the result.&quot;),
     
     p(&quot;Finally, copy, paste and tweet. &quot;,strong(&quot;Enjoy it!&quot;)),
     tags$br(),
     tags$blockquote(textOutput(&quot;tweet1&quot;)),
     tags$br()

)))
</pre>
<p><code>Server.R</code>:</p>
<pre>
library(shiny)

function(input, output) {
  
  values &lt;- reactiveValues(default = 0)
  
  observeEvent(input$do,{
    values$default &lt;- 1
  })

  book &lt;- eventReactive(input$do, {
    GetTweet(input$book, input$meth, input$sent, input$char,
             input$auth, input$titl, input$post, input$adds)
  })
  
  output$tweet1 &lt;- renderText({
    if(values$default == 0){
      &quot;Your tweet will appear here ...&quot;
    }
    else{
      book()
    }
  })
}
</pre>
<p><code>Global.R</code>:</p>
<pre>
library(gutenbergr)
library(dplyr)
library(stringr)
library(syuzhet)

x &lt;- tempdir() # Read the Project Gutenberg catalog and filter english works. I also create a column with # title and author to make searchings gutenberg_metadata %>%
  filter(has_text, language==&quot;en&quot;, gutenberg_id>0, !is.na(author)) %>%
  mutate(searchstr=ifelse(is.na(author), title, paste(title, author, sep= &quot; - &quot;))) %>%
  mutate(searchstr=str_replace_all(searchstr, &quot;[\r\n]&quot; , &quot;&quot;)) %>%
  group_by(searchstr) %>%
  summarize(gutenberg_id=min(gutenberg_id)) %>%
  ungroup() %>%
  na.omit() %>%
  filter(str_length(searchstr)&lt;100)-> gutenberg_works

# This function generates a tweet according the UI settings (book, method, sentiment and
# number of characters). It also appends some optional strings at the end
GetTweet = function (string, method, sentim, characters,
                     author, title, link, hastag)
 {
  # Obtain gutenberg_id from book 
  gutenberg_works %>%
     filter(searchstr == string) %>%
     select(gutenberg_id) %>% .$gutenberg_id -> result
  
  # Download text, divide into sentences and score sentiment. Save results to do it once and
  # optimize performance
  if(!file.exists(paste0(x,&quot;/&quot;,&quot;book&quot;,result,&quot;_&quot;,method,&quot;.RDS&quot;)))
  {
    book=gutenberg_download(result)
    book[,2] %>% 
      as.data.frame() %>% 
      .$text %>% 
      paste(collapse=&quot; &quot;) -> text
    
    sentences_v &lt;- get_sentences(text)
    sentiment_v &lt;- get_sentiment(sentences_v, method=method) data.frame(sentence=sentences_v, sentiment=sentiment_v) %>% 
      mutate(length=str_length(sentence)) -> results
    saveRDS(results, paste0(x,&quot;/&quot;,&quot;book&quot;,result,&quot;_&quot;,method,&quot;.RDS&quot;))
  }
   
  results=readRDS(paste0(x,&quot;/&quot;,&quot;book&quot;,result,&quot;_&quot;,method,&quot;.RDS&quot;))
  book_info=gutenberg_metadata %>% filter(gutenberg_id==result)
  
  # Paste optional strings to append at the end
  post=&quot;&quot;
  if (title)  post=paste(&quot;-&quot;, book_info[,&quot;title&quot;], post, sep=&quot; &quot;)
  if (author) post=paste0(post, &quot; (&quot;, str_trim(book_info[,&quot;author&quot;]), &quot;)&quot;)
  if (link)   post=paste(post, &quot;https://wp.me/p7VZWY-16S&quot;, sep=&quot; &quot;)
  post=paste(post, hastag, sep=&quot; &quot;)
  length_post=nchar(post)

  # Calculate 5% quantiles
  results %>% 
    filter(length&lt;=(as.numeric(characters)-length_post)) %>%
     mutate(sentiment=jitter(sentiment)) %>% 
     mutate(group = cut(sentiment, 
                        include.lowest = FALSE,
                        labels = FALSE,
                        breaks = quantile(sentiment, probs = seq(0, 1, 0.05)))) -> results
   
  # Obtain a sample sentence according sentiment and append optional string to create tweet
  results %>% 
     filter(group==as.numeric(sentim)) %>% 
     sample_n(1) %>% 
     select(sentence) %>% 
     .$sentence %>% 
     as.character() %>% 
     str_replace_all(&quot;[.]&quot;, &quot;&quot;) %>% 
    paste(post, sep=&quot; &quot;) -> tweet
  
  return(tweet)

 }
</pre>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://fronkonstin.com/2017/10/24/a-shiny-app-to-create-sentimental-tweets-based-on-project-gutenberg-books/"> R – Fronkonstin</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/qsjJTTgdZ04" height="1"></a></div>