<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/handling-happy-vs-not-happy-better-sentiment-analysis-with-sentimentr-in-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://datascienceplus.com/handling-happy-vs-not-happy-better-sentiment-analysis-with-sentimentr-in-r/"> R Programming – DataScience+</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Sentiment Analysis is one of the most obvious things Data Analysts with unlabelled Text data (with no score or no rating) end up doing in an attempt to extract some insights out of it and the same Sentiment analysis is also one of the potential research areas for any NLP (Natural Language Processing) enthusiasts. </p>
<p>For an analyst, the same sentiment analysis is a pain in the neck because most of the primitive packages/libraries handling sentiment analysis perform a simple dictionary lookup and calculate a final composite score based on the number of occurrences of positive and negative words. But that often ends up in a lot of false positives, with a very obvious case being ‘happy’ vs ‘not happy’ – Negations, in general <strong>Valence Shifters</strong>. </p>
<p>Consider this sentence: ‘I am not very happy’. Any Primitive Sentiment Analysis Algorithm would just flag this sentence positive because of the word ‘happy’ that apparently would appear in the positive dictionary. But reading this sentence we know this is not a positive sentence. </p>
<p>While we could build our own way to handle these negations, there are couple of new R-packages that could do this with ease. One such package is <code><a rel="nofollow" href="https://github.com/trinker/sentimentr" target="_blank">sentimentr</a></code> developed by <a rel="nofollow" href="https://github.com/trinker" target="_blank">Tyler Rinker</a>. </p>
<h2>Installing the package</h2>
<p>sentimentr can be installed from <a rel="nofollow" href="https://cran.r-project.org/web/packages/sentimentr/index.html" target="_blank">CRAN</a> or the development version can be installed from github. </p>
<pre>
install.packages('sentimentr')
#or
library(devtools)
install_github('trinker/sentimentr')
</pre>
<h2>Why sentimentr? </h2>
<p>The author of the package himself explaining what does <code>sentimentr</code> do that other packages don’t and why does it matter?</p>
<p>“sentimentr attempts to take into account valence shifters (i.e., negators, amplifiers (intensifiers), de-amplifiers (downtoners), and adversative conjunctions) while maintaining speed. Simply put, sentimentr is an augmented dictionary lookup. The next questions address why it matters.”</p>
<h3>Sentiment Scoring:</h3>
<p>sentimentr offers sentiment analysis with two functions: 1. <code>sentiment_by()</code> 2. <code>sentiment()</code></p>
<p>Aggregated (Averaged) Sentiment Score for a given text with <code>sentiment_by</code></p>
<pre>
sentiment_by('I am not very happy', by = NULL)

<em> element_id sentence_id word_count sentiment
1: 1 1 5 -0.06708204
</em>
</pre>
<p>But this might not help much when we have multiple sentences with different polarity, hence sentence-level scoring with <code>sentiment</code> would help here. </p>
<pre>
sentiment('I am not very happy. He is very happy')

<em> element_id sentence_id word_count sentiment
1: 1 1 5 -0.06708204
2: 1 2 4 0.67500000
</em>
</pre>
<p>Both the functions return a dataframe with four columns:</p>
<p>1. <code>element_id </code>– ID / Serial Number of the given text<br>
2. <code>sentence_id</code> – ID / Serial Number of the sentence and this is equal to element_id in case of <code>sentiment_by</code><br>
3. <code>word_count</code> – Number of words in the given sentence<br>
4. <code>sentiment</code> – Sentiment Score of the given sentence</p>
<h2>Extract Sentiment Keywords</h2>
<p>The <code>extract_sentiment_terms()</code> function helps us extract the keywords – both positive and negative that was part of the sentiment score calculation. sentimentr also supports pipe operator <code>%>%</code> which makes it easier to write multiple lines of code with less assignment and also cleaner code. </p>
<pre>
'My life has become terrible since I met you and lost money' %>% extract_sentiment_terms()
<em> element_id sentence_id negative positive
1: 1 1 terrible,lost money
</em>
</pre>
<h3>Sentiment Highlighting:</h3>
<p>And finally, the <code>highight()</code> function coupled with <code>sentiment_by()</code> that gives a html output with parts of sentences nicely highlighted with green and red color to show its polarity. Trust me, This might seem trivial but it really helps while making Presentations to share the results, discuss False positives and to identify the room for improvements in the accuracy.</p>
<pre>
'My life has become terrible since I met you and lost money. But I still have got a little hope left in me' %>% 
  sentiment_by(by = NULL) %>%
  highlight()
</pre>
<p>Output Screenshot:<br>
<a rel="nofollow" href="https://i1.wp.com/datascienceplus.com/wp-content/uploads/2017/11/polarity.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/11/polarity-490x87.png?w=450&ssl=1" alt="" class="wp-image-9546" data-recalc-dims="1"></a></p>
<p>Try using <code>sentimentr</code> for your sentiment analysis and text analytics project and do share your feedback in comments. Complete code used here is available on <a rel="nofollow" href="https://github.com/amrrs/datascienceplus_codes/blob/master/sentimentr_basics.R" target="_blank">my github</a>.</p>
<ul></ul>
<p><strong>
<p>Related Post</p>
<p></p></strong>
<ol>
<li><a rel="nofollow" href="https://datascienceplus.com/creating-reporting-template-with-glue-in-r/" title="Permanent Link toCreating Reporting Template with Glue in R" target="_blank">Creating Reporting Template with Glue in R</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/predict-employee-turnover-with-python/" title="Permanent Link toPredict Employee Turnover With Python" target="_blank">Predict Employee Turnover With Python</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/making-a-shiny-dashboard-using-highcharter-analyzing-inflation-rates/" title="Permanent Link toMaking a Shiny dashboard using ‘highcharter’ – Analyzing Inflation Rates" target="_blank">Making a Shiny dashboard using ‘highcharter’ – Analyzing Inflation Rates</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/time-series-analysis-in-r-part-2-time-series-transformations/" title="Permanent Link toTime Series Analysis in R Part 2: Time Series Transformations" target="_blank">Time Series Analysis in R Part 2: Time Series Transformations</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/time-series-analysis-in-r-part-1-the-time-series-object/" title="Permanent Link toTime Series Analysis in R Part 1: The Time Series Object" target="_blank">Time Series Analysis in R Part 1: The Time Series Object</a></li>
</ol>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://datascienceplus.com/handling-happy-vs-not-happy-better-sentiment-analysis-with-sentimentr-in-r/"> R Programming – DataScience+</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/ligE_L-eSIw" height="1"></a></div>