<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/sentiment-analysis-of-5-popular-romantic-comedies/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://appsilondatascience.com/blog/rstats/2018/02/14/valentines.html"> Appsilon Data Science Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h1>Background</h1>
<p>With Valentine’s Day coming up, I was thinking about a fun analysis that I could convert into a blog post.</p>
<p><br>
<center><br>
<img src="https://i1.wp.com/media.giphy.com/media/3ohuAc7s47ZTXyWLS0/giphy.gif?w=456&ssl=1" data-recalc-dims="1"><br>
</center><br>
</p>
<p>Inspired by a beautiful visualization <a rel="nofollow" href="https://informationisbeautiful.net/visualizations/based-on-a-true-true-story/" target="_blank">“Based on a True True story?”</a> I decided to do something similar and analyze the sentiment in the most popular romantic comedies.</p>
<p>After searching for <em>romantic comedies</em> Google suggests a list of movies, where the top 5 are: <a rel="nofollow" href="http://www.imdb.com/title/tt0098635/?ref_=nv_sr_3" target="_blank">“When Harry Met Sally”</a>, <a rel="nofollow" href="http://www.imdb.com/title/tt0314331/" target="_blank">“Love Actually”</a>, <a rel="nofollow" href="http://www.imdb.com/title/tt0100405/?ref_=nv_sr_1" target="_blank">“Pretty Woman”</a>, <a rel="nofollow" href="http://www.imdb.com/title/tt0125439/?ref_=nv_sr_1" target="_blank">“Notting Hill”</a>, and <a rel="nofollow" href="http://www.imdb.com/title/tt0108160/?ref_=nv_sr_2" target="_blank">“Sleepless in Seattle”</a>.</p>
<p><img src="https://i0.wp.com/appsilondatascience.com/blog/assets/article_images/2018-02-13valentines/comedies.png?w=50%25" data-recalc-dims="1"></p>
<h1>How to do text analysis in R?</h1>
<p>We can use the <code>subtools</code> package to analyze the movies’ sentiment in R by loading the movie subtitles into R and then use <code>tidytext</code> to work with the text data.</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>subtools</span><span>)</span><span>
</span><span>library</span><span>(</span><span>tidytext</span><span>)</span><span>
</span><span>library</span><span>(</span><span>dplyr</span><span>)</span><span>
</span><span>library</span><span>(</span><span>plotly</span><span>)</span><span>
</span><span>library</span><span>(</span><span>purrr</span><span>)</span><span>
</span><span>library</span><span>(</span><span>lubridate</span><span>)</span><span>
</span><span>library</span><span>(</span><span>methods</span><span>)</span></code></pre>
</figure>
<h2>Working with movie data</h2>
<p>I downloaded the srt subtitles for 5 comedies from <a rel="nofollow" href="https://www.opensubtitles.org/en/?" target="_blank">Open Subtitles</a> before the analysis.</p>
<p>Now let’s load them into R and have a sneak peak of what the data looks like.</p>
<figure>
<pre><code data-lang="r"><span>romantic_comedies_titles</span><span> </span><span>&lt;-</span><span> </span><span>c</span><span>(</span><span>&quot;Love Actually&quot;</span><span>,</span><span> </span><span>&quot;Notting Hill&quot;</span><span>,</span><span>
</span><span>&quot;Pretty Woman&quot;</span><span>,</span><span> </span><span>&quot;Sleepless in Seattle&quot;</span><span>,</span><span> </span><span>&quot;When Harry Met Sally&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span>subtitles_path</span><span> </span><span>&lt;-</span><span> </span><span>&quot;../assets/data/valentines/&quot;</span><span>
</span><span>romantic_comedies</span><span> </span><span>&lt;-</span><span> </span><span>romantic_comedies_titles</span><span> </span><span>%>%</span><span> </span><span>map</span><span>(</span><span>function</span><span>(</span><span>title</span><span>){</span><span>
</span><span>title_no_space</span><span> </span><span>&lt;-</span><span> </span><span>gsub</span><span>(</span><span>&quot; &quot;</span><span>,</span><span> </span><span>&quot;_&quot;</span><span>,</span><span> </span><span>tolower</span><span>(</span><span>title</span><span>))</span><span>
</span><span>title_file_name</span><span> </span><span>&lt;-</span><span> </span><span>paste0</span><span>(</span><span>subtitles_path</span><span>,</span><span> </span><span>title_no_space</span><span>,</span><span> </span><span>&quot;.srt&quot;</span><span>)</span><span>
</span><span>read.subtitles</span><span>(</span><span>title_file_name</span><span>)</span><span>$</span><span>subtitles</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>movie_title</span><span> </span><span>=</span><span> </span><span>title</span><span>)</span><span>
</span><span>})</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span>head</span><span>(</span><span>romantic_comedies</span><span>[[</span><span>1</span><span>]])</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">##   ID  Timecode.in Timecode.out
## 1  1 00:01:12.292 00:01:14.668
## 2  2 00:01:14.753 00:01:18.130
## 3  3 00:01:18.339 00:01:19.715
## 4  4 00:01:19.799 00:01:22.134
## 5  5 00:01:22.218 00:01:23.802
## 6  6 00:01:23.887 00:01:26.430
##                                                   Text   movie_title
## 1   Whenever I get gloomy with the state of the world, Love Actually
## 2 I think about the arrivals gate at Heathrow airport. Love Actually
## 3                  General opinion started to make out Love Actually
## 4         that we live in a world of hatred and greed, Love Actually
## 5                                but I don't see that. Love Actually
## 6                 Seems to me that love is everywhere. Love Actually</code></pre>
</figure>
<h2>Subtitles preprocessing</h2>
<p>The next step is <strong>tokenization</strong>, chopping up the subtitles into single words. At this stage I also perform a minor cleaning task, which is removing <strong>stop words</strong> and adding information about the line and its duration.</p>
<figure>
<pre><code data-lang="r"><span>tokenize_clean_subtitles</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>subtitles</span><span>,</span><span> </span><span>stop_words</span><span>)</span><span> </span><span>{</span><span>
</span><span>subtitles</span><span> </span><span>%>%</span><span>
</span><span>unnest_tokens</span><span>(</span><span>word</span><span>,</span><span> </span><span>Text</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>anti_join</span><span>(</span><span>stop_words</span><span>,</span><span> </span><span>by</span><span> </span><span>=</span><span> </span><span>&quot;word&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>left_join</span><span>(</span><span>subtitles</span><span> </span><span>%>%</span><span> </span><span>select</span><span>(</span><span>ID</span><span>,</span><span> </span><span>Text</span><span>),</span><span> </span><span>by</span><span> </span><span>=</span><span> </span><span>&quot;ID&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>
</span><span>line</span><span> </span><span>=</span><span> </span><span>paste</span><span>(</span><span>Timecode.in</span><span>,</span><span> </span><span>Timecode.out</span><span>),</span><span>
</span><span>duration</span><span> </span><span>=</span><span> </span><span>as.numeric</span><span>(</span><span>hms</span><span>(</span><span>Timecode.out</span><span>)</span><span> </span><span>-</span><span> </span><span>hms</span><span>(</span><span>Timecode.in</span><span>)))</span><span>
</span><span>}</span><span>
</span><span>data</span><span>(</span><span>&quot;stop_words&quot;</span><span>)</span><span>
</span><span>head</span><span>(</span><span>stop_words</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 6 x 2
##   word      lexicon
##   </code></pre></figure>