<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/rick-and-morty-and-tidy-data-principles-part-3/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://pachamaltese.github.io/2017-11-06_rick_and_morty_tidy_data_3.html"> Pachá (Batteries Included)</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h1>Motivation</h1>
<p>The <a rel="nofollow" href="http://pacha.hk/2017-10-13_rick_and_morty_tidy_data.html" target="_blank">first</a> and <a rel="nofollow" href="http://pacha.hk/2017-10-22_rick_and_morty_tidy_data_2.html" target="_blank">second</a> part of this analysis gave the idea that I did too much scrapping and processing and that deserves more analysis to use that information well. In this third and final part I’m also taking a lot of ideas from <a rel="nofollow" href="https://juliasilge.com/" target="_blank">Julia Silge</a>‘s blog. </p>
<p>In the <a rel="nofollow" href="https://github.com/pachamaltese/rick_and_morty_tidy_text" target="_blank">GitHub repo</a> of this project you shall find not just Rick and Morty processed subs, but also for Archer, Bojack Horseman, Gravity Falls and Stranger Things. Why? In post post I’m gonna compare the different shows.</p>
<p><img src="https://i1.wp.com/pachamaltese.github.io/images/2017-10-13_rick_and_morty_tidy_data/intro.jpg?w=456" data-recalc-dims="1"></p>
<p><em>Note: If some images appear too small on your screen you can open them in a new tab to show them in their original size.</em></p>
<h1>Word Frequencies</h1>
<p>Comparing frequencies across different shows can tell us how similar Gravity Falls, for example, is similar to Rick and Morty. I’ll use the subtitles from different shows that I scraped using the same procedure I did with Rick and Morty.</p>
<div>
<pre><span></span><span>if</span> <span>(</span><span>!</span><span>require</span><span>(</span><span>&quot;pacman&quot;</span><span>))</span> install.packages<span>(</span><span>&quot;pacman&quot;</span><span>)</span>
p_load<span>(</span>data.table<span>,</span>tidyr<span>,</span>tidytext<span>,</span>dplyr<span>,</span>ggplot2<span>,</span>viridis<span>,</span>ggstance<span>,</span>stringr<span>,</span>scales<span>)</span>
p_load_gh<span>(</span><span>&quot;dgrtwo/widyr&quot;</span><span>)</span>

rick_and_morty_subs   <span>=</span> as_tibble<span>(</span>fread<span>(</span><span>&quot;2017-10-13_rick_and_morty_tidy_data/rick_and_morty_subs.csv&quot;</span><span>))</span>
archer_subs           <span>=</span> as_tibble<span>(</span>fread<span>(</span><span>&quot;2017-10-13_rick_and_morty_tidy_data/archer_subs.csv&quot;</span><span>))</span>
bojack_horseman_subs  <span>=</span> as_tibble<span>(</span>fread<span>(</span><span>&quot;2017-10-13_rick_and_morty_tidy_data/bojack_horseman_subs.csv&quot;</span><span>))</span>
gravity_falls_subs    <span>=</span> as_tibble<span>(</span>fread<span>(</span><span>&quot;2017-10-13_rick_and_morty_tidy_data/gravity_falls_subs.csv&quot;</span><span>))</span>
stranger_things_subs  <span>=</span> as_tibble<span>(</span>fread<span>(</span><span>&quot;2017-10-13_rick_and_morty_tidy_data/stranger_things_subs.csv&quot;</span><span>))</span>

rick_and_morty_subs_tidy <span>=</span> rick_and_morty_subs <span>%>%</span> 
  unnest_tokens<span>(</span>word<span>,</span>text<span>)</span> <span>%>%</span> 
  anti_join<span>(</span>stop_words<span>)</span>

archer_subs_tidy <span>=</span> archer_subs <span>%>%</span> 
  unnest_tokens<span>(</span>word<span>,</span>text<span>)</span> <span>%>%</span> 
  anti_join<span>(</span>stop_words<span>)</span>

bojack_horseman_subs_tidy <span>=</span> bojack_horseman_subs <span>%>%</span> 
  unnest_tokens<span>(</span>word<span>,</span>text<span>)</span> <span>%>%</span> 
  anti_join<span>(</span>stop_words<span>)</span>

gravity_falls_subs_tidy <span>=</span> gravity_falls_subs <span>%>%</span> 
  unnest_tokens<span>(</span>word<span>,</span>text<span>)</span> <span>%>%</span> 
  anti_join<span>(</span>stop_words<span>)</span>

stranger_things_subs_tidy <span>=</span> stranger_things_subs <span>%>%</span> 
  unnest_tokens<span>(</span>word<span>,</span>text<span>)</span> <span>%>%</span> 
  anti_join<span>(</span>stop_words<span>)</span>
</pre>
</div>
<p>With this processing we can compare frequencies across different shows. Here’s an example of the top ten words for each show:</p>
<div>
<pre><span></span>bind_cols<span>(</span>rick_and_morty_subs_tidy <span>%>%</span> count<span>(</span>word<span>,</span> sort <span>=</span> <span>TRUE</span><span>)</span> <span>%>%</span> filter<span>(</span>row_number<span>()</span> <span>&lt;=</span> <span>10</span><span>),</span>
          archer_subs_tidy <span>%>%</span> count<span>(</span>word<span>,</span> sort <span>=</span> <span>TRUE</span><span>)</span> <span>%>%</span> filter<span>(</span>row_number<span>()</span> <span>&lt;=</span> <span>10</span><span>),</span>
          bojack_horseman_subs_tidy <span>%>%</span> count<span>(</span>word<span>,</span> sort <span>=</span> <span>TRUE</span><span>)</span> <span>%>%</span> filter<span>(</span>row_number<span>()</span> <span>&lt;=</span> <span>10</span><span>),</span>
          gravity_falls_subs_tidy <span>%>%</span> count<span>(</span>word<span>,</span> sort <span>=</span> <span>TRUE</span><span>)</span> <span>%>%</span> filter<span>(</span>row_number<span>()</span> <span>&lt;=</span> <span>10</span><span>),</span>
          stranger_things_subs_tidy <span>%>%</span> count<span>(</span>word<span>,</span> sort <span>=</span> <span>TRUE</span><span>)</span> <span>%>%</span> filter<span>(</span>row_number<span>()</span> <span>&lt;=</span> <span>10</span><span>))</span> <span>%>%</span> 
  setNames<span>(</span><span>.</span><span>,</span> <span>c</span><span>(</span><span>&quot;rm_word&quot;</span><span>,</span><span>&quot;rm_n&quot;</span><span>,</span><span>&quot;a_word&quot;</span><span>,</span><span>&quot;a_n&quot;</span><span>,</span><span>&quot;bh_word&quot;</span><span>,</span><span>&quot;bh_n&quot;</span><span>,</span><span>&quot;gf_word&quot;</span><span>,</span><span>&quot;gf_n&quot;</span><span>,</span><span>&quot;st_word&quot;</span><span>,</span><span>&quot;st_n&quot;</span><span>))</span>
</pre>
</div>
<div>
<pre><span></span># A tibble: 10 x 10
   rm_word  rm_n a_word   a_n bh_word  bh_n gf_word  gf_n st_word  st_n
     </pre></div>