<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/rick-and-morty-and-tidy-data-principles/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://pachamaltese.github.io/2017-10-13_rick_and_morty_tidy_data.html"> Pachá (Batteries Included)</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h1>Motivation</h1>
<p>After reading <a rel="nofollow" href="https://juliasilge.com/blog/life-changing-magic/" target="_blank">The Life Changing Magic of Tidying Text</a> and <a rel="nofollow" href="http://tamaszilagyi.com/blog/a-tidy-text-analysis-of-rick-and-morty/" target="_blank">A tidy text analysis of Rick and Morty </a> I thought about doing something similar but reproducible and focused on Rick and Morty.</p>
<p>In this post I’ll focus on the Tidy Data principles. However, <a rel="nofollow" href="https://github.com/pachamaltese/rick_and_morty_tidy_text" target="_blank">here</a> is the Github repo with the scripts to scrap the transcripts and subtitles of Rick and Morty.</p>
<p>Here I’m using the subtitles of the TV show, as some of the transcripts I could scrap were incomplete.</p>
<p><img src="https://i1.wp.com/pachamaltese.github.io/images/2017-10-13_rick_and_morty_tidy_data/intro.jpg?w=456" data-recalc-dims="1"></p>
<p><em>Note: If some images appear too small on your screen you can open them in a new tab to show them in their original size.</em></p>
<h1>Let’s scrap</h1>
<p>The subtools package returns a data frame after reading srt files. In addition to that resulting data frame I wanted to explicitly point the season and chapter of each line of the subtitles. To do that I had to scrap the subtitles and then use <code>str_replace_all</code>. To follow the steps clone the repo from Github:</p>
<div>
<pre><span></span>git clone https<span>://</span>github.com<span>/</span>pachamaltese<span>/</span>rick_and_morty_tidy_text
</pre>
</div>
<h1>Rick and Morty Can Be So Tidy</h1>
<p>After reading the tidy file I created after scraping the subtitles, I use <code>unnest_tokens</code> to divide the subtitles in words. This function uses the tokenizers package to separate each line into words. The default tokenizing is for words, but other options include characters, sentences, lines, paragraphs, or separation around a regex pattern.</p>
<div>
<pre><span></span><span>if</span> <span>(</span><span>!</span><span>require</span><span>(</span><span>&quot;pacman&quot;</span><span>))</span> install.packages<span>(</span><span>&quot;pacman&quot;</span><span>)</span>
p_load<span>(</span>data.table<span>,</span>tidyr<span>,</span>stringr<span>,</span>tidytext<span>,</span>dplyr<span>,</span>janitor<span>,</span>ggplot2<span>,</span>viridis<span>,</span>ggstance<span>,</span>igraph<span>)</span>
p_load_gh<span>(</span><span>&quot;thomasp85/ggraph&quot;</span><span>,</span><span>&quot;dgrtwo/widyr&quot;</span><span>)</span>

rick_and_morty_subs <span>=</span> as_tibble<span>(</span>fread<span>(</span><span>&quot;2017-10-13_rick_and_morty_tidy_data/rick_and_morty_subs.csv&quot;</span><span>))</span>

rick_and_morty_subs_tidy <span>=</span> rick_and_morty_subs <span>%>%</span> 
  unnest_tokens<span>(</span>word<span>,</span>text<span>)</span> <span>%>%</span> 
  anti_join<span>(</span>stop_words<span>)</span>
</pre>
</div>
<p>The data is in one-word-per-row format, and we can manipulate it with tidy tools like dplyr. For example, in the last chunk I used an <code>anti_join</code> to remove words such a “a”, “an” or “the”.</p>
<p>Then we can use <code>count</code> to find the most common words in all of Rick and Morty episodes as a whole.</p>
<div>
<pre><span></span>rick_and_morty_subs_tidy <span>%>%</span>
  count<span>(</span>word<span>,</span> sort <span>=</span> <span>TRUE</span><span>)</span>
</pre>
</div>
<div>
<pre><span></span># A tibble: 8,100 x 2
     word     n
    </pre></div>