<div>Last week, I showed <a href="http://www.deeplytrivial.com/2018/05/statistics-sunday-tokenizing-text.html">how to tokenize text</a>. Today I’ll use those functions to do some text analysis of one of my favorite types of text: song lyrics. Plus, this is a great opportunity to demonstrate a new R package I discovered: <a href="https://github.com/JosiahParry/geniusR">geniusR</a>, which will download lyrics from Genius.
<p>There are two packages – geniusR and geniusr – which will do this. I played with both and found geniusR easier to use. Neither is perfect, but what is perfect, anyway?</p>
<p>To install geniusR, you’ll use a different method than usual – you’ll need to install the package devtools, then call the install_github function to download the R package directly from GitHub.</p>
<div>
<div>
<div>
<pre><span>install.packages</span><span>(</span><span>&quot;devtools&quot;</span><span>)</span><br></pre>
</div>
<div>
<pre><span>devtools</span><span>::</span><span>install_github</span><span>(</span><span>&quot;josiahparry/geniusR&quot;</span><span>)</span><br></pre>
</div>
<div>
<pre>## Downloading GitHub repo josiahparry/geniusR@master<br>## from URL https://api.github.com/repos/josiahparry/geniusR/zipball/master<br></pre>
</div>
<div>
<pre>## Installing geniusR<br></pre>
</div>
<div>
<pre>## '/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file  \<br>##   --no-environ --no-save --no-restore --quiet CMD INSTALL  \<br>##   '/private/var/folders/85/9ygtlz0s4nxbmx3kgkvbs5g80000gn/T/Rtmpl3bwRx/devtools33c73e3f989/JosiahParry-geniusR-5907d82'  \<br>##   --library='/Library/Frameworks/R.framework/Versions/3.4/Resources/library'  \<br>##   --install-tests<br></pre>
</div>
<div>
<pre>## <br></pre>
</div>
</div>
</div>
<p>Now you’ll want to load geniusR and tidyverse so we can work with our data.</p>
<div>
<div>
<div>
<pre><span>library</span><span>(geniusR)</span><br><span>library</span><span>(tidyverse)</span><br></pre>
</div>
<div>
<pre>## ── Attaching packages ────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 1.2.1 ──<br></pre>
</div>
<div>
<pre>## <img alt="✔" src="https://s.w.org/images/core/emoji/2.4/72x72/2714.png" class="wp-smiley"> ggplot2 2.2.1     <img alt="✔" src="https://s.w.org/images/core/emoji/2.4/72x72/2714.png" class="wp-smiley"> purrr   0.2.4<br>## <img alt="✔" src="https://s.w.org/images/core/emoji/2.4/72x72/2714.png" class="wp-smiley"> tibble  1.4.2     <img alt="✔" src="https://s.w.org/images/core/emoji/2.4/72x72/2714.png" class="wp-smiley"> dplyr   0.7.4<br>## <img alt="✔" src="https://s.w.org/images/core/emoji/2.4/72x72/2714.png" class="wp-smiley"> tidyr   0.8.0     <img alt="✔" src="https://s.w.org/images/core/emoji/2.4/72x72/2714.png" class="wp-smiley"> stringr 1.3.0<br>## <img alt="✔" src="https://s.w.org/images/core/emoji/2.4/72x72/2714.png" class="wp-smiley"> readr   1.1.1     <img alt="✔" src="https://s.w.org/images/core/emoji/2.4/72x72/2714.png" class="wp-smiley"> forcats 0.3.0<br></pre>
</div>
<div>
<pre>## ── Conflicts ───────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──<br>## <img alt="✖" src="https://s.w.org/images/core/emoji/2.4/72x72/2716.png" class="wp-smiley"> dplyr::filter() masks stats::filter()<br>## <img alt="✖" src="https://s.w.org/images/core/emoji/2.4/72x72/2716.png" class="wp-smiley"> dplyr::lag()    masks stats::lag()<br></pre>
</div>
</div>
</div>
<p>For today’s demonstration, I’ll be working with data from two artists I love: Taylor Swift and Lorde. Both dropped new albums last year, Reputation and Melodrama, respectively, and both, though similar in age and friends with each other, have very different writing and musical styles.</p>
<p>geniusR has a function genius_album that will download lyrics from an entire album, labeling it by track.</p>
<div>
<div>
<div>
<pre><span>swift_lyrics</span> <span>&lt;-</span> <span>genius_album</span><span>(</span><span>artist</span><span>=</span><span>&quot;Taylor Swift&quot;</span><span>,</span> <span>album</span><span>=</span><span>&quot;Reputation&quot;</span><span>)</span><br></pre>
</div>
<div>
<pre>## Joining, by = c(&quot;track_title&quot;, &quot;track_n&quot;, &quot;track_url&quot;)<br></pre>
</div>
<div>
<pre><span>lorde_lyrics</span> <span>&lt;-</span> <span>genius_album</span><span>(</span><span>artist</span><span>=</span><span>&quot;Lorde&quot;</span><span>,</span> <span>album</span><span>=</span><span>&quot;Melodrama&quot;</span><span>)</span><br></pre>
</div>
<div>
<pre>## Joining, by = c(&quot;track_title&quot;, &quot;track_n&quot;, &quot;track_url&quot;)<br></pre>
</div>
</div>
</div>
<p>Now we want to tokenize our datasets, remove stop words, and count word frequency – this code should look familiar, except this time, I’m combining them using the pipeline symbol (%>%) from the tidyverse, which allows you to string together multiple functions without having to nest them.</p>
<div>
<div>
<div>
<pre><span>library</span><span>(tidytext)</span><br><span>tidy_swift</span> <span>&lt;-</span> <span>swift_lyrics</span> <span>%>%</span><br>  <span>unnest_tokens</span><span>(word,lyric)</span> <span>%>%</span><br>  <span>anti_join</span><span>(stop_words)</span> <span>%>%</span><br>  <span>count</span><span>(word,</span> <span>sort</span><span>=</span><span>TRUE</span><span>)</span><br></pre>
</div>
<div>
<pre>## Joining, by = &quot;word&quot;<br></pre>
</div>
<div>
<pre><span>head</span><span>(tidy_swift)</span><br></pre>
</div>
<div>
<pre>## # A tibble: 6 x 2<br>##   word      n<br>##   </pre></div></div></div></div>