<div> As a mixed methods researcher, I love working with qualitative data, but I also love the idea of using quantitative methods to add some meaning and context to the words. This is the main reason I’ve started digging into using R for text mining, and these skills have paid off in not only fun blog posts about <a rel="nofollow" href="http://www.deeplytrivial.com/2018/05/how-has-taylor-swifts-word-choice.html" target="_blank">Taylor Swift</a>, <a rel="nofollow" href="http://www.deeplytrivial.com/2018/05/statistics-sunday-taylor-swift-vs-lorde.html" target="_blank">Lorde</a>, and <a rel="nofollow" href="http://www.deeplytrivial.com/2018/05/statistics-sunday-welcome-to-sentiment.html" target="_blank">“Hotel California”</a>, but also in analyzing data for my job (blog post about that to be posted soon). So today, I thought I’d keep moving forward to other tools you can use in text analysis: term frequency and inverse document frequency.
<p>These tools are useful when you have multiple documents you’re analyzing, such as interview text from different people or books by the same author. For my demonstration today, I’ll be using (what else?) song lyrics, this time from Florence + the Machine (one of my all-time favorites), who just dropped a new album, <i>High as Hope</i>. So let’s get started by pulling in those lyrics.</p>
<div>
<div>
<div>
<pre><span>library</span><span>(geniusR)</span><br><br><span>high_as_hope</span> <span>&lt;-</span> <span>genius_album</span><span>(</span><span>artist</span> <span>=</span> <span>&quot;Florence the Machine&quot;</span><span>,</span> <span>album</span> <span>=</span> <span>&quot;High as Hope&quot;</span><span>)</span><br></pre>
</div>
<div>
<pre>## Joining, by = c(&quot;track_title&quot;, &quot;track_n&quot;, &quot;track_url&quot;)<br></pre>
</div>
<div>
<pre><span>library</span><span>(tidyverse)</span><br></pre>
</div>
<div>
<pre><span>library</span><span>(tidytext)</span><br><span>tidy_hope</span> <span>&lt;-</span> <span>high_as_hope</span> <span>%>%</span><br>  <span>unnest_tokens</span><span>(word,lyric)</span> <span>%>%</span><br>  <span>anti_join</span><span>(stop_words)</span><br></pre>
</div>
<div>
<pre>## Joining, by = &quot;word&quot;<br></pre>
</div>
<div>
<pre><span>head</span><span>(tidy_hope)</span><br></pre>
</div>
<div>
<pre>## # A tibble: 6 x 4<br>##   track_title track_n  line word   <br>##   </pre></div></div></div></div>