<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/text-classification-with-tidy-data-principles/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://juliasilge.com/blog/tidy-text-classification/"> Rstats on Julia Silge</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I am an enthusiastic proponent of using <a rel="nofollow" href="https://www.tidytextmining.com/" target="_blank">tidy data principles for dealing with text data</a>. This kind of approach offers a fluent and flexible option not just for exploratory data analysis, but also for machine learning for text, including both <a rel="nofollow" href="https://juliasilge.com/blog/evaluating-stm/" target="_blank">unsupervised machine learning</a> and supervised machine learning. I haven’t written much about supervised machine learning for text, i.e. predictive modeling, using tidy data principles, so let’s walk through an example workflow for this a text classification task.</p>
<p>This post lays out a workflow similar to <a rel="nofollow" href="https://www.hvitfeldt.me/2018/01/predicting-authorship-in-the-federalist-papers-with-tidytext/" target="_blank">the approach taken by Emil Hvitfeldt</a> in predicting authorship of the Federalist Papers, so be sure to check out that post to see more examples. Also, I’ve been giving some workshops lately that included material on this, such as for <a rel="nofollow" href="https://github.com/juliasilge/ibm-ai-day" target="_blank">IBM Community Day: AI</a> and at the <a rel="nofollow" href="https://github.com/juliasilge/deming2018" target="_blank">2018 Deming Conference</a>. I have slides and code available at those links. This material is also some of what we’ll cover in the <a rel="nofollow" href="https://ww2.amstat.org/meetings/sdss/2019/onlineprogram/Program.cfm?date=05-29-19" target="_blank">short course I am teaching at the SDSS conference in 2019</a> so come on out to Bellevue if you are interested!</p>
<div>
<h2>Jane Austen vs. H. G. Wells</h2>
<p>Let’s build a supervised machine learning model that learns the difference between text from <em>Pride and Prejudice</em> and text from <em>The War of the Worlds</em>. We can access the full texts of these works from <a rel="nofollow" href="https://www.gutenberg.org/" target="_blank">Project Gutenberg</a> via the <a rel="nofollow" href="https://ropensci.org/tutorials/gutenbergr_tutorial/" target="_blank">gutenbergr</a> package.</p>
<pre><code>library(tidyverse)
library(gutenbergr)

titles &lt;- c(
  &quot;The War of the Worlds&quot;,
  &quot;Pride and Prejudice&quot;
)
books &lt;- gutenberg_works(title %in% titles) %>%
  gutenberg_download(meta_fields = &quot;title&quot;) %>%
  mutate(document = row_number())

books</code></pre>
<pre><code>## # A tibble: 19,504 x 4
##    gutenberg_id text                                 title         document
##           </code></pre></div>