<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/who-wrote-the-anti-trump-new-york-times-op-ed-using-tidytext-to-find-document-similarity/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://varianceexplained.org/r/op-ed-text-analysis/"> Variance Explained</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Like a lot of people, I was intrigued by <a rel="nofollow" href="https://www.nytimes.com/2018/09/05/opinion/trump-white-house-anonymous-resistance.html" target="_blank">“I Am Part of the Resistance Inside the Trump Administration”</a>, an anonymous New York Times op-ed written by a “senior official in the Trump administration”. And like many data scientists, I was curious about what role text mining could play.</p>
<blockquote class="twitter-tweet" data-lang="en">
<p lang="en" dir="ltr">Ok NLP people, now’s your chance to shine. Just spitballing here but TF-IDF on “the op-ed” compared to the published writing of every senior Trump admin official? I want likelihood estimates with standard errors. GO!</p>
<p>— Drew Conway (@drewconway) <a rel="nofollow" href="https://twitter.com/drewconway/status/1037475836042117120?ref_src=twsrc%5Etfw" target="_blank">September 5, 2018</a></p></blockquote>
<p></p>
<p>This is a useful opportunity to demonstrate how to use the <a rel="nofollow" href="https://www.tidytextmining.com/" target="_blank">tidytext package</a> that Julia Silge and I developed, and in particular to apply three methods:</p>
<ul>
<li>Using TF-IDF to find words specific to each document (examined in more detail in <a rel="nofollow" href="https://www.tidytextmining.com/tfidf.html" target="_blank">Chapter 3 of our book</a>)</li>
<li>Using <a rel="nofollow" href="https://github.com/dgrtwo/widyr" target="_blank">widyr</a> to compute pairwise cosine similarity</li>
<li>How to make similarity interpretable by breaking it down by word</li>
</ul>
<p>Since my goal is R education more than it is political analysis, I show all the code in the post.</p>
<p>Even in the less than 24 hours since the article was posted, I’m far from the first to run text analysis on it. In particular <a rel="nofollow" href="https://twitter.com/kearneymw" target="_blank">Mike Kearney</a> has shared a <a rel="nofollow" href="https://github.com/mkearney/resist_oped" target="_blank">great R analysis on GitHub</a> (which in particular pointed me towards <a rel="nofollow" href="https://twitter.com/cspan/lists/the-cabinet" target="_blank">CSPAN’s cabinet Twitter list</a>), and <a rel="nofollow" href="https://twitter.com/iamasharkskin" target="_blank">Kanishka Misra</a> has done some exciting work <a rel="nofollow" href="https://github.com/kanishkamisra/inside-trumps-white-house" target="_blank">here</a>.</p>
<h3>Downloading data</h3>
<p>Getting the text of the op-ed is doable with the <a rel="nofollow" href="https://github.com/hadley/rvest" target="_blank">rvest package</a>.</p>
<figure>
<pre><code data-lang="r"><span># setup</span><span>
</span><span>library</span><span>(</span><span>tidyverse</span><span>)</span><span>
</span><span>library</span><span>(</span><span>tidytext</span><span>)</span><span>
</span><span>library</span><span>(</span><span>rvest</span><span>)</span><span>
</span><span>theme_set</span><span>(</span><span>theme_light</span><span>())</span><span>
</span><span>url</span><span> </span><span>&lt;-</span><span> </span><span>&quot;https://www.nytimes.com/2018/09/05/opinion/trump-white-house-anonymous-resistance.html&quot;</span><span>
</span><span># tail(-1) removes the first paragraph, which is an editorial header</span><span>
</span><span>op_ed</span><span> </span><span>&lt;-</span><span> </span><span>read_html</span><span>(</span><span>url</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>html_nodes</span><span>(</span><span>&quot;.e2kc3sl0&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>html_text</span><span>()</span><span> </span><span>%>%</span><span>
</span><span>tail</span><span>(</span><span>-1</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>data_frame</span><span>(</span><span>text</span><span> </span><span>=</span><span> </span><span>.</span><span>)</span></code></pre>
</figure>
<p>The harder step is getting a set of documents representing “senior officials”. An imperfect but fast approach is to collect text from their Twitter accounts. (If you find an interesting dataset of, say, government FOIA documents, I recommend you try extending this analysis!)</p>
<p>We can look at a combination of two (overlapping) Twitter lists containing administration staff members:</p>
<ul>
<li><a rel="nofollow" href="https://twitter.com/cspan/lists/the-cabinet" target="_blank">CSPAN’s list of Cabinet accounts</a></li>
<li><a rel="nofollow" href="https://twitter.com/cspan/lists/digiphile" target="_blank">digiphile’s list of White House staff</a></li>
</ul>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>rtweet</span><span>)</span><span>
</span><span>cabinet_accounts</span><span> </span><span>&lt;-</span><span> </span><span>lists_members</span><span>(</span><span>owner_user</span><span> </span><span>=</span><span> </span><span>&quot;CSPAN&quot;</span><span>,</span><span> </span><span>slug</span><span> </span><span>=</span><span> </span><span>&quot;the-cabinet&quot;</span><span>)</span><span>
</span><span>staff</span><span> </span><span>&lt;-</span><span> </span><span>lists_members</span><span>(</span><span>slug</span><span> </span><span>=</span><span> </span><span>&quot;white-house-staff&quot;</span><span>,</span><span> </span><span>owner</span><span> </span><span>=</span><span> </span><span>&quot;digiphile&quot;</span><span>)</span><span>
</span><span># Find unique screen names from either account</span><span>
</span><span>accounts</span><span> </span><span>&lt;-</span><span> </span><span>unique</span><span>(</span><span>c</span><span>(</span><span>cabinet_accounts</span><span>$</span><span>screen_name</span><span>,</span><span> </span><span>staff</span><span>$</span><span>screen_name</span><span>))</span><span>
</span><span># Download ~3200 from each account</span><span>
</span><span>tweets</span><span> </span><span>&lt;-</span><span> </span><span>map_df</span><span>(</span><span>accounts</span><span>,</span><span> </span><span>get_timeline</span><span>,</span><span> </span><span>n</span><span> </span><span>=</span><span> </span><span>3200</span><span>)</span></code></pre>
</figure>
<p>This results in a set of 136,501 from 69 Twitter handles. There’s certainly no guarantee that the op-ed writer is among these Twitter accounts (or, if they are, that they even write their tweets themselves). But it still serves as an interesting case study of text analysis. How do we find the tweets with the closest use of language?</p>
<h3>Tokenizing tweets</h3>
<p>First, we need to tokenize the tweets: to turn them from full messages into individual words. We probably want to avoid retweets, and we need to use a custom regular expression for splitting it and remove links (just as I’d done when <a rel="nofollow" href="http://varianceexplained.org/r/trump-tweets/" target="_blank">analyzing Trump’s Twitter account</a>).</p>
<figure>
<pre><code data-lang="r"><span># When multiple tweets across accounts are identical (common in government</span><span>
</span><span># accounts), use distinct() to keep only the earliest</span><span>
</span><span>reg</span><span> </span><span>&lt;-</span><span> </span><span>&quot;([^A-Za-z\\d#@']|'(?![A-Za-z\\d#@]))&quot;</span><span>
</span><span>tweet_words</span><span> </span><span>&lt;-</span><span> </span><span>tweets</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>!</span><span>is_retweet</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>arrange</span><span>(</span><span>created_at</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>distinct</span><span>(</span><span>text</span><span>,</span><span> </span><span>.keep_all</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>screen_name</span><span>,</span><span> </span><span>status_id</span><span>,</span><span> </span><span>text</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>text</span><span> </span><span>=</span><span> </span><span>str_replace_all</span><span>(</span><span>text</span><span>,</span><span> </span><span>&quot;https?://t.co/[A-Za-z\\d]+|&amp;&quot;</span><span>,</span><span> </span><span>&quot;&quot;</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>unnest_tokens</span><span>(</span><span>word</span><span>,</span><span> </span><span>text</span><span>,</span><span> </span><span>token</span><span> </span><span>=</span><span> </span><span>&quot;regex&quot;</span><span>,</span><span> </span><span>pattern</span><span> </span><span>=</span><span> </span><span>reg</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>str_detect</span><span>(</span><span>word</span><span>,</span><span> </span><span>&quot;[a-z]&quot;</span><span>))</span></code></pre>
</figure>
<p>This parses the corpus of tweets into almost 1.5 million words.</p>
<p>Among this population of accounts, and ignoring “stop words” like “the” and “of”, what are the most common words? We can use ggplot2 to visualize this.</p>
<figure>
<pre><code data-lang="r"><span>tweet_words</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>!</span><span>word</span><span> </span><span>%in%</span><span> </span><span>stop_words</span><span>$</span><span>word</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>count</span><span>(</span><span>word</span><span>,</span><span> </span><span>sort</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>head</span><span>(</span><span>16</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>word</span><span> </span><span>=</span><span> </span><span>reorder</span><span>(</span><span>word</span><span>,</span><span> </span><span>n</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>word</span><span>,</span><span> </span><span>n</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_col</span><span>()</span><span> </span><span>+</span><span>
</span><span>coord_flip</span><span>()</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>y</span><span> </span><span>=</span><span> </span><span>&quot;# of uses among staff Twitter accounts&quot;</span><span>)</span></code></pre>
</figure>
<p><img alt="center" src="https://i2.wp.com/varianceexplained.org/figs/2018-09-06-op-ed-text-analysis/common_words-1.png?w=456" data-recalc-dims="1"></p>
<p>No real surprises here. Many accounts mention @POTUS often, as well as words like “people”, “American”, and “Trump” that you’d expect from administration accounts.</p>
<h3>Finding a text signature: TF-IDF vectors</h3>
<p>What words make up someone’s “signature”? What make up mine, or Trump’s, or Mike Pence’s, or the op-ed’s?</p>
<p>We could start with the most common words someone uses. But there are some words, like “the” and “of” that just about everyone uses, as well as words like “President” that everyone in our dataset will use. So we also want to downweight words that appear across many documents. A common tool for balancing these two considerations and turning them into a “signature” vector is <a rel="nofollow" href="https://en.wikipedia.org/wiki/Tf%E2%80%93idf" target="_blank">tf-idf</a>: <strong>term-frequency inverse-document-frequency</strong>. This takes how frequently someone uses a term, but divides it by (the log of) how many documents mention it. For more details, see <a rel="nofollow" href="https://www.tidytextmining.com/tfidf.html" target="_blank">Chapter 3 of Text Mining with R</a>.</p>
<p>The <code>bind_tf_idf</code> function from tidytext lets us compute tf-idf on a dataset of word counts like this. Before we do, we bring in the op-ed as an additional document (since we’re interesting in considering it as one “special” document in our corpus).</p>
<figure>
<pre><code data-lang="r"><span># Combine in the op_ed wordswith the name &quot;OP-ED&quot; </span><span>
</span><span>op_ed_words</span><span> </span><span>&lt;-</span><span> </span><span>op_ed</span><span> </span><span>%>%</span><span>
</span><span>unnest_tokens</span><span>(</span><span>word</span><span>,</span><span> </span><span>text</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>count</span><span>(</span><span>word</span><span>)</span><span>
</span><span>word_counts</span><span> </span><span>&lt;-</span><span> </span><span>tweet_words</span><span> </span><span>%>%</span><span>
</span><span>count</span><span>(</span><span>screen_name</span><span>,</span><span> </span><span>word</span><span>,</span><span> </span><span>sort</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>bind_rows</span><span>(</span><span>op_ed_words</span><span> </span><span>%>%</span><span> </span><span>mutate</span><span>(</span><span>screen_name</span><span> </span><span>=</span><span> </span><span>&quot;OP-ED&quot;</span><span>))</span><span>
</span><span># Compute TF-IDF using &quot;word&quot; as term and &quot;screen_name&quot; as document</span><span>
</span><span>word_tf_idf</span><span> </span><span>&lt;-</span><span> </span><span>word_counts</span><span> </span><span>%>%</span><span>
</span><span>bind_tf_idf</span><span>(</span><span>word</span><span>,</span><span> </span><span>screen_name</span><span>,</span><span> </span><span>n</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>arrange</span><span>(</span><span>desc</span><span>(</span><span>tf_idf</span><span>))</span><span>
</span><span>word_tf_idf</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 226,410 x 6
##    screen_name    word                 n      tf   idf tf_idf
##    </code></pre></figure>