<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/text-analysis-of-trumps-tweets-confirms-he-writes-only-the-angrier-android-half/"></div></div><div><a data-url="https://www.r-bloggers.com/text-analysis-of-trumps-tweets-confirms-he-writes-only-the-angrier-android-half/" data-counturl="https://www.r-bloggers.com/text-analysis-of-trumps-tweets-confirms-he-writes-only-the-angrier-android-half/" data-count="horizontal" data-text="Text analysis of Trump’s tweets confirms he writes only the (angrier) Android half" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://varianceexplained.org/r/trump-tweets/"> Variance Explained</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I don’t normally post about politics (I’m not particularly savvy about polling, which is where data science <a target="_blank" href="http://fivethirtyeight.com/">has had the largest impact on politics</a>). But this weekend I saw a hypothesis about Donald Trump’s twitter account that simply begged to be investigated with data:</p>
<blockquote data-lang="en" class="twitter-tweet"><p dir="ltr" lang="en">Every non-hyperbolic tweet is from iPhone (his staff). </p>
<p>Every hyperbolic tweet is from Android (from him). <a target="_blank" href="https://t.co/GWr6D8h5ed">pic.twitter.com/GWr6D8h5ed</a></p>
<p>— Todd Vaziri (@tvaziri) <a target="_blank" href="https://twitter.com/tvaziri/status/762005541388378112">August 6, 2016</a></p></blockquote>
<p></p>
<p>When Trump wishes the Olympic team good luck, he’s tweeting from his iPhone. When he’s insulting a rival, he’s usually tweeting from an Android. Is this an artifact showing which tweets are Trump’s own and which are by some handler?</p>
<p>Others have <a target="_blank" href="http://www.cnet.com/news/trumps-tweets-android-for-nasty-iphone-for-nice/">explored Trump’s timeline</a> and noticed this tends to hold up- and Trump himself <a target="_blank" href="http://www.theverge.com/2015/10/5/9453935/donald-trump-twitter-strategy">does indeed tweet from a Samsung Galaxy</a>. But how could we examine it quantitatively? I’ve been writing about text mining and <a target="_blank" href="http://varianceexplained.org/r/trump-tweets/varianceexplained.org/r/yelp-sentiment/">sentiment analysis</a> recently, particularly during my development of the <a target="_blank" href="http://github.com/juliasilge/tidytext">tidytext R package</a> with Julia Silge, and this is a great opportunity to apply it again.</p>
<p>My analysis, shown below, concludes that <strong>the Android and iPhone tweets are clearly from different people</strong>, posting during different times of day and using hashtags, links, and retweets in distinct ways. What’s more, we can see that <strong>the Android tweets are angrier and more negative</strong>, while the iPhone tweets tend to be benign announcements and pictures. Overall I’d agree with <a target="_blank" href="https://twitter.com/tvaziri">@tvaziri</a>’s analysis: this lets us tell the difference between the campaign’s tweets (iPhone) and Trump’s own (Android).</p>
<h3>The dataset</h3>
<p>First we’ll retrieve the content of Donald Trump’s timeline using the <code>userTimeline</code> function in the <a target="_blank" href="https://cran.r-project.org/web/packages/twitteR">twitteR</a> package:<sup><a target="_blank" href="http://varianceexplained.org/r/trump-tweets/#fn:fullcode">1</a></sup></p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>dplyr</span><span>)</span><span>
</span><span>library</span><span>(</span><span>purrr</span><span>)</span><span>
</span><span>library</span><span>(</span><span>twitteR</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span># You'd need to set global options with an authenticated app
</span><span>setup_twitter_oauth</span><span>(</span><span>getOption</span><span>(</span><span>&quot;twitter_consumer_key&quot;</span><span>),</span><span>
</span><span>getOption</span><span>(</span><span>&quot;twitter_consumer_secret&quot;</span><span>),</span><span>
</span><span>getOption</span><span>(</span><span>&quot;twitter_access_token&quot;</span><span>),</span><span>
</span><span>getOption</span><span>(</span><span>&quot;twitter_access_token_secret&quot;</span><span>))</span><span>
</span><span># We can request only 3200 tweets at a time; it will return fewer
# depending on the API
</span><span>trump_tweets</span><span> </span><span>&lt;-</span><span> </span><span>userTimeline</span><span>(</span><span>&quot;realDonaldTrump&quot;</span><span>,</span><span> </span><span>n</span><span> </span><span>=</span><span> </span><span>3200</span><span>)</span><span>
</span><span>trump_tweets_df</span><span> </span><span>&lt;-</span><span> </span><span>tbl_df</span><span>(</span><span>map_df</span><span>(</span><span>trump_tweets</span><span>,</span><span> </span><span>as.data.frame</span><span>))</span></code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span># if you want to follow along without setting up Twitter authentication,
# just use my dataset:
</span><span>load</span><span>(</span><span>url</span><span>(</span><span>&quot;http://varianceexplained.org/files/trump_tweets_df.rda&quot;</span><span>))</span></code></pre>
</figure>
<p>We clean this data a bit, extracting the source application. (We’re looking only at the iPhone and Android tweets- a much smaller number are from the web client or iPad).</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>tidyr</span><span>)</span><span>
</span><span>tweets</span><span> </span><span>&lt;-</span><span> </span><span>trump_tweets_df</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>id</span><span>,</span><span> </span><span>statusSource</span><span>,</span><span> </span><span>text</span><span>,</span><span> </span><span>created</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>extract</span><span>(</span><span>statusSource</span><span>,</span><span> </span><span>&quot;source&quot;</span><span>,</span><span> </span><span>&quot;Twitter for (.*?)&lt;&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>source</span><span> </span><span>%in%</span><span> </span><span>c</span><span>(</span><span>&quot;iPhone&quot;</span><span>,</span><span> </span><span>&quot;Android&quot;</span><span>))</span></code></pre>
</figure>
<p>Overall, this includes 628 tweets from iPhone, and 762 tweets from Android.</p>
<p>Another consideration is what time of day the tweets occur, which we’d expect to be a “signature” of their user. Here we can spot a difference as well:</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>lubridate</span><span>)</span><span>
</span><span>library</span><span>(</span><span>scales</span><span>)</span><span>
</span><span>tweets</span><span> </span><span>%>%</span><span>
</span><span>count</span><span>(</span><span>source</span><span>,</span><span> </span><span>hour</span><span> </span><span>=</span><span> </span><span>hour</span><span>(</span><span>with_tz</span><span>(</span><span>created</span><span>,</span><span> </span><span>&quot;EST&quot;</span><span>)))</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>percent</span><span> </span><span>=</span><span> </span><span>n</span><span> </span><span>/</span><span> </span><span>sum</span><span>(</span><span>n</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>hour</span><span>,</span><span> </span><span>percent</span><span>,</span><span> </span><span>color</span><span> </span><span>=</span><span> </span><span>source</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_line</span><span>()</span><span> </span><span>+</span><span>
</span><span>scale_y_continuous</span><span>(</span><span>labels</span><span> </span><span>=</span><span> </span><span>percent_format</span><span>())</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>&quot;Hour of day (EST)&quot;</span><span>,</span><span>
</span><span>y</span><span> </span><span>=</span><span> </span><span>&quot;% of tweets&quot;</span><span>,</span><span>
</span><span>color</span><span> </span><span>=</span><span> </span><span>&quot;&quot;</span><span>)</span></code></pre>
</figure>
<p><img alt="center" src="http://varianceexplained.org/figs/2016-08-09-trump-tweets/unnamed-chunk-3-1.svg"></p>
<p>Trump on the Android does a lot more tweeting in the morning, while the campaign posts from the iPhone more in the afternoon and early evening.</p>
<p>Another place we can spot a difference is in Trump’s anachronistic behavior of “manually retweeting” people by copy-pasting their tweets, then surrounding them with quotation marks:</p>
<blockquote data-lang="en" class="twitter-tweet"><p dir="ltr" lang="en">&quot;<a target="_blank" href="https://twitter.com/trumplican2016">@trumplican2016</a>: <a target="_blank" href="https://twitter.com/realDonaldTrump">@realDonaldTrump</a> <a target="_blank" href="https://twitter.com/DavidWohl">@DavidWohl</a> stay the course mr trump your message is resonating with the PEOPLE&quot;</p>
<p>— Donald J. Trump (@realDonaldTrump) <a target="_blank" href="https://twitter.com/realDonaldTrump/status/758512401629192192">July 28, 2016</a></p></blockquote>
<p></p>
<p>Almost all of these quoted tweets are posted from the Android:</p>
<p><img alt="center" src="http://varianceexplained.org/figs/2016-08-09-trump-tweets/unnamed-chunk-4-1.svg"></p>
<p>In the remaining by-word analyses in this text, I’ll filter these quoted tweets out (since they contain text from followers that may not be representative of Trump’s own tweets).</p>
<p>Somewhere else we can see a difference involves sharing links or pictures in tweets.</p>
<figure>
<pre><code data-lang="r"><span>tweet_picture_counts</span><span> </span><span>&lt;-</span><span> </span><span>tweets</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>!</span><span>str_detect</span><span>(</span><span>text</span><span>,</span><span> </span><span>'^&quot;'</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>count</span><span>(</span><span>source</span><span>,</span><span>
</span><span>picture</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>str_detect</span><span>(</span><span>text</span><span>,</span><span> </span><span>&quot;t.co&quot;</span><span>),</span><span>
</span><span>&quot;Picture/link&quot;</span><span>,</span><span> </span><span>&quot;No picture/link&quot;</span><span>))</span><span>
</span><span>ggplot</span><span>(</span><span>tweet_picture_counts</span><span>,</span><span> </span><span>aes</span><span>(</span><span>source</span><span>,</span><span> </span><span>n</span><span>,</span><span> </span><span>fill</span><span> </span><span>=</span><span> </span><span>picture</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_bar</span><span>(</span><span>stat</span><span> </span><span>=</span><span> </span><span>&quot;identity&quot;</span><span>,</span><span> </span><span>position</span><span> </span><span>=</span><span> </span><span>&quot;dodge&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>&quot;&quot;</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>&quot;Number of tweets&quot;</span><span>,</span><span> </span><span>fill</span><span> </span><span>=</span><span> </span><span>&quot;&quot;</span><span>)</span></code></pre>
</figure>
<p><img alt="center" src="http://varianceexplained.org/figs/2016-08-09-trump-tweets/unnamed-chunk-5-1.svg"></p>
<p>It turns out tweets from the iPhone were <strong>38 times as likely to contain either a picture or a link.</strong> This also makes sense with our narrative: the iPhone (presumably run by the campaign) tends to write “announcement” tweets about events, like this:</p>
<blockquote data-lang="en" class="twitter-tweet"><p dir="ltr" lang="en">Thank you Windham, New Hampshire! <a target="_blank" href="https://twitter.com/hashtag/TrumpPence16?src=hash">#TrumpPence16</a> <a target="_blank" href="https://twitter.com/hashtag/MAGA?src=hash">#MAGA</a> <a target="_blank" href="https://t.co/ZL4Q01Q49s">pic.twitter.com/ZL4Q01Q49s</a></p>
<p>— Donald J. Trump (@realDonaldTrump) <a target="_blank" href="https://twitter.com/realDonaldTrump/status/762110918721310721">August 7, 2016</a></p></blockquote>
<p></p>
<p>While Android (Trump himself) tends to write picture-less tweets like:</p>
<blockquote data-lang="en" class="twitter-tweet"><p dir="ltr" lang="en">The media is going crazy. They totally distort so many things on purpose. Crimea, nuclear, &quot;the baby&quot; and so much more. Very dishonest!</p>
<p>— Donald J. Trump (@realDonaldTrump) <a target="_blank" href="https://twitter.com/realDonaldTrump/status/762400869858115588">August 7, 2016</a></p></blockquote>
<p></p>
<h3>Comparison of words</h3>
<p>Now that we’re sure there’s a difference between these two accounts, what can we say about the difference in the <em>content</em>? We’ll use the <a target="_blank" href="https://cran.r-project.org/web/packages/tidytext">tidytext</a> package that <a target="_blank" href="http://juliasilge.com/">Julia Silge</a> and I developed.</p>
<p>We start by dividing into individual words using the <code>unnest_tokens</code> function (see <a target="_blank" href="https://cran.r-project.org/web/packages/tidytext/vignettes/tidytext.html">this vignette</a> for more), and removing some common “stopwords”<sup><a target="_blank" href="http://varianceexplained.org/r/trump-tweets/#fn:regex">2</a></sup>:</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>tidytext</span><span>)</span><span>
</span><span>reg</span><span> </span><span>&lt;-</span><span> </span><span>&quot;([^A-Za-z\\d#@']|'(?![A-Za-z\\d#@]))&quot;</span><span>
</span><span>tweet_words</span><span> </span><span>&lt;-</span><span> </span><span>tweets</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>!</span><span>str_detect</span><span>(</span><span>text</span><span>,</span><span> </span><span>'^&quot;'</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>text</span><span> </span><span>=</span><span> </span><span>str_replace_all</span><span>(</span><span>text</span><span>,</span><span> </span><span>&quot;https://t.co/[A-Za-z\\d]+|&amp;amp;&quot;</span><span>,</span><span> </span><span>&quot;&quot;</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>unnest_tokens</span><span>(</span><span>word</span><span>,</span><span> </span><span>text</span><span>,</span><span> </span><span>token</span><span> </span><span>=</span><span> </span><span>&quot;regex&quot;</span><span>,</span><span> </span><span>pattern</span><span> </span><span>=</span><span> </span><span>reg</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>!</span><span>word</span><span> </span><span>%in%</span><span> </span><span>stop_words</span><span>$</span><span>word</span><span>,</span><span>
</span><span>str_detect</span><span>(</span><span>word</span><span>,</span><span> </span><span>&quot;[a-z]&quot;</span><span>))</span><span>
</span><span>tweet_words</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 8,753 x 4
##                    id source             created                   word
##                 &lt;chr>  &lt;chr>              &lt;time>                  &lt;chr>
## 1  676494179216805888 iPhone 2015-12-14 20:09:15                 record
## 2  676494179216805888 iPhone 2015-12-14 20:09:15                 health
## 3  676494179216805888 iPhone 2015-12-14 20:09:15 #makeamericagreatagain
## 4  676494179216805888 iPhone 2015-12-14 20:09:15             #trump2016
## 5  676509769562251264 iPhone 2015-12-14 21:11:12               accolade
## 6  676509769562251264 iPhone 2015-12-14 21:11:12             @trumpgolf
## 7  676509769562251264 iPhone 2015-12-14 21:11:12                 highly
## 8  676509769562251264 iPhone 2015-12-14 21:11:12              respected
## 9  676509769562251264 iPhone 2015-12-14 21:11:12                   golf
## 10 676509769562251264 iPhone 2015-12-14 21:11:12                odyssey
## # ... with 8,743 more rows</code></pre>
</figure>
<p>What were the most common words in Trump’s tweets overall?</p>
<p><img alt="center" src="http://varianceexplained.org/figs/2016-08-09-trump-tweets/tweet_words_plot-1.svg"></p>
<p>These should look familiar for anyone who has seen the feed. Now let’s consider which words are most common from the Android relative to the iPhone, and vice versa. We’ll use the simple measure of log odds ratio, calculated for each word as:<sup><a target="_blank" href="http://varianceexplained.org/r/trump-tweets/#fn:plusone">3</a></sup></p>
<p></p>
<figure>
<pre><code data-lang="r"><span>android_iphone_ratios</span><span> </span><span>&lt;-</span><span> </span><span>tweet_words</span><span> </span><span>%>%</span><span>
</span><span>count</span><span>(</span><span>word</span><span>,</span><span> </span><span>source</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>sum</span><span>(</span><span>n</span><span>)</span><span> </span><span>>=</span><span> </span><span>5</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>spread</span><span>(</span><span>source</span><span>,</span><span> </span><span>n</span><span>,</span><span> </span><span>fill</span><span> </span><span>=</span><span> </span><span>0</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>ungroup</span><span>()</span><span> </span><span>%>%</span><span>
</span><span>mutate_each</span><span>(</span><span>funs</span><span>((</span><span>.</span><span> </span><span>+</span><span> </span><span>1</span><span>)</span><span> </span><span>/</span><span> </span><span>sum</span><span>(</span><span>.</span><span> </span><span>+</span><span> </span><span>1</span><span>)),</span><span> </span><span>-</span><span>word</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>logratio</span><span> </span><span>=</span><span> </span><span>log2</span><span>(</span><span>Android</span><span> </span><span>/</span><span> </span><span>iPhone</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>arrange</span><span>(</span><span>desc</span><span>(</span><span>logratio</span><span>))</span></code></pre>
</figure>
<p>Which are the words most likely to be from Android and most likely from iPhone?</p>
<p><img alt="center" src="http://varianceexplained.org/figs/2016-08-09-trump-tweets/android_iphone_ratios_plot-1.svg"></p>
<p>A few observations:</p>
<ul>
<li>
<p><strong>Most hashtags come from the iPhone</strong>. Indeed, almost no tweets from Trump’s Android contained hashtags, with some rare exceptions like <a target="_blank" href="https://twitter.com/realDonaldTrump/status/753960134422900736">this one</a>. (This is true only because we filtered out the quoted “retweets”, as Trump does sometimes quote tweets <a target="_blank" href="https://twitter.com/realDonaldTrump/status/731805331425218560">like this</a> that contain hashtags).</p>
</li>
<li>
<p><strong>Words like “join” and “tomorrow”, and times like “7pm”, also came only from the iPhone</strong>. The iPhone is clearly responsible for event announcements like <a target="_blank" href="https://twitter.com/realDonaldTrump/status/743522630230228993">this one</a> (“Join me in Houston, Texas tomorrow night at 7pm!”)</p>
</li>
<li>
<p><strong>A lot of “emotionally charged” words, like “badly”, “crazy”, “weak”, and “dumb”, were overwhelmingly more common on Android.</strong> This supports the original hypothesis that this is the “angrier” or more hyperbolic account.</p>
</li>
</ul>
<h3>Sentiment analysis: Trump’s tweets are much more negative than his campaign’s</h3>
<p>Since we’ve observed a difference in sentiment between the Android and iPhone tweets, let’s try quantifying it. We’ll work with the <a target="_blank" href="http://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm">NRC Word-Emotion Association</a> lexicon, available from the tidytext package, which associates words with 10 sentiments: <strong>positive</strong>, <strong>negative</strong>, <strong>anger</strong>, <strong>anticipation</strong>, <strong>disgust</strong>, <strong>fear</strong>, <strong>joy</strong>, <strong>sadness</strong>, <strong>surprise</strong>, and <strong>trust</strong>.</p>
<figure>
<pre><code data-lang="r"><span>nrc</span><span> </span><span>&lt;-</span><span> </span><span>sentiments</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>lexicon</span><span> </span><span>==</span><span> </span><span>&quot;nrc&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>dplyr</span><span>::</span><span>select</span><span>(</span><span>word</span><span>,</span><span> </span><span>sentiment</span><span>)</span><span>
</span><span>nrc</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 13,901 x 2
##           word sentiment
##          &lt;chr>     &lt;chr>
## 1       abacus     trust
## 2      abandon      fear
## 3      abandon  negative
## 4      abandon   sadness
## 5    abandoned     anger
## 6    abandoned      fear
## 7    abandoned  negative
## 8    abandoned   sadness
## 9  abandonment     anger
## 10 abandonment      fear
## # ... with 13,891 more rows</code></pre>
</figure>
<p>To measure the sentiment of the Android and iPhone tweets, we can count the number of words in each category:</p>
<figure>
<pre><code data-lang="r"><span>sources</span><span> </span><span>&lt;-</span><span> </span><span>tweet_words</span><span> </span><span>%>%</span><span>
</span><span>group_by</span><span>(</span><span>source</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>total_words</span><span> </span><span>=</span><span> </span><span>n</span><span>())</span><span> </span><span>%>%</span><span>
</span><span>ungroup</span><span>()</span><span> </span><span>%>%</span><span>
</span><span>distinct</span><span>(</span><span>id</span><span>,</span><span> </span><span>source</span><span>,</span><span> </span><span>total_words</span><span>)</span><span>
</span><span>by_source_sentiment</span><span> </span><span>&lt;-</span><span> </span><span>tweet_words</span><span> </span><span>%>%</span><span>
</span><span>inner_join</span><span>(</span><span>nrc</span><span>,</span><span> </span><span>by</span><span> </span><span>=</span><span> </span><span>&quot;word&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>count</span><span>(</span><span>sentiment</span><span>,</span><span> </span><span>id</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>ungroup</span><span>()</span><span> </span><span>%>%</span><span>
</span><span>complete</span><span>(</span><span>sentiment</span><span>,</span><span> </span><span>id</span><span>,</span><span> </span><span>fill</span><span> </span><span>=</span><span> </span><span>list</span><span>(</span><span>n</span><span> </span><span>=</span><span> </span><span>0</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>inner_join</span><span>(</span><span>sources</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>group_by</span><span>(</span><span>source</span><span>,</span><span> </span><span>sentiment</span><span>,</span><span> </span><span>total_words</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>summarize</span><span>(</span><span>words</span><span> </span><span>=</span><span> </span><span>sum</span><span>(</span><span>n</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>ungroup</span><span>()</span><span>
</span><span>head</span><span>(</span><span>by_source_sentiment</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 6 x 4
##    source    sentiment total_words words
##     &lt;chr>        &lt;chr>       &lt;int> &lt;dbl>
## 1 Android        anger        4901   321
## 2 Android anticipation        4901   256
## 3 Android      disgust        4901   207
## 4 Android         fear        4901   268
## 5 Android          joy        4901   199
## 6 Android     negative        4901   560</code></pre>
</figure>
<p>(For example, we see that 321 of the 4901 words in the Android tweets were associated with “anger”). We then want to measure how much more likely the Android account is to use an emotionally-charged term relative to the iPhone account. Since this is count data, we can use a <a target="_blank" href="https://stat.ethz.ch/R-manual/R-devel/library/stats/html/poisson.test.html">Poisson test</a> to measure the difference:</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>broom</span><span>)</span><span>
</span><span>sentiment_differences</span><span> </span><span>&lt;-</span><span> </span><span>by_source_sentiment</span><span> </span><span>%>%</span><span>
</span><span>group_by</span><span>(</span><span>sentiment</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>do</span><span>(</span><span>tidy</span><span>(</span><span>poisson.test</span><span>(</span><span>.</span><span>$</span><span>words</span><span>,</span><span> </span><span>.</span><span>$</span><span>total_words</span><span>)))</span><span>
</span><span>sentiment_differences</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## Source: local data frame [10 x 9]
## Groups: sentiment [10]
## 
##       sentiment estimate statistic      p.value parameter  conf.low
##           (chr)    (dbl)     (dbl)        (dbl)     (dbl)     (dbl)
## 1         anger 1.492863       321 2.193242e-05  274.3619 1.2353162
## 2  anticipation 1.169804       256 1.191668e-01  239.6467 0.9604950
## 3       disgust 1.677259       207 1.777434e-05  170.2164 1.3116238
## 4          fear 1.560280       268 1.886129e-05  225.6487 1.2640494
## 5           joy 1.002605       199 1.000000e+00  198.7724 0.8089357
## 6      negative 1.692841       560 7.094486e-13  459.1363 1.4586926
## 7      positive 1.058760       555 3.820571e-01  541.4449 0.9303732
## 8       sadness 1.620044       303 1.150493e-06  251.9650 1.3260252
## 9      surprise 1.167925       159 2.174483e-01  148.9393 0.9083517
## 10        trust 1.128482       369 1.471929e-01  350.5114 0.9597478
## Variables not shown: conf.high (dbl), method (fctr), alternative (fctr)</code></pre>
</figure>
<p>And we can visualize it with a 95% confidence interval:</p>
<p><img alt="center" src="http://varianceexplained.org/figs/2016-08-09-trump-tweets/unnamed-chunk-8-1.svg"></p>
<p>Thus, Trump’s Android account uses about 40-80% more words related to <strong>disgust</strong>, <strong>sadness</strong>, <strong>fear</strong>, <strong>anger</strong>, and other “negative” sentiments than the iPhone account does. (The positive emotions weren’t different to a statistically significant extent).</p>
<p>We’re especially interested in which words drove this different in sentiment. Let’s consider the words with the largest changes within each category:</p>
<p><img alt="center" src="http://varianceexplained.org/figs/2016-08-09-trump-tweets/unnamed-chunk-9-1.svg"></p>
<p>This confirms that lots of words annotated as negative sentiments (with a few exceptions like “crime” and “terrorist”) are more common in Trump’s Android tweets than the campaign’s iPhone tweets.</p>
<h3>Conclusion: the ghost in the political machine</h3>
<p>I was fascinated by the recent <a target="_blank" href="http://www.newyorker.com/magazine/2016/07/25/donald-trumps-ghostwriter-tells-all">New Yorker article</a> about Tony Schwartz, Trump’s ghostwriter for The Art of the Deal. Of particular interest was how Schwartz imitated Trump’s voice and philosophy:</p>
<blockquote>
<p>In his journal, Schwartz describes the process of trying to make Trump’s voice palatable in the book. It was kind of “a trick,” he writes, to mimic Trump’s blunt, staccato, no-apologies delivery while making him seem almost boyishly appealing…. Looking back at the text now, Schwartz says, “I created a character far more winning than Trump actually is.”</p>
</blockquote>
<p>Like any journalism, data journalism is ultimately about human interest, and there’s one human I’m interested in: who is writing these iPhone tweets?</p>
<p>The majority of the tweets from the iPhone are fairly benign declarations. But consider cases like these, both posted from an iPhone:</p>
<blockquote data-lang="en" class="twitter-tweet"><p dir="ltr" lang="en">Like the worthless <a target="_blank" href="https://twitter.com/NYDailyNews">@NYDailyNews</a>, looks like <a target="_blank" href="https://twitter.com/politico">@politico</a> will be going out of business. Bad reporting- no money, no cred!</p>
<p>— Donald J. Trump (@realDonaldTrump) <a target="_blank" href="https://twitter.com/realDonaldTrump/status/697404736211959808">February 10, 2016</a></p></blockquote>
<p></p>
<blockquote data-lang="en" class="twitter-tweet"><p dir="ltr" lang="en">Failing <a target="_blank" href="https://twitter.com/nytimes">@NYTimes</a> will always take a good story about me and make it bad. Every article is unfair and biased. Very sad!</p>
<p>— Donald J. Trump (@realDonaldTrump) <a target="_blank" href="https://twitter.com/realDonaldTrump/status/733691591198703616">May 20, 2016</a></p></blockquote>
<p></p>
<p>These tweets certainly sound like the Trump we all know. Maybe our above analysis isn’t complete: maybe Trump has sometimes, however rarely, tweeted from an iPhone (perhaps dictating, or just using it when his own battery ran out). But what if our hypothesis is right, and these weren’t authored by the candidate- just someone trying their best to sound like him?</p>
<p>Or what about tweets like this (also iPhone), which defend Trump’s slogan- but doesn’t really sound like something he’d write?</p>
<blockquote data-lang="en" class="twitter-tweet"><p dir="ltr" lang="en">Our country does not feel 'great already' to the millions of wonderful people living in poverty, violence and despair.</p>
<p>— Donald J. Trump (@realDonaldTrump) <a target="_blank" href="https://twitter.com/realDonaldTrump/status/758492727583576064">July 28, 2016</a></p></blockquote>
<p></p>
<p>A lot has been written about Trump’s mental state. But I’d really rather get inside the head of this anonymous staffer, whose job is to imitate Trump’s unique cadence (“Very sad!”), or to put a positive spin on it, to millions of followers. Is he a true believer, or just a cog in a political machine, mixing whatever mainstream appeal he can into the @realDonaldTrump concoction? Like Tony Schwartz, will he one day regret his involvement?</p>
<div>
<ol>
<li>
<p>To keep the post concise I don’t show all of the code, especially code that generates figures. But you can find the full code <a target="_blank" href="https://github.com/dgrtwo/dgrtwo.github.com/blob/master/_R/2016-08-09-trump-tweets.Rmd">here</a>. <a target="_blank" href="http://varianceexplained.org/r/trump-tweets/#fnref:fullcode">↩</a></p>
</li>
<li>
<p>We had to use a custom regular expression for Twitter, since typical tokenizers would split the # off of hashtags and @ off of usernames. We also removed links and ampersands (<code>&amp;amp;</code>) from the text. <a target="_blank" href="http://varianceexplained.org/r/trump-tweets/#fnref:regex">↩</a></p>
</li>
<li>
<p>The “plus ones,” called <a target="_blank" href="https://en.wikipedia.org/wiki/Additive_smoothing">Laplace smoothing</a> are to avoid dividing by zero and to put <a target="_blank" href="http://varianceexplained.org/r/empirical_bayes_baseball/">more trust in common words</a>. <a target="_blank" href="http://varianceexplained.org/r/trump-tweets/#fnref:plusone">↩</a></p>
</li>
</ol>
</div>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/text-analysis-of-trumps-tweets-confirms-he-writes-only-the-angrier-android-half/"></div></div><div><a data-url="https://www.r-bloggers.com/text-analysis-of-trumps-tweets-confirms-he-writes-only-the-angrier-android-half/" data-counturl="https://www.r-bloggers.com/text-analysis-of-trumps-tweets-confirms-he-writes-only-the-angrier-android-half/" data-count="vertical" data-text="Text analysis of Trump’s tweets confirms he writes only the (angrier) Android half" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://varianceexplained.org/r/trump-tweets/"> Variance Explained</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/04egoksAdzE"></a></div>