<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/analyzing-rtweet-data-with-kerasformula/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://tensorflow.rstudio.com/blog/analyzing-rtweet-data-with-kerasformula.html"> TensorFlow for R</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>This is guest post contributed by <a rel="nofollow" href="https://sites.google.com/site/petemohanty/" target="_blank">Pete Mohanty</a>, creator of the <a rel="nofollow" href="https://cran.r-project.org/web/packages/kerasformula/index.html" target="_blank">kerasformula</a> package.</p>
</div>
<div>
<h2>Overview</h2>
<p>The <a rel="nofollow" href="https://cran.r-project.org/web/packages/kerasformula/index.html" target="_blank">kerasformula</a> package offers a high-level interface for the R interface to <a rel="nofollow" href="https://keras.rstudio.com/" target="_blank">Keras</a>. It’s main interface is the <code>kms</code> function, a regression-style interface to <code>keras_model_sequential</code> that uses formulas and sparse matrices.</p>
<p>The kerasformula package is available on CRAN, and can be installed with:</p>
<div>
<pre><code><span># install the kerasformula package</span>
<span>install.packages</span>(<span>&quot;kerasformula&quot;</span>)    <span># or devtools::install_github(&quot;rdrr1990/kerasformula&quot;)</span>
<span>library</span>(kerasformula)

<span># install the core keras library (if you haven't already done so)</span>
<span># see ?install_keras() for options e.g. install_keras(tensorflow = &quot;gpu&quot;)</span>
<span>install_keras</span>()</code></pre>
</div>
</div>
<div>
<h2>The kms() function</h2>
<p>Many classic machine learning tutorials assume that data come in a relatively homogenous form (e.g., pixels for digit recognition or word counts or ranks) which can make coding somewhat cumbersome when data is contained in a heterogenous data frame. <code><a rel="nofollow" href="http://www.rdocumentation.org/packages/kerasformula/topics/kms?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">kms()</a></code> takes advantage of the flexibility of R formulas to smooth this process.</p>
<p><code>kms</code> builds dense neural nets and, after fitting them, returns a single object with predictions, measures of fit, and details about the function call. <code>kms</code> accepts a number of parameters including the loss and activation functions found in <code>keras</code>. <code>kms</code> also accepts compiled <code>keras_model_sequential</code> objects allowing for even further customization. This little demo shows how <code>kms</code> can aid is model building and hyperparameter selection (e.g., batch size) starting with raw data gathered using <code>library(rtweet)</code>.</p>
<p>Let’s look at #rstats tweets (excluding retweets) for a six-day period ending January 24, 2018 at 10:40. This happens to give us a nice reasonable number of observations to work with in terms of runtime (and the purpose of this document is to show syntax, not build particularly predictive models).</p>
<div>
<pre><code>rstats &lt;-<span> </span><span>search_tweets</span>(<span>&quot;#rstats&quot;</span>, <span>n =</span> <span>10000</span>, <span>include_rts =</span> <span>FALSE</span>)
<span>dim</span>(rstats)</code></pre>
</div>
<pre><code>  [1] 2840   42</code></pre>
<p>Suppose our goal is to predict how popular tweets will be based on how often the tweet was retweeted and favorited (which correlate strongly).</p>
<div>
<pre><code><span>cor</span>(rstats<span>$</span>favorite_count, rstats<span>$</span>retweet_count, <span>method=</span><span>&quot;spearman&quot;</span>)</code></pre>
</div>
<pre><code>    [1] 0.7051952</code></pre>
<p>Since few tweeets go viral, the data are quite skewed towards zero.</p>
<div>
<img src="https://i1.wp.com/tensorflow.rstudio.com/blog/images/analyzing-rtweet-data-with-kerasformula/densities-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
</div>
<div>
<h2>Getting the most out of formulas</h2>
<p>Let’s suppose we are interested in putting tweets into categories based on popularity but we’re not sure how finely-grained we want to make distinctions. Some of the data, like <code>rstats$mentions_screen_name</code> comes in a list of varying lengths, so let’s write a helper function to count non-NA entries.</p>
<div>
<pre><code>n &lt;-<span> </span><span>function</span>(x) {
  <span>unlist</span>(<span>lapply</span>(x, <span>function</span>(y){<span>length</span>(y) <span>-</span><span> </span><span>is.na</span>(y[<span>1</span>])}))
}</code></pre>
</div>
<p>Let’s start with a dense neural net, the default of <code>kms</code>. We can use base R functions to help clean the data–in this case, <code>cut</code> to discretize the outcome, <code>grepl</code> to look for key words, and <code>weekdays</code> and <code>format</code> to capture different aspects of the time the tweet was posted.</p>
<div>
<pre><code>breaks &lt;-<span> </span><span>c</span>(<span>-</span><span>1</span>, <span>0</span>, <span>1</span>, <span>10</span>, <span>100</span>, <span>1000</span>, <span>10000</span>)
popularity &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/kerasformula/topics/kms?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">kms</a></span>(<span>cut</span>(retweet_count <span>+</span><span> </span>favorite_count, breaks) <span>~</span><span> </span>screen_name <span>+</span><span> </span>source <span>+</span><span> </span>
<span> </span><span>n</span>(hashtags) <span>+</span><span> </span><span>n</span>(mentions_screen_name) <span>+</span><span> </span>
<span> </span><span>n</span>(urls_url) <span>+</span><span> </span><span>nchar</span>(text) <span>+</span>
<span> </span><span>grepl</span>(<span>'photo'</span>, media_type) <span>+</span>
<span> </span><span>weekdays</span>(created_at) <span>+</span><span> </span>
<span> </span><span>format</span>(created_at, <span>'%H'</span>), rstats)
<span>plot</span>(popularity<span>$</span>history) <span>+</span><span> </span><span>ggtitle</span>(<span>paste</span>(<span>&quot;#rstat popularity:&quot;</span>,
                                         <span>paste0</span>(<span>round</span>(<span>100</span><span>*</span>popularity<span>$</span>evaluations<span>$</span>acc, <span>1</span>), <span>&quot;%&quot;</span>),
                                         <span>&quot;out-of-sample accuracy&quot;</span>)) <span>+</span><span> </span><span>theme_minimal</span>()
popularity<span>$</span>confusion</code></pre>
</div>
<div>
<img src="https://i0.wp.com/tensorflow.rstudio.com/blog/images/analyzing-rtweet-data-with-kerasformula/first_model-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<pre><code>popularity$confusion

                    (-1,0] (0,1] (1,10] (10,100] (100,1e+03] (1e+03,1e+04]
      (-1,0]            37    12     28        2           0             0
      (0,1]             14    19     72        1           0             0
      (1,10]             6    11    187       30           0             0
      (10,100]           1     3     54       68           0             0
      (100,1e+03]        0     0      4       10           0             0
      (1e+03,1e+04]      0     0      0        1           0             0</code></pre>
<p>The model only classifies about 55% of the out-of-sample data correctly and that predictive accuracy doesn’t improve after the first ten epochs. The confusion matrix suggests that model does best with tweets that are retweeted a handful of times but overpredicts the 1-10 level. The <code>history</code> plot also suggests that out-of-sample accuracy is not very stable. We can easily change the breakpoints and number of epochs.</p>
<div>
<pre><code>breaks &lt;-<span> </span><span>c</span>(<span>-</span><span>1</span>, <span>0</span>, <span>1</span>, <span>25</span>, <span>50</span>, <span>75</span>, <span>100</span>, <span>500</span>, <span>1000</span>, <span>10000</span>)
popularity &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/kerasformula/topics/kms?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">kms</a></span>(<span>cut</span>(retweet_count <span>+</span><span> </span>favorite_count, breaks) <span>~</span><span> </span>
<span> </span><span>n</span>(hashtags) <span>+</span><span> </span><span>n</span>(mentions_screen_name) <span>+</span><span> </span><span>n</span>(urls_url) <span>+</span>
<span> </span><span>nchar</span>(text) <span>+</span>
<span> </span>screen_name <span>+</span><span> </span>source <span>+</span>
<span> </span><span>grepl</span>(<span>'photo'</span>, media_type) <span>+</span>
<span> </span><span>weekdays</span>(created_at) <span>+</span><span> </span>
<span> </span><span>format</span>(created_at, <span>'%H'</span>), rstats, <span>Nepochs =</span> <span>10</span>)
<span>plot</span>(popularity<span>$</span>history) <span>+</span><span> </span><span>ggtitle</span>(<span>paste</span>(<span>&quot;#rstat popularity (new breakpoints):&quot;</span>,
                                         <span>paste0</span>(<span>round</span>(<span>100</span><span>*</span>popularity<span>$</span>evaluations<span>$</span>acc, <span>1</span>), <span>&quot;%&quot;</span>),
                                         <span>&quot;out-of-sample accuracy&quot;</span>)) <span>+</span><span> </span><span>theme_minimal</span>()</code></pre>
</div>
<div>
<img src="https://i2.wp.com/tensorflow.rstudio.com/blog/images/analyzing-rtweet-data-with-kerasformula/change_breaks-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p>That helped some (about 5% additional predictive accuracy). Suppose we want to add a little more data. Let’s first store the input formula.</p>
<div>
<pre><code>pop_input &lt;-<span> &quot;cut(retweet_count + favorite_count, breaks) ~ </span>
<span> n(hashtags) + n(mentions_screen_name) + n(urls_url) +</span>
<span> nchar(text) +</span>
<span> screen_name + source +</span>
<span> grepl('photo', media_type) +</span>
<span> weekdays(created_at) + </span>
<span> format(created_at, '%H')&quot;</span></code></pre>
</div>
<p>Here we use <code>paste0</code> to add to the formula by looping over user IDs adding something like:</p>
<pre><code>grepl(&quot;12233344455556&quot;, mentions_user_id)</code></pre>
<div>
<pre><code>mentions &lt;-<span> </span><span>unlist</span>(rstats<span>$</span>mentions_user_id)
mentions &lt;-<span> </span><span>unique</span>(mentions[<span>which</span>(<span>table</span>(mentions) <span>></span><span> </span><span>5</span>)]) <span># remove infrequent mentions</span>
mentions &lt;-<span> </span>mentions[<span>!</span><span>is.na</span>(mentions)] <span># drop NA</span>

<span>for</span>(i <span>in</span> mentions)
  pop_input &lt;-<span> </span><span>paste0</span>(pop_input, <span>&quot; + &quot;</span>, <span>&quot;grepl(&quot;</span>, i, <span>&quot;, mentions_user_id)&quot;</span>)

popularity &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/kerasformula/topics/kms?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">kms</a></span>(pop_input, rstats)</code></pre>
</div>
<div>
<img src="https://i1.wp.com/tensorflow.rstudio.com/blog/images/analyzing-rtweet-data-with-kerasformula/mentionsplot-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p>That helped a touch but the predictive accuracy is still fairly unstable across epochs…</p>
</div>
<div>
<h2>Customizing layers with kms()</h2>
<p>We could add more data, perhaps add individual words from the text or some other summary stat (<code>mean(text %in% LETTERS)</code> to see if all caps explains popularity). But let’s alter the neural net.</p>
<p>The <code>input.formula</code> is used to create a sparse model matrix. For example, <code>rstats$source</code> (Twitter or Twitter-client application type) and <code>rstats$screen_name</code> are character vectors that will be dummied out. How many columns does it have?</p>
<div>
<pre><code>popularity<span>$</span>P</code></pre>
</div>
<pre><code>    [1] 1277</code></pre>
<p>Say we wanted to reshape the layers to transition more gradually from the input shape to the output.</p>
<div>
<pre><code>popularity &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/kerasformula/topics/kms?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">kms</a></span>(pop_input, rstats,
                  <span>layers =</span> <span>list</span>(<span>units =</span> <span>c</span>(<span>1024</span>, <span>512</span>, <span>256</span>, <span>128</span>, <span>NA</span>),
                                <span>activation =</span> <span>c</span>(<span>&quot;relu&quot;</span>, <span>&quot;relu&quot;</span>, <span>&quot;relu&quot;</span>, <span>&quot;relu&quot;</span>, <span>&quot;softmax&quot;</span>), 
                                <span>dropout =</span> <span>c</span>(<span>0.5</span>, <span>0.45</span>, <span>0.4</span>, <span>0.35</span>, <span>NA</span>)))</code></pre>
</div>
<div>
<img src="https://i0.wp.com/tensorflow.rstudio.com/blog/images/analyzing-rtweet-data-with-kerasformula/customplot-1.png?w=456&ssl=1" data-recalc-dims="1">
</div>
<p><code>kms</code> builds a <code>keras_sequential_model()</code>, which is a stack of linear layers. The input shape is determined by the dimensionality of the model matrix (<code>popularity$P</code>) but after that users are free to determine the number of layers and so on. The <code>kms</code> argument <code>layers</code> expects a list, the first entry of which is a vector <code>units</code> with which to call <code><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/layer_dense.html" target="_blank">keras::layer_dense()</a></code>. The first element the number of <code>units</code> in the first layer, the second element for the second layer, and so on (<code>NA</code> as the final element connotes to auto-detect the final number of units based on the observed number of outcomes). <code>activation</code> is also passed to <code>layer_dense()</code> and may take values such as <code>softmax</code>, <code>relu</code>, <code>elu</code>, and <code>linear</code>. (<code>kms</code> also has a separate parameter to control the optimizer; by default <code>kms(... optimizer = 'rms_prop')</code>.) The <code>dropout</code> that follows each dense layer rate prevents overfitting (but of course isn’t applicable to the final layer).</p>
</div>
<div>
<h2>Choosing a Batch Size</h2>
<p>By default, <code>kms</code> uses batches of 32. Suppose we were happy with our model but didn’t have any particular intuition about what the size should be.</p>
<div>
<pre><code>Nbatch &lt;-<span> </span><span>c</span>(<span>16</span>, <span>32</span>, <span>64</span>)
Nruns &lt;-<span> </span><span>4</span>
accuracy &lt;-<span> </span><span>matrix</span>(<span>nrow =</span> Nruns, <span>ncol =</span> <span>length</span>(Nbatch))
<span>colnames</span>(accuracy) &lt;-<span> </span><span>paste0</span>(<span>&quot;Nbatch_&quot;</span>, Nbatch)

est &lt;-<span> </span><span>list</span>()
<span>for</span>(i <span>in</span> <span>1</span><span>:</span>Nruns){
  <span>for</span>(j <span>in</span> <span>1</span><span>:</span><span>length</span>(Nbatch)){
   est[[i]] &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/kerasformula/topics/kms?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">kms</a></span>(pop_input, rstats, <span>Nepochs =</span> <span>2</span>, <span>batch_size =</span> Nbatch[j])
   accuracy[i,j] &lt;-<span> </span>est[[i]][[<span>&quot;evaluations&quot;</span>]][[<span>&quot;acc&quot;</span>]]
  }
}
  
<span>colMeans</span>(accuracy)</code></pre>
</div>
<pre><code>    Nbatch_16 Nbatch_32 Nbatch_64 
    0.5088407 0.3820850 0.5556952 </code></pre>
<p>For the sake of curtailing runtime, the number of epochs was set arbitrarily short but, from those results, 64 is the best batch size.</p>
</div>
<div>
<h2>Making predictions for new data</h2>
<p>Thus far, we have been using the default settings for <code>kms</code> which first splits data into 80% training and 20% testing. Of the 80% training, a certain portion is set aside for validation and that’s what produces the epoch-by-epoch graphs of loss and accuracy. The 20% is only used at the end to assess predictive accuracy. But suppose you wanted to make predictions on a new data set…</p>
<div>
<pre><code>popularity &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/kerasformula/topics/kms?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">kms</a></span>(pop_input, rstats[<span>1</span><span>:</span><span>1000</span>,])
predictions &lt;-<span> </span><span>predict</span>(popularity, rstats[<span>1001</span><span>:</span><span>2000</span>,])
predictions<span>$</span>accuracy</code></pre>
</div>
<pre><code>    [1] 0.579</code></pre>
<div>
<pre><code><span># predictions$confusion</span></code></pre>
</div>
<p>Because the formula creates a dummy variable for each screen name and mention, any given set of tweets is all but guaranteed to have different columns. <code>predict.kms_fit</code> is an <code>S3 method</code> that takes the new data and constructs a (sparse) model matrix that preserves the original structure of the training matrix. <code>predict</code> then returns the predictions along with a confusion matrix and accuracy score.</p>
<p>If your newdata has the same observed levels of y and columns of x_train (the model matrix), you can also use <code><a rel="nofollow" href="https://tensorflow.rstudio.com/keras/reference/predict_proba.html" target="_blank">keras::predict_classes</a></code> on <code>object$model</code>.</p>
</div>
<div>
<h2>Using a compiled Keras model</h2>
<p>This section shows how to input a model compiled in the fashion typical to <code>library(keras)</code>, which is useful for more advanced models. Here is an example for <code>lstm</code> analogous to the <a rel="nofollow" href="https://tensorflow.rstudio.com/keras/articles/examples/imdb_lstm.html" target="_blank">imbd with Keras example</a>.</p>
<div>
<pre><code>k &lt;-<span> </span><span>keras_model_sequential</span>()
k <span>%>%</span>
<span> </span><span>layer_embedding</span>(<span>input_dim =</span> popularity<span>$</span>P, <span>output_dim =</span> popularity<span>$</span>P) <span>%>%</span><span> </span>
<span> </span><span>layer_lstm</span>(<span>units =</span> <span>512</span>, <span>dropout =</span> <span>0.4</span>, <span>recurrent_dropout =</span> <span>0.2</span>) <span>%>%</span><span> </span>
<span> </span><span>layer_dense</span>(<span>units =</span> <span>256</span>, <span>activation =</span> <span>&quot;relu&quot;</span>) <span>%>%</span>
<span> </span><span>layer_dropout</span>(<span>0.3</span>) <span>%>%</span>
<span> </span><span>layer_dense</span>(<span>units =</span> <span>8</span>, <span># number of levels observed on y (outcome) </span>
              <span>activation =</span> <span>'sigmoid'</span>)

k <span>%>%</span><span> </span><span>compile</span>(
  <span>loss =</span> <span>'categorical_crossentropy'</span>,
  <span>optimizer =</span> <span>'rmsprop'</span>,
  <span>metrics =</span> <span>c</span>(<span>'accuracy'</span>)
)

popularity_lstm &lt;-<span> </span><span><a rel="nofollow" href="http://www.rdocumentation.org/packages/kerasformula/topics/kms?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">kms</a></span>(pop_input, rstats, k)</code></pre>
</div>
</div>
<div>
<h2>Questions? Comments?</h2>
<p>Drop me a line via the project’s <a rel="nofollow" href="https://github.com/rdrr1990/kerasformula" target="_blank">Github repo</a>. Special thanks to <a rel="nofollow" href="https://github.com/dfalbel" target="_blank">@dfalbel</a> and <a rel="nofollow" href="https://github.com/jjallaire" target="_blank">@jjallaire</a> for helpful suggestions!!</p>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://tensorflow.rstudio.com/blog/analyzing-rtweet-data-with-kerasformula.html"> TensorFlow for R</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/tSHp2jXwb0c" height="1"></a></div>