<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/data-wonderland-christmas-songs-from-the-viewpoint-of-a-data-scientist/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://blog.eoda.de/2017/12/19/data-wonderland-christmas-songs-from-the-viewpoint-of-a-data-scientist/"> eoda english R news</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Whether „Driving Home for Christmas“, „Winter Wonderland“, „Let it snow!“ or „Last Christmas“ – every year christmas songs are taking over the charts again. While average Joe is joyfully putting on the next christmas song, the data scientist starts his journey of discovery through the snowy music history.</p>
<p><a rel="nofollow" href="https://i2.wp.com/blog.eoda.de/wp-content/uploads/2017/12/hidden_xmas.png?ssl=1" target="_blank"><img sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px" src="https://i2.wp.com/blog.eoda.de/wp-content/uploads/2017/12/hidden_xmas.png?w=450&ssl=1" alt="" class="wp-image-4543" data-recalc-dims="1"></a></p>
<p> </p>
<p>The data set comes from <a rel="nofollow" href="https://www.kaggle.com/mousehead/songlyrics" target="_blank">55000+ Song Lyrics</a>, which contains over 55,000+ songs. It is a data frame with 55,000+ rows and four columns:</p>
<ul>
<li><strong>artist</strong></li>
<li><strong>song</strong></li>
<li><strong>link</strong></li>
<li><strong>text</strong></li>
</ul>
<pre class="wp-code-highlight"><code>library(igraph)
library(ggraph)
library(ggplot2)
library(wordcloud2)
library(dplyr)
library(widyr)
library(tidytext)
library(tm)
library(stringr)
library(topicmodels)
library(reshape2)
library(quanteda)
library(Rtsne)
library(DT)
library(knitr)
library(animation)
library(ldatuning)

set.seed(201712)

# Preperation -------------------------------------------------------------
songs &lt;- read.csv(&quot;songdata.csv&quot;)
songs$song &lt;- songs$song %>% as.character()
songs$artist &lt;- songs$artist %>% as.character()
songs$link &lt;- songs$link %>% as.character()
songs$text &lt;- songs$text %>% as.character()</code></pre>
<p>Our goal is to perform a comprehensive analysis of the song texts to identify the Christmas songs. In order to do so, first we add an additional column to the data frame to give each song a <strong>label</strong> of either <strong>Christmas</strong> or <strong>Not Christmas</strong>, where every song which contains the words <strong>Christmas</strong>, <strong>Xmas</strong> or <strong>X-mas</strong> will be labeled as <strong>Christmas</strong> and otherwise as <strong>Not Christmas</strong>.</p>
<pre class="wp-code-highlight"><code># Initialization of the Labels
label &lt;- character(dim(songs)[1])

for(i in 1:dim(songs)[1]){
  if(str_detect(songs$song[i], &quot;Christmas&quot;) |
     str_detect(songs$song[i], &quot;X-mas&quot;) |
     str_detect(songs$song[i], &quot;Xmas&quot;)){
    label[i] &lt;- &quot;Christmas&quot;
  } else{
    label[i] &lt;- &quot;Not Christmas&quot;
  }
}

songs &lt;- songs %>% 
  mutate(Label = label)

</code></pre>
<p>This is just the initialization of the labels, later we will apply Naive Bayes to a training set to identify the other Christmas songs. First of all, we will start by exploring the data set by means of some intuitive descriptive approaches.</p>
<pre class="wp-code-highlight"><code>D3Vis &lt;- function(edgeList, directed){
  colnames(edgeList) &lt;- c(&quot;SourceName&quot;, &quot;TargetName&quot;, &quot;Weight&quot;)
  
  # Min-Max &amp; Inverse scaling, because the weights should represent distance/similarity
  edgeList$Weight &lt;- 1 - edgeList$Weight
  weight.min &lt;- edgeList$Weight %>% min
  weight.max &lt;- edgeList$Weight %>% max
  
  edgeList$Weight &lt;- (edgeList$Weight - weight.min)/(weight.max - weight.min)
  
  # Create a graph. Use simplyfy to ensure that there are no duplicated edges or self loops
  gD &lt;- igraph::simplify(igraph::graph.data.frame(edgeList, directed=directed))
  
  # Create a node list object (actually a data frame object) that will contain information about nodes
  nodeList &lt;- data.frame(ID = c(0:(igraph::vcount(gD) - 1)), # because networkD3 library requires IDs to start at 0
                         nName = igraph::V(gD)$name)
  
  # Map node names from the edge list to node IDs
  getNodeID &lt;- function(x){
    which(x == igraph::V(gD)$name) - 1 # to ensure that IDs start at 0
  }
  # And add them to the edge list
  edgeList &lt;- plyr::ddply(edgeList, .variables = c(&quot;SourceName&quot;, &quot;TargetName&quot;, &quot;Weight&quot;), 
                          function (x) data.frame(SourceID = getNodeID(x$SourceName), 
                                                  TargetID = getNodeID(x$TargetName)))
  
  #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++#
  # Calculate some node properties and node similarities that will be used to illustrate 
  # different plotting abilities and add them to the edge and node lists
  
  # Calculate degree for all nodes
  nodeList &lt;- cbind(nodeList, nodeDegree=igraph::degree(gD, v = igraph::V(gD), mode = &quot;all&quot;))
  
  # Calculate betweenness for all nodes
  betAll &lt;- igraph::betweenness(gD, v = igraph::V(gD), directed = directed) / (((igraph::vcount(gD) - 1) * (igraph::vcount(gD)-2)) / 2)
  betAll.norm &lt;- (betAll - min(betAll))/(max(betAll) - min(betAll))
  nodeList &lt;- cbind(nodeList, nodeBetweenness=100*betAll.norm) # We are scaling the value by multiplying it by 100 for visualization purposes only (to create larger nodes)
  rm(betAll, betAll.norm)
  
  #Calculate Dice similarities between all pairs of nodes
  dsAll &lt;- igraph::similarity.dice(gD, vids = igraph::V(gD), mode = &quot;all&quot;)
  
  F1 &lt;- function(x) {data.frame(diceSim = dsAll[x$SourceID +1, x$TargetID + 1])}
  edgeList &lt;- plyr::ddply(edgeList, .variables=c(&quot;SourceName&quot;, &quot;TargetName&quot;, &quot;Weight&quot;, &quot;SourceID&quot;, &quot;TargetID&quot;), 
                          function(x) data.frame(F1(x)))
  
  rm(dsAll, F1, getNodeID, gD)
  
  #++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++#
  # We will also create a set of colors for each edge, based on their dice similarity values
  # We'll interpolate edge colors based on the using the &quot;colorRampPalette&quot; function, that 
  # returns a function corresponding to a collor palete of &quot;bias&quot; number of elements (in our case, that
  # will be a total number of edges, i.e., number of rows in the edgeList data frame)
  F2 &lt;- colorRampPalette(c(&quot;#FFFF00&quot;, &quot;#FF0000&quot;), bias = nrow(edgeList), space = &quot;rgb&quot;, interpolate = &quot;linear&quot;)
  colCodes &lt;- F2(length(unique(edgeList$diceSim)))
  edges_col &lt;- sapply(edgeList$diceSim, function(x) colCodes[which(sort(unique(edgeList$diceSim)) == x)])
  
  rm(colCodes, F2)
  
  #++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++#
  # revise transformation of the weights
  edgeList$Weight &lt;- -(edgeList$Weight*(weight.max - weight.min) + weight.min - 1)
  
  # Let's create a network
  D3_network_LM &lt;- networkD3::forceNetwork(Links = edgeList, # data frame that contains info about edges
                                           Nodes = nodeList, # data frame that contains info about nodes
                                           Source = &quot;SourceID&quot;, # ID of source node 
                                           Target = &quot;TargetID&quot;, # ID of target node
                                           Value = &quot;Weight&quot;, # value from the edge list (data frame) that will be used to value/weight relationship amongst nodes
                                           NodeID = &quot;nName&quot;, # value from the node list (data frame) that contains node description we want to use (e.g., node name)
                                           Nodesize = &quot;nodeBetweenness&quot;,  # value from the node list (data frame) that contains value we want to use for a node size
                                           Group = &quot;nodeDegree&quot;,  # value from the node list (data frame) that contains value we want to use for node color
                                           #                                         height = 500, # Size of the plot (vertical)
                                           #                                         width = 1000,  # Size of the plot (horizontal)
                                           fontSize = 20, # Font size
                                           linkDistance = networkD3::JS(&quot;function(d) { return 10*d.value; }&quot;), # Function to determine distance between any two nodes, uses variables already defined in forceNetwork function (not variables from a data frame)
                                           #                                         linkWidth = networkD3::JS(&quot;function(d) { return d.value/5; }&quot;),# Function to determine link/edge thickness, uses variables already defined in forceNetwork function (not variables from a data frame)
                                           opacity = 0.85, # opacity
                                           arrows = directed,
                                           zoom = TRUE, # ability to zoom when click on the node
                                           #                                         opacityNoHover = 0.1, # opacity of labels when static
                                           legend = F,
                                           linkColour = edges_col) # edge colors
  
  # Plot network
  D3_network_LM 
}
</code></pre>
<h1><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#exploration-the-initial-christmas-songs" target="_blank"></a><strong>Exploration of the initial Christmas songs</strong></h1>
<h2><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#cleaning--tokenization" target="_blank"></a><em>Cleaning &amp; Tokenization</em></h2>
<p>We should start with the data cleaning and tokenization. Afterwards, the Christmas songs will be selected and saved as a variable.</p>
<pre class="wp-code-highlight"><code>songs.unnest &lt;- songs %>% 
  unnest_tokens(word, text) %>% 
  anti_join(tibble(word = stop_words$word)) %>% 
  filter(!str_detect(word, &quot;\\d+&quot;))

xmas.unnest &lt;- songs.unnest %>% 
  filter(Label == &quot;Christmas&quot;)
</code></pre>
<h2><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#correlation-analysis" target="_blank"></a><em>Correlation Analysis</em></h2>
<p>Now we can start analyzing the initial Christmas songs by means of correlations from different perspectives. In the following, we will visualize the correlations with the <a rel="nofollow" href="http://christophergandrud.github.io/networkD3/" target="_blank">networkD3 html widget</a> where nodes with the same total number of connections will be given the same color and the color of the edge implies the number of common neighbors shared by two nodes. Moreover, the size of a node indicates the centrality of it, which is defined by the betweenness, i.e. the number of shortest paths going through it. Where the distance between two nodes is the minimum maximum transformation of 1 minus the correlation, which makes sense because intuitively the higher the correlation, the nearer two nodes should be. Moreover, the shorter the distance, the wider the edge.</p>
<p>Note that the correlations are always based on lyrics.</p>
<h3><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#correlation-between-words" target="_blank"></a><em>Correlation between words</em></h3>
<p>The correlation between words which appeared more than 100 times and are correlated with at least one other word with a correlation greater than 0.55.</p>
<pre class="wp-code-highlight"><code>correlation.words &lt;- xmas.unnest %>% 
  group_by(word) %>% 
  filter(n() > 100) %>% 
  ungroup() %>% 
  pairwise_cor(word, song, sort = T)

# Network visualization
correlation.words %>% 
  filter(correlation > 0.55) %>% 
  D3Vis(directed = F)
</code></pre>
<h3><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#correlation-between-songs" target="_blank"></a><em>Correlation between songs</em></h3>
<p>The correlation between songs which are correlated with at least 3 other songs with a correlation greater than 0.75. With this, we may detect similiar or just slightly modified songs.</p>
<pre class="wp-code-highlight"><code>correlation.songs &lt;- xmas.unnest %>%
  pairwise_cor(song, word, sort = T)

# Network visualization
correlation.songs %>% 
  filter(correlation > 0.75) %>% 
  group_by(item1) %>% 
  filter(n() >= 3) %>% 
  ungroup() %>% 
  D3Vis(directed = F)
</code></pre>
<h3><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#correlation-between-certain-words" target="_blank"></a><em>Correlation between certain words</em></h3>
<p>The correlation between certain words</p>
<pre class="wp-code-highlight"><code>correlation.words %>% 
  filter(item1 == &quot;christus&quot; |
           item1 == &quot;jesus&quot;  |
           item1 == &quot;snow&quot;   |
           item1 == &quot;reindeer&quot; |
           item1 == &quot;home&quot;   |
           item1 == &quot;holy&quot;   |
           item1 == &quot;love&quot;   |
           item1 == &quot;tree&quot;   |
           item1 == &quot;white&quot;  |
           item1 == &quot;christmas&quot;, 
         correlation > 0.4) %>% 
  D3Vis(directed = F)
</code></pre>
<h3><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#correlation-between-artists" target="_blank"></a><em>Correlation between artists</em></h3>
<p>The correlation between artists</p>
<pre class="wp-code-highlight"><code>correlation.artists &lt;- xmas.unnest %>% 
  pairwise_cor(artist, word, sort = T)

# Network Visualization
correlation.artists %>% 
  filter(correlation > 0.8) %>% 
  group_by(item1) %>% 
  filter(n() >= 3) %>% 
  ungroup() %>% 
  D3Vis(directed = F)
</code></pre>
<h2><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#word-cloud" target="_blank"></a><em><span>Word Cloud</span></em></h2>
<p>Word cloud of the initial Christmas songs</p>
<pre class="wp-code-highlight"><code>xmas.cloud &lt;- xmas.unnest %>% 
  count(word) %>% 
  as.data.frame()

xmas.cloud %>% 
  wordcloud2(minSize = 3, shape = 'star')
</code></pre>
<h1><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#naive-bayes" target="_blank"></a><strong>Naive Bayes</strong></h1>
<p>Naive Bayes is a popular supervised machine learning algorithm to handle classification problems with a huge amount of features. It is „naive“ in the sense that, conditioned on a class, the features are assumed to be independently distributed. In our case, we would like to know, given a bunch of features, i.e. the <a rel="nofollow" href="https://en.wikipedia.org/wiki/Tf%E2%80%93idf" target="_blank">tf-idf</a> of words in a document, whether a song should be classified as Christmas song or not by Naive Bayes.</p>
<p>Generally, given features $\mathbf{x} = (x_1, …, x_p)$ we have $$\begin{aligned} \mathbb{P}(C_k|\mathbf{x}) &amp;= \frac{\mathbb{P}(C_k)\mathbb{P}(\mathbf{x}|C_k)}{\mathbb{P(\mathbf{x})}} \ &amp;= \frac{\mathbb{P}(C_k)\prod_{i = 1}^p\mathbb{P}(x_i|C_k)}{\mathbb{P(\mathbf{x})}} \varpropto \mathbb{P}(C_k)\prod_{i = 1}^p\mathbb{P}(x_i|C_k) \end{aligned} $$ Where $\mathbb{P}(C_k)$ is called the <em>prior</em> and $\mathbb{P}(C_k|\mathbf{x})$ the <em>posterior</em> and $\mathbb{P}(\mathbf{x}|C_k)$ the <em>likelihood</em>. The MLE is obviously $$\hat{C}:= \underset{k}{\arg \max},\mathbb{P}(C_k)\prod_{i = 1}^p\mathbb{P}(x_i|C_k)$$</p>
<p>Because we assume that the features are independent conditioning on an arbitrary class. We may therefore estimate $\mathbb{P}(x_i|C_k), \forall i = 1,…, p$ independently of other features using a training set, which makes the whole thing much easier. The popular assumptions of the likelihood are Gaussian, multinomial or Bernoulli. The harder part of constructing the maximum likelihood estimator is the choice of the prior distribution, i.e. the probability distribution of the classes. Where it is usually assumed to be uniformly distributed or estimated by the class frequencies. In our case the multinomial distribution for the likelihood and the uniform distribution for the prior are used, which means we have no prejudice regarding the categorization of the songs without given further information.</p>
<h2><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#find-out-the-hidden-christmas-songs" target="_blank"></a><em>Identify the hidden Christmas songs</em></h2>
<pre class="wp-code-highlight"><code># Document Feature Matrix 
songs.dfm.tfidf &lt;- corpus(songs, text_field = &quot;text&quot;,
                          docid_field = &quot;song&quot;) %>% 
  dfm(tolower = T, 
      stem = TRUE,
      remove_punct = TRUE,
      remove = stopwords(&quot;english&quot;)) %>% 
  dfm_trim(min_count = 5, min_docfreq = 3) %>% 
  dfm_weight(type = &quot;tfidf&quot;)

# Determine the Indizes for the training set
christmas.index &lt;- which(label == &quot;Christmas&quot;)
not_christmas.index &lt;- which(label == &quot;Not Christmas&quot;)

christmas.train.index &lt;- christmas.index
not_christmas.train.index &lt;- sample(not_christmas.index, length(christmas.index))

train.index &lt;- c(christmas.train.index, not_christmas.train.index)

label.train &lt;- label[train.index]

trainning.set &lt;- songs.dfm.tfidf[train.index, ]

# Train the Model
classifier_NB &lt;- textmodel_NB(trainning.set, label.train)

# Prediction
predictions &lt;- classifier_NB %>% 
  predict(newdata = songs.dfm.tfidf)

# Confusion Matrix
confusion &lt;- table(predictions$nb.predicted, label)

confusion
</code></pre>
<p>So we have identified 2965 hidden Christmas songs and there are 2 songs out of the initial 500 Christmas songs that are rejected by Naive Bayes as Christmas songs.</p>
<h2><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#explore-the-hidden-christmas-songs" target="_blank"></a><em>Explore the hidden Christmas songs</em></h2>
<pre class="wp-code-highlight"><code>#Determine the Indizes for the hidden (not) Christmas Songs.
hidden.index &lt;- (predictions$nb.predicted == &quot;Christmas&quot;) &amp; (songs$Label == &quot;Not Christmas&quot;)
hidden_not.index &lt;- (predictions$nb.predicted == &quot;Not Christmas&quot;) &amp; (songs$Label == &quot;Christmas&quot;)

# Change the labels
label[hidden.index] &lt;- &quot;Hidden Christmas&quot;
label[hidden_not.index] &lt;- &quot;Hidden Not Christmas&quot;

songs$Label &lt;- label
songs.dfm.tfidf@docvars$Label &lt;- label

# Wordcloud for the hidden Christmas Songs
hidden.xmas &lt;- songs[hidden.index, ]

hidden.unnest &lt;- hidden.xmas %>% 
  unnest_tokens(word, text) %>% 
  anti_join(tibble(word = stop_words$word)) %>% 
  filter(!str_detect(word, &quot;\\d+&quot;))

hidden.unnest %>% 
  count(word) %>% 
  filter(n >= 5) %>% 
  as.data.frame() %>% 
  wordcloud2(shape = &quot;star&quot;, minSize = 5)
</code></pre>
<pre class="wp-code-highlight"><code>
# Correlation
hidden.correlation.words &lt;- hidden.unnest %>% 
  group_by(word) %>% 
  filter(n() > 15) %>% 
  ungroup() %>% 
  pairwise_cor(word, song, sort = T)

# Network visualization
hidden.correlation.words %>% 
  filter(correlation > 0.65) %>% 
  group_by(item1) %>% 
  filter(n() >= 20) %>% 
  ungroup() %>% 
  D3Vis(directed = F)
</code></pre>
<p>We have therefore successfully identified a bunch of religous christmas songs, whose titles usually do not contain the word „Christmas“ or „X-mas“.</p>
<h1><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#latent-dirichtlet-allocation--t-statistics-stochastic-neighbor-embedding" target="_blank"></a><strong>Latent Dirichtlet Allocation &amp; t-Statistics Stochastic Neighbor Embedding</strong></h1>
<h2><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#data-preparation" target="_blank"></a><em>Data Preparation</em></h2>
<p>Only the top 300 features for the Christmas songs including the hidden ones will be used to calculate the Rtsne &amp; LDA, else the memory space will not be sufficient.</p>
<pre class="wp-code-highlight"><code>xmas.dfm.tfidf &lt;- songs.dfm.tfidf %>% 
  dfm_subset(Label == &quot;Christmas&quot; | Label == &quot;Hidden Christmas&quot;)

songs.dfm.tfidf_300 &lt;- songs.dfm.tfidf %>% 
  dfm_select(pattern = xmas.dfm.tfidf %>% 
               topfeatures(300) %>% 
               names(), selection = &quot;keep&quot;)

xmas.dfm.tfidf_300 &lt;- xmas.dfm.tfidf %>% 
  dfm_select(pattern = xmas.dfm.tfidf %>% 
               topfeatures(300) %>% 
               names(), selection = &quot;keep&quot;)
</code></pre>
<h2><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#lda" target="_blank"></a><em>LDA</em></h2>
<p>LDA stands for Latent Dirichtlet Allocation, which was introduced in <a rel="nofollow" href="http://www.jmlr.org/papers/volume3/blei03a/blei03a.pdf" target="_blank">Blei, Ng, Jordan (2003)</a>. It is a generative probabilistic model of a corpus, where the documents are represented as random mixtures over latent topics and for a single document there are usually only a few topics that are assigned unneglectable probabilities. Moreover, each topic is characterized by a distribution over words, where usually only a small set of words will be assigned significant probabilities for a certain topic. Either the variational expectation maximization algorithm or Gibbs sampling is used for the statistical inference of the parameters.</p>
<p>LDA requires a fixed number of topics, i.e. it assumes that the number of topics should already be known before applying the algorithm. However, there are possibilities to determine the optimal number of topics by different performance metrics, see <a rel="nofollow" href="https://cran.r-project.org/web/packages/ldatuning/vignettes/topics.html" target="_blank">Nikita</a>, by using the package <strong>ldatuning</strong>.</p>
<pre class="wp-code-highlight"><code>OptimalNumber &lt;- FindTopicsNumber(LDA_xmas &lt;- xmas.dfm.tfidf_300 %>% 
                                    convert(to = &quot;topicmodels&quot;), 
                                  topics = seq(2, 8, by = 1),
                                  mc.cores = 2, 
                                  metrics = c(&quot;CaoJuan2009&quot;, &quot;Arun2010&quot;, &quot;Deveaud2014&quot;),
                                  method = &quot;VEM&quot;,
                                  verbose = T)
</code></pre>
<pre class="wp-code-highlight"><code>FindTopicsNumber_plot(OptimalNumber)
</code></pre>
<p>Therefore, we will choose 8 as the optimal number of topics.</p>
<pre class="wp-code-highlight"><code>LDA_xmas &lt;- xmas.dfm.tfidf_300 %>% 
  convert(to = &quot;topicmodels&quot;) %>% 
  LDA(k = 8)
</code></pre>
<p>We may use the package <strong>tidytext</strong> to inspect the topic probability distribution of each document, i.e. for each document the sum of the probabilities that it belongs to a topic from 1 to 8 is equal to 1.</p>
<pre class="wp-code-highlight"><code>LDA_xmas %>%
  tidy(matrix = &quot;gamma&quot;) %>% 
  datatable(rownames = F)
</code></pre>
<p>Analogously, we can also obtain the probability distribution of words for each topic, i.e. for each topic the sum of probabilities that it generates different words is equal to 1.</p>
<pre class="wp-code-highlight"><code>LDA_xmas %>%
  tidy(matrix = &quot;beta&quot;) %>% 
  datatable(rownames = F)
</code></pre>
<p>The top terms for each topic are:</p>
<pre class="wp-code-highlight"><code># LDA for the Christmas songs
terms(LDA_xmas, 10)
</code></pre>
<h2><a rel="nofollow" href="https://github.com/eodaGmbH/christmas_songs/blob/master/notebook.Rmd#t-sne" target="_blank"></a><em>t-SNE</em></h2>
<p>Developed by <a rel="nofollow" href="http://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf" target="_blank">van der Maaten and Hinton (2008)</a>, t-SNE stands for t-Statistics Stochastic Neighborhood Embedding, which is a dimensionality reduction technique that is formulated to capture the local clustering structure of the original data points. It is non-linear and non-deterministic.</p>
<p>We have generally speaking data points with high dimensionality $$x_1, …, x_n \in \mathbb{R}^N$$ and would like to calculate its counterparts $$y_1, …, y_n \in \mathbb{R}^M$$ in a low dimensional space, i.e. where $M