<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/the-bold-beautiful-character-similarities-using-word-embeddings/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://longhowlam.wordpress.com/2017/10/12/the-bold-beautiful-character-similarities-using-word-embeddings/"> R – Longhow Lam's Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2><img data-orig-file="https://longhowlam.files.wordpress.com/2017/10/bbpost.png?w=456" data-orig-size="790,409" data-medium-file="https://longhowlam.files.wordpress.com/2017/10/bbpost.png?w=456?w=300" data-attachment-id="2546" data-image-meta="{" data-image-description="" src="https://longhowlam.files.wordpress.com/2017/10/bbpost.png?w=456" data-permalink="https://longhowlam.wordpress.com/2017/10/12/the-bold-beautiful-character-similarities-using-word-embeddings/bbpost/" data-large-file="https://longhowlam.files.wordpress.com/2017/10/bbpost.png?w=456?w=440" alt="BBpost" data-image-title="BBpost" sizes="(max-width: 440px) 100vw, 440px" data-comments-opened="1" class="wp-image-2546" data-recalc-dims="1"></h2>
<h2>Introduction</h2>
<p>I often see advertisement for The Bold and The Beautiful, I have never watched a single episode of the series. Still, even as a data scientist you might be wondering how these beautiful ladies and gentlemen from the show are related to each other. I do not have the time to watch all these episodes to find out, so I am going to use <strong>word embeddings</strong> on recaps instead…</p>
<h2>Calculating word embeddings</h2>
<p>First, we need some data, from the first few google hits I got to the site <a rel="nofollow" href="http://soapcentral.com/bb/recaps/index.php" target="_blank">soap central</a>. Recaps can be found from the show that date back to 1997. Then, I used a little bit of rvest code to scrape the daily recaps into an R data set.</p>
<p>Word embedding is a technique to transform a word onto a vector of numbers, there are several approaches to do this. I have used the so-called Global Vector word embedding. See <a rel="nofollow" href="https://nlp.stanford.edu/projects/glove/" target="_blank">here</a> for details, it makes use of word co-occurrences that are determined from a (large) collection of documents and there is fast implementation in the R <a rel="nofollow" href="http://text2vec.org/index.html" target="_blank">text2vec</a> package.</p>
<p>Once words are transformed to vectors, you can calculate distances (similarities) between the words, for a specific word you can calculate the top 10 closest words for example. More over <strong>linguistic regularities</strong> can be determined, for example:</p>
<pre>amsterdam - netherlands + germany
</pre>
<p>would result in a vector that would be close to the vector for berlin.</p>
<h2>Results for The B&amp;B recaps</h2>
<p>It takes about an hour on my laptop to determine the word vectors (length 250) from 3645 B&amp;B recaps (15 seasons). After removing some common stop words, I have 10.293 unique words, text2vec puts the embeddings in a matrix (10.293 by 250).</p>
<p>Lets take the lovely <strong>steffy</strong>,</p>
<div><img src="https://i1.wp.com/media.licdn.com/mpr/mpr/AAIA_wDGAAAAAQAAAAAAAAucAAAAJDAzYzQwZWVhLTRiMzEtNGM3Yi1iOTU4LTg3MjQ5NWU4NGI3NQ.png?resize=200%2C213&ssl=1" data-recalc-dims="1"></div>
<p>the ten closest words are:</p>
<pre>    from     to     value
    </pre>