<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/new-chapters-for-50-shades-of-grey/"></div></div><div><a data-url="http://www.r-bloggers.com/new-chapters-for-50-shades-of-grey/" data-counturl="http://www.r-bloggers.com/new-chapters-for-50-shades-of-grey/" data-count="horizontal" data-text="New chapters for 50 shades of grey…." class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://longhowlam.wordpress.com/2016/07/27/new-chapters-for-50-shades-of-grey/"> R – Longhow Lam's Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2><img alt="WP" class="wp-image-1842" src="http://i0.wp.com/longhowlam.files.wordpress.com/2016/07/wp.png?w=456&ssl=1" data-recalc-dims="1"></h2>
<h2>Introduction</h2>
<p>Some time ago I had the honor to follow an interesting talk from <a target="_blank" href="https://www.linkedin.com/in/tijmen-blankevoort-a5633a24">Tijmen Blankevoort</a> on neural networks and deeplearning. Convolutional and recurrent neural networks were topics that already caught my interest and this talk inspired me to dive into these topics deeper and do some more experiments with it.</p>
<p>In the same session organized by <a target="_blank" href="https://www.linkedin.com/in/martindelusenet">Martin de Lusenet</a> for Ziggo (a Dutch cable company) I also had the honor to give a talk, my presentation contained <a target="_blank" href="https://longhowlam.wordpress.com/2015/08/08/soap-analytics-text-mining-goede-tijden-slechte-tijden-plot-summaries/">a text mining experiment </a>that I did earlier on the Dutch TV soap GTST <em>“Goede Tijden Slechte Tijden”</em>. A nice idea by Tijmen was: Why not use deep learning to generate new episode plots for GTST?</p>
<p>So I did that, see my <a target="_blank" href="https://www.linkedin.com/pulse/script-schrijvers-van-goede-tijden-slechte-uw-baan-staat-longhow-lam?trk=pulse_spock-articles">LinkedIn post on GTST</a>. However, these episodes are in Dutch and I guess only interesting for people here in the Netherlands. So to make things <strong>more international</strong> and more <strong>spicier</strong> I generated <em>some new texts</em> based on deep learning and the erotic romance novel <em><strong>50 shades of grey</strong></em><img height="16" alt=":-)" width="16" class="wp-smiley" src="https://s1.wp.com/wp-content/mu-plugins/wpcom-smileys/simple-smile.svg" draggable="false"></p>
<h2>More than plain vanilla networks</h2>
<p>In R or SAS you could already train plain vanilla neural networks for a long time. The so-called fully connected networks where all input nodes are connected to all nodes in the following hidden layer.And all nodes in a hidden layer are connected to all nodes in the following hidden layer or output layer.</p>
<p><img alt="neural" class="wp-image-1922" src="http://i2.wp.com/longhowlam.files.wordpress.com/2016/07/neural.jpg?resize=406%2C202&ssl=1" data-recalc-dims="1"></p>
<p>In more recent years deep learning frame works have become very popular. For example <a target="_blank" href="http://caffe.berkeleyvision.org/">Caffe</a>, <a target="_blank" href="http://torch.ch/">Torch</a>, <a target="_blank" href="https://github.com/Microsoft/CNTK">CTNK</a>, <a target="_blank" href="https://www.tensorflow.org/">Tensorflow </a>and <a target="_blank" href="https://mxnet.readthedocs.io/en/latest/">MXNET</a>. The additional value of these frame works compared to SAS for example are:</p>
<ul>
<li>They support more network types than plain vanilla networks. For example, convolutional networks, where not all input nodes are connected to a next layer. And recurrent networks, where loops are present. A nice introduction to these networks can be found <a target="_blank" href="http://colah.github.io/posts/2015-09-NN-Types-FP/">here </a>and <a target="_blank" href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">here</a>.</li>
<li>They support computations on GPU’s, which could speed up things dramatically.</li>
<li>They are open-source and free. No need for long sales and implementation cycles<img height="16" alt=":-)" width="16" class="wp-smiley" src="https://s1.wp.com/wp-content/mu-plugins/wpcom-smileys/simple-smile.svg" draggable="false"> Just download it and use it!</li>
</ul>
<div data-shortcode="caption" class="wp-caption"><img alt="RNN" class="wp-image-1926" src="http://i2.wp.com/longhowlam.files.wordpress.com/2016/07/rnn.png?resize=456%2C134&ssl=1" data-recalc-dims="1">
<p class="wp-caption-text">recurrent neural network</p>
</div>
<h2>My 50 Shades of Grey experiment</h2>
<p>For my experiment I used the text of the erotic romance novel <strong><em>50 shades of grey</em></strong>. A pdf can be found <a target="_blank" href="https://jenseyatvajameh.files.wordpress.com/2008/07/50shades.pdf">here</a>, I used <strong>xpdfbin</strong> to extract all the words into a plain text file. I trained a Long Short Term Memory network (LSTM, a special type of recurrent networks), with MXNET. The reason to use MXNET is that they have a nice R interface, so that I can just stay in my comfortable RStudio environment.</p>
<p>Moreover, the R example script of MXNET is ready to run, I just changed the input data and used more rounds of training and more hidden layers. The script and the data can be found on <a target="_blank" href="https://github.com/longhowlam/GTST">Github</a>.</p>
<p>The LSTM model is fit on character level, the complete romance novel contains 817,204 characters, all these characters are mapped to a number (91 unique numbers). The first few numbers are shown in the following figure.</p>
<p><img alt="numbers" class="wp-image-1949" src="http://i2.wp.com/longhowlam.files.wordpress.com/2016/07/numbers.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Once the model has been trained it can generate new text, character by character!</p>
<pre><em>arsess whatever</em>
<em>yuu’re still expeliar a sally. </em><em>Reftion while break in a limot.”</em>
<em>“Yes, ald what’s at my artmer and brow maned, but I’m so then for a</em>
<em>dinches suppretion. If you think vining. “Anastasia, and depregineon posing rave.</em>
<em>He’d deharing minuld, him drits.</em>

<em>“Miss Steele</em>
<em>“Fasting at liptfel, Miss I’ve dacind her leaches reme,” he knimes.</em>
<em>“I want to blight on to the wriptions of my great. I find sU she asks the stroke, to read with what’s old both – in our fills into his ear, surge • whirl happy, this is subconisue. Mrs. I can say about the battractive see. I slues</em>
<em>is her ever returns. </em><em>“Anab.</em>

<em>It’s too even ullnes. </em><em>“By heaven. Grey</em>
<em>about his voice. “Rest of the meriction.”</em>
<em>He scrompts to the possible. I shuke my too sucking four finishessaures. I need to fush quint the only more eat at me.</em>
<em>“Oh my. Kate. He’s follower socks?</em>
<em>“Lice in Quietly. In so morcieut wait to obsed teach beside my tired steately liked trying that.”</em>
<em>Kate for new of its street of confcinged. I haven’t Can regree.</em>
<em>“Where.” I fluscs </em><em>up hwindwer-and I have
</em>
<em>I’ll staring for conisure, pain!”</em>
<em>I know he’s just doesn’t walk to my backeting on Kate has hotelby of confidered Christaal side, supproately. Elliot, but it’s the ESca, that feel posing, it make my just drinking my eyes bigror on my head. S I’ll tratter topality butterch,” I mud</em>
<em>a nevignes, bleamn. </em><em>“It’s not by there soup. He’s washing, and I arms and have. I wave to make my eyes. It’s forgately? Dash I’d desire to come your drink my heathman legt</em>
<em>you hay D1 Eyep, Christian Gry, husder with a truite sippking, I coold behind, it didn’t want to mive not to my stop?”</em>
<em>“Yes.”
</em>
<em>“Sire, stcaring it was do and he licks his viice ever.”</em>
<em>I murmurs, </em><em>most stare thut’s the then staraline for neced outsive. She</em>
<em>so know what differ at,” he murmurs?</em>
<em>“I shake my headanold.” Jeez.</em>
<em>“Are you?” Eviulder keep “Oh,_ I frosing gylaced in – angred. I am most drink to start and try aparts through. I really thrial you, dly woff you stund, there, I care an right dains to rainer.” He likes his eye finally finally my eyes to over opper heaven, places my trars his Necked her jups.</em>
<em>“Do you think your or Christian find at me, is so with that stand at my mouth sait the laxes any litee, this is a memory rude. It</em>
<em>flush,” He says usteer?” </em><em>“Are so that front up.</em>
<em>I preparraps. I don’t scomine Kneat for from Christian.</em>
<em>“Christian,’! he leads the acnook. I can’t see. I breathing Kate’ve bill more over keen by. He releases?”</em>
<em>“I’m kisses take other in to peekies my tipgents my</em></pre>
<h2>Conclusion</h2>
<p>The generated text <strong>does not make any sense</strong>, nor will it win any literature prize soon<img height="16" alt=":-)" width="16" class="wp-smiley" src="https://s1.wp.com/wp-content/mu-plugins/wpcom-smileys/simple-smile.svg" draggable="false"> Keep in mind, that the model is based ‘only’ on 817,204 characters  (which is considered a small number), and I did not bother to fine-tune the model at all. But still it is <strong>funny and remarkable</strong> to see that when you use it to generate text, character by character, it can still produce a lot of correct English words and even some correct basic grammar patterns!</p>
<p>cheers, Longhow.</p>
<p> </p>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/longhowlam.wordpress.com/1829/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/longhowlam.wordpress.com/1829/" border="0"></a> <img alt="" src="http://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/new-chapters-for-50-shades-of-grey/"></div></div><div><a data-url="http://www.r-bloggers.com/new-chapters-for-50-shades-of-grey/" data-counturl="http://www.r-bloggers.com/new-chapters-for-50-shades-of-grey/" data-count="vertical" data-text="New chapters for 50 shades of grey…." class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://longhowlam.wordpress.com/2016/07/27/new-chapters-for-50-shades-of-grey/"> R – Longhow Lam's Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/n3XA31YOUvo"></a></div>