<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/how-to-create-a-sequential-model-in-keras-for-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://blog.datascienceheroes.com/how-to-create-a-sequential-model-in-keras-for-r/"> R - Data Science Heroes Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div><img alt="How to create a sequential model in Keras for R" src="https://i2.wp.com/blog.datascienceheroes.com/content/images/2018/09/sequential_model_keras_tw_r-2.png?w=456&ssl=1" data-recalc-dims="1">
<p>tl;dr: This tutorial will introduce the Deep Learning classification task with Keras. We will particularly focus on the shape of the arrays, which is one of the most common pitfalls.</p>
<p>The topics we’ll cover are:</p>
<ul>
<li>How to do one-hot encoding</li>
<li>Choosing the input and output shape/dimensions in the layers</li>
<li>How to train the model</li>
<li>How to evaluate the model (training vs. test)</li>
</ul>
<p>You can find the complete code of <a rel="nofollow" href="https://github.com/pablo14/Keras-R-tutorials/blob/master/keras_sequential_model.R" target="_blank">this tutorial on Github</a> <img alt="👇" src="https://s.w.org/images/core/emoji/2.4/72x72/1f447.png" class="wp-smiley"></p>
<p><img alt="How to create a sequential model in Keras for R" src="https://i1.wp.com/blog.datascienceheroes.com/content/images/2018/09/keras_model.gif?w=456&ssl=1" data-recalc-dims="1"></p>
<h2>Creating the data</h2>
<p>The input data will be 10000 rows and three columns coming from the uniform distribution.</p>
<p>The model will recognize that the sum of the three numbers is above a threshold of <code>1.5</code>, otherwise it will be 0.</p>
<pre><code># Input: 10000 rows and 3 columns of uniform distribution
x_data=matrix(data=runif(30000), nrow=10000, ncol=3)

# Output
y_data=ifelse(rowSums(x_data) > 1.5, 1, 0)

head(x_data)
</code></pre>
<pre><code>##            [,1]      [,2]      [,3]
## [1,] 0.49224109 0.6096538 0.7855929
## [2,] 0.56710301 0.8707570 0.3428543
## [3,] 0.08246592 0.7504529 0.6979251
## [4,] 0.79197829 0.6330420 0.7307403
## [5,] 0.65170373 0.9052915 0.1395298
## [6,] 0.23479406 0.2611791 0.2780037
</code></pre>
<pre><code>head(y_data)
</code></pre>
<pre><code>## [1] 1 1 1 1 1 0
</code></pre>
<h2>Installing / Loading Keras</h2>
<pre><code># install.packages(&quot;keras&quot;)
library(keras)
library(tidyverse)
</code></pre>
<h3>One-hot encoding in Keras</h3>
<p>One of the key points in Deep Learning is to understand the dimensions of the vector, matrices and/or arrays that the model needs. I found that these are the types supported by Keras.</p>
<p>In Python’s words, it is the shape of the array.</p>
<p>To do a binary classification task, we are going to create a one-hot vector. It works the same way for more than 2 classes.</p>
<p>For instance:</p>
<ul>
<li>The value <code>1</code> will be the vector <code>[0,1]</code></li>
<li>The value <code>0</code> will be the vector <code>[1,0]</code></li>
</ul>
<p>Keras provides the <code>to_categorical</code> function to achieve this goal.</p>
<pre><code>y_data_oneh=to_categorical(y_data, num_classes = 2)

head(y_data_oneh)
</code></pre>
<pre><code>##      [,1] [,2]
## [1,]    0    1
## [2,]    0    1
## [3,]    0    1
## [4,]    0    1
## [5,]    0    1
## [6,]    1    0
</code></pre>
<p>It’s easy to get categorical variables like: &quot;yes/no&quot;, &quot;CatA,CatB,CatC&quot;, etc. But <code>to_categorical</code> doesn’t accept non-numeric values as input. We need to convert them first.</p>
<p><code>num_classes</code> is necessary to create a vector length.</p>
<p>Alternatives to <code>to_categorical</code>:</p>
<ul>
<li>Package <code>CatEncoders</code>, <a rel="nofollow" href="https://cran.r-project.org/web/packages/CatEncoders/CatEncoders.pdf" target="_blank">OneHotEncoder</a> (same as Python <code>scikit-learn</code>).</li>
<li>Package <code>caret</code>, function <a rel="nofollow" href="https://amunategui.github.io/dummyVar-Walkthrough/" target="_blank">dummyVars</a>.</li>
</ul>
<p>Note: We don’t need to convert the <strong>input variables</strong> since they are numerical.</p>
<h2>Creating a sequential model in Keras</h2>
<p>The simplest model in Keras is the <strong>sequential</strong>, which is built by stacking layers sequentially.</p>
<p>In the next example, we are stacking three dense layers, and keras builds an implicit input layer with your data, using the input_shape parameter. So in total we’ll have an input layer and the output layer.</p>
<pre><code>model = keras_model_sequential() %>%   
  layer_dense(units = 64, activation = &quot;relu&quot;, input_shape = ncol(x_data)) %>%
  layer_dense(units = 64, activation = &quot;relu&quot;) %>%
  layer_dense(units = ncol(y_data_oneh), activation = &quot;softmax&quot;)
</code></pre>
<p>The most important parameters by now are:</p>
<ul>
<li>
<p>In the first layer the <code>input_shape</code> represents a vector with the value 3 (<code>ncol(x_data)</code>) indicating the number of input variables. In deep learning almost everything is vectors (or tensors).</p>
</li>
<li>
<p>The second layer doesn’t have an <code>input_shape</code> since Keras infers it from the previous layer.</p>
</li>
<li>
<p>The third <code>layer_dense</code>, which represents the final output, has 2 (<code>ncol(y_data_oneh)</code>) units representing the two possible outcomes.</p>
</li>
</ul>
<p>You can check the model and the shapes per layer:</p>
<pre><code>model
</code></pre>
<pre><code>## Model
## ___________________________________________________________________________
## Layer (type)                     Output Shape                  Param #     
## ===========================================================================
## dense_28 (Dense)                 (None, 64)                    256         
## ___________________________________________________________________________
## dense_29 (Dense)                 (None, 64)                    4160        
## ___________________________________________________________________________
## dense_30 (Dense)                 (None, 2)                     130         
## ===========================================================================
## Total params: 4,546
## Trainable params: 4,546
## Non-trainable params: 0
## ___________________________________________________________________________
</code></pre>
<p>Now it’s time to define the loss and optimizer functions, and the metric to optimize.</p>
<p>Although it says &quot;accuracy&quot;, keras recognizes the nature of the output (classification), and uses the <code>categorical_accuracy</code> on the backend.</p>
<pre><code>compile(model, loss = &quot;categorical_crossentropy&quot;, optimizer = optimizer_rmsprop(), metrics = &quot;accuracy&quot;)
</code></pre>
<p>Let’s fit (train) the model:</p>
<pre><code>history = fit(model,  x_data, y_data_oneh, epochs = 20, batch_size = 128, validation_split = 0.2)
</code></pre>
<p><img alt="How to create a sequential model in Keras for R" src="https://i2.wp.com/blog.datascienceheroes.com/content/images/2018/09/training_keras.gif?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Done!</p>
<p>Plotting results:</p>
<pre><code>plot(history)
</code></pre>
<p><img alt="How to create a sequential model in Keras for R" src="https://i2.wp.com/blog.datascienceheroes.com/content/images/2018/09/unnamed-chunk-8-1.png?w=456&ssl=1" data-recalc-dims="1"></p>
<h2>Validating with unseen data</h2>
<p>Creating ‘unseen’ input test data (1000 rows, 3 columns):</p>
<pre><code>x_data_test=matrix(data=runif(3000), nrow=1000, ncol=3)
dim(x_data_test) 
</code></pre>
<pre><code>## [1] 1000    3
</code></pre>
<p>Predicting new cases:</p>
<pre><code>y_data_pred=predict_classes(model, x_data_test)

glimpse(y_data_pred)
</code></pre>
<pre><code>##  num [1:1000(1d)] 0 0 1 1 1 1 0 1 0 1 ...
</code></pre>
<p>Please note that the dimension is 1000 rows and 2 columns. <code>predict_classes</code> automatically does the one-hot decoding.</p>
<p>On the contrary, <code>predict</code> returns the same dimension that was received when training (n-rows, n-classes to predict).</p>
<pre><code>y_data_pred_oneh=predict(model, x_data_test)

dim(y_data_pred_oneh)
</code></pre>
<pre><code>## [1] 1000    2
</code></pre>
<pre><code>head(y_data_pred_oneh)
</code></pre>
<pre><code>##                     [,1]              [,2]
## [1,] 0.99207872152328491 0.007921278476715
## [2,] 1.00000000000000000 0.000000001157425
## [3,] 0.00000002739444227 1.000000000000000
## [4,] 0.41273152828216553 0.587268412113190
## [5,] 0.03470309823751450 0.965296924114227
## [6,] 0.00000000007147296 1.000000000000000
</code></pre>
<p>In classification, it is always recommended to return the probabilities for each class, just like we did with <code>predict</code> (the row sum is 1). More info at: <a rel="nofollow" href="https://livebook.datascienceheroes.com/model-performance.html#scoring_data" target="_blank">DSLB – Scoring Data</a>.</p>
<h3>The softmax activation</h3>
<p>We got the probabilities thanks to the <code>activation = &quot;softmax&quot;</code> in the last layer.</p>
<p>The output of softmax ranges from 0 to 1 for each class, and the sum of all the classes is, naturally, 1.</p>
<h2>Evaluating the model (Training vs. Test)</h2>
<p>Creating the &quot;real&quot; y target to compare against the predicted:</p>
<pre><code>y_data_real=ifelse(rowSums(x_data_test) > 1.5, 1, 0)
y_data_real_oneh=to_categorical(y_data_real)
</code></pre>
<pre><code>## Training data
evaluate(model, x_data, y_data_oneh, verbose = 0)
</code></pre>
<pre><code>## $loss
## [1] 0.03008356
## 
## $acc
## [1] 0.9918
</code></pre>
<pre><code>## Test data (we need the one-hot version)
evaluate(model, x_data_test, y_data_real_oneh, verbose = 0)
</code></pre>
<pre><code>## $loss
## [1] 0.03146484
## 
## $acc
## [1] 0.991
</code></pre>
<p>The accuracy with the unseen data is almost perfect: Around <code>1</code> (and a loss near to 0, which is more important than the accuracy).</p>
<p>The neural network learned the pattern we defined at the beginning <img alt="🎉" src="https://s.w.org/images/core/emoji/2.4/72x72/1f389.png" class="wp-smiley">!</p>
<p><img alt="How to create a sequential model in Keras for R" src="https://i0.wp.com/blog.datascienceheroes.com/content/images/2018/09/yoda_dancing.gif?w=456&ssl=1" data-recalc-dims="1"></p>
<h2>Some questions <img alt="🤔" src="https://s.w.org/images/core/emoji/2.4/72x72/1f914.png" class="wp-smiley"></h2>
<p><em>How many layers are needed?</em> We tried with 3, but how about 4? It will learn the same, but in more time. But in other scenarios the performance is increased. It depends on each case.</p>
<p><em>How many neurons per layer are needed?</em> The same reasoning as before. We should expect that maximum accuracy with the simplest network. One approach that tends to do a good standard fit is to start with a layer with (i.e.) 128 neurons. Then add a layer with 64 and the other with 16. The pattern is to reduce the number of neurons per layer.</p>
<h2>Try-it yourself <img alt="🙌" src="https://s.w.org/images/core/emoji/2.4/72x72/1f64c.png" class="wp-smiley"></h2>
<p>Reading is just one tiny part of learning, try it yourself by, for example, changing the function we used (we tried the <code>y > 1.5 = 1</code>) and put the network to learn it.</p>
<p>Learn by coding and expose your practice to errors <img alt="🙂" src="https://s.w.org/images/core/emoji/2.4/72x72/1f642.png" class="wp-smiley"></p>
<h2>Further reading</h2>
<ul>
<li><a rel="nofollow" href="https://keras.rstudio.com/articles/sequential_model.html" target="_blank">Guide to the Sequential Models</a> (from Rstudio)</li>
<li><a rel="nofollow" href="https://blog.rstudio.com/2018/09/12/getting-started-with-deep-learning-in-r" target="_blank">Getting started with deep learning in R</a> (from Rstudio)</li>
<li><a rel="nofollow" href="https://blog.datascienceheroes.com/sample-size-and-class-balance-on-model-performance" target="_blank">Sample size and class balance on model performance</a> (the underlying model was built with Keras)</li>
</ul>
<hr>
<p><em>Any questions? Leave them below</em> <img alt="📩" src="https://s.w.org/images/core/emoji/2.4/72x72/1f4e9.png" class="wp-smiley"></p>
<p><strong>Thanks</strong> for reading! <img alt="🚀" src="https://s.w.org/images/core/emoji/2.4/72x72/1f680.png" class="wp-smiley"></p>
<p><img alt="📗" src="https://s.w.org/images/core/emoji/2.4/72x72/1f4d7.png" class="wp-smiley"> <a rel="nofollow" href="http://livebook.datascienceheroes.com/" target="_blank">Data Science Live Book</a> – An online book to learn machine learning</p>
<p><a rel="nofollow" href="https://blog.datascienceheroes.com/how-to-create-a-sequential-model-in-keras-for-r/blog.datascienceheroes.com" target="_blank">Blog</a> | <a rel="nofollow" href="https://www.linkedin.com/in/pcasas/" target="_blank">Linkedin</a> | <a rel="nofollow" href="https://twitter.com/pabloc_ds" target="_blank">Twitter</a></p>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://blog.datascienceheroes.com/how-to-create-a-sequential-model-in-keras-for-r/"> R - Data Science Heroes Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/-kDpBOId8pA" height="1"></a></div>