<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/text-message-classification/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://datascienceplus.com/text-message-classification/"> R Programming – DataScience+</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Classification is a supervised machine learning technique in which the dataset which we are analyzing has some inputs \(X_i\) and a response variable \(Y\) which is a <b>discrete</b> valued variable.Discrete valued means the variable has a finite set of values.In more specific terms in classification the response variable has some categorical values.In R we call such values as <b>factor</b> variables. For example-\(Y \in\) (Male,Female) or (0,1) or (High,low,medium) etc are the best examples of the response variable \(Y\) in a classification problem.</p>
<p>Now our task in a typical classification problem is to compute the mapping function(also called hypotheses) \(f : X–>Y\) which maps and relates inputs to target variable which it learns from the training data that we feed the learner.Now given some inputs the learning algorithm will learn from the data and will compute the <b>probabilities</b> of occurrence of each class label \(K\) where \(Y \in K\).</p>
<p>Almost all the classifiers use a <b>conditional probability model</b> \(C(X)\) to compute the probabilities of the class labels.<br>
$$C(X) = P(Y \in K | X) = \frac{P(X \cap Y)}{P(X)} $$</p>
<p>Definitely, there are a few exceptions which directly don’t use a conditional probability model(e.g-SVM) to classify data but in general, all classifiers use the conditional probability model.</p>
<h3>Text Classification</h3>
<p>Now in this article I am going to classify text messages as either <b>Spam</b> or <b>Ham</b>.As the dataset will have text messages which are unstructured in nature so we will require some basic natural language processing to compute word frequencies, tokenizing texts, and calculating document-feature matrix etc.</p>
<p>The dataset is taken from Kaggle’s SMS Spam Collection <a rel="nofollow" href="https://www.kaggle.com/uciml/sms-spam-collection-dataset" target="_blank">Spam Dataset</a>.</p>
<h3>Implementation in R</h3>
<p>We will use a very nice package called <code>quanteda</code> which is used for managing, processing and analyzing text data.I urge the readers to go and read the documentation for the package and how it works.</p>
<p>Requiring the necessary packages–</p>
<pre>
require(quanteda)#natural language processing package
?quanteda 
require(RColorBrewer)
require(ggplot2)
</pre>
<p>Let’s load the dataset.</p>
<pre>
spam=read.csv(&quot;../Datasets/spam.csv&quot;,header=TRUE, sep=&quot;,&quot;, quote='\&quot;\&quot;', stringsAsFactors=FALSE)

table(spam$v1)
<em>## ## ham spam ## 4825 747
</em>
</pre>
<p>As we can observe there are more number of ham messages than spam.<br>
let’s plot a barplot-<br>
<a rel="nofollow" href="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/09/download.png?ssl=1" target="_blank"><img sizes="(max-width: 1344px) 100vw, 1344px" src="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/09/download.png?w=450&ssl=1" alt="" class="wp-image-8241" data-recalc-dims="1"></a></p>
<p>Now let’s add appropriate names to the columns.</p>
<pre>
names(spam)&lt;-c(&quot;type&quot;,&quot;message&quot;)
head(spam)
<em>## type
## 1 ham
## 2 ham
## 3 spam
## 4 ham
## 5 ham
## 6 spam
## message
## 1 Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...
## 2 Ok lar... Joking wif u oni...
## 3 Free entry in 2 a wkly comp to win FA Cup final tkts 21st May 2005. Text FA to 87121 to receive entry question(std txt rate)T&amp;C's apply 08452810075over18's
## 4 U dun say so early hor... U c already then say...
## 5 Nah I don't think he goes to usf, he lives around here though
## 6 FreeMsg Hey there darling it's been 3 week's now and no word back! I'd like some fun you up for it still? Tb ok! XxX std chgs to send, å£1.50 to rcv
## NA NA NA
## 1 ## 2 ## 3 ## 4 ## 5 ## 6
</em>
</pre>
<p>Next thing we need to do is a random sampling of data i.e shuffling it.We can randomize our data using the <code>sample()</code> command.If the data is not stored in a random distribution, this will help to ensure that we are dealing with a random draw from our data. The <code>set.seed()</code> is to ensure reproducible results.</p>
<pre>
set.seed(2012)
spam&lt;-spam[sample(nrow(spam)),] #randomly shuffling the dataset
</pre>
<h3>Building the Wordclouds</h3>
<p>Firstly, I will use quanteda’s <code>corpus()</code> command to construct a corpus from the text field of our raw data.A corpus can be thought of as a master copy of our dataset from which we can pull subsets or observations as needed.</p>
<p>After this I will attach the label field as a document variable to the corpus using the <code>docvars()</code> command. We attach label as a variable directly to our corpus so that we can associate SMS messages with their respective ham/spam label later in the analysis.</p>
<pre>
?corpus #to search more on this method
msg.corpus&lt;-corpus(spam$message)

docvars(msg.corpus)&lt;-spam$type   #attaching the class labels to the corpus message text


</pre>
<p>Let’s plot the word-cloud now-<br>
First, we will subset and filter all the spam text messages from the message corpus.Then we will generate a <b>document feature matrix</b> which is a sparse matrix consisting of the frequency of words that occur in a document.The rows represent the document and column represent the words/terms of the sentence which show which documents contain which terms and how many times they appear.</p>
<pre>
#subsetting only the spam messages
spam.plot&lt;-corpus_subset(msg.corpus,docvar1==&quot;spam&quot;)

#now creating a document-feature matrix using dfm()
spam.plot&lt;-dfm(spam.plot, tolower = TRUE, remove_punct = TRUE, remove_twitter = TRUE, remove_numbers = TRUE, remove=stopwords(&quot;SMART&quot;))

spam.col &lt;- brewer.pal(10, &quot;BrBG&quot;)  

textplot_wordcloud(spam.plot, min.freq = 16, color = spam.col)  
title(&quot;Spam Wordcloud&quot;, col.main = &quot;grey14&quot;)
</pre>
<p>Word cloud for Spam messages–</p>
<p><a rel="nofollow" href="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/09/download-1.png?ssl=1" target="_blank"><img sizes="(max-width: 1536px) 100vw, 1536px" src="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/09/download-1.png?w=450&ssl=1" alt="" class="wp-image-8243" data-recalc-dims="1"></a></p>
<p>The above plot is a <b>wordcloud</b> which is an amazing way of visualizing and understanding textual data and visually represent the contents in sentences.What is does is it picks and selects the most commonly occurring words in the sentences i.e the words having the highest frequencies and plots them, the more the frequency of a particular word the greater is the size of the word in the word-cloud. </p>
<h3>Generating the Ham wordcloud</h3>
<p>We will use the same procedure like we used in generating the spam wordcloud.</p>
<pre>
ham.plot&lt;-corpus_subset(msg.corpus,docvar1==&quot;ham&quot;)
ham.plot&lt;-dfm(ham.plot,tolower = TRUE, remove_punct = TRUE, remove_twitter = TRUE, remove_numbers = TRUE,remove=c(&quot;gt&quot;, &quot;lt&quot;, stopwords(&quot;SMART&quot;)))
ham.col=brewer.pal(10, &quot;BrBG&quot;)  
textplot_wordcloud(ham.plot,min.freq=50,colors=ham.col,fixed.asp=TRUE)
title(&quot;Ham Wordcloud&quot;,col.main = &quot;grey14&quot;)
</pre>
<p><a rel="nofollow" href="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/09/download-2.png?ssl=1" target="_blank"><img sizes="(max-width: 1536px) 100vw, 1536px" src="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/09/download-2.png?w=450&ssl=1" alt="" class="wp-image-8247" data-recalc-dims="1"></a></p>
<p>The above-plotted word-clouds are an amazing tool for knowing what are the most frequently occurring words that appear in Spam and Ham messages.</p>
<h2>Predictive Modelling</h2>
<p>Now let’s train a Naive Bayes text classifier to compute the probabilities of a message being \((spam,ham)\) i.e \( P(Y=Spam | message) \) and \( P(Y=ham | message) \).<br>
Naive Bayes classifiers are a class of simple linear classifiers which use conditional probability models based on <b>Bayes Theoram</b> i.e</p>
<p> $$C(X)=P(Y=K_j | X_i)=P(X_1|Y).P(X_2|Y)…P(X_i|Y)P(Y=K_j)$$<br>
Where \( X_i \) are the number of inputs and \(Y\) is a categorical response variable and \(K_j \) are the number of class labels.</p>
<p>The special thing about Naive Bayes classifiers is that they follow <b>Conditional Independence Theoram</b> i.e they assume that the features \(X_i\) are uncorrelated and independent of each other which is often a crude and impractical assumption,but still they are nice and simple classifiers which perform well most of the times too, because we are only concerned with the probability values.Secondly, they assume that the data samples are drawn from an identical and independent distribution- IID is the term which is famous in Statistics.</p>
<p>Let’s separate the training and test data-</p>
<pre>
#separating Train and test data
spam.train&lt;-spam[1:4458,]
spam.test&lt;-spam[4458:nrow(spam),]

msg.dfm &lt;- dfm(msg.corpus, tolower = TRUE)  #generating document freq matrix
msg.dfm &lt;- dfm_trim(msg.dfm, min_count = 5, min_docfreq = 3)  
msg.dfm &lt;- dfm_weight(msg.dfm, type = &quot;tfidf&quot;) 

head(msg.dfm)

#trining and testing data of dfm 
msg.dfm.train&lt;-msg.dfm[1:4458,]

msg.dfm.test&lt;-msg.dfm[4458:nrow(spam),]

<em> > #head(msg.dfm)
Document-feature matrix of: 5,572 documents, 1,932 features (99.3% sparse).
(showing first 6 documents and first 6 features)
features
docs nothing much , at home .
text1 2.166228 1.712587 0.644264 1.170823 1.539185 0.1999622
text2 0.000000 0.000000 0.000000 0.000000 0.000000 0.1999622
text3 0.000000 0.000000 0.000000 0.000000 1.539185 0.1999622
text4 0.000000 0.000000 0.000000 0.000000 0.000000 0.0000000
text5 0.000000 0.000000 0.000000 0.000000 0.000000 1.7996602
text6 0.000000 0.000000 0.644264 0.000000 0.000000 0.1999622
</em>
</pre>
<p>Please read the documentation of the above functions used for processing the document-feature-matrix to know more about the functions and their major use.</p>
<p>Training the Naive Bayes classifier-</p>
<pre>
nb.classifier&lt;-textmodel_NB(msg.dfm.train,spam.train[,1])
nb.classifier
<em>## Fitted Naive Bayes model:
## Call:
## textmodel_NB.dfm(x = msg.dfm.train, y = spam.train[, 1])
## ## ## Training classes and priors:
## spam ham ## 0.5 0.5 ## ## Likelihoods: Class Posteriors:
## 30 x 4 Matrix of class &quot;dgeMatrix&quot;
## spam ham spam ham
## you 5.001507e-03 0.0096798156 0.34067144 0.6593286
## have 4.322289e-03 0.0042303673 0.50537386 0.4946261
## 1 2.695748e-03 0.0009529526 0.73882413 0.2611759
## new 3.492485e-03 0.0010753934 0.76457487 0.2354251
## . 6.965338e-03 0.0168302131 0.29271598 0.7072840
## please 2.339097e-03 0.0011593603 0.66860811 0.3313919
## call 1.058603e-02 0.0021859571 0.82884759 0.1711524
## i 8.439760e-04 0.0112106647 0.07001254 0.9299875
## wait 1.860817e-04 0.0011538316 0.13887596 0.8611240
## for 5.699340e-03 0.0045025239 0.55865674 0.4413433
## hope 2.334040e-04 0.0017258550 0.11912872 0.8808713
## tonight 1.137075e-04 0.0011106417 0.09287182 0.9071282
## too 3.802754e-05 0.0017024748 0.02184860 0.9781514
## bad 1.232420e-04 0.0006045270 0.16934219 0.8306578
## as 1.339518e-03 0.0020699791 0.39287852 0.6071215
## well 2.938089e-04 0.0017334850 0.14492664 0.8550734
## but 2.528948e-04 0.0043933716 0.05442968 0.9455703
## rock 3.802754e-05 0.0002684845 0.12406542 0.8759346
## night 3.003905e-04 0.0017976398 0.14317739 0.8568226
## anyway 3.802754e-05 0.0005405216 0.06572915 0.9342709
## going 1.538819e-04 0.0023951976 0.06036762 0.9396324
## a 7.856726e-03 0.0064918622 0.54756091 0.4524391
## now 6.254232e-03 0.0028758075 0.68501697 0.3149830
## good 6.723203e-04 0.0030342352 0.18138681 0.8186132
## speak 8.003838e-04 0.0004728416 0.62862694 0.3713731
## to 1.113210e-02 0.0075761991 0.59503541 0.4049646
## soon 2.642059e-04 0.0010608467 0.19939274 0.8006073
## today 1.120666e-03 0.0019041688 0.37048833 0.6295117
## is 4.451802e-03 0.0058153446 0.43359683 0.5664032
## accept 3.802754e-05 0.0003188419 0.10655871 0.8934413
</em>
</pre>
<p>The model outputs the Probabilities of the message being Spam or ham.</p>
<h3>Let’s Test the Model</h3>
<pre>
pred&lt;-predict(nb.classifier,msg.dfm.test)

#generating a confusion matrix

# use pred$nb.predicted to extract the class labels
table(predicted=pred$nb.predicted,actual=spam.test[,1])
<em>## actual
## predicted ham spam
## ham 952 7
## spam 16 140
</em>
</pre>
<p>16 text examples wrongly classified for ham and 7 examples wrongly classified for spam.<br>
In the Confusion matrix,the <b>diagonals</b> are the correctly classified examples and the <b>off-diagonals</b> are the incorrectly classified text instances.</p>
<p>Now let’s calculate the accuracy of the model –</p>
<pre>
#acccuracy of the classifier on Test data
mean(pred$nb.predicted==spam.test[,1])*100
<em>## [1] 97.93722
</em>
</pre>
<p>Now 98% accuracy is a good amount of accuracy on unseen random test data.</p>
<h3>Conclusion</h3>
<p>This was a simple article on classifying text messages as ham or spam using some basic natural language processing and then building a naive Bayes text classifier.I urge the readers to implement and use the knowledge acquired from this article in making their own text classifiers and solving different problems related to text processing and NLP etc.<br>
Ofcourse, there are various other packages to do text processing and building such models.</p>
<p>Hope you guys liked the article, make sure to like and share it.</p>
<ul></ul>
<p><strong>
<p>Related Post</p>
<p>
<ol>
<li><a rel="nofollow" href="https://datascienceplus.com/analyzing-google-trends-data-in-r/" title="Permanent Link toAnalyzing Google Trends Data in R" target="_blank">Analyzing Google Trends Data in R</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/gradient-boosting-in-r/" title="Permanent Link toGradient boosting in R" target="_blank">Gradient boosting in R</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/radial-kernel-support-vector-classifier/" title="Permanent Link toRadial kernel Support Vector Classifier" target="_blank">Radial kernel Support Vector Classifier</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/random-forests-in-r/" title="Permanent Link toRandom Forests in R" target="_blank">Random Forests in R</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/network-analysis-of-game-of-thrones/" title="Permanent Link toNetwork analysis of Game of Thrones" target="_blank">Network analysis of Game of Thrones</a></li>
</ol></strong>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://datascienceplus.com/text-message-classification/"> R Programming – DataScience+</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/uu2vlvxW0Cs" height="1"></a></div>