<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/machine-learning-classification-using-naive-bayes/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://feedproxy.google.com/~r/DataRValue/~3/bmOMvDq73Ns/machine-learning-classification-naive-bayes.html"> Data R Value</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><span><span lang="en"><span>We will develop a classification exercise using Naive-Bayes algorithm. The exercise was originally published in <i>“Machine Learning in R” </i>by Brett Lantz, PACKT publishing 2015 (open source community experience destilled).</span></span></span></p>
<p><span><span lang="en"><span>Naive Bayes is a probabilistic classification algorithm that can be applied to problems of text classification such as spam filtering, intrusion detection or network anomalies, diagnosis of medical conditions given a set of symptoms, among others.</span></span></span></p>
<p><span><span lang="en"><span>The exercise we will develop is about filtering spam and ham sms messages</span></span><span lang="en"><span>.</span></span></span></p>
<p><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>We will carry out the exercise verbatim as published in the aforementioned reference.</span></span></span></span></span></span></span></span></p>
<p><span><span lang="en"><span>To develop the Naive Bayes classifier, we will use data adapted from the SMS Spam Collection at http://www.dt.fee.unicamp.br/~tiago/smsspamcollection/ .</span></span></span><br><span><span lang="en"><span><br></span></span></span><span><span lang="en"><span><b>### install required packages</b>
<p>install.packages(“tm”)<br>install.packages(“NLP”)<br>install.packages(“SnowballC”)<br>install.packages(“wordcloud”)<br>install.packages(“e1071”)<br>install.packages(“RColorBrewer”)<br>install.packages(“gmodels”)</p>
<p>library(tm)<br>library(NLP)<br>library(SnowballC)<br>library(RColorBrewer)<br>library(wordcloud)<br>library(e1071)<br>library(gmodels)</p>
<p><b>### read the data</b><br>sms_raw &lt;- read.csv(“sms_spam.csv”, stringsAsFactors = FALSE)</p>
<p><b>### transform type into factor (ham/spam)</b><br>sms_raw$type &lt;- factor(sms_raw$type)</p>
<p><b>### see frequencies</b><br>table(sms_raw$type)</p>
<p><b>### create volatile (stored in memory) corpus <br>### Vcorpus create a complex list, we can use list manipulation<br>### commands to manage it</b><br>sms_corpus &lt;- VCorpus(VectorSource(sms_raw$text))</p>
<p><b>### inspect the two first elements</b> <br>inspect(sms_corpus[1:2])</p>
<p><b>### see the first sms (as text)</b><br>as.character(sms_corpus[[1]])</p>
<p><b>### to view multiple documents</b><br>lapply(sms_corpus[1:2], as.character)</p>
<p><b>### cleaning the text in documents (corpus)
<p>### tm_map to map all over the corpus<br>### content_transformer to access the corpus<br>### tolower to lowercase all strings<br>sms_corpus_clean &lt;- tm_map(sms_corpus,<br>                           content_transformer(tolower))<br><b>### check and compare the result of the cleaning</b><br>as.character(sms_corpus[[1]])<br>as.character(sms_corpus_clean[[1]])</p></b>
<p><b>### remove numbers</b><br>sms_corpus_clean &lt;- tm_map(sms_corpus_clean, removeNumbers)</p>
<p><b>### remove “stop words”<br>### see the “stop words” list</b><br>stopwords()</p>
<p>sms_corpus_clean &lt;- tm_map(sms_corpus_clean,<br>                           removeWords, stopwords())<br><b>### remove punctuation</b><br>sms_corpus_clean &lt;- tm_map(sms_corpus_clean, removePunctuation)</p>
<p><b>### stemming (transform words into it’s root form)</b><br>sms_corpus_clean &lt;- tm_map(sms_corpus_clean, stemDocument)</p>
<p><b>### romove additional whitespaces</b><br>sms_corpus_clean &lt;- tm_map(sms_corpus_clean, stripWhitespace)</p>
<p><b>### check and compare result of cleaning</b><br><span><span lang="en"><span>as.character(sms_corpus[[1]])<br>as.character(sms_corpus_clean[[1]])
<p><b>### tokenization<br>### document term matrix (DTM)<br>### rows are sms and columns are words</b><br>sms_dtm &lt;- DocumentTermMatrix(sms_corpus_clean)</p>
<p><b>### data preparation (training and test sets)<br>### 75% train 25% test</b><br>sms_dtm_train &lt;- sms_dtm[1:4169, ]<br>sms_dtm_test &lt;- sms_dtm[4170:5559, ]</p>
<p><b>### labels for train and test sets<br>### feature to be classified</b><br>sms_train_labels &lt;- sms_raw[1:4169, ]$type<br>sms_test_labels &lt;- sms_raw[4170:5559, ]$type</p>
<p><b>### confirm that the subsets are representative of the <br>### complete set of SMS data</b><br>prop.table(table(sms_train_labels))<br>prop.table(table(sms_test_labels))</p>
<p><b>### wordcloud</b><br>wordcloud(sms_corpus_clean, min.freq = 50, random.order = FALSE)<br><span><span lang="en"><span><br></span></span></span>
<div><a target="_blank" rel="nofollow" href="https://i0.wp.com/4.bp.blogspot.com/-MXNi69WgdS4/WQNZt1w75NI/AAAAAAAAC00/EPhxPMhLCyY6EFPVJSfyiAbN2EfpYOoOACLcB/s1600/naive_cloud.png?ssl=1"><img src="https://i2.wp.com/4.bp.blogspot.com/-MXNi69WgdS4/WQNZt1w75NI/AAAAAAAAC00/EPhxPMhLCyY6EFPVJSfyiAbN2EfpYOoOACLcB/s400/naive_cloud.png?resize=400%2C353&ssl=1" border="0" data-recalc-dims="1"></a></div></span>
<p><span><span lang="en"><span></span></span></span><br><span><span lang="en"><span><b>### separated wordclouds</b></span></span></span><br><span><span lang="en"><span>spam &lt;- subset(sms_raw, type == “spam”)<br>ham &lt;- subset(sms_raw, type == “ham”)
<p>wordcloud(spam$text, max.words = 40, scale = c(3, 0.5))<br>wordcloud(ham$text, max.words = 40, scale = c(3, 0.5))</p>
<p><b>### creating indicator features for frequent words</b><br>findFreqTerms(sms_dtm_train, 5)<br>sms_freq_words &lt;- findFreqTerms(sms_dtm_train, 5)<br>str(sms_freq_words)</p>
<p><b>### filter DTM by frequent terms</b><br>sms_dtm_freq_train &lt;- sms_dtm_train[ , sms_freq_words]<br>sms_dtm_freq_test &lt;- sms_dtm_test[ , sms_freq_words]</p>
<p><b>### change DTM frequency to factor (categorical)</b><br>convert_counts &lt;- function(x) {<br>  x &lt;- ifelse(x > 0, “Yes”, “No”)<br>}</p>
<p>sms_train &lt;- apply(sms_dtm_freq_train, MARGIN = 2,<br>                   convert_counts)<br>sms_test &lt;- apply(sms_dtm_freq_test, MARGIN = 2,<br>                  convert_counts)</p>
<p><b>### training a model on the data<br>### alternative package for Naives-Bayes (“klaR”)</b><br>sms_classifier &lt;- naiveBayes(sms_train, sms_train_labels)</p>
<p><b>### evaluating model performance<br>### make predictions on test set</b><br>sms_test_pred &lt;- predict(sms_classifier, sms_test)</p>
<p><b>### compare predictions (classifications) with the<br>### true values</b><br>CrossTable(sms_test_pred, sms_test_labels,<br>           prop.chisq = FALSE, prop.t = FALSE,<br>           dnn = c(‘predicted’, ‘actual’))
<div><span><span lang="en"><span> </span></span></span><a target="_blank" rel="nofollow" href="https://i1.wp.com/2.bp.blogspot.com/-sxR9L8de7XY/WQNYxlnQHII/AAAAAAAAC0o/NS21Qv9jrqQ2JzKC6i-y6C7FkMghRSb-ACLcB/s1600/naive_1.png?ssl=1"><img src="https://i1.wp.com/2.bp.blogspot.com/-sxR9L8de7XY/WQNYxlnQHII/AAAAAAAAC0o/NS21Qv9jrqQ2JzKC6i-y6C7FkMghRSb-ACLcB/s400/naive_1.png?resize=360%2C400&ssl=1" border="0" data-recalc-dims="1"></a></div></span>
<p><span><span lang="en"><span></span></span></span><br><span><span lang="en"><span></span></span></span><br><span><span lang="en"><span><b>### improving model performance?<br>### set Laplace estimator to 1</b><br>sms_classifier2 &lt;- naiveBayes(sms_train, sms_train_labels,<br>                              laplace = 1)
<p>sms_test_pred2 &lt;- predict(sms_classifier2, sms_test)</p>
<p>CrossTable(sms_test_pred2, sms_test_labels,<br>           prop.chisq = FALSE, prop.t = FALSE, prop.r = FALSE,<br>           dnn = c(‘predicted’, ‘actual’))</p></span>
<div><a target="_blank" rel="nofollow" href="https://i0.wp.com/2.bp.blogspot.com/-ErvtFfmmddE/WQNY9u0x0bI/AAAAAAAAC0s/Kzu8PAB541MqjbyUUQlIzu9fPhTz3RUFACLcB/s1600/naive_2.png?ssl=1"><img src="https://i0.wp.com/2.bp.blogspot.com/-ErvtFfmmddE/WQNY9u0x0bI/AAAAAAAAC0s/Kzu8PAB541MqjbyUUQlIzu9fPhTz3RUFACLcB/s400/naive_2.png?resize=390%2C400&ssl=1" border="0" data-recalc-dims="1"></a></div>
<p><span><span lang="en"><span></span></span></span><span lang="en"><span><span></span></span></span><span><span lang="en"><span>You can get the exercise and the data set in:</span></span></span><br><span><span><span lang="en"><span><a target="_blank" rel="nofollow" href="https://github.com/pakinja/Data-R-Value/tree/master/MachineLearning_NaiveBayes">https://github.com/pakinja/Data-R-Value</a> </span></span></span></span><img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/DataRValue/~4/bmOMvDq73Ns"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://feedproxy.google.com/~r/DataRValue/~3/bmOMvDq73Ns/machine-learning-classification-naive-bayes.html"> Data R Value</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/qwCwBsxr4CI"></a></div></span></span></span></span></span></span></span></span></span>