<span><span lang="en"><span>We will develop a classification exercise using Naive-Bayes algorithm. The exercise was originally published in <i>&quot;Machine Learning in R&quot; </i>by Brett Lantz, PACKT publishing 2015 (open source community experience destilled).</span></span></span><br><br><span><span lang="en"><span>Naive Bayes is a probabilistic classification algorithm that can be applied to problems of text classification such as spam filtering, intrusion detection or network anomalies, diagnosis of medical conditions given a set of symptoms, among others.</span></span></span><br><br><span><span lang="en"><span>The exercise we will develop is about filtering spam and ham sms messages</span></span><span lang="en"><span>.</span></span></span><br><br><span><span lang="en"><span><span lang="en"><span><span><span lang="en"><span>We will carry out the exercise verbatim as published in the aforementioned reference.</span></span></span></span></span></span></span></span><br><br><span><span lang="en"><span>To develop the Naive Bayes classifier, we will use data adapted from the SMS Spam Collection at http://www.dt.fee.unicamp.br/~tiago/smsspamcollection/ .</span></span></span><br><span><span lang="en"><span><br></span></span></span><span><span lang="en"><span>### install required packages<br><br>install.packages(&quot;tm&quot;)<br>install.packages(&quot;NLP&quot;)<br>install.packages(&quot;SnowballC&quot;)<br>install.packages(&quot;wordcloud&quot;)<br>install.packages(&quot;e1071&quot;)<br>install.packages(&quot;RColorBrewer&quot;)<br>install.packages(&quot;gmodels&quot;)<br><br>library(tm)<br>library(NLP)<br>library(SnowballC)<br>library(RColorBrewer)<br>library(wordcloud)<br>library(e1071)<br>library(gmodels)<br><br>### read the data<br>sms_raw &lt;- read.csv(&quot;sms_spam.csv&quot;, stringsAsFactors = FALSE)<br><br>### transform type into factor (ham/spam)<br>sms_raw$type &lt;- factor(sms_raw$type)<br><br>### see frequencies<br>table(sms_raw$type)<br><br>### create volatile (stored in memory) corpus <br>### Vcorpus create a complex list, we can use list manipulation<br>### commands to manage it<br>sms_corpus &lt;- VCorpus(VectorSource(sms_raw$text))<br><br>### inspect the two first elements <br>inspect(sms_corpus[1:2])<br><br>### see the first sms (as text)<br>as.character(sms_corpus[[1]])<br><br>### to view multiple documents<br>lapply(sms_corpus[1:2], as.character)<br><br>### cleaning the text in documents (corpus)<br><br>### tm_map to map all over the corpus<br>### content_transformer to access the corpus<br>### tolower to lowercase all strings<br>sms_corpus_clean &lt;- tm_map(sms_corpus,<br>                           content_transformer(tolower))<br>### check and compare the result of the cleaning<br>as.character(sms_corpus[[1]])<br>as.character(sms_corpus_clean[[1]])<br><br>### remove numbers<br>sms_corpus_clean &lt;- tm_map(sms_corpus_clean, removeNumbers)<br><br>### remove &quot;stop words&quot;<br>### see the &quot;stop words&quot; list<br>stopwords()<br><br>sms_corpus_clean &lt;- tm_map(sms_corpus_clean,<br>                           removeWords, stopwords())<br>### remove punctuation<br>sms_corpus_clean &lt;- tm_map(sms_corpus_clean, removePunctuation)<br><br>### stemming (transform words into it's root form)<br>sms_corpus_clean &lt;- tm_map(sms_corpus_clean, stemDocument)<br><br>### romove additional whitespaces<br>sms_corpus_clean &lt;- tm_map(sms_corpus_clean, stripWhitespace)<br><br>### check and compare result of cleaning</span></span></span><br><span><span lang="en"><span>as.character(sms_corpus[[1]])<br>as.character(sms_corpus_clean[[1]])<br><br>### tokenization<br>### document term matrix (DTM)<br>### rows are sms and columns are words<br>sms_dtm &lt;- DocumentTermMatrix(sms_corpus_clean)<br><br>### data preparation (training and test sets)<br>### 75% train 25% test<br>sms_dtm_train &lt;- sms_dtm[1:4169, ]<br>sms_dtm_test &lt;- sms_dtm[4170:5559, ]<br><br>### labels for train and test sets<br>### feature to be classified<br>sms_train_labels &lt;- sms_raw[1:4169, ]$type<br>sms_test_labels &lt;- sms_raw[4170:5559, ]$type<br><br>### confirm that the subsets are representative of the <br>### complete set of SMS data<br>prop.table(table(sms_train_labels))<br>prop.table(table(sms_test_labels))<br><br>### wordcloud<br>wordcloud(sms_corpus_clean, min.freq = 50, random.order = FALSE)</span></span></span><br><span><span lang="en"><span><br></span></span></span><div><a href="https://4.bp.blogspot.com/-MXNi69WgdS4/WQNZt1w75NI/AAAAAAAAC00/EPhxPMhLCyY6EFPVJSfyiAbN2EfpYOoOACLcB/s1600/naive_cloud.png"></a></div><span><span lang="en"><span></span></span></span><br><span><span lang="en"><span>### separated wordclouds</span></span></span><br><span><span lang="en"><span>spam &lt;- subset(sms_raw, type == &quot;spam&quot;)<br>ham &lt;- subset(sms_raw, type == &quot;ham&quot;)<br><br>wordcloud(spam$text, max.words = 40, scale = c(3, 0.5))<br>wordcloud(ham$text, max.words = 40, scale = c(3, 0.5))<br><br>### creating indicator features for frequent words<br>findFreqTerms(sms_dtm_train, 5)<br>sms_freq_words &lt;- findFreqTerms(sms_dtm_train, 5)<br>str(sms_freq_words)<br><br>### filter DTM by frequent terms<br>sms_dtm_freq_train &lt;- sms_dtm_train[ , sms_freq_words]<br>sms_dtm_freq_test &lt;- sms_dtm_test[ , sms_freq_words]<br><br>### change DTM frequency to factor (categorical)<br>convert_counts &lt;- function(x) {<br>  x &lt;- ifelse(x > 0, &quot;Yes&quot;, &quot;No&quot;)<br>}<br><br>sms_train &lt;- apply(sms_dtm_freq_train, MARGIN = 2,<br>                   convert_counts)<br>sms_test &lt;- apply(sms_dtm_freq_test, MARGIN = 2,<br>                  convert_counts)<br><br>### training a model on the data<br>### alternative package for Naives-Bayes (&quot;klaR&quot;)<br>sms_classifier &lt;- naiveBayes(sms_train, sms_train_labels)<br><br>### evaluating model performance<br>### make predictions on test set<br>sms_test_pred &lt;- predict(sms_classifier, sms_test)<br><br>### compare predictions (classifications) with the<br>### true values<br>CrossTable(sms_test_pred, sms_test_labels,<br>           prop.chisq = FALSE, prop.t = FALSE,<br>           dnn = c('predicted', 'actual'))</span></span></span><br><div><span><span lang="en"><span> </span></span></span><a href="https://2.bp.blogspot.com/-sxR9L8de7XY/WQNYxlnQHII/AAAAAAAAC0o/NS21Qv9jrqQ2JzKC6i-y6C7FkMghRSb-ACLcB/s1600/naive_1.png"></a></div><span><span lang="en"><span></span></span></span><br><span><span lang="en"><span></span></span></span><br><span><span lang="en"><span>### improving model performance?<br>### set Laplace estimator to 1<br>sms_classifier2 &lt;- naiveBayes(sms_train, sms_train_labels,<br>                              laplace = 1)<br><br>sms_test_pred2 &lt;- predict(sms_classifier2, sms_test)<br><br>CrossTable(sms_test_pred2, sms_test_labels,<br>           prop.chisq = FALSE, prop.t = FALSE, prop.r = FALSE,<br>           dnn = c('predicted', 'actual'))</span></span></span><br><br><div><a href="https://2.bp.blogspot.com/-ErvtFfmmddE/WQNY9u0x0bI/AAAAAAAAC0s/Kzu8PAB541MqjbyUUQlIzu9fPhTz3RUFACLcB/s1600/naive_2.png"></a></div><span><span lang="en"><span></span></span></span><span lang="en"><span><span></span></span></span><span><span lang="en"><span>You can get the exercise and the data set in:</span></span></span><br><span><span><span lang="en"><span><a target="_blank" href="https://github.com/pakinja/Data-R-Value/tree/master/MachineLearning_NaiveBayes">https://github.com/pakinja/Data-R-Value</a> </span></span></span></span>