<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/qualitative-research-in-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://datascienceplus.com/qualitative-research-in-r/"> R Programming – DataScience+</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>In the last two posts, I’ve focused purely on statistical topics – one-way ANOVA and dealing with multicollinearity in R. In this post, I’ll deviate from the pure statistical topics and will try to highlight some aspects of qualitative research. More specifically, I’ll show you the procedure of analyzing text mining and visualizing the text analysis using word cloud.<br>
Some of typical usage of the text mining are mentioned below:<br>
• Marketing managers often use the text mining approach to study the needs and complaints of their customers;<br>
• Politicians and journalists also use effectively the text mining to critically analyze the lectures delivered by the opposition leaders;<br>
• Social media experts uses this technique to collect, analyze and share user posts, comments etc.;<br>
• Social science researchers use text mining approach for analysing the qualitative data.</p>
<h2>What is Text mining?</h2>
<p>It is method which enables us to highlight the most frequently used keywords in a paragraph of texts or compilation of several text documents.</p>
<h2>What is Word Cloud?</h2>
<p>It is the visual representation of text data, especially the keywords in the text documents.<br>
R has very simple and straightforward approaches for text mining and creating word clouds. </p>
<p>The text mining package “(tm)” will be used for mining the text and the word cloud generator package (wordcloud) will be used for visualizing the keywords as a word cloud.</p>
<h2>Preparing the Text Documents:</h2>
<p>As the starting point of qualitative research, you need to create the text file. Here I’ve used the lecture delivered by great Indian Hindu monk Swami Vivekananda at the first World’s Parliament of Religions held from 11 to 27 September 1893. Only two lecture notes – opening and closing address, will be used.<br>
Both the lectures are saved in text file (<a rel="nofollow" href="https://datascienceplus.com/wp-content/uploads/2017/10/chicago.txt" target="_blank">chicago</a>).</p>
<h2>Loading the Required Packages:</h2>
<pre>
library(&quot;tm&quot;)
library(&quot;SnowballC&quot;)
library(&quot;wordcloud&quot;)
library(&quot;RColorBrewer&quot;)
</pre>
<h2>Load the text:</h2>
<h3>Importing the text file:</h3>
<p> The text file (<a rel="nofollow" href="https://datascienceplus.com/wp-content/uploads/2017/10/chicago.txt" target="_blank">chicago</a>) is imported using the following code in R.</p>
<p>The R code for leading the text is given below:</p>
<pre>
text &lt;- readLines(file.choose())
</pre>
<h3>Build the data as a corpus:</h3>
<p>The ‘text’ object will now be loaded as ‘Corpora’ which are collections of documents containing (natural language) text. The Corpus() function from text mining(tm) package will be used for this purpose.<br>
The R code for building the corpus is given below:</p>
<pre>
docs &lt;- Corpus(VectorSource(text))
</pre>
<p>Next use the function inspect() under the tm package to display detailed information of the text document.<br>
The R code for inspecting the text is given below:</p>
<pre>
inspect(docs)
<em>The output is not, however, produced here due to space constraint
</em>
</pre>
<h2>Text transformation:</h2>
<p>After inspecting the text document (corpora), it is required to perform some text transformation for replacing special characters from the text. To do this, use the ‘tm_map()’ function.<br>
The R code for transformation of the text is given below:</p>
<pre>
toSpace &lt;- content_transformer(function (x , pattern ) gsub(pattern, &quot; &quot;, x))
docs &lt;- tm_map(docs, toSpace, &quot;/&quot;)
docs &lt;- tm_map(docs, toSpace, &quot;@&quot;)
docs &lt;- tm_map(docs, toSpace, &quot;\\|&quot;)
</pre>
<h2>Text Cleaning:</h2>
<p>After removing the special characters from the text, it is now the time to remove the to remove unnecessary white space, to convert the text to lower case, to remove common stopwords like ‘the’, “we”. This is required as the The information value of ‘stopwords’ is near zero due to the fact that they are so common in a language. For doing this exercise, the same ‘tm_map()’ function will be used.<br>
The R code for cleaning the text along with the short self-explanation is given below:</p>
<pre>
# Convert the text to lower case
docs &lt;- tm_map(docs, content_transformer(tolower))
# Remove numbers
docs &lt;- tm_map(docs, removeNumbers)
# Remove english common stopwords
docs &lt;- tm_map(docs, removeWords, stopwords(&quot;english&quot;))
# Remove your own stop word
# specify your stopwords as a character vector
docs &lt;- tm_map(docs, removeWords, c(&quot;I&quot;, &quot;my&quot;)) 
# Remove punctuations
docs &lt;- tm_map(docs, removePunctuation)
# Eliminate extra white spaces
docs &lt;- tm_map(docs, stripWhitespace)
</pre>
<h2>Building a document matrix:</h2>
<p>Document matrix is the frequency distribution of the words used in the given text. I hope that readers will easily understand this frequency distribution of words.<br>
The R function TermDocumentMatrix() from the text mining package ‘tm’ will be used for building this frequency table for words in the given text.<br>
The R code is given below:</p>
<pre>
docs_matrix &lt;- TermDocumentMatrix(docs)
m &lt;- as.matrix(docs_matrix)
v &lt;- sort(rowSums(m),decreasing=TRUE)
d &lt;- data.frame(word = names(v),freq=v)
</pre>
<h3> Have a look at the document matrix for the top ten keywords:</h3>
<pre>
head(d, 10)
<em> word freq
religions religions 7
world world 6
earth earth 6
become become 6
hindu hindu 5
religion religion 5
thanks thanks 5
different different 4
men men 4
proud proud 4
</em>
</pre>
<h2>Generate the Word cloud:</h2>
<p>Finally, the frequency table of the words (document matrix) will be visualized graphically by plotting in a word cloud with the help of the following R code. </p>
<pre>
wordcloud(words = d$word, freq = d$freq, min.freq = 1,
          max.words=200, random.order=FALSE, rot.per=0.35, 
          colors=brewer.pal(8, &quot;Dark2&quot;))
</pre>
<p><a rel="nofollow" href="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/10/plot.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2017/10/plot-490x279.png?w=450&ssl=1" alt="" class="wp-image-9120" data-recalc-dims="1"></a></p>
<p>You can also use barplot to plot the frequencies of the keywords using the following R code:</p>
<pre>
barplot(d[1:10,]$freq, las = 2, names.arg = d[1:10,]$word,
        col =&quot;lightblue&quot;, main =&quot;Most commonly used words&quot;,
        ylab = &quot;Word frequencies&quot;, xlab=&quot;Keywords&quot;)
</pre>
<p><a rel="nofollow" href="https://i1.wp.com/datascienceplus.com/wp-content/uploads/2017/10/plot-1.png?ssl=1" target="_blank"><img sizes="(max-width: 490px) 100vw, 490px" src="https://i2.wp.com/datascienceplus.com/wp-content/uploads/2017/10/plot-1-490x280.png?w=450&ssl=1" alt="" class="wp-image-9121" data-recalc-dims="1"></a></p>
<h2>Conclusion:</h2>
<p>The above word cloud clearly shows that “religions”, “earth”, “world”, “hindu”, “one” etc. are the most important words in the lecture delivered by Swamiji in Chicago World’s Parliament of Religions.</p>
<ul></ul>
<p><strong>
<p>Related Post</p>
<p>
<ol>
<li><a rel="nofollow" href="https://datascienceplus.com/multi-dimensional-reduction-and-visualisation-with-t-sne/" title="Permanent Link toMulti-Dimensional Reduction and Visualisation with t-SNE" target="_blank">Multi-Dimensional Reduction and Visualisation with t-SNE</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/comparing-trump-and-clintons-facebook-pages-during-the-us-presidential-election-2016/" title="Permanent Link toComparing Trump and Clinton’s Facebook pages during the US presidential election, 2016" target="_blank">Comparing Trump and Clinton’s Facebook pages during the US presidential election, 2016</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/analyzing-obesity-across-usa/" title="Permanent Link toAnalyzing Obesity across USA" target="_blank">Analyzing Obesity across USA</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/can-we-predict-flu-deaths-with-machine-learning-and-r/" title="Permanent Link toCan we predict flu deaths with Machine Learning and R?" target="_blank">Can we predict flu deaths with Machine Learning and R?</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/graphical-presentation-of-missing-data-vim-package/" title="Permanent Link toGraphical Presentation of Missing Data; VIM Package" target="_blank">Graphical Presentation of Missing Data; VIM Package</a></li>
</ol></strong>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://datascienceplus.com/qualitative-research-in-r/"> R Programming – DataScience+</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/qJiyMZRaT3w" height="1"></a></div>