<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know"> Easy Guides</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p></p>
<p>  </p>
<div>
<p><strong>Text mining</strong> methods allow us to highlight the most frequently used keywords in a paragraph of texts. One can create a <strong>word cloud</strong>, also referred as <em>text cloud</em> or <em>tag cloud</em>, which is a visual representation of text data.</p>
<p>The procedure of creating word clouds is very simple in R if you know the different steps to execute. The text mining package (<em>tm</em>) and the word cloud generator package (<em>wordcloud</em>) are available in R for helping us to analyze texts and to quickly visualize the keywords as a word cloud.</p>
<p><span>In this article, we’ll describe, step by step, how to generate <strong>word clouds</strong> using the R software.</span></p>
<div>
<img alt="word cloud and text mining, I have a dream speech from Martin luther king" src="https://i2.wp.com/www.sthda.com/sthda/RDoc/images/word-cloud.png?w=456" data-recalc-dims="1">
<p>word cloud and text mining, I have a dream speech from Martin luther king</p>
</div>
<p></p>
<div>
<strong>Contents</strong>
<ul>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#reasons-you-should-use-word-clouds-to-present-your-text-data">3 reasons you should use word clouds to present your text data</a></li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#who-is-using-word-clouds">Who is using word clouds ?</a></li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#the-5-main-steps-to-create-word-clouds-in-r">The 5 main steps to create word clouds in R</a>
<ul>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#step-1-create-a-text-file">Step 1: Create a text file</a></li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#step-2-install-and-load-the-required-packages">Step 2 : Install and load the required packages</a></li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#step-3-text-mining">Step 3 : Text mining</a></li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#step-4-build-a-term-document-matrix">Step 4 : Build a term-document matrix</a></li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#step-5-generate-the-word-cloud">Step 5 : Generate the Word cloud</a></li>
</ul>
</li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#go-further">Go further</a>
<ul>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#explore-frequent-terms-and-their-associations">Explore frequent terms and their associations</a></li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#the-frequency-table-of-words">The frequency table of words</a></li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#plot-word-frequencies">Plot word frequencies</a></li>
</ul>
</li>
<li><a target="_blank" rel="nofollow" href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know#infos">Infos</a></li>
</ul>
</div>
<p></p>
<div>
<h2>3 reasons you should use word clouds to present your text data</h2>
<ol>
<li><strong>Word clouds</strong> add simplicity and clarity. The most used keywords stand out better in a word cloud
</li>
<li><strong>Word clouds</strong> are a potent communication tool. They are easy to understand, to be shared and are impactful</li>
<li><strong>Word clouds</strong> are visually engaging than a table data</li>
</ol>
</div>
<div>
<h2>Who is using word clouds ?</h2>
<ul>
<li>Researchers : for reporting qualitative data</li>
<li>Marketers : for highlighting the needs and pain points of customers</li>
<li>Educators : to support essential issues</li>
<li>Politicians and journalists</li>
<li>social media sites : to collect, analyze and share user sentiments</li>
</ul>
</div>
<div>
<h2>The 5 main steps to create word clouds in R</h2>
<div>
<h3>Step 1: Create a text file</h3>
<p>In the following examples, I’ll process the “<strong>I have a dream speech</strong>” from “<strong>Martin Luther King</strong>” but you can use any text you want :</p>
<ul>
<li>Copy and paste the text in a plain text file (e.g : ml.txt)</li>
<li>Save the file</li>
</ul>
<p><span>Note that, the text should be saved in a plain text (.txt) file format using your favorite text editor.</span></p>
</div>
<div>
<h3>Step 2 : Install and load the required packages</h3>
<p>Type the R code below, to install and load the required packages:</p>
<pre><code># Install
install.packages(&quot;tm&quot;)  # for text mining
install.packages(&quot;SnowballC&quot;) # for text stemming
install.packages(&quot;wordcloud&quot;) # word-cloud generator 
install.packages(&quot;RColorBrewer&quot;) # color palettes
# Load
library(&quot;tm&quot;)
library(&quot;SnowballC&quot;)
library(&quot;wordcloud&quot;)
library(&quot;RColorBrewer&quot;)</code></pre>
</div>
<div>
<h3>Step 3 : Text mining</h3>
<div>
<h4>load the text</h4>
<p>The text is loaded using <strong>Corpus()</strong> function from <strong>text mining</strong> ™ package. Corpus is a list of a document (in our case, we only have one document).</p>
<ol>
<li><strong>We start by importing the text file created in Step 1</strong></li>
</ol>
<p>To import the file saved locally in your computer, type the following R code. You will be asked to choose the text file interactively.</p>
<pre><code>text &lt;- readLines(file.choose())</code></pre>
<p>In the example below, I’ll load a .txt file hosted on STHDA website:</p>
<pre><code># Read the text file from internet
filePath &lt;- &quot;http://www.sthda.com/sthda/RDoc/example-files/martin-luther-king-i-have-a-dream-speech.txt&quot;
text &lt;- readLines(filePath)</code></pre>
<ol start="2">
<li><strong>Load the data as a corpus</strong></li>
</ol>
<pre><code># Load the data as a corpus
docs &lt;- Corpus(VectorSource(text))</code></pre>
<p><span>VectorSource() function creates a corpus of character vectors</span></p>
<ol start="3">
<li><strong>Inspect the content of the document</strong></li>
</ol>
<pre><code>inspect(docs)</code></pre>
</div>
<div>
<h4>Text transformation</h4>
<p>Transformation is performed using <strong>tm_map()</strong> function to replace, for example, special characters from the text.</p>
<p>Replacing “/”, “@” and “|” with space:</p>
<pre><code>toSpace &lt;- content_transformer(function (x , pattern ) gsub(pattern, &quot; &quot;, x))
docs &lt;- tm_map(docs, toSpace, &quot;/&quot;)
docs &lt;- tm_map(docs, toSpace, &quot;@&quot;)
docs &lt;- tm_map(docs, toSpace, &quot;\\|&quot;)</code></pre>
</div>
<div>
<h4>Cleaning the text</h4>
<p>the <strong>tm_map()</strong> function is used to remove unnecessary white space, to convert the text to lower case, to remove common stopwords like ‘the’, “we”.</p>
<p>The information value of ‘stopwords’ is near zero due to the fact that they are so common in a language. Removing this kind of words is useful before further analyses. For ‘stopwords’, supported languages are danish, dutch, english, finnish, french, german, hungarian, italian, norwegian, portuguese, russian, spanish and swedish. Language names are case sensitive.</p>
<p><span>I’ll also show you how to make your own list of stopwords to remove from the text.</span></p>
<p>You could also remove numbers and punctuation with <strong>removeNumbers</strong> and <strong>removePunctuation</strong> arguments.</p>
<p>Another important preprocessing step is to make a <strong>text stemming</strong> which reduces words to their root form. In other words, this process removes suffixes from words to make it simple and to get the common origin. For example, a stemming process reduces the words “moving”, “moved” and “movement” to the root word, “move”.</p>
<p><span>Note that, text stemming require the package ‘SnowballC’. </span></p>
<p>The R code below can be used to clean your text :</p>
<pre><code># Convert the text to lower case
docs &lt;- tm_map(docs, content_transformer(tolower))
# Remove numbers
docs &lt;- tm_map(docs, removeNumbers)
# Remove english common stopwords
docs &lt;- tm_map(docs, removeWords, stopwords(&quot;english&quot;))
# Remove your own stop word
# specify your stopwords as a character vector
docs &lt;- tm_map(docs, removeWords, c(&quot;blabla1&quot;, &quot;blabla2&quot;)) 
# Remove punctuations
docs &lt;- tm_map(docs, removePunctuation)
# Eliminate extra white spaces
docs &lt;- tm_map(docs, stripWhitespace)
# Text stemming
# docs &lt;- tm_map(docs, stemDocument)</code></pre>
</div>
</div>
<div>
<h3>Step 4 : Build a term-document matrix</h3>
<p>Document matrix is a table containing the frequency of the words. Column names are words and row names are documents. The function <em>TermDocumentMatrix()</em> from <strong>text mining</strong> package can be used as follow :</p>
<pre><code>dtm &lt;- TermDocumentMatrix(docs)
m &lt;- as.matrix(dtm)
v &lt;- sort(rowSums(m),decreasing=TRUE)
d &lt;- data.frame(word = names(v),freq=v)
head(d, 10)</code></pre>
<pre><code>             word freq
will         will   17
freedom   freedom   13
ring         ring   12
day           day   11
dream       dream   11
let           let   11
every       every    9
able         able    8
one           one    8
together together    7</code></pre>
</div>
<div>
<h3>Step 5 : Generate the Word cloud</h3>
<p>The importance of words can be illustrated as a <strong>word cloud</strong> as follow :</p>
<pre><code>set.seed(1234)
wordcloud(words = d$word, freq = d$freq, min.freq = 1,
          max.words=200, random.order=FALSE, rot.per=0.35, 
          colors=brewer.pal(8, &quot;Dark2&quot;))</code></pre>
<div>
<img alt="word cloud and text mining, I have a dream speech from Martin Luther King" src="https://i0.wp.com/www.sthda.com/sthda/RDoc/figure/text-mining/word-cloud-martin-luther-king-i-have-a-dream-speech.png?w=456" data-recalc-dims="1">
<p>
word cloud and text mining, I have a dream speech from Martin Luther King
</p>
</div>
<p>The above <strong>word cloud</strong> clearly shows that “Will”, “freedom”, “dream”, “day” and “together” are the five most important words in the “<strong>I have a dream speech</strong>” from <strong>Martin Luther King</strong>.</p>
<p>Arguments of the <strong>word cloud generator</strong> function :</p>
<p></p>
<div>
<ul>
<li>words : the words to be plotted</li>
<li>freq : their frequencies</li>
<li>min.freq : words with frequency below min.freq will not be plotted</li>
<li>max.words : maximum number of words to be plotted</li>
<li>random.order : plot words in random order. If false, they will be plotted in decreasing frequency</li>
<li>rot.per : proportion words with 90 degree rotation (vertical text)</li>
<li>colors : color words from least to most frequent. Use, for example, colors =“black” for single color.</li>
</ul>
</div>
<p></p>
</div>
</div>
<div>
<h2>Go further</h2>
<div>
<h3>Explore frequent terms and their associations</h3>
<p>You can have a look at the frequent terms in the term-document matrix as follow. In the example below we want to find words that occur at least four times :</p>
<pre><code>findFreqTerms(dtm, lowfreq = 4)</code></pre>
<pre><code> [1] &quot;able&quot;     &quot;day&quot;      &quot;dream&quot;    &quot;every&quot;    &quot;faith&quot;    &quot;free&quot;     &quot;freedom&quot;  &quot;let&quot;      &quot;mountain&quot; &quot;nation&quot;  
[11] &quot;one&quot;      &quot;ring&quot;     &quot;shall&quot;    &quot;together&quot; &quot;will&quot;    </code></pre>
<p>You can analyze the association between frequent terms (i.e., terms which correlate) using findAssocs() function. The R code below identifies which words are associated with “freedom” in <strong>I have a dream speech</strong> :</p>
<pre><code>findAssocs(dtm, terms = &quot;freedom&quot;, corlimit = 0.3)</code></pre>
<pre><code>$freedom
         let         ring  mississippi mountainside        stone        every     mountain        state 
        0.89         0.86         0.34         0.34         0.34         0.32         0.32         0.32 </code></pre>
</div>
<div>
<h3>The frequency table of words</h3>
<pre><code>head(d, 10)</code></pre>
<pre><code>             word freq
will         will   17
freedom   freedom   13
ring         ring   12
day           day   11
dream       dream   11
let           let   11
every       every    9
able         able    8
one           one    8
together together    7</code></pre>
</div>
<div>
<h3>Plot word frequencies</h3>
<p>The frequency of the first 10 frequent words are plotted :</p>
<pre><code>barplot(d[1:10,]$freq, las = 2, names.arg = d[1:10,]$word,
        col =&quot;lightblue&quot;, main =&quot;Most frequent words&quot;,
        ylab = &quot;Word frequencies&quot;)</code></pre>
<div>
<img alt="word cloud and text mining" src="https://i2.wp.com/www.sthda.com/sthda/RDoc/figure/text-mining/word-cloud-frequency-plot.png?w=384" data-recalc-dims="1">
<p>
word cloud and text mining
</p>
</div>
</div>
</div>
<div>
<h1>Infos</h1>
<p><span> This analysis has been performed using R (ver. 3.3.2). </span></p>
</div>
<p></p>

</div>
<p><br>
</p>
<p></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know"> Easy Guides</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/JVhU40E7HgI"></a></div>