<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/clustering/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://realdataweb.wordpress.com/2017/06/12/clustering/"> R – Real Data</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><img src="https://i0.wp.com/kids.nationalgeographic.com/content/dam/kids/photos/animals/Birds/A-G/emperor-penguin-colony.ngsversion.1412630284744.jpg"></p>
<p>Hello, everyone! I’ve been meaning to get a new blog post out for the<br>
past couple of weeks. During that time I’ve been messing around with<br>
clustering. Clustering, or cluster analysis, is a method of data mining<br>
that groups similar observations together. Classification and clustering<br>
are quite alike, but clustering is more concerned with exploration than<br>
an end result.</p>
<p>Note: This post is far from an exhaustive look at all clustering has to<br>
offer. Check out <a rel="nofollow" href="https://home.deib.polimi.it/matteucc/Clustering/tutorial_html/" target="_blank">this guide</a><br>
for more. I am reading <a rel="nofollow" href="http://www.springer.com/us/book/9783319141411" target="_blank">Data Mining</a> by Aggarwal presently, which is very informative.</p>
<pre>
data(&quot;iris&quot;)
head(iris)
</pre>
<pre><code>##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1          5.1         3.5          1.4         0.2  setosa
## 2          4.9         3.0          1.4         0.2  setosa
## 3          4.7         3.2          1.3         0.2  setosa
## 4          4.6         3.1          1.5         0.2  setosa
## 5          5.0         3.6          1.4         0.2  setosa
## 6          5.4         3.9          1.7         0.4  setosa
</code></pre>
<p>For simplicity, we’ll use the <code>iris</code> dataset. We’re going to try to use<br>
the numeric data to correctly group the observations by species. There<br>
are 50 of each species in the dataset, so ideally we would end up with<br>
three clusters of 50.</p>
<pre>
library(ggplot2)
ggplot() +
geom_point(aes(iris$Sepal.Length, iris$Sepal.Width, col = iris$Species))
</pre>
<p><img data-orig-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-2-1.png?w=456" data-orig-size="672,480" data-medium-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-2-1.png?w=456?w=300" data-attachment-id="622" data-image-meta="{" data-image-description="" src="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-2-1.png?w=456" data-permalink="https://realdataweb.wordpress.com/2017/06/12/clustering/unnamed-chunk-2-1-2/" data-large-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-2-1.png?w=456?w=656" alt="unnamed-chunk-2-1" data-image-title="unnamed-chunk-2-1" sizes="(max-width: 656px) 100vw, 656px" data-comments-opened="1" class="wp-image-622" data-recalc-dims="1"></p>
<p>As you can see, there is already some groupings present. Let’s use<br>
<a rel="nofollow" href="https://en.wikipedia.org/wiki/Hierarchical_clustering" target="_blank">hierarcical clustering</a> first.</p>
<pre>
iris2 &lt;- iris[,c(1:4)]  #not going to use the `Species` column.
medians &lt;- apply(iris2, 2, median)
mads &lt;- apply(iris2,2,mad)
iris3 &lt;- scale(iris2, center = medians, scale = mads)

dist &lt;- dist(iris3)
hclust &lt;- hclust(dist, method = 'median')  #there are several methods for hclust
cut &lt;- cutree(hclust, 3)

table(cut)
</pre>
<pre><code>## cut
##  1  2  3 
## 49 34 67
</code></pre>
<pre>
iris &lt;- cbind(iris, cut)
iris$cut &lt;- factor(iris$cut)
levels(iris$cut) &lt;- c('setosa', 'versicolor', 'virginica')
err &lt;- iris$Species == iris$cut
table(err)
</pre>
<pre><code>## err
## FALSE  TRUE 
##    38   112
</code></pre>
<pre>
ggplot() +
geom_point(aes(iris2$Sepal.Length, iris2$Sepal.Width, col = iris$cut))
</pre>
<p><img data-orig-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-3-11.png?w=456" data-orig-size="672,480" data-medium-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-3-11.png?w=456?w=300" data-attachment-id="693" data-image-meta="{" data-image-description="" src="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-3-11.png?w=456" data-permalink="https://realdataweb.wordpress.com/2017/06/12/clustering/unnamed-chunk-3-1/" data-large-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-3-11.png?w=456?w=656" alt="unnamed-chunk-3-1" data-image-title="unnamed-chunk-3-1" sizes="(max-width: 656px) 100vw, 656px" data-comments-opened="1" class="wp-image-693" data-recalc-dims="1"></p>
<p>Nice groupings here, but it looks like the algorithm has some trouble<br>
determining between versicolor and virginica. If we used this<br>
information to classify the observations, we’d get an error rate of<br>
about .25. Let’s try another clustering technique: DBSCAN.</p>
<pre>
library(dbscan)
</pre>
<pre>
db &lt;- dbscan(iris3, eps = 1, minPts = 20)
table(db$cluster)
</pre>
<pre><code>## 
##  0  1  2 
##  5 48 97
</code></pre>
<pre>
iris2 &lt;- cbind(iris2, db$cluster)

iris2$cluster &lt;- factor(iris2$`db$cluster`)

ggplot() +
geom_point(aes(iris2$Sepal.Length, iris2$Sepal.Width, col = iris2$cluster))
</pre>
<p><img data-orig-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-5-11.png?w=456" data-orig-size="672,480" data-medium-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-5-11.png?w=456?w=300" data-attachment-id="706" data-image-meta="{" data-image-description="" src="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-5-11.png?w=456" data-permalink="https://realdataweb.wordpress.com/2017/06/12/clustering/unnamed-chunk-5-1/" data-large-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-5-11.png?w=456?w=656" alt="unnamed-chunk-5-1" data-image-title="unnamed-chunk-5-1" sizes="(max-width: 656px) 100vw, 656px" data-comments-opened="1" class="wp-image-706" data-recalc-dims="1"></p>
<p>DBSCAN classifies points into three different categories: core, border,<br>
and noise points on the basis of density. Thus, the versicolor/<br>
virginica cluster is treated as one group. Since our data is not<br>
structured in such a way that density is meaningful, DBSCAN is probably<br>
not a wise choice here.</p>
<p>Let’s look at one last algo: the ROCK. No, not <em>that</em> ROCK.</p>
<pre>
library(cba)
</pre>
<pre>
distm &lt;- as.matrix(dist)
rock &lt;- rockCluster(distm, 3, theta = .02)
</pre>
<pre><code>## Clustering:
## computing distances ...
## computing links ...
## computing clusters ...
</code></pre>
<pre>
iris$rock &lt;- rock$cl
levels(iris$rock) &lt;- c('setosa', 'versicolor', 'virginica')

ggplot() +
geom_point(aes(iris2$Sepal.Length, iris2$Sepal.Width, col = rock$cl))
</pre>
<p><img data-orig-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-7-1.png?w=456" data-orig-size="672,480" data-medium-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-7-1.png?w=456?w=300" data-attachment-id="707" data-image-meta="{" data-image-description="" src="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-7-1.png?w=456" data-permalink="https://realdataweb.wordpress.com/2017/06/12/clustering/unnamed-chunk-7-1/" data-large-file="https://realdataweb.files.wordpress.com/2017/06/unnamed-chunk-7-1.png?w=456?w=656" alt="unnamed-chunk-7-1" data-image-title="unnamed-chunk-7-1" sizes="(max-width: 656px) 100vw, 656px" data-comments-opened="1" class="wp-image-707" data-recalc-dims="1"></p>
<pre>
err &lt;- (iris$Species == iris$rock)
table(err)
</pre>
<pre><code>## err
## FALSE  TRUE 
##    24   126
</code></pre>
<p>While it may not look like it, the ROCK does the best job at determining<br>
clusters in this data – the error rate dropped to 16%. Typically this<br>
method is reserved for categorical data, but since we used <code>dist</code> it<br>
shouldn't cause any problems.</p>
<p>I have been working on a project using some of these (and similar) data<br>
mining procedures to explore spatial data and search for distinct<br>
groups. While clustering the <code>iris</code> data may not be all that meaningful,<br>
I think it is illustrative of the power of clustering. I have yet to try<br>
higher-dimension clustering techniques, which might be even better at<br>
determining <code>Species</code>.</p>
<p>Have any comments? Questions? Suggestions for future posts? I am always<br>
happy to hear from readers; please contact me!</p>
<p>Happy clustering,</p>
<p>Kiefer</p>
<p> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/realdataweb.wordpress.com/617/" target="_blank"><img border="0" alt="" src="http://feeds.wordpress.com/1.0/comments/realdataweb.wordpress.com/617/"></a> <img border="0" alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" data-recalc-dims="1"></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://realdataweb.wordpress.com/2017/06/12/clustering/"> R – Real Data</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...

	<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/t5lWKso0C3U" height="1"></a></div>