<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/intuition-for-principal-component-analysis-pca/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.ephorie.de/intuition-for-principal-component-analysis-pca"> R-Bloggers – Learning Machines</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div data-backend-url="http://blog.ephorie.de/wp-content/plugins/shariff-sharing/backend/index.php" data-temp="/tmp" data-orientation="horizontal" data-lang="en" data-theme="color" data-service="gftlr" data-services="[&quot;googleplus&quot;,&quot;facebook&quot;,&quot;twitter&quot;,&quot;linkedin&quot;,&quot;reddit&quot;,&quot;whatsapp&quot;,&quot;mail&quot;,&quot;info&quot;]" data-title="Intuition for principal component analysis (PCA)" data-info-url="http://ct.de/-2467514" data-ttl="60" data-url="http://blog.ephorie.de/intuition-for-principal-component-analysis-pca" data-image="http://blog.ephorie.de/wp-content/uploads/2018/12/teapot-in-3D-1024x731.pnghttp://blog.ephorie.de/wp-content/uploads/2018/12/teapot-in-3D-1024x731.png 1024w, http://blog.ephorie.de/wp-content/uploads/2018/12/teapot-in-3D-300x214.png 300w, http://blog.ephorie.de/wp-content/uploads/2018/12/teapot-in-3D-768x549.png 768w, http://blog.ephorie.de/wp-content/uploads/2018/12/teapot-in-3D-1200x857.png 1200w, http://blog.ephorie.de/wp-content/uploads/2018/12/teapot-in-3D.png 1344w"></div>
<p>Principal component analysis (PCA) is a dimension-reduction method that can be used to reduce a large set of (often correlated) variables into a smaller set of (uncorrelated) variables, called principal components, which still contain most of the information.</p>
<p>PCA is a concept that is traditionally hard to grasp so instead of giving you the n’th mathematical derivation I will provide you with some intuition.</p>
<p>Basically PCA is nothing else but a projection of some higher dimensional object into a lower dimension. What sounds complicated is really something we encounter every day: when we watch TV we see a 2D-projection of 3D-objects!</p>
<p>Have a look at this instructive <a rel="nofollow" href="https://youtu.be/BfTMmoDFXyE" target="_blank">youtube video</a>.</p>
<p>So what we want is not only a projection but a projection which retains as much information as possible. A good strategy is to find the longest axis of the object and after that turn the object around this axis so that we again find the second longest axis.</p>
<p>Mathematically this can be done by calculating the so called eigenvectors of the covariance matrix, after that we use just use the n eigenvectors with the biggest eigenvalues to project the object into n-dimensional space (in our example n = 2) – but as I said I won’t go into the mathematical details here (many good explanations can be found here on <a rel="nofollow" href="https://stats.stackexchange.com/questions/2691/making-sense-of-principal-component-analysis-eigenvectors-eigenvalues" target="_blank">stackexchange</a>).</p>
<p>To make matters concrete we will now recreate the example from the video above! The teapot data is from <a rel="nofollow" href="https://www.khanacademy.org/computer-programming/3d-teapot/971436783" target="_blank">https://www.khanacademy.org/computer-programming/3d-teapot/971436783</a> – there you can also turn it around interactively.</p>
<p>Have a look at the following code:</p>
<pre>
library(scatterplot3d)

teapot &lt;- read.csv(&quot;data/teapot.csv&quot;, header = FALSE)
dat &lt;- matrix(unlist(teapot), ncol = 3, byrow = TRUE)
head(dat)
##      [,1] [,2] [,3]
## [1,] 40.6 28.3 -1.1
## [2,] 40.1 30.4 -1.1
## [3,] 40.7 31.1 -1.1
## [4,] 42.0 30.4 -1.1
## [5,] 43.5 28.3 -1.1
## [6,] 37.5 28.3 14.5

# teapot in 3D
scatterplot3d(dat[ , 1], dat[ , 2], dat[ , 3], highlight.3d = TRUE, angle = 75, pch = 19, lwd = 15, xlab = &quot;&quot;, ylab = &quot;&quot;, zlab = &quot;&quot;, main = &quot;teapot in 3D&quot;)
</pre>
<p><img sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px" src="https://i1.wp.com/blog.ephorie.de/wp-content/uploads/2018/12/teapot-in-3D-1024x731.png?w=450" alt="" class="wp-image-142" data-recalc-dims="1"></p>
<p>And now we project that teapot from 3D-space onto a 2D-plane while retaining as much information as possible by taking the two eigenvectors with biggest eigenvalues:</p>
<pre>
# PCA
(eigenval &lt;- eigen(cov(dat))$values)
## [1] 2050.6935  747.5764  553.4070

eigenvec &lt;- eigen(cov(dat))$vectors
# take only the two eigenvectors with biggest eigenvalues
PCA_2 &lt;- dat %*% eigenvec[ , 1:2] 
plot(PCA_2, ylim = c(50, -55), pch = 19, lwd = 35, col= &quot;blue&quot;, xlab = &quot;&quot;, ylab = &quot;&quot;, main = &quot;teapot in 2D&quot;)
</pre>
<p><img sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px" src="https://i0.wp.com/blog.ephorie.de/wp-content/uploads/2018/12/teapot-in-2D-1024x731.png?w=450" alt="" class="wp-image-141" data-recalc-dims="1"></p>
<pre>
round(cumsum(eigenval)/sum(eigenval) * 100, 2) # cumulative percentage of retrieved information
## [1]  61.18  83.49 100.00
</pre>
<p>That was fun, wasn’t it!</p>
<p>In data science you will often use PCA to condense large datasets into smaller datasets while retaining most of the information. PCA is like clustering an unsupervised learning method.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.ephorie.de/intuition-for-principal-component-analysis-pca"> R-Bloggers – Learning Machines</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/J-aQntbRcww" height="1"></a></div>