<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/singular-value-decomposition-svd-tutorial-using-examples-in-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://www.displayr.com/singular-value-decomposition-in-r/"> R – Displayr</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><img sizes="(max-width: 300px) 100vw, 300px" src="https://i0.wp.com/www.displayr.com/wp-content/uploads/2017/07/svdish-300x185.png?resize=300%2C185&ssl=1" alt="singular value decomposition" class="webfeedsFeaturedVisual wp-post-image" data-recalc-dims="1">
<p>If you have ever looked with any depth at statistical computing for multivariate analysis, there is a good chance you have come across the <em>singular value decomposition (SVD). </em>It is a workhorse for techniques that decompose data, such as <em>correspondence analysis </em>and <em>principal components analysis. </em>In this post I explain, at an intuitive level, how it works. I demonstrate this using examples in R. If you have not come across the SVD before, skip this post! It is only for that rare connoisseur, who has heard of it, wants to understand it a bit better, but is averse to lots of maths.</p>
<hr>
<h1><span>A singular value decomposition case study in R</span></h1>
<p>The table below shows the <em>standardized residuals </em>from a <em>contingency table </em>showing the relationship between education and readership of a newspaper. The R code used to generate the table is below. More about this data and R code, and why it is interesting, will be available in my forthcoming post about the maths of correspondence analysis.</p>
<p><iframe src="https://embedded.azureedge.net/75668/224251/6e4b1a5f-7285-456b-b7fa-5b54a3c48cfe.html?v=28297-90b5b922692055048071" width="450" height="245.3243"></iframe>
</p>
<pre> 
education.by.readership = matrix(c(5, 18, 19, 12, 3, 7, 46, 29, 40, 7, 2, 20, 39, 49, 16), 
    nrow = 5,
    dimnames = list(
        &quot;Level of education&quot; = c(&quot;Some primary&quot;, &quot;Primary completed&quot;, &quot;Some secondary&quot;, &quot;Secondary completed&quot;, &quot;Some tertiary&quot;),
        &quot;Category of readership&quot; = c(&quot;Glance&quot;, &quot;Fairly thorough&quot;, &quot;Very thorough&quot;))) 
O = education.by.readership / sum(education.by.readership)
E = rowSums(O) %o% colSums(O)
Z = (O - E) / sqrt(E)</pre>
<hr>
<h1><span>How to compute the SVD</span></h1>
<p>The table above is a <em>matrix </em>of numbers. I am going to call it <b>Z</b>. The singular value decomposition is computed using the <span>svd</span> function. The following code computes the singular value decomposition of the matrix <strong>Z</strong>, and assigns it to a new object called <span>SVD<span>, which contains<b> </b></span></span>one vector, <span>d</span>, and two matrices, <span>u</span> and <span>v</span>. The vector, <span>d</span>, contains the <em>singular values. </em>The first matrix, <span>u</span>, contains the <em>left singular vectors, </em>and <span>v</span> contains the <em>right singular vectors. </em>The left singular vectors represent the rows of the input table, and the right singular vectors represent their columns.</p>
<pre> 
SVD = svd(Z)
</pre>
<p><iframe src="https://embedded.azureedge.net/75668/224251/a87b4000-1c3f-401f-8df9-01f9d4159c21.html?v=28297-90b5b922691100640493" width="380.3874" height="311"></iframe>
</p>
<hr>
<h1><span>Recovering the data</span></h1>
<p>The singular value decomposition (SVD) has four useful properties. The first is that these two matrices and vector can be “multiplied” together to re-create the original input data, <strong>Z</strong>.  In the data we started with (<strong>Z</strong>), we have a value of -0.064751 in the 5th row, 2nd column. We can work this out from the results of the SVD by multiplying each element of <span>d</span> with the elements of the 5th row of <span>u</span> and the 2nd row <span>v</span>.</p>
<p>That is: -0.064751  = 0.2652708*0.468524*(-0.4887795) + 0.1135421*(-0.0597979)*0.5896041 + 0*(-0.6474922)*(-0.6430097)</p>
<p>This can be achieved in R using the code:</p>
<pre> 
sum(SVD$d * SVD$u[5, ] * SVD$v[2, ])
</pre>
<p>Better yet, if we want to recompute the whole table of numbers at once, we can use a bit of matrix algebra:</p>
<pre> 
SVD$u %*% diag(SVD$d) %*% t(SVD$v)
</pre>
<p>Now, at first glance this property may not seem so useful. Indeed, it does not even seem very clever. We started with a table of 15 numbers. Now, we have one vector and two tables, containing a total of 27 numbers. We seem to be going backwards!</p>
<hr>
<h1><span>Reducing the data</span></h1>
<p>The second useful property of the SVD relates to the values in <span>d</span>. They are sorted in descending order (ties are possible). Why is this important? Take a look at the last value in <span>d</span>. It is 2.71825390754254E-17. In reality, this is 0 (computers struggle to compute 0 exactly). When recovering the data, we can ignore the last value of <span>d</span>, and also the last column of each of <span>u</span> and <span>v.</span> Their values are multiplied by 0 and thus are irrelevant. Now, we only have 18 numbers to look at. This is still more than the 15 we started with.</p>
<p>The values of <span>d</span> tell us the relative importance of each of the columns in <span>u</span> and <span>v</span> in describing the original data. We can compute the <em>variance</em> in the original data (<strong>Z</strong>) that is explained by the columns by first squaring the values in <span>d</span>, and then expressing these as proportions. If you run the following  Rcode, it shows that the first <em>dimension</em> explains 85% of variance in the data.</p>
<pre>
variance.explained = prop.table(svd(Z)$d^2)
</pre>
<p>So, if we are happy to ignore 15% of the information in the original data, we only need to look at the first column in <span>u</span> and the first column in <span>v</span>. Now we have to look at less than half the numbers that we started with.</p>
<p>Halving the number of numbers to consider may not seem like a sufficient benefit. However, the bigger the data set, the bigger the saving. For example, if we had a table with 20 rows and 20 columns, we may only need to look at the first couple of columns, only needing to consider 10% of the number of values that we started with. This is the basic logic of techniques like principle components analysis and correspondence analysis. In addition to reducing the number of values we need to look at, this also allows us to chart the values, which saves more time. There is rarely a good way to chart 20 columns of data, but charting 2 columns is usually straightforward.</p>
<hr>
<h1><span>Two more properties</span></h1>
<p>The third property of the SVD is that the rows of <span>u</span> represents the row categories of the original table, and the rows of <span>v</span> represent the column categories. The fourth property is that the columns of <span>u</span> are orthogonal to each other, and the columns of <span>v</span> are orthogonal to each other. With these two properties combined, we end up with considerable simplicity in future analyses. For example, this allows us to compute uncorrelated principal components in principal components analysis and to produce plots of correspondence analysis. I will walk through this in detail in my forthcoming post on the math of correspondence analysis.</p>
<p>All the <em>R</em> code in this post has been run using Displayr. Anyone can <a rel="nofollow" href="https://app.displayr.com/Try/SVD" target="_blank">explore SVD and the R code</a> used in this post by logging into Displayr.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://www.displayr.com/singular-value-decomposition-in-r/"> R – Displayr</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/WigPytLpVyo" height="1"></a></div>