<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/goodness-of-fit-in-mds-and-t-sne-with-shepard-diagrams/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://www.displayr.com/goodness-of-fit-in-mds-and-t-sne-with-shepard-diagrams/?utm_medium=Feed&utm_source=Syndication"> R – Displayr</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>The goodness of fit for data reduction techniques such as MDS and t-SNE can be easily assessed with Shepard diagrams. A Shepard diagram compares how far apart your data points are before and after you transform them (ie: goodness-of-fit) as a scatter plot. Shepard diagrams can be used for data reduction techniques like <em>principal components analysis</em> (PCA), <em>multidimensional scaling</em> (MDS), or <a rel="nofollow" href="https://www.displayr.com/using-t-sne-to-visualize-data-before-prediction" target="_blank">t-SNE</a>.</p>
<p>In this post, I illustrate goodness of fit with Shapard diagrams using a simple example which maps the locations of cities in Europe using t-SNE and MDS. You will see that the t-SNE approach, which is not designed to preserve all distances in the data, produces an odd-looking map of Europe and a distorted Shepard diagram. While the MDS approach produces an ideal-looking Shepard diagram. This is because MDS does not introduce any distortions in data which has only two dimensions. I then look at real (high dimensional) data using t-SNE and MDS Shepard diagrams.</p>
<hr>
<h2><span>The t-SNE example</span></h2>
<p>I’ll start with an example of t-SNE. The chart below uses t-SNE to place European cities on a map using a matrix of distances between cities of Europe. Having <a rel="nofollow" href="https://www.displayr.com/using-t-sne-to-visualize-data-before-prediction" target="_blank">previously championed</a> t-SNE, something has clearly gone wrong on this map.</p>
<p><iframe width="450" src="https://embedded.azureedge.net/75668/234955/f60cb1bb-4516-47d0-b548-366c951484ce.html?v=29062-354f82b68e2019595222"></iframe>
</p>
<p>First and most obvious, the orientation of the map is incorrect. More southern cities are at the top of the map. In fact, this is not a failure of t-SNE. The algorithm doesn’t know that conventionally we put north at the top, so we are free to rotate the output.</p>
<p>More seriously, the relative distances and placements on the map are wrong. For example, you would not make a halfway stop in Rome if traveling from London to Copenhagen. I <a rel="nofollow" href="https://www.displayr.com/using-t-sne-to-visualize-data-before-prediction" target="_blank">described in an earlier post</a> why this happens: t-SNE tries to maintain the placement of each point amongst its closest neighbors. So it does not aim to get the distances correct.</p>
<p>This is a trivial example because we know how the true map should look. If we didn’t know where the cities of Europe really are, how could we tell if t-SNE produced an accurate visualization?</p>
<hr>
<h2><span><strong>Using a Shepard Diagram for t-SNE</strong></span></h2>
<p>A Shepard diagram is one way of assessing if the t-SNEW produced an accurate visualization. It is a scatterplot of distances between data points. On the x-axis we plot the original distances. On the y-axis we plot the distances output by a dimension reduction algorithm. Below I show a Shepard diagram for t-SNE applied the map of European cities.</p>
<p>The scatter plot shows a rough correlation in that cities closer together in <em>input space</em> tend to be closer together in <em>output space</em>. However, by hovering over the points we can see that, for example, Brussels and Hamburg are too far apart. The fact that the Spearman’s rank correlation is only 78% shows that the ordering of the distances is wrong in some cases.</p>
<p><iframe width="450" src="https://embedded.azureedge.net/75668/234955/15dc6f51-06fa-446b-99cd-a8bb568d5805.html?v=29062-354f82b68e1182490078"></iframe>
</p>
<hr>
<h2><span><strong>Using Shepard Diagrams with Multidimensional Scaling (MDS) </strong></span></h2>
<p>Whilst t-SNE preserves local neighbors, MDS takes a different approach to mapping. It has 2 main variants:</p>
<ul>
<li><em>Metric MDS</em> minimizes the difference between distances in input and output spaces.</li>
<li><em>Non-metric MDS</em> aims to preserve the ranking of distances between input and output spaces.</li>
</ul>
<p>Applying metric MDS to the European cities gives the map below. You might recognize this as being correct (if you rotate it around a little).</p>
<p><iframe width="450" src="https://embedded.azureedge.net/75668/234955/d0c0fefd-1f6c-49e9-a5b5-58dcfb90a657.html?v=29062-354f82b68e2096358383"></iframe>
</p>
<p>Because we start with a distance matrix derived from 2 dimensions, MDS is capable of faithfully recreating the map. In other words, there is no information loss. This is true of both the metric and non-metric versions. We can confirm the accuracy from the Shepard diagram below. This shows that the mapped distances are in the same order as the original distances.</p>
<p><iframe width="450" src="https://embedded.azureedge.net/75668/234955/1b111adc-4122-4b2f-bbef-411a30c1a76c.html?v=29062-354f82b68e1010411706"></iframe>
</p>
<p>While a really accurate dimension reduction like the one above will produce a straight line. However since information is almost always lost during data reduction, at least on real, high-dimension data, Shepard diagrams rarely look this straight.</p>
<hr>
<h2><span><strong>Applying t-SNE and MDS to High Dimension Data</strong></span></h2>
<p>Let’s try and apply the techniques above on a real data set. In a <a rel="nofollow" href="https://www.displayr.com/using-t-sne-to-visualize-data-before-prediction" target="_blank">previous analysis</a>, I used t-SNE to reduce the dimensionality of a data set which described the physical characteristics of leaves from a variety of plants. The Shepard diagram for the t-SNE analysis reveals a rank correlation of 88%.</p>
<p>Metric MDS produces the following chart for the leaf dataset.</p>
<p><iframe width="450" src="https://embedded.azureedge.net/75668/234955/a09eacd3-cd09-432c-9533-c6a674c1561b.html?v=29062-354f82b68e2060920362"></iframe>
</p>
<p>The MDS groups species more loosely than the t-SNE. So the rank correlation from the Shepard diagram is 90%, which is slightly better than the t-SNE.</p>
<p>Non-metric MDS aims to maintain the distance ranking. So it is no surprise that it has an even higher rank correlation of 97% as shown below.</p>
<p><iframe width="450" src="https://embedded.azureedge.net/75668/234955/8589c965-4b8a-47bc-8038-322bd61efe21.html?v=29062-354f82b68e596315714"></iframe>
</p>
<hr>
<h2><span>t-SNE versus MDS: which is better?</span></h2>
<p>Which method is better? That depends on what we mean by “better”. t-SNE’s strength lies in creating tight clusters for visualization. Often we care more about relative positioning than absolute differences, in which case non-metric is preferred to metric MDS.</p>
<hr>
<h2><span>Software for Shepard diagrams</span></h2>
<p>In <a rel="nofollow" href="http://www.displayr.com/" target="_blank">Displayr</a>, PCA, t-SNE, and MDS options are all available under <strong>Insert > More > Dimension Reduction</strong>. You can create a Shepard diagram by selecting <strong>Insert > More > Dimension Reduction > Plot > Goodness of Fit</strong>. Select your PCA, t-SNE, or MDS in the <strong>Dimension Reduction</strong> menu under <span>Properties</span>, and then click <strong><span>Calculate</span></strong>.</p>
<p>In R, these Shepard diagrams are available using the <span>GoodnessOfFitPlot()</span> function from the <span>flipDimensionReduction</span> package.</p>
<hr>
<h2><span>Replicate this analysis</span></h2>
<p>All the analysis in this post was conducted using R in Displayr. You can review the underlying data and code used in my analysis and <a rel="nofollow" href="https://app.displayr.com/Try/Shepard%20Diagrams" target="_blank">create your own Shepard diagram analysis here</a>.  The <span>flipDimensionReduction<span> package (<a rel="nofollow" href="https://github.com/Displayr/flipDimensionReduction" target="_blank">available on GitHub</a>) was used, which itself uses the <span>Rtsne</span> and <span>MASS</span> packages.</span></span></p>
<hr>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://www.displayr.com/goodness-of-fit-in-mds-and-t-sne-with-shepard-diagrams/?utm_medium=Feed&utm_source=Syndication"> R – Displayr</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/zxj448NqWxU" height="1"></a></div>