<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/manifold-visualization-polynomials-to-the-rescue/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://matloff.wordpress.com/2018/09/30/manifold-visualization-polynomials-to-the-rescue/"> Mad (Data) Scientist</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Our <a rel="nofollow" href="https://arxiv.org/abs/1806.06850" target="_blank">arXiv paper</a> and the associated R package <a rel="nofollow" href="http://github.com/matloff/polyreg" target="_blank">polyreg</a> caused a bit of a stir, both pro and con, when we first <a rel="nofollow" href="https://matloff.wordpress.com/2018/06/20/neural-networks-are-essentially-polynomial-regression/" target="_blank">announced them</a> here in June. The discussion even spread as far as Twitter, Reddit and Hacker News. We’ll be announcing a revised paper, and various new features to the package, very soon.</p>
<p>But the purpose of this blog post is to focus on one particular new feature, a visualization tool. Over the years a number of “nonlinear” methods generalizing Principal Components Analysis (PCA) have been proposed, such as ICA and KPCA some time ago, and more recently t-SNE and UMAP.</p>
<p>I’ve long felt that applying PCA to “polynomial-ized” versions of one’s data should do well too, and of course much more simply, a major virtue. So, now  that we have machinery, the <strong>polyreg</strong> package, to conveniently build multivariate polynomial models — including for categorical variables, an important case — I developed a new function in <strong>polyreg</strong>, named <strong>prVis()</strong>. I’ll illustrate it in this blog post, and compare it to UMAP.</p>
<p>A popular example in the “manifold visualization” (MV) business is the Swiss Roll model, which works as follows: A 4-component mixture of bivariate normals is generated, yielding a 2-column data frame whose column names are ‘x’ and ‘y’. Now derive from that a 3-column data frame, consisting triples of the form (x cos (x), y, x sin(x)). Here is the goal:</p>
<p>Using one of the MV methods on the 3-column data, and not knowing that the 2-column data was a 4-component mixture, could a person discern that latter fact? We’ll try UMAP (said to be faster than t-SNE) and <strong>prVis()</strong>.</p>
<p>First, what UMAP implementation to use? Several excellent ones are available for R, such as <a rel="nofollow" href="https://github.com/ropenscilabs/umapr/blob/master/DESCRIPTION" target="_blank">umapr</a>, <a rel="nofollow" href="https://cran.r-project.org/web/packages/umap/index.html" target="_blank">umap</a> and <a rel="nofollow" href="https://github.com/jlmelville/uwot" target="_blank">uwot.</a> I use the last one, as it was the easiest for me to install, and especially because it includes a <strong>predict()</strong> method.</p>
<p>Here is the code. We read in the data from a file included in the <strong>polyreg</strong> package into <strong>sw</strong>, and change the mode of the last column to an R factor. (Code not shown.) The data frame sw actually has 4 columns, the last being the mixture component ID, 1-4.</p>
<p>We first try PCA:</p>
<p>plot(prcomp(sw[,-4])$x)<br>
<a rel="nofollow" href="https://matloff.files.wordpress.com/2018/09/swpca.png" target="_blank"><img data-orig-file="https://matloff.files.wordpress.com/2018/09/swpca.png" data-orig-size="480,480" data-medium-file="https://matloff.files.wordpress.com/2018/09/swpca.png?w=300&amp;h=300&amp;fit=300%2C300&amp;resize=300%2C300" data-attachment-id="2105" data-image-meta="{" data-image-description="" src="https://matloff.files.wordpress.com/2018/09/swpca.png?w=300&h=300&fit=300%2C300&resize=300%2C300" data-permalink="https://matloff.wordpress.com/2018/09/30/manifold-visualization-polynomials-to-the-rescue/swpca/" data-large-file="https://matloff.files.wordpress.com/2018/09/swpca.png?w=474" alt="" data-image-title="swPCA" sizes="(max-width: 300px) 100vw, 300px" width="300" data-comments-opened="1" class="wp-image-2105" data-recalc-dims="1" height="300"></a></p>
<p>That swirl is where the Swiss Roll model gets its name, and clearly, the picture gives no hint at all as to how many components were in the original data. The model was constructed in such a way that PCA would fail.</p>
<p>So let’s try UMAP.</p>
<p>plot(umap(sw[,-4]))</p>
<p>We get this:<br>
<a rel="nofollow" href="https://matloff.files.wordpress.com/2018/09/swumap.png" target="_blank"><img data-orig-file="https://matloff.files.wordpress.com/2018/09/swumap.png" data-orig-size="480,480" data-medium-file="https://matloff.files.wordpress.com/2018/09/swumap.png?w=300&amp;h=300&amp;fit=300%2C300&amp;resize=300%2C300" data-attachment-id="2103" data-image-meta="{" data-image-description="" src="https://matloff.files.wordpress.com/2018/09/swumap.png?w=300&h=300&fit=300%2C300&resize=300%2C300" data-permalink="https://matloff.wordpress.com/2018/09/30/manifold-visualization-polynomials-to-the-rescue/swumap/" data-large-file="https://matloff.files.wordpress.com/2018/09/swumap.png?w=474" alt="" data-image-title="swUmap" sizes="(max-width: 300px) 100vw, 300px" width="300" data-comments-opened="1" class="wp-image-2103" data-recalc-dims="1" height="300"></a></p>
<p>So, how many components are there? (Remember, we’re pretending we don’t know it’s 4.) On the left, for instance, is that loop 1 component? 2? 3? We might go for 1. Well, let’s un-pretend now, and use the component labels, color-coded:</p>
<p>plot(umap(sw[,-4]),col=sw[,4])<br>
<a rel="nofollow" href="https://matloff.files.wordpress.com/2018/09/swumapcolor.png" target="_blank"><img data-orig-file="https://matloff.files.wordpress.com/2018/09/swumapcolor.png" data-orig-size="480,480" data-medium-file="https://matloff.files.wordpress.com/2018/09/swumapcolor.png?w=300&amp;h=300&amp;fit=300%2C300&amp;resize=300%2C300" data-attachment-id="2104" data-image-meta="{" data-image-description="" src="https://matloff.files.wordpress.com/2018/09/swumapcolor.png?w=300&h=300&fit=300%2C300&resize=300%2C300" data-permalink="https://matloff.wordpress.com/2018/09/30/manifold-visualization-polynomials-to-the-rescue/swumapcolor/" data-large-file="https://matloff.files.wordpress.com/2018/09/swumapcolor.png?w=474" alt="" data-image-title="swUmapColor" sizes="(max-width: 300px) 100vw, 300px" width="300" data-comments-opened="1" class="wp-image-2104" data-recalc-dims="1" height="300"></a></p>
<p>Wow, that “loop” actually contained 2 of the original components, not just 1. We were led astray.</p>
<p>Let’s try <strong>prVis()</strong>.</p>
<p>prVis(sw[,-4],labels=FALSE)<br>
<a rel="nofollow" href="https://matloff.files.wordpress.com/2018/09/swprvis.png" target="_blank"><img data-orig-file="https://matloff.files.wordpress.com/2018/09/swprvis.png" data-orig-size="480,480" data-medium-file="https://matloff.files.wordpress.com/2018/09/swprvis.png?w=300&amp;h=300&amp;fit=300%2C300&amp;resize=300%2C300" data-attachment-id="2106" data-image-meta="{" data-image-description="" src="https://matloff.files.wordpress.com/2018/09/swprvis.png?w=300&h=300&fit=300%2C300&resize=300%2C300" data-permalink="https://matloff.wordpress.com/2018/09/30/manifold-visualization-polynomials-to-the-rescue/swprvis/" data-large-file="https://matloff.files.wordpress.com/2018/09/swprvis.png?w=474" alt="" data-image-title="swPRvis" sizes="(max-width: 300px) 100vw, 300px" width="300" data-comments-opened="1" class="wp-image-2106" data-recalc-dims="1" height="300"></a></p>
<p>Still rather swirly, but it definitely suggests 4 components. Let’s check, revealing the color-coded actual components:</p>
<p>prVis(sw,labels=TRUE)</p>
<p><a rel="nofollow" href="https://matloff.files.wordpress.com/2018/09/swprviscolor.png" target="_blank"><img data-orig-file="https://matloff.files.wordpress.com/2018/09/swprviscolor.png" data-orig-size="480,480" data-medium-file="https://matloff.files.wordpress.com/2018/09/swprviscolor.png?w=300&amp;h=300&amp;fit=300%2C300&amp;resize=300%2C300" data-attachment-id="2107" data-image-meta="{" data-image-description="" src="https://matloff.files.wordpress.com/2018/09/swprviscolor.png?w=300&h=300&fit=300%2C300&resize=300%2C300" data-permalink="https://matloff.wordpress.com/2018/09/30/manifold-visualization-polynomials-to-the-rescue/swprviscolor/" data-large-file="https://matloff.files.wordpress.com/2018/09/swprviscolor.png?w=474" alt="" data-image-title="swPRvisColor" sizes="(max-width: 300px) 100vw, 300px" width="300" data-comments-opened="1" class="wp-image-2107" data-recalc-dims="1" height="300"></a></p>
<p>Very nice! Each of the 4 apparent components really did correspond to an actual component.</p>
<p>Of course, usually the difference is not this dramatic, but in applying <strong>prVis()</strong> to a number of examples, I’ve found that it does indeed do as well as, or better than, UMAP and t-SNE, but in a simpler, more easily explained manner, as noted a major virtue.</p>
<p>I’ll post another example, showing an interesting interpretation of a real dataset, in the next day or so.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://matloff.wordpress.com/2018/09/30/manifold-visualization-polynomials-to-the-rescue/"> Mad (Data) Scientist</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/Tg0tEMMscjg" height="1"></a></div>