<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/how-to-build-an-image-recognizer-in-r-using-just-a-few-images/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2017/08/image-featurizer.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p><a rel="nofollow" href="https://azure.microsoft.com/en-us/services/cognitive-services/#vision" target="_blank">Microsoft Cognitive Services</a> provides several APIs for image recognition, but if you want to build your own recognizer (or create one that works offline), you can use the new <a rel="nofollow" href="https://docs.microsoft.com/en-us/r-server/r-reference/microsoftml/featurizeimage" target="_blank">Image Featurizer</a> capabilities of Microsoft R Server. </p>
<p>The process of training an image recognition system requires LOTS of images — millions and millions of them. The process involves feeding those images into a deep neural network, and during that process the network generates &quot;features&quot; from the image. These features might be versions of the image including just the outlines, or maybe the image with only the green parts. You could further boil those features down into a single number, say the length of the outline or the percentage of the image that is green. With enough of these &quot;features&quot;, you could use them in a traditional machine learning model to classify the images, or perform other recognition tasks.</p>
<p>But if you don't <em>have</em> millions of images, it's still possible to generate these features from a model that has already been trained <em>on</em> millions of images. <a rel="nofollow" href="http://www.cv-foundation.org/openaccess/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html" target="_blank">ResNet</a> is a very deep neural network model trained for the task of image recognition which has been used to <a rel="nofollow" href="http://image-net.org/challenges/LSVRC/2015/results" target="_blank">win major computer-vision competitions</a>. With the <code>rxFeaturize</code> function in Microsoft R Client and Microsoft R Server, you can generate 4096 features from this model on any image you provide. The features themselves are meaningful only to a computer, but that vector of 4096 numbers between zero and one is (ideally) a distillation of the unique characteristics of that image as a human would recognize it. You can then use that features vector to create your own image-recognition system without the burden of training your own neural network on a large corpus of images.</p>
<p>On the <a rel="nofollow" href="https://blogs.technet.microsoft.com/machinelearning/2017/07/31/find-images-with-images/" target="_blank">Cortana Intelligence and ML blog</a>, Remko de Lange provides a simple example: given a collection of 60-or-so images of chairs like those below, how can you find the image that most looks like a deck chair?</p>
<p><a rel="nofollow" href="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb09b9a1d1970d-pi" target="_blank"><img border="0" alt="Chairs" title="Chairs" src="http://revolution-computing.typepad.com/.a/6a010534b1db25970b01bb09b9a1d1970d-800wi"></a></p>
<p>First, you need a representative image of a deck chair:</p>
<p><a rel="nofollow" href="http://a2.typepad.com/6a0105360ba1c6970c01bb09b9a1f2970d-pi" target="_blank"><img alt="Chair56" title="Chair56" src="http://a2.typepad.com/6a0105360ba1c6970c01bb09b9a1f2970d-400wi"></a></p>
<p>Then, you calculate the features vector for that image using <a rel="nofollow" href="https://docs.microsoft.com/en-us/r-server/r-reference/microsoftml/rxfeaturize" target="_blank">rxFeaturize</a>. </p>
<div>
<pre>imageToMatchDF <span>&lt;-</span> rxFeaturize(
  data <span>=</span> data.frame(Image<span>=</span><span>&quot;deck-chair.jpg&quot;</span>),
  mlTransforms <span>=</span> list(
    loadImage(vars <span>=</span> list(Features <span>=</span> <span>&quot;Image&quot;</span>)),
    resizeImage(vars <span>=</span> <span>&quot;Features&quot;</span>, width <span>=</span> <span>227</span>, height <span>=</span> <span>227</span>),
    extractPixels(vars <span>=</span> <span>&quot;Features&quot;</span>),
    featurizeImage(var <span>=</span> <span>&quot;Features&quot;</span>, dnnModel <span>=</span> <span>&quot;alexnet&quot;</span>)   
  ))
</pre>
</div>
<p>Note that when featurizing an image, you need to shrink it down to a specified size (the built-in function <code>resizeImage</code> handles that). There are also several pretrained models to choose from: three variants of ResNet and also <a rel="nofollow" href="http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks" target="_blank">Alexnet</a>, which we use here. This gives us a features vector of 4096 numbers to represent our deck chair image. Then, we just need to use the same process to calculate the features vector for our 60 other images, and find the one that's closest to our deck chair. We can simply use the <code><a rel="nofollow" href="https://www.rdocumentation.org/packages/stats/versions/3.4.1/topics/dist?tap_a=5644-dce66f&tap_s=10907-287229" target="_blank">dist</a></code> function in R to do that, for example, and that's exactly what <a rel="nofollow" href="https://github.com/delange/image2image/blob/master/image2image.R" target="_blank">Remko's script on Github</a> does. The image with the closest features vector to our representative image is this one:</p>
<p><a rel="nofollow" href="http://a4.typepad.com/6a0105360ba1c6970c01b7c9166934970b-pi" target="_blank"><img alt="Chair43" title="Chair43" src="http://a4.typepad.com/6a0105360ba1c6970c01b7c9166934970b-400wi"></a></p>
<p>So, even with a relatively small collection of images, it's possible to build a powerful image recognition system using image featurization and the powerful image recognition neural networks provided with Microsoft R. The <a rel="nofollow" href="https://github.com/delange/image2image" target="_blank">complete code and sample images</a> used in this example are available on Github. (Note, you'll need to have a license for <a rel="nofollow" href="https://docs.microsoft.com/en-us/r-server/what-is-microsoft-r-server" target="_blank">Microsoft R Server</a> or install the free <a rel="nofollow" href="https://docs.microsoft.com/en-us/r-server/r-client/what-is-microsoft-r-client" target="_blank">Microsoft R Client</a> with the <a rel="nofollow" href="https://docs.microsoft.com/en-us/r-server/install/microsoftml-install-pretrained-models" target="_blank">pretrained models</a> option to use the image featurization functions in the script.) And for more details on creating this recognizer, check out the blog post below.</p>
<p>Cortana Intelligence and Machine Learning Blog: <a rel="nofollow" href="https://blogs.technet.microsoft.com/machinelearning/2017/07/31/find-images-with-images/" target="_blank">Find Images With Images</a></p>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2017/08/image-featurizer.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/RXOdmkcYR3A" height="1"></a></div>