<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/a-poor-mans-video-analyzer/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://longhowlam.wordpress.com/2017/06/16/a-poor-mans-video-analyzer/"> R – Longhow Lam's Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2><img data-orig-file="https://longhowlam.files.wordpress.com/2017/06/video01.png?w=450&amp;h=361&amp;fit=658%2C361" data-orig-size="933,512" data-medium-file="https://longhowlam.files.wordpress.com/2017/06/video01.png?w=450&amp;h=361&amp;fit=658%2C361?w=300" data-attachment-id="2236" data-image-meta="{" data-image-description="" src="https://longhowlam.files.wordpress.com/2017/06/video01.png?w=450&h=361&fit=658%2C361" data-permalink="https://longhowlam.wordpress.com/2017/06/16/a-poor-mans-video-analyzer/video01/" data-large-file="https://longhowlam.files.wordpress.com/2017/06/video01.png?w=450&amp;h=361&amp;fit=658%2C361?w=440" alt="video01" data-image-title="video01" sizes="(max-width: 658px) 100vw, 658px" data-comments-opened="1" class="wp-image-2236" data-recalc-dims="1"></h2>
<h2>Introduction</h2>
<p>Not so long ago there was a nice <a rel="nofollow" href="https://www.meetup.com/Dataiku-Dutch-Meetup-Group/events/240023945/?eventId=240023945&from=ref" target="_blank">dataiku meetup with Pierre Gutierrez</a> talking about transfer learning. RStudio recently released <a rel="nofollow" href="https://rstudio.github.io/keras/" target="_blank">the keras package</a>, an R interface to work with <a rel="nofollow" href="https://keras.io/" target="_blank">keras</a> for deep learning and transfer learning. Both events inspired me to do some experiments at my work here at RTL and explore the usability of it for us at RTL. I like to share the slides of the presentation that I gave internally at RTL, you can find them on <a rel="nofollow" href="https://www.slideshare.net/LonghowLam/keras-on-tensorflow-in-r-python" target="_blank">slide-share</a>.</p>
<p>As a side effect, another experiment that I like to share is the “<strong>poor man’s video analyzer</strong>“. There are several vendors now that offer API’s to analyze videos. See for example the one that <a rel="nofollow" href="https://www.videoindexer.ai/" target="_blank">Microsoft </a>offers. With just <strong>a few lines of R</strong> code I came up with a shiny app that is a very cheap imitation <img alt="🙂" src="https://i0.wp.com/s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f642.png?w=456&ssl=1" class="wp-smiley" data-recalc-dims="1"></p>
<h2>Set up of the R Shiny app</h2>
<p>To run the shiny app a few things are needed. Make sure that <a rel="nofollow" href="https://ffmpeg.org/" target="_blank">ffmpeg</a> is installed, it is used to extract images from a video. Tensorflow and keras need to be installed as well. The extracted images from the video are parsed through a pre-trained VGG16 network so that each image is tagged.</p>
<p>After this tagging a data table will appear with the images and their tags. <strong>That’s it! </strong>I am sure there are better visualizations than a data table to show a lot of images. If you have a better idea just adjust <a rel="nofollow" href="https://github.com/longhowlam/videoanalyser" target="_blank">my shiny app on GitHub</a>…. <img alt="🙂" src="https://i0.wp.com/s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f642.png?w=456&ssl=1" class="wp-smiley" data-recalc-dims="1"></p>
<h2>Using the app, some screen shots</h2>
<p>There is a simple interface, specify the number of frames per second that you want to analyse. And then upload a video, many formats are supported (by ffmpeg), like *.mp4, *.mpeg, *.mov</p>
<div><img src="https://i2.wp.com/media.licdn.com/mpr/mpr/AAEAAQAAAAAAAAxCAAAAJGE2NTQwNzYyLTdhN2YtNGU4Yi1iYmE0LTQ3MDc2NTExMDAzYw.png?w=456&ssl=1" data-recalc-dims="1"></div>
<p>Click on video images to start the analysis process. This can take a few minutes, when it is finished you will see a data table with extracted images and their tags from VGG-16.</p>
<div><img src="https://i2.wp.com/media.licdn.com/mpr/mpr/AAEAAQAAAAAAAAoJAAAAJDQ2MWUwZDUyLTI1ZGYtNGY3ZC04OTRhLThkNGZlZjE3MGIyMA.png?w=456&ssl=1" data-recalc-dims="1"></div>
<p>Click on ‘info on extracted classes’ to see an overview of the class. You will see a bar chart of tags that where found and the output of ffmpeg. It shows some info on the video.</p>
<div><img src="https://i0.wp.com/media.licdn.com/mpr/mpr/AAEAAQAAAAAAAAvIAAAAJDgzMzNlNTkzLTkwMmItNDI4Yy1iYjhkLTliZmNlZmFkMzJkYw.png?w=456&ssl=1" data-recalc-dims="1"></div>
<p>If you have code to improve the data table output in a more fancy visualization, just go to my <a rel="nofollow" href="https://github.com/longhowlam/videoanalyser" target="_blank">GitHub</a>. For those who want to play around, look at a <em>live</em> video analyzer shiny app <a rel="nofollow" href="http://145.131.21.163:3838/sample-apps/videoanalyzer/" target="_blank">here</a>.</p>
<p>A <a rel="nofollow" href="http://145.131.21.163:3838/sample-apps/videoanalyzer2/" target="_blank">Shiny app version using miniUI</a> will be a better fit for small mobile screens.</p>
<p>Cheers, Longhow</p>
<p> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/longhowlam.wordpress.com/2232/" target="_blank"><img border="0" alt="" src="http://feeds.wordpress.com/1.0/comments/longhowlam.wordpress.com/2232/"></a> <img border="0" alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" data-recalc-dims="1"></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://longhowlam.wordpress.com/2017/06/16/a-poor-mans-video-analyzer/"> R – Longhow Lam's Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/wldSjwzYSQY" height="1"></a></div>