<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/the-instant-rise-of-machine-intelligence/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://flovv.github.io/MachineLearning_Improvement/"> Florian Teschner</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Currently the news are filled with articles about the rise of machine intelligence, artificial intelligence and deep learning.<br>
For the average reader it seems that there was this single technical breakthrough that made AI possible. While I strongly believe in the fascinating opportunities around deep learning for image recognition, natural language processing and even end-to-end “intelligent” systems (e.g. chat bots), I wanted to get a better feeling of the recent technological progress.</p>
<p>First I read about tensorflow (for R) and watched a number of great talks about it. Do not miss <a target="_blank" rel="nofollow" href="https://www.youtube.com/watch?v=F1ka6a13S9I">Nuts and Bolts of Applying Deep Learning (Andrew Ng)</a> and <a target="_blank" rel="nofollow" href="https://www.youtube.com/watch?v=vq2nnJ4g6N0">Tensorflow and deep learning – without at PhD by Martin Görner</a>. Second I started to look at publications and error improvements on public datasets.<br>
There is surprisingly little information about the improvement rate of machine learning on public datasets. I found one great <a target="_blank" rel="nofollow" href="http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html">resource</a> I would like to analyse in the following post.<br>
All datasets (“MINST”,”CIFAR-10”, “CIFAR-100”, “STL-10”, “SVHN”) are image classification tasks and results are published in academic (peer-reviewed) outlets.<br>
In order to better aggregate the results, I report the trimmed (10 percent) mean error rate per year per dataset.</p>
<p><img alt="plot of chunk unnamed-chunk-2" src="https://i1.wp.com/flovv.github.io/figures/post17/unnamed-chunk-2-1.png?w=456" data-recalc-dims="1"></p>
<p>We see that the mean reported error drops in all datasets per year. Each panel has it’s own x,y-scales, however inspected closely, we see that there is no apparent drop in the error rate in one particular year. Rather, it seems that the improvement rate per dataset is a linear function of the time.<br>
To get a better look at the best performer, let’s do the same plot with just the lowest reported error rates per year.</p>
<p><img alt="plot of chunk unnamed-chunk-3" src="https://i1.wp.com/flovv.github.io/figures/post17/unnamed-chunk-3-1.png?w=456" data-recalc-dims="1"></p>
<p>Again, there is not a single year that appears to mark the rise of machines but it looks like a continuous process.<br>
If it is a continuous process, let’s quickly summarise the learning rate per dataset.</p>
<table>
<thead>
<tr>
<th>Dataset</th>
<th>Improvement</th>
<th>Years</th>
<th>PP. Improvement per Year</th>
</tr>
</thead>
<tbody>
<tr>
<td>CIFAR-10</td>
<td>3%</td>
<td>6</td>
<td>0.4%</td>
</tr>
<tr>
<td>CIFAR-100</td>
<td>29%</td>
<td>5</td>
<td>5.9%</td>
</tr>
<tr>
<td>MINST</td>
<td>63%</td>
<td>13</td>
<td>4.8%</td>
</tr>
<tr>
<td>STL-10</td>
<td>36%</td>
<td>5</td>
<td>7.1%</td>
</tr>
<tr>
<td>SVHN</td>
<td>16%</td>
<td>6</td>
<td>2.6%</td>
</tr>
</tbody>
</table>
<p>The improvement column lists the percent improvement from the first year best publication to the current best publication. The dataset have been around for various timeframes (indicated in column 2). Finally we get the percentage point increase per year. While the improvement varies, across the board it seem that 5% improvement is reasonable.</p>
<p>So if there is not a single year that marks instant spike in improvement, what is the hype about? I assume that with the steady process in the recent years AI seems to approach or even surpass human-level performance on some tasks. Basically the news is not a technology breakthrough but rather a passing of an important threshold.</p>
<p>In case you want to have a look at the data yourself:<br>
</p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://flovv.github.io/MachineLearning_Improvement/"> Florian Teschner</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/_8WBw9Zki80"></a></div>