<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/teaching-to-machines-what-is-learning-in-machine-learning-entails/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://memosisland.blogspot.com/2017/11/teaching-to-machines-what-is-learning.html"> Memo's Island</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><b>Preamble</b><br><b><br></b></p>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i2.wp.com/upload.wikimedia.org/wikipedia/commons/f/fc/Laurentius_de_Voltolina_001.jpg?ssl=1" target="_blank"><img border="0" data-original-height="646" src="https://i2.wp.com/upload.wikimedia.org/wikipedia/commons/f/fc/Laurentius_de_Voltolina_001.jpg?resize=450%2C646&ssl=1" data-recalc-dims="1" data-original-width="450"></a></td>
</tr>
<tr>
<td>Figure 1: The oldest learning <span>institution </span><br><span> in the</span><span> world;</span><span> </span><a rel="nofollow" href="https://en.wikipedia.org/wiki/University_of_Bologna" target="_blank">University of Bologna</a><span>. </span><br><span>(Source: Wikipedia).</span></td>
</tr>
</tbody>
</table>
<div><a rel="nofollow" href="https://en.wikipedia.org/wiki/Machine_learning" target="_blank">Machine Learning (ML)</a> is now a de-facto skill for every quantitative job and almost every industry embraced it, even though fundamentals of the field is not new at all. However, <i>what does it mean to teach to a machine?</i> Unfortunately, for even moderate technical people coming from different backgrounds, answer to this question is not apparent in the first instance. This sounds like a conceptual and jargon issue, but it lies in the very success of <a rel="nofollow" href="https://en.wikipedia.org/wiki/Supervised_learning" target="_blank">supervised learning</a> algorithms.<br><b><br></b><b>What is a machine in machine learning</b>
<p>First of all here, <i>machine</i> does not mean a <a rel="nofollow" href="https://en.wikipedia.org/wiki/Machine" target="_blank">machine</a> in conventional sense, but computational modules or set of instructions. It is called <i>machine</i> because of <i>thermodynamics</i> can be applied to analyse this computational modules, their algorithmic complexity can be map to an energy consumption and work production, recall <a rel="nofollow" href="https://en.wikipedia.org/wiki/Landauer%27s_principle" target="_blank">Landauer Principle.</a> Charles Bennett has an article on this principle, <a rel="nofollow" href="http://www.cs.princeton.edu/courses/archive/fall06/cos576/papers/bennett03.pdf" target="_blank">here</a>.</p>
</div>
<p><b>Learning curve for machines</b><br><b><br></b> </p>
<div>What about <i>learning</i> bit? <i>Learning</i> is a natural process for humans, specially for the younger ones. We learn new things naturally without explicitly thinking with instruction and some training, such as practised in University of Bologna a millennia ago, see Figure 1. Usually,  it takes time for us to learn new thing, and the process is called <i><a rel="nofollow" href="https://en.wikipedia.org/wiki/Learning_curve" target="_blank">learning curve</a></i>, due to <a rel="nofollow" href="https://en.wikipedia.org/wiki/Hermann_Ebbinghaus" target="_blank">Hermann Ebinghauss</a>. Essentially, <i>learning</i> in machine learning exactly roots in Ebinghauss approach.  But question remains, how this manifests quantitatively in machine learning. Answer to this question is going to be given here.</div>
<p>The concept of <i>learning curve</i> and  effect of sample size in training machine learning models for both experienced and novice practitioners. It is often this type of analysis is omitted in producing supervised learning solutions. In the advent of deep learning architecture, multi-layered neural networks, this concept becomes more pronounced.</p>
<p>Quantifying learning curve lies in measuring <i>performance </i>over increasing <i>experience</i>. If the performance of the machine, or human, increases over experience we denote that learning is achieved. We distinguish good learner.<br><b><br>On the misconception of unsupervised learning</b></p>
<table cellspacing="0" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i1.wp.com/2.bp.blogspot.com/-uWp4IYJosBU/Wg5Zn8EOXzI/AAAAAAAAF5s/RTK6JaSki600TAY9zf9hGyCsJ4JaGtaegCLcBGAs/s1600/Donald_Hebb.gif?ssl=1" target="_blank"><img border="0" data-original-height="162" src="https://i1.wp.com/2.bp.blogspot.com/-uWp4IYJosBU/Wg5Zn8EOXzI/AAAAAAAAF5s/RTK6JaSki600TAY9zf9hGyCsJ4JaGtaegCLcBGAs/s1600/Donald_Hebb.gif?resize=126%2C162&ssl=1" data-recalc-dims="1" data-original-width="126"></a></td>
</tr>
<tr>
<td>Figure 2: <a rel="nofollow" href="https://en.wikipedia.org/wiki/Donald_O._Hebb" target="_blank">Donald Webb</a>, <br>father of unsupervised<br>learning. <br>(Source: Wikipedia)</td>
</tr>
</tbody>
</table>
<p>A generic misconception appear on what unsupervised learning means. Clustering, or categorising unlabelled data, is not learning in Ebbinghaus sense. <span>The goodness of fit or cluster validation exercise do not account an increase in experience, at least this </span><span>is not established in the literature, judging from cluster validation techniques, see, Jain-Dubes’s </span><span><a rel="nofollow" href="https://dl.acm.org/citation.cfm?id=46712" target="_blank">Algorithms for clustering data</a>. </span><span><span>Wikipedia defines unsupervised learning <i>“inferring a function to describe hidden structure from “unlabeled” data”</i>, this is </span>a function inference is not learning,.</span></p>
<p><span><span>Then, what is unsupervised learning? It originates from <a rel="nofollow" href="https://en.wikipedia.org/wiki/Hebbian_theory" target="_blank">Hebbian Theory</a> from neuroscience that <i>“Cells that fire together wire together”</i>. This implies, <i>unsupervised learning</i> is about how information is encoded, not how it is labelled. One good practical model that could be classified as unsupervised learning, so called </span><span><a rel="nofollow" href="https://en.wikipedia.org/wiki/Spiking_neural_network" target="_blank">spiking network model</a>.</span></span></p>
<p><b>Outlook</b></p>
<p>The concept of learning from Machine Learning perspective is summarised in this post. In data science, it is a good practice to differentiate what we call learning and function inference/optimisation. Being attentive to this details would help us.</p>
<p></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://memosisland.blogspot.com/2017/11/teaching-to-machines-what-is-learning.html"> Memo's Island</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/kS6188xszeM" height="1"></a></div>