<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/practical-machine-learning-with-r-and-python-part-1/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/"> R – Giga thoughts …</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h1>Introduction</h1>
<p>This is the 1st part of a series of posts I intend to write on some common Machine Learning Algorithms in R and Python. In this first part I cover the following Machine Learning Algorithms</p>
<ul>
<li>Univariate Regression</li>
<li>Multivariate Regression</li>
<li>Polynomial Regression</li>
<li>K Nearest Neighbors Regression</li>
</ul>
<p>The code includes the implementation in both R and Python. This series of posts are based on the following 2 MOOC courses I did at Stanford Online and at Coursera</p>
<ol>
<li><a rel="nofollow" href="https://lagunita.stanford.edu/courses/HumanitiesandScience/StatLearning/Winter2015/about" target="_blank">Statistical Learning</a>, Prof Trevor Hastie &amp; Prof Robert Tibesherani, Online Stanford</li>
<li><a rel="nofollow" href="https://www.coursera.org/learn/python-machine-learning" target="_blank">Applied Machine Learning in Python</a> Prof Kevyn-Collin Thomson, University Of Michigan, Coursera</li>
</ol>
<p>I have used the data sets from <a rel="nofollow" href="https://archive.ics.uci.edu/ml/datasets.html" target="_blank">UCI Machine Learning repository</a>(<a rel="nofollow" href="https://archive.ics.uci.edu/ml/datasets/Communities+and+Crime" target="_blank">Communities and Crime</a> and <a rel="nofollow" href="https://archive.ics.uci.edu/ml/datasets/Auto+MPG" target="_blank">Auto MPG</a>). I also use the Boston data set from MASS package</p>
<p>While coding in R and Python I found that there were some aspects that were more convenient in one language and some in the other. For example, plotting the fit in R is straightforward in R, while computing the R squared, splitting as Train &amp; Test sets etc. are already available in Python. In any case, these minor inconveniences can be easily be implemented in either language.</p>
<p>R squared computation in R is computed as follows<br>
<img alt="RSS=sum (y-yhat)^{2}" title="RSS=sum (y-yhat)^{2}" src="https://s0.wp.com/latex.php?latex=RSS%3D%5Csum+%28y-yhat%29%5E%7B2%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"><br>
<img alt="TSS= sum(y-mean(y))^{2}" title="TSS= sum(y-mean(y))^{2}" src="https://s0.wp.com/latex.php?latex=TSS%3D+%5Csum%28y-mean%28y%29%29%5E%7B2%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"><br>
<img alt="Rsquared- 1-frac{RSS}{TSS}" title="Rsquared- 1-frac{RSS}{TSS}" src="https://s0.wp.com/latex.php?latex=Rsquared-+1-%5Cfrac%7BRSS%7D%7BTSS%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p><strong>Note</strong>: You can download this R Markdown file and the associated data sets from Github at <a rel="nofollow" href="https://github.com/tvganesh/MachineLearning-RandPython" target="_blank">MachineLearning-RandPython</a><br>
<strong>Note 1</strong>: This post was created as an R Markdown file in RStudio which has a cool feature of including R and Python snippets. The plot of matplotlib needs a workaround but otherwise this is a real cool feature of RStudio!</p>
<h2>1.1a Univariate Regression – R code</h2>
<p>Here a simple linear regression line is fitted between a single input feature and the target variable</p>
<pre><code><span># Source in the R function library</span>
<span>source</span><span>(</span><span>&quot;RFunctions.R&quot;</span><span>)</span></code></pre>
<pre><code><span># Read the Boston data file</span>
<span>df</span><span>=</span><span>read.csv</span><span>(</span><span>&quot;Boston.csv&quot;</span>,<span>stringsAsFactors</span> <span>=</span> <span>FALSE</span><span>)</span> <span># Data from MASS - Statistical Learning</span>

<span># Split the data into training and test sets (75:25)</span>
<span>train_idx</span> <span>&lt;-</span> <span>trainTestSplit</span><span>(</span><span>df</span>,<span>trainPercent</span><span>=</span><span>75</span>,<span>seed</span><span>=</span><span>5</span><span>)</span>
<span>train</span> <span>&lt;-</span> <span>df</span><span>[</span><span>train_idx</span>, <span>]</span>
<span>test</span> <span>&lt;-</span> <span>df</span><span>[</span><span>-</span><span>train_idx</span>, <span>]</span>

<span># Fit a linear regression line between 'Median value of owner occupied homes' vs 'lower status of </span>
<span># population'</span>
<span>fit</span><span>=</span><span>lm</span><span>(</span><span>medv</span><span>~</span><span>lstat</span>,<span>data</span><span>=</span><span>df</span><span>)</span>
<span># Display details of fir</span>
<span>summary</span><span>(</span><span>fit</span><span>)</span></code></pre>
<pre><code>## 
## Call:
## lm(formula = medv ~ lstat, data = df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -15.168  -3.990  -1.318   2.034  24.500 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 34.55384    0.56263   61.41   &lt;2e-16 ***
## lstat       -0.95005    0.03873  -24.53   &lt;2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 6.216 on 504 degrees of freedom
## Multiple R-squared:  0.5441, Adjusted R-squared:  0.5432 
## F-statistic: 601.6 on 1 and 504 DF,  p-value: &lt; 2.2e-16</code></pre>
<pre><code><span># Display the confidence intervals</span>
<span>confint</span><span>(</span><span>fit</span><span>)</span></code></pre>
<pre><code>##                 2.5 %     97.5 %
## (Intercept) 33.448457 35.6592247
## lstat       -1.026148 -0.8739505</code></pre>
<pre><code><span>plot</span><span>(</span><span>df</span><span>$</span><span>lstat</span>,<span>df</span><span>$</span><span>medv</span>, <span>xlab</span><span>=</span><span>&quot;Lower status (%)&quot;</span>,<span>ylab</span><span>=</span><span>&quot;Median value of owned homes ($1000)&quot;</span>, <span>main</span><span>=</span><span>&quot;Median value of homes ($1000) vs Lowe status (%)&quot;</span><span>)</span>
<span>abline</span><span>(</span><span>fit</span><span>)</span>
<span>abline</span><span>(</span><span>fit</span>,<span>lwd</span><span>=</span><span>3</span><span>)</span>
<span>abline</span><span>(</span><span>fit</span>,<span>lwd</span><span>=</span><span>3</span>,<span>col</span><span>=</span><span>&quot;red&quot;</span><span>)</span></code></pre>
<p><img><img data-orig-file="https://gigadom.files.wordpress.com/2017/10/a-11.png" data-orig-size="1344,960" data-medium-file="https://gigadom.files.wordpress.com/2017/10/a-11.png?w=300" data-attachment-id="5613" data-image-meta="{" data-image-description="" src="https://gigadom.files.wordpress.com/2017/10/a-11.png?w=450&h=483&fit=676%2C483" data-permalink="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/a-1-4/" data-large-file="https://gigadom.files.wordpress.com/2017/10/a-11.png?w=450&amp;h=483&amp;fit=676%2C483" alt="" data-image-title="a-1" sizes="(max-width: 676px) 100vw, 676px" data-comments-opened="1" class="wp-image-5613" data-recalc-dims="1"></p>
<pre><code><span>rsquared</span><span>=</span><span>Rsquared</span><span>(</span><span>fit</span>,<span>test</span>,<span>test</span><span>$</span><span>medv</span><span>)</span>
<span>sprintf</span><span>(</span><span>&quot;R-squared for uni-variate regression (Boston.csv) is : %f&quot;</span>, <span>rsquared</span><span>)</span></code></pre>
<pre><code>## [1] &quot;R-squared for uni-variate regression (Boston.csv)  is : 0.556964&quot;</code></pre>
<div>
<h2>1.1b Univariate Regression – Python code</h2>
<pre><code><span>import</span> numpy <span>as</span> np
<span>import</span> pandas <span>as</span> pd
<span>import</span> os
<span>import</span> matplotlib.pyplot <span>as</span> plt
<span>from</span> sklearn.model_selection <span>import</span> train_test_split
<span>from</span> sklearn.linear_model <span>import</span> LinearRegression
<span>#os.chdir(&quot;C:\\software\\machine-learning\\RandPython&quot;)</span>

<span># Read the CSV file</span>
df = pd.read_csv(<span>&quot;Boston.csv&quot;</span>,encoding = <span>&quot;ISO-8859-1&quot;</span>)
<span># Select the feature variable</span>
X=df[<span>'lstat'</span>]

<span># Select the target </span>
y=df[<span>'medv'</span>]

<span># Split into train and test sets (75:25)</span>
X_train, X_test, y_train, y_test = train_test_split(X, y,random_state = <span>0</span>)
X_train=X_train.values.reshape(-<span>1</span>,<span>1</span>)
X_test=X_test.values.reshape(-<span>1</span>,<span>1</span>)

<span># Fit a linear model</span>
linreg = LinearRegression().fit(X_train, y_train)

<span># Print the training and test R squared score</span>
<span>print</span>(<span>'R-squared score (training): {:.3f}'</span>.format(linreg.score(X_train, y_train)))
<span>print</span>(<span>'R-squared score (test): {:.3f}'</span>.format(linreg.score(X_test, y_test)))
     
<span># Plot the linear regression line</span>
fig=plt.scatter(X_train,y_train)

<span># Create a range of points. Compute yhat=coeff1*x + intercept and plot</span>
x=np.linspace(<span>0</span>,<span>40</span>,<span>20</span>)
fig1=plt.plot(x, linreg.coef_ * x + linreg.intercept_, color=<span>'red'</span>)
fig1=plt.title(<span>&quot;Median value of homes ($1000) vs Lowe status (%)&quot;</span>)
fig1=plt.xlabel(<span>&quot;Lower status (%)&quot;</span>)
fig1=plt.ylabel(<span>&quot;Median value of owned homes ($1000)&quot;</span>)
fig.figure.savefig(<span>'foo.png'</span>, bbox_inches=<span>'tight'</span>)
fig1.figure.savefig(<span>'foo1.png'</span>, bbox_inches=<span>'tight'</span>)
<span>print</span> <span>&quot;Finished&quot;</span>
</code></pre>
<pre><code>## R-squared score (training): 0.571
## R-squared score (test): 0.458
## Finished</code></pre>
<p><img data-orig-file="https://gigadom.files.wordpress.com/2017/10/foo11.png?w=456" data-orig-size="566,455" data-medium-file="https://gigadom.files.wordpress.com/2017/10/foo11.png?w=456?w=300" data-attachment-id="5614" data-image-meta="{" data-image-description="" src="https://gigadom.files.wordpress.com/2017/10/foo11.png?w=456" data-permalink="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/foo1-2/" data-large-file="https://gigadom.files.wordpress.com/2017/10/foo11.png?w=456?w=566" alt="" data-image-title="foo1" sizes="(max-width: 566px) 100vw, 566px" data-comments-opened="1" class="wp-image-5614" data-recalc-dims="1"></p>
<div>
<h2>1.2a Multivariate Regression – R code</h2>
<pre><code><span># Read crimes data</span>
<span>crimesDF</span> <span>&lt;-</span> <span>read.csv</span><span>(</span><span>&quot;crimes.csv&quot;</span>,<span>stringsAsFactors</span> <span>=</span> <span>FALSE</span><span>)</span>

<span># Remove the 1st 7 columns which do not impact output</span>
<span>crimesDF1</span> <span>&lt;-</span> <span>crimesDF</span><span>[</span>,<span>7</span><span>:</span><span>length</span><span>(</span><span>crimesDF</span><span>)</span><span>]</span>

<span># Convert all to numeric</span>
<span>crimesDF2</span> <span>&lt;-</span> <span>sapply</span><span>(</span><span>crimesDF1</span>,<span>as.numeric</span><span>)</span>

<span># Check for NAs</span>
<span>a</span> <span>&lt;-</span> <span>is.na</span><span>(</span><span>crimesDF2</span><span>)</span>
<span># Set to 0 as an imputation</span>
<span>crimesDF2</span><span>[</span><span>a</span><span>]</span> <span>&lt;-</span><span>0</span>
<span>#Create as a dataframe</span>
<span>crimesDF2</span> <span>&lt;-</span> <span>as.data.frame</span><span>(</span><span>crimesDF2</span><span>)</span>
<span>#Create a train/test split</span>
<span>train_idx</span> <span>&lt;-</span> <span>trainTestSplit</span><span>(</span><span>crimesDF2</span>,<span>trainPercent</span><span>=</span><span>75</span>,<span>seed</span><span>=</span><span>5</span><span>)</span>
<span>train</span> <span>&lt;-</span> <span>crimesDF2</span><span>[</span><span>train_idx</span>, <span>]</span>
<span>test</span> <span>&lt;-</span> <span>crimesDF2</span><span>[</span><span>-</span><span>train_idx</span>, <span>]</span>

<span># Fit a multivariate regression model between crimesPerPop and all other features</span>
<span>fit</span> <span>&lt;-</span> <span>lm</span><span>(</span><span>ViolentCrimesPerPop</span><span>~</span>.,<span>data</span><span>=</span><span>train</span><span>)</span>

<span># Compute and print R Squared</span>
<span>rsquared</span><span>=</span><span>Rsquared</span><span>(</span><span>fit</span>,<span>test</span>,<span>test</span><span>$</span><span>ViolentCrimesPerPop</span><span>)</span>
<span>sprintf</span><span>(</span><span>&quot;R-squared for multi-variate regression (crimes.csv) is : %f&quot;</span>, <span>rsquared</span><span>)</span></code></pre>
<pre><code>## [1] &quot;R-squared for multi-variate regression (crimes.csv)  is : 0.653940&quot;</code></pre>
</div>
<div>
<h2>1.2b Multivariate Regression – Python code</h2>
<pre><code><span>import</span> numpy <span>as</span> np
<span>import</span> pandas <span>as</span> pd
<span>import</span> os
<span>import</span> matplotlib.pyplot <span>as</span> plt
<span>from</span> sklearn.model_selection <span>import</span> train_test_split
<span>from</span> sklearn.linear_model <span>import</span> LinearRegression
<span># Read the data</span>
crimesDF =pd.read_csv(<span>&quot;crimes.csv&quot;</span>,encoding=<span>&quot;ISO-8859-1&quot;</span>)
<span>#Remove the 1st 7 columns</span>
crimesDF1=crimesDF.iloc[:,<span>7</span>:crimesDF.shape[<span>1</span>]]
<span># Convert to numeric</span>
crimesDF2 = crimesDF1.apply(pd.to_numeric, errors=<span>'coerce'</span>)
<span># Impute NA to 0s</span>
crimesDF2.fillna(<span>0</span>, inplace=<span>True</span>)

<span># Select the X (feature vatiables - all)</span>
X=crimesDF2.iloc[:,<span>0</span>:<span>120</span>]

<span># Set the target</span>
y=crimesDF2.iloc[:,<span>121</span>]

X_train, X_test, y_train, y_test = train_test_split(X, y,random_state = <span>0</span>)
<span># Fit a multivariate regression model</span>
linreg = LinearRegression().fit(X_train, y_train)

<span># compute and print the R Square</span>
<span>print</span>(<span>'R-squared score (training): {:.3f}'</span>.format(linreg.score(X_train, y_train)))
<span>print</span>(<span>'R-squared score (test): {:.3f}'</span>.format(linreg.score(X_test, y_test)))</code></pre>
<pre><code>## R-squared score (training): 0.699
## R-squared score (test): 0.677</code></pre>
</div>
<div>
<h2>1.3a Polynomial Regression – R</h2>
<p>For Polynomial regression , polynomials of degree 1,2 &amp; 3 are used and R squared is computed. It can be seen that the quadaratic model provides the best R squared score and hence the best fit</p>
<pre><code> <span># Polynomial degree 1</span>
<span>df</span><span>=</span><span>read.csv</span><span>(</span><span>&quot;auto_mpg.csv&quot;</span>,<span>stringsAsFactors</span> <span>=</span> <span>FALSE</span><span>)</span> <span># Data from UCI</span>
<span>df1</span> <span>&lt;-</span> <span>as.data.frame</span><span>(</span><span>sapply</span><span>(</span><span>df</span>,<span>as.numeric</span><span>)</span><span>)</span>

<span># Select key columns</span>
<span>df2</span> <span>&lt;-</span> <span>df1</span> <span>%>%</span> <span>select</span><span>(</span><span>cylinder</span>,<span>displacement</span>, <span>horsepower</span>,<span>weight</span>, <span>acceleration</span>, <span>year</span>,<span>mpg</span><span>)</span>
<span>df3</span> <span>&lt;-</span> <span>df2</span><span>[</span><span>complete.cases</span><span>(</span><span>df2</span><span>)</span>,<span>]</span>

<span># Split as train and test sets</span>
<span>train_idx</span> <span>&lt;-</span> <span>trainTestSplit</span><span>(</span><span>df3</span>,<span>trainPercent</span><span>=</span><span>75</span>,<span>seed</span><span>=</span><span>5</span><span>)</span>
<span>train</span> <span>&lt;-</span> <span>df3</span><span>[</span><span>train_idx</span>, <span>]</span>
<span>test</span> <span>&lt;-</span> <span>df3</span><span>[</span><span>-</span><span>train_idx</span>, <span>]</span>

<span># Fit a model of degree 1</span>
<span>fit</span> <span>&lt;-</span> <span>lm</span><span>(</span><span>mpg</span><span>~</span>. ,<span>data</span><span>=</span><span>train</span><span>)</span>
<span>rsquared1</span> <span>&lt;-</span><span>Rsquared</span><span>(</span><span>fit</span>,<span>test</span>,<span>test</span><span>$</span><span>mpg</span><span>)</span>
<span>sprintf</span><span>(</span><span>&quot;R-squared for Polynomial regression of degree 1 (auto_mpg.csv) is : %f&quot;</span>, <span>rsquared1</span><span>)</span></code></pre>
<pre><code>## [1] &quot;R-squared for Polynomial regression of degree 1 (auto_mpg.csv)  is : 0.763607&quot;</code></pre>
<pre><code><span># Polynomial degree 2 - Quadratic</span>
<span>x</span> <span>=</span> <span>as.matrix</span><span>(</span><span>df3</span><span>[</span><span>1</span><span>:</span><span>6</span><span>]</span><span>)</span>
<span># Make a polynomial of degree 2 for feature variables before split</span>
<span>df4</span><span>=</span><span>as.data.frame</span><span>(</span><span>poly</span><span>(</span><span>x</span>,<span>2</span>,<span>raw</span><span>=</span><span>TRUE</span><span>)</span><span>)</span>
<span>df5</span> <span>&lt;-</span> <span>cbind</span><span>(</span><span>df4</span>,<span>df3</span><span>[</span><span>7</span><span>]</span><span>)</span>

<span># Split into train and test set</span>
<span>train_idx</span> <span>&lt;-</span> <span>trainTestSplit</span><span>(</span><span>df5</span>,<span>trainPercent</span><span>=</span><span>75</span>,<span>seed</span><span>=</span><span>5</span><span>)</span>
<span>train</span> <span>&lt;-</span> <span>df5</span><span>[</span><span>train_idx</span>, <span>]</span>
<span>test</span> <span>&lt;-</span> <span>df5</span><span>[</span><span>-</span><span>train_idx</span>, <span>]</span>

<span># Fit the quadratic model</span>
<span>fit</span> <span>&lt;-</span> <span>lm</span><span>(</span><span>mpg</span><span>~</span>. ,<span>data</span><span>=</span><span>train</span><span>)</span>
<span># Compute R squared</span>
<span>rsquared2</span><span>=</span><span>Rsquared</span><span>(</span><span>fit</span>,<span>test</span>,<span>test</span><span>$</span><span>mpg</span><span>)</span>
<span>sprintf</span><span>(</span><span>&quot;R-squared for Polynomial regression of degree 2 (auto_mpg.csv) is : %f&quot;</span>, <span>rsquared2</span><span>)</span></code></pre>
<pre><code>## [1] &quot;R-squared for Polynomial regression of degree 2 (auto_mpg.csv)  is : 0.831372&quot;</code></pre>
<pre><code><span>#Polynomial degree 3</span>
<span>x</span> <span>=</span> <span>as.matrix</span><span>(</span><span>df3</span><span>[</span><span>1</span><span>:</span><span>6</span><span>]</span><span>)</span>
<span># Make polynomial of degree 4 of feature variables before split</span>
<span>df4</span><span>=</span><span>as.data.frame</span><span>(</span><span>poly</span><span>(</span><span>x</span>,<span>3</span>,<span>raw</span><span>=</span><span>TRUE</span><span>)</span><span>)</span>
<span>df5</span> <span>&lt;-</span> <span>cbind</span><span>(</span><span>df4</span>,<span>df3</span><span>[</span><span>7</span><span>]</span><span>)</span>
<span>train_idx</span> <span>&lt;-</span> <span>trainTestSplit</span><span>(</span><span>df5</span>,<span>trainPercent</span><span>=</span><span>75</span>,<span>seed</span><span>=</span><span>5</span><span>)</span>

<span>train</span> <span>&lt;-</span> <span>df5</span><span>[</span><span>train_idx</span>, <span>]</span>
<span>test</span> <span>&lt;-</span> <span>df5</span><span>[</span><span>-</span><span>train_idx</span>, <span>]</span>
<span># Fit a model of degree 3</span>
<span>fit</span> <span>&lt;-</span> <span>lm</span><span>(</span><span>mpg</span><span>~</span>. ,<span>data</span><span>=</span><span>train</span><span>)</span>
<span># Compute R squared</span>
<span>rsquared3</span><span>=</span><span>Rsquared</span><span>(</span><span>fit</span>,<span>test</span>,<span>test</span><span>$</span><span>mpg</span><span>)</span>
<span>sprintf</span><span>(</span><span>&quot;R-squared for Polynomial regression of degree 2 (auto_mpg.csv) is : %f&quot;</span>, <span>rsquared3</span><span>)</span></code></pre>
<pre><code>## [1] &quot;R-squared for Polynomial regression of degree 2 (auto_mpg.csv)  is : 0.773225&quot;</code></pre>
<pre><code><span>df</span><span>=</span><span>data.frame</span><span>(</span><span>degree</span><span>=</span><span>c</span><span>(</span><span>1</span>,<span>2</span>,<span>3</span><span>)</span>,<span>Rsquared</span><span>=</span><span>c</span><span>(</span><span>rsquared1</span>,<span>rsquared2</span>,<span>rsquared3</span><span>)</span><span>)</span>
<span># Make a plot of Rsquared and degree</span>
<span>ggplot</span><span>(</span><span>df</span>,<span>aes</span><span>(</span><span>x</span><span>=</span><span>degree</span>,<span>y</span><span>=</span><span>Rsquared</span><span>)</span><span>)</span> <span>+</span><span>geom_point</span><span>(</span><span>)</span> <span>+</span> <span>geom_line</span><span>(</span><span>color</span><span>=</span><span>&quot;blue&quot;</span><span>)</span> <span>+</span>
    <span>ggtitle</span><span>(</span><span>&quot;Polynomial regression - R squared vs Degree of polynomial&quot;</span><span>)</span> <span>+</span>
    <span>xlab</span><span>(</span><span>&quot;Degree&quot;</span><span>)</span> <span>+</span> <span>ylab</span><span>(</span><span>&quot;R squared&quot;</span><span>)</span></code></pre>
<p><img><img data-orig-file="https://gigadom.files.wordpress.com/2017/10/b-1.png" data-orig-size="1344,960" data-medium-file="https://gigadom.files.wordpress.com/2017/10/b-1.png?w=300" data-attachment-id="5616" data-image-meta="{" data-image-description="" src="https://gigadom.files.wordpress.com/2017/10/b-1.png?w=450&h=483&fit=676%2C483" data-permalink="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/b-1/" data-large-file="https://gigadom.files.wordpress.com/2017/10/b-1.png?w=450&amp;h=483&amp;fit=676%2C483" alt="" data-image-title="b -1" sizes="(max-width: 676px) 100vw, 676px" data-comments-opened="1" class="wp-image-5616" data-recalc-dims="1"></p>
</div>
<div>
<h2>1.3a Polynomial Regression – Python</h2>
<p>For Polynomial regression , polynomials of degree 1,2 &amp; 3 are used and R squared is computed. It can be seen that the quadaratic model provides the best R squared score and hence the best fit</p>
<pre><code><span>import</span> numpy <span>as</span> np
<span>import</span> pandas <span>as</span> pd
<span>import</span> os
<span>import</span> matplotlib.pyplot <span>as</span> plt
<span>from</span> sklearn.model_selection <span>import</span> train_test_split
<span>from</span> sklearn.linear_model <span>import</span> LinearRegression
<span>from</span> sklearn.preprocessing <span>import</span> PolynomialFeatures
autoDF =pd.read_csv(<span>&quot;auto_mpg.csv&quot;</span>,encoding=<span>&quot;ISO-8859-1&quot;</span>)
autoDF.shape
autoDF.columns
<span># Select key columns</span>
autoDF1=autoDF[[<span>'mpg'</span>,<span>'cylinder'</span>,<span>'displacement'</span>,<span>'horsepower'</span>,<span>'weight'</span>,<span>'acceleration'</span>,<span>'year'</span>]]
<span># Convert columns to numeric</span>
autoDF2 = autoDF1.apply(pd.to_numeric, errors=<span>'coerce'</span>)
<span># Drop NAs</span>
autoDF3=autoDF2.dropna()
autoDF3.shape
X=autoDF3[[<span>'cylinder'</span>,<span>'displacement'</span>,<span>'horsepower'</span>,<span>'weight'</span>,<span>'acceleration'</span>,<span>'year'</span>]]
y=autoDF3[<span>'mpg'</span>]

<span># Polynomial degree 1</span>
X_train, X_test, y_train, y_test = train_test_split(X, y,random_state = <span>0</span>)
linreg = LinearRegression().fit(X_train, y_train)
<span>print</span>(<span>'R-squared score - Polynomial degree 1 (training): {:.3f}'</span>.format(linreg.score(X_train, y_train)))
<span># Compute R squared </span>
rsquared1 =linreg.score(X_test, y_test)
<span>print</span>(<span>'R-squared score - Polynomial degree 1 (test): {:.3f}'</span>.format(linreg.score(X_test, y_test)))

<span># Polynomial degree 2</span>
poly = PolynomialFeatures(degree=<span>2</span>)
X_poly = poly.fit_transform(X)
X_train, X_test, y_train, y_test = train_test_split(X_poly, y,random_state = <span>0</span>)
linreg = LinearRegression().fit(X_train, y_train)

<span># Compute R squared</span>
<span>print</span>(<span>'R-squared score - Polynomial degree 2 (training): {:.3f}'</span>.format(linreg.score(X_train, y_train)))
rsquared2 =linreg.score(X_test, y_test)
<span>print</span>(<span>'R-squared score - Polynomial degree 2 (test): {:.3f}\n'</span>.format(linreg.score(X_test, y_test)))

<span>#Polynomial degree 3</span>

poly = PolynomialFeatures(degree=<span>3</span>)
X_poly = poly.fit_transform(X)
X_train, X_test, y_train, y_test = train_test_split(X_poly, y,random_state = <span>0</span>)
linreg = LinearRegression().fit(X_train, y_train)
<span>print</span>(<span>'(R-squared score -Polynomial degree 3 (training): {:.3f}'</span>
     .format(linreg.score(X_train, y_train)))
<span># Compute R squared </span>
rsquared3 =linreg.score(X_test, y_test)
<span>print</span>(<span>'R-squared score Polynomial degree 3 (test): {:.3f}\n'</span>.format(linreg.score(X_test, y_test)))
degree=[<span>1</span>,<span>2</span>,<span>3</span>]
rsquared =[rsquared1,rsquared2,rsquared3]
fig2=plt.plot(degree,rsquared)
fig2=plt.title(<span>&quot;Polynomial regression - R squared vs Degree of polynomial&quot;</span>)
fig2=plt.xlabel(<span>&quot;Degree&quot;</span>)
fig2=plt.ylabel(<span>&quot;R squared&quot;</span>)
fig2.figure.savefig(<span>'foo2.png'</span>, bbox_inches=<span>'tight'</span>)
<span>print</span> <span>&quot;Finished plotting and saving&quot;</span>
</code></pre>
<pre><code>## R-squared score - Polynomial degree 1 (training): 0.811
## R-squared score - Polynomial degree 1 (test): 0.799
## R-squared score - Polynomial degree 2 (training): 0.861
## R-squared score - Polynomial degree 2 (test): 0.847
## 
## (R-squared score -Polynomial degree 3  (training): 0.933
## R-squared score Polynomial degree 3 (test): 0.710
## 
## Finished plotting and saving</code></pre>
<p><img data-orig-file="https://gigadom.files.wordpress.com/2017/10/foo2.png?w=456" data-orig-size="579,455" data-medium-file="https://gigadom.files.wordpress.com/2017/10/foo2.png?w=456?w=300" data-attachment-id="5617" data-image-meta="{" data-image-description="" src="https://gigadom.files.wordpress.com/2017/10/foo2.png?w=456" data-permalink="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/foo2/" data-large-file="https://gigadom.files.wordpress.com/2017/10/foo2.png?w=456?w=579" alt="" data-image-title="foo2" sizes="(max-width: 579px) 100vw, 579px" data-comments-opened="1" class="wp-image-5617" data-recalc-dims="1"></p>
</div>
<div>
<h2>1.4 K Nearest Neighbors</h2>
<p>The code below implements KNN Regression both for R and Python. This is done for different neighbors. The R squared is computed in each case. This is repeated after performing feature scaling. It can be seen the model fit is much better after feature scaling. Normalization refers to</p>
<p><img alt="X_{normalized} = frac{X-min(X)}{max(X-min(X))}" title="X_{normalized} = frac{X-min(X)}{max(X-min(X))}" src="https://s0.wp.com/latex.php?latex=X_%7Bnormalized%7D+%3D+%5Cfrac%7BX-min%28X%29%7D%7Bmax%28X-min%28X%29%29%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p>Another technique that is used is Standardization which is</p>
<p><img alt="X_{standardized} = frac{X-mean(X)}{sd(X)}" title="X_{standardized} = frac{X-mean(X)}{sd(X)}" src="https://s0.wp.com/latex.php?latex=X_%7Bstandardized%7D+%3D+%5Cfrac%7BX-mean%28X%29%7D%7Bsd%28X%29%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
</div>
<div>
<h2>1.4a K Nearest Neighbors Regression – R( Unnormalized)</h2>
<p>The R code below does not use feature scaling</p>
<pre><code><span># KNN regression requires the FNN package</span>
<span>df</span><span>=</span><span>read.csv</span><span>(</span><span>&quot;auto_mpg.csv&quot;</span>,<span>stringsAsFactors</span> <span>=</span> <span>FALSE</span><span>)</span> <span># Data from UCI</span>
<span>df1</span> <span>&lt;-</span> <span>as.data.frame</span><span>(</span><span>sapply</span><span>(</span><span>df</span>,<span>as.numeric</span><span>)</span><span>)</span></code></pre>
<pre><code><span>df2</span> <span>&lt;-</span> <span>df1</span> <span>%>%</span> <span>select</span><span>(</span><span>cylinder</span>,<span>displacement</span>, <span>horsepower</span>,<span>weight</span>, <span>acceleration</span>, <span>year</span>,<span>mpg</span><span>)</span>
<span>df3</span> <span>&lt;-</span> <span>df2</span><span>[</span><span>complete.cases</span><span>(</span><span>df2</span><span>)</span>,<span>]</span>

<span># Split train and test</span>
<span>train_idx</span> <span>&lt;-</span> <span>trainTestSplit</span><span>(</span><span>df3</span>,<span>trainPercent</span><span>=</span><span>75</span>,<span>seed</span><span>=</span><span>5</span><span>)</span>
<span>train</span> <span>&lt;-</span> <span>df3</span><span>[</span><span>train_idx</span>, <span>]</span>
<span>test</span> <span>&lt;-</span> <span>df3</span><span>[</span><span>-</span><span>train_idx</span>, <span>]</span>
<span># Select the feature variables</span>
<span>train.X</span><span>=</span><span>train</span><span>[</span>,<span>1</span><span>:</span><span>6</span><span>]</span>
<span># Set the target for training</span>
<span>train.Y</span><span>=</span><span>train</span><span>[</span>,<span>7</span><span>]</span>
<span># Do the same for test set</span>
<span>test.X</span><span>=</span><span>test</span><span>[</span>,<span>1</span><span>:</span><span>6</span><span>]</span>
<span>test.Y</span><span>=</span><span>test</span><span>[</span>,<span>7</span><span>]</span>

<span>rsquared</span> <span>&lt;-</span> <span>NULL</span>
<span># Create a list of neighbors</span>
<span>neighbors</span> <span>&lt;-</span><span>c</span><span>(</span><span>1</span>,<span>2</span>,<span>4</span>,<span>8</span>,<span>10</span>,<span>14</span><span>)</span>
<span>for</span><span>(</span><span>i</span> <span>in</span> <span>seq_along</span><span>(</span><span>neighbors</span><span>)</span><span>)</span><span>{</span>
    <span># Perform a KNN regression fit</span>
    <span>knn</span><span>=</span><span>knn.reg</span><span>(</span><span>train.X</span>,<span>test.X</span>,<span>train.Y</span>,<span>k</span><span>=</span><span>neighbors</span><span>[</span><span>i</span><span>]</span><span>)</span>
    <span># Compute R sqaured</span>
    <span>rsquared</span><span>[</span><span>i</span><span>]</span><span>=</span><span>knnRSquared</span><span>(</span><span>knn</span><span>$</span><span>pred</span>,<span>test.Y</span><span>)</span>
<span>}</span>

<span># Make a dataframe for plotting</span>
<span>df</span> <span>&lt;-</span> <span>data.frame</span><span>(</span><span>neighbors</span>,<span>Rsquared</span><span>=</span><span>rsquared</span><span>)</span>
<span># Plot the number of neighors vs the R squared</span>
<span>ggplot</span><span>(</span><span>df</span>,<span>aes</span><span>(</span><span>x</span><span>=</span><span>neighbors</span>,<span>y</span><span>=</span><span>Rsquared</span><span>)</span><span>)</span> <span>+</span> <span>geom_point</span><span>(</span><span>)</span> <span>+</span><span>geom_line</span><span>(</span><span>color</span><span>=</span><span>&quot;blue&quot;</span><span>)</span> <span>+</span>
    <span>xlab</span><span>(</span><span>&quot;Number of neighbors&quot;</span><span>)</span> <span>+</span> <span>ylab</span><span>(</span><span>&quot;R squared&quot;</span><span>)</span> <span>+</span>
    <span>ggtitle</span><span>(</span><span>&quot;KNN regression - R squared vs Number of Neighors (Unnormalized)&quot;</span><span>)</span></code></pre>
<p><img><img data-orig-file="https://gigadom.files.wordpress.com/2017/10/c-1.png" data-orig-size="1344,960" data-medium-file="https://gigadom.files.wordpress.com/2017/10/c-1.png?w=300" data-attachment-id="5618" data-image-meta="{" data-image-description="" src="https://gigadom.files.wordpress.com/2017/10/c-1.png?w=450&h=483&fit=676%2C483" data-permalink="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/c-1/" data-large-file="https://gigadom.files.wordpress.com/2017/10/c-1.png?w=450&amp;h=483&amp;fit=676%2C483" alt="" data-image-title="c-1" sizes="(max-width: 676px) 100vw, 676px" data-comments-opened="1" class="wp-image-5618" data-recalc-dims="1"></p>
</div>
<div>
<h2>1.4b K Nearest Neighbors Regression – Python( Unnormalized)</h2>
<p>The Python code below does not use feature scaling</p>
<pre><code><span>import</span> numpy <span>as</span> np
<span>import</span> pandas <span>as</span> pd
<span>import</span> os
<span>import</span> matplotlib.pyplot <span>as</span> plt
<span>from</span> sklearn.model_selection <span>import</span> train_test_split
<span>from</span> sklearn.linear_model <span>import</span> LinearRegression
<span>from</span> sklearn.preprocessing <span>import</span> PolynomialFeatures
<span>from</span> sklearn.neighbors <span>import</span> KNeighborsRegressor
autoDF =pd.read_csv(<span>&quot;auto_mpg.csv&quot;</span>,encoding=<span>&quot;ISO-8859-1&quot;</span>)
autoDF.shape
autoDF.columns
autoDF1=autoDF[[<span>'mpg'</span>,<span>'cylinder'</span>,<span>'displacement'</span>,<span>'horsepower'</span>,<span>'weight'</span>,<span>'acceleration'</span>,<span>'year'</span>]]
autoDF2 = autoDF1.apply(pd.to_numeric, errors=<span>'coerce'</span>)
autoDF3=autoDF2.dropna()
autoDF3.shape
X=autoDF3[[<span>'cylinder'</span>,<span>'displacement'</span>,<span>'horsepower'</span>,<span>'weight'</span>,<span>'acceleration'</span>,<span>'year'</span>]]
y=autoDF3[<span>'mpg'</span>]

<span># Perform a train/test split</span>
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = <span>0</span>)
<span># Create a list of neighbors</span>
rsquared=[]
neighbors=[<span>1</span>,<span>2</span>,<span>4</span>,<span>8</span>,<span>10</span>,<span>14</span>]
<span>for</span> i <span>in</span> neighbors:
        <span># Fit a KNN model</span>
        knnreg = KNeighborsRegressor(n_neighbors = i).fit(X_train, y_train)
        <span># Compute R squared</span>
        rsquared.append(knnreg.score(X_test, y_test))
        <span>print</span>(<span>'R-squared test score: {:.3f}'</span>
        .format(knnreg.score(X_test, y_test)))
<span># Plot the number of neighors vs the R squared </span>
fig3=plt.plot(neighbors,rsquared)
fig3=plt.title(<span>&quot;KNN regression - R squared vs Number of neighbors(Unnormalized)&quot;</span>)
fig3=plt.xlabel(<span>&quot;Neighbors&quot;</span>)
fig3=plt.ylabel(<span>&quot;R squared&quot;</span>)
fig3.figure.savefig(<span>'foo3.png'</span>, bbox_inches=<span>'tight'</span>)
<span>print</span> <span>&quot;Finished plotting and saving&quot;</span></code></pre>
<pre><code>## R-squared test score: 0.527
## R-squared test score: 0.678
## R-squared test score: 0.707
## R-squared test score: 0.684
## R-squared test score: 0.683
## R-squared test score: 0.670
## Finished plotting and saving</code></pre>
<h2><img data-orig-file="https://gigadom.files.wordpress.com/2017/10/foo3.png?w=456" data-orig-size="618,455" data-medium-file="https://gigadom.files.wordpress.com/2017/10/foo3.png?w=456?w=300" data-attachment-id="5620" data-image-meta="{" data-image-description="" src="https://gigadom.files.wordpress.com/2017/10/foo3.png?w=456" data-permalink="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/foo3/" data-large-file="https://gigadom.files.wordpress.com/2017/10/foo3.png?w=456?w=618" alt="" data-image-title="foo3" sizes="(max-width: 618px) 100vw, 618px" data-comments-opened="1" class="wp-image-5620" data-recalc-dims="1"></h2>
<h2>1.4c K Nearest Neighbors Regression – R( Normalized)</h2>
<p>It can be seen that R squared improves when the features are normalized.</p>
<pre><code><span>df</span><span>=</span><span>read.csv</span><span>(</span><span>&quot;auto_mpg.csv&quot;</span>,<span>stringsAsFactors</span> <span>=</span> <span>FALSE</span><span>)</span> <span># Data from UCI</span>
<span>df1</span> <span>&lt;-</span> <span>as.data.frame</span><span>(</span><span>sapply</span><span>(</span><span>df</span>,<span>as.numeric</span><span>)</span><span>)</span></code></pre>
<pre><code><span>df2</span> <span>&lt;-</span> <span>df1</span> <span>%>%</span> <span>select</span><span>(</span><span>cylinder</span>,<span>displacement</span>, <span>horsepower</span>,<span>weight</span>, <span>acceleration</span>, <span>year</span>,<span>mpg</span><span>)</span>
<span>df3</span> <span>&lt;-</span> <span>df2</span><span>[</span><span>complete.cases</span><span>(</span><span>df2</span><span>)</span>,<span>]</span>

<span># Perform MinMaxScaling of feature variables </span>
<span>train.X.scaled</span><span>=</span><span>MinMaxScaler</span><span>(</span><span>train.X</span><span>)</span>
<span>test.X.scaled</span><span>=</span><span>MinMaxScaler</span><span>(</span><span>test.X</span><span>)</span>

<span># Create a list of neighbors</span>
<span>rsquared</span> <span>&lt;-</span> <span>NULL</span>
<span>neighbors</span> <span>&lt;-</span><span>c</span><span>(</span><span>1</span>,<span>2</span>,<span>4</span>,<span>6</span>,<span>8</span>,<span>10</span>,<span>12</span>,<span>15</span>,<span>20</span>,<span>25</span>,<span>30</span><span>)</span>
<span>for</span><span>(</span><span>i</span> <span>in</span> <span>seq_along</span><span>(</span><span>neighbors</span><span>)</span><span>)</span><span>{</span>
    <span># Fit a KNN model</span>
    <span>knn</span><span>=</span><span>knn.reg</span><span>(</span><span>train.X.scaled</span>,<span>test.X.scaled</span>,<span>train.Y</span>,<span>k</span><span>=</span><span>i</span><span>)</span>
    <span># Compute R ssquared</span>
    <span>rsquared</span><span>[</span><span>i</span><span>]</span><span>=</span><span>knnRSquared</span><span>(</span><span>knn</span><span>$</span><span>pred</span>,<span>test.Y</span><span>)</span>
    
<span>}</span>

<span>df</span> <span>&lt;-</span> <span>data.frame</span><span>(</span><span>neighbors</span>,<span>Rsquared</span><span>=</span><span>rsquared</span><span>)</span>
<span># Plot the number of neighors vs the R squared </span>
<span>ggplot</span><span>(</span><span>df</span>,<span>aes</span><span>(</span><span>x</span><span>=</span><span>neighbors</span>,<span>y</span><span>=</span><span>Rsquared</span><span>)</span><span>)</span> <span>+</span> <span>geom_point</span><span>(</span><span>)</span> <span>+</span><span>geom_line</span><span>(</span><span>color</span><span>=</span><span>&quot;blue&quot;</span><span>)</span> <span>+</span>
    <span>xlab</span><span>(</span><span>&quot;Number of neighbors&quot;</span><span>)</span> <span>+</span> <span>ylab</span><span>(</span><span>&quot;R squared&quot;</span><span>)</span> <span>+</span>
    <span>ggtitle</span><span>(</span><span>&quot;KNN regression - R squared vs Number of Neighors(Normalized)&quot;</span><span>)</span></code></pre>
<p><img><img data-orig-file="https://gigadom.files.wordpress.com/2017/10/d-1.png" data-orig-size="1344,960" data-medium-file="https://gigadom.files.wordpress.com/2017/10/d-1.png?w=300" data-attachment-id="5621" data-image-meta="{" data-image-description="" src="https://gigadom.files.wordpress.com/2017/10/d-1.png?w=450&h=483&fit=676%2C483" data-permalink="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/d-1/" data-large-file="https://gigadom.files.wordpress.com/2017/10/d-1.png?w=450&amp;h=483&amp;fit=676%2C483" alt="" data-image-title="d-1" sizes="(max-width: 676px) 100vw, 676px" data-comments-opened="1" class="wp-image-5621" data-recalc-dims="1"></p>
</div>
<div>
<h2>1.4d K Nearest Neighbors Regression – Python( Normalized)</h2>
<p>R squared improves when the features are normalized with MinMaxScaling</p>
<pre><code><span>import</span> numpy <span>as</span> np
<span>import</span> pandas <span>as</span> pd
<span>import</span> os
<span>import</span> matplotlib.pyplot <span>as</span> plt
<span>from</span> sklearn.model_selection <span>import</span> train_test_split
<span>from</span> sklearn.linear_model <span>import</span> LinearRegression
<span>from</span> sklearn.preprocessing <span>import</span> PolynomialFeatures
<span>from</span> sklearn.neighbors <span>import</span> KNeighborsRegressor
<span>from</span> sklearn.preprocessing <span>import</span> MinMaxScaler
autoDF =pd.read_csv(<span>&quot;auto_mpg.csv&quot;</span>,encoding=<span>&quot;ISO-8859-1&quot;</span>)
autoDF.shape
autoDF.columns
autoDF1=autoDF[[<span>'mpg'</span>,<span>'cylinder'</span>,<span>'displacement'</span>,<span>'horsepower'</span>,<span>'weight'</span>,<span>'acceleration'</span>,<span>'year'</span>]]
autoDF2 = autoDF1.apply(pd.to_numeric, errors=<span>'coerce'</span>)
autoDF3=autoDF2.dropna()
autoDF3.shape
X=autoDF3[[<span>'cylinder'</span>,<span>'displacement'</span>,<span>'horsepower'</span>,<span>'weight'</span>,<span>'acceleration'</span>,<span>'year'</span>]]
y=autoDF3[<span>'mpg'</span>]

<span># Perform a train/ test split</span>
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = <span>0</span>)
<span># Use MinMaxScaling</span>
scaler = MinMaxScaler()
X_train_scaled = scaler.fit_transform(X_train)
<span># Apply scaling on test set</span>
X_test_scaled = scaler.transform(X_test)

<span># Create a list of neighbors</span>
rsquared=[]
neighbors=[<span>1</span>,<span>2</span>,<span>4</span>,<span>6</span>,<span>8</span>,<span>10</span>,<span>12</span>,<span>15</span>,<span>20</span>,<span>25</span>,<span>30</span>]
<span>for</span> i <span>in</span> neighbors:
    <span># Fit a KNN model</span>
    knnreg = KNeighborsRegressor(n_neighbors = i).fit(X_train_scaled, y_train)
    <span># Compute R squared</span>
    rsquared.append(knnreg.score(X_test_scaled, y_test))
    <span>print</span>(<span>'R-squared test score: {:.3f}'</span>
        .format(knnreg.score(X_test_scaled, y_test)))

<span># Plot the number of neighors vs the R squared </span>
fig4=plt.plot(neighbors,rsquared)
fig4=plt.title(<span>&quot;KNN regression - R squared vs Number of neighbors(Normalized)&quot;</span>)
fig4=plt.xlabel(<span>&quot;Neighbors&quot;</span>)
fig4=plt.ylabel(<span>&quot;R squared&quot;</span>)
fig4.figure.savefig(<span>'foo4.png'</span>, bbox_inches=<span>'tight'</span>)
<span>print</span> <span>&quot;Finished plotting and saving&quot;</span></code></pre>
<pre><code>## R-squared test score: 0.703
## R-squared test score: 0.810
## R-squared test score: 0.830
## R-squared test score: 0.838
## R-squared test score: 0.834
## R-squared test score: 0.828
## R-squared test score: 0.827
## R-squared test score: 0.826
## R-squared test score: 0.816
## R-squared test score: 0.815
## R-squared test score: 0.809
## Finished plotting and saving</code></pre>
<p><img data-orig-file="https://gigadom.files.wordpress.com/2017/10/foo4.png?w=456" data-orig-size="599,455" data-medium-file="https://gigadom.files.wordpress.com/2017/10/foo4.png?w=456?w=300" data-attachment-id="5622" data-image-meta="{" data-image-description="" src="https://gigadom.files.wordpress.com/2017/10/foo4.png?w=456" data-permalink="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/foo4/" data-large-file="https://gigadom.files.wordpress.com/2017/10/foo4.png?w=456?w=599" alt="" data-image-title="foo4" sizes="(max-width: 599px) 100vw, 599px" data-comments-opened="1" class="wp-image-5622" data-recalc-dims="1"></p>
<h1>Conclusion</h1>
<p>In this initial post I cover the regression models when the output is continous. I intend to touch upon other Machine Learning algorithms.<br>
Comments, suggestions and corrections are welcome.</p>
<p>Watch this this space!</p>
<p>To be continued….</p>
</div>
</div>
<p>You may like<br>
1. <a rel="nofollow" href="https://gigadom.wordpress.com/2017/09/29/using-linear-programming-lp-for-optimizing-bowling-change-or-batting-lineup/" target="_blank">Using Linear Programming (LP) for optimizing bowling change or batting lineup in T20 cricket</a><br>
2. <a rel="nofollow" href="https://gigadom.wordpress.com/2017/01/21/neural-networks-the-mechanics-of-backpropagation/" target="_blank">Neural Networks: The mechanics of backpropagation</a><br>
3. <a rel="nofollow" href="https://gigadom.wordpress.com/2017/03/26/more-book-more-cricket-2nd-edition-of-my-books-now-on-amazon/" target="_blank">More book, more cricket! 2nd edition of my books now on Amazon</a><br>
4. <a rel="nofollow" href="https://gigadom.wordpress.com/2014/08/07/spicing-up-a-ibm-bluemix-cloud-app-with-mongodb-and-nodeexpress/" target="_blank">Spicing up a IBM Bluemix cloud app with MongoDB and NodeExpress</a><br>
5. <a rel="nofollow" href="https://gigadom.wordpress.com/2016/04/12/introducing-cricket-package-yorkrpart-4-in-the-block-hole/" target="_blank">Introducing cricket package yorkr:Part 4-In the block hole!</a></p>
<p>To see all posts see <a rel="nofollow" href="https://gigadom.wordpress.com/aa-2/" target="_blank">Index of posts</a></p>
<p> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/gigadom.wordpress.com/5605/" target="_blank"><img border="0" alt="" src="http://feeds.wordpress.com/1.0/comments/gigadom.wordpress.com/5605/"></a> <img border="0" alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" data-recalc-dims="1"></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://gigadom.wordpress.com/2017/10/06/practical-machine-learning-with-r-and-python-part-1/"> R – Giga thoughts …</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/fqZHN-64bgE" height="1"></a></div>