<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/automatically-fitting-the-support-vector-machine-cost-parameter/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://www.displayr.com/automatically-fitting-the-support-vector-machine-cost-parameter/"> R – Displayr</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><img sizes="(max-width: 300px) 100vw, 300px" src="https://i0.wp.com/www.displayr.com/wp-content/uploads/2017/04/glass-300x185.png?resize=300%2C185&ssl=1" alt="preview Show information about the snippet editorYou can click on each element in the preview to jump to the Snippet Editor. SEO title preview: Displayr | Automatically Fitting the Support Vector Machine Cost Parameter" class="webfeedsFeaturedVisual wp-post-image" data-recalc-dims="1">
<p><img sizes="(max-width: 780px) 100vw, 780px" src="https://i2.wp.com/www.displayr.com/wp-content/uploads/2017/04/glass.png?w=450" alt="Glass insulators" class="wp-image-1539" data-recalc-dims="1"></p>
<p>In <a rel="nofollow" href="https://www.displayr.com/using-support-vector-machines-in-displayr/" target="_blank">an earlier post</a> I discussed how to avoid overfitting when using Support Vector Machines. This was achieved using cross validation. In cross validation, prediction accuracy is maximized by varying the cost parameter. Importantly, prediction accuracy is calculated on a different subset of the data from that used for training.</p>
<p>In this blog post I take that concept a step further, by automating the manual search for the optimal cost.</p>
<p>The data set I’ll be using describes different types of glass based upon physical attributes and chemical composition.  You can read more about the data <a rel="nofollow" href="https://archive.ics.uci.edu/ml/datasets/Glass+Identification" target="_blank">here</a>, but for the purposes of my analysis all you need to know is that the outcome variable is categorical (7 types of glass) and the 4 predictor variables are numeric.</p>
<hr>
<h1><span>Creating the base support vector machine model</span></h1>
<p>I start, as in my <a rel="nofollow" href="https://www.displayr.com/using-support-vector-machines-in-displayr/" target="_blank">earlier analysis</a>, by splitting the data into a larger 70% training sample and a smaller 30% testing sample. Then I train a support vector machine on the training sample with the following code:</p>
<pre>
library(flipMultivariates)
svm = SupportVectorMachine(Type ~ RefractiveIndex + Ca + Ba + Fe,
                           subset = training,
                           cost = 1)
</pre>
<p>This produces output as shown below. There are 2 reasons why we can largely disregard the 64.67% accuracy:</p>
<ol>
<li>We used the training data (and not the independent testing data) to calculate accuracy.</li>
<li>We have used a default value for the cost of 1 and not attempted to optimize.</li>
</ol>
<p><iframe src="https://embedded.azureedge.net/75668/207841/72c1985d-091e-4170-bcd7-f9addd264962.html?v=28297-90b5b92269934767632" width="446" height="335"></iframe></p>
<hr>
<h1><span>Amending the R code</span></h1>
<p>I am going to amend the code above in order to loop over a range of values of cost. For each value, I will calculate the accuracy on the test sample. The updated code is as follows:</p>
<pre>
library(flipMultivariates)
library(flipRegression)
costs = c(0.1, 1, 10, 100, 1000, 10000)
i = 1
accuracies = rep(0, length(costs))

for (cost in costs)
{
    svm = SupportVectorMachine(Type ~ RefractiveIndex + Ca + Ba + Fe,
                               subset = training,
                               cost = cost)
    accuracies[i] = attr(ConfusionMatrix(svm, subset = (testing == 1)), &quot;accuracy&quot;)
    i = i + 1
}
plot(costs, accuracies, type = &quot;l&quot;, log = &quot;x&quot;)
</pre>
<p>The first 5 lines set things up. I load libraries required to run the Support Vector Machine and calculate the accuracy. Next I choose a range of costs, initialize a loop counter <span>i</span> and an empty vector <span>accuracies, </span>where I store the results.</p>
<p>Then I add a loop around the code that created the base model to iterate over <span>costs</span><span>. </span>The next line calculates and stores the accuracy on the testing sample. Finally I plot the results which tells me that the greatest accuracy appears around 100. This allows us to go back and update <span>costs</span> to a more granular range around this value.</p>
<p>Re-running the code again using the new costs (10, 20, 50, 75, 100, 150, 200, 300, 500, 1000) I get the final chart shown below. This indicates that a cost of 50 gives best performance.</p>
<p><iframe width="450" src="https://embedded.azureedge.net/75668/207841/797faa03-278b-4af6-98f5-383913acbaab.html?v=28297-90b5b92269-1357995370"></iframe></p>
<hr>
<p><span>TRY IT OUT</span><br>
The analysis in this post used R in Displayr. The <span>flipMultivariates</span> package (available on <a rel="nofollow" href="https://github.com/Displayr/flipMultivariates" target="_blank">GitHub</a>), which uses the <span>e1071 <span>package, performed the calculations. You can try automatically fitting the <a rel="nofollow" href="https://app.displayr.com/Try/Automatically%20Fitting%20SVM%20Cost%20Parameter" target="_blank">Support Vector Machine Cost Parameter</a> yourself using the data in this example. </span></span></p>
<hr>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://www.displayr.com/automatically-fitting-the-support-vector-machine-cost-parameter/"> R – Displayr</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/w4YrCjqcZ6M" height="1"></a></div>