<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/cost-weighted-logistic-loss/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://aviadklein.wordpress.com/2016/12/16/cost-weighted-logistic-loss/"> R – My contRibution</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>The problem of weighting the type 1,2 errors on binary classification came up in a forum I visit.</p>
<p>My solution:</p>
<pre>
# normal log-loss
ll &amp;amp;amp;lt;- function(y) function(p) -(y * log(p) + (1-y)*log(1-p))

plot(ll(0),0,1,col=1,main=&quot;log loss&quot;,ylab=&quot;loss&quot;,xlab=&quot;p&quot;)
plot(ll(1),0,1,col=2,add=TRUE)
legend(&quot;topleft&quot;,legend = c(&quot;y=0&quot;,&quot;y=1&quot;), lty=1, col=1:2, bty=&quot;n&quot;)

# cost weighted log-loss
cwll &amp;amp;amp;lt;- function(y,cost) function(p) -(cost * y * log(p) + (1-cost)*(1-y)*log(1-p))

plot(cwll(0,0.1),0,1,col=1,main=&quot;cost weighted log loss\n(cost=0.1)&quot;,ylab=&quot;loss&quot;,xlab=&quot;p&quot;)
plot(cwll(1,0.1),0,1,col=2,add=TRUE)
legend(&quot;topleft&quot;,legend = c(&quot;y=0&quot;,&quot;y=1&quot;), lty=1, col=1:2, bty=&quot;n&quot;)
</pre>
<p>Here we can see the different loss behaviours</p>
<p><a target="_blank" rel="nofollow" href="https://aviadklein.wordpress.com/2016/12/16/cost-weighted-logistic-loss/plot-log-loss/#main"><img sizes="(max-width: 150px) 100vw, 150px" data-image-description="" data-orig-file="https://aviadklein.files.wordpress.com/2016/12/plot-log-loss.png" data-orig-size="637,612" data-image-title="plot-log-loss" data-comments-opened="1" data-recalc-dims="1" data-attachment-id="324" data-permalink="https://aviadklein.wordpress.com/2016/12/16/cost-weighted-logistic-loss/plot-log-loss/#main" alt="plot-log-loss" data-medium-file="https://aviadklein.files.wordpress.com/2016/12/plot-log-loss.png?w=300" data-large-file="https://aviadklein.files.wordpress.com/2016/12/plot-log-loss.png?w=637" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i0.wp.com/aviadklein.files.wordpress.com/2016/12/plot-log-loss.png?resize=150%2C144&ssl=1"></a><br>
<a target="_blank" rel="nofollow" href="https://aviadklein.wordpress.com/2016/12/16/cost-weighted-logistic-loss/rplot-cw-log-loss/#main"><img sizes="(max-width: 150px) 100vw, 150px" data-image-description="" data-orig-file="https://aviadklein.files.wordpress.com/2016/12/rplot-cw-log-loss.png" data-orig-size="637,612" data-image-title="rplot-cw-log-loss" data-comments-opened="1" data-recalc-dims="1" data-attachment-id="325" data-permalink="https://aviadklein.wordpress.com/2016/12/16/cost-weighted-logistic-loss/rplot-cw-log-loss/#main" alt="rplot-cw-log-loss" data-medium-file="https://aviadklein.files.wordpress.com/2016/12/rplot-cw-log-loss.png?w=300" data-large-file="https://aviadklein.files.wordpress.com/2016/12/rplot-cw-log-loss.png?w=637" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i2.wp.com/aviadklein.files.wordpress.com/2016/12/rplot-cw-log-loss.png?resize=150%2C144&ssl=1"></a></p>
<p>We will try to classify the Virginica species from the iris dataset</p>
<pre>
# let's take the iris data set with a species that is hard to differentiate by sepal length and petal length
d &lt;- transform(iris, y = Species == &quot;virginica&quot;)
plot(Sepal.Length~Petal.Length,data=d,col=0,pch=21,bg=rgb(0,y,0), main=&quot;is it Virginica?&quot;)
legend(&quot;topleft&quot;,legend=c(&quot;virginica&quot;,&quot;others&quot;),pch=21,col=0,pt.bg=c(&quot;green&quot;,&quot;black&quot;),bty=&quot;n&quot;)
</pre>
<p> </p>
<p><img sizes="(max-width: 637px) 100vw, 637px" data-image-description="" data-orig-file="https://i1.wp.com/aviadklein.files.wordpress.com/2016/12/rplot-virignica.png?w=456&amp;ssl=1" data-orig-size="637,612" data-image-title="rplot-virignica" data-comments-opened="1" class="wp-image-335" data-recalc-dims="1" data-attachment-id="335" data-permalink="https://aviadklein.wordpress.com/2016/12/16/cost-weighted-logistic-loss/rplot-virignica/#main" alt="rplot-virignica" data-medium-file="https://i1.wp.com/aviadklein.files.wordpress.com/2016/12/rplot-virignica.png?w=456&amp;ssl=1?w=300" data-large-file="https://i1.wp.com/aviadklein.files.wordpress.com/2016/12/rplot-virignica.png?w=456&amp;ssl=1?w=637" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i1.wp.com/aviadklein.files.wordpress.com/2016/12/rplot-virignica.png?w=456&ssl=1"></p>
<p>And now the loss functions:</p>
<pre>
# normal loss function
Loss &lt;- function(par, data) {
xb &lt;- cbind(1,d$Sepal.Length, d$Petal.Length) %*% par
p &lt;- 1 / (1 + exp(-xb))
sum(-(d$y * log(p) + (1-d$y)*log(1-p)))
}
# cost weighted version, cost refers to error on the virginica species
cwLoss &lt;- function(par, data, cost = 0.01) {
xb &lt;- cbind(1,d$Sepal.Length, d$Petal.Length) %*% par
p &lt;- 1 / (1 + exp(-xb))
sum(-(cost * d$y * log(p) + (1-cost)*(1-d$y)*log(1-p)))
}

beta &lt;- optim(par=c(0,0,0),fn=Loss,data=d)$par
cw.beta &lt;- optim(par=c(0,0,0),fn=cwLoss,data=d)$par

xRange &lt;- range(d$Petal.Length)
yRange &lt;- range(d$Sepal.Length)
X &lt;- seq(xRange[1],xRange[2],by = 0.05)
Y &lt;- seq(yRange[1],yRange[2],by = 0.05)
z &lt;- outer(X,Y, FUN = function(x,y) -(beta[1] + x * beta[3] + y * beta[2]))
cwz &lt;- outer(X,Y, FUN = function(x,y) -(cw.beta[1] + x * cw.beta[3] + y * cw.beta[2]))

contour(X,Y,z, add=TRUE,levels = 0,labels = &quot;normal&quot;,labcex = 1)
contour(X,Y,cwz, add=TRUE,levels = 0,col=4, labels = &quot;cost-weighted&quot;,labcex = 1)
</pre>
<p><img sizes="(max-width: 637px) 100vw, 637px" data-image-description="" data-orig-file="https://i1.wp.com/aviadklein.files.wordpress.com/2016/12/plot-cost-weighted-resutlts.png?w=456&amp;ssl=1" data-orig-size="637,612" data-image-title="plot-cost-weighted-resutlts" data-comments-opened="1" class="wp-image-341" data-recalc-dims="1" data-attachment-id="341" data-permalink="https://aviadklein.wordpress.com/2016/12/16/cost-weighted-logistic-loss/plot-cost-weighted-resutlts/#main" alt="Plot-Cost-weighted -resutlts.png" data-medium-file="https://i1.wp.com/aviadklein.files.wordpress.com/2016/12/plot-cost-weighted-resutlts.png?w=456&amp;ssl=1?w=300" data-large-file="https://i1.wp.com/aviadklein.files.wordpress.com/2016/12/plot-cost-weighted-resutlts.png?w=456&amp;ssl=1?w=637" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i1.wp.com/aviadklein.files.wordpress.com/2016/12/plot-cost-weighted-resutlts.png?w=456&ssl=1"></p>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/aviadklein.wordpress.com/314/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/aviadklein.wordpress.com/314/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://aviadklein.wordpress.com/2016/12/16/cost-weighted-logistic-loss/"> R – My contRibution</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/PnvbBstn2gs"></a></div>