<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/how-to-detect-hatespeech-in-plain-text-schildnvrienden/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.bnosac.be/index.php/blog/80-how-to-detect-hatespeech-in-plain-text-schildnvrienden"> bnosac :: open analytical helpers - bnosac :: open analytical helpers</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Yesterday there was a pretty controversial Pano TV documentary called ‘Wie is Schild &amp; Vrienden echt’ at the national television channel ‘één’ (<a rel="nofollow" href="https://www.vrt.be/vrtnu/a-z/pano/2018/pano-s2018a10" target="_blank">https://www.vrt.be/vrtnu/a-z/pano/2018/pano-s2018a10</a>). The documentary revealed the internal communication of a right-wing group from Belgium, called #schildnvrienden.</p>
<p>After that, there was a show by <a rel="nofollow" href="https://www.een.be/van-gils-gasten" target="_blank">Van Gils &amp; gasten</a> where a representative of the police explained or tried not to explain how the police can or can not monitor online private groups. It was pretty hilarious how she tried to manage not to say anything about the internal online monitoring system they apparently have.</p>
<p>That reminded me that a few years ago, I created an R package which can easily detect hate speech. I finally put it online on github today. You can find it <a rel="nofollow" href="https://github.com/weRbelgium/hatespeech.dutch" target="_blank">here</a>. <strong><span>The R package used a dictionary which is made available by the University of Antwerp which I think is the basis of the hate speech detection algorithms that currently the police in Belgium is running</span>.</strong></p>
<h3><strong>Example</strong></h3>
<p>How does that hate speech detection system work? Pretty simple, a <strong>dictionary of hate speech terminology and hate speech regular expressions</strong> are set up and next you just provide some text to it, the data is being cut up into words and it sees which words are part of the dictionary. As an example below, let’s try it out on a message by the leader of that #schildnvrienden group to see if it is considered hate speech.</p>
<p><img src="https://i1.wp.com/www.bnosac.be/images/bnosac/blog/screenshot-twitter-dvanlangenhove-20180901.png?resize=232%2C190" alt="screenshot twitter dvanlangenhove 20180901" width="232" data-recalc-dims="1" height="190"></p>
<pre>library(udpipe)<br>library(hatespeech.dutch)<br>detect_hatespeech(&quot;Europa wordt élke dag geteisterd door geweld van illegalen. <br> Zowel voor mensen die zich zorgen maken over dit geweld als voor mensen <br> die zich zorgen maken over de boze reactie van Europeanen òp dit geweld zou <br> oplossing duidelijk moeten zijn: alle illegalen opsporen en deporteren.&quot;, <br> type = &quot;udpipe&quot;)</pre>
<pre tabindex="0"><span> Neutral-Country Neutral-Migration Neutral-Nationality 0 1 0 Neutral-Religion Neutral-Skin_color Racist-Animals 0 0 0 Racist-Country Racist-Crime Racist-Culture 0 0 0 Racist-Diseases Racist-Migration Racist-Nationality 0 0 0 Racist-Race Racist-Religion Racist-Skin_color 0 0 0 Racist-Stereotypes 0</span></pre>
<p>So apparently the dictionary logic considers this statement as Neutral-Migration. Hope the police have improved on the natural language processing a bit such that they have incorporated a bit more than just word lookup and regular expressions. Feel free to try the hate speech detector out on your own text using the R package made available at <a rel="nofollow" href="https://github.com/weRbelgium/hatespeech.dutch" target="_blank">https://github.com/weRbelgium/hatespeech.dutch</a>. Or visit the website to see to the dictionaries which are used to detect hate speech.</p>
<h3><strong>Training on Text Mining <br></strong></h3>
<p>Are you interested in how text mining techniques work, then you might be interested in the following data science courses that are held in the coming months.</p>
<ul>
<li>08-09/10/2018: Text mining with R. Brussels (Belgium). <a rel="nofollow" href="http://di-academy.com/bootcamp/" target="_blank">http://di-academy.com/bootcamp</a> + send mail to <a rel="nofollow" href="mailto:training@di-academy.com" target="_blank">training@di-academy.com</a></li>
<li>15-16/10/2018: Statistical machine learning with R. Leuven (Belgium). Subscribe <a rel="nofollow" href="https://lstat.kuleuven.be/training/coursedescriptions/statistical-machine-learning-with-r" target="_blank">here</a></li>
<li>05-06/11/2018: Text mining with R. Leuven (Belgium). Subscribe <a rel="nofollow" href="https://lstat.kuleuven.be/training/coursedescriptions/text-mining-with-r" target="_blank">here</a></li>
<li>19-20/12/2018: Applied spatial modelling with R. Leuven (Belgium). Subscribe <a rel="nofollow" href="https://lstat.kuleuven.be/training/applied-spatial-modelling-with-r" target="_blank">here</a></li>
<li>21-22/02/2018: Advanced R programming. Leuven (Belgium). Subscribe <a rel="nofollow" href="https://lstat.kuleuven.be/training/coursedescriptions/AdvancedprogramminginR.html" target="_blank">here</a></li>
<li>13-14/03/2018: Computer Vision with R and Python. Leuven (Belgium). Subscribe <a rel="nofollow" href="https://lstat.kuleuven.be/training/coursedescriptions/ComputervisionwithRandPython" target="_blank">here</a></li>
<li>     15/03/2019: Image Recognition with R and Python: Subscribe <a rel="nofollow" href="https://lstat.kuleuven.be/training/coursedescriptions/Imagerecognition" target="_blank">here</a></li>
<li>01-02/04/2019: Text Mining with R. Leuven (Belgium). Subscribe <a rel="nofollow" href="https://lstat.kuleuven.be/training/coursedescriptions/text-mining-with-r" target="_blank">here</a></li>
</ul>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.bnosac.be/index.php/blog/80-how-to-detect-hatespeech-in-plain-text-schildnvrienden"> bnosac :: open analytical helpers - bnosac :: open analytical helpers</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/2FEH_HryAes" height="1"></a></div>