<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/natural-language-processing-on-40-languages-with-the-ripple-down-rules-based-part-of-speech-tagger/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.bnosac.be/index.php/blog/65-natural-language-processing-on-40-languages-with-the-ripple-down-rules-based-part-of-speech-tagger"> bnosac :: open analytical helpers</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><strong>Parts of Speech (POS) tagging</strong> is a crucial part in natural language processing. It consists of labelling each word in a text document with a certain category like noun, verb, adverb, pronoun, … . At <a target="_blank" rel="nofollow" href="http://www.bnosac.be/">BNOSAC</a>, we use it on a dayly basis in order to select only nouns before we do topic detection or in <a target="_blank" rel="nofollow" href="http://www.bnosac.be/index.php/services/data-science-activities/text-mining">specific NLP flows</a>. For R users working with different languages, the number of POS tagging options is small and all have up or downsides. The following taggers are commonly used.</p>
<ul>
<li>The <strong>Stanford Part-Of-Speech Tagger</strong> which is terribly slow, the language set is limited to English/French/German/Spanish/Arabic/Chinese (<span>no Dutch</span>). R packages for this are available at <a target="_blank" rel="nofollow" href="http://datacube.wu.ac.at./">http://datacube.wu.ac.at.</a></li>
<li><strong>Treetagger</strong> (<a target="_blank" rel="nofollow" href="http://www.cis.uni-muenchen.de/~schmid/tools/TreeTagger">http://www.cis.uni-muenchen.de/~schmid/tools/TreeTagger</a>) contains more languages but is only usable for <span>non-commercial</span> purposes (can be used based on the <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=koRpus">koRpus </a>R package)</li>
<li><strong><a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=openNLP">OpenNLP </a></strong>is faster and allows to do POS tagging for Dutch, Spanish, Polish, Swedish, English, Danish, German but <span>no French</span> or Eastern-European languages. R packages for this are available at <a target="_blank" rel="nofollow" href="http://datacube.wu.ac.at./">http://datacube.wu.ac.at.</a></li>
<li>Package <strong>pattern.nlp</strong> (<a target="_blank" rel="nofollow" href="https://github.com/bnosac/pattern.nlp">https://github.com/bnosac/pattern.nlp</a>) allows Parts of Speech tagging and lemmatisation for Dutch, French, English, German, Spanish, Italian but<span> needs Python</span> installed which is not always easy to request at IT departments</li>
<li>SyntaxNet and Parsey McParseface (<a target="_blank" rel="nofollow" href="https://github.com/tensorflow/models/tree/master/syntaxnet">https://github.com/tensorflow/models/tree/master/syntaxnet</a>) have good accuracy for POS tagging but need tensorflow installed which might be too much <span>installation hassle</span> in a corporate setting not to mention the computational resources needed.</li>
</ul>
<p>Comes in <span><strong>RDRPOSTagger</strong> </span>which BNOSAC released at <a target="_blank" rel="nofollow" href="https://github.com/bnosac/RDRPOSTagger.">https://github.com/bnosac/RDRPOSTagger.</a> It has the following features:</p>
<p><img alt="zinsontledingmogelijk" src="https://i1.wp.com/www.bnosac.be/images/bnosac/blog/zinsontledingmogelijk.png?w=456" data-recalc-dims="1"></p>
<ol>
<li><strong>Easily installable</strong> in a corporate environment as a simple R package based on rJava</li>
<li>Covering <strong>more than 40 languages</strong>:<br><span>UniversalPOS</span> annotation for languages: <strong>Ancient_Greek, Ancient_Greek-PROIEL, Arabic, Basque, Bulgarian, Catalan, Chinese, Croatian, Czech, Czech-CAC, Czech-CLTT, Danish, Dutch, Dutch-LassySmall, English, English-LinES, Estonian, Finnish, Finnish-FTB, French, Galician, German, Gothic, Greek, Hebrew, Hindi, Hungarian, Indonesian, Irish, Italian, Kazakh, Latin, Latin-ITTB, Latin-PROIEL, Latvian, Norwegian, Old_Church_Slavonic, Persian, Polish, Portuguese, Portuguese-BR, Romanian, Russian-SynTagRus, Slovenian, Slovenian-SST, Spanish, Spanish-AnCora, Swedish, Swedish-LinES, Tamil, Turkish</strong>. Prepend the UD_ to the language if you want to used these models.<br><span>MORPH</span> annotation for languages:<strong> Bulgarian, Czech, Dutch, French, German, Portuguese, Spanish, Swedish</strong><br><span>POS</span> annotation for languages:<strong> English, French, German, Hindi, Italian, Thai, Vietnamese</strong></li>
<li><strong>Fast</strong> tagging as the Single Classification Ripple Down Rules are easy to execute and hence are quick on larger text volumes</li>
<li><strong>Competitive accuracy</strong> in comparison to state-of-the-art POS and morphological taggers</li>
<li>Cross-platform running on Windows/Linux/Mac</li>
<li>It allows to do the Morphological, POS tagging and universal POS tagging of sentences</li>
</ol>
<p>The Ripple Down Rules a basic binary classification trees which are built on top of the Universal Dependencies datasets available at <a target="_blank" rel="nofollow" href="http://universaldependencies.org./">http://universaldependencies.org.</a> The methodology of this is explained in detail at the paper ‘ A Robust Transformation-Based Learning Approach Using Ripple Down Rules for Part-Of-Speech Tagging’ available at <a target="_blank" rel="nofollow" href="http://content.iospress.com/articles/ai-communications/aic698">http://content.iospress.com/articles/ai-communications/aic698.</a> If you just want to apply POS tagging on your text, you can go ahead as follows:</p>
<pre>library(RDRPOSTagger)<br>rdr_available_models()<br><br>## POS annotation<br>x &lt;- c(&quot;Oleg Borisovich Kulik is a Ukrainian-born Russian performance artist&quot;)<br>tagger &lt;- rdr_model(language = &quot;English&quot;, annotation = &quot;POS&quot;)<br>rdr_pos(tagger, x = x)<br><br>## MORPH/POS annotation<br>x &lt;- c(&quot;Dus godvermehoeren met pus in alle puisten , zei die schele van Van Bukburg .&quot;, <br>       &quot;Er was toen dat liedje van tietenkonttieten kont tieten kontkontkont&quot;,<br>       &quot;  &quot;, &quot;&quot;, NA)<br>tagger &lt;- rdr_model(language = &quot;Dutch&quot;, annotation = &quot;MORPH&quot;)<br>rdr_pos(tagger, x = x)<br><br>## Universal POS tagging annotation<br>tagger &lt;- rdr_model(language = &quot;UD_Dutch&quot;, annotation = &quot;UniversalPOS&quot;)<br>rdr_pos(tagger, x = x)<br><br>## This gives the following output<br> sentence.id word.id             word word.type<br>           1       1              Dus       ADV<br>           1       2   godvermehoeren      VERB<br>           1       3              met       ADP<br>           1       4              pus      NOUN<br>           1       5               in       ADP<br>           1       6             alle      PRON<br>           1       7          puisten      NOUN<br>           1       8                ,     PUNCT<br>           1       9              zei      VERB<br>           1      10              die      PRON<br>           1      11           schele       ADJ<br>           1      12              van       ADP<br>           1      13              Van     PROPN<br>           1      14          Bukburg     PROPN<br>           1      15                .     PUNCT<br>           2       1               Er       ADV<br>           2       2              was       AUX<br>           2       3             toen     SCONJ<br>           2       4              dat     SCONJ<br>           2       5           liedje      NOUN<br>           2       6              van       ADP<br>           2       7 tietenkonttieten      VERB<br>           2       8             kont     PROPN<br>           2       9           tieten      VERB<br>           2      10     kontkontkont     PROPN<br>           2      11                .     PUNCT<br>           3       0             &lt;NA>      &lt;NA><br>           4       0             &lt;NA>      &lt;NA><br>           5       0             &lt;NA>      &lt;NA></pre>
<p>The function rdr_pos requests as input a vector of sentences. If you need to transform you text data to sentences, just use tokenize_sentences from the <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=tokenizers">tokenizers </a>package.</p>
<p>Good luck with text mining. <br>If you need our help for a text mining project. <a target="_blank" rel="nofollow" href="http://www.bnosac.be/index.php/contact/get-in-touch">Let us know</a>, we’ll be glad to get you started.</p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.bnosac.be/index.php/blog/65-natural-language-processing-on-40-languages-with-the-ripple-down-rules-based-part-of-speech-tagger"> bnosac :: open analytical helpers</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/2kfh1Rc1G4o"></a></div>