<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-8-intro-to-text-mining-in-r-ml-estimation-binomial-logistic-regression/"></div></div><div><a data-url="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-8-intro-to-text-mining-in-r-ml-estimation-binomial-logistic-regression/" data-counturl="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-8-intro-to-text-mining-in-r-ml-estimation-binomial-logistic-regression/" data-count="horizontal" data-text="Introduction to R for Data Science :: Session 8 [Intro to Text Mining in R, ML Estimation + Binomial Logistic Regression]" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.exactness.net/post/146247550535"> The Exactness of Mind</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><br></p>
<p>Welcome to Introduction to R for Data Science, Session 8: Intro to Text Mining in R, ML Estimation + Binomial Logistic Regression [<i>Web-scraping with tm.plugin.webmining. The tm package corpora structures: assessing document metadata and content. Typical corpus transformations and Term-Document Matrix production. A simple binomial regression model with tf-idf scores as features and its shortcommings due to sparse data. Reminder: Maximum Likelihood Estimation with Nelder-Mead from optim().<i>]</i></i></p>
<p>The course is co-organized by <a target="_blank" href="http://www.datascience.rs/">Data Science Serbia</a> and <a target="_blank" href="http://en.startit.rs/">Startit</a>. You will find all course material (R scripts, data sets, SlideShare presentations, readings) on these pages.</p>
<p>Check out the <a target="_blank" href="http://www.exactness.net/post/143651799365/introduction-to-r-for-data-science">Course Overview</a> to acess the learning material presented thus far.</p>
<p><a target="_blank" href="http://www.datascience.rs/kurs-r-za-data-science/">Data Science Serbia Course Pages [in Serbian]</a></p>
<p><a target="_blank" href="http://startit.rs/uvod-u-r-za-data-science/">Startit Course Pages [in Serbian]</a></p>
<figure data-orig-height="436" data-orig-width="500"><img alt="image" data-orig-height="436" src="http://i1.wp.com/66.media.tumblr.com/17e8497a226e52c554da4fc1d0fda08e/tumblr_inline_o93e35J5JH1qa0hyw_540.png?resize=456%2C398" data-recalc-dims="1" data-orig-width="500"></figure>
<h2><strong>Lecturers</strong></h2>
<ul>
<li><a target="_blank" href="https://rs.linkedin.com/in/kovacbranko">dipl. ing Branko Kovač</a>, Data Analyst at <a target="_blank" href="http://cube.rs/">CUBE</a>, Data Science Mentor at <a target="_blank" href="https://www.springboard.com/">Springboard</a>, Data Science Serbia</li>
<li><a target="_blank" href="http://www.exactness.net/post/72461375500/services">Goran S. Milovanović, Phd</a>, <a target="_blank" href="http://www.diplomacy.edu/">DataScientist@DiploFoundation</a>, Data Science Mentor at <a target="_blank" href="https://www.springboard.com/">Springboard</a>, Data Science Serbia</li>
</ul>
<h2><strong>Summary of Session 8, 17. June 2016 :: Intro to Text Mining in R + Binomial Logistic Regression.</strong></h2>
<p><em>Intro to Text Mining in R + Binomial Logistic Regression. <span>Intro to Text Mining in R + Binomiral Logistic Regression: Web-scraping with tm.plugin.webmining. The tm package corpora structures: assessing document metadata and content. Typical corpus transformations and Term-Document Matrix production. A simple binomial regression model with tf-idf scores as features and its shortcommings due to sparse data. Reminder: Maximum Likelihood Estimation with Nelder-Mead from optim().</span></em></p>
<ul>
<li><a target="_blank" href="http://www.exactness.net/post/146247550535#slides">Session 8 SlideShare</a></li>
<li><a target="_blank" href="http://www.exactness.net/post/146247550535#script">Session 8 R Script</a></li>
<li><a target="_blank" href="http://www.exactness.net/post/146247550535#readings">Further Readings</a></li>
</ul>
<h2><strong>Intro to R for Data Science SlideShare :: Session 8</strong></h2>
<p><iframe height="485" width="595" allowfullscreen src="http://www.slideshare.net/slideshow/embed_code/key/ztN1pu81fDHDJP"> </iframe>
<div> <strong> <a title="Introduction to R for Data Science :: Session 8 [Intro to Text Mining in R, ML Estimation + Binomial Logistic Regression]" target="_blank" href="http://www.slideshare.net/Krule/introduction-to-r-for-data-science-session-8-intro-to-text-mining-in-r-ml-estimation-binomiral-logistic-regression">Introduction to R for Data Science :: Session 8 [Intro to Text Mining in R, ML Estimation + Binomial Logistic Regression] </a> </strong> from <strong><a target="_blank" href="http://www.slideshare.net/Krule">Goran S. Milovanovic</a></strong> </div>
<p><br><br>
<h2><strong>R script :: Session 8</strong></h2>
<div>
<div>
<pre><span>########################################################</span>
<span># Introduction to R for Data Science</span>
<span># SESSION 8 :: 16 June, 2016</span>
<span># Binomiral Logistic Regression + Intro to Text Mining in R</span>
<span># Data Science Community Serbia + Startit</span>
<span># :: Goran S. Milovanović and Branko Kovač ::</span>
<span>########################################################</span>
 
<span># clear</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/rm"><span>rm</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/list"><span>list</span></a>=<a target="_blank" href="http://inside-r.org/r-doc/base/ls"><span>ls</span></a><span>(</span><span>)</span><span>)</span>
 
<span># libraries</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/library"><span>library</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/tm"><span>tm</span></a><span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/library"><span>library</span></a><span>(</span>tm.plugin.webmining<span>)</span>
 
<span>#### NOTE</span>
<span>#### Suggestion: Skip WebCorpus() calls from the tm.plugin.webmining</span>
<span>#### (skip everything before: # START HERE load)</span>
<span>#### Data set is available from GitHub :: <a target="_blank" href="https://github.com/GoranMilovanovic/IntroRDataScience">https://github.com/GoranMilovanovic/IntroRDataScience</a></span>
<span>#### File: Session8.RData</span>
<span>#### Download link: <a target="_blank" href="https://goo.gl/cgJ3J3">https://goo.gl/cgJ3J3</a></span>
 
<span>#### Part I Information Retrieval: ICT market</span>
<span># 2 categories: dotcom vs hardware companies</span>
<span># dotCom category:</span>
<span># NASDAQ:GOOGL is Alphabet Inc, NASDAQ:AMZN is Amazon, NASDAQ:JD is JD.com, NASDAQ:FB is Facebook,</span>
<span># NYSE:BABA is Alibaba</span>
<span># hardware category:</span>
<span># NYSE:HPQ is HP, NASDAQ:AAPL is Apple, KRX:005930 is Samsung Electronics, TPE:2354 is Foxconn,</span>
<span># NYSE:IBM is IBM</span>
 
<span># source: Google Finance</span>
<span># search queries: .com vs. hardware companies</span>
searchQueries <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/list"><span>list</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>'NASDAQ:GOOGL'</span><span>,</span><span>'NASDAQ:AMZN'</span><span>,</span><span>'NASDAQ:JD'</span><span>,</span><span>'NASDAQ:FB'</span><span>,</span><span>'NYSE:BABA'</span><span>)</span><span>,</span>
                      <a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>'NYSE:HPQ'</span><span>,</span><span>'NASDAQ:AAPL'</span><span>,</span><span>'KRX:005930'</span><span>,</span><span>'TPE:2354'</span><span>,</span><span>'NYSE:IBM'</span><span>)</span><span>)</span><span>;</span>
 
<span># retrive w. tm.plugin.webmining </span>
<span># retrieve news for dotCom</span>
dotCom <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>searchQueries<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
                 WebCorpus<span>(</span>GoogleFinanceSource<span>(</span>x<span>)</span><span>)</span>
<span>}</span><span>)</span><span>;</span>
<span># now retrieve news for hardware companies</span>
hardware <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>searchQueries<span>[</span><span>[</span><span>2</span><span>]</span><span>]</span><span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  WebCorpus<span>(</span>GoogleFinanceSource<span>(</span>x<span>)</span><span>)</span>
<span>}</span><span>)</span><span>;</span>
 
<span># source: Google News</span>
searchQueriesNews <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/list"><span>list</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>&quot;Google&quot;</span><span>,</span> <span>&quot;Amazon&quot;</span><span>,</span> <span>&quot;JD.com&quot;</span><span>,</span> <span>&quot;Facebook&quot;</span><span>,</span> <span>&quot;Alibaba&quot;</span><span>)</span><span>,</span> 
                       <a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>&quot;Hewlett Packard&quot;</span><span>,</span> <span>&quot;Apple&quot;</span><span>,</span> <span>&quot;Samsung&quot;</span><span>,</span> <span>&quot;Foxconn&quot;</span><span>,</span> <span>&quot;IBM&quot;</span><span>)</span><span>)</span>
 
<span># retrieve news for dotCom companies </span>
dotComNews <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>searchQueriesNews<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  googleNewsSRC <span>&lt;-</span> GoogleNewsSource<span>(</span>x<span>,</span>
                                    params = <a target="_blank" href="http://inside-r.org/r-doc/base/list"><span>list</span></a><span>(</span>hl = <span>&quot;en&quot;</span><span>,</span>
                                                  <a target="_blank" href="http://inside-r.org/r-doc/base/q"><span>q</span></a> = x<span>,</span>
                                                  ie = <span>&quot;utf-8&quot;</span><span>,</span>
                                                  num = <span>30</span><span>,</span>
                                                  output = <span>&quot;rss&quot;</span><span>)</span><span>)</span>
 
  WebCorpus<span>(</span>googleNewsSRC<span>)</span>
<span>}</span><span>)</span><span>;</span>
 
<span># retrieve news for hardware companies</span>
hardwareNews <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>searchQueriesNews<span>[</span><span>[</span><span>2</span><span>]</span><span>]</span><span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  googleNewsSRC <span>&lt;-</span> GoogleNewsSource<span>(</span>x<span>,</span>
                                    params = <a target="_blank" href="http://inside-r.org/r-doc/base/list"><span>list</span></a><span>(</span>hl = <span>&quot;en&quot;</span><span>,</span>
                                                  <a target="_blank" href="http://inside-r.org/r-doc/base/q"><span>q</span></a> = x<span>,</span>
                                                  ie = <span>&quot;utf-8&quot;</span><span>,</span>
                                                  num = <span>30</span><span>,</span>
                                                  output = <span>&quot;rss&quot;</span><span>)</span><span>)</span>
 
  WebCorpus<span>(</span>googleNewsSRC<span>)</span>
<span>}</span><span>)</span><span>;</span>
 
<span># save workspace; NOTE: tm has functions to save corpora</span>
sessionDir <span>&lt;-</span> 
  <span>&quot;YOUR WORKING DIRECTORY HERE&quot;</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/setwd"><span>setwd</span></a><span>(</span>sessionDir<span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/save.image"><span>save.image</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/file"><span>file</span></a>=<span>&quot;Session8.RData&quot;</span><span>)</span>
 
<span># START HERE load</span>
sessionDir <span>&lt;-</span> 
  <span>&quot;YOUR WORKING DIRECTORY HERE&quot;</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/setwd"><span>setwd</span></a><span>(</span>sessionDir<span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/load"><span>load</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/file"><span>file</span></a>=<span>&quot;Session8.RData&quot;</span><span>)</span>
 
<span># tm corpora are lists</span>
 
<span># a tm corpus (WebCorpus, more specifically)</span>
dotCom<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span>
<span># a PlaintTextDocument in the first dotCom corpus</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/class"><span>class</span></a><span>(</span>dotCom<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>)</span>
dotCom<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>[</span><span>[</span><span>1</span><span>]</span><span>]</span>
<span># document metadata</span>
dotCom<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>meta
<span># document content</span>
dotCom<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content
 
<span># let's add another tag the document metadata structure: dotCom</span>
dotCom <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>dotCom<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  x <span>&lt;-</span> tm_map<span>(</span>x<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>doc<span>)</span> <span>{</span> <span># tm_map works over tm corpora, similarly to lapply</span>
    <a target="_blank" href="http://inside-r.org/packages/cran/meta"><span>meta</span></a><span>(</span>doc<span>,</span> <span>&quot;category&quot;</span><span>)</span> <span>&lt;-</span> <span>&quot;dotCom&quot;</span>
    <a target="_blank" href="http://inside-r.org/r-doc/base/return"><span>return</span></a><span>(</span>doc<span>)</span>
    <span>}</span><span>)</span>
<span>}</span><span>)</span>
dotCom<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>meta
 
<span># add a category tag to the document metadata structure: dotComNews</span>
dotComNews <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>dotComNews<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  x <span>&lt;-</span> tm_map<span>(</span>x<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>doc<span>)</span> <span>{</span>
    <a target="_blank" href="http://inside-r.org/packages/cran/meta"><span>meta</span></a><span>(</span>doc<span>,</span> <span>&quot;category&quot;</span><span>)</span> <span>&lt;-</span> <span>&quot;dotCom&quot;</span>
    <a target="_blank" href="http://inside-r.org/r-doc/base/return"><span>return</span></a><span>(</span>doc<span>)</span>
  <span>}</span><span>)</span>
<span>}</span><span>)</span>
dotComNews<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>meta
 
<span># add a category tag to the document metadata structure: hardware</span>
hardware <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>hardware<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  x <span>&lt;-</span> tm_map<span>(</span>x<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>doc<span>)</span> <span>{</span>
    <a target="_blank" href="http://inside-r.org/packages/cran/meta"><span>meta</span></a><span>(</span>doc<span>,</span> <span>&quot;category&quot;</span><span>)</span> <span>&lt;-</span> <span>&quot;hardware&quot;</span>
    <a target="_blank" href="http://inside-r.org/r-doc/base/return"><span>return</span></a><span>(</span>doc<span>)</span>
  <span>}</span><span>)</span>
<span>}</span><span>)</span>
hardware<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>meta
 
<span># add a category tag to the document metadata structure: hardwareNews</span>
hardwareNews <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>hardwareNews<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  x <span>&lt;-</span> tm_map<span>(</span>x<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>doc<span>)</span> <span>{</span>
    <a target="_blank" href="http://inside-r.org/packages/cran/meta"><span>meta</span></a><span>(</span>doc<span>,</span> <span>&quot;category&quot;</span><span>)</span> <span>&lt;-</span> <span>&quot;hardware&quot;</span>
    <a target="_blank" href="http://inside-r.org/r-doc/base/return"><span>return</span></a><span>(</span>doc<span>)</span>
  <span>}</span><span>)</span>
<span>}</span><span>)</span>
hardwareNews<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>meta
 
<span># combine corpora w. do.call() and c()</span>
dotCom <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/do.call"><span>do.call</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>,</span> dotCom<span>)</span> <span># do.call comes handy; similar to lapply</span>
<span># Learn more about do.call: <a target="_blank" href="http://www.stat.berkeley.edu/~s133/Docall.html">http://www.stat.berkeley.edu/~s133/Docall.html</a></span>
hardware <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/do.call"><span>do.call</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>,</span> hardware<span>)</span>
dotComNews <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/do.call"><span>do.call</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>,</span> dotComNews<span>)</span>
hardwareNews <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/do.call"><span>do.call</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>,</span> hardwareNews<span>)</span>
workCorpus <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span>dotCom<span>,</span> dotComNews<span>,</span> hardware<span>,</span> hardwareNews<span>)</span>
 
<span># are there any duplicates?</span>
urls <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/as.character"><span>as.character</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/meta"><span>meta</span></a><span>(</span>workCorpus<span>,</span>tag=<span>&quot;origin&quot;</span><span>)</span><span>)</span>
uurls <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/unique"><span>unique</span></a><span>(</span>urls<span>)</span>
w <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/unlist"><span>unlist</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>uurls<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span><a target="_blank" href="http://inside-r.org/r-doc/base/which"><span>which</span></a><span>(</span>urls <span>%in%</span> x<span>)</span><span>[</span><span>1</span><span>]</span><span>}</span><span>)</span><span>)</span>
workCorpus <span>&lt;-</span> workCorpus<span>[</span>w<span>]</span>
 
<span># empty docs?</span>
docLengths <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/as.numeric"><span>as.numeric</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/lapply"><span>lapply</span></a><span>(</span>workCorpus<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span><a target="_blank" href="http://inside-r.org/r-doc/base/nchar"><span>nchar</span></a><span>(</span>x<span>$</span>content<span>)</span><span>}</span><span>)</span><span>)</span>
w <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/which"><span>which</span></a><span>(</span>docLengths<span>&lt;</span><span>10</span><span>)</span>
workCorpus <span>&lt;-</span> workCorpus<span>[</span><span>-</span>w<span>]</span>
 
<span>#### Part II Text Preprocessing</span>
 
<span>### text pre-processing</span>
workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content <span># we need to clean-up the docs</span>
<span># reminder: <a target="_blank" href="https://stat.ethz.ch/R-manual/R-devel/library/base/html/regex.html">https://stat.ethz.ch/R-manual/R-devel/library/base/html/regex.html</a></span>
<span># Regex in R</span>
removeSpecial <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  <span># replacing w. space might turn out to be handy</span>
  x<span>$</span>content <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/gsub"><span>gsub</span></a><span>(</span><span>&quot;<span>\\</span>t|<span>\\</span>r|<span>\\</span>n&quot;</span><span>,</span><span>&quot; &quot;</span><span>,</span>x<span>$</span>content<span>)</span>
  <a target="_blank" href="http://inside-r.org/r-doc/base/return"><span>return</span></a><span>(</span>x<span>)</span>
<span>}</span>
<span># example: </span>
cleanDoc <span>&lt;-</span> removeSpecial<span>(</span>workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>)</span>
cleanDoc<span>$</span>content
<span># removeSpecial with tm_map {tm}</span>
workCorpus <span>&lt;-</span> tm_map<span>(</span>workCorpus<span>,</span> removeSpecial<span>)</span>
workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content
 
<span>#### {tm} by the book text pre-processing</span>
 
<span># remove punctuation</span>
workCorpus <span>&lt;-</span> tm_map<span>(</span>workCorpus<span>,</span> removePunctuation<span>)</span> <span># built in</span>
workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content
 
<span># remove numbers</span>
workCorpus <span>&lt;-</span> tm_map<span>(</span>workCorpus<span>,</span> removeNumbers<span>)</span> <span># built in</span>
workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content
 
<span># all characters tolower</span>
toLower <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  x<span>$</span>content <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/tolower"><span>tolower</span></a><span>(</span>x<span>$</span>content<span>)</span>
  <a target="_blank" href="http://inside-r.org/r-doc/base/return"><span>return</span></a><span>(</span>x<span>)</span>
<span>}</span>
workCorpus <span>&lt;-</span> tm_map<span>(</span>workCorpus<span>,</span> toLower<span>)</span>
workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content
 
<span># remove stop words</span>
workCorpus <span>&lt;-</span> tm_map<span>(</span>workCorpus<span>,</span> removeWords<span>,</span> stopwords<span>(</span><span>&quot;english&quot;</span><span>)</span><span>)</span> <span># built in</span>
workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content
 
<span># stemming</span>
<span># see: <a target="_blank" href="http://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html">http://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html</a></span>
<a target="_blank" href="http://inside-r.org/r-doc/base/library"><span>library</span></a><span>(</span>SnowballC<span>)</span> <span># nice stemming algorithm (Porter, 1980)</span>
<span># see: <a target="_blank" href="https://cran.r-project.org/web/packages/SnowballC/index.html">https://cran.r-project.org/web/packages/SnowballC/index.html</a></span>
workCorpus <span>&lt;-</span> tm_map<span>(</span>workCorpus<span>,</span> stemDocument<span>)</span>
workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content
 
<span># remove potentially dangerous predictors: company names</span>
<span># Why? Why? (In a nutshell: we are trying to discover something here,</span>
<span># not to 'confirm' our potentially redundant knowledge)</span>
predWords <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/tolower"><span>tolower</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>&quot;Alphabet&quot;</span><span>,</span> <span>&quot;Google&quot;</span><span>,</span> <span>&quot;Amazon&quot;</span><span>,</span> <span>&quot;JD.com&quot;</span><span>,</span> <span>&quot;Facebook&quot;</span><span>,</span> <span>&quot;Alibaba&quot;</span><span>,</span> 
                       <span>&quot;HP&quot;</span><span>,</span> <span>&quot;Hewlett Packard&quot;</span><span>,</span> <span>&quot;Apple&quot;</span><span>,</span> <span>&quot;Samsung&quot;</span><span>,</span> <span>&quot;Foxconn&quot;</span><span>,</span> <span>&quot;IBM&quot;</span><span>)</span><span>)</span>
predWordsRegex <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span>stemDocument<span>(</span>predWords<span>)</span><span>,</span><a target="_blank" href="http://inside-r.org/r-doc/nlme/collapse"><span>collapse</span></a>=<span>&quot;|&quot;</span><span>)</span><span>;</span>
replacePreds <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>x<span>)</span> <span>{</span>
  x<span>$</span>content <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/gsub"><span>gsub</span></a><span>(</span>predWordsRegex<span>,</span>
                    <span>&quot; &quot;</span><span>,</span> 
                    x<span>$</span>content<span>)</span>
  <a target="_blank" href="http://inside-r.org/r-doc/base/return"><span>return</span></a><span>(</span>x<span>)</span>
<span>}</span>
workCorpus <span>&lt;-</span> tm_map<span>(</span>workCorpus<span>,</span> replacePreds<span>)</span>
workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content
 
<span># strip whitespace</span>
workCorpus <span>&lt;-</span> tm_map<span>(</span>workCorpus<span>,</span> stripWhitespace<span>)</span>
workCorpus<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><span>$</span>content
 
<span>#### Part III Feature Selection: Term-Document Frequency Matrix (TDM)</span>
 
<span># TDM</span>
<span># rows = terms; columns = docs</span>
<span># we will use the td-idf weighting</span>
<span># see: <a target="_blank" href="https://en.wikipedia.org/wiki/Tf%E2%80%93idf">https://en.wikipedia.org/wiki/Tf%E2%80%93idf</a></span>
dT <span>&lt;-</span> TermDocumentMatrix<span>(</span>workCorpus<span>,</span>
                              <a target="_blank" href="http://inside-r.org/r-doc/boot/control"><span>control</span></a> = <a target="_blank" href="http://inside-r.org/r-doc/base/list"><span>list</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/tolower"><span>tolower</span></a> = <span>FALSE</span><span>,</span>
                                             wordLengths <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>3</span><span>,</span> <span>Inf</span><span>)</span><span>,</span>
                                             weighting = weightTfIdf<span>)</span>
<span>)</span>
 
<span># dT is a *sparse matrix*; maybe you want to learn more about the R {slam} package</span>
<span># <a target="_blank" href="https://cran.r-project.org/web/packages/slam/index.html">https://cran.r-project.org/web/packages/slam/index.html</a></span>
<a target="_blank" href="http://inside-r.org/r-doc/base/class"><span>class</span></a><span>(</span>dT<span>)</span>
dT<span>$</span>i <span># rows w. non-zero entries</span>
dT<span>$</span>j <span># columns w. non-zero entries</span>
dT<span>$</span>v <span># non-zero entry in [i,j]</span>
dT<span>$</span>nrow <span># &quot;true&quot; row dimension</span>
dT<span>$</span>ncol <span># &quot;true&quot; column dimension</span>
dT<span>$</span>dimnames<span>$</span>Terms <span># self-explanatory</span>
dT<span>$</span>dimnames<span>$</span>Docs <span># self-explanatory</span>
 
<span># remove sparse terms</span>
docTerm <span>&lt;-</span> removeSparseTerms<span>(</span>dT<span>,</span> sparse = <span>.75</span><span>)</span> <span># sparse is in (0,1]</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/dim"><span>dim</span></a><span>(</span>docTerm<span>)</span>
docTerm<span>$</span>dimnames<span>$</span>Terms
<span># see: [1] <a target="_blank" href="http://www.inside-r.org/packages/cran/tm/docs/removeSparseTerms">http://www.inside-r.org/packages/cran/tm/docs/removeSparseTerms</a></span>
<span># A term-document matrix where those terms from x are removed which have </span>
<span># at least a sparse percentage of empty (i.e., terms occurring 0 times in a document) </span>
<span># elements. [1]</span>
<span># NOTE: **very important**; in a real-world application</span>
<span># you would probably need to run many models with features obtained from various levels of sparcity</span>
<span># and perform model selection.</span>
 
<span># as.matrix</span>
docTerm <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/as.matrix"><span>as.matrix</span></a><span>(</span>docTerm<span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/colnames"><span>colnames</span></a><span>(</span>docTerm<span>)</span> <span>&lt;-</span> paste0<span>(</span><span>&quot;d&quot;</span><span>,</span><a target="_blank" href="http://inside-r.org/r-doc/base/seq"><span>seq</span></a><span>(</span><span>1</span><span>:</span><a target="_blank" href="http://inside-r.org/r-doc/base/dim"><span>dim</span></a><span>(</span>docTerm<span>)</span><span>[</span><span>2</span><span>]</span><span>)</span><span>)</span>
<span># dimensions</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/dim"><span>dim</span></a><span>(</span>docTerm<span>)</span>
<span># inspect candidate features</span>
tfIdf <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/rowSums"><span>rowSums</span></a><span>(</span>docTerm<span>)</span>
tfIdf <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/sort"><span>sort</span></a><span>(</span>tfIdf<span>,</span> decreasing = T<span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/utils/head"><span>head</span></a><span>(</span>tfIdf<span>,</span> <span>10</span><span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/utils/tail"><span>tail</span></a><span>(</span>tfIdf<span>,</span> <span>10</span><span>)</span>
<span># list candidate features</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/length"><span>length</span></a><span>(</span>tfIdf<span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/names"><span>names</span></a><span>(</span>tfIdf<span>)</span>
<span># pick some words w</span>
w <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/sample"><span>sample</span></a><span>(</span><span>1</span><span>:</span><a target="_blank" href="http://inside-r.org/r-doc/base/length"><span>length</span></a><span>(</span>tfIdf<span>)</span><span>,</span><span>4</span><span>,</span><a target="_blank" href="http://inside-r.org/r-doc/base/replace"><span>replace</span></a> = F<span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/graphics/par"><span>par</span></a><span>(</span>mfcol = <a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>2</span><span>,</span><span>2</span><span>)</span><span>)</span>
<span>for</span> <span>(</span>i <span>in</span> <span>1</span><span>:</span><a target="_blank" href="http://inside-r.org/r-doc/base/length"><span>length</span></a><span>(</span>w<span>)</span><span>)</span> <span>{</span>
  <a target="_blank" href="http://inside-r.org/r-doc/graphics/hist"><span>hist</span></a><span>(</span>docTerm<span>[</span>w<span>[</span>i<span>]</span><span>,</span><span>]</span><span>,</span>
       main = paste0<span>(</span><span>&quot;Distribution of '&quot;</span><span>,</span><a target="_blank" href="http://inside-r.org/r-doc/base/names"><span>names</span></a><span>(</span>tfIdf<span>)</span><span>[</span>w<span>[</span>i<span>]</span><span>]</span><span>,</span><span>&quot;'&quot;</span><span>)</span><span>,</span>
       cex.main = <span>.85</span><span>,</span>
       xlab = <span>&quot;Tf-Idf&quot;</span><span>,</span>
       ylab = <span>&quot;Count&quot;</span>
       <span>)</span>
<span>}</span> <span># quite interesting, isn't it? - How do you perform regression with these?</span></pre>
</div>
</div>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="436" data-orig-width="500"><img alt="image" data-orig-height="436" src="http://i1.wp.com/66.media.tumblr.com/17e8497a226e52c554da4fc1d0fda08e/tumblr_inline_o93e35J5JH1qa0hyw_540.png?resize=456%2C398" data-recalc-dims="1" data-orig-width="500"></figure>
<div>
<div>
<pre><span>#### Part IV Binomial Logistic Regression</span>
 
<span># How well can the selected words predict the document category?</span>
<span># How to relate continuous, non-normally distributed predictors, to a categorical outcome?</span>
<span># Idea: Logistic function</span>
<a target="_blank" href="http://inside-r.org/r-doc/graphics/par"><span>par</span></a><span>(</span>mfcol=<a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>1</span><span>,</span><span>1</span><span>)</span><span>)</span>
logistic <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/t"><span>t</span></a><span>)</span> <span>{</span><a target="_blank" href="http://inside-r.org/r-doc/base/exp"><span>exp</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/t"><span>t</span></a><span>)</span><span>/</span><span>(</span><span>1</span><span>+</span><a target="_blank" href="http://inside-r.org/r-doc/base/exp"><span>exp</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/t"><span>t</span></a><span>)</span><span>)</span><span>}</span>
<a target="_blank" href="http://inside-r.org/r-doc/graphics/curve"><span>curve</span></a><span>(</span>logistic<span>,</span> from = <span>-</span><span>10</span><span>,</span> to = <span>10</span><span>,</span> n = <span>1000</span><span>,</span>
      main=<span>&quot;Logistic Function&quot;</span><span>,</span>
      cex.main = <span>.85</span><span>,</span>
      xlab = <span>&quot;t&quot;</span><span>,</span>
      ylab = <span>&quot;Logistic(t)&quot;</span><span>,</span>
      cex.lab = <span>.85</span><span>)</span>
<span># and then let t be b0 + b1*x1 + b2*x2 + ... + bn*xn</span></pre>
</div>
</div>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="373" data-orig-width="500"><img alt="image" data-orig-height="373" src="http://i0.wp.com/67.media.tumblr.com/01862efcc0c93623b72edcbfc083ac91/tumblr_inline_o93e35iaP41qa0hyw_540.png?resize=456%2C340" data-recalc-dims="1" data-orig-width="500"></figure>
<div>
<div>
<pre><span># prepare data set</span>
dataSet <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/data.frame"><span>data.frame</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/t"><span>t</span></a><span>(</span>docTerm<span>)</span><span>)</span>
dataSet<span>$</span>Category <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/as.character"><span>as.character</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/meta"><span>meta</span></a><span>(</span>workCorpus<span>,</span> tag=<span>&quot;category&quot;</span><span>)</span><span>)</span> 
<a target="_blank" href="http://inside-r.org/r-doc/base/table"><span>table</span></a><span>(</span>dataSet<span>$</span>Category<span>)</span>
 
<span># does every word play at least some role in each category?</span>
w1 <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/which"><span>which</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/colSums"><span>colSums</span></a><span>(</span>dataSet<span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/which"><span>which</span></a><span>(</span>dataSet<span>$</span>Category <span>==</span> <span>&quot;dotCom&quot;</span><span>)</span><span>,</span> <span>1</span><span>:</span><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/dim"><span>dim</span></a><span>(</span>dataSet<span>)</span><span>[</span><span>2</span><span>]</span><span>-</span><span>1</span><span>)</span><span>]</span><span>)</span><span>==</span><span>0</span><span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/colnames"><span>colnames</span></a><span>(</span>dataSet<span>)</span><span>[</span>w1<span>]</span>
w2 <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/which"><span>which</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/colSums"><span>colSums</span></a><span>(</span>dataSet<span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/which"><span>which</span></a><span>(</span>dataSet<span>$</span>Category <span>==</span> <span>&quot;hardware&quot;</span><span>)</span><span>,</span> <span>1</span><span>:</span><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/dim"><span>dim</span></a><span>(</span>dataSet<span>)</span><span>[</span><span>2</span><span>]</span><span>-</span><span>1</span><span>)</span><span>]</span><span>)</span><span>==</span><span>0</span><span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/colnames"><span>colnames</span></a><span>(</span>dataSet<span>)</span><span>[</span>w2<span>]</span>
 
<span># recode category</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/library"><span>library</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/plyr"><span>plyr</span></a><span>)</span>
dataSet<span>$</span>Category <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/as.numeric"><span>as.numeric</span></a><span>(</span>revalue<span>(</span>dataSet<span>$</span>Category<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>&quot;dotCom&quot;</span>=<span>1</span><span>,</span> <span>&quot;hardware&quot;</span>=<span>0</span><span>)</span><span>)</span><span>)</span>
 
<span>#### Logistic Regression</span>
<span># Binomial Logistic Regression: use glm w. logit link (link='logit' is default)</span>
bLRmodel <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/stats/glm"><span>glm</span></a><span>(</span>Category <span>~</span>.<span>,</span>
             <a target="_blank" href="http://inside-r.org/r-doc/stats/family"><span>family</span></a>=<a target="_blank" href="http://inside-r.org/r-doc/stats/binomial"><span>binomial</span></a><span>(</span>link=<span>'logit'</span><span>)</span><span>,</span>
             <a target="_blank" href="http://inside-r.org/r-doc/boot/control"><span>control</span></a> = <a target="_blank" href="http://inside-r.org/r-doc/base/list"><span>list</span></a><span>(</span>maxit = <span>500</span><span>)</span><span>,</span>
             <a target="_blank" href="http://inside-r.org/r-doc/utils/data"><span>data</span></a>=dataSet<span>)</span>
 
sumLR <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/summary"><span>summary</span></a><span>(</span>bLRmodel<span>)</span>
sumLR
 
<span># Coefficients</span>
<span># NOTE: coefficients here relate a unit change in the predictor to the logit[P(Outcome)] </span>
<span># logit(p) = log(p/(1-p)) - also known as log-odds</span>
sumLR<span>$</span>coefficients
<a target="_blank" href="http://inside-r.org/r-doc/base/class"><span>class</span></a><span>(</span>sumLR<span>$</span>coefficients<span>)</span>
coefLR <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/as.data.frame"><span>as.data.frame</span></a><span>(</span>sumLR<span>$</span>coefficients<span>)</span>
<span># Wald statistics significant? (this Wald z is normally distributed)</span>
coefLR <span>&lt;-</span> coefLR<span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/order"><span>order</span></a><span>(</span><span>-</span>coefLR<span>$</span>Estimate<span>)</span><span>,</span> <span>]</span>
w <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/which"><span>which</span></a><span>(</span><span>(</span>coefLR<span>$</span><span>`Pr(>|z|)`</span> <span>&lt;</span> <span>.05</span><span>)</span><span>&amp;</span><span>(</span><span>!</span><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/rownames"><span>rownames</span></a><span>(</span>coefLR<span>)</span> <span>==</span> <span>&quot;(Intercept)&quot;</span><span>)</span><span>)</span><span>)</span>
<span># which predictors worked?</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/rownames"><span>rownames</span></a><span>(</span>coefLR<span>)</span><span>[</span>w<span>]</span>
<span># NOTE: Wald statistic (z) is dangerous:</span>
<span># as the coefficient gets higher, its standard error inflates thus underestimating z</span>
<span># Beware of z...</span>
 
<span># plot coefficients {ggplot2}</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/library"><span>library</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/ggplot2"><span>ggplot2</span></a><span>)</span>
plotFrame <span>&lt;-</span> coefLR<span>[</span>w<span>,</span><span>]</span>
plotFrame<span>$</span>Estimate <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/round"><span>round</span></a><span>(</span>plotFrame<span>$</span>Estimate<span>,</span><span>2</span><span>)</span>
plotFrame<span>$</span>Features <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/rownames"><span>rownames</span></a><span>(</span>plotFrame<span>)</span>
plotFrame <span>&lt;-</span> plotFrame<span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/order"><span>order</span></a><span>(</span><span>-</span>plotFrame<span>$</span>Estimate<span>)</span><span>,</span> <span>]</span>
plotFrame<span>$</span>Features <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/factor"><span>factor</span></a><span>(</span>plotFrame<span>$</span>Features<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/levels"><span>levels</span></a> = plotFrame<span>$</span>Features<span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/ordered"><span>ordered</span></a>=T<span>)</span>
<a target="_blank" href="http://inside-r.org/packages/cran/ggplot"><span>ggplot</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/utils/data"><span>data</span></a> = plotFrame<span>,</span> aes<span>(</span>x = plotFrame<span>$</span>Features<span>,</span> y = plotFrame<span>$</span>Estimate<span>)</span><span>)</span> <span>+</span>
  geom_line<span>(</span><a target="_blank" href="http://inside-r.org/r-doc/grDevices/group"><span>group</span></a>=<span>1</span><span>)</span> <span>+</span> geom_point<span>(</span>color=<span>&quot;red&quot;</span><span>,</span> size=<span>2.5</span><span>)</span> <span>+</span> geom_point<span>(</span>color=<span>&quot;white&quot;</span><span>,</span> size=<span>2</span><span>)</span> <span>+</span>
  xlab<span>(</span><span>&quot;Features&quot;</span><span>)</span> <span>+</span> ylab<span>(</span><span>&quot;Regression Coefficients&quot;</span><span>)</span> <span>+</span>
  ggtitle<span>(</span><span>&quot;Logistic Regression: Coeficients (sig. Wald test)&quot;</span><span>)</span> <span>+</span>
  theme<span>(</span>axis.text.x = element_text<span>(</span>angle=<span>90</span><span>)</span><span>)</span></pre>
</div>
</div>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="373" data-orig-width="500"><img alt="image" data-orig-height="373" src="http://i1.wp.com/65.media.tumblr.com/e6eb830034d74d1139645fa23260ed63/tumblr_inline_o93e36kWkH1qa0hyw_540.png?resize=456%2C340" data-recalc-dims="1" data-orig-width="500"></figure>
<div>
<div>
<pre><span># fitted probabilities</span>
<a target="_blank" href="http://inside-r.org/r-doc/stats/fitted"><span>fitted</span></a><span>(</span>bLRmodel<span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/graphics/hist"><span>hist</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/stats/fitted"><span>fitted</span></a><span>(</span>bLRmodel<span>)</span><span>,</span><span>50</span><span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/graphics/plot"><span>plot</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/stats/density"><span>density</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/stats/fitted"><span>fitted</span></a><span>(</span>bLRmodel<span>)</span><span>)</span><span>,</span>
     main = <span>&quot;Predicted Probabilities: Density&quot;</span><span>)</span>
<a target="_blank" href="http://inside-r.org/r-doc/graphics/polygon"><span>polygon</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/stats/density"><span>density</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/stats/fitted"><span>fitted</span></a><span>(</span>bLRmodel<span>)</span><span>)</span><span>,</span> 
                <a target="_blank" href="http://inside-r.org/r-doc/base/col"><span>col</span></a>=<span>&quot;red&quot;</span><span>,</span> 
                border=<span>&quot;black&quot;</span><span>)</span></pre>
</div>
</div>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<figure data-orig-height="373" data-orig-width="500"><img alt="image" data-orig-height="373" src="http://i1.wp.com/66.media.tumblr.com/087105df6f0a7f257bcab1f607d5b4dd/tumblr_inline_o93e36OWPT1qa0hyw_540.png?resize=456%2C340" data-recalc-dims="1" data-orig-width="500"></figure>
<div>
<div>
<pre><span># coefficients related to odds (and not log-odds): simply</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/exp"><span>exp</span></a><span>(</span>bLRmodel<span>$</span>coefficients<span>)</span><span>[</span>w<span>]</span>
<span># check max</span>
<a target="_blank" href="http://inside-r.org/r-doc/base/max"><span>max</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/exp"><span>exp</span></a><span>(</span>bLRmodel<span>$</span>coefficients<span>)</span><span>[</span>w<span>]</span><span>)</span> <span># huge? why? - think!</span>
 
<span>#### Reminder: Maximum Likelihood Estimation</span>
normData <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/stats/rnorm"><span>rnorm</span></a><span>(</span><span>10000</span><span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/mean"><span>mean</span></a> = <span>5.75</span><span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/stats/sd"><span>sd</span></a> = <span>1.25</span><span>)</span>
normLogLike <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>params<span>,</span>x<span>)</span> <span>{</span>
  <a target="_blank" href="http://inside-r.org/r-doc/base/mean"><span>mean</span></a> <span>&lt;-</span> params<span>[</span><span>1</span><span>]</span>
  <a target="_blank" href="http://inside-r.org/r-doc/stats/sd"><span>sd</span></a> <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/abs"><span>abs</span></a><span>(</span>params<span>[</span><span>2</span><span>]</span><span>)</span> <span># dnorm generates NaNs if sd &lt; 0</span>
  dens <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/stats/dnorm"><span>dnorm</span></a><span>(</span>x<span>,</span><a target="_blank" href="http://inside-r.org/r-doc/base/mean"><span>mean</span></a><span>,</span><a target="_blank" href="http://inside-r.org/r-doc/stats/sd"><span>sd</span></a><span>)</span>
  w <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/which"><span>which</span></a><span>(</span>dens<span>==</span><span>0</span><span>)</span>
  dens<span>[</span>w<span>]</span> <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/base/.Machine"><span>.Machine</span></a><span>$</span>double.xmin	
  <a target="_blank" href="http://inside-r.org/r-doc/base/return"><span>return</span></a><span>(</span><span>-</span><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/sum"><span>sum</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/log"><span>log</span></a><span>(</span>dens<span>)</span><span>)</span><span>)</span><span>)</span> <span># negative logLike, for minimization w. optim()</span>
<span>}</span>
<span># test</span>
normLogLike<span>(</span>normData<span>,</span><a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span><span>2</span><span>,</span><span>.77</span><span>)</span><span>)</span>
<span># ML estimation</span>
<span># random initial values</span>
startMean <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/stats/runif"><span>runif</span></a><span>(</span><span>1</span><span>,</span><span>-</span><span>100</span><span>,</span><span>100</span><span>)</span>
startSd <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/stats/runif"><span>runif</span></a><span>(</span><span>1</span><span>,</span><span>-</span><span>100</span><span>,</span><span>100</span><span>)</span>
mlFit <span>&lt;-</span> <a target="_blank" href="http://inside-r.org/r-doc/stats/optim"><span>optim</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/c"><span>c</span></a><span>(</span>startMean<span>,</span> startSd<span>)</span><span>,</span>
               fn = normLogLike<span>,</span>
               x=normData<span>,</span>
               <a target="_blank" href="http://inside-r.org/r-doc/boot/control"><span>control</span></a> = <a target="_blank" href="http://inside-r.org/r-doc/base/list"><span>list</span></a><span>(</span>maxit = <span>50000</span><span>)</span><span>)</span>
<span># ML estimates</span>
mlFit<span>$</span>par
<span># cmp. true paramaters: mean = 5.75, sd = 1.25</span>
 
<span># model Chi-Square</span>
CSq <span>&lt;-</span> bLRmodel<span>$</span>null.deviance <span>-</span> bLRmodel<span>$</span>deviance <span># this difference ~ Chi-Square Distribution</span>
CSq
dfCSq <span>&lt;-</span> bLRmodel<span>$</span>df.null <span>-</span> bLRmodel<span>$</span>df.residual <span># null - residual (model) degrees of freedom</span>
dfCSq
<span># Chi-Square significance test in R:</span>
pCSq = <span>1</span><span>-</span><a target="_blank" href="http://inside-r.org/r-doc/stats/pchisq"><span>pchisq</span></a><span>(</span>CSq<span>,</span> dfCSq<span>)</span> <span># 1 - c.d.f. = P(Chi-Square larger than this obtained by chance)</span>
pCSq
 
<span># AIC = Akaike information criterion (-2*LogLikelihood+2*k, k = num.parameters)</span>
bLRmodel<span>$</span>aic</pre>
</div>
</div>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<h2><b><a target="_blank" href="http://www.exactness.net/post/146230982005/introduction-to-r-for-data-science-session-8">Session 8 Appendix</a></b></h2>
<p><br></p>
<h2><strong>Readings :: Session 9: Binomial and Multinomial Logistic Regression [23. June, 2016, @Startit.rs, 19h CET]</strong></h2>
<ul>
<li>Yves Croissant, <a target="_blank" href="https://cran.r-project.org/web/packages/mlogit/vignettes/mlogit.pdf"> Estimation of multinomial logit models in R: The mlogit Packages</a>
</li>
</ul>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-8-intro-to-text-mining-in-r-ml-estimation-binomial-logistic-regression/"></div></div><div><a data-url="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-8-intro-to-text-mining-in-r-ml-estimation-binomial-logistic-regression/" data-counturl="http://www.r-bloggers.com/introduction-to-r-for-data-science-session-8-intro-to-text-mining-in-r-ml-estimation-binomial-logistic-regression/" data-count="vertical" data-text="Introduction to R for Data Science :: Session 8 [Intro to Text Mining in R, ML Estimation + Binomial Logistic Regression]" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.exactness.net/post/146247550535"> The Exactness of Mind</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/346fDs314qQ"></a></div>