<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/tidy-text-mining-with-r/"></div></div><div><a data-url="https://www.r-bloggers.com/tidy-text-mining-with-r/" data-counturl="https://www.r-bloggers.com/tidy-text-mining-with-r/" data-count="horizontal" data-text="Tidy Text Mining with R" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://juliasilge.com/blog/Tidy-Text-Mining/"> data science ish</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I am so pleased to announce that <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=tidytext">tidytext 0.1.2</a> is now available on CRAN. This release of tidytext, a package for text mining using tidy data principles by <a target="_blank" rel="nofollow" href="http://varianceexplained.org/">Dave Robinson</a> and me, includes some bug fixes and performance improvements, as well as some new functionality.</p>
<p>There is now a handy function for accessing the various lexicons in the <code>sentiments</code> dataset without the columns that are not used in that particular dataset; this makes these datasets even easier to use with pipes and joins from dplyr.</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>tidytext</span><span>)</span><span>
</span><span>library</span><span>(</span><span>dplyr</span><span>)</span><span>
</span><span>get_sentiments</span><span>(</span><span>&quot;afinn&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 2,476 × 2
##          word score
##         &lt;chr> &lt;int>
## 1     abandon    -2
## 2   abandoned    -2
## 3    abandons    -2
## 4    abducted    -2
## 5   abduction    -2
## 6  abductions    -2
## 7       abhor    -3
## 8    abhorred    -3
## 9   abhorrent    -3
## 10     abhors    -3
## # ... with 2,466 more rows</code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span>get_sentiments</span><span>(</span><span>&quot;nrc&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>sentiment</span><span> </span><span>==</span><span> </span><span>&quot;joy&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 689 × 2
##             word sentiment
##            &lt;chr>     &lt;chr>
## 1     absolution       joy
## 2      abundance       joy
## 3       abundant       joy
## 4       accolade       joy
## 5  accompaniment       joy
## 6     accomplish       joy
## 7   accomplished       joy
## 8        achieve       joy
## 9    achievement       joy
## 10       acrobat       joy
## # ... with 679 more rows</code></pre>
</figure>
<p>The <code>unnest_tokens</code> package can now process html, xml, latex or man pages (using the <a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=hunspell">hunspell</a> package).</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>readr</span><span>)</span><span>
</span><span>data_frame</span><span>(</span><span>text</span><span> </span><span>=</span><span> </span><span>read_lines</span><span>(</span><span>&quot;https://en.wikipedia.org/wiki/Pride_and_Prejudice_and_Zombies&quot;</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>unnest_tokens</span><span>(</span><span>word</span><span>,</span><span> </span><span>text</span><span>,</span><span> </span><span>format</span><span> </span><span>=</span><span> </span><span>&quot;html&quot;</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## # A tibble: 3,068 × 1
##         word
##        &lt;chr>
## 1      pride
## 2        and
## 3  prejudice
## 4        and
## 5    zombies
## 6  wikipedia
## 7      pride
## 8        and
## 9  prejudice
## 10       and
## # ... with 3,058 more rows</code></pre>
</figure>
<p>This only works for tokenizing by single words.</p>
<p>Since the last release of this package, we published a <a target="_blank" rel="nofollow" href="http://joss.theoj.org/papers/89fd1099620268fe0342ffdcdf66776f">short paper on tidytext</a> in the Journal of Open Source Software. Our experience with JOSS was great, and you can find the citation within R by typing <code>citation(&quot;tidytext&quot;)</code>.</p>
<h2>BUT WAIT THERE’S MORE</h2>
<p>I am even <em>more</em> excited to <a target="_blank" rel="nofollow" href="http://tidytextmining.com/">publicly announce the book</a> that Dave and I have been working on.</p>
<p><a target="_blank" rel="nofollow" href="http://tidytextmining.com/"><img alt="center" src="https://i1.wp.com/juliasilge.com/figs/2016-10-28-Tidy-Text-Mining/screenshot.png?w=456" data-recalc-dims="1"></a></p>
<p>Dave and I have used the <a target="_blank" rel="nofollow" href="https://github.com/rstudio/bookdown">bookdown</a> package by Yihui Xie to write and build our book on text mining using tidy data principles. Our goal in writing this book is to provide resources and examples for people who want to use tidy tools to approach natural language processing tasks. The intended audience for this book includes people (like us!) who don’t have extensive backgrounds in computational linguistics but who need or want to analyze unstructured, text-heavy data. Using tidy data principles can make text mining easier, more effective, and consistent with tools that are already being used widely by many data scientists and analysts.</p>
<p>The book is partly composed of material from blog posts by both of us, the package’s vignettes, and various tutorials we have put together. We think it will be helpful to have all this material in one place, organized in a cohesive way for both newcomers to using this package and other users who need to look something up.</p>
<p>The book also includes new material that we haven’t published elsewhere. <a target="_blank" rel="nofollow" href="http://tidytextmining.com/twitter.html">One chapter</a> is about analyzing our own Twitter archives. How have Dave and I used our Twitter accounts?</p>
<p><a target="_blank" rel="nofollow" href="http://tidytextmining.com/twitter.html"><img alt="center" src="https://i1.wp.com/juliasilge.com/figs/2016-10-28-Tidy-Text-Mining/spread-1.png?w=456" data-recalc-dims="1"></a></p>
<p>QUITE DIFFERENTLY, IT TURNS OUT.</p>
<p><a target="_blank" rel="nofollow" href="http://tidytextmining.com/nasa.html">Another chapter</a> that I am really excited about focuses on exploring NASA metadata. I am part of the <a target="_blank" rel="nofollow" href="https://open.nasa.gov/explore/datanauts/">NASA Datanauts</a> class of 2016; it has been a great experience with lots of opportunities to meet interesting people and learn. The program is mostly oriented toward underrepresented groups in tech and is open to a variety of skill levels, including those who are true newcomers. I highly recommend applying for next year’s program if it might be a good fit for you! One of the projects I’ve worked on through this program is exploring NASA metadata. There are 32,000+ datasets at NASA, and we can use the metadata for these datasets to understand the connections between them. For just a sneak peek, here is a network of the most commonly co-occurring words in the titles of NASA datasets.</p>
<p><a target="_blank" rel="nofollow" href="http://tidytextmining.com/nasa.html"><img alt="center" src="https://i1.wp.com/juliasilge.com/figs/2016-10-28-Tidy-Text-Mining/plot_title-1.png?w=456" data-recalc-dims="1"></a></p>
<h2>The End</h2>
<p>Some of the later chapters are less developed currently than the earlier ones; Dave and I are still in the writing process and will be actively developing and honing the book in the near future. We are eager to share our work with the world at large, though, so go ahead and take a look! We plan to finish writing this book in the open, and we are open to suggestions and edits via <a target="_blank" rel="nofollow" href="https://github.com/dgrtwo/tidy-text-mining/issues">the book’s GitHub repository</a>. The R Markdown file used to make this blog post is available <a target="_blank" rel="nofollow" href="https://github.com/juliasilge/juliasilge.github.io/blob/master/_R/2016-10-28-Tidy-Text-Mining.Rmd">here</a>. I am very happy to hear feedback or questions!</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/tidy-text-mining-with-r/"></div></div><div><a data-url="https://www.r-bloggers.com/tidy-text-mining-with-r/" data-counturl="https://www.r-bloggers.com/tidy-text-mining-with-r/" data-count="vertical" data-text="Tidy Text Mining with R" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://juliasilge.com/blog/Tidy-Text-Mining/"> data science ish</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/uyPE10vkldQ"></a></div>