<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/writing-code-to-read-quotes-about-writing-code/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://georgejmount.com/writecodereadquoteswritecode/"> George J. Mount</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><img sizes="(max-width: 740px) 100vw, 740px" src="https://i2.wp.com/georgejmount.com/wp-content/uploads/2018/10/WritingCodeReadingQuotes.png?w=450&ssl=1" alt="" class="wp-image-4871" data-recalc-dims="1"></p>
<p>A recent project of mine has been setting up a <a rel="nofollow" href="https://twitter.com/QuotesInno" target="_blank">Twitter bot on innovation quotes</a>. I enjoy this project because in addition to curating a great set of content and growing an audience around it, I have also learned a lot about coding.</p>
<p><img sizes="(max-width: 465px) 100vw, 465px" src="https://i0.wp.com/georgejmount.com/wp-content/uploads/2018/10/2jxbr0.jpg?w=450&ssl=1" alt="" class="wp-image-4873" data-recalc-dims="1"></p>
<p>From web scraping to regular expressions to social media automation, I’ve learned a lot collecting a list of over 30,000 quotes related to innovation.</p>
<p>Lately I’ve been turning my attention to finding quotes about computer programming, as digital-savvy is crucial to innovation today. These exercises prove great blog post material and quite “meta,” too… <em><span>writing code to read quotes about writing code</span></em>. I will cover one of what I hope to make a series below. For this example…</p>
<p><strong>Scraping DevTopics.com’s <a rel="nofollow" href="http://www.devtopics.com/101-great-computer-programming-quotes/" target="_blank">“101 Great Computer Programming Quotes”</a></strong></p>
<p>This is a nice set of quotes but we can’t quite copy-and-paste them into a .csv file as in doing so each quote is split across multiple rows and begins with its numeric position. I also want to eliminate the quotation marks and parentheses from these quotations as stylistically I tend to avoid them for Twitter.</p>
<p>While we might despair about the orderliness of this page based on this first attempt, make no mistake that there is well-reasoned logic running under the code with its HTML, and we will need to go there instead.</p>
<h3><strong>Part I: Scrape</strong></h3>
<p>To do this I will load up the <a rel="nofollow" href="https://blog.rstudio.com/2014/11/24/rvest-easy-web-scraping-with-r/" target="_blank">rvest package</a> for R and <a rel="nofollow" href="https://chrome.google.com/webstore/detail/selectorgadget/mhjhnkcfbdhnjickkkdbjoemdmbfginb" target="_blank">SelectorGadget extension for Chrome</a>.</p>
<p><img sizes="(max-width: 642px) 100vw, 642px" src="https://i1.wp.com/georgejmount.com/wp-content/uploads/2018/10/SelectorGadget.png?w=450&ssl=1" alt="" class="wp-image-4872" data-recalc-dims="1"></p>
<p>I want to identify the HTML nodes which hold the quotes we want, then collect that text. To do that, I will initialize the SelectorGadget, then hover and click on the first quote.</p>
<p>In the bottom toolbar we see the value is set as <code>li</code>, a common HTML tag for <a rel="nofollow" href="http://bradleyboehmke.github.io/2015/12/scraping-html-text.html" target="_blank">items of a list</a>.</p>
<p>Knowing this, we will use the <code>html_nodes</code> function in R to parse those nodes, then <code>html_text</code> to extract the text they hold.</p>
<p>Doing this will return a character vector, but I will convert it to a dataframe for ease of manipulation.</p>
<p>Our code thus far is below.</p>
<pre data-enlighter-language="null">#initialize packages and URL
library(rvest)
library(tidyverse)
library(stringr)

link &lt;- c(&quot;http://www.devtopics.com/101-great-computer-programming-quotes/&quot;)

#read in our url
quotes &lt;- read_html(link)

#gather text held in the &quot;li&quot; html nodes
quote &lt;- quotes %>% 
  html_nodes(&quot;li&quot;) %>% 
  html_text()

is.vector(quote)

#convert to data frame
quote &lt;- as.data.frame(quote)</pre>
<h3><strong>Part II: Clean</strong></h3>
<p>Gathering our quotes via rvest versus copying-and-pasting, we get one quote per line, making it more legible to store in our final workbook. We’ve also left the numerical position of each quote. But some issues with the text remain.</p>
<p>First off, looking through the gathered selection of text, I will see that not all text held in the <code>li</code> node is a quote. This takes some manual intervention to spot, but here I will use dplyr’s <code>slice</code> function to keep only rows 26 through 126 (corresponding to 100 quotes).</p>
<p>We still want to eliminate the parentheses and quotation markers, and to do this I will use regular expression functions from <a rel="nofollow" href="https://cran.r-project.org/web/packages/stringr/vignettes/stringr.html" target="_blank">stringr</a> to replace them.</p>
<p><strong>a. Replace “(“, “)”, and ““” with “”</strong></p>
<p>This is not meant as a comprehensive guide to the notorious regular expression, and if you are not familiar I suggest Chapter 14 of <a rel="nofollow" href="http://r4ds.had.co.nz/strings.html" target="_blank"><em>R for Data Science</em></a>. So I assume some familiarity here as otherwise it becomes quite tedious.</p>
<p>Because “(” and “)” are both <a rel="nofollow" href="http://uc-r.github.io/regex_syntax" target="_blank">metacharacters</a> we will need to escape them. Placing these three characters together with the “or” pipe (|) we then use the <code>str_replace_all </code> function to replace strings matching any of the three with nothing “”.</p>
<p><strong>b. Replace “”” with ” “</strong></p>
<p>The end of a quotation is handled differently as we need a space between the quotation and the author; thus this expression is moved to its own function and we use <code>str_replace</code> to replace matches with ” “.</p>
<p><em>Bonus: Set it up for social media</em></p>
<p>Because I intend to send these quotes to Twitter so I will put a couple finishing touches on here.</p>
<p>First, using the <code>paste</code> function from base R, I will concatenate our quotes with a couple select hashtags.</p>
<p>Next, I use dplyr’s <code>filter</code> function to exclude lines that are longer than 240 characters, using another stringr function, <code>str_length</code>.</p>
<p>The quote for Part II is displayed below.</p>
<pre data-enlighter-language="null">#get the rows I want
quote &lt;- slice(quote, 26:126)

#delete the characters I don't want

charsd &lt;- c(&quot;\\(|\\)|“&quot;)

quote$quote &lt;- str_replace_all(quote$quote,charsd,&quot;&quot;)

quote$quote &lt;- str_replace(quote$quote,&quot;”&quot;,&quot; &quot;)

#filter lines >240 characters
quote$quote &lt;- paste(quote$quote, &quot;#quote #coding&quot;)
quote &lt;- filter(quote, str_length(quote)&lt; 240)

#write csv
write.csv(quote,&quot;C:/RFiles/tech2quotes.csv&quot;)
</pre>
<p>Finally, find the complete code below.</p>
<p></p>
<p>From web scraping to dataframe manipulation to regular expression, this exercise packs a punch in dealing with real-world unstructured text data — and it comes with some enjoyable reading, too.</p>
<p>I hope this post inspires you to tackle the world of text, and I plan to walk through a couple more of these.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://georgejmount.com/writecodereadquoteswritecode/"> George J. Mount</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/0RqGgfxmKOw" height="1"></a></div>