<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/statistics-sunday-introduction-to-regular-expressions/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.deeplytrivial.com/2018/11/statistics-sunday-introduction-to.html"> Deeply Trivial</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>In my last <a rel="nofollow" href="http://www.deeplytrivial.com/2018/11/statistics-sunday-reading-and-creating.html" target="_blank">Statistics Sunday post</a>, I briefly mentioned the concept of regular expressions, also known as regex (though note that in some contexts, these refer to different things – see <a rel="nofollow" href="https://www.rexegg.com/regex-vs-regular-expression.html" target="_blank">here</a>). A regular expression is a text string, which you ask your program to match. You can use this to look for files with a particular name or extension, or search a corpus of text for a specific word or word(s) that match a certain pattern. This concept is used in many programming languages, and R is no exception. In order to use a regular expression, you need a regular expression engine. Fortunately, R base and many R packages come with a regular expression engine, which you call up with a function. In R base, those functions are things like grep, grepl, regexpr, and so on.</p>
<p>Today, I want to talk about some of the syntax you use to describe different patterns of text characters, which can include letters, digits, white space, and punctuation, starting from the most simple regular expressions. Typing out a specific text string matches only that. In last week’s example, I was working with files that all start with the word “exam.” So if I had other files in that folder that didn’t follow that naming convention, I could look specifically for those by simply typing exam into the pattern attribute of my list.files() function. These files follow exam with a date, formatted as YYYYMMDD, and a letter to delineate each file from that day (a to z). Pretty much all of our files end in a, but in the rare instances where they send two files for a day, that one would end in b. How can I write a regular expression that matches this pattern?</p>
<p>First, the laziest approach, using the * character (which is sometimes called a greedy operator), which tells the regular expression engine to look for any number of characters following a string. For that, I could use this pattern:</p>
<p><span>list.files(pattern = “exam*”)</span></p>
<p>A less lazy approach that still uses the + greedy operator would be specify the numbers I know will be the same for all files, which would be the year, 2018. In fact, if all of my exam files from every year were in a single folder, I could use that to select just one year:</p>
<p><span>list.files(pattern = “exam2018*”)</span></p>
<p>Let’s say I want all files from October 2018. I could do this:</p>
<p><span>list.files(pattern = “exam201810*”)</span></p>
<p>That might be as fancy as I need to get for my approach.</p>
<p>We can get more specific by putting information into brackets, such as [a-z] to tell the program to match something with a lowercase letter in that range or [aA] to tell the program to look for something with either lowercase or uppercase a. For example, what if I wanted to find every instance of the word “witch” in <i>The Wizard of Oz</i>?<i> </i>Sometimes, the characters refer to a specific witch (Good Witch, Bad Witch, and so on) or to the concept of being a witch (usually lowercase, such as “I am a good witch”). I could download <i>The Wizard of Oz </i>through Project Gutenberg (see <a rel="nofollow" href="http://www.deeplytrivial.com/2018/05/statistics-sunday-tokenizing-text.html" target="_blank">this post</a> for how to use the guternbergr package; the ID for <i>The Wizard of Oz </i>is 55), then run some text analysis to look for any instance of Witch or witch:</p>
<p><span>witch &lt;-WoO[grep(“[Ww]itch”, WoO$text),] </span></p>
<p>Here’s what the beginning of the witch dataset looks like:</p>
<p>> <span>witch</span><br># A tibble: 141 x 2<br>   gutenberg_id text     <br>         