How Has Taylor Swift's Word Choice Changed Over Time?Sunday night was a big night for Taylor Swift - not only was she nominated for multiple Billboard Music Awards; she took home Top Female Artist and Top Selling Album. So I thought it was a good time for some more Taylor Swift-themed statistical analysis.When I started this blog back in 2011, my goal was to write deep thoughts on trivial topics - specifically, to overthink and overanalyze pop culture and related topics that appear fluffy until you really dig into them. Recently, I've been blogging more about statistics, research, R, and data science, and I've loved getting to teach and share.But sometimes, you just want to overthink and overanalyze pop culture.So in a similar vein to the text analysis I've been demonstrating on my blog, I decided to answer a question I'm sure we all have - as Taylor Swift moved from country sweetheart to mega pop star, how have the words she uses in her songs changed?I've used the geniusR package on a couple posts, and I'll be using it again today to answer this question. I'll be pulling in some additional code, some based on code from the Text Mining with R: A Tidy Approach book I recently devoured, some written to try to tackle this problem I've created for myself to solve. I've shared all my code and tried to credit those who helped me write it where I can.First, we want to pull in the names of Taylor Swift's 6 studio albums. I found these and their release dates on Wikipedia. While there are only 6 and I could easily copy and paste them to create my data frame, I wanted to pull that data directly from Wikipedia, to write code that could be used on a larger set in the future. Thanks to this post, I could, with a couple small tweaks.library(rvest)## Loading required package: xml2TSdisc %html_nodes(xpath = '//*[@id=&quot;mw-content-text&quot;]/div/table[2]') %__%html_table(fill = TRUE)Since html() is deprecated, I replaced it with read_html(), and I got errors if I didn't add fill = TRUE. The result is a list of 1, with an 8 by 14 data frame within that single list object. I can pull that out as a separate data frame.TS_albums %anti_join(stop_words)## Joining, by = &quot;word&quot;tidy_TS %__%count(word, sort = TRUE)## # A tibble: 2,024 x 2## word n## ## 1 time 198## 2 love 180## 3 baby 118## 4 ooh 104## 5 stay 89## 6 night 85## 7 wanna 84## 8 yeah 83## 9 shake 80## 10 ey 72## # ... with 2,014 more rowsThere are a little over 2,000 unique words across TS's 6 albums. But how have they changed over time? To examine this, I'll create a dataset that counts word by year (or album, really). Then I'll use a binomial regression model to look at changes over time, one model per word. In their book, Julia Silge and David Robinson demonstrated how to use binomial regression to examine word use on the authors' Twitter accounts over time, including an adjustment to the p-values to correct for multiple comparisons. So I based on my code off that.words_by_year %count(Year, word) %__%group_by(Year) %__%mutate(time_total = sum(n)) %__%group_by(word) %__%mutate(word_total = sum(n)) %__%ungroup() %__%rename(count = n) %__%filter(word_total __ 50)nested_words %nest(-word)word_models %mutate(models = map(data, ~glm(cbind(count, time_total) ~ Year, .,family = &quot;binomial&quot;)))This nests our regression results in a data frame called word_models. While I could unnest and keep all, I don't care about every value the GLM gives me. What I care about is the slope for Year, so the filter selects only that slope and the associated p-value. I can then filter to select the significant/marginally significant slopes for plotting (p library(broom)slopes %unnest(map(models, tidy)) %__%filter(term == &quot;Year&quot;) %__%mutate(adjusted.p.value = p.adjust(p.value))top_slopes %filter(adjusted.p.value %select(-statistic, -p.value)This gives me five words that show changes in usage over time: bad, call, dancing, eyes, and yeah. We can plot those five words to see how they've changed in usage over her 6 albums. And because I still have my TS_albums data frame, I can use that information to label the axis of my plot (which is why I needed year to be numeric). I also added a vertical line and annotations to note where TS believes she shifted from country to pop.library(scales)words_by_year %__%inner_join(top_slopes, by = &quot;word&quot;) %__%ggplot(aes(Year, count/time_total, color = word, lty = word)) +geom_line(size = 1.3) +labs(x = NULL, y = &quot;Word Frequency&quot;) +scale_x_continuous(breaks=TS_albums$Year,labels=TS_albums$Title) +scale_y_continuous(labels=scales::percent) +geom_vline(xintercept = 2014) +theme(panel.grid.major = element_blank(),panel.grid.minor = element_blank(),panel.background = element_blank()) +annotate(&quot;text&quot;, x = c(2009.5,2015.5), y = c(0.025,0.025),label = c(&quot;Country&quot;, &quot;Pop&quot;) , size=5)The biggest change appears to be in the word &quot;call,&quot; which she didn't use at all in her self-titled album, and used at low rates until &quot;1989&quot; and, especially, &quot;Reputation.&quot; I can ask for a few examples of &quot;call&quot; in her song lyrics, with grep.library(expss)callsubset