In a number of upcoming posts, I'll be analyzing an interesting dataset I found on Kaggle. The dataset contains information on 18,393 music reviews from the Pitchfork website. The data cover reviews posted between January 1999 and January 2016. I downloaded the data and did an extensive data munging exercise to turn the data into a tidy dataset for analysis (not described here, but perhaps in a future blog post).The goal of this post is to describe the similarities and differences among the music genres of the reviewed albums using cluster analysis. The Data
After data munging, I was left with 18,389 reviews for analysis (there were 4 identical review id's in the dataset; visual inspection showed that the reviews were identical so I removed the duplicates).One of the pieces of information about each album is the music genre, with the following options available: electronic, experimental, folk/country, global, jazz, metal, pop/rnb, rap and rock. Each album can have 0, 1 or multiple genres attached to it. I represented this information in the wide format, with one column to represent each genre. The presence of a given genre for a given album is represented with a 1, and the absence of a given genre for a given album is represented with a 0.The head of the dataset, called &quot;categories,&quot; looks like this:
title
artist
electronic
experimental
folk_country
global
jazz
metal
pop_rnb
rap
rock
mezzanine
massive attack
1
0
0
0
0
0
0
0
0
prelapsarian
krallice
0
0
0
0
0
1
0
0
0
all of them naturals
uranium club
0
0
0
0
0
0
0
0
1
first songs
kleenex liliput
0
0
0
0
0
0
0
0
1
new start
taso
1
0
0
0
0
0
0
0
0
insecure (music from the hbo original series)
various artists
0
0
0
0
0
0
0
0
0
As we can see, each album has numeric values on all of our 9 genres. The album in the last row of the data shown above does not have a genre attached to it.Let's first answer some basic questions about the music genres. How often is each genre represented in our 18,389 reviews? We can make a simple bar plot using base R with the following code:# histogram of genre frequenciespar(mai=c(1,1.5,1,1))barplot(sort(colSums(categories[,3:11]), decreasing = FALSE), horiz = TRUE, cex.names = 1, col = 'springgreen4', main = 'Genre Frequency', las = 1)
Which gives us the following plot:
Rock is by far the most frequently-occurring genre in the dataset!Let's not forget that albums can have more than one genre. How many albums have more than 1 genre attached to them? What is the maximum number of genres in these data? What number of albums have what number of genres? It's possible to extract these figures with dplyr, but for basic summary statistics I find it quicker and easier to use base R:# how many of the albums have more than 1 genre?table(rowSums(categories[,3:11])__1)# FALSE TRUE # 14512 3877# what is the maximum number of genres?max(rowSums(categories[,3:11]))# [1] 4# how many albums have what number of genres?table(rowSums(categories[,3:11]))# 0 1 2 3 4 # 2365 12147 3500 345 32
It looks like 3,877 of the albums have more than 1 genre. The table shows that the vast majority of albums with more than 1 genre have 2 genres, with a much smaller number having 3 and 4 genres. There are 2,365 albums with no recorded genre.Data Preparation In order to cluster the music genres, we first must make a matrix which contains the co-occurrences of our dummy-coded genre variables with one another. We can use matrix multiplication to accomplish this, as explained in this excellent StackOverflow answer:# make a co-occurrence matrix# select the relevant columns and convert to matrix formatlibrary(plyr); library(dplyr)co_occur % select(electronic, experimental, folk_country, global,jazz, metal, pop_rnb, rap, rock) %__% as.matrix() # calculate the co-occurrences of genresout