<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/analyzing-english-team-of-the-year-data-since-1973/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://worldsocceranalytics.com/2018/10/18/analyzing-english-team-of-the-year-data-since-1973/"> World Soccer Analytics</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>The Professional Footballers’ Association (PFA) <a rel="nofollow" href="https://en.wikipedia.org/wiki/PFA_Team_of_the_Year" target="_blank">Team of the Year</a> is released in England at the end of each season, picking the 11 most influential players in each of Britain’s leagues.</p>
<p>The Team of the Year award was launched in the 1973-1974 season, meaning there was 44 years worth of data to web scrape. Using Wikipedia’s <a rel="nofollow" href="https://en.wikipedia.org/wiki/PFA_Team_of_the_Year" target="_blank">PFA Team of the Year pages</a> (filtered by decade) and the <code>rvest</code> package, I was left with a dataframe of 484 soccer players (44 years * 11 players per/year).</p>
<p>Here are some visualizations I thought were cool:</p>
<ul>
<li>Liverpool, Arsenal, and (most significantly) Manchester United are represented more than other clubs. United need just three more players to hit the 100 mark.</li>
</ul>
<p><a rel="nofollow" href="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_per_club1.png" target="_blank"><img data-orig-file="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_per_club1.png?w=450&amp;h=349&amp;fit=698%2C349" data-orig-size="3543,1771" data-medium-file="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_per_club1.png?w=450&amp;h=349&amp;fit=698%2C349?w=300" data-attachment-id="174" data-image-meta="{" data-image-description="" src="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_per_club1.png?w=450&h=349&fit=698%2C349" data-permalink="https://worldsocceranalytics.com/2018/10/18/analyzing-english-team-of-the-year-data-since-1973/static_plot_players_per_club-2/" data-large-file="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_per_club1.png?w=450&amp;h=349&amp;fit=698%2C349?w=1024" alt="static_plot_players_per_club" data-image-title="static_plot_players_per_club" sizes="(max-width: 698px) 100vw, 698px" data-comments-opened="1" class="wp-image-174" data-recalc-dims="1"></a></p>
<ul>
<li>The PFA Team of the Year formation has changed over time from 4-3-3 to 4-4-2. The graph below shows the number of <span>forwards</span> and <span>midfielders</span> who featured in the ranking each year. Up until the late 1980s, the formation included three midfielders and three forwards (the Team of the Year has always had four defenders and one goalkeeper). In the 1990s, both formations were used but since the 2000s, the use of two strikers has been the status quo.</li>
</ul>
<p><img data-orig-file="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_per_formation1.png?w=456" data-orig-size="3543,1771" data-medium-file="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_per_formation1.png?w=456?w=300" data-attachment-id="175" data-image-meta="{" data-image-description="" src="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_per_formation1.png?w=456" data-permalink="https://worldsocceranalytics.com/2018/10/18/analyzing-english-team-of-the-year-data-since-1973/static_plot_players_per_formation-2/" data-large-file="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_per_formation1.png?w=456?w=1024" alt="static_plot_players_per_formation" data-image-title="static_plot_players_per_formation" sizes="(max-width: 1060px) 100vw, 1060px" data-comments-opened="1" class="wp-image-175" data-recalc-dims="1"></p>
<ul>
<li>Additionally, I thought it was interesting to see the top individual players in each position. Peter Shilton remains the only player to hit 10 First Division PFA Team of the Year Awards.</li>
</ul>
<p> </p>
<p><a rel="nofollow" href="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_appearances_number1.png" target="_blank"><img data-orig-file="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_appearances_number1.png?w=456" data-orig-size="3543,1771" data-medium-file="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_appearances_number1.png?w=456?w=300" data-attachment-id="166" data-image-meta="{" data-image-description="" src="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_appearances_number1.png?w=456" data-permalink="https://worldsocceranalytics.com/2018/10/18/analyzing-english-team-of-the-year-data-since-1973/static_plot_players_appearances_number-2/" data-large-file="https://worldsocceranalytics.files.wordpress.com/2018/10/static_plot_players_appearances_number1.png?w=456?w=1024" alt="static_plot_players_appearances_number.png" data-image-title="static_plot_players_appearances_number" sizes="(max-width: 1060px) 100vw, 1060px" data-comments-opened="1" class="wp-image-166" data-recalc-dims="1"></a></p>
<ul>
<li>The more interesting analysis involved merging Team of the Year data with <a rel="nofollow" href="https://en.wikipedia.org/wiki/List_of_English_football_champions" target="_blank">historical league position.</a> We can now ask questions along the lines of: did the majority of players in the Team of the Year come from the champions that year or other teams? Looking at the specific seasons with the most representation in the award, only 14 times did a team have 5+ players on the list; the majority of these teams came from the <span>champions</span> (12 out of 14); only Manchester United in 1998 and Arsenal in 2003 (both <span>runner-ups</span>) feature in this list.</li>
</ul>
<p> </p>
<p><a rel="nofollow" href="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year2.png" target="_blank"><img data-orig-file="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year2.png?w=456" data-orig-size="3543,1771" data-medium-file="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year2.png?w=456?w=300" data-attachment-id="169" data-image-meta="{" data-image-description="" src="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year2.png?w=456" data-permalink="https://worldsocceranalytics.com/2018/10/18/analyzing-english-team-of-the-year-data-since-1973/teams_most_representation_per_year-3/" data-large-file="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year2.png?w=456?w=1024" alt="teams_most_representation_per_year.png" data-image-title="teams_most_representation_per_year" sizes="(max-width: 1060px) 100vw, 1060px" data-comments-opened="1" class="wp-image-169" data-recalc-dims="1"></a></p>
<ul>
<li>And here are the champions of England with the <em>fewest</em> players on the Team of the Year. There are many from the 1970s and 1980s, which hints to a change in the way the Team of the Year was designed over the years (it may have been purposefully less top-heavy in the past, whereas today the English champions often features more significantly on the list). We can see that more recently Manchester City in 2014 and Chelsea in 2010 were on the lower end in terms of representation.</li>
</ul>
<p><a rel="nofollow" href="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year_lowest_champions1.png" target="_blank"><img data-orig-file="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year_lowest_champions1.png?w=456" data-orig-size="3543,1771" data-medium-file="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year_lowest_champions1.png?w=456?w=300" data-attachment-id="172" data-image-meta="{" data-image-description="" src="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year_lowest_champions1.png?w=456" data-permalink="https://worldsocceranalytics.com/2018/10/18/analyzing-english-team-of-the-year-data-since-1973/teams_most_representation_per_year_lowest_champions-2/" data-large-file="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year_lowest_champions1.png?w=456?w=1024" alt="teams_most_representation_per_year_lowest_champions.png" data-image-title="teams_most_representation_per_year_lowest_champions" sizes="(max-width: 1060px) 100vw, 1060px" data-comments-opened="1" class="wp-image-172" data-recalc-dims="1"></a></p>
<p> </p>
<ul>
<li>While we’re at it, which teams that didn’t finish 1st had the most players represented? It’s clear that there were some pretty high quality teams that didn’t win the league these years (runners-up are in <span>red</span>, third place teams in <span>green</span>, and any from 4th to 20th in <span>grey</span>).</li>
</ul>
<p><img data-orig-file="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year_losers1.png?w=456" data-orig-size="3543,1771" data-medium-file="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year_losers1.png?w=456?w=300" data-attachment-id="177" data-image-meta="{" data-image-description="" src="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year_losers1.png?w=456" data-permalink="https://worldsocceranalytics.com/2018/10/18/analyzing-english-team-of-the-year-data-since-1973/teams_most_representation_per_year_losers-2/" data-large-file="https://worldsocceranalytics.files.wordpress.com/2018/10/teams_most_representation_per_year_losers1.png?w=456?w=1024" alt="teams_most_representation_per_year_losers" data-image-title="teams_most_representation_per_year_losers" sizes="(max-width: 1060px) 100vw, 1060px" data-comments-opened="1" class="wp-image-177" data-recalc-dims="1"></p>
<p>I think that’s enough visualizations for today, but there’s definitely a lot more we can analyze with this data. Let me know if you have any questions or feedback.</p>
<p><strong>R Code Snapshot (full code can be found on <a rel="nofollow" href="https://github.com/stefangouyet/epl_team_of_year" target="_blank">Github</a>):</strong></p>
<p>Step 1 – Web Scraping:</p>
<pre>#initialize data.frame
df &lt;- data.frame(`Pos.`= as.character(),
Player = as.character(),
Club = as.character(),
`App.` = as.double(),
year = as.character())

div_table_numbers &lt;- c(1,5,9,13,17,21,25,29,33,37)

urls &lt;- c(&quot;https://en.wikipedia.org/wiki/PFA_Team_of_the_Year_(2000s)&quot;,
&quot;https://en.wikipedia.org/wiki/PFA_Team_of_the_Year_(1990s)&quot;,
&quot;https://en.wikipedia.org/wiki/PFA_Team_of_the_Year_(1980s)&quot;)


for (i in 1:length(urls)){

  for (j in 1:length(div_table_numbers)) {
    xpath_base &lt;- '//*[@id=&quot;mw-content-text&quot;]/div/table['
    new_data &lt;- urls[i] %>%
      html() %>%
      html_nodes(xpath = paste0(xpath_base,
                              div_table_numbers[j],
                              &quot;]&quot;)) %>%
      html_table()

     new_data &lt;- new_data[[1]]

     year_xpath_base &lt;- '//*[@id=&quot;mw-content-text&quot;]/div/h3['
     year &lt;- urls[i] %>%
      html() %>%
      html_nodes(xpath = paste0(year_xpath_base,
                              j,
                              &quot;]&quot;)) %>%
      html_text()

      year &lt;- year %>% str_remove_all(fixed(&quot;[edit]&quot;))
    
    new_data$year &lt;- year
     
    df &lt;- bind_rows(df,
                    new_data)
  }
}</pre>
<p>Step 2: Exploratory data analysis and visualizations</p>
<pre>club_count &lt;- df %>% count(Club, sort = TRUE)

club_count %>% 
  top_n(n = 15) %>% 
  ggplot(aes(x = reorder(Club,n), 
                  y = n,
                  fill = Club)) +
  geom_col() +
  coord_flip()+
  theme_few()+
  guides(fill=FALSE)+
  labs(y = &quot;# of Players&quot;,
       x = &quot;Club&quot;,
       title = &quot;Number of Players in PFA Team of the Year (1973 - 2017)&quot;,
       caption = &quot;Data Source: Wikipedia&quot;)+
  theme(plot.title = element_text(hjust = 0.5)) +
   scale_fill_manual(values = c(&quot;Manchester United&quot; = &quot;darkred&quot;, 
                                 &quot;Liverpool&quot; = &quot;orange&quot;,
                                 &quot;Arsenal&quot; = &quot;yellow&quot;,
                                 &quot;Chelsea&quot; = &quot;blue&quot;,
                                 &quot;Blackburn Rovers&quot; = &quot;lightblue&quot;,
                                 &quot;Leeds United&quot; = &quot;grey&quot;,
                                 &quot;Manchester City&quot; = &quot;dark green&quot;,
                               &quot;Derby County&quot; = &quot;black&quot;,
                               &quot;Everton&quot; = &quot;gold&quot;,
                               &quot;Nottingham Forest&quot; = &quot;red&quot;,
                               &quot;Ipswich Town&quot; = &quot;darkblue&quot;,
                               &quot;Southampton&quot; = &quot;orange&quot;,
                               &quot;Newcastle United&quot; = &quot;darkgrey&quot;,
                               &quot;Aston Villa&quot; = &quot;purple&quot;, 
                               &quot;Tottenham Hotspur&quot; = &quot;navy&quot;
                               ))</pre>
<pre>df &lt;- df %>%
  mutate(short_year = str_sub(year,1,4) %>% 
           as.numeric() + 1)

order_positions &lt;- c(&quot;GK&quot;,&quot;DF&quot;,&quot;MF&quot;,&quot;FW&quot;)

df &lt;- df %>% mutate(Pos. = fct_relevel(Pos., order_positions))

count_position &lt;- df %>%
  filter(Pos. %in% c(&quot;MF&quot;,&quot;FW&quot;)) %>%
    count(Pos.,short_year, sort = TRUE)

count_position %>%
  ggplot(aes(x=short_year,y=n,color=Pos.,group=Pos.)) + 
  geom_point(position=position_jitter(h=0.005))+
  geom_smooth(method = &quot;loess&quot;)+
  scale_x_continuous(breaks = seq(1973, 2017, 5))+
  scale_y_continuous(breaks=seq(2,4,1))+
  labs(x = &quot;Year&quot;,
       y = &quot;Number of Players per Position&quot;,
       title = &quot;Count of Midfielders and Forwards in English PFA Team of the Year (1973 - 2017)&quot;,
       caption = &quot;Data Source: Wikipedia&quot;)+
  theme_few()+
  theme(plot.title = element_text(hjust = 0.5))</pre>
<p>Step 3: More web scraping and merging datasets</p>
<pre>first_div_top_three_url &lt;- &quot;https://en.wikipedia.org/wiki/List_of_English_football_champions&quot;
first_div_top_three_xpath &lt;- '//*[@id=&quot;mw-content-text&quot;]/div/table[2]'

first_div_top_three &lt;- first_div_top_three_url %>%
  html() %>%
  html_nodes(xpath = first_div_top_three_xpath) %>%
  html_table()

first_div_top_three &lt;- first_div_top_three[[1]]           

first_div_top_three &lt;- first_div_top_three %>% filter(!(Year %in% c(&quot;1915/16–1918/19&quot;,
                                                                    &quot;1939/40–1945/46&quot;)))

first_div_top_three$Goals &lt;-first_div_top_three$Goals %>% as.numeric()

first_div_top_three &lt;- first_div_top_three %>% 
  rename(`Champions` = `Champions(number of titles)`,
         `Top goalscorer` = `Leading goalscorer`)

epl_top_three_url &lt;- &quot;https://en.wikipedia.org/wiki/List_of_English_football_champions&quot;
epl_top_three_xpath &lt;- '//*[@id=&quot;mw-content-text&quot;]/div/table[3]'

epl_top_three &lt;- epl_top_three_url %>%
  html() %>%
  html_nodes(xpath = epl_top_three_xpath) %>%
  html_table()

epl_top_three &lt;- epl_top_three[[1]] 

epl_top_three &lt;- epl_top_three %>% 
  rename(`Champions` = `Champions (number of titles)`)

english_top_three_total &lt;- bind_rows(first_div_top_three,
                                     epl_top_three)

english_top_three_total$Champions &lt;- english_top_three_total$Champions %>% str_remove_all(regex(&quot;\\([^)]*\\)&quot;))

english_top_three_total$Champions &lt;- english_top_three_total$Champions %>% str_remove_all(regex(&quot;\\[.*?\\]&quot;))

english_top_three_total &lt;- english_top_three_total %>%
  mutate(short_year = str_sub(Year,1,4) %>% as.numeric() + 1)

english_top_three_total &lt;- english_top_three_total %>%
  filter(short_year > 1973) %>%
  select(-c(`Top goalscorer`,Goals))

english_top_three_total_melted &lt;- english_top_three_total %>%
  melt(id.vars=c(&quot;Year&quot;,&quot;short_year&quot;),
       value.name = &quot;Club&quot;,
       variable.name = &quot;Team_Ranking&quot;)

english_top_three_total_melted$Club &lt;- english_top_three_total_melted$Club %>% str_trim(side = c( &quot;right&quot;))

df_merged &lt;- df %>% left_join(english_top_three_total_melted,
                              by = c(&quot;Club&quot;,&quot;short_year&quot;))</pre>
<p>Step 4: More data visualizations with merged dataset</p>
<pre>club_count_year &lt;- df_merged %>% 
  count(Club, year, Team_Ranking, sort = TRUE) %>%
  mutate(club_year = paste(Club, year))

club_count_year %>% 
  top_n(n = 10, wt = n) %>% 
  ggplot(aes(x = reorder(club_year,n), 
                  y = n))+
  geom_col(aes(fill = factor(ifelse(Team_Ranking == &quot;Champions&quot;, 
                                1,
                                2)))) +
  coord_flip()+
  theme_few()+
  guides(fill=FALSE)+
  labs(y = &quot;# of Players&quot;,
       x = &quot;Club&quot;,
       title = &quot;Teams with Most Representation in PFA Team of the Year (1973 - 2017)&quot;,
       caption = &quot;Data Source: Wikipedia&quot;)+
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_fill_solarized()</pre>
<pre>club_count_year_champions &lt;- df_merged %>% 
  count(Club, year, Team_Ranking, sort = TRUE) %>%
  mutate(club_year = paste(Club, year)) %>%
  filter(Team_Ranking == &quot;Champions&quot;)

club_count_year_champions %>% 
  top_n(n = -10, wt = n) %>% 
  ggplot(aes(x = reorder(club_year,n), 
                  y = n))+
  geom_col(aes(fill = Club)) +
  coord_flip()+
  theme_few()+
  guides(fill=FALSE)+
  labs(y = &quot;# of Players&quot;,
       x = &quot;Club&quot;,
       title = &quot;English Champions with Fewest Players in PFA Team of the Year (1973 - 2017)&quot;,
       caption = &quot;Data Source: Wikipedia&quot;)+
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_y_continuous(breaks = seq(0,2,1)) +
  scale_fill_manual(values = c(&quot;Manchester United&quot; = &quot;darkred&quot;, 
                                 &quot;Liverpool&quot; = &quot;orange&quot;,
                                 &quot;Arsenal&quot; = &quot;yellow&quot;,
                                 &quot;Chelsea&quot; = &quot;blue&quot;,
                                 &quot;Blackburn Rovers&quot; = &quot;lightblue&quot;,
                                 &quot;Leeds United&quot; = &quot;grey&quot;,
                                 &quot;Manchester City&quot; = &quot;dark green&quot;,
                               &quot;Derby County&quot; = &quot;black&quot;,
                               &quot;Everton&quot; = &quot;gold&quot;,
                               &quot;Nottingham Forest&quot; = &quot;red&quot;))</pre>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://worldsocceranalytics.com/2018/10/18/analyzing-english-team-of-the-year-data-since-1973/"> World Soccer Analytics</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/FGKFEhSmTzk" height="1"></a></div>