Research about adequacy of Elo based models applied to snooker match results. Contains a novel approach (EloBeta) targeted for sport results with variable &quot;best of N&quot; format.
Prologue
For many years I’ve been following snooker as a sport. It has it all: hypnotic beauty of smart play, elegance of cue strikes and psychological tension of competition. The one thing I don’t like that much is its ranking system.
Generally speaking, current snooker ranking is based on player accomplishments in tournaments (events) with different “weights” for different tournaments. Long time ago, it just used World Championships. Then, after more events had emerged, there was a table of points player could earn for winning at certain stage of tournament. Now it has the form of “rolling” sum of prize money player won during (roughly) past two calendar years.
This system has two main advantages: it is simple (win more money -__ rise in rankings) and predictabile (want to get certain ranking -__ win certain amount of money, other things being equal). The problem is that this type of rankings doesn’t account the strength (skill, form) of player’s opponents. The usual counter-argument for this is that if player reached high stage of tournament then he/she is “strong” at this moment of time (“weak players don’t win tournaments”). Well, it does sound quite convincing. However, in snooker, as in any sport, the role of chance should be taken into account: if player is “weaker” it doesn’t mean that he can’t ever win in a match with “stronger” player. It means that this happens less often then the other way around. Here where Elo model comes into play.
The idea behind Elo model is that each player is associated with numerical rating. The assumption is that a result of a game between two players can be predicted based on difference of their ratings: more value indicates more probability of “stronger” (with higher rating) player to win. Elo ranking is based on current player “strength” derived by wins against other players. This eliminates main disadvantage of current official ranking system. It is also capable of updating player rating during tournament to numerically react to player’s strong tournament performance.
Having some practical experience with Elo ratings, I think it can do well in snooker too. However, there is one obstacle: it is devised for competitions with uniform type of games. Yes, there are some variations to account for home field advantage in football or first move advantage in chess (both by adding fixed amount of rating points to “less advantageous” player). In snooker, however, matches are played in the “best of \(N\)” format: the first one to win \(n = \frac{N + 1}{2}\) frames wins a match. We will also call this format “\(n\) to win”.
Intuitively, winning a “10 to win” match (final of major tournament) should be harder for “weaker” player then “4 to win” match (first rounds of current Home Nations tournaments). This is taken into account by my proposed EloBeta model.
To celebrate actual start of 2018/19 snooker season I decided to write this post in which I explore adequacy of both Elo and EloBeta models on snooker match results. Note that the goal is not to build models for forecasting and gambling purposes but for assessing players “strength” and creating “fair” ranking.
The idea of using Elo rating in snooker is not new at all. There are works on this topic, for example:
Snooker Analyst provides “Elo inspired” (more like Bradley–Terry model) rating system based on the idea of updating rating based on difference between “actual frames won” and “expected frames won”. This approach is a little bit questionable. Surely, more frame difference should indicate more difference in strength, however, achieving that is not player’s goal. In snooker aim is “just” to win match, i.e. get certain amount of frame wins before the opponent.
This forum discussion with implementation of basic Elo model.
Possibly, there are other works that I’ve missed. I will highly appreciate any information on this topic.
Overview
This post is intended for both R users interested in Elo ratings and snooker analysts and fans. All experiments are written with intention to be reproducible. All code is hidden under spoilers (text appears after clicking on its summary, usually beginning with “Code for …”). It has commentaries and uses tidyverse packages, so it might be an interesting read for R users and programmers.
This post is organized as follows:
Models describes Elo and EloBeta models with their R implementations.
Experiment describes details and intentions of computations: which data and methodology were used (and why) and what are the results.
Exploration of EloBeta ratings has application results of EloBeta model to actual snooker data. This section is written more for Snooker fans than for R enthusiasts.
We will need the following setup:
Code for setup
# Data wrangling packages
suppressPackageStartupMessages(library(dplyr))
library(tidyr)
library(purrr)
# Visualization package
library(ggplot2)
# Package for ratings
suppressPackageStartupMessages(library(comperank))
theme_set(theme_bw())
# Shouldn't be needed. Added just in case.
set.seed(20180703)
Models
Both models are based on the following assumptions:
There is a fixed set of players which should be ranked from “strongest” (first place) to “weakest” (last place).
Ranking is done by associating player \(i\) with numerical rating \(r_i\): a number indicating the “strength” of player (more value -__ “stronger”).
The more difference in player ratings before the match the less favorable is “weaker” player to win it.
Ratings are updated after every match based on its result and the ratings before it.
Winning against “stronger” opponent should lead to bigger increase in rating than winning against “weaker” opponent. The opposite should be true for losing.
Elo
Code for Elo model
#' @details This function is vectorized by all its arguments. Also usage of
#' `...` is crucial to allow supplying unrelated arguments in the future.
#' #' @return A probability of player 1 (rating `rating1`) wins in a match with
#' player 2 (rating `rating2`). Here difference in ratings directly affects
#' the outcome.
elo_win_prob %
add_iterative_ratings(
rate_fun = rate_fun_gen(cur_k), initial_ratings = initial_ratings
) %__%
left_join(y = matches %__% select(game, matchType), by = &quot;game&quot;) %__%
filter(matchType %in% test_type) %__%
mutate(
# Number of frames needed to win in a match
framesToWin = pmax(score1, score2),
# Probability of player 1 winning a match with `frame_to_win` frames
# needed to win.
winProb = get_win_prob(
rating1 = rating1Before, rating2 = rating2Before,
frames_to_win = framesToWin
),
result = get_match_result(score1, score2),
squareError = (result - winProb)^2
) %__%
summarise(goodness = sqrt(mean(squareError)))
}) %__%
mutate(k = k_vec) %__%
select(k, goodness)
}
#' A wrapper for `compute_goodness()` to be used with design matrix data.
compute_goodness_wrap %
left_join(
y = snooker_players %__% select(id, playerName = name), by = c(player = &quot;id&quot;)
) %__%
mutate(rankEloBeta = order(ratingEloBeta, decreasing = TRUE)) %__%
select(player, playerName, ratingEloBeta, rankEloBeta)
elobeta_top16 %
filter(rankEloBeta %
mutate(
rankChr = formatC(rankEloBeta, width = 2, format = &quot;d&quot;, flag = &quot;0&quot;),
ratingEloBeta = round(ratingEloBeta, 1)
)
Top 16 by EloBeta model at the end of 2017/18 season looks like this (official data is also taken from snooker.org site):
Player
EloBeta rank
EloBeta rating
Official rank
Official rating
EloBeta rank increase
Ronnie O’Sullivan
1
128.8
2
905 750
1
Mark J Williams
2
123.4
3
878 750
1
John Higgins
3
112.5
4
751 525
1
Mark Selby
4
102.4
1
1 315 275
-3
Judd Trump
5
92.2
5
660 250
0
Barry Hawkins
6
83.1
7
543 225
1
Ding Junhui
7
82.8
6
590 525
-1
Stuart Bingham
8
74.3
13
324 587
5
Ryan Day
9
71.9
16
303 862
7
Neil Robertson
10
70.6
10
356 125
0
Shaun Murphy
11
70.1
8
453 875
-3
Kyren Wilson
12
70.1
9
416 250
-3
Jack Lisowski
13
68.8
26
180 862
13
Stephen Maguire
14
63.7
17
291 025
3
Mark Allen
15
63.7
12
332 450
-3
Yan Bingtao
16
61.6
23
215 125
7
Some observations:
Current official #1 Mark Selby is ranked 3 places lower in EloBeta. This might be a sign that current distribution of prize money doesn’t quite reflect efforts needed to win them (on average).
Most “underrated” players according to official ranking are Jack Lisowski (astonishing 13 place difference), Ryan Day and Yan Bingtao (both have 7 place difference).
Stuart Bingham is ranked 5 positions higher by EloBeta probably because he didn’t play for six month due to WPBSA ban. His EloBeta rating didn’t change during this period but in official rating he lost points because of its “rolling” nature. This case demonstrates one important differences between two approaches: official system is good to account for “not playing” and EloBeta is good to account for “playing”.
Judd Trump and Neil Robertson are ranked the same under both methods.
With EloBeta model Allister Carter (officially ranked #11), Anthony McGill (#14) and Luca Brecel (#15) are not in top 16. Instead, Jack Lisowski (#26), Yan Bingtao (#23) and Stephen Maguire (#17) are in.
Here is an example of predictions Based on EloBeta model. The probability of 16-th player (Yan Bingtao) win one frame in a match against first player (Ronnie O’Sullivan) equals to 0.404. In “4 to win” match it drops to 0.299, in “10 to win” - 0.197 and in World Championship final “18 to win” - 0.125. In my opinion, these numbers might be close to reality.
Collective evolution of EloBeta ratings
Code for rating evolution
# Helper data
seasons_break %
gather_to_longcr() %__%
left_join(y = pro_matches_test_off %__% select(game, endDate), by = &quot;game&quot;)
# Generate plot
plot_all_elobeta_history %
mutate(isTop16 = player %in% elobeta_top16$player) %__%
ggplot(aes(endDate, ratingAfter, group = player)) +
geom_step(data = . %__% filter(!isTop16), colour = &quot;#C2DF9A&quot;) +
geom_step(data = . %__% filter(isTop16), colour = &quot;#22A01C&quot;) +
geom_hline(yintercept = 0, colour = &quot;#AAAAAA&quot;) +
geom_vline(
xintercept = seasons_break, linetype = &quot;dotted&quot;,
colour = &quot;#E41A1C&quot;, size = 1
) +
geom_text(
x = seasons_break, y = Inf, label = &quot;End of 2016/17&quot;,
colour = &quot;#E41A1C&quot;, hjust = 1.05, vjust = 1.2
) +
scale_x_datetime(date_labels = &quot;%Y-%m&quot;) +
labs(
x = NULL, y = &quot;EloBeta rating&quot;,
title = paste0(
&quot;Most of current top 16 established at the end of 2016/17 season&quot;
),
subtitle = paste0(
&quot;Winning of event is well noticable as rapid increase without &quot;,
&quot;descrease at the end.&quot;
)
) +
theme(title = element_text(size = 14))
}
plot_all_elobeta_history(elobeta_history)
Evolution of EloBeta top 16
Code for rating evolution of top 16
# Compute plot data
top16_rating_evolution %
# Using `inner_join` to leave only players from `elobeta_top16`
inner_join(y = elobeta_top16 %__% select(-ratingEloBeta), by = &quot;player&quot;) %__%
# Leave games only from 2017/18 season
semi_join(
y = pro_matches_test_off %__% filter(season == 2017), by = &quot;game&quot;
) %__%
mutate(playerLabel = paste(rankChr, playerName))
# Generate plot
plot_top16_elobeta_history