Abstract
In this post, I will analyse the data I scraped and put into an R package, which I called {nethack}.
NetHack is a roguelike game; for more context, read my previous blog
post.
You can install the {nethack} package and play around with the data yourself by installing it from github:
devtools::install_github(&quot;b-rodrigues/nethack&quot;)
And to use it:
library(nethack)
data(&quot;nethack&quot;)
The data contains information on games played from 2001 to 2018; 322485 rows and 14 columns. I
will analyze the data in a future blog post. This post focuses on getting and then sharing the
data. By the way, all the content from the public server I scrape is under the CC BY 4.0 license.
I built the package by using the very useful {devtools} package.
Introduction
What I want from this first analysis are several, simple things: how many players manage to ascend
(meaning, winning), what monster kills most players, and finally extract data from the dumplog
column. The dumplog column is a bit special; each element of the dumplog column is a log file
that contains a lot of information from the last turns of a player. I will leave this for a future
blog post, though.
Let’s load some packages first:
library(nethack)
library(tidyverse)
library(lubridate)
library(magrittr)
library(brotools)
{brotools} is my own package that contains some functions that I use daily. If you want to
install it, run the following line:
devtools::install_github(&quot;b-rodrigues/brotools&quot;)
The documentation is not up-to-date, I think I’ll do that and release it on CRAN. Some day.
Now, let’s load the “nethack” data, included in the {nethack} package:
data(&quot;nethack&quot;)
head(nethack)
## rank score name time turns lev_max hp_max role race gender alignment
## 1 1 360 jkm NA 2/2 -2/25 Sam Hum Mal Law
## 2 2 172 yosemite NA 1/1 -1/10 Tou Hum Fem Neu
## 3 3 2092 dtype NA 6/7 -2/47 Val Hum Fem Neu
## 4 4 32 joorko NA 1/1 0/15 Sam Hum Mal Law
## 5 5 118 jorko NA 1/1 0/11 Rog Orc Fem Cha
## 6 6 1757 aaronl NA 5/5 0/60 Bar Hum Mal Neu
## death date
## 1 killed by a brown mold 2001-10-24
## 2 killed by a jackal 2001-10-24
## 3 killed by a fire ant 2001-10-24
## 4 killed by a jackal 2001-10-24
## 5 killed by a jackal 2001-10-24
## 6 killed by a hallucinogen-distorted ghoul, while helpless 2001-10-24
## dumplog
## 1 NA
## 2 NA
## 3 NA
## 4 NA
## 5 NA
## 6 NA
Let’s create some variables that might be helpful (or perhaps not, we’ll see):
nethack %% mutate(date = ymd(date),
year = year(date),
month = month(date),
day = day(date))
This makes it easy to look at the data from, say, June 2017:
nethack %__%
filter(year == 2017, month == 6) %__%
brotools::describe()
## # A tibble: 15 x 13
## variable type nobs mean sd mode min max q25 median
## ## 1 day Nume… 1451 17.4 9.00e0 1 1 30 10 19
## 2 month Nume… 1451 6 0. 6 6 6 6 6
## 3 rank Nume… 1451 47.1 2.95e1 1 1 100 20 47
## 4 score Nume… 1451 38156. 3.39e5 488 0 5966425 402. 953
## 5 turns Nume… 1451 4179. 1.23e4 812 1 291829 860. 1796
## 6 year Nume… 1451 2017 0. 2017 2017 2017 2017 2017
## 7 alignme… Char… 1451 NA NA Law NA NA NA NA
## 8 death Char… 1451 NA NA kill… NA NA NA NA
## 9 gender Char… 1451 NA NA Mal NA NA NA NA
## 10 hp_max Char… 1451 NA NA -1/16 NA NA NA NA
## 11 lev_max Char… 1451 NA NA 4/4 NA NA NA NA
## 12 name Char… 1451 NA NA ohno… NA NA NA NA
## 13 race Char… 1451 NA NA Hum NA NA NA NA
## 14 role Char… 1451 NA NA Kni NA NA NA NA
## 15 time Char… 1451 NA NA 01:1… NA NA NA NA
## # ... with 3 more variables: q75 , n_missing , n_unique Let’s also take a look at a dumplog:
Click to expand; the dumplog is quite long
nethack %__%
filter(year == 2018, month == 10) %__%
slice(1) %__%
pull(dumplog)
## [[1]]
## [1] &quot;Unix NetHack Version 3.6.1 - last build Fri Apr 27 19:25:48 2018. (d4ebae12f1a709d1833cf466dd0c553fb97518d2)&quot;
## [2] &quot;&quot; ## [3] &quot;Game began 2018-09-30 22:27:18, ended 2018-10-01 00:01:12.&quot; ## [4] &quot;&quot; ## [5] &quot;brothertrebius, neutral female gnomish Ranger&quot; ## [6] &quot;&quot; ## [7] &quot; -----&quot; ## [8] &quot; -------- |....# ----- --------&quot; ## [9] &quot; |/..%.=| #...^|######|...| ##.......|&quot; ## [10] &quot; |/[%..%| #|...| #|...| # |......|&quot; ## [11] &quot; |......| #----- #-...-######......#&quot; ## [24] &quot; -----&quot; ## [25] &quot;&quot; ## [26] &quot;Brothertre the Trailblazer St:15 Dx:12 Co:16 In:13 Wi:15 Ch:6 Neutral&quot; ## [27] &quot;Dlvl:6 $:59 HP:0(54) Pw:40(40) AC:0 Exp:8 T:7398 Satiated Burdened&quot; ## [28] &quot;&quot; ## [29] &quot;Latest messages:&quot; ## [30] &quot; In what direction? l&quot; ## [31] &quot; You shoot 2 arrows.&quot; ## [32] &quot; The 1st arrow hits the ape.&quot; ## [33] &quot; The 2nd arrow hits the ape!&quot; ## [34] &quot; The ape hits!&quot; ## [35] &quot; The ape hits!&quot; ## [36] &quot; The ape bites!&quot; ## [37] &quot; You ready: q - 9 uncursed arrows.&quot; ## [38] &quot; In what direction? l&quot; ## [39] &quot; The arrow hits the ape.&quot; ## [40] &quot; The ape hits!&quot; ## [41] &quot; The ape hits!&quot; ## [42] &quot; The ape bites!&quot; ## [43] &quot; The ape hits!&quot; ## [44] &quot; The ape hits!&quot; ## [45] &quot; The ape misses!&quot; ## [46] &quot; In what direction? l&quot; ## [47] &quot; You shoot 2 arrows.&quot; ## [48] &quot; The 1st arrow hits the ape!&quot; ## [49] &quot; The 2nd arrow hits the ape.&quot; ## [50] &quot; The ape misses!&quot; ## [51] &quot; The ape hits!&quot; ## [52] &quot; The ape misses!&quot; ## [53] &quot; In what direction? l&quot; ## [54] &quot; You shoot 2 arrows.&quot; ## [55] &quot; The 1st arrow misses the ape.&quot; ## [56] &quot; The 2nd arrow hits the ape.&quot; ## [57] &quot; The ape misses!&quot; ## [58] &quot; The ape hits!&quot; ## [59] &quot; The ape bites!&quot; ## [60] &quot; In what direction? l&quot; ## [61] &quot; The arrow hits the ape!&quot; ## [62] &quot; The ape hits!&quot; ## [63] &quot; The ape misses!&quot; ## [64] &quot; The ape bites!&quot; ## [65] &quot; You hear someone cursing shoplifters.&quot; ## [66] &quot; The ape misses!&quot; ## [67] &quot; The ape hits!&quot; ## [68] &quot; The ape bites!&quot; ## [69] &quot; What do you want to write with? [- amnqsvBJM-OWZ or ?*] -&quot; ## [70] &quot; You write in the dust with your fingertip.&quot; ## [71] &quot; What do you want to write in the dust here? Elbereth&quot; ## [72] &quot; The ape hits!&quot; ## [73] &quot; The ape hits!&quot; ## [74] &quot; You die...&quot; ## [75] &quot; Do you want your possessions identified? [ynq] (y) y&quot; ## [76] &quot; Do you want to see your attributes? [ynq] (y) n&quot; ## [77] &quot; Do you want an account of creatures vanquished? [ynaq] (y) n&quot; ## [78] &quot; Do you want to see your conduct? [ynq] (y) n&quot; ## [79] &quot; Do you want to see the dungeon overview? [ynq] (y) q&quot; ## [80] &quot;&quot; ## [81] &quot;Inventory:&quot; ## [82] &quot; Coins&quot; ## [83] &quot; $ - 59 gold pieces&quot; ## [84] &quot; Weapons&quot; ## [85] &quot; m - 17 blessed +1 arrows&quot; ## [86] &quot; n - a blessed +0 arrow&quot; ## [87] &quot; q - 3 +0 arrows (in quiver)&quot; ## [88] &quot; s - a +0 bow (weapon in hand)&quot; ## [89] &quot; B - 11 +1 darts&quot; ## [90] &quot; N - 11 +0 darts&quot; ## [91] &quot; a - a +1 dagger (alternate weapon; not wielded)&quot; ## [92] &quot; Armor&quot; ## [93] &quot; T - an uncursed +0 dwarvish iron helm (being worn)&quot; ## [94] &quot; z - an uncursed +0 pair of leather gloves (being worn)&quot; ## [95] &quot; U - a cursed -4 pair of iron shoes (being worn)&quot; ## [96] &quot; e - an uncursed +2 cloak of displacement (being worn)&quot; ## [97] &quot; h - a blessed +0 dwarvish mithril-coat (being worn)&quot; ## [98] &quot; Comestibles&quot; ## [99] &quot; f - 3 uncursed cram rations&quot; ## [100] &quot; j - 2 uncursed food rations&quot; ## [101] &quot; L - an uncursed food ration&quot; ## [102] &quot; P - an uncursed lembas wafer&quot; ## [103] &quot; I - an uncursed lizard corpse&quot; ## [104] &quot; o - an uncursed tin of spinach&quot; ## [105] &quot; Scrolls&quot; ## [106] &quot; G - 2 uncursed scrolls of blank paper&quot; ## [107] &quot; t - an uncursed scroll of confuse monster&quot; ## [108] &quot; V - an uncursed scroll of identify&quot; ## [109] &quot; Potions&quot; ## [110] &quot; x - an uncursed potion of gain ability&quot; ## [111] &quot; H - a blessed potion of sleeping&quot; ## [112] &quot; g - 3 uncursed potions of water&quot; ## [113] &quot; Rings&quot; ## [114] &quot; O - an uncursed ring of slow digestion (on left hand)&quot; ## [115] &quot; v - an uncursed ring of stealth (on right hand)&quot; ## [116] &quot; Tools&quot; ## [117] &quot; p - an uncursed magic lamp&quot; ## [118] &quot; k - an uncursed magic whistle&quot; ## [119] &quot; Q - an uncursed mirror&quot; ## [120] &quot; C - an uncursed saddle&quot; ## [121] &quot; D - an uncursed stethoscope&quot; ## [122] &quot; y - a +0 unicorn horn&quot; ## [123] &quot; i - 7 uncursed wax candles&quot; ## [124] &quot; Gems/Stones&quot; ## [125] &quot; W - an uncursed flint stone&quot; ## [126] &quot; M - an uncursed worthless piece of red glass&quot; ## [127] &quot; Z - an uncursed worthless piece of violet glass&quot; ## [128] &quot; J - an uncursed worthless piece of white glass&quot; ## [129] &quot;&quot; ## [130] &quot;Brothertrebius the Ranger's attributes:&quot; ## [131] &quot;&quot; ## [132] &quot;Background:&quot; ## [133] &quot; You were a Trailblazer, a level 8 female gnomish Ranger.&quot; ## [134] &quot; You were neutral, on a mission for Venus&quot; ## [135] &quot; who was opposed by Mercury (lawful) and Mars (chaotic).&quot; ## [136] &quot;&quot; ## [137] &quot;Final Characteristics:&quot; ## [138] &quot; You had 0 hit points (max:54).&quot; ## [139] &quot; You had 40 magic power (max:40).&quot; ## [140] &quot; Your armor class was 0.&quot; ## [141] &quot; You had 1552 experience points.&quot; ## [142] &quot; You entered the dungeon 7398 turns ago.&quot; ## [143] &quot; Your strength was 15 (limit:18/50).&quot; ## [144] &quot; Your dexterity was 12 (limit:18).&quot; ## [145] &quot; Your constitution was 16 (limit:18).&quot; ## [146] &quot; Your intelligence was 13 (limit:19).&quot; ## [147] &quot; Your wisdom was 15 (limit:18).&quot; ## [148] &quot; Your charisma was 6 (limit:18).&quot; ## [149] &quot;&quot; ## [150] &quot;Final Status:&quot; ## [151] &quot; You were satiated.&quot; ## [152] &quot; You were burdened; movement was slightly slowed.&quot; ## [153] &quot; You were wielding a bow.&quot; ## [154] &quot;&quot; ## [155] &quot;Final Attributes:&quot; ## [156] &quot; You were piously aligned.&quot; ## [157] &quot; You were telepathic.&quot; ## [158] &quot; You had automatic searching.&quot; ## [159] &quot; You had infravision.&quot; ## [160] &quot; You were displaced.&quot; ## [161] &quot; You were stealthy.&quot; ## [162] &quot; You had slower digestion.&quot; ## [163] &quot; You were guarded.&quot; ## [164] &quot; You are dead.&quot; ## [165] &quot;&quot; ## [166] &quot;Vanquished creatures:&quot; ## [167] &quot; a warhorse&quot; ## [168] &quot; a tengu&quot; ## [169] &quot; a quivering blob&quot; ## [170] &quot; an iron piercer&quot; ## [171] &quot; 2 black lights&quot; ## [172] &quot; a gold golem&quot; ## [173] &quot; a werewolf&quot; ## [174] &quot; 3 lizards&quot; ## [175] &quot; 2 dingoes&quot; ## [176] &quot; a housecat&quot; ## [177] &quot; a white unicorn&quot; ## [178] &quot; 2 dust vortices&quot; ## [179] &quot; a plains centaur&quot; ## [180] &quot; an ape&quot; ## [181] &quot; a Woodland-elf&quot; ## [182] &quot; 2 soldier ants&quot; ## [183] &quot; a bugbear&quot; ## [184] &quot; an imp&quot; ## [185] &quot; a wood nymph&quot; ## [186] &quot; a water nymph&quot; ## [187] &quot; a rock piercer&quot; ## [188] &quot; a pony&quot; ## [189] &quot; 3 fog clouds&quot; ## [190] &quot; a yellow light&quot; ## [191] &quot; a violet fungus&quot; ## [192] &quot; 2 gnome lords&quot; ## [193] &quot; 2 gnomish wizards&quot; ## [194] &quot; 2 gray oozes&quot; ## [195] &quot; 2 elf zombies&quot; ## [196] &quot; a straw golem&quot; ## [197] &quot; a paper golem&quot; ## [198] &quot; 2 giant ants&quot; ## [199] &quot; 2 little dogs&quot; ## [200] &quot; 3 floating eyes&quot; ## [201] &quot; 8 dwarves&quot; ## [202] &quot; a homunculus&quot; ## [203] &quot; 3 kobold lords&quot; ## [204] &quot; 3 kobold shamans&quot; ## [205] &quot; 13 hill orcs&quot; ## [206] &quot; 4 rothes&quot; ## [207] &quot; 2 centipedes&quot; ## [208] &quot; 3 giant bats&quot; ## [209] &quot; 6 dwarf zombies&quot; ## [210] &quot; a werejackal&quot; ## [211] &quot; 3 iguanas&quot; ## [212] &quot; 23 killer bees&quot; ## [213] &quot; an acid blob&quot; ## [214] &quot; a coyote&quot; ## [215] &quot; 3 gas spores&quot; ## [216] &quot; 5 hobbits&quot; ## [217] &quot; 7 manes&quot; ## [218] &quot; 2 large kobolds&quot; ## [219] &quot; a hobgoblin&quot; ## [220] &quot; 2 giant rats&quot; ## [221] &quot; 2 cave spiders&quot; ## [222] &quot; a yellow mold&quot; ## [223] &quot; 6 gnomes&quot; ## [224] &quot; 8 garter snakes&quot; ## [225] &quot; 2 gnome zombies&quot; ## [226] &quot; 8 geckos&quot; ## [227] &quot; 11 jackals&quot; ## [228] &quot; 5 foxes&quot; ## [229] &quot; 2 kobolds&quot; ## [230] &quot; 2 goblins&quot; ## [231] &quot; a sewer rat&quot; ## [232] &quot; 6 grid bugs&quot; ## [233] &quot; 3 lichens&quot; ## [234] &quot; 2 kobold zombies&quot; ## [235] &quot; 5 newts&quot; ## [236] &quot;206 creatures vanquished.&quot; ## [237] &quot;&quot; ## [238] &quot;No species were genocided or became extinct.&quot; ## [239] &quot;&quot; ## [240] &quot;Voluntary challenges:&quot; ## [241] &quot; You never genocided any monsters.&quot; ## [242] &quot; You never polymorphed an object.&quot; ## [243] &quot; You never changed form.&quot; ## [244] &quot; You used no wishes.&quot; ## [245] &quot;&quot; ## [246] &quot;The Dungeons of Doom: levels 1 to 6&quot; ## [247] &quot; Level 1:&quot; ## [248] &quot; A fountain.&quot; ## [249] &quot; Level 2:&quot; ## [250] &quot; A sink.&quot; ## [251] &quot; Level 3:&quot; ## [252] &quot; A general store, a fountain.&quot; ## [253] &quot; Level 4:&quot; ## [254] &quot; A general store, a fountain.&quot; ## [255] &quot; Stairs down to The Gnomish Mines.&quot; ## [256] &quot; Level 5:&quot; ## [257] &quot; A fountain.&quot; ## [258] &quot; Level 6: