{"id":12977,"date":"2021-03-01T07:45:14","date_gmt":"2021-03-01T12:45:14","guid":{"rendered":"https:\/\/rud.is\/b\/?p=12977"},"modified":"2021-03-01T07:45:14","modified_gmt":"2021-03-01T12:45:14","slug":"brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r","status":"publish","type":"post","link":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/","title":{"rendered":"Brimming With Possibilities: Query zqd & Mine Logs with zq from R"},"content":{"rendered":"

Brim Security<\/a> maintains a free, Electron-based desktop GUI for exploration of PCAPs and select cybersecurity logs:<\/p>\n

\"\"<\/a><\/p>\n

along with a broad ecosystem of tools<\/a> which can be used independently of the GUI.<\/p>\n

The standalone or embedded zqd<\/code> server, as well as the zq<\/code> command line utility let analysts run ZQL (a domain-specific query language) queries on cybersecurity data sources.<\/p>\n

The Brim team maintains a Python module that is capable of working with the zqd<\/code> HTTP API and my nascent {brimr}gitea<\/a>|gh<\/a>|gl<\/a>|bb<\/a><\/sup> R package provides a similar API structure to perform similar operations in R, along with a wrapper for the zq<\/code> commmand line tool.<\/p>\n

PCAPs! In! Spaaaaacce[s]!<\/h3>\n

Brim Desktop organizes input sources into something called “spaces”. We can check for available spaces with brim_spaces()<\/code>:<\/p>\n

library(brimr)\nlibrary(tibble)\n\nbrim_spaces()\n##                               id                                                            name\n## 1 sp_1p6pwLgtsESYBTHU9PL9fcl2iBn 2021-02-17-Trickbot-gtag-rob13-infection-in-AD-environment.pcap\n##                                                                                              data_path storage_kind\n## 1 file:\/\/\/Users\/demo\/Library\/Application%20Support\/Brim\/data\/spaces\/sp_1p6pwLgtsESYBTHU9PL9fcl2iBn    filestore\n<\/code><\/pre>\n
This single space availble is a sample capture of Trickbot<\/a><\/p>\n
Let’s profile the network connections in this capture:<\/p>\n
# ZQL query to fetch Zeek connection data\nzql1 <- '_path=conn | count() by id.orig_h, id.resp_h, id.resp_p | sort id.orig_h, id.resp_h, id.resp_p'\n\nspace <- \"2021-02-17-Trickbot-gtag-rob13-infection-in-AD-environment.pcap\"\n\nr1 <- brim_search(space, zql1)\n\nr1\n## ZQL query took 0.0000 seconds; 384 records matched; 1,082 records read; 238,052 bytes read\n\n(r1 <- as_tibble(tidy_brim(r1)))\n## # A tibble: 74 x 4\n##    orig_h      resp_h       resp_p count\n##    <chr>       <chr>        <chr>  <int>\n##  1 10.2.17.2   10.2.17.101  49787      1\n##  2 10.2.17.101 3.222.126.94 80         1\n##  3 10.2.17.101 10.2.17.1    445        1\n##  4 10.2.17.101 10.2.17.2    53        97\n##  5 10.2.17.101 10.2.17.2    88        27\n##  6 10.2.17.101 10.2.17.2    123        5\n##  7 10.2.17.101 10.2.17.2    135        8\n##  8 10.2.17.101 10.2.17.2    137        2\n##  9 10.2.17.101 10.2.17.2    138        2\n## 10 10.2.17.101 10.2.17.2    389       37\n## # \u2026 with 64 more rows\n<\/code><\/pre>\n
Brim auto-processed the PCAP into Zeek<\/a> log format and _path=conn<\/code> in query string indicates that’s where we’re going to perform further data operations (the queries are structured a bit like jq<\/code><\/a> filters). We then ask Brim\/zqd<\/code> to summarize and sort source IP, destination IP, and port counts. {brimr} sends this query over to the server. The raw response is a custom data structure that we can turn into a tidy data frame via tidy_brim()<\/code>.<\/p>\n
We can do something similar with the Suricata data that Brim also auto-processes for us:<\/p>\n
# Z query to fetch Suricata alerts including the count of alerts per source:destination \nzql2 <- \"event_type=alert | count() by src_ip, dest_ip, dest_port, alert.severity, alert.signature | sort src_ip, dest_ip, dest_port, alert.severity, alert.signature\"\n\nr2 <- brim_search(space, zql2)\n\nr2\n## ZQL query took 0.0000 seconds; 47 records matched; 870 records read; 238,660 bytes read\n\n(r2 <- (as_tibble(tidy_brim(r2))))\n## # A tibble: 35 x 6\n##    src_ip     dest_ip    dest_port severity signature                                                              count\n##    <chr>      <chr>          <int>    <int> <chr>                                                                  <int>\n##  1 10.2.17.2  10.2.17.1\u2026     49674        3 SURICATA Applayer Detect protocol only one direction                       1\n##  2 10.2.17.2  10.2.17.1\u2026     49680        3 SURICATA Applayer Detect protocol only one direction                       1\n##  3 10.2.17.2  10.2.17.1\u2026     49687        3 SURICATA Applayer Detect protocol only one direction                       1\n##  4 10.2.17.2  10.2.17.1\u2026     49704        3 SURICATA Applayer Detect protocol only one direction                       1\n##  5 10.2.17.2  10.2.17.1\u2026     49709        3 SURICATA Applayer Detect protocol only one direction                       1\n##  6 10.2.17.2  10.2.17.1\u2026     49721        3 SURICATA Applayer Detect protocol only one direction                       1\n##  7 10.2.17.2  10.2.17.1\u2026     50126        3 SURICATA Applayer Detect protocol only one direction                       1\n##  8 10.2.17.1\u2026 3.222.126\u2026        80        2 ET POLICY curl User-Agent Outbound                                         1\n##  9 10.2.17.1\u2026 36.95.27.\u2026       443        1 ET HUNTING Suspicious POST with Common Windows Process Names - Possib\u2026     1\n## 10 10.2.17.1\u2026 36.95.27.\u2026       443        1 ET MALWARE Win32\/Trickbot Data Exfiltration                                1\n## # \u2026 with 25 more rows\n<\/code><\/pre>\n
Finally, for this toy example, we’ll also generate a visual overview of these connections:<\/p>\n
library(igraph)\nlibrary(ggraph)\nlibrary(tidyverse)\n\ngdf <- count(r1, orig_h, resp_h, wt=count)\n\ncount(gdf, node = resp_h, wt=n, name = \"in_degree\") %>% \n  full_join(\n    count(gdf, node = orig_h, name = \"out_degree\")\n  ) %>% \n  mutate_at(\n    vars(in_degree, out_degree),\n    replace_na, 1\n  ) %>% \n  arrange(in_degree) -> vdf\n\ng <- graph_from_data_frame(gdf, vertices = vdf)\n\nggraph(g, layout = \"linear\") +\n  geom_node_point(\n    aes(size = in_degree), shape = 21\n  ) +\n  geom_edge_arc(\n    width = 0.125, \n    arrow = arrow(\n      length = unit(5, \"pt\"),\n      type = \"closed\"\n    )\n  )\n<\/code><\/pre>\n
\"\"<\/a><\/p>\n
We can also process log files directly (i.e. without any server) with zq_cmd()<\/code>:<\/p>\n
zq_cmd(\n  c(\n    '\"* | cut ts,id.orig_h,id.orig_p\"', # note the quotes\n    system.file(\"logs\", \"conn.log.gz\", package = \"brimr\")\n   )\n )\n##           id.orig_h id.orig_p                          ts\n##   1:  10.164.94.120     39681 2018-03-24T17:15:21.255387Z\n##   2:    10.47.25.80     50817 2018-03-24T17:15:21.411148Z\n##   3:    10.47.25.80     50817 2018-03-24T17:15:21.926018Z\n##   4:    10.47.25.80     50813 2018-03-24T17:15:22.690601Z\n##   5:    10.47.25.80     50813 2018-03-24T17:15:23.205187Z\n##  ---                                                     \n## 988: 10.174.251.215     33003 2018-03-24T17:15:21.429238Z\n## 989: 10.174.251.215     33003 2018-03-24T17:15:21.429315Z\n## 990: 10.174.251.215     33003 2018-03-24T17:15:21.429479Z\n## 991:  10.164.94.120     38265 2018-03-24T17:15:21.427375Z\n## 992: 10.174.251.215     33003 2018-03-24T17:15:21.433306Z\n<\/code><\/pre>\n
FIN<\/h3>\n
This package is less than 24 hrs old (as of the original blog post date) and there are still a few bits missing, which means y’all have the ability to guide the direction it heads in. So kick the tyres and interact where you’re most comfortable.<\/p>\n","protected":false},"excerpt":{"rendered":"
Brim Security maintains a free, Electron-based desktop GUI for exploration of PCAPs and select cybersecurity logs: along with a broad ecosystem of tools which can be used independently of the GUI. The standalone or embedded zqd server, as well as the zq command line utility let analysts run ZQL (a domain-specific query language) queries on […]<\/p>\n","protected":false},"author":1,"featured_media":12980,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[681,91],"tags":[],"class_list":["post-12977","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cybersecurity","category-r"],"yoast_head":"\nBrimming With Possibilities: Query zqd & Mine Logs with zq from R - rud.is<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Brimming With Possibilities: Query zqd & Mine Logs with zq from R - rud.is\" \/>\n<meta property=\"og:description\" content=\"Brim Security maintains a free, Electron-based desktop GUI for exploration of PCAPs and select cybersecurity logs: along with a broad ecosystem of tools which can be used independently of the GUI. The standalone or embedded zqd server, as well as the zq command line utility let analysts run ZQL (a domain-specific query language) queries on […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/\" \/>\n<meta property=\"og:site_name\" content=\"rud.is\" \/>\n<meta property=\"article:published_time\" content=\"2021-03-01T12:45:14+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/03\/brimr-graph.png?fit=1728%2C960&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"1728\" \/>\n\t<meta property=\"og:image:height\" content=\"960\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"hrbrmstr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hrbrmstr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/\"},\"author\":{\"name\":\"hrbrmstr\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"headline\":\"Brimming With Possibilities: Query zqd & Mine Logs with zq from R\",\"datePublished\":\"2021-03-01T12:45:14+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/\"},\"wordCount\":316,\"commentCount\":2,\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/brimr-graph.png?fit=1728%2C960&ssl=1\",\"articleSection\":[\"Cybersecurity\",\"R\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/\",\"name\":\"Brimming With Possibilities: Query zqd & Mine Logs with zq from R - rud.is\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/brimr-graph.png?fit=1728%2C960&ssl=1\",\"datePublished\":\"2021-03-01T12:45:14+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/#primaryimage\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/brimr-graph.png?fit=1728%2C960&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/brimr-graph.png?fit=1728%2C960&ssl=1\",\"width\":1728,\"height\":960},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2021\\\/03\\\/01\\\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/rud.is\\\/b\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Brimming With Possibilities: Query zqd & Mine Logs with zq from R\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/\",\"name\":\"rud.is\",\"description\":\""In God we trust. All others must bring data"\",\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/rud.is\\\/b\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\",\"name\":\"hrbrmstr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"width\":460,\"height\":460,\"caption\":\"hrbrmstr\"},\"logo\":{\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\"},\"description\":\"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7\",\"sameAs\":[\"http:\\\/\\\/rud.is\"],\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/author\\\/hrbrmstr\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Brimming With Possibilities: Query zqd & Mine Logs with zq from R - rud.is","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/","og_locale":"en_US","og_type":"article","og_title":"Brimming With Possibilities: Query zqd & Mine Logs with zq from R - rud.is","og_description":"Brim Security maintains a free, Electron-based desktop GUI for exploration of PCAPs and select cybersecurity logs: along with a broad ecosystem of tools which can be used independently of the GUI. The standalone or embedded zqd server, as well as the zq command line utility let analysts run ZQL (a domain-specific query language) queries on […]","og_url":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/","og_site_name":"rud.is","article_published_time":"2021-03-01T12:45:14+00:00","og_image":[{"width":1728,"height":960,"url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/03\/brimr-graph.png?fit=1728%2C960&ssl=1","type":"image\/png"}],"author":"hrbrmstr","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hrbrmstr","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/#article","isPartOf":{"@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/"},"author":{"name":"hrbrmstr","@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"headline":"Brimming With Possibilities: Query zqd & Mine Logs with zq from R","datePublished":"2021-03-01T12:45:14+00:00","mainEntityOfPage":{"@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/"},"wordCount":316,"commentCount":2,"publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"image":{"@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/03\/brimr-graph.png?fit=1728%2C960&ssl=1","articleSection":["Cybersecurity","R"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/","url":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/","name":"Brimming With Possibilities: Query zqd & Mine Logs with zq from R - rud.is","isPartOf":{"@id":"https:\/\/rud.is\/b\/#website"},"primaryImageOfPage":{"@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/#primaryimage"},"image":{"@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/03\/brimr-graph.png?fit=1728%2C960&ssl=1","datePublished":"2021-03-01T12:45:14+00:00","breadcrumb":{"@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/#primaryimage","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/03\/brimr-graph.png?fit=1728%2C960&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/03\/brimr-graph.png?fit=1728%2C960&ssl=1","width":1728,"height":960},{"@type":"BreadcrumbList","@id":"https:\/\/rud.is\/b\/2021\/03\/01\/brimming-with-possibilities-query-zqd-mine-logs-with-zq-from-r\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/rud.is\/b\/"},{"@type":"ListItem","position":2,"name":"Brimming With Possibilities: Query zqd & Mine Logs with zq from R"}]},{"@type":"WebSite","@id":"https:\/\/rud.is\/b\/#website","url":"https:\/\/rud.is\/b\/","name":"rud.is","description":""In God we trust. All others must bring data"","publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/rud.is\/b\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886","name":"hrbrmstr","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","width":460,"height":460,"caption":"hrbrmstr"},"logo":{"@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1"},"description":"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7","sameAs":["http:\/\/rud.is"],"url":"https:\/\/rud.is\/b\/author\/hrbrmstr\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/03\/brimr-graph.png?fit=1728%2C960&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p23idr-3nj","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":13120,"url":"https:\/\/rud.is\/b\/2021\/07\/20\/packet-maze-solving-a-cyberdefenders-pcap-puzzle-with-r-zeek-and-tshark\/","url_meta":{"origin":12977,"position":0},"title":"Packet Maze: Solving a CyberDefenders PCAP Puzzle with R, Zeek, and tshark","author":"hrbrmstr","date":"2021-07-20","format":false,"excerpt":"It was a rainy weekend in southern Maine and I really didn't feel like doing chores, so I was skimming through RSS feeds and noticed a link to a PacketMaze challenge in the latest This Week In 4n6. Since it's also been a while since I've done any serious content\u2026","rel":"","context":"In "Cybersecurity"","block_context":{"text":"Cybersecurity","link":"https:\/\/rud.is\/b\/category\/cybersecurity\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":8230,"url":"https:\/\/rud.is\/b\/2018\/02\/16\/pym-js-library-vulnerability-in-widgetframe-package\/","url_meta":{"origin":12977,"position":1},"title":"Pym.js Library Vulnerability in widgetframe Package","author":"hrbrmstr","date":"2018-02-16","format":false,"excerpt":"What's Up? The NPR Visuals Team created and maintains a javascript library that makes it super easy to embed iframes on web pages and have said documents still be responsive. The widgetframe R htmlwidget uses pym.js to bring this (much needed) functionality into widgets and (eventually) shiny apps. NPR reported\u2026","rel":"","context":"In "Cybersecurity"","block_context":{"text":"Cybersecurity","link":"https:\/\/rud.is\/b\/category\/cybersecurity\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":24954,"url":"https:\/\/rud.is\/b\/2025\/04\/17\/trumps-retaliation-against-chris-krebs-and-the-cybersecurity-industrys-deafening-silence\/","url_meta":{"origin":12977,"position":2},"title":"Trump\u2019s Retaliation Against Chris Krebs \u2014 and the Cybersecurity Industry\u2019s Deafening Silence","author":"hrbrmstr","date":"2025-04-17","format":false,"excerpt":"Chris Krebs, the former director of the Cybersecurity and Infrastructure Security Agency (CISA), was fired by Donald Trump in 2020 for publicly affirming that the presidential election was secure and free from widespread fraud. Fast-forward to April 2025: Trump, now back in the White House, issued an executive order revoking\u2026","rel":"","context":"In "Commentary"","block_context":{"text":"Commentary","link":"https:\/\/rud.is\/b\/category\/commentary\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2025\/04\/kalea-morgan-zFPTvo0aZ0g-unsplash.jpg?fit=1129%2C1200&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2025\/04\/kalea-morgan-zFPTvo0aZ0g-unsplash.jpg?fit=1129%2C1200&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2025\/04\/kalea-morgan-zFPTvo0aZ0g-unsplash.jpg?fit=1129%2C1200&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2025\/04\/kalea-morgan-zFPTvo0aZ0g-unsplash.jpg?fit=1129%2C1200&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2025\/04\/kalea-morgan-zFPTvo0aZ0g-unsplash.jpg?fit=1129%2C1200&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":25134,"url":"https:\/\/rud.is\/b\/2025\/04\/29\/rsac-2025-sets-a-dangerous-precedent-for-cybersecurity-leadership\/","url_meta":{"origin":12977,"position":3},"title":"RSAC 2025 Sets A Dangerous Precedent for Cybersecurity Leadership","author":"hrbrmstr","date":"2025-04-29","format":false,"excerpt":"(I posted this on LI, but I like to own my content, so am also posting here.) The cybersecurity community deserves better than what we're witnessing at RSAC 2025, today. While Kristi Noem delivers today's keynote, the absence of traditional cybersecurity leaders from agencies like NSA and CISA speaks volumes\u2026","rel":"","context":"In "Commentary"","block_context":{"text":"Commentary","link":"https:\/\/rud.is\/b\/category\/commentary\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4852,"url":"https:\/\/rud.is\/b\/2017\/01\/08\/2017-01-authored-package-updates\/","url_meta":{"origin":12977,"position":4},"title":"2017-01 Authored Package Updates","author":"hrbrmstr","date":"2017-01-08","format":false,"excerpt":"The rest of the month is going to be super-hectic and it's unlikely I'll be able to do any more to help the push to CRAN 10K, so here's a breakdown of CRAN and GitHub new packages & package updates that I felt were worth raising awareness on: epidata I\u2026","rel":"","context":"In "dplyr"","block_context":{"text":"dplyr","link":"https:\/\/rud.is\/b\/category\/dplyr\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/01\/epi2.png?fit=982%2C1200&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/01\/epi2.png?fit=982%2C1200&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/01\/epi2.png?fit=982%2C1200&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2017\/01\/epi2.png?fit=982%2C1200&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":3775,"url":"https:\/\/rud.is\/b\/2015\/11\/08\/visualizing_survey_data\/","url_meta":{"origin":12977,"position":5},"title":"Visualizing Survey Data : Comparison Between Observations","author":"hrbrmstr","date":"2015-11-08","format":false,"excerpt":"Cybersecurity is a domain that really likes surveys, or at the very least it has many folks within it that like to conduct and report on surveys. One recent survey on threat intelligence is in it's second year, so it sets about comparing answers across years. Rather than go into\u2026","rel":"","context":"In "Cybersecurity"","block_context":{"text":"Cybersecurity","link":"https:\/\/rud.is\/b\/category\/cybersecurity\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2015\/11\/Visualizing_Survey_Data___Comparison_Between_Observations.png?fit=1200%2C721&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2015\/11\/Visualizing_Survey_Data___Comparison_Between_Observations.png?fit=1200%2C721&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2015\/11\/Visualizing_Survey_Data___Comparison_Between_Observations.png?fit=1200%2C721&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2015\/11\/Visualizing_Survey_Data___Comparison_Between_Observations.png?fit=1200%2C721&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2015\/11\/Visualizing_Survey_Data___Comparison_Between_Observations.png?fit=1200%2C721&ssl=1&resize=1050%2C600 3x"},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/12977","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/comments?post=12977"}],"version-history":[{"count":0,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/12977\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media\/12980"}],"wp:attachment":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media?parent=12977"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/categories?post=12977"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/tags?post=12977"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}