{"id":11479,"date":"2018-09-09T15:58:06","date_gmt":"2018-09-09T20:58:06","guid":{"rendered":"https:\/\/rud.is\/b\/?p=11479"},"modified":"2018-09-09T15:58:06","modified_gmt":"2018-09-09T20:58:06","slug":"driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant","status":"publish","type":"post","link":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/","title":{"rendered":"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant"},"content":{"rendered":"<p>The <a href=\"gitlab.com\/hrbrmstr\/sergeant\"><code>sergeant<\/code>?<\/a> package has a minor update that adds REST API coverage for <a href=\"https:\/\/drill.apache.org\/docs\/rest-api-introduction\/\">two &#8220;new&#8221; <code>storage<\/code> endpoints<\/a> that make it possible to add, update and remove storage configurations on-the-fly without using the GUI or manually updating a config file.<\/p>\n<p>This is an especially handy feature when paired with Drill&#8217;s new, official <a href=\"https:\/\/drill.apache.org\/docs\/running-drill-on-docker\/\">Docker container<\/a> since that means we can:<\/p>\n<ul>\n<li>fire up a clean Drill instance<\/li>\n<li>modify the storage configuration (to, say, point to a local file system directory)<\/li>\n<li>execute SQL ops<\/li>\n<li>destroy the Drill instance<\/li>\n<\/ul>\n<p>all from within R.<\/p>\n<p>This is even more handy for those of us who prefer handling JSON data in Drill than in R directly or with <code>sparklyr<\/code>.<\/p>\n<h3>Quick Example<\/h3>\n<p>In a few weeks most of the following verbose-code-snippets will have a more diminutive and user-friendly interface within <code>sergeant<\/code>, but for now we&#8217;ll perform the above bulleted steps with some data that was used in <a href=\"https:\/\/gitlab.com\/hrbrmstr\/dnshelpers\/\">a recent new package<\/a> which was also generated by another <a href=\"https:\/\/gitlab.com\/hrbrmstr\/zdnsr\">recent new package<\/a>. The <code>zdnsr::zdns_exec()<\/code> function ultimately generates a deeply nested JSON file that I really prefer working with in Drill before shunting it into R. Said file is stored, say, in the <code>~\/drilldat<\/code> directory.<\/p>\n<p>Now, I have Drill running all the time on almost every system I use, but we&#8217;ll pretend I don&#8217;t for this example. I&#8217;ve run <code>zdns_exec()<\/code> and generated the JSON file and it&#8217;s in the aforementioned directory. Let&#8217;s fire up an instance and connect to it:<\/p>\n<pre><code class=\"language-r\">library(sergeant) # git[hu|la]b:hrbrmstr\/sergeant\nlibrary(dplyr)\n\ndocker <- Sys.which(\"docker\") # you do need docker. it's a big dependency, but worth it IMO\n\n(system2(\n  command = docker,  \n  args = c(\n    \"run\", \"-i\", \n    \"--name\", \"drill-1.14.0\", \n    \"-p\", \"8047:8047\", \n    \"-v\", paste0(c(path.expand(\"~\/drilldat\"), \"\/drilldat\"), collapse=\":\"),\n    \"--detach\", \n    \"-t\", \"drill\/apache-drill:1.14.0\",\n    \"\/bin\/bash\"\n  ),\n  stdout = TRUE\n) -> drill_container)\n## [1] \"d6bc79548fa073d3bfbd32528a12669d753e7a19a6258e1be310e1db378f0e0d\"<\/code><\/pre>\n<p>The above snippet fires up a Drill Docker container (downloads it, too, if not already local) and wires up a virtual directory to it.<\/p>\n<p>We should wait a couple seconds and make sure we can connect to it:<\/p>\n<pre><code class=\"language-r\">drill_connection() %>% \n  drill_active()\n## [1] TRUE<\/code><\/pre>\n<p>Now, we need to add a storage configuration so we can access our virtual directory. Rather than modify <code>dfs<\/code> we&#8217;ll add a <code>drilldat<\/code> plugin that will work with the local filesystem just like <code>dfs<\/code> does:<\/p>\n<pre><code class=\"language-r\">drill_connection() %>%\n  drill_mod_storage(\n    name = \"drilldat\",\n    config = '\n{\n  \"config\" : {\n    \"connection\" : \"file:\/\/\/\", \n    \"enabled\" : true,\n    \"formats\" : null,\n    \"type\" : \"file\",\n    \"workspaces\" : {\n      \"root\" : {\n        \"location\" : \"\/drilldat\",\n        \"writable\" : true,\n        \"defaultInputFormat\": null\n      }\n    }\n  },\n  \"name\" : \"drilldat\"\n}\n')\n## $result\n## [1] \"success\"<\/code><\/pre>\n<p>Now, we can perform all the Drill ops <code>sergeant<\/code> has to offer, including ones like this:<\/p>\n<pre><code class=\"language-r\">(db <- src_drill(\"localhost\"))\n## src:  DrillConnection\n## tbls: cp.default, dfs.default, dfs.root, dfs.tmp, drilldat.default, drilldat.root,\n##   INFORMATION_SCHEMA, sys\n\ntbl(db, \"drilldat.root.`\/*.json`\")\n## # Source:   table<drilldat.root.`\/*.json`> [?? x 10]\n## # Database: DrillConnection\n##    data                                              name   error class status timestamp          \n##    <chr>                                             <chr>  <lgl> <chr> <chr>  <dttm>             \n##  1 \"{\\\"authorities\\\":[{\\\"ttl\\\":180,\\\"type\\\":\\\"SOA\\\"\u2026 _dmar\u2026 NA    IN    NOERR\u2026 2018-09-09 13:18:07\n##  2 \"{\\\"authorities\\\":[],\\\"protocol\\\":\\\"udp\\\",\\\"flag\u2026 _dmar\u2026 NA    IN    NXDOM\u2026 2018-09-09 13:18:07\n##  3 \"{\\\"authorities\\\":[],\\\"protocol\\\":\\\"udp\\\",\\\"flag\u2026 _dmar\u2026 NA    IN    NXDOM\u2026 2018-09-09 13:18:07\n##  4 \"{\\\"authorities\\\":[],\\\"protocol\\\":\\\"udp\\\",\\\"flag\u2026 _dmar\u2026 NA    IN    NXDOM\u2026 2018-09-09 13:18:07\n##  5 \"{\\\"authorities\\\":[],\\\"protocol\\\":\\\"udp\\\",\\\"flag\u2026 _dmar\u2026 NA    IN    NXDOM\u2026 2018-09-09 13:18:07\n##  6 \"{\\\"authorities\\\":[{\\\"ttl\\\":1799,\\\"type\\\":\\\"SOA\\\u2026 _dmar\u2026 NA    IN    NOERR\u2026 2018-09-09 13:18:07\n##  7 \"{\\\"authorities\\\":[],\\\"protocol\\\":\\\"udp\\\",\\\"flag\u2026 _dmar\u2026 NA    IN    NXDOM\u2026 2018-09-09 13:18:07\n##  8 \"{\\\"authorities\\\":[],\\\"protocol\\\":\\\"udp\\\",\\\"flag\u2026 _dmar\u2026 NA    IN    NXDOM\u2026 2018-09-09 13:18:07\n##  9 \"{\\\"authorities\\\":[],\\\"protocol\\\":\\\"udp\\\",\\\"flag\u2026 _dmar\u2026 NA    IN    NXDOM\u2026 2018-09-09 13:18:07\n## 10 \"{\\\"authorities\\\":[],\\\"protocol\\\":\\\"udp\\\",\\\"flag\u2026 _dmar\u2026 NA    IN    NOERR\u2026 2018-09-09 13:18:07\n## # ... with more rows\n\n(tbl(db, \"(\nSELECT\n b.answers.name AS question,\n b.answers.answer AS answer\nFROM (\n SELECT \n   FLATTEN(a.data.answers) AS answers\n FROM \n   drilldat.root.`\/*.json` a\n WHERE \n   (a.status = 'NOERROR')\n) b\n)\") %>% \n collect() -> dmarc_recs)\n## # A tibble: 1,250 x 2\n##    question             answer                                                                    \n##  * <chr>                <chr>                                                                     \n##  1 _dmarc.washjeff.edu  v=DMARC1; p=none                                                          \n##  2 _dmarc.barry.edu     v=DMARC1; p=none; rua=mailto:dmpost@barry.edu,mailto:7cc566d7@mxtoolbox.d\u2026\n##  3 _dmarc.yhc.edu       v=DMARC1; pct=100; p=none                                                 \n##  4 _dmarc.aacc.edu      v=DMARC1;p=none; rua=mailto:DKIM_DMARC@aacc.edu;ruf=mailto:DKIM_DMARC@aac\u2026\n##  5 _dmarc.sagu.edu      v=DMARC1; p=none; rua=mailto:Office365contact@sagu.edu; ruf=mailto:Office\u2026\n##  6 _dmarc.colostate.edu v=DMARC1; p=none; pct=100; rua=mailto:re+anahykughvo@dmarc.postmarkapp.co\u2026\n##  7 _dmarc.wne.edu       v=DMARC1;p=quarantine;sp=none;fo=1;ri=86400;pct=50;rua=mailto:dmarcreply@\u2026\n##  8 _dmarc.csuglobal.edu v=DMARC1; p=none;                                                         \n##  9 _dmarc.devry.edu     v=DMARC1; p=none; pct=100; rua=mailto:devry@rua.agari.com; ruf=mailto:dev\u2026\n## 10 _dmarc.sullivan.edu  v=DMARC1; p=none; rua=mailto:mcambron@sullivan.edu; ruf=mailto:mcambron@s\u2026\n## # ... with 1,240 more rows<\/code><\/pre>\n<p>Finally (when done), we can terminate the Drill container:<\/p>\n<pre><code class=\"language-r\">system2(\n  command = \"docker\",\n  args = c(\"rm\", \"-f\", drill_container)\n)<\/code><\/pre>\n<h3>FIN<\/h3>\n<p>Those <code>system2()<\/code> calls are hard on the ? and a pain to type\/remember, so they&#8217;ll be wrapped in some <code>sergeant<\/code> utility functions (I&#8217;m hesitant to add a <code>reticulate<\/code> dependency to <code>sergeant<\/code> which is necessary to use the <code>docker<\/code> package, hence the system call wrapper approach).<\/p>\n<p>Check your favorite repository for more <code>sergeant<\/code> updates and file issues if you have suggestions for how you&#8217;d like this Docker API for Drill to be controlled.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The sergeant? package has a minor update that adds REST API coverage for two &#8220;new&#8221; storage endpoints that make it possible to add, update and remove storage configurations on-the-fly without using the GUI or manually updating a config file. This is an especially handy feature when paired with Drill&#8217;s new, official Docker container since that [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[819,781,91],"tags":[],"class_list":["post-11479","post","type-post","status-publish","format-standard","hentry","category-apache-drill","category-drill","category-r"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant - rud.is<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant - rud.is\" \/>\n<meta property=\"og:description\" content=\"The sergeant? package has a minor update that adds REST API coverage for two &#8220;new&#8221; storage endpoints that make it possible to add, update and remove storage configurations on-the-fly without using the GUI or manually updating a config file. This is an especially handy feature when paired with Drill&#8217;s new, official Docker container since that [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/\" \/>\n<meta property=\"og:site_name\" content=\"rud.is\" \/>\n<meta property=\"article:published_time\" content=\"2018-09-09T20:58:06+00:00\" \/>\n<meta name=\"author\" content=\"hrbrmstr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hrbrmstr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/\"},\"author\":{\"name\":\"hrbrmstr\",\"@id\":\"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886\"},\"headline\":\"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant\",\"datePublished\":\"2018-09-09T20:58:06+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/\"},\"wordCount\":415,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886\"},\"articleSection\":[\"Apache Drill\",\"drill\",\"R\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/\",\"url\":\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/\",\"name\":\"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant - rud.is\",\"isPartOf\":{\"@id\":\"https:\/\/rud.is\/b\/#website\"},\"datePublished\":\"2018-09-09T20:58:06+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/rud.is\/b\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/rud.is\/b\/#website\",\"url\":\"https:\/\/rud.is\/b\/\",\"name\":\"rud.is\",\"description\":\"&quot;In God we trust. All others must bring data&quot;\",\"publisher\":{\"@id\":\"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/rud.is\/b\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886\",\"name\":\"hrbrmstr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1\",\"url\":\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1\",\"width\":460,\"height\":460,\"caption\":\"hrbrmstr\"},\"logo\":{\"@id\":\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1\"},\"description\":\"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7\",\"sameAs\":[\"http:\/\/rud.is\"],\"url\":\"https:\/\/rud.is\/b\/author\/hrbrmstr\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant - rud.is","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/","og_locale":"en_US","og_type":"article","og_title":"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant - rud.is","og_description":"The sergeant? package has a minor update that adds REST API coverage for two &#8220;new&#8221; storage endpoints that make it possible to add, update and remove storage configurations on-the-fly without using the GUI or manually updating a config file. This is an especially handy feature when paired with Drill&#8217;s new, official Docker container since that [&hellip;]","og_url":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/","og_site_name":"rud.is","article_published_time":"2018-09-09T20:58:06+00:00","author":"hrbrmstr","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hrbrmstr","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/#article","isPartOf":{"@id":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/"},"author":{"name":"hrbrmstr","@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"headline":"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant","datePublished":"2018-09-09T20:58:06+00:00","mainEntityOfPage":{"@id":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/"},"wordCount":415,"commentCount":1,"publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"articleSection":["Apache Drill","drill","R"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/","url":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/","name":"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant - rud.is","isPartOf":{"@id":"https:\/\/rud.is\/b\/#website"},"datePublished":"2018-09-09T20:58:06+00:00","breadcrumb":{"@id":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/rud.is\/b\/"},{"@type":"ListItem","position":2,"name":"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant"}]},{"@type":"WebSite","@id":"https:\/\/rud.is\/b\/#website","url":"https:\/\/rud.is\/b\/","name":"rud.is","description":"&quot;In God we trust. All others must bring data&quot;","publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/rud.is\/b\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886","name":"hrbrmstr","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","width":460,"height":460,"caption":"hrbrmstr"},"logo":{"@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1"},"description":"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7","sameAs":["http:\/\/rud.is"],"url":"https:\/\/rud.is\/b\/author\/hrbrmstr\/"}]}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p23idr-2Z9","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":12772,"url":"https:\/\/rud.is\/b\/2020\/06\/01\/sergeant-0-9-0-is-on-its-way-to-cran-mirrors\/","url_meta":{"origin":11479,"position":0},"title":"{sergeant} 0.9.0 Is On Its Way to CRAN Mirrors!","author":"hrbrmstr","date":"2020-06-01","format":false,"excerpt":"Tis been a long time coming, but a minor change to default S3 parameters in tibbles finally caused a push of {sergeant} \u2014\u00a0the R package that lets you use the Apache Drill REST API via {DBI}, {dplyr}, or directly \u2014 to CRAN. The CRAN automatic processing system approved the release\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":11737,"url":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/","url_meta":{"origin":11479,"position":1},"title":"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package","author":"hrbrmstr","date":"2019-01-09","format":false,"excerpt":"It was probably not difficult to discern from my previous Drill-themed post that I'm fairly excited about the Apache Drill 1.15.0 release. I've rounded out most of the existing corners for it in preparation for a long-overdue CRAN update and have been concentrating on two helper features: configuring & launching\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4929,"url":"https:\/\/rud.is\/b\/2017\/01\/22\/create-parquet-files-from-r-data-frames-with-sergeant-apache-drill-a-k-a-make-parquet-files-great-again-in-r\/","url_meta":{"origin":11479,"position":2},"title":"Create Parquet Files From R Data Frames With sergeant &#038; Apache Drill (a.k.a. Make Parquet Files Great Again in R)","author":"hrbrmstr","date":"2017-01-22","format":false,"excerpt":"2021-11-04 UPDATE: Just use {arrow}. Apache Drill is a nice tool to have in the toolbox as it provides a SQL front-end to a wide array of database and file back-ends and runs in standalone\/embedded mode on every modern operating system (i.e. you can get started with or play locally\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4753,"url":"https:\/\/rud.is\/b\/2016\/12\/20\/sergeant-a-r-boot-camp-for-apache-drill\/","url_meta":{"origin":11479,"position":3},"title":"sergeant : An R Boot Camp for Apache Drill","author":"hrbrmstr","date":"2016-12-20","format":false,"excerpt":"I recently mentioned that I've been working on a development version of an Apache Drill R package called sergeant. Here's a lifted \"TLDR\" on Drill: Drill supports a variety of NoSQL databases and file systems, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift,\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":6127,"url":"https:\/\/rud.is\/b\/2017\/07\/27\/reading-pcap-files-with-apache-drill-and-the-sergeant-r-package\/","url_meta":{"origin":11479,"position":4},"title":"Reading PCAP Files with Apache Drill and the sergeant R Package","author":"hrbrmstr","date":"2017-07-27","format":false,"excerpt":"It's no secret that I'm a fan of Apache Drill. One big strength of the platform is that it normalizes the access to diverse data sources down to ANSI SQL calls, which means that I can pull data from parquet, Hie, HBase, Kudu, CSV, JSON, MongoDB and MariaDB with the\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":6111,"url":"https:\/\/rud.is\/b\/2017\/07\/17\/ten-hut-the-apache-drill-r-interface-package-sergeant-is-now-on-cran\/","url_meta":{"origin":11479,"position":5},"title":"Ten-HUT! The Apache Drill R interface package \u2014\u00a0sergeant \u2014\u00a0is now on CRAN","author":"hrbrmstr","date":"2017-07-17","format":false,"excerpt":"I'm extremely pleased to announce that the sergeant package is now on CRAN or will be hitting your local CRAN mirror soon. sergeant provides JDBC, DBI and dplyr\/dbplyr interfaces to Apache Drill. I've also wrapped a few goodies into the dplyr custom functions that work with Drill and if you\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/11479","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/comments?post=11479"}],"version-history":[{"count":0,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/11479\/revisions"}],"wp:attachment":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media?parent=11479"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/categories?post=11479"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/tags?post=11479"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}