{"id":11737,"date":"2019-01-09T14:32:59","date_gmt":"2019-01-09T19:32:59","guid":{"rendered":"https:\/\/rud.is\/b\/?p=11737"},"modified":"2019-01-09T16:51:43","modified_gmt":"2019-01-09T21:51:43","slug":"on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package","status":"publish","type":"post","link":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/","title":{"rendered":"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package"},"content":{"rendered":"

It was probably not difficult to discern from my previous Drill-themed post<\/a> that I’m fairly excited about the Apache Drill 1.15.0 release. I’ve rounded out most of the existing corners for it in preparation for a long-overdue CRAN update and have been concentrating on two helper features: configuring & launching Drill embedded Docker containers and auto-generation of Drill CTAS queries.<\/p>\n

Drill Docker Goodness<\/h3>\n

Starting with version 1.14.0, Apache provides Drill Docker images<\/a> for use in experimenting\/testing\/building-off-of. They run Drill in single node standalone mode so you’re not going to be running this in “production” (unless you have light or just personal workloads). Docker is a great way to get to know Drill if you haven’t already played with it since you don’t have do do much except run the Docker image.<\/p>\n

I’ve simplified this even more thanks to @rgfitzjohn’s most excellent stevedore<\/code>?<\/a> package which adds a robust R wrapper to the Docker client without relying on any heavy external dependencies such as reticulate<\/code>. The new drill_up()<\/code> function will auto-fetch the latest Drill image and launch a container so you can have a running Drill instance with virtually no effort on your part.<\/p>\n

Just running the vanilla image isn’t enough since your goal is likely to do more than work with the built-in cp<\/code> data source. The default container launch scenario also doesn’t hook up any local filesystem paths to the container so you really can’t do much other than cp<\/code>-oriented queries. Rather than make you do all the work of figuring out how to machinate Docker command line arguments and manually configure a workspace that points to a local filesystem area in the Drill web admin GUI the drill_up()<\/code> function provides a data_dir<\/code> argument (that defaults to the getwd()<\/code> where you are in your R session) which will then auto-wire up that path into the container and<\/strong> create a dfs.d<\/code> workspace which auto-points to it for you. Here’s a sample execution:<\/p>\n

library(sergeant)\nlibrary(tidyverse)\n\ndr <- drill_up(data_dir = \"~\/Data\")\n## Drill container started. Waiting for the service to become active (this may take up to 30s).\n## Drill container ID: f02a11b50e1647e44c4e233799180da3e907c8aa27900f192b5fd72acfa67ec0\n<\/code><\/pre>\n
You can use dc$stop()<\/code> to stop the container or use the printed container id to do it from the command line.<\/p>\n
We’ll use this containerized Drill instance with the next feature but I need to thank @cboettig for the suggestion to make an auto-downloader-runner-thingy<\/a> before doing that. (Thank you @cboettig!)<\/p>\n
Taking the Tedium out of CTAS<\/h3>\n
@dseverski, an intrepid R, Drill & sergeant<\/code> user noticed some new package behavior with Drill 1.15.0<\/a> that ended up spawning a new feature: automatic generation of Drill CTAS<\/a> statements.<\/p>\n
Prior to 1.14.0 sergeant<\/code> had no way to accurately, precisely tell data types of the columns coming back since the REST API didn’t provide them (as noted in the previous Drill post). Now, it did<\/em> rely on the JSON types to create the initial data frames but<\/em> id also did something **kinda horribad*: it ran readr::type_convert()<\/code> on the result sets ?. Said operation had the singular benefit of auto-converting CSV\/CSVH\/TSV\/PSV\/etc data to something sane without having to worry about writing lengthy CTAS queries (at the expense of potentially confusing everyone, though that didn’t seem to happen).<\/p>\n
With 1.15.0, the readr::type_convert()<\/code> crutch is gone, which results in less-than-helpful things like this when you have delimiter-separated values data:<\/p>\n
# using the Drill container we just started above\n\nwrite_csv(nycflights13::flights, \"~\/Data\/flights.csvh\")\n\ncon <- src_drill(\"localhost\")\n\ntbl(con, \"dfs.d.`flights.csvh`\") %>% \n  glimpse()\n## Observations: ??\n## Variables: 19\n## Database: DrillConnection\n## $ year           <chr> \"2013\", \"2013\", \"2013\", \"2013\", \"2013\", \"2013\", \"2013\", \"2013\u2026\n## $ month          <chr> \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"\u2026\n## $ day            <chr> \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"1\", \"\u2026\n## $ dep_time       <chr> \"517\", \"533\", \"542\", \"544\", \"554\", \"554\", \"555\", \"557\", \"557\"\u2026\n## $ sched_dep_time <chr> \"515\", \"529\", \"540\", \"545\", \"600\", \"558\", \"600\", \"600\", \"600\"\u2026\n## $ dep_delay      <chr> \"2\", \"4\", \"2\", \"-1\", \"-6\", \"-4\", \"-5\", \"-3\", \"-3\", \"-2\", \"-2\"\u2026\n## $ arr_time       <chr> \"830\", \"850\", \"923\", \"1004\", \"812\", \"740\", \"913\", \"709\", \"838\u2026\n## $ sched_arr_time <chr> \"819\", \"830\", \"850\", \"1022\", \"837\", \"728\", \"854\", \"723\", \"846\u2026\n## $ arr_delay      <chr> \"11\", \"20\", \"33\", \"-18\", \"-25\", \"12\", \"19\", \"-14\", \"-8\", \"8\",\u2026\n## $ carrier        <chr> \"UA\", \"UA\", \"AA\", \"B6\", \"DL\", \"UA\", \"B6\", \"EV\", \"B6\", \"AA\", \"\u2026\n## $ flight         <chr> \"1545\", \"1714\", \"1141\", \"725\", \"461\", \"1696\", \"507\", \"5708\", \u2026\n## $ tailnum        <chr> \"N14228\", \"N24211\", \"N619AA\", \"N804JB\", \"N668DN\", \"N39463\", \"\u2026\n## $ origin         <chr> \"EWR\", \"LGA\", \"JFK\", \"JFK\", \"LGA\", \"EWR\", \"EWR\", \"LGA\", \"JFK\"\u2026\n## $ dest           <chr> \"IAH\", \"IAH\", \"MIA\", \"BQN\", \"ATL\", \"ORD\", \"FLL\", \"IAD\", \"MCO\"\u2026\n## $ air_time       <chr> \"227\", \"227\", \"160\", \"183\", \"116\", \"150\", \"158\", \"53\", \"140\",\u2026\n## $ distance       <chr> \"1400\", \"1416\", \"1089\", \"1576\", \"762\", \"719\", \"1065\", \"229\", \u2026\n## $ hour           <chr> \"5\", \"5\", \"5\", \"5\", \"6\", \"5\", \"6\", \"6\", \"6\", \"6\", \"6\", \"6\", \"\u2026\n## $ minute         <chr> \"15\", \"29\", \"40\", \"45\", \"0\", \"58\", \"0\", \"0\", \"0\", \"0\", \"0\", \"\u2026\n## $ time_hour      <chr> \"2013-01-01T10:00:00Z\", \"2013-01-01T10:00:00Z\", \"2013-01-01T1\u2026\n<\/code><\/pre>\n
\"\"<\/a><\/p>\n
So the package does what it finally should have been doing all along. But, as noted, that’s not great if you just wanted to quickly work with a directory of CSV files. In theory, you’re supposed to use Drill’s CREATE TABLE AS<\/code> then do a bunch of CASTS<\/code> and TO_<\/code>s to get proper data types. But who has time for that?<\/p>\n
David had a stellar<\/em> idea, might sergeant<\/code> be able to automagically create CTAS statements from a query?<\/em>. Yes. Yes it just might be able to do that with the new ctas_profile()<\/code> function.<\/p>\n
Let’s pipe the previous tbl()<\/code> into ctas_profile()<\/code> and see what we get:<\/p>\n
tbl(con, \"dfs.d.`flights.csvh`\") %>% \n  ctas_profile() %>% \n  cat()\n<\/code><\/pre>\n
-- ** Created by ctas_profile() in the R sergeant package, version 0.8.0 **\n\nCREATE TABLE CHANGE____ME AS\nSELECT\n  CAST(`year` AS DOUBLE) AS `year`,\n  CAST(`month` AS DOUBLE) AS `month`,\n  CAST(`day` AS DOUBLE) AS `day`,\n  CAST(`dep_time` AS DOUBLE) AS `dep_time`,\n  CAST(`sched_dep_time` AS DOUBLE) AS `sched_dep_time`,\n  CAST(`dep_delay` AS DOUBLE) AS `dep_delay`,\n  CAST(`arr_time` AS DOUBLE) AS `arr_time`,\n  CAST(`sched_arr_time` AS DOUBLE) AS `sched_arr_time`,\n  CAST(`arr_delay` AS DOUBLE) AS `arr_delay`,\n  CAST(`carrier` AS VARCHAR) AS `carrier`,\n  CAST(`flight` AS DOUBLE) AS `flight`,\n  CAST(`tailnum` AS VARCHAR) AS `tailnum`,\n  CAST(`origin` AS VARCHAR) AS `origin`,\n  CAST(`dest` AS VARCHAR) AS `dest`,\n  CAST(`air_time` AS DOUBLE) AS `air_time`,\n  CAST(`distance` AS DOUBLE) AS `distance`,\n  CAST(`hour` AS DOUBLE) AS `hour`,\n  CAST(`minute` AS DOUBLE) AS `minute`,\n  TO_TIMESTAMP(`time_hour`, 'FORMATSTRING') AS `time_hour` -- *NOTE* You need to specify the format string. Sample character data is: [2013-01-01T10:00:00Z]. \nFROM (SELECT * FROM dfs.d.`flights.csvh`)\n\n\n-- TIMESTAMP and\/or DATE columns were detected.\nDrill's date\/time format string reference can be found at:\n--\n-- <http:\/\/joda-time.sourceforge.net\/apidocs\/org\/joda\/time\/format\/DateTimeFormat.html>\n<\/code><\/pre>\n
There’s a parameter for the new table name which will cause the CHANGE____ME<\/code> to go away and when the function finds TIMESTAMP<\/code> or DATE<\/code> fields it knows to switch to their TO_<\/code> cousins and gives sample data with a reminder that you need to make a format string (I’ll eventually auto-generate them unless someone PRs it first). And, since nodoby but Java programmers remember Joda format strings (they’re different than what you’re used to) it provides a handy link to them if it detects the presence of those column types.<\/p>\n
Now, we don’t need to actually create a new table (though converting a bunch of CSVs to Parquet is likely a good idea for performance reasons) to use that output. We can pass most of that new query right to tbl()<\/code>:<\/p>\n
tbl(con, sql(\"\nSELECT\n  CAST(`year` AS DOUBLE) AS `year`,\n  CAST(`month` AS DOUBLE) AS `month`,\n  CAST(`day` AS DOUBLE) AS `day`,\n  CAST(`dep_time` AS DOUBLE) AS `dep_time`,\n  CAST(`sched_dep_time` AS DOUBLE) AS `sched_dep_time`,\n  CAST(`dep_delay` AS DOUBLE) AS `dep_delay`,\n  CAST(`arr_time` AS DOUBLE) AS `arr_time`,\n  CAST(`sched_arr_time` AS DOUBLE) AS `sched_arr_time`,\n  CAST(`arr_delay` AS DOUBLE) AS `arr_delay`,\n  CAST(`carrier` AS VARCHAR) AS `carrier`,\n  CAST(`flight` AS DOUBLE) AS `flight`,\n  CAST(`tailnum` AS VARCHAR) AS `tailnum`,\n  CAST(`origin` AS VARCHAR) AS `origin`,\n  CAST(`dest` AS VARCHAR) AS `dest`,\n  CAST(`air_time` AS DOUBLE) AS `air_time`,\n  CAST(`distance` AS DOUBLE) AS `distance`,\n  CAST(`hour` AS DOUBLE) AS `hour`,\n  CAST(`minute` AS DOUBLE) AS `minute`,\n  TO_TIMESTAMP(`time_hour`, 'yyyy-MM-dd''T''HH:mm:ssZ') AS `time_hour` -- [2013-01-01T10:00:00Z].\nFROM (SELECT * FROM dfs.d.`flights.csvh`)\n\")) %>% \n  glimpse()\n## Observations: ??\n## Variables: 19\n## Database: DrillConnection\n## $ year           <dbl> 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2\u2026\n## $ month          <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1\u2026\n## $ day            <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1\u2026\n## $ dep_time       <dbl> 517, 533, 542, 544, 554, 554, 555, 557, 557, 558, 558, 558, 5\u2026\n## $ sched_dep_time <dbl> 515, 529, 540, 545, 600, 558, 600, 600, 600, 600, 600, 600, 6\u2026\n## $ dep_delay      <dbl> 2, 4, 2, -1, -6, -4, -5, -3, -3, -2, -2, -2, -2, -2, -1, 0, -\u2026\n## $ arr_time       <dbl> 830, 850, 923, 1004, 812, 740, 913, 709, 838, 753, 849, 853, \u2026\n## $ sched_arr_time <dbl> 819, 830, 850, 1022, 837, 728, 854, 723, 846, 745, 851, 856, \u2026\n## $ arr_delay      <dbl> 11, 20, 33, -18, -25, 12, 19, -14, -8, 8, -2, -3, 7, -14, 31,\u2026\n## $ carrier        <chr> \"UA\", \"UA\", \"AA\", \"B6\", \"DL\", \"UA\", \"B6\", \"EV\", \"B6\", \"AA\", \"\u2026\n## $ flight         <dbl> 1545, 1714, 1141, 725, 461, 1696, 507, 5708, 79, 301, 49, 71,\u2026\n## $ tailnum        <chr> \"N14228\", \"N24211\", \"N619AA\", \"N804JB\", \"N668DN\", \"N39463\", \"\u2026\n## $ origin         <chr> \"EWR\", \"LGA\", \"JFK\", \"JFK\", \"LGA\", \"EWR\", \"EWR\", \"LGA\", \"JFK\"\u2026\n## $ dest           <chr> \"IAH\", \"IAH\", \"MIA\", \"BQN\", \"ATL\", \"ORD\", \"FLL\", \"IAD\", \"MCO\"\u2026\n## $ air_time       <dbl> 227, 227, 160, 183, 116, 150, 158, 53, 140, 138, 149, 158, 34\u2026\n## $ distance       <dbl> 1400, 1416, 1089, 1576, 762, 719, 1065, 229, 944, 733, 1028, \u2026\n## $ hour           <dbl> 5, 5, 5, 5, 6, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5, 6, 6, 6, 6, 6\u2026\n## $ minute         <dbl> 15, 29, 40, 45, 0, 58, 0, 0, 0, 0, 0, 0, 0, 0, 0, 59, 0, 0, 0\u2026\n## $ time_hour      <dttm> 2013-01-01 10:00:00, 2013-01-01 10:00:00, 2013-01-01 10:00:0\u2026\n<\/code><\/pre>\n
Ahhhh\u2026 Useful data types. (And, see what I mean about that daft format string? Also, WP is mangling the format string so add a comment if you need the actual string.)<\/p>\n
FIN<\/h3>\n
As you can see questions, suggestions (and PRs!) are welcome and heeded on your social-coding platform of choice (though y’all still seem to be stuck on GH ?).<\/p>\n
NOTE: I’ll be subbing out most install_github()<\/code> links in READMEs and future blog posts for install_git()<\/code> counterparts pointing to my sr.ht<\/a> repos (as I co-locate\/migrate them there).<\/p>\n
You can play with the new 0.8.0 features via devtools::install_git(\"https:\/\/git.sr.ht\/~hrbrmstr\/sergeant\", ref=\"0.8.0\")<\/code>.<\/p>\n","protected":false},"excerpt":{"rendered":"
It was probably not difficult to discern from my previous Drill-themed post that I’m fairly excited about the Apache Drill 1.15.0 release. I’ve rounded out most of the existing corners for it in preparation for a long-overdue CRAN update and have been concentrating on two helper features: configuring & launching Drill embedded Docker containers and […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[819,91],"tags":[],"class_list":["post-11737","post","type-post","status-publish","format-standard","hentry","category-apache-drill","category-r"],"yoast_head":"\nOn the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package - rud.is<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package - rud.is\" \/>\n<meta property=\"og:description\" content=\"It was probably not difficult to discern from my previous Drill-themed post that I’m fairly excited about the Apache Drill 1.15.0 release. I’ve rounded out most of the existing corners for it in preparation for a long-overdue CRAN update and have been concentrating on two helper features: configuring & launching Drill embedded Docker containers and […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/\" \/>\n<meta property=\"og:site_name\" content=\"rud.is\" \/>\n<meta property=\"article:published_time\" content=\"2019-01-09T19:32:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-01-09T21:51:43+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/rud.is\/b\/wp-content\/uploads\/2019\/01\/chrs.png\" \/>\n<meta name=\"author\" content=\"hrbrmstr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hrbrmstr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/\"},\"author\":{\"name\":\"hrbrmstr\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"headline\":\"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package\",\"datePublished\":\"2019-01-09T19:32:59+00:00\",\"dateModified\":\"2019-01-09T21:51:43+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/\"},\"wordCount\":864,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2019\\\/01\\\/chrs.png\",\"articleSection\":[\"Apache Drill\",\"R\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/\",\"name\":\"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package - rud.is\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2019\\\/01\\\/chrs.png\",\"datePublished\":\"2019-01-09T19:32:59+00:00\",\"dateModified\":\"2019-01-09T21:51:43+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/#primaryimage\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2019\\\/01\\\/chrs.png?fit=750%2C500&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2019\\\/01\\\/chrs.png?fit=750%2C500&ssl=1\",\"width\":750,\"height\":500},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/01\\\/09\\\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/rud.is\\\/b\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/\",\"name\":\"rud.is\",\"description\":\""In God we trust. All others must bring data"\",\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/rud.is\\\/b\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\",\"name\":\"hrbrmstr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"width\":460,\"height\":460,\"caption\":\"hrbrmstr\"},\"logo\":{\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\"},\"description\":\"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7\",\"sameAs\":[\"http:\\\/\\\/rud.is\"],\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/author\\\/hrbrmstr\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package - rud.is","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/","og_locale":"en_US","og_type":"article","og_title":"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package - rud.is","og_description":"It was probably not difficult to discern from my previous Drill-themed post that I’m fairly excited about the Apache Drill 1.15.0 release. I’ve rounded out most of the existing corners for it in preparation for a long-overdue CRAN update and have been concentrating on two helper features: configuring & launching Drill embedded Docker containers and […]","og_url":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/","og_site_name":"rud.is","article_published_time":"2019-01-09T19:32:59+00:00","article_modified_time":"2019-01-09T21:51:43+00:00","og_image":[{"url":"https:\/\/rud.is\/b\/wp-content\/uploads\/2019\/01\/chrs.png","type":"","width":"","height":""}],"author":"hrbrmstr","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hrbrmstr","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/#article","isPartOf":{"@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/"},"author":{"name":"hrbrmstr","@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"headline":"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package","datePublished":"2019-01-09T19:32:59+00:00","dateModified":"2019-01-09T21:51:43+00:00","mainEntityOfPage":{"@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/"},"wordCount":864,"commentCount":1,"publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"image":{"@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/#primaryimage"},"thumbnailUrl":"https:\/\/rud.is\/b\/wp-content\/uploads\/2019\/01\/chrs.png","articleSection":["Apache Drill","R"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/","url":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/","name":"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package - rud.is","isPartOf":{"@id":"https:\/\/rud.is\/b\/#website"},"primaryImageOfPage":{"@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/#primaryimage"},"image":{"@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/#primaryimage"},"thumbnailUrl":"https:\/\/rud.is\/b\/wp-content\/uploads\/2019\/01\/chrs.png","datePublished":"2019-01-09T19:32:59+00:00","dateModified":"2019-01-09T21:51:43+00:00","breadcrumb":{"@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/#primaryimage","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2019\/01\/chrs.png?fit=750%2C500&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2019\/01\/chrs.png?fit=750%2C500&ssl=1","width":750,"height":500},{"@type":"BreadcrumbList","@id":"https:\/\/rud.is\/b\/2019\/01\/09\/on-the-road-to-0-8-0-some-additional-new-features-coming-in-the-sergeant-package\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/rud.is\/b\/"},{"@type":"ListItem","position":2,"name":"On the Road to 0.8.0 \u2014 Some Additional New Features Coming in the sergeant Package"}]},{"@type":"WebSite","@id":"https:\/\/rud.is\/b\/#website","url":"https:\/\/rud.is\/b\/","name":"rud.is","description":""In God we trust. All others must bring data"","publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/rud.is\/b\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886","name":"hrbrmstr","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","width":460,"height":460,"caption":"hrbrmstr"},"logo":{"@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1"},"description":"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7","sameAs":["http:\/\/rud.is"],"url":"https:\/\/rud.is\/b\/author\/hrbrmstr\/"}]}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p23idr-33j","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":11479,"url":"https:\/\/rud.is\/b\/2018\/09\/09\/driving-drill-dynamically-with-docker-and-updating-storage-configurations-on-the-fly-with-sergeant\/","url_meta":{"origin":11737,"position":0},"title":"Driving Drill Dynamically with Docker and Updating Storage Configurations On-the-fly with sergeant","author":"hrbrmstr","date":"2018-09-09","format":false,"excerpt":"The sergeant? package has a minor update that adds REST API coverage for two \"new\" storage endpoints that make it possible to add, update and remove storage configurations on-the-fly without using the GUI or manually updating a config file. This is an especially handy feature when paired with Drill's new,\u2026","rel":"","context":"In "Apache Drill"","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":12772,"url":"https:\/\/rud.is\/b\/2020\/06\/01\/sergeant-0-9-0-is-on-its-way-to-cran-mirrors\/","url_meta":{"origin":11737,"position":1},"title":"{sergeant} 0.9.0 Is On Its Way to CRAN Mirrors!","author":"hrbrmstr","date":"2020-06-01","format":false,"excerpt":"Tis been a long time coming, but a minor change to default S3 parameters in tibbles finally caused a push of {sergeant} \u2014\u00a0the R package that lets you use the Apache Drill REST API via {DBI}, {dplyr}, or directly \u2014 to CRAN. The CRAN automatic processing system approved the release\u2026","rel":"","context":"In "Apache Drill"","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":11392,"url":"https:\/\/rud.is\/b\/2018\/08\/16\/updates-to-the-sergeant-apache-drill-connector-package-apache-drill-1-14-0-release\/","url_meta":{"origin":11737,"position":2},"title":"Updates to the sergeant (Apache Drill connector) Package & a look at Apache Drill 1.14.0 release","author":"hrbrmstr","date":"2018-08-16","format":false,"excerpt":"Apache Drill 1.14.0 was recently released, bringing with it many new features and a temporary incompatibility with the current rev of the MapR ODBC drivers. The Drill community expects new ODBC drivers to arrive shortly. The sergeant? is an alternative to ODBC for R users as it provides a dplyr\u2026","rel":"","context":"In "Apache Drill"","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":6111,"url":"https:\/\/rud.is\/b\/2017\/07\/17\/ten-hut-the-apache-drill-r-interface-package-sergeant-is-now-on-cran\/","url_meta":{"origin":11737,"position":3},"title":"Ten-HUT! The Apache Drill R interface package \u2014\u00a0sergeant \u2014\u00a0is now on CRAN","author":"hrbrmstr","date":"2017-07-17","format":false,"excerpt":"I'm extremely pleased to announce that the sergeant package is now on CRAN or will be hitting your local CRAN mirror soon. sergeant provides JDBC, DBI and dplyr\/dbplyr interfaces to Apache Drill. I've also wrapped a few goodies into the dplyr custom functions that work with Drill and if you\u2026","rel":"","context":"In "Apache Drill"","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":7637,"url":"https:\/\/rud.is\/b\/2017\/12\/20\/r%e2%81%b6-series-random-sampling-from-apache-drill-tables-with-r-sergeant\/","url_meta":{"origin":11737,"position":4},"title":"R\u2076 Series \u2014 Random Sampling From Apache Drill Tables With R & sergeant","author":"hrbrmstr","date":"2017-12-20","format":false,"excerpt":"(For first-timers, R\u2076 tagged posts are short & sweet with minimal expository; R\u2076 feed) At work-work I mostly deal with medium-to-large-ish data. I often want to poke at new or existing data sets w\/o working across billions of rows. I also use Apache Drill for much of my exploratory work.\u2026","rel":"","context":"In "Apache Drill"","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":11369,"url":"https:\/\/rud.is\/b\/2018\/08\/11\/connecting-apache-zeppelin-and-apache-drill-postgresql-etc\/","url_meta":{"origin":11737,"position":5},"title":"Connecting Apache Zeppelin and Apache Drill, PostgreSQL, etc.","author":"hrbrmstr","date":"2018-08-11","format":false,"excerpt":"A previous post showed how to use a different authentication provider to wire up Apache Zeppelin and Amazon Athena. As noted in that post, Zeppelin is a \"notebook\" alternative to Jupyter (and other) notebooks. Unlike Jupyter, I can tolerate Zeppelin and it's got some nifty features like plug-and-play JDBC access.\u2026","rel":"","context":"In "Apache Drill"","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/z-drill-2.png?fit=1200%2C542&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/z-drill-2.png?fit=1200%2C542&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/z-drill-2.png?fit=1200%2C542&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/z-drill-2.png?fit=1200%2C542&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/z-drill-2.png?fit=1200%2C542&ssl=1&resize=1050%2C600 3x"},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/11737","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/comments?post=11737"}],"version-history":[{"count":0,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/11737\/revisions"}],"wp:attachment":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media?parent=11737"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/categories?post=11737"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/tags?post=11737"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}