{"id":11921,"date":"2019-02-17T22:37:14","date_gmt":"2019-02-18T03:37:14","guid":{"rendered":"https:\/\/rud.is\/b\/?p=11921"},"modified":"2019-02-17T22:37:14","modified_gmt":"2019-02-18T03:37:14","slug":"conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages","status":"publish","type":"post","link":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/","title":{"rendered":"Conquering Caffeinated Amazon Athena with the metis Trio of Packages"},"content":{"rendered":"<p>I must preface this post with the posit that if you&#8217;re doing anything <code>interactive()<\/code> with Amazon Athena you should seriously consider just using their <a href=\"https:\/\/docs.aws.amazon.com\/athena\/latest\/ug\/connect-with-odbc.html\">free ODBC drivers<\/a> as it&#8217;s the easiest way to wire them up to R DBI- and tidyverse-wise. I&#8217;ve <a href=\"https:\/\/rud.is\/b\/2018\/04\/20\/painless-odbc-dplyr-connections-to-amazon-athena-and-apache-drill-with-r-odbc\/\">said as much in previous posts<\/a>. Drop a note in the comments if you don&#8217;t know the incantations for repackaging the provided Linux ODBC drivers to work on your flavor of Linux.<\/p>\n<p><em>However<\/em>&hellip;<\/p>\n<p>There are times&mdash;say, when you&#8217;re trying to stand up an R service in your kubernetes cluster which bridges data in Athena to analyses &amp; visualizations in R&mdash;when ODBC drivers can be more of a hindrance than help and JDBC is the path of least resistance.<\/p>\n<p>Sure, there&#8217;s the in-CRAN <code>AWR.Athena<\/code> package but it&#8217;s a fairly constrained and low-feature RJDBC shim which gets the basic job done but not much more.<\/p>\n<p>Enter:<\/p>\n<ul>\n<li><a href=\"https:\/\/git.sr.ht\/~hrbrmstr\/metis-jars\"><code>metis.jars<\/code>?<\/a> | <a href=\"https:\/\/gitlab.com\/hrbrmstr\/metis-jars\">GL<\/a> | <a href=\"https:\/\/github.com\/hrbrmstr\/metis-jars\">GH<\/a>;<\/li>\n<li><a href=\"https:\/\/git.sr.ht\/~hrbrmstr\/metis\"><code>metis<\/code>?<\/a> | <a href=\"https:\/\/gitlab.com\/hrbrmstr\/metis\">GL<\/a> | <a href=\"https:\/\/github.com\/hrbrmstr\/metis\">GH<\/a>; and,<\/li>\n<li><a href=\"https:\/\/git.sr.ht\/~hrbrmstr\/metis-tidy\"><code>metis.tidy<\/code>?<\/a> | <a href=\"https:\/\/gitlab.com\/hrbrmstr\/metis-tidy\">GL<\/a> | <a href=\"https:\/\/github.com\/hrbrmstr\/metis-tidy\">GH<\/a><\/li>\n<\/ul>\n<p>a trio of packages which aims to make it super-straightforward to wire up R to Amazon Athena when ODBC is not available.<\/p>\n<h3>Why Three Packages?<\/h3>\n<p>For starters, there are CRAN hopes for the <code>metis<\/code>-trio and one key component of that is separating out the JARs into one package (<code>metis.jars<\/code>) and actual functionality into others (<code>metis<\/code> and <code>metis.tidy<\/code>). We&#8217;ll see how the CRAN attempt goes since the JAR package weighs in at sufficient weight to warrant a <code>NOTE<\/code>. The packaging of the driver reduces the need for you to pre-load the JAR (locally or into, say, a Docker image) or perform a package-initiated download-dance like <code>AWR.Athena<\/code> does (which I still don&#8217;t understand why that hasn&#8217;t kicked it out of CRAN the way it does it but \u00af&#92;_(\u30c4)_\/\u00af).<\/p>\n<p><code>metis.jars<\/code> also has three helper functions which do some (basic) fun things:<\/p>\n<pre><code class=\"language-r\">library(metis.jars)\n\nsimba_driver_version()\n## [1] \"02.00.06.1008\"\n\nathena_supported_types()\n##  [1] \"BOOLEAN\"   \"TINYINT\"   \"SMALLINT\"  \"INT\"       \"INTEGER\"  \n##  [6] \"BIGINT\"    \"REAL\"      \"FLOAT\"     \"DOUBLE\"    \"DECIMAL\"  \n## [11] \"DATE\"      \"TIMESTAMP\" \"BINARY\"    \"VARBINARY\" \"CHAR\"     \n## [16] \"VARCHAR\"   \"STRING\"    \"ARRAY\"     \"MAP\"       \"ROW\"      \n## [21] \"STRUCT\"   \n\nmetis_jar_path()\n## [1] \"\/Library\/Frameworks\/R.framework\/Versions\/3.5\/Resources\/library\/metis.jars\/java\/AthenaJDBC42_2.0.6.jar\"\n<\/code><\/pre>\n<p>The first uses the <code>rJava<\/code> interface to directly query the version (since Amazon seems to update the Simba JAR twice a year). By separating out the JAR into a separate package, updates can be made to the other two sibling packages more frequently without crushing CRAN&#8217;s disk space. <code>metis.jars<\/code> is also versioned to the included JAR so configuration management will be easier for folks.<\/p>\n<p>The reason for the second type-lister function is that there&#8217;s hope Amazon will add support for <em>all<\/em> Presto data types, especially <code>IPADDRESS<\/code>. It, again, performs JDBC driver introspection to collect the supported types.<\/p>\n<p>Finally, the third function abstracts the JAR location from the <code>metis<\/code> package or even <em>your own<\/em> interface package should you choose to depend on it.<\/p>\n<h3>OK, But Why Not Just Two?<\/h3>\n<p>The <code>metis<\/code> package is a more functional RJDBC superclass of a DBI wrapper than <code>AWR.Athena<\/code>. One thing it does that its CRAN cousin cannot is handle <code>BIGINT<\/code>s properly:<\/p>\n<pre><code class=\"language-r\">library(metis)\n\ndbConnect(\n  metis::Athena(),\n  Schema = \"sampledb\",\n  AwsCredentialsProviderClass = \"com.simba.athena.amazonaws.auth.PropertiesFileCredentialsProvider\",\n  AwsCredentialsProviderArguments = path.expand(\"~\/.aws\/athenaCredentials.props\")\n) -&gt; con\n\ndbGetQuery(con, \"\nSELECT\n  CAST('chr' AS CHAR(4)) achar,\n  CAST('varchr' AS VARCHAR) avarchr,\n  CAST(SUBSTR(timestamp, 1, 10) AS DATE) AS tsday,\n  CAST(100.1 AS DOUBLE) AS justadbl,\n  CAST(127 AS TINYINT) AS asmallint,\n  CAST(100 AS INTEGER) AS justanint,\n  CAST(100000000000000000 AS BIGINT) AS abigint,\n  CAST(('GET' = 'GET') AS BOOLEAN) AS is_get,\n  ARRAY[1, 2, 3] AS arr1,\n  ARRAY['1', '2, 3', '4'] AS arr2,\n  MAP(ARRAY['foo', 'bar'], ARRAY[1, 2]) AS mp,\n  CAST(ROW(1, 2.0) AS ROW(x BIGINT, y DOUBLE)) AS rw,\n  CAST('{\\\"a\\\":1}' AS JSON) js\nFROM elb_logs\nLIMIT 1\n\") %&gt;% \n  dplyr::glimpse()\n## Observations: 1\n## Variables: 13\n## $ achar     &lt;chr&gt; \"chr \"\n## $ avarchr   &lt;chr&gt; \"varchr\"\n## $ tsday     &lt;date&gt; 2014-09-29\n## $ justadbl  &lt;dbl&gt; 100.1\n## $ asmallint &lt;int&gt; 127\n## $ justanint &lt;int&gt; 100\n## $ abigint   &lt;S3: integer64&gt; 100000000000000000\n## $ is_get    &lt;lgl&gt; TRUE\n## $ arr1      &lt;chr&gt; \"1, 2, 3\"\n## $ arr2      &lt;chr&gt; \"1, 2, 3, 4\"\n## $ mp        &lt;chr&gt; \"{bar=2, foo=1}\"\n## $ rw        &lt;chr&gt; \"{x=1, y=2.0}\"\n## $ js        &lt;chr&gt; \"\\\"{\\\\\\\"a\\\\\\\":1}\\\"\"\n<\/code><\/pre>\n<p><strike>Presto<\/strike>Athena arrays and maps and rows and JSON come across as characters from the Athena driver and they&#8217;re formatted so badly that there&#8217;s little hope of full R support for <code>list<\/code> columns for them. <em>But<\/em>, you do get real, big integers with <code>metis<\/code> along with full support for all other current Athena types.<\/p>\n<p>R folk who may be users of the old, standalone <code>metis<\/code> package need to be aware of some things.<\/p>\n<p>First, <code>dbConnect()<\/code> has breaking changes. The snake_case names that still exist in the higher-level <code>athena_jdbc()<\/code> function are gone. In exchange for this pain, you now have full naming-parity with all the Athena JDBC connection properties and can more easily use alternate credential providers which <code>metis<\/code>&#8216; cousin <a href=\"https:\/\/github.com\/nfultz\/AWR.Athena\/blob\/master\/R\/athena.R#L77\">totally cannot do for you<\/a> which is illustrated in the example above and in the package README.<\/p>\n<p>The <code>metis<\/code> package also makes it easier to see documentation for <em>all<\/em> available Athena connection properties since it has a vignette with a descriptive table of all of them (rendered <a href=\"https:\/\/rud.is\/dl\/athena-connection-parameters.html\">here<\/a>).<\/p>\n<p>There is also nascent support for the &#8220;streaming API&#8221; (TLDR: faster result set downloads) but that won&#8217;t be fully tested until some AWS policy tweaks happen this week.<\/p>\n<h3>Gotcha. But, Why Not Just Two?<\/h3>\n<p>As awesome as it is (including base Docker image support) the <code>tidyverse<\/code> is not without overhead in terms of compilation time and dependencies, both of which are especially painful on Linux systems and some Docker environments. You can absolutely get by with some well-crafted SQL and JDBC and the thinner the image the easier it is to deploy and scale.<\/p>\n<p>But! The tidyverse is <em>so helpful<\/em> that ensuring smooth support for Athena is critical. On its own, <code>metis<\/code> wires up to <code>dplyr<\/code>\/<code>dbplyr<\/code> fine, but by providing (in <code>metis.tidy<\/code>) some enhanced <code>db_data_type()<\/code> support (primarily for <code>BIGINT<\/code>) and some extra ? in <code>sql_translate_env()<\/code> )for those of us who continue to mindlessly use R-only verbs like <code>grep()<\/code> or <code>as.POSIXct()<\/code> in non-R contexts) we can level-up <code>interactive()<\/code> use and tidyverse-infused service use:<\/p>\n<pre><code class=\"language-r\">library(metis.tidy)\nlibrary(dbplyr)\nlibrary(dplyr)\n\nmetis::dbConnect(\n  metis::Athena(),\n  Schema = \"sampledb\",\n  AwsCredentialsProviderClass = \"com.simba.athena.amazonaws.auth.PropertiesFileCredentialsProvider\",\n  AwsCredentialsProviderArguments = path.expand(\"~\/.aws\/athenaCredentials.props\")\n) -&gt; con\n\nelb_logs &lt;- tbl(con, \"elb_logs\")\n\nfilter(elb_logs, grepl(\"20\", elbresponsecode)) %&gt;%\n  mutate(\n    tsday = as.Date(substring(timestamp, 1L, 10L)),\n    host = url_extract_host(url),\n    proto_version = regexp_extract(protocol, \"([[:digit:]\\\\.]+)\"),\n  ) %&gt;%\n  select(tsday, host, receivedbytes, requestprocessingtime, proto_version) %&gt;%\n  head(1) %&gt;%\n  glimpse()\n## Observations: ??\n## Variables: 5\n## Database: AthenaConnection\n## $ tsday                 &lt;date&gt; 2014-09-29\n## $ host                  &lt;chr&gt; \"www.abcxyz.com\"\n## $ receivedbytes         &lt;S3: integer64&gt; 0\n## $ requestprocessingtime &lt;dbl&gt; 9.5e-05\n## $ proto_version         &lt;chr&gt; \"1.1\"\n<\/code><\/pre>\n<h3>FIN<\/h3>\n<p>A fairly big impetus for this radical refactoring was the need to use the Athena JDBC interface in R at $DAYJOB in a serverless context. So, if I\/we needed it, others may as well. All three packages have tests (that work with my personal Athena setup which is easily replicated since it&#8217;s just the default schema &amp; table you get when you enable Athena), pass CRAN checks and will be live in a real production environment by the time you read this.<\/p>\n<p>Note that I do have CRAN plans for these three amigos, but all three packages will need to go in at the same time and I need to get tests into- and prove tests are live in Travis before submitting. Now&#8217;s the time for feature requests, problem reports or issues. Until SourceHut&#8217;s (sr.ht) API is finished, said contributions are best left to GitLab (preferably) or GitHub (if you must continue to fill the coffers of giant multional companies that undermine your freedom).<\/p>\n<h3>POSTSCRIPT<\/h3>\n<p>One other reason for re-visiting <code>metis<\/code> was <a href=\"https:\/\/github.com\/s-u\/RJDBC\/issues\/71\">this R-crashing <code>rJava<\/code> issue<\/a> that is really a Simba Athena implementation issue (OS signals in a JDBC driver, <em>rly<\/em>?)<\/p>\n<p>This <code>Rprofile<\/code> entry:<\/p>\n<pre><code class=\"language-r\">options(\n  \"java.parameters\" = c(getOption(\"java.parameters\", default = NULL), \"-Xrs\")\n)\n<\/code><\/pre>\n<p>has been a solid workaround until <code>rJava<\/code> is updated. Note that <code>metis.jars<\/code> warns about this on load if it detects your setup is at risk.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I must preface this post with the posit that if you&#8217;re doing anything interactive() with Amazon Athena you should seriously consider just using their free ODBC drivers as it&#8217;s the easiest way to wire them up to R DBI- and tidyverse-wise. I&#8217;ve said as much in previous posts. Drop a note in the comments if [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[818,75,13,91],"tags":[],"class_list":["post-11921","post","type-post","status-publish","format-standard","hentry","category-athena","category-java","category-linux","category-r"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Conquering Caffeinated Amazon Athena with the metis Trio of Packages - rud.is<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Conquering Caffeinated Amazon Athena with the metis Trio of Packages - rud.is\" \/>\n<meta property=\"og:description\" content=\"I must preface this post with the posit that if you&#8217;re doing anything interactive() with Amazon Athena you should seriously consider just using their free ODBC drivers as it&#8217;s the easiest way to wire them up to R DBI- and tidyverse-wise. I&#8217;ve said as much in previous posts. Drop a note in the comments if [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/\" \/>\n<meta property=\"og:site_name\" content=\"rud.is\" \/>\n<meta property=\"article:published_time\" content=\"2019-02-18T03:37:14+00:00\" \/>\n<meta name=\"author\" content=\"hrbrmstr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hrbrmstr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/02\\\/17\\\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/02\\\/17\\\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\\\/\"},\"author\":{\"name\":\"hrbrmstr\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"headline\":\"Conquering Caffeinated Amazon Athena with the metis Trio of Packages\",\"datePublished\":\"2019-02-18T03:37:14+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/02\\\/17\\\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\\\/\"},\"wordCount\":993,\"commentCount\":4,\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"articleSection\":[\"athena\",\"Java\",\"Linux\",\"R\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/02\\\/17\\\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/02\\\/17\\\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\\\/\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/02\\\/17\\\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\\\/\",\"name\":\"Conquering Caffeinated Amazon Athena with the metis Trio of Packages - rud.is\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\"},\"datePublished\":\"2019-02-18T03:37:14+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/02\\\/17\\\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/02\\\/17\\\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2019\\\/02\\\/17\\\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/rud.is\\\/b\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Conquering Caffeinated Amazon Athena with the metis Trio of Packages\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/\",\"name\":\"rud.is\",\"description\":\"&quot;In God we trust. All others must bring data&quot;\",\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/rud.is\\\/b\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\",\"name\":\"hrbrmstr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"width\":460,\"height\":460,\"caption\":\"hrbrmstr\"},\"logo\":{\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\"},\"description\":\"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7\",\"sameAs\":[\"http:\\\/\\\/rud.is\"],\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/author\\\/hrbrmstr\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Conquering Caffeinated Amazon Athena with the metis Trio of Packages - rud.is","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/","og_locale":"en_US","og_type":"article","og_title":"Conquering Caffeinated Amazon Athena with the metis Trio of Packages - rud.is","og_description":"I must preface this post with the posit that if you&#8217;re doing anything interactive() with Amazon Athena you should seriously consider just using their free ODBC drivers as it&#8217;s the easiest way to wire them up to R DBI- and tidyverse-wise. I&#8217;ve said as much in previous posts. Drop a note in the comments if [&hellip;]","og_url":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/","og_site_name":"rud.is","article_published_time":"2019-02-18T03:37:14+00:00","author":"hrbrmstr","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hrbrmstr","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/#article","isPartOf":{"@id":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/"},"author":{"name":"hrbrmstr","@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"headline":"Conquering Caffeinated Amazon Athena with the metis Trio of Packages","datePublished":"2019-02-18T03:37:14+00:00","mainEntityOfPage":{"@id":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/"},"wordCount":993,"commentCount":4,"publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"articleSection":["athena","Java","Linux","R"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/","url":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/","name":"Conquering Caffeinated Amazon Athena with the metis Trio of Packages - rud.is","isPartOf":{"@id":"https:\/\/rud.is\/b\/#website"},"datePublished":"2019-02-18T03:37:14+00:00","breadcrumb":{"@id":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/rud.is\/b\/2019\/02\/17\/conquering-caffeinated-amazon-athena-with-the-metis-trio-of-packages\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/rud.is\/b\/"},{"@type":"ListItem","position":2,"name":"Conquering Caffeinated Amazon Athena with the metis Trio of Packages"}]},{"@type":"WebSite","@id":"https:\/\/rud.is\/b\/#website","url":"https:\/\/rud.is\/b\/","name":"rud.is","description":"&quot;In God we trust. All others must bring data&quot;","publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/rud.is\/b\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886","name":"hrbrmstr","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","width":460,"height":460,"caption":"hrbrmstr"},"logo":{"@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1"},"description":"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7","sameAs":["http:\/\/rud.is"],"url":"https:\/\/rud.is\/b\/author\/hrbrmstr\/"}]}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p23idr-36h","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":10121,"url":"https:\/\/rud.is\/b\/2018\/04\/20\/painless-odbc-dplyr-connections-to-amazon-athena-and-apache-drill-with-r-odbc\/","url_meta":{"origin":11921,"position":0},"title":"Painless ODBC  + dplyr Connections to Amazon Athena and Apache Drill with R &#038; odbc","author":"hrbrmstr","date":"2018-04-20","format":false,"excerpt":"I spent some time this morning upgrading the JDBC driver (and changing up some supporting code to account for changes to it) for my metis package? which connects R up to Amazon Athena via RJDBC. I'm used to JDBC and have to deal with Java separately from R so I'm\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/04\/today-is-a-good-day-to-query.jpg?fit=700%2C535&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/04\/today-is-a-good-day-to-query.jpg?fit=700%2C535&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/04\/today-is-a-good-day-to-query.jpg?fit=700%2C535&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/04\/today-is-a-good-day-to-query.jpg?fit=700%2C535&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":11070,"url":"https:\/\/rud.is\/b\/2018\/07\/14\/alleviating-aws-athena-aggravation-with-asynchronous-assistance\/","url_meta":{"origin":11921,"position":1},"title":"Alleviating AWS Athena Aggravation with Asynchronous Assistance","author":"hrbrmstr","date":"2018-07-14","format":false,"excerpt":"I've blogged about how to use Amazon Athena with R before and if you are a regular Athena user, you've likely run into a situation where you prepare a dplyr chain, fire off a collect() and then wait. And, wait. And, wait. And, wait. Queries that take significant processing time\u2026","rel":"","context":"In &quot;athena&quot;","block_context":{"text":"athena","link":"https:\/\/rud.is\/b\/category\/athena\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":11077,"url":"https:\/\/rud.is\/b\/2018\/07\/20\/a-new-boto3-amazon-athena-client-wrapper-with-dplyr-async-query-support\/","url_meta":{"origin":11921,"position":2},"title":"A new &#8216;boto3&#8217; Amazon Athena client wrapper with dplyr async query support","author":"hrbrmstr","date":"2018-07-20","format":false,"excerpt":"A previous post explored how to deal with Amazon Athena queries asynchronously. The function presented is a beast, though it is on purpose (to provide options for folks). In reality, nobody really wants to use rJava wrappers much anymore and dealing with icky Python library calls directly just feels wrong,\u2026","rel":"","context":"In &quot;athena&quot;","block_context":{"text":"athena","link":"https:\/\/rud.is\/b\/category\/athena\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":12938,"url":"https:\/\/rud.is\/b\/2021\/02\/02\/amazon-athena-dbplyr-implicit-usage-of-presto-functions-and-making-json-casting-great-again\/","url_meta":{"origin":11921,"position":3},"title":"Amazon Athena {dbplyr} Implicit Usage of Presto Functions and Making JSON Casting Great Again","author":"hrbrmstr","date":"2021-02-02","format":false,"excerpt":"I was chatting with a fellow Amazon Athena user and the topic of using Presto functions such as approx_distinct() via {d[b]plyr} came up and it seems it might not be fully common knowledge that any non-already translated function is passed to the destination intact. That means you can just \"use\"\u2026","rel":"","context":"In &quot;R&quot;","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4696,"url":"https:\/\/rud.is\/b\/2016\/12\/05\/interacting-with-amazon-athena-from-r\/","url_meta":{"origin":11921,"position":4},"title":"Interacting With Amazon Athena from R","author":"hrbrmstr","date":"2016-12-05","format":false,"excerpt":"This is a short post for those looking to test out Amazon Athena with R. Amazon makes Athena available via JDBC, so you can use RJDBC to query data. All you need is their JAR file and some setup information. Here's how to get the JAR file to the current\u2026","rel":"","context":"In &quot;R&quot;","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":12841,"url":"https:\/\/rud.is\/b\/2020\/11\/18\/apple-silicon-big-sur-rstudio-r-field-report\/","url_meta":{"origin":11921,"position":5},"title":"Apple Silicon + Big Sur + RStudio + R Field Report","author":"hrbrmstr","date":"2020-11-18","format":false,"excerpt":"It's been a while since I've posted anything R-related and, while this one will be brief, it may be of use to some R folks who have taken the leap into Big Sur and\/or Apple Silicon. Stay to the end for an early Christmas ?! Big Sur Report As #rstats\u2026","rel":"","context":"In &quot;macOS&quot;","block_context":{"text":"macOS","link":"https:\/\/rud.is\/b\/category\/macos\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/11921","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/comments?post=11921"}],"version-history":[{"count":0,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/11921\/revisions"}],"wp:attachment":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media?parent=11921"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/categories?post=11921"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/tags?post=11921"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}