

{"id":6146,"date":"2017-08-01T14:50:11","date_gmt":"2017-08-01T19:50:11","guid":{"rendered":"https:\/\/rud.is\/b\/?p=6146"},"modified":"2018-03-07T17:07:29","modified_gmt":"2018-03-07T22:07:29","slug":"r%e2%81%b6-reticulating-parquet-files","status":"publish","type":"post","link":"https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/","title":{"rendered":"R\u2076 \u2014 Reticulating Parquet Files"},"content":{"rendered":"<p>The <code>reticulate<\/code> package provides a very clean &amp; concise interface bridge between R and Python which makes it handy to work with modules that have yet to be ported to R (going native is always better when you can do it). This post shows how to use <code>reticulate<\/code> to create parquet files directly from R using <code>reticulate<\/code> as a bridge to the <code>pyarrow<\/code> module, which has the ability to natively create parquet files.<\/p>\n<p>Now, you can <a href=\"https:\/\/rud.is\/b\/2017\/01\/22\/create-parquet-files-from-r-data-frames-with-sergeant-apache-drill-a-k-a-make-parquet-files-great-again-in-r\/\">create parquet files through R with Apache Drill<\/a> &mdash; and, I&#8217;ll provide another example for that here &mdash; but, you may have need to generate such files and not have the ability to run Drill.<\/p>\n<p>The Python parquet process is pretty simple since you can convert a <code>pandas<\/code> <code>DataFrame<\/code> directly to a <code>pyarrow<\/code> <code>Table<\/code> which can be written out in parquet format with <code>pyarrow.parquet<\/code>. We just need to follow this process through <code>reticulate<\/code> in R:<\/p>\n<pre id=\"reticulatepq01\"><code class=\"language-r\">library(reticulate)\r\n\r\npd &lt;- import(&quot;pandas&quot;, &quot;pd&quot;)\r\npa &lt;- import(&quot;pyarrow&quot;, &quot;pa&quot;)\r\npq &lt;- import(&quot;pyarrow.parquet&quot;, &quot;pq&quot;)\r\n\r\nmtcars_py &lt;- r_to_py(mtcars)\r\nmtcars_df &lt;- pd$DataFrame$from_dict(mtcars_py)\r\nmtcars_tab &lt;- pa$Table$from_pandas(mtcars_df)\r\n\r\npq$write_table(mtcars_tab, path.expand(&quot;~\/Data\/mtcars_python.parquet&quot;))<\/code><\/pre>\n<p>I wouldn&#8217;t want to do that for ginormous data frames, but it should work pretty well for modest use cases (you&#8217;re likely using Spark, Drill, Presto or other &#8220;big data&#8221; platforms for creation of larger parquet structures). Here&#8217;s how we&#8217;d do that with Drill via the <code>sergeant<\/code> package:<\/p>\n<pre id=\"reticulatepq02\"><code class=\"language-r\">readr::write_csv(mtcars, &quot;~\/Data\/mtcars_r.csvh&quot;)\r\ndc &lt;- drill_connection(&quot;localhost&quot;)\r\ndrill_query(dc, &quot;CREATE TABLE dfs.tmp.`\/mtcars_r.parquet` AS SELECT * FROM dfs.root.`\/Users\/bob\/Data\/mtcars_r.csvh`&quot;)<\/code><\/pre>\n<p>Without additional configuration parameters, the reticulated-Python version (above) generates larger parquet files and also has an index column since they&#8217;re needed in Python <code>DataFrame<\/code>s (ugh), but small-ish data frames will end up in a single file whereas the Drill created ones will be in a directory with an additional CRC file (and, much smaller by default). NOTE: You <em>can<\/em> use <code>preserve_index=False<\/code> on the call to <code>Table.from_pandas<\/code> to get rid of that icky index.<\/p>\n<p>It&#8217;s fairly efficient even for something like <code>nycflights13::flights<\/code> which has ~330K rows and 19 columns:<\/p>\n<pre id=\"reticulatepq03\"><code class=\"language-r\">system.time(\r\n  r_to_py(nycflights13::flights) %&gt;% \r\n  pd$DataFrame$from_dict() %&gt;% \r\n  pa$Table$from_pandas() %&gt;% \r\n  pq$write_table(where = &quot;\/tmp\/flights.parquet&quot;)\r\n)\r\n##    user  system elapsed \r\n##   1.285   0.108   1.398 <\/code><\/pre>\n<p>If you need to generate parquet files in a pinch, <code>reticulate<\/code> seems to be a good way to go.<\/p>\n<p><strong>UPDATE<\/strong> (2018-01-25)<\/p>\n<p>API&#8217;s change and while the above still works, there&#8217;s a slightly simpler way, now:<\/p>\n<pre id=\"reticulatepq04\"><code class=\"language-r\">library(reticulate)\r\n\r\npd &lt;- import(&quot;pandas&quot;, &quot;pd&quot;)\r\n\r\nmtcars_py &lt;- r_to_py(mtcars)\r\nmtcars_df &lt;- pd$DataFrame$from_dict(mtcars_py)\r\n\r\ncity_wx_df$to_parquet(path.expand(&quot;~\/Data\/mtcars_python.parquet&quot;), &quot;pyarrow&quot;)<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>The reticulate package provides a very clean &amp; concise interface bridge between R and Python which makes it handy to work with modules that have yet to be ported to R (going native is always better when you can do it). This post shows how to use reticulate to create parquet files directly from R [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[764,91],"tags":[810],"class_list":["post-6146","post","type-post","status-publish","format-standard","hentry","category-data-wrangling","category-r","tag-post"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>R\u2076 \u2014 Reticulating Parquet Files - rud.is<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/rud.is\/b\/2017\/08\/01\/r\u2076-reticulating-parquet-files\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"R\u2076 \u2014 Reticulating Parquet Files - rud.is\" \/>\n<meta property=\"og:description\" content=\"The reticulate package provides a very clean &amp; concise interface bridge between R and Python which makes it handy to work with modules that have yet to be ported to R (going native is always better when you can do it). This post shows how to use reticulate to create parquet files directly from R [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/rud.is\/b\/2017\/08\/01\/r\u2076-reticulating-parquet-files\/\" \/>\n<meta property=\"og:site_name\" content=\"rud.is\" \/>\n<meta property=\"article:published_time\" content=\"2017-08-01T19:50:11+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-03-07T22:07:29+00:00\" \/>\n<meta name=\"author\" content=\"hrbrmstr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hrbrmstr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/08\\\/01\\\/r%e2%81%b6-reticulating-parquet-files\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/08\\\/01\\\/r%e2%81%b6-reticulating-parquet-files\\\/\"},\"author\":{\"name\":\"hrbrmstr\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"headline\":\"R\u2076 \u2014 Reticulating Parquet Files\",\"datePublished\":\"2017-08-01T19:50:11+00:00\",\"dateModified\":\"2018-03-07T22:07:29+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/08\\\/01\\\/r%e2%81%b6-reticulating-parquet-files\\\/\"},\"wordCount\":340,\"commentCount\":3,\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"keywords\":[\"post\"],\"articleSection\":[\"data wrangling\",\"R\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/08\\\/01\\\/r%e2%81%b6-reticulating-parquet-files\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/08\\\/01\\\/r%e2%81%b6-reticulating-parquet-files\\\/\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/08\\\/01\\\/r%e2%81%b6-reticulating-parquet-files\\\/\",\"name\":\"R\u2076 \u2014 Reticulating Parquet Files - rud.is\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\"},\"datePublished\":\"2017-08-01T19:50:11+00:00\",\"dateModified\":\"2018-03-07T22:07:29+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/08\\\/01\\\/r%e2%81%b6-reticulating-parquet-files\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/08\\\/01\\\/r%e2%81%b6-reticulating-parquet-files\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2017\\\/08\\\/01\\\/r%e2%81%b6-reticulating-parquet-files\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/rud.is\\\/b\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"R\u2076 \u2014 Reticulating Parquet Files\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/\",\"name\":\"rud.is\",\"description\":\"&quot;In God we trust. All others must bring data&quot;\",\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/rud.is\\\/b\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\",\"name\":\"hrbrmstr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"width\":460,\"height\":460,\"caption\":\"hrbrmstr\"},\"logo\":{\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\"},\"description\":\"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7\",\"sameAs\":[\"http:\\\/\\\/rud.is\"],\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/author\\\/hrbrmstr\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"R\u2076 \u2014 Reticulating Parquet Files - rud.is","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/rud.is\/b\/2017\/08\/01\/r\u2076-reticulating-parquet-files\/","og_locale":"en_US","og_type":"article","og_title":"R\u2076 \u2014 Reticulating Parquet Files - rud.is","og_description":"The reticulate package provides a very clean &amp; concise interface bridge between R and Python which makes it handy to work with modules that have yet to be ported to R (going native is always better when you can do it). This post shows how to use reticulate to create parquet files directly from R [&hellip;]","og_url":"https:\/\/rud.is\/b\/2017\/08\/01\/r\u2076-reticulating-parquet-files\/","og_site_name":"rud.is","article_published_time":"2017-08-01T19:50:11+00:00","article_modified_time":"2018-03-07T22:07:29+00:00","author":"hrbrmstr","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hrbrmstr","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/#article","isPartOf":{"@id":"https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/"},"author":{"name":"hrbrmstr","@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"headline":"R\u2076 \u2014 Reticulating Parquet Files","datePublished":"2017-08-01T19:50:11+00:00","dateModified":"2018-03-07T22:07:29+00:00","mainEntityOfPage":{"@id":"https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/"},"wordCount":340,"commentCount":3,"publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"keywords":["post"],"articleSection":["data wrangling","R"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/","url":"https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/","name":"R\u2076 \u2014 Reticulating Parquet Files - rud.is","isPartOf":{"@id":"https:\/\/rud.is\/b\/#website"},"datePublished":"2017-08-01T19:50:11+00:00","dateModified":"2018-03-07T22:07:29+00:00","breadcrumb":{"@id":"https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/rud.is\/b\/2017\/08\/01\/r%e2%81%b6-reticulating-parquet-files\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/rud.is\/b\/"},{"@type":"ListItem","position":2,"name":"R\u2076 \u2014 Reticulating Parquet Files"}]},{"@type":"WebSite","@id":"https:\/\/rud.is\/b\/#website","url":"https:\/\/rud.is\/b\/","name":"rud.is","description":"&quot;In God we trust. All others must bring data&quot;","publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/rud.is\/b\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886","name":"hrbrmstr","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","width":460,"height":460,"caption":"hrbrmstr"},"logo":{"@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1"},"description":"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7","sameAs":["http:\/\/rud.is"],"url":"https:\/\/rud.is\/b\/author\/hrbrmstr\/"}]}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p23idr-1B8","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":4929,"url":"https:\/\/rud.is\/b\/2017\/01\/22\/create-parquet-files-from-r-data-frames-with-sergeant-apache-drill-a-k-a-make-parquet-files-great-again-in-r\/","url_meta":{"origin":6146,"position":0},"title":"Create Parquet Files From R Data Frames With sergeant &#038; Apache Drill (a.k.a. Make Parquet Files Great Again in R)","author":"hrbrmstr","date":"2017-01-22","format":false,"excerpt":"2021-11-04 UPDATE: Just use {arrow}. Apache Drill is a nice tool to have in the toolbox as it provides a SQL front-end to a wide array of database and file back-ends and runs in standalone\/embedded mode on every modern operating system (i.e. you can get started with or play locally\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":6046,"url":"https:\/\/rud.is\/b\/2017\/05\/31\/drilling-into-csvs-teaser-trailer\/","url_meta":{"origin":6146,"position":1},"title":"Drilling Into CSVs \u2014 Teaser Trailer","author":"hrbrmstr","date":"2017-05-31","format":false,"excerpt":"I used reading a directory of CSVs as the foundational example in my recent post on idioms. During my exchange with Matt, Hadley and a few others -- in the crazy Twitter thread that spawned said post -- I mentioned that I'd personally \"just use Drill\u201d. I'll use this post\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4753,"url":"https:\/\/rud.is\/b\/2016\/12\/20\/sergeant-a-r-boot-camp-for-apache-drill\/","url_meta":{"origin":6146,"position":2},"title":"sergeant : An R Boot Camp for Apache Drill","author":"hrbrmstr","date":"2016-12-20","format":false,"excerpt":"I recently mentioned that I've been working on a development version of an Apache Drill R package called sergeant. Here's a lifted \"TLDR\" on Drill: Drill supports a variety of NoSQL databases and file systems, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift,\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":6127,"url":"https:\/\/rud.is\/b\/2017\/07\/27\/reading-pcap-files-with-apache-drill-and-the-sergeant-r-package\/","url_meta":{"origin":6146,"position":3},"title":"Reading PCAP Files with Apache Drill and the sergeant R Package","author":"hrbrmstr","date":"2017-07-27","format":false,"excerpt":"It's no secret that I'm a fan of Apache Drill. One big strength of the platform is that it normalizes the access to diverse data sources down to ANSI SQL calls, which means that I can pull data from parquet, Hie, HBase, Kudu, CSV, JSON, MongoDB and MariaDB with the\u2026","rel":"","context":"In &quot;Apache Drill&quot;","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":11215,"url":"https:\/\/rud.is\/b\/2018\/08\/04\/digging-into-mbox-details-a-tale-of-tm-reticulate\/","url_meta":{"origin":6146,"position":4},"title":"Digging into mbox details: A tale of tm &#038; reticulate","author":"hrbrmstr","date":"2018-08-04","format":false,"excerpt":"\u2728 I had to processes a bunch of emails for a $DAYJOB task this week and my \"default setting\" is to use R for pretty much everything (this should come as no surprise). Treating mail as data is not an uncommon task and many R packages exist that can reach\u2026","rel":"","context":"In &quot;Python&quot;","block_context":{"text":"Python","link":"https:\/\/rud.is\/b\/category\/python-2\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":22211,"url":"https:\/\/rud.is\/b\/2024\/08\/26\/reading-pcap-files-directly-with-duckdb\/","url_meta":{"origin":6146,"position":5},"title":"Reading PCAP Files (Directly) With DuckDB","author":"hrbrmstr","date":"2024-08-26","format":false,"excerpt":"2024-08-30 UPDATE: Binary versions of this extension are available for amd64 Linux (linux_amd64 & linux_amd64_gcc4) and Apple Silicon. (osx_arm64). $ duckdb -unsigned v1.0.0 1f98600c2c Enter \".help\" for usage hints. Connected to a transient in-memory database. Use \".open FILENAME\" to reopen on a persistent database. D SET custom_extension_repository='https:\/\/w3c2.c20.e2-5.dev\/ppcap\/latest'; D INSTALL ppcap;\u2026","rel":"","context":"In &quot;duckdb&quot;","block_context":{"text":"duckdb","link":"https:\/\/rud.is\/b\/category\/duckdb\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/6146","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/comments?post=6146"}],"version-history":[{"count":0,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/6146\/revisions"}],"wp:attachment":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media?parent=6146"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/categories?post=6146"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/tags?post=6146"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}