{"id":10287,"date":"2018-05-18T07:20:08","date_gmt":"2018-05-18T12:20:08","guid":{"rendered":"https:\/\/rud.is\/b\/?p=10287"},"modified":"2018-05-18T13:49:44","modified_gmt":"2018-05-18T18:49:44","slug":"lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations","status":"publish","type":"post","link":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/","title":{"rendered":"“LMX ot NOSJ!” Interchanging Classic Data Formats With Single blackmagic Incantations"},"content":{"rendered":"

\"\"<\/p>\n

The D.C. Universe magic hero Zatanna<\/a> used spells (i.e. incantations) to battle foes and said spells were just sentences said backwards, hence the mixed up jumble in the title. But, now I’m regretting not naming the package zatanna<\/code> and reversing the function names to help ensure they’re only used deliberately & carefully. You’ll see why in a bit.<\/p>\n

Just like their ore-seeking speleological counterparts, workers in our modern day data mines process a multitude of mineralic data formats to achieve our goals of world domination<\/strike> finding meaning, insight & solutions to hard problems.<\/p>\n

Two formats in particular are common occurrences in many of our $DAYJOB<\/code>s: XML and JSON. The rest of this (hopefully short-ish) post is going to assume you have at least a passing familiarity with — if not full-on battle scars from working with — them.<\/p>\n

XML and JSON are, in many ways, very similar. This similarity is on purpose since JSON was originally created to make is easier to process data in browsers and help make data more human-readable. If your $DAYJOB<\/code> involves processing small or large streams of nested data, you likely prefer JSON over XML.<\/p>\n

There are times, though, that — even if one generally works with only JSON data — one comes across a need to ingest some XML and turn it into JSON. This was the case for a question-poser on Stack Overflow this week (I won’t point-shill with a direct link but it’ll be easy to find if you are interested in this latest SODD<\/a> package).<\/p>\n

Rather than take on the potentially painful task of performing the XML to JSON transformation on their own the OP<\/a> wished for a simple incantation to transform the entirety of the incoming XML into JSON.<\/p>\n

\"\"<\/p>\n

We’ll switch comic universes for a moment to issue a warning that all magic comes with a price. And, the cost for automatic XML<->JSON conversion can be quite high. XML has namespaces, attributes tags and values and requires schemas to convey data types and help validate content structure. JSON has no attributes, implicitly conveys types and is generally schema-less (though folks have bolted on<\/a> that concept).<\/p>\n

If one is going to use magic for automatic data conversion there must be rules (no, not<\/em> those kind<\/a> of Magic rules), otherwise how various aspects of XML become encoded into JSON (and the reverse) will generate inconsistency and may even result in significant data corruption. Generally speaking, you are always<\/em> better off writing your own<\/em> conversion utility vs rely on specific settings in a general conversion script\/function. However, if your need is a one-off (which anyone who has been doing this type of work for a while knows is also generally never the case) you may have cause to throw caution to the wind, get your quick data fix, and move on. If that is the case, the blackmagic<\/code>?<\/a> package may be of use to you.<\/p>\n

gnitrevnoC eht ANAI sserddA ecapS yrtsigeR ot NOSJ<\/h3>\n

One file that’s in XML that I only occasionally have to process is the IANA IPv4 Address Space Registry<\/a>. If you visited that link you may have uttered “Hey! That’s not XML it’s HTML!”<\/em>, to wit — I would respond — “Well, HTML is really XML anyway, but use the View Source, Luke! and see that it is indeed XML with some clever XSL style sheet processing being applied in-browser to make the gosh awful XML human readable.”<\/em>.<\/p>\n

With blackmagic<\/code> we can make quick work of converting this monstrosity into JSON.<\/p>\n

The blackmagic<\/code> package itself uses even darker<\/em> magic to accomplish its goals. The package is just a thin V8<\/code> wrapper around the xml-js<\/code>?<\/a> javascript library. Because of this, it is recommended that you do not try to process gigabytes of XML with it as there is a round trip of data marshalling<\/a> between R and the embedded v8 engine.<\/p>\n

\nrequireNamespace(\"jsonlite\") # jsonlite::flatten clobbers purrr::flatten in the wrong order so I generally fully-qualify what I need\n## Loading required namespace: jsonlite\nlibrary(xml2)\nlibrary(blackmagic) # devtools::install_github(\"hrbrmstr\/blackmagic\")\nlibrary(purrr)\nrequireNamespace(\"dplyr\") # I'm going to fully qualify use of dplyr:data_frame() below\n## Loading required namespace: dplyr\n<\/code><\/pre>\n
You can thank @yoniceedee for the URL processing capability in blackmagic<\/code><\/a>:<\/p>\n
\nsource_url <- \"https:\/\/www.iana.org\/assignments\/ipv4-address-space\/ipv4-address-space.xml\"\n\niana_json <- blackmagic::xml_to_json(source_url)\n\n# NOTE: cat the whole iana_json locally to see it \u2014 perhaps to file=\"...\" vs clutter your console\ncat(substr(iana_json, 1800, 2300))\n## me\":\"prefix\",\"elements\":[{\"type\":\"text\",\"text\":\"000\/8\"}]},{\"type\":\"element\",\"name\":\"designation\",\"elements\":[{\"type\":\"text\",\"text\":\"IANA - Local Identification\"}]},{\"type\":\"element\",\"name\":\"date\",\"elements\":[{\"type\":\"text\",\"text\":\"1981-09\"}]},{\"type\":\"element\",\"name\":\"status\",\"elements\":[{\"type\":\"text\",\"text\":\"RESERVED\"}]},{\"type\":\"element\",\"name\":\"xref\",\"attributes\":{\"type\":\"note\",\"data\":\"2\"}}]},{\"type\":\"element\",\"name\":\"record\",\"elements\":[{\"type\":\"element\",\"name\":\"prefix\",\"elements\":[{\"type\":\"\n<\/code><\/pre>\n
By by the hoary hosts of Hoggoth that's not very \"human readable\"! And, it looks super-verbose. Thankfully, Yousuf Almarzooqi knew we'd want to fine-tune the output<\/a> and we can use those options to make this a bit better:<\/p>\n
\nblackmagic::xml_to_json(\n  doc = source_url, \n  spaces = 2,                # Number of spaces to be used for indenting XML output\n  compact = FALSE,           # Whether to produce detailed object or compact object\n  ignoreDeclaration = TRUE   # No declaration property will be generated.\n) -> iana_json\n\n# NOTE: cat the whole iana_json locally to see it \u2014 perhaps to file=\"...\" vs clutter your console\ncat(substr(iana_json, 3000, 3300))\n## pe\": \"element\",\n##               \"name\": \"prefix\",\n##               \"elements\": [\n##                 {\n##                   \"type\": \"text\",\n##                   \"text\": \"000\/8\"\n##                 }\n##               ]\n##             },\n##             {\n##               \"type\": \"element\",\n##               \"name\": \"designation\",\n## \n<\/code><\/pre>\n
One \"plus side\" for doing the mass-conversion is that we don't really need to do much processing to have it be \"usable\" data in R:<\/p>\n
\nblackmagic::xml_to_json(\n  doc = source_url, \n  compact = FALSE,        \n  ignoreDeclaration = TRUE\n) -> iana_json\n\n# NOTE: consider taking some more time to explore this monstrosity than this\nstr(processed <- jsonlite::fromJSON(iana_json), 3)\n## List of 1\n##  $ elements:'data.frame':    3 obs. of  5 variables:\n##   ..$ type       : chr [1:3] \"instruction\" \"instruction\" \"element\"\n##   ..$ name       : chr [1:3] \"xml-stylesheet\" \"oxygen\" \"registry\"\n##   ..$ instruction: chr [1:3] \"type=\\\"text\/xsl\\\" href=\\\"ipv4-address-space.xsl\\\"\" \"RNGSchema=\\\"ipv4-address-space.rng\\\" type=\\\"xml\\\"\" NA\n##   ..$ attributes :'data.frame':  3 obs. of  2 variables:\n##   .. ..$ xmlns: chr [1:3] NA NA \"http:\/\/www.iana.org\/assignments\"\n##   .. ..$ id   : chr [1:3] NA NA \"ipv4-address-space\"\n##   ..$ elements   :List of 3\n##   .. ..$ : NULL\n##   .. ..$ : NULL\n##   .. ..$ :'data.frame':  280 obs. of  4 variables:\n\ncompact(processed$elements$elements[[3]]$elements) %>% \n  head(6) %>% \n  str(3) \n## List of 6\n##  $ :'data.frame':    1 obs. of  2 variables:\n##   ..$ type: chr \"text\"\n##   ..$ text: chr \"IANA IPv4 Address Space Registry\"\n##  $ :'data.frame':    1 obs. of  2 variables:\n##   ..$ type: chr \"text\"\n##   ..$ text: chr \"Internet Protocol version 4 (IPv4) Address Space\"\n##  $ :'data.frame':    1 obs. of  2 variables:\n##   ..$ type: chr \"text\"\n##   ..$ text: chr \"2018-04-23\"\n##  $ :'data.frame':    3 obs. of  4 variables:\n##   ..$ type      : chr [1:3] \"text\" \"element\" \"text\"\n##   ..$ text      : chr [1:3] \"Allocations to RIRs are made in line with the Global Policy published at \" NA \". \\nAll other assignments require IETF Review.\"\n##   ..$ name      : chr [1:3] NA \"xref\" NA\n##   ..$ attributes:'data.frame':   3 obs. of  2 variables:\n##   .. ..$ type: chr [1:3] NA \"uri\" NA\n##   .. ..$ data: chr [1:3] NA \"http:\/\/www.icann.org\/en\/resources\/policy\/global-addressing\" NA\n##  $ :'data.frame':    3 obs. of  4 variables:\n##   ..$ type      : chr [1:3] \"text\" \"element\" \"text\"\n##   ..$ text      : chr [1:3] \"The allocation of Internet Protocol version 4 (IPv4) address space to various registries is listed\\nhere. Origi\"| __truncated__ NA \" documents most of these allocations.\"\n##   ..$ name      : chr [1:3] NA \"xref\" NA\n##   ..$ attributes:'data.frame':   3 obs. of  2 variables:\n##   .. ..$ type: chr [1:3] NA \"rfc\" NA\n##   .. ..$ data: chr [1:3] NA \"rfc1466\" NA\n##  $ :'data.frame':    5 obs. of  4 variables:\n##   ..$ type      : chr [1:5] \"element\" \"element\" \"element\" \"element\" ...\n##   ..$ name      : chr [1:5] \"prefix\" \"designation\" \"date\" \"status\" ...\n##   ..$ elements  :List of 5\n##   .. ..$ :'data.frame':  1 obs. of  2 variables:\n##   .. ..$ :'data.frame':  1 obs. of  2 variables:\n##   .. ..$ :'data.frame':  1 obs. of  2 variables:\n##   .. ..$ :'data.frame':  1 obs. of  2 variables:\n##   .. ..$ : NULL\n##   ..$ attributes:'data.frame':   5 obs. of  2 variables:\n##   .. ..$ type: chr [1:5] NA NA NA NA ...\n##   .. ..$ data: chr [1:5] NA NA NA NA ...\n<\/code><\/pre>\n
As noted previously, all magic comes with a price and we just traded XML processing for some gnarly list processing. This isn't the case for all XML files and you can try to tweak the parameters to xml_to_json()<\/code> to make the output more usable (NOTE: key name transformation parameters<\/a> still need to be implemented in the package), but this seems a whole<\/em> lot easier (to me):<\/p>\n
\ndoc <- read_xml(source_url)\n\nxml_ns_strip(doc)\n\ndplyr::data_frame(\n  \n  prefix = xml_find_all(doc, \".\/\/record\/prefix\") %>% xml_text(),\n  \n  designation = xml_find_all(doc, \".\/\/record\/designation\") %>% xml_text(),\n  \n  date = xml_find_all(doc, \".\/\/record\/date\") %>% \n    xml_text() %>% \n    sprintf(\"%s-01\", .) %>% \n    as.Date(),\n  \n  whois = xml_find_all(doc, \".\/\/record\") %>% \n    map(xml_find_first, \".\/whois\") %>% \n    map_chr(xml_text),\n  \n  status = xml_find_all(doc, \".\/\/record\/status\") %>% xml_text()\n  \n)\n## # A tibble: 256 x 5\n##    prefix designation                     date       whois        status  \n##                                                 \n##  1 000\/8  IANA - Local Identification     1981-09-01          RESERVED\n##  2 001\/8  APNIC                           2010-01-01 whois.apnic\u2026 ALLOCAT\u2026\n##  3 002\/8  RIPE NCC                        2009-09-01 whois.ripe.\u2026 ALLOCAT\u2026\n##  4 003\/8  Administered by ARIN            1994-05-01 whois.arin.\u2026 LEGACY  \n##  5 004\/8  Level 3 Parent, LLC             1992-12-01 whois.arin.\u2026 LEGACY  \n##  6 005\/8  RIPE NCC                        2010-11-01 whois.ripe.\u2026 ALLOCAT\u2026\n##  7 006\/8  Army Information Systems Center 1994-02-01 whois.arin.\u2026 LEGACY  \n##  8 007\/8  Administered by ARIN            1995-04-01 whois.arin.\u2026 LEGACY  \n##  9 008\/8  Administered by ARIN            1992-12-01 whois.arin.\u2026 LEGACY  \n## 10 009\/8  Administered by ARIN            1992-08-01 whois.arin.\u2026 LEGACY  \n## # ... with 246 more rows\n<\/code><\/pre>\n
NIF<\/h3>\n
xml_to_json()<\/code> has a sibling function --- json_to_xml()<\/code> for the reverse operation and you're invited to fill in the missing parameters with a PR<\/a> as there is a fairly consistent and straightforward way to do that. Note that a small parameter tweak can radically change the output, which is one of the aforementioned potentially costly pitfalls of this automagic conversion.<\/p>\n
Before using either function, seriously consider taking the time to write a dedicated, small package that exposes a function or two to perform the necessary conversions.<\/p>\n","protected":false},"excerpt":{"rendered":"
The D.C. Universe magic hero Zatanna used spells (i.e. incantations) to battle foes and said spells were just sentences said backwards, hence the mixed up jumble in the title. But, now I’m regretting not naming the package zatanna and reversing the function names to help ensure they’re only used deliberately & carefully. You’ll see why […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[91,732],"tags":[],"class_list":["post-10287","post","type-post","status-publish","format-standard","hentry","category-r","category-xml"],"yoast_head":"\n"LMX ot NOSJ!" Interchanging Classic Data Formats With Single blackmagic Incantations - rud.is<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\""LMX ot NOSJ!" Interchanging Classic Data Formats With Single blackmagic Incantations - rud.is\" \/>\n<meta property=\"og:description\" content=\"The D.C. Universe magic hero Zatanna used spells (i.e. incantations) to battle foes and said spells were just sentences said backwards, hence the mixed up jumble in the title. But, now I’m regretting not naming the package zatanna and reversing the function names to help ensure they’re only used deliberately & carefully. You’ll see why […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/\" \/>\n<meta property=\"og:site_name\" content=\"rud.is\" \/>\n<meta property=\"article:published_time\" content=\"2018-05-18T12:20:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-05-18T18:49:44+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/rud.is\/b\/wp-content\/uploads\/2018\/05\/blackmagic.jpg\" \/>\n<meta name=\"author\" content=\"hrbrmstr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hrbrmstr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/\"},\"author\":{\"name\":\"hrbrmstr\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"headline\":\"“LMX ot NOSJ!” Interchanging Classic Data Formats With Single blackmagic Incantations\",\"datePublished\":\"2018-05-18T12:20:08+00:00\",\"dateModified\":\"2018-05-18T18:49:44+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/\"},\"wordCount\":883,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2018\\\/05\\\/blackmagic.jpg\",\"articleSection\":[\"R\",\"xml\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/\",\"name\":\"\\\"LMX ot NOSJ!\\\" Interchanging Classic Data Formats With Single blackmagic Incantations - rud.is\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2018\\\/05\\\/blackmagic.jpg\",\"datePublished\":\"2018-05-18T12:20:08+00:00\",\"dateModified\":\"2018-05-18T18:49:44+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/#primaryimage\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2018\\\/05\\\/blackmagic.jpg?fit=700%2C648&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2018\\\/05\\\/blackmagic.jpg?fit=700%2C648&ssl=1\",\"width\":\"700\",\"height\":\"648\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/2018\\\/05\\\/18\\\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/rud.is\\\/b\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"“LMX ot NOSJ!” Interchanging Classic Data Formats With Single blackmagic Incantations\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#website\",\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/\",\"name\":\"rud.is\",\"description\":\""In God we trust. All others must bring data"\",\"publisher\":{\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/rud.is\\\/b\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/rud.is\\\/b\\\/#\\\/schema\\\/person\\\/d7cb7487ab0527447f7fda5c423ff886\",\"name\":\"hrbrmstr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\",\"width\":460,\"height\":460,\"caption\":\"hrbrmstr\"},\"logo\":{\"@id\":\"https:\\\/\\\/i0.wp.com\\\/rud.is\\\/b\\\/wp-content\\\/uploads\\\/2023\\\/10\\\/ukr-shield.png?fit=460%2C460&ssl=1\"},\"description\":\"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7\",\"sameAs\":[\"http:\\\/\\\/rud.is\"],\"url\":\"https:\\\/\\\/rud.is\\\/b\\\/author\\\/hrbrmstr\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"\"LMX ot NOSJ!\" Interchanging Classic Data Formats With Single blackmagic Incantations - rud.is","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/","og_locale":"en_US","og_type":"article","og_title":"\"LMX ot NOSJ!\" Interchanging Classic Data Formats With Single blackmagic Incantations - rud.is","og_description":"The D.C. Universe magic hero Zatanna used spells (i.e. incantations) to battle foes and said spells were just sentences said backwards, hence the mixed up jumble in the title. But, now I’m regretting not naming the package zatanna and reversing the function names to help ensure they’re only used deliberately & carefully. You’ll see why […]","og_url":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/","og_site_name":"rud.is","article_published_time":"2018-05-18T12:20:08+00:00","article_modified_time":"2018-05-18T18:49:44+00:00","og_image":[{"url":"https:\/\/rud.is\/b\/wp-content\/uploads\/2018\/05\/blackmagic.jpg","type":"","width":"","height":""}],"author":"hrbrmstr","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hrbrmstr","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/#article","isPartOf":{"@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/"},"author":{"name":"hrbrmstr","@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"headline":"“LMX ot NOSJ!” Interchanging Classic Data Formats With Single blackmagic Incantations","datePublished":"2018-05-18T12:20:08+00:00","dateModified":"2018-05-18T18:49:44+00:00","mainEntityOfPage":{"@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/"},"wordCount":883,"commentCount":1,"publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"image":{"@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/#primaryimage"},"thumbnailUrl":"https:\/\/rud.is\/b\/wp-content\/uploads\/2018\/05\/blackmagic.jpg","articleSection":["R","xml"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/","url":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/","name":"\"LMX ot NOSJ!\" Interchanging Classic Data Formats With Single blackmagic Incantations - rud.is","isPartOf":{"@id":"https:\/\/rud.is\/b\/#website"},"primaryImageOfPage":{"@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/#primaryimage"},"image":{"@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/#primaryimage"},"thumbnailUrl":"https:\/\/rud.is\/b\/wp-content\/uploads\/2018\/05\/blackmagic.jpg","datePublished":"2018-05-18T12:20:08+00:00","dateModified":"2018-05-18T18:49:44+00:00","breadcrumb":{"@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/#primaryimage","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/05\/blackmagic.jpg?fit=700%2C648&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/05\/blackmagic.jpg?fit=700%2C648&ssl=1","width":"700","height":"648"},{"@type":"BreadcrumbList","@id":"https:\/\/rud.is\/b\/2018\/05\/18\/lmx-ot-nosj-interchanging-classic-data-formats-with-single-blackmagic-incantations\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/rud.is\/b\/"},{"@type":"ListItem","position":2,"name":"“LMX ot NOSJ!” Interchanging Classic Data Formats With Single blackmagic Incantations"}]},{"@type":"WebSite","@id":"https:\/\/rud.is\/b\/#website","url":"https:\/\/rud.is\/b\/","name":"rud.is","description":""In God we trust. All others must bring data"","publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/rud.is\/b\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886","name":"hrbrmstr","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","width":460,"height":460,"caption":"hrbrmstr"},"logo":{"@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1"},"description":"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7","sameAs":["http:\/\/rud.is"],"url":"https:\/\/rud.is\/b\/author\/hrbrmstr\/"}]}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p23idr-2FV","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":3898,"url":"https:\/\/rud.is\/b\/2016\/01\/13\/cobble-xpath-interactively-with-the-xmlview-package\/","url_meta":{"origin":10287,"position":0},"title":"Cobble XPath Interactively with the xmlview Package","author":"hrbrmstr","date":"2016-01-13","format":false,"excerpt":"(If you don't know what XML is, you should probably [read a primer](https:\/\/en.wikipedia.org\/wiki\/XML) before reading this post,) When working with data, one inevitably comes across things encoded in XML. I'm in the \"anti-XML\" camp, but deal with my fair share of XML in \"cyber\" and help out enough people who\u2026","rel":"","context":"In "data wrangling"","block_context":{"text":"data wrangling","link":"https:\/\/rud.is\/b\/category\/data-wrangling\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2016\/01\/RStudioScreenSnapz003.png?fit=865%2C523&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2016\/01\/RStudioScreenSnapz003.png?fit=865%2C523&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2016\/01\/RStudioScreenSnapz003.png?fit=865%2C523&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2016\/01\/RStudioScreenSnapz003.png?fit=865%2C523&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":12917,"url":"https:\/\/rud.is\/b\/2021\/01\/26\/making-it-easier-to-experiment-with-compiled-swift-code-in-r\/","url_meta":{"origin":10287,"position":1},"title":"Making It Easier To Experiment With Compiled Swift Code In R","author":"hrbrmstr","date":"2021-01-26","format":false,"excerpt":"The past two posts have (lightly) introduced how to use compiled Swift code in R, but they've involved a bunch of \"scary\" command line machinations and incantations. One feature of {Rcpp} I've always ? is the cppFunction() (\"r-lib\" zealots have a similar cpp11::cpp_function()) which lets one experiment with C[++] code\u2026","rel":"","context":"In "macOS"","block_context":{"text":"macOS","link":"https:\/\/rud.is\/b\/category\/macos\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":12951,"url":"https:\/\/rud.is\/b\/2021\/02\/14\/extracting-heart-rate-data-from-apple-health-xml-export-files-using-r-a-k-a-the-least-romantic-valentines-day-r-post\/","url_meta":{"origin":10287,"position":2},"title":"Extracting Heart Rate Data (Two Ways!) from Apple Health XML Export Files Using R (a.k.a. The Least Romantic Valentine’s Day R Post Ever)","author":"hrbrmstr","date":"2021-02-14","format":false,"excerpt":"? Expand for EKG code library(hrbrthemes) library(elementalist) # remotes::install_github(\"teunbrand\/elementalist\") library(ggplot2) read_csv( file = \"~\/Data\/apple_health_export\/electrocardiograms\/ecg_2020-09-24.csv\", # this is extracted below skip = 12, col_names = \"\u00b5V\" ) %>% mutate( idx = 1:n() ) -> ekg ggplot() + geom_line_theme( data = ekg %>% tail(3000) %>% head(2500), aes(idx, \u00b5V), size = 0.125, color\u2026","rel":"","context":"In "R"","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/02\/ekg.png?fit=1200%2C437&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/02\/ekg.png?fit=1200%2C437&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/02\/ekg.png?fit=1200%2C437&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/02\/ekg.png?fit=1200%2C437&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2021\/02\/ekg.png?fit=1200%2C437&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":11427,"url":"https:\/\/rud.is\/b\/2018\/08\/24\/friday-rstats-twofer-finding-macos-32-bit-apps-processing-data-from-system-commands\/","url_meta":{"origin":10287,"position":3},"title":"Friday #rstats twofer: Finding macOS 32-bit apps & Processing Data from System Commands","author":"hrbrmstr","date":"2018-08-24","format":false,"excerpt":"Apple has run the death bell on 32-bit macOS apps and, if you're running a recent macOS version on your Mac (which you should so you can get security updates) you likely see this alert from time-to-time: If you're like me, you click through that and keep working but later\u2026","rel":"","context":"In "Apple"","block_context":{"text":"Apple","link":"https:\/\/rud.is\/b\/category\/apple\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/Screen-Shot-2018-08-24-at-4.58.41-AM.png?fit=1200%2C612&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/Screen-Shot-2018-08-24-at-4.58.41-AM.png?fit=1200%2C612&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/Screen-Shot-2018-08-24-at-4.58.41-AM.png?fit=1200%2C612&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/Screen-Shot-2018-08-24-at-4.58.41-AM.png?fit=1200%2C612&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2018\/08\/Screen-Shot-2018-08-24-at-4.58.41-AM.png?fit=1200%2C612&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":3633,"url":"https:\/\/rud.is\/b\/2015\/08\/23\/using-r-to-get-data-out-of-word-docs\/","url_meta":{"origin":10287,"position":4},"title":"Using R To Get Data *Out Of* Word Docs","author":"hrbrmstr","date":"2015-08-23","format":false,"excerpt":"NOTE: after reading this post head on over to this new one as it has wrapped this functionality (and more!) into a package. Also: docxtractr is now on CRAN This was asked on twitter recently: Is it possible to import data entered in MS Word into R - I have\u2026","rel":"","context":"In "R"","block_context":{"text":"R","link":"https:\/\/rud.is\/b\/category\/r\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":6127,"url":"https:\/\/rud.is\/b\/2017\/07\/27\/reading-pcap-files-with-apache-drill-and-the-sergeant-r-package\/","url_meta":{"origin":10287,"position":5},"title":"Reading PCAP Files with Apache Drill and the sergeant R Package","author":"hrbrmstr","date":"2017-07-27","format":false,"excerpt":"It's no secret that I'm a fan of Apache Drill. One big strength of the platform is that it normalizes the access to diverse data sources down to ANSI SQL calls, which means that I can pull data from parquet, Hie, HBase, Kudu, CSV, JSON, MongoDB and MariaDB with the\u2026","rel":"","context":"In "Apache Drill"","block_context":{"text":"Apache Drill","link":"https:\/\/rud.is\/b\/category\/apache-drill\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/10287","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/comments?post=10287"}],"version-history":[{"count":0,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/10287\/revisions"}],"wp:attachment":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media?parent=10287"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/categories?post=10287"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/tags?post=10287"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}