rud.is

Both my osqueryr and macthekinfe packages have had a few updates and I wanted to put together a fun example (it being Friday, and all) for what you can do with them. All my packages are now on GitHub and GitLab and I’ll be maintaining them on both so I can accommodate the comfort-level of any and all contributors but will be prioritizing issues and PRs on GitLab ahead of any other platform. Having said that, I’ll mark non-CRAN packages with a # notcran comment in the source views so you know you need to install it from wherever you like to grab sketch packages from.

One table that osquery makes available under macOS is an inventory of all “apps” that macOS knows about. Previous posts have shown how to access these tables via the dplyr interface I built for osquery, but they involved multiple steps and as I started to use it more regularly (especially to explore the macOS 10.14 beta I’m running) I noticed that it could use some helper functions. One in particular — osq_expose_tables() — is pretty helpful in that it handles all the dplyr boilerplate code and makes table(s) available in the global environment by name. It takes a single table name or regular expression and then exposes all matching entities. While the function has a help page, it’s easier just to see it in action. Let’s expose the apps table:

library(osqueryr) # notcran
library(tidyverse)

osq_expose_tables("apps")

apps
## # Source:   table [?? x 19]
## # Database: OsqueryConnection
##    applescript_enab… bundle_executable    bundle_identifier   bundle_name  bundle_package_…
##                                                                   
##  1 0                 1Password 6          com.agilebits.onep… 1Password 6  APPL            
##  2 0                 2BUA8C4S2C.com.agil… 2BUA8C4S2C.com.agi… 1Password m… APPL            
##  3 1                 Adium                com.adiumX.adiumX   Adium        APPL            
##  4 1                 Adobe Connect        com.adobe.adobecon… Adobe Conne… APPL            
##  5 1                 Adobe Illustrator    com.adobe.illustra… Illustrator… APPL            
##  6 ""                AIGPUSniffer         com.adobe.AIGPUSni… AIGPUSniffer APPL            
##  7 ""                CEPHtmlEngine Helper com.adobe.cep.CEPH… CEPHtmlEngi… APPL            
##  8 ""                CEPHtmlEngine        com.adobe.cep.CEPH… CEPHtmlEngi… APPL            
##  9 ""                LogTransport2        com.adobe.headligh… LogTranspor… APPL            
## 10 ""                droplet              ""                  Analyze Doc… APPL            
## # ... with more rows, and 14 more variables: bundle_short_version ,
## #   bundle_version , category , compiler , copyright ,
## #   development_region , display_name , element , environment ,
## #   info_string , last_opened_time , minimum_system_version , name ,
## #   path 

There’s tons of info on all the apps macOS knows about, some of which are system services and “helper” apps (like Chrome’s auto-updater). One field — last_opened_time — caught my eye and I thought it would be handy to see which apps had little use (i.e. ones that haven’t been opened in a while) and which apps I might use more frequently (i.e. ones with more recent “open” times). That last_open_time is a fractional POSIX timestamp and, due to the way osquery created the schemas, it’s in a character field. That’s easy enough to convert and then arrange() the whole list in descending order to let you see what you use most frequently.

But, this is R and we can do better than a simple table or even a DT::datatable().

I recently added the ability to read macOS property lists (a.k.a. “plists”) to mactheknife by wrapping a Python module (plistlib). Since all (OK, “most”) macOS apps have an icon, I thought it would be fun to visualize the last opened frequency for each app using the app icons and ggplot2. Unfortunately, the ImageMagick (and, thus the magick package) cannot read macOS icns files, so you’ll need to do a brew install libicns before working with any of the remaining code since we’ll be relying on a command-line utility from that formula.

Let’s get the frontmatter out of the way:

library(sys)
library(magick)
library(osqueryr) # notcran
library(mactheknife) #notcran
library(ggimage)
library(hrbrthemes)
library(ggbeeswarm)
library(tidyverse)

osq_expose_tables("apps")

# macOS will use a generic app icon when none is present in an app bundle; this is the location and we'll
# need to use it when our plist app spelunking comes up short

default_app <- "/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/GenericApplicationIcon.icns"

Next, we'll:

• collect the apps table locally
• filter out system-ish things (which we really don't care about for this post)
• convert the last used time to something useful (and reduce it to a day resolution)
• try to locate the property list for the app and read the path to the app icon file, substituting the generic one if not found (or other errors pop up):

select(apps, name, path, last_opened_time) %>%
  collect() %>%
  filter(!str_detect(path, "(^/System|usr|//System|/Library/|Helper|/Contents/|\\.service$)")) %>%
  mutate(lop_day = as.Date(anytime::anytime(as.numeric(last_opened_time)))) %>%
  mutate(icon = map_chr(path, ~{
    p <- read_plist(file.path(.x, "Contents", "Info.plist"))
    icns <- p$CFBundleIconFile[1]
    if (is.null(icns)) return(default_app)
    if (!str_detect(icns, "\\.icns$")) icns <- sprintf("%s.icns", icns)
    file.path(.x, "Contents", "Resources", icns)
  })) -> apps_df

apps_df
## # A tibble: 274 x 5
##    last_opened_time name                       path                      lop_day    icon                       
##                                                                                      
##  1 1529958322.11297 1Password 6.app            /Applications/1Password … 2018-06-25 /Applications/1Password 6.…
##  2 1523889402.80918 Adium.app                  /Applications/Adium.app   2018-04-16 /Applications/Adium.app/Co…
##  3 1516307513.7606  Adobe Connect.app          /Applications/Adobe Conn… 2018-01-18 /Applications/Adobe Connec…
##  4 1530044681.76677 Adobe Illustrator.app      /Applications/Adobe Illu… 2018-06-26 /Applications/Adobe Illust…
##  5 -1.0             Analyze Documents.app      /Applications/Adobe Illu… 1969-12-31 /Applications/Adobe Illust…
##  6 -1.0             Make Calendar.app          /Applications/Adobe Illu… 1969-12-31 /Applications/Adobe Illust…
##  7 -1.0             Contact Sheets.app         /Applications/Adobe Illu… 1969-12-31 /Applications/Adobe Illust…
##  8 -1.0             Export Flash Animation.app /Applications/Adobe Illu… 1969-12-31 /Applications/Adobe Illust…
##  9 -1.0             Web Gallery.app            /Applications/Adobe Illu… 1969-12-31 /Applications/Adobe Illust…
## 10 -1.0             Adobe InDesign CC 2018.app /Applications/Adobe InDe… 1969-12-31 /Applications/Adobe InDesi…
## # ... with 264 more rows

Since I really didn't feel like creating a package wrapper for libicns, we're going to use the sys package to make system calls to convert the icns files to png files. We really don't want to do this repeatedly for the same files if we ever run this again, so we'll setup a cache directory to hold our converted pngs.

Apps can (and, usually do) have multiple icons with varying sizes and are not guaranteed to have every common size available. So, we'll have the libicns icns2png utility extract all the icons and use the highest resolution one, using magick to reduce it to a 32x32 png bitmap.

# setup the cache dir -- use whatever you want
cache_dir <- path.expand("~/.r-icns-cache")
dir.create(cache_dir)

# create a unique name hash for more compact names
mutate(apps_df, icns_png = map_chr(icon, ~{
  hash <- digest::digest(.x, serialize=FALSE)
  file.path(cache_dir, sprintf("%s.png", hash))
})) -> apps_df

# find the icns2png program
icns2png <- unname(Sys.which("icns2png"))

# go through each icon file 
pb <- progress_estimated(length(apps_df$icns_png))
walk2(apps_df$icon, apps_df$icns_png, ~{

  pb$tick()$print() # progress!

  if (!file.exists(.y)) { # don't create it if it already exists

    td <- tempdir()

    # no icon file == use default one
    if (!file.exists(.x)) .x <- default_app

    # convert all of them to pngs
    sys::exec_internal(
      cmd = icns2png,
      args = c("-x", "-o", td, .x),
      error = FALSE
    ) -> res

    rawToChar(res$stdout) %>% # go through icns2png output
      str_split("\n") %>%
      flatten_chr() %>%
      keep(str_detect, "  Saved") %>% # find all the extracted icons
      last() %>% # use the last one
      str_replace(".* to /", "/") %>% # clean up the filename so we can read it in
      str_replace("\\.$", "") -> png

    # read and convert
    image_read(png) %>%
      image_resize(geometry_area(32, 32)) %>%
      image_write(.y)

  }

})

You can open up that cache directory with the macOS finder to find all the extracted/converted pngs.

Now, we're on the final leg of our app-use visualization journey.

Some system/utility apps have start-of-epoch dates due to the way the macOS installer tags them. We only want "recent" ones so I set an arbitrary cutoff date of the year 2000. Since many apps would have the same last opened date, I wanted to get a spread out layout "for free". One way to do that is to use ggbeeswarm::position_beswarm():

filter(apps_df, lop_day > as.Date("2000-01-01")) %>%
  ggplot() +
  geom_image(
    aes(x="", lop_day, image = icns_png), size = 0.033,
    position = position_quasirandom(width = 0.5)
  ) +
  geom_text(
    data = data_frame(
      x = c(0.6, 0.6),
      y = as.Date(c("2018-05-01", "2017-09-15")),
      label = c("More recently used ↑", "Not used in a while ↓")
    ), 
    aes(x, y, label=label), family = font_an, size = 5 , hjust = 0,
    color = "lightslategray"
  ) +
  labs(x = NULL, y = "Last Opened Time") +
  labs(
    x = NULL, y = NULL,
    title = "macOS 'Last Used' App History"
  ) +
  theme_ipsum_rc(grid="Y") +
  theme(axis.text.x = element_blank())

There are tons of other ways to look at this data and you can use the osquery daemon to log this data regularly so you can get an extra level of detail. An interesting offshot project would be to grab the latest RStudio dailies and see if you can wrangle a sweet D3 visualization from the app data we collected. Make sure to drop a comment with your creations in the comments. You can find the full code in this snippet.

UPDATE (2018-07-07)

A commenter really wanted tooltips with app names. So did I, but neither plotly nor ggiraph support ggimage so we can't get tooltips for free.

However, if you're willing to use the latest RStudio Preview or Daily editions, then we can "easily" use the new built-in D3 support to get some sketch tooltips.

First, we need to change up the plotting code a bit so we can get some base data to feed to D3:

filter(apps_df, lop_day > as.Date("2000-01-01")) %>%
  mutate(name = sub("\\.app", "", name)) %>% 
  ggplot() +
  geom_image(
    aes(x="", lop_day, image = icns_png, name=name), size = 0.033,
    position = position_quasirandom(width = 0.5)
  ) +
  geom_text(
    data = data_frame(
      x = c(0.6, 0.6),
      y = as.Date(c("2018-05-01", "2017-09-15")),
      label = c("More recently used ↑", "Not used in a while ↓")
    ), 
    aes(x, y, label=label), family = font_an, size = 5 , hjust = 0,
    color = "lightslategray"
  ) +
  labs(x = NULL, y = "Last Opened Time") +
  labs(
    x = NULL, y = NULL,
    title = "macOS 'Last Used' App History"
  ) +
  theme_ipsum_rc(grid="Y") +
  theme(axis.text.x = element_blank()) -> gg

gb <- ggplot_build(gg) # compute the layout

idf <- tbl_df(gb$data[[1]]) # extract the data
idf <- mutate(idf, image = sprintf("lib/imgs-1.0.0/%s", basename(image))) # munge the png paths so D3 can find them

write_rds(idf, "~/Data/apps.rds") # save off the data

Now, we just need some D3 javascript glue:

// !preview r2d3 data=data.frame(readRDS("~/Data/apps.rds")), d3_version = 4, dependencies = htmltools::htmlDependency(name = "imgs", version = "1.0.0", src = "~/.r-icns-cache", all_files = TRUE)

var margin = {top: 16, right: 32, bottom: 16, left: 32},
    width = width - margin.left - margin.right,
    height = height - margin.top - margin.bottom;

var x = d3.scaleLinear().range([0, width]);
var y = d3.scaleLinear().range([height, 0]);

x.domain([
  d3.min(data, function(d) { return d.x; }) - 0.05,
  d3.max(data, function(d) { return d.x; }) + 0.05
]);

y.domain([
  d3.min(data, function(d) { return d.y; }) - 16,
  d3.max(data, function(d) { return d.y; }) + 16
]);

var tooltip = d3.select("body")
    .append("div")
    .style("position", "absolute")
    .style("z-index", "10")
    .style("visibility", "hidden")
    .style("color", "blue")
    .style("background", "white")
    .style("padding", "5px")
    .style("font-family", "sans-serif")
    .text("");

svg.attr("width", width + margin.left + margin.right)
    .attr("height", height + margin.top + margin.bottom)
  .append("g")
    .attr("transform",
          "translate(" + margin.left + "," + margin.top + ")");

var images = svg.selectAll("appimg")
      .data(data)
    .enter().append("svg:image")
      .attr("xlink:href",  function(d) { return d.image;})
      .attr("x", function(d) { return x(d.x);})
      .attr("y", function(d) { return y(d.y);})
      .attr("height", 32)
      .attr("width", 32)
      .on("mouseover", function(d) { return tooltip.style("visibility", "visible").text(d.name); })
  .on("mousemove", function(){ return tooltip.style("top", (event.pageY-10)+"px").style("left",(event.pageX+10)+"px"); })
  .on("mouseout", function(){ return tooltip.style("visibility", "hidden"); });

If you don't want to live dangerously, you can also save that script off and just use r2d3 directly:

r2d3::r2d3(
  data = data.frame(readRDS("~/Data/apps.rds")), 
  script = "~/Desktop/app-d3.js",
  d3_version = 4, 
  dependencies = htmltools::htmlDependency(
    name = "imgs", 
    version = "1.0.0", 
    src = "~/.r-icns-cache", 
    all_files = TRUE
  )
)

Either way gives us interactive tooltips: