Skip navigation

Category Archives: Observable

Foliage 2023

The days are getting shorter and when we were visiting Down East Maine the other week, there was just a hint of some trees starting to change up their leaf palettes. It was a solid reminder to re-up my ~annual “foliage” plotting that I started way back in 2017.

The fine folks over at Smoky Mountains — (“the most authoritative source for restaurants, attractions, & cabin rentals in the Smoky Mountains”) — have been posting an interactive map of ConUS foliage predictions for many years and the dataset they curate and use for that is also very easy to use in R and other contexts.

This year, along with the usual R version, I have also made:

The only real changes to R version were to add some code to make a more usable JSON for the JavaScript versions of the project, and to take advantage of the .progress parameter to {purrr}’s walk function.

The Observable notebook version (one frame of that is above) makes use of Observable Plot’s super handy geo mark, and also shows how to do some shapefile surgery to avoid plotting Alaska & Hawaii (the Smoky Mountains folks only provide predictions for ConUS).

After using the Reveal QMD extension to make the Quarto project, the qmd document rendered fine, but I tweaked the YAML to send the output to the GH Pages-renderable docs/ directory, and combined some of the OJS blocks to tighten up the document. You’ll see some Quarto “error” blocks, briefly, since there the QMD fetches imports from Observable. You can get around that by moving all the imported resources to the Observable notebook before generating the QMD, but that’s an exercise left to the reader.

And, since I’m a fan of both Lit WebComponents and Tachyons CSS, I threw together a version using them (+ Observable Plot) to further encourage folks to get increasingly familiar with core web tech. Tachyons + Plot make it pretty straightforward to create responsive pages, too (resize the browser and toggle system dark/light mode to prove that). The Lit element’s CSS section also shows how to style Plot’s legend a bit.

Hit up the GH page to see the animated gif (I’ve stared at it a bit too much to include it in the post).

Drop any q’s here or in the GH issues, and — if anyone makes a Shiny version — please let me know, and I’ll add all links to any of those here and on the GH page.

FIN

While it is all well and good to plot foliage prediction maps, please also remember to take some time away from your glowing rectangles to go and actually observe the fall palette changes IRL.

Keeping Track Of URLs Shared On Bluesky

While the future of Bluesky is nowhere near certain, it is most certainly growing. It’s also the largest community of users for the AT Protocol.

Folks are using Bluesky much the same way as any online forum/chat. One of those ways is to share URLs to content.

For the moment, it is possible to eavesdrop on the Bluesky “firehose” sans authentication. I’ve been curious as to what folks are sharing on the platform and decided to do more than poke at it casually in my hacky terminal firehose viewer.

This GitLab project contains all the code necessary to log URLs seen in the firehose to a local SQLite database. As Bluesky grows, this will definitely not scale, but it’s fine for right now, and scaling just means moving the websocket capture client to a more capable environment than my home server and setting up something like a Kafka stream. Might as well move to Postgres while we’re at it.

But, for now, this lightweight script/database is fine.

NOTE: I’m deliberately not tracking any other data, but the code is easy to modify to log whatever you want from the firehose post.

I’m syncing the data to this server every ~30 minutes or so and have created an Observable notebook which keeps track of the most popular domains.

I don’t know what card.syui.ai is (Perplexity had some ideas), but it appears to be some AI-driven “card” game that has AT protocol and ActivityPub integration. Due to the programmatic nature of the posts with URLs containing that domain, I suspect it’ll be in the lead for quite some time.

There are some neat sites in the long tail of the distribution.

I think I’ll set up one to monitor post with CVE’s, soon, too.

Supreme Annotations Plot Redux & An OJS Plot↔ggplot2 Rosetta Stone

Back in 2016, I did a post on {ggplot2} text annotations because it was a tad more challenging to do some of the things in that post back in the day.

Since I’ve been moving back and forth between R and Observable (and JavaScript in general), I decided to recreate that post in OJS Plot, as it is also somewhat challenging to use this nascent new plot player in town.

Getting Observable embeds right in a theme that supports both dark/light mode is kind of a pain (either that or I’m just lazy), and I already have enough holes in this site’s content security policy (and, I’d have to poke more holes to get fonts working “properly”); so, you can visit the link to see the crisp SVG version and suffer the PNG below to see that Plot is a very capable companion to {ggplot2}.

Speaking of {ggplot2}, Daily Drop subscribers were given some homework (happens every Friday) that involved working in Observable Plot. The catch was that said work was to recreate {ggplot2} geom_ examples in the R manual pages for {ggplot2}.

You can check out that post, go to a hosted version of the starter “Rosetta Stone” set of examples I provided, poke at the Observable notebook version, or head over to GitHub where the online play ground source lives, along with a Quarto document with all the examples.

Fall Foliage: JavaScript/OJS Edition

I’ve been (mostly) keeping up with annual updates for my R/{sf} U.S. foliage post which you can find on GH. This year, we have Quarto, and it comes with so many batteries included that you’d think it was Christmas. One of those batteries is full support for the Observable runtime. These are used in {ojs} Quarto blocks, and rendered versions can run anywhere.

The Observable platform is great for both tinkering and publishing (we’re using it at work for some quick or experimental vis work), and with a few of the recent posts, here, showing how to turn Observable notebooks into Quarto documents, you’re literally two clicks or one command line away from using any public Observable notebook right in Quarto.

I made a version of the foliage vis in Observable and then did the qmd conversion using the Chrome extension, tweaked the source a bit and published the same in Quarto.

The interactive datavis uses some foundational Observable/D3 libraries:

In the JS code we set some datavis-centric values:

foliage_levels = [0.01, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6]
foliage_colors = ["#83A87C", "#FCF6B4", "#FDCC5C", "#F68C3F", "#EF3C23", "#BD1F29", "#98371F"]
foliage_labels = ["No Change", "Minimal", "Patchy", "Partial", "Near Peak", "Peak", "Past Peak"]
week_label = ["Sept 5th", "Sept 12th", "Sept 19th", "Sept 26th", "Oct 3rd", "Oct 10th", "Oct 17th", "Oct 24th", "Oct 31st", "Nov 7th", "Nov 14th", "Nov 21st"]

We then borrow the U.S. Albers-projected topojson file from the Choropleth example notebook and rebuild the outline mesh and county geometry collections, since we need to get rid of Alaska and Hawaii (they’re not present in the source data). We do this by filtering out two FIPS codes:

counties = {
  var cty = topojson.feature(us, us.objects.counties);
  cty.features = cty.features.filter(
    (d) => (d.id.substr(0, 2) != "02") & (d.id.substr(0, 2) != "15")
  );
  return cty;
}

I also ended up modifying the source CSV a bit to account for missing counties.

After that, it was a straightforward call to our imported Choropleth function:

chart = Choropleth(rendered2022, {
  id: (d) => d.id.toString().padStart(5, "0"), // this is needed since the CSV id column is numeric
  value: (d) => d[week_label.indexOf(week) + 1], // this gets the foliage value based on which index the selected week is at
  scale: d3.scaleLinear, // this says to map foliage_levels to foliage_colors directly
  domain: foliage_levels,
  range: foliage_colors,
  title: (f, d) =>
    `${f.properties.name}, ${statemap.get(f.id.slice(0, 2)).properties.name}`, // this makes the county hover text the county + state names
  features: counties, // this is the counties we modified
  borders: statemesh, // this is the statemesh
  width: 975,
  height: 610
})

and placing the legend and scrubbing slider.

The only real difference between the notebook and qmd is the inclusion of the source functions rather than use Observable’s import (I’ve found that there’s a slight load delay for imports when network conditions aren’t super perfect and the inclusion of the source — WITH copyrights — makes up for that).

I’ve set up the Quarto project so that renders go to the docs/ directory, which makes it easy to publish as a GH page.

FIN

Drop issues on GH if anything needs clarifying or fixing and go experiment! You can’t break anything either on Observable or locally that version control can’t fix (yes, Observable has version control!).

Some things to consider modifying/adding:

  • have a click take you to a (selectable?) mapping service, so folks can get driving directions
  • turn the hover text into a proper tooltip
  • speed up or slow down the animation when ‘Play’ is tapped
  • use different colors
  • bring in older datasets (see the foliage GH repo) and make multiple maps or let the user select them or have them compare across years

COVID-19 U.S. County Vaccination Tracker With An Observable Notebook Using Datasettes and {Plot}

Rather than continue to generate daily images with R, I threw together an Observable notebook that takes advantage of the CDC COVID-19 county data datasette (provided by Simon Willison) and the new {Plot} library (by the @ObservableHQ team) that enables users to interactively see the daily county resident vaccination “series complete” percentage distribution.

The full notebook is here — https://observablehq.com/@hrbrmstr/us-county-covid-vaccination-progress — and the interactive visualization is embedded below (though it doesn’t support “dark mode” well):