macOS R users who tend to work on the bleeding edge likely noticed some downtime at <mac.r-project.org> this past weekend. Part of the issue was an SSL/TLS certificate expiration situation. Moving forward, we can monitor this with R using the super spiffy {openssl} and {pushoverr} packages whilst also generating a daily report with {rmarkdown} and {DT}.
The Basic Process
The {openssl} package has a handy function — download_ssl_cert()
— which will, by default, hit a given host on the standard HTTPS port (443/TCP) and grab the site certificate and issuer. We’ll grab the “validity end” field and convert that to a date to use for comparison.
To get the target list of sites to check I used Rapid7’s FDNS data set and a glance at a few certificate transparency logs to put together a current list of “r-project” domains that have been known to have SSL certs. This process could be made more dynamic, but things don’t change that quickly in r-project domain land.
Finally, we use the {DT} package to build a pretty HTML table and the {pushoverr} package to send notifications at normal priority for certs expiring within a week and critical priority for certs that have expired (the package has excellent documentation which will guide you through setting up a Pushover account).
I put this all in a plain R script named r-project-ssl-notify.R
that’s then called from a Linux CRON job which runs:
/usr/bin/Rscript -e 'rmarkdown::render(input="PATH_TO/r-project-ssl-notify.R", output_file="PATH_TO/r-project-cert-status/index.html", quiet=TRUE)'
once a day at 0930 ET to make this status page and also fire off any notifications which I have going to my watch and phone (I did a test send by expanding the delta to 14 days):
Here’s the contents of
#' ---
#' title: "r-project SSL/TLS Certificate Status"
#' date: "`r format(Sys.time(), '%Y-%m-%d')`"
#' output:
#' html_document:
#' keep_md: false
#' theme: simplex
#' highlight: monochrome
#' ---
#+ init, include=FALSE
knitr::opts_chunk$set(
message = FALSE,
warning = FALSE,
echo = FALSE,
collapse=TRUE
)
#+ libs
library(DT)
library(openssl)
library(pushoverr)
library(tidyverse)
# Setup -----------------------------------------------------------------------------------------------------------
# This env config file contains two lines:
#
# PUSHOVER_USER=YOUR_PUSHOVER_USER_STRING
# PUSHOVER_APP=YOUR_PUSHOVER_APP_KEY
#
# See the {pushoverr} package for how to setup your Pushover account
readRenviron("~/jobs/conf/r-project-ssl-notify.env")
# Check certs -----------------------------------------------------------------------------------------------------
# r-project.org domains retrieved from Rapid7's FDNS data set
# (https://opendata.rapid7.com/sonar.fdns_v2/) and cert transparency logs
#+ work
c(
"beta.r-project.org", "bugs.r-project.org", "cloud.r-project.org",
"cran-archive.r-project.org", "cran.at.r-project.org", "cran.ch.r-project.org",
"cran.es.r-project.org", "cran.r-project.org", "cran.uk.r-project.org",
"cran.us.r-project.org", "developer.r-project.org", "ess.r-project.org",
"ftp.cran.r-project.org", "journal.r-project.org", "lists.r-forge.r-project.org",
"mac.r-project.org", "r-project.org", "svn.r-project.org", "translation.r-project.org",
"user2011.r-project.org", "user2014.r-project.org", "user2016.r-project.org",
"user2018.r-project.org", "user2019.r-project.org", "user2020.r-project.org",
"user2020muc.r-project.org", "win-builder.r-project.org", "www.cran.r-project.org",
"www.r-project.org", "www.user2019.fr"
) -> r_doms
# grab each cert
r_certs <- map(r_doms, openssl::download_ssl_cert)
# make a nice table
tibble(
dom = r_doms,
expires = map_chr(r_certs, ~.x[[1]][["validity"]][[2]]) %>% # this gets us the "validity end"
as.Date(format = "%b %d %H:%M:%S %Y", tz = "GMT"), # and converts it to a date object
delta = as.numeric(expires - Sys.Date(), "days") # this computes the delta from the day this script was called
) %>%
arrange(expires) -> r_certs_expir
# Status page generation ------------------------------------------------------------------------------------------
# output nice table
DT::datatable(r_certs_expir, list(pageLength = nrow(r_certs_expir))) # if the # of r-proj doms gets too large we'll cap this for pagination
# Notifications ---------------------------------------------------------------------------------------------------
# See if we need to notify abt things expiring within 1 week
# REMOVE THIS or edit the delta max if you want less noise
one_week <- filter(r_certs_expir, between(delta, 1, 7))
if (nrow(one_week) > 0) {
pushover_normal(
title = "There are r-project SSL Certs Expiring Within 1 Week",
message = "Check which ones: https://rud.is/r-project-cert-status"
)
}
# See if we have expired certs
expired <- filter(r_certs_expir, delta <= 0)
if (nrow(expired) > 0) {
pushover_critical(
title = "There are expired r-project SSL Certs!",
message = "Check which ones: https://rud.is/r-project-cert-status"
)
}
FIN
With just a tiny bit of R code we have the ability to monitor expiring SSL certs via a diminutive status page and alerts to any/all devices at our disposal.
6 Comments
Dear Bob, thank you for this service to the community.
I assume you were not aware of the sites svn.r-project.org (where the sources are maintained, hosted here at ETH Zurich) and the even less prominent developer.r-project.org sites.. We’d be grateful if you’d add them as well.
Martin
I’ll get them in today (and I’ll poke at what was wrong with my FDNS query since it shld have picked these up).
and ess.r-project.org (I’m sorry these come bit by bit … OTOH, shouldn’t there be a cool 1-liner to find all hosts *.r-project.org ?)
Got them in and they’re in the latest cron run. The query was almost a one-liner but my eyes are the reason I missed thing. The query against our FDNS dataset came back with 769 unique names (https://rud.is/dl/r-project-doms.txt) and I eyeballed ones I “knew” were web-ones. I’ll also do a query against our own certificate db in a bit and add what I find there, too.
come back as everything that isn’t r-forge related. I’ll validate the ones that have SSL enabled and get those in too.
30 have SSL/TLS certs and they’re all in the monitoring/report now.
One Trackback/Pingback
[…] by data_admin [This article was first published on R – rud.is, and kindly contributed to R-bloggers]. (You can report issue about the content on this page […]