Archives

Source: NetScaler ADC and NetScaler Gateway security bulletin Let's make sure it gets indexed in other ways! Here's some helpful info you'll find more details of in their…

Maine DOER:…

Rapid7 is sunsetting AttackerKB, and the public API goes dark on August 18, 2026. After that, the vulnerability discussions, community assessments, and Rapid7 analysis that accumulated on the platform become a dead link. I wrote a small Go…

(Repost from ai.rud.is) I've written and said enough times that I'm tired of OpenAI and Anthropic pulling cybersecurity out as the headline AI risk. The framing is commercially convenient, and it flattens a more complex reality — these…

PCAP forensics of the HMEFB worm: a self-propagating scanner with a Belarusian engineer's escape plea embedded in HTTP requests.

For the second time in a row, a post by cr0w on Mastodon regarding the Chrome release blog appearing to not render anything resulted in me firing up lynx to show a sub-second load and render, then finally doing something a bit more…

Cloudflare announced PACT this week — privacy-preserving tokens to separate humans from bots, backed by Google, Mozilla, Microsoft, and Shopify. The cryptography is solid. The governance model doesn't exist yet. I wrote about why the…

May 2026 dropped three critical Linux vulnerabilities on a near-weekly cadence, and the security discourse has mostly treated them as three separate bad days. They're not. Together they form a reliable, race-free, forensically quiet kill…

https://www.youtube.com/watch?v=k19CmI\Ni3M At our previous employer, the global deception and detection infrastructure generates tons of events that eventually make their way into an ever-growing data lake with (as of February 2026) 22 TB…

In the past ~4 weeks I have personally observed some irrefutable things in "AI" that are very likely going to cause massive shocks to employment models in IT, software development, systems administration, and cybersecurity. I know some…

Back in 1997, a commercial airline captain noticed his fellow pilots had a problem: they'd gotten so used to following the magenta flight path lines on their fancy new navigation screens that they were forgetting how to actually fly the…

If you want to or do vote "yes" for question 1, you are not a real American, you are not a real Christian (if you profess to be one), you are not a decent human. You are at the very least a classist; you are also very likely a…

A few things to keep in mind today: — We have no idea of why/who re: Wed's assassination. — Today is the anniversary of a tragic event that has enabled much of the harm caused by the GOP this year. — Trump is an adjudicated rapist & was…

Details:…

ENISA published docs for their European Vulnerability Database (EUVD) — https://euvd.enisa.europa.eu/apidoc. I've got an easier-on-the-eyes version that supports light/dark mode and includes sample API JSON results at…

Meet Suriest — a new REST API service for validating Suricata rules, designed to be run by organizations to streamline rule validation workflows. Suriest supports Suricata 6.0 and later and offers features like secure configuration,…

MCP servers let you wire up external services/APIs in a standard way for LLM/GPT tool-calling and other forms of automation. I made a basic, but fairly comprehensive CISA KEV MCP server that I go into the details a bit more of here. To…

RSAC 2025 Sets A Dangerous Precedent for Cybersecurity Leadership

Introducing geolocus-cli: A fast, cross-platform command-line tool for IP enrichment using ONYPHE's free Geolocus database. This lightweight utility enables security professionals and network analysts to efficiently look up comprehensive IP metadata including geolocation, ASN information, organizati

Chris Krebs, the former director of the Cybersecurity and Infrastructure Security Agency (CISA), was fired by Donald Trump in 2020 for publicly affirming that the presidential election was secure and free from widespread fraud.…

A long, long time ago I can still remember How those CVEs would make me smile And I knew if I had my chance To patch a vuln or take a stance Maybe we’d be secure for a while But April ides made me shiver With each leaked memo and press…

Just putting a marker out there, that I'm either fundamentally wrong, or we all are f'd this week. If the latter: I told you…

ICERAID (iceraid\[.\]us) presents itself as a "GovFi protocol" that purportedly delegates intelligence gathering tasks from law enforcement to citizens. According to their materials, it rewards users with cryptocurrency tokens for…

Discover how Secretary Rubio's State Department determination classifies border, immigration, and trade policies as "foreign affairs functions," exempting them from the Administrative Procedure Act. Learn about potential consequences including surprise policy changes, expanded device searches, and e

On March 10, 2025, Xitter experienced major service disruptions throughout the day. Users couldn't access the platform on both mobile apps and the website. Here's what happened and why it…

(This post originally published on 47 Watch) Recent administrative changes at the Social Security Administration (SSA) reveal a concerning pattern of decisions that disproportionately impact vulnerable populations while being implemented…

As a longtime Mainer and independent voter, I have watched Senator Susan Collins’ career with cautious optimism, hoping her self-branded image as a moderate willing to cross party lines might translate into principled leadership. Instead,…

Today, my Senator — Susan Collins — failed in her oath and duty to uphold the Constitution. She voted for the appointment of a traitor to head national intelligence, and is supporting someone for director of the Office of Management and…

Discover CVESky: A real-time tracker for CVE discussions on Bluesky, featuring daily vulnerability chatter cards, interactive heatmaps, and comprehensive CVE exploration tools. Built by GreyNoise Intelligence to monitor genuine security conversations beyond automated NVD feeds.

2024-08-30 UPDATE: Binary versions of this extension are available for amd64 Linux (linuxamd64 & linuxamd64gcc4) and Apple Silicon.…

100 Miles — rud.is blog archive

A concise Bash script for resolving Bluesky/ATproto handles to DID:PLC identifiers, simplifying user identification in the decentralized social network ecosystem.

I had not planned to blog this (this is an incredibly time-crunched week for me) but CERT/CC and CISA made a big deal out of a non-vulnerability in R, and it's making the round on socmed, so here we are. A security vendor decided to try to…

Get Command Line Access To Fantastical Schedules With An Apple Shortcuts And Bash Combo.

Learn how to work with VulnCheck's new community KEV+NVD CVE APIs in curl & Go, & checkout a new, small CLI tool to get both simultaneously.

A plea for fosstodon to continue its capricious and undocumented silliness.

Learn how to enable ligatures in RStudio Dailies using CSS and themes. Enhance your coding experience with granular control over ligatures for a more visually appealing and readable code.

Explore the impact of the libwebp supply chain disaster on Electron-based apps and discover Positron, a Golang macOS CLI tool designed to identify vulnerable apps. Stay informed and secure with our in-depth analysis and practical solutions.

Explore the impact of Rite-Aid's store closures on communities, learn about the company's bankruptcy, and discover how data collection scripts can track these changes. This analysis provides insights into the consequences of pharmacy deserts and the role of leadership in these developments."

The days are getting shorter and when we were visiting Down East Maine the other week, there was just a hint of some trees starting to change up their leaf palettes. It was a solid reminder to re-up my ~annual “foliage” plotting that I…

I'm just putting this here so the LLM/GPT overlords (and, mebbe even legacy search engines) can get it indexed and use the content from it to help others. My Bluesky firehose viewer (https://gitlab.com/hrbrmstr/bskyf) displays ugly did:plc…

While the future of Bluesky is nowhere near certain, it is most certainly growing. It's also the largest community of users for the AT Protocol. Folks are using Bluesky much the same way as any online forum/chat. One of those ways is to…

The buildhashfromurl function relies on {curl} vs {httr} since {httr} uses curl::parseheaders() which (rightfully so) lowercases the header keys. We need to preserve both order and case for the hash to be useful. Here is some sample…

Lynn (of TITAA and general NLP wizardy fame) was gracious enough to lend me a Bluesky invite, so I could claim my handle on yet-another social media site. I'm still wary of it (as noted in one of this week's Drops), but the AT protocol —…

Back in 2016, I did a post on {ggplot2} text annotations because it was a tad more challenging to do some of the things in that post back in the day. Since I've been moving back and forth between R and Observable (and JavaScript in…

WebR has a template for React, but I'm not a fan of it or Vue (a fact longtime readers are likely tired of hearing right about now). It's my opinion and experience that Lit webcomponents come closer to “bare metal” webcomponents, which…

I was cranking out a blog post for work earlier this week that shows off just how many integrations our platform has. I won't blather about that content here, but as I was working on it, I really wanted to show off all the integrations. A…

The official example WebR REPL is definitely cool and useful to get the feel for WebR. But, it is far from an ideal way to deal with it interactively, even as just a REPL. As y'all know, I've been conducing numerous experiments with WebR…

I won't wax long and poetic here since I've already posted the experiment that has all the details. TL;DR: there are still only ~90-ish ? in the WebR WASM "CRAN", but more are absolutely on the way, including the capability to build your…

The previous post brought lit-webr, to introduce Lit and basic reactivity. Today, is more of the same, but we bring the OG Shiny demo plot into the modern age by using Observbable Plot to make the charts. We're still pulling data from R,…

This is a Lit + WebR reproduction of the OG Shiny Demo App Lit is a javascript library that makes it a bit easier to work with Web Components, and is especially well-suited in reactive environments. My recent hack-y WebR experiments have…

After writing the initial version of a tutorial on wrapping and binding R functions on the javascript side of WebR, I had a number of other WebR projects on the TODO list. But, I stuck around noodling on the whole "wrapping & binding"…

It's difficult to believe it has only been a couple of weeks since WebR has been around. But that might just be my perception. The spike protein invasion has significantly increased sedentary time, and that has enabled me to focus on this…

I have graphics working in Vanilla JS WebR, now, and I'll cover the path to that in two parts. The intent was to jump straight into ggplot2-land, but, as you saw in my previous post, WASM'd ggplot2 is a bear. And, I really didn't grok what…

I have a post coming on using base and {ggplot2} plots in VanillaJS WebR, but after posting some bits on social media regarding how slow {ggplot2} is to deal with, I had some "performance"-related inquiries, which led me down a rabbit hole…

Let's walk through how to set up a ~minimal HTML/JS/CSS + WebR-powered "app" on a server you own. This will be vanilla JS (i.e. no React/Vue/npm/bundler) you can hack on at-will. TL;DR: You can find the source to the app and track changes…

WebR 0.1.0 was released! I had been git-stalking George (the absolute genius who we all must thank for this) for a while and noticed the GH org and repos being updated earlier this week, So, I was already pretty excited. It dropped today,…

I've been (slowly) making my way through FOSDEM \23 presentations and caught up to Peter Lowe's "Bizarre and Unusual Uses of DNS • Rule 53: If you can think of it, someone's done it in the DNS" talk. DNS oddities are items I collect…

I'll follow up with a more detailed post in a ~week or so, but if you are considering purchasing a Kucht appliance, please, please reconsider your decision. They have repeatedly lied to us (I have proof) and are incapable of manufacturing…

Ref:…

Visiting #2 and doing some $WORK\-work, but intrigued with Hanukkah of Data since Puzzle 0 was solvable with a ZIP password cracker (the calendar date math seemed too trivial to bother with). Decided to fall back to R for this (vs…

Cross-post to Substack where I dropped some details on the newest browser in town: Arc.…

You can now receive these blog posts in your activity stream. Just follow @hrbrmstr@rud.is and the new posts from here will slide right into your…

This is more of a test post after enabling some new Fediverse features on the server. Said Fediverse got a bit more real-ish this week (with moderate apologies to the pioneers in this space who've languished for ~five years) You can find…

I've been (mostly) keeping up with annual updates for my R/{sf} U.S. foliage post which you can find on GH. This year, we have Quarto, and it comes with so many batteries included that you'd think it was Christmas. One of those batteries…

My previous post announced a Rust-based command line tool for generating Quarto projects from Observable Notebooks. Some folks may not want to use yet-another command line tool, and it dawned on me that it'd be more convenient to just do…

The previous post had some hacky R code to grab seekrit JSON data in ObservableHQ (OHQ) Notebooks and spit out a directory with a Quarto qmd and any associated FileAttachments. Holding firm to my "no more generic public R packages" decree,…

Quarto is amazing! And, it's eating the world! OK. Perhaps not the entire world. But it's still amazing! If you browse around the HQ, you'll find many interesting notebooks. You may even have a few yourself! Wouldn't it be great if you…

Apple is in the final stages of shuttering the DarkSky service/API. They've replaced it with WeatherKit, which has both an xOS framework version as well as a REST API. To use either, you need to be a member of the Apple Developer Program…

My {darksky} package has been around for years, now, and the service that powers it was purchased by Apple before the pandemic. The DarkSky API is slated to be shuttered in December of this year and is being replaced by Apple's WeatherKit…

I realize you have to be living under a rock in the U.S. to not know that yesterday, was Juneteenth (a portmanteau of "June Nineteenth"). Still, I feel compelled to explain that said date marks the day when federal troops arrived in…

In today's newsletter Leonardo, an open source project and free online too from Adobe that lets you make great and accessible color palettes for use in UX/UI design and data visualizations! You can read the one newsletter section to get a…

Before digging into this post, I need to set some context. Friday, May 13, 2022 was my last day at my, now, former employer of nearly seven years. I'm not mentioning the company name 1 because this post is not about them 2 . This post is…

In my M-F newsletter today I mentioned an awesome Rust-based HTML/JS/CSS minifier library that also include batteries for a few other languages. There was no R port, so I made one using {rextendr}. The {rextendr} package makes is as easy…

Just a quick post to note that I'll still be doing long-form, single-topic posts here on the blog, but I've also started an M-F daily Substack newsletter \[free\], that introduces 1-3(ish) topics per-issue on stuff I find during my daily…

I came across a neat site that uses a Golang wasm function called from javascript on the page to help you see if your GitHub public SSH keys are "safe". What does "safe" mean? This is what the function checks for (via that…

The morning before work was super productive and there is a nigh-complete DSL for ESC/POS commands along with the ability to just print {ggplot2}/{grid} object. I changed the package name to {escpos} since it is no longer just plot object…

At the end of March, I caught a fleeting tweet that showcased an Epson thermal receipt printer generating a new "ticket" whenever a new GitHub issue was filed on a repository. @aschmelyun documents it well in this blog post. It's a pretty…

RStudio's macOS Electron build is coming along quite nicely and is blazing fast on Apple Silicon. I like to install the dailies, well, daily!; and, of late, RStudio and Quarto are joined at the hip. As a result, I regularly found myself…

via Zack Whittaker on Twitter… Download Tor Browser: http://torproject.org/download Open this URL in Tor to access BBC (??): https://www.bbcweb3hytmzhn5d532owbu6oqadra5z3ar726vq5kgwwn6aucdccrad.onion/ukrainian Open this URL in Tor to…

Ukrainian Presidential Press Office via AP Ukrainian President Volodymyr Zelenskyy addresses the nation on a live TV broadcast in Kyiv, Ukraine, on Feb. 22,…

If you've been following my inane tweets and non-technical blog posts for any length of time since 2015, you likely know the 2016 election cycle broke me more than just a tad, with each subsequent month of the Trump presidency adding a bit…

After a Twitter convo about weather stations I picked up a WeatherFlow Tempest. Setup was quick, but the sensor package died within 24 hours. I was going to give up on it but I had written an R package (for the REST API & UDP broadcast…

The New York Times had a \[tragic\] story on Covid deaths today and one of their plots really stuck with me for how well it told that part of the story. NOTE: The red panel highlights are off a bit as I manually typed the data in (I only…

The Moderna booster level drained me all day on Dec 1 and did what jab two did during the overnight period (achy enough to wake me up and not get back to slumber easily). To try to wear myself down, I decided to practice a bit of R with…

I've been wanting to create a custom MaxMind mmdb file for alternate IPv4 range classifications for a while, and finally had an opportunity to do so for the $DAYJOB. MaxMind mmdb files are small and easy to work with and provide lightning…

The 30-Day Map Challenge is on again, and I'm hoping to be able to scrounge some time to get an entry for each day. Day 2 is lines (Day 1 was posted on Twitter only) and — while I'm hoping to focus on saving U.S. democracy for the majority…

RStudio is continuing to refine their ?? new product version string scheme, with the most recent change using something like the following for daily…

Vox grabbed some data from the Kaiser Family Foundation and did a story a few days ago on it, then posted a different visualization of it that attracted some attention: I'm a pretty ardent donut detractor, but I have to also admit that…

Version 2 of RSwitch — the macOS menubar utility that enables fast and seamless switching between R versions on macOS and also provides notifications for new versions of RStudio Dailies has had a hasty update to deal with an error…

Hot on the heels of the previous CyberDefenders Challenge Solution comes this noisy installment which solves their Acoustic challenge. You can find the source Rmd on GitHub, but I'm also testing the limits of WP's markdown rendering and…

It was a rainy weekend in southern Maine and I really didn't feel like doing chores, so I was skimming through RSS feeds and noticed a link to a PacketMaze challenge in the latest This Week In 4n6. Since it's also been a while since I've…

WWDC 2021 is on this week and many new fun things are being introduced, including some data science-friendly additions to the frameworks that come with Xcode 13 and available on macOS 12+ (and its \OS cousins). Specifically, Apple has made…

My archinfo utility is a small macOS command line application that displays process id, name, and architecture (arm64 or x86\64). Version 0.4.0 adds the following…

My {cdcfluview} package started tossing erros on CRAN just over a week ago when the CDC added an extra parameter to one of the hidden API endpoints that the package wraps. After a fairly hectic set of days since said NOTE came, I had time…

The fine folks over at @ObservableHQ released a new javascript exploratory visualization library called Plot last week with great fanfare. It was primarily designed to be used in Observable notebooks and I quickly tested it out there (you…

On or about Friday evening (May 7, 2021) Edge notified me that the Feedly Mini extension (one of the only extensions I use as extensions are dangerous things) was remove from the store due to "malware". Feedly is used by many newshounds,…

Rather than continue to generate daily images with R, I threw together an Observable notebook that takes advantage of the CDC COVID-19 county data datasette (provided by Simon Willison) and the new {Plot} library (by the @ObservableHQ…

There's a semi-infrequent-but-frequent-enough-to-be-annoying manual task at $DAYJOB that involves extracting a particular set of strings (identifiable by a fairly benign set of regular expressions) from various interactive text sources…

There are a plethora of amazingly useful Golang libraries, and it has been possible for quite some time to use Go libraries with Swift. The advent of the release of the new Apple Silicon/M1/arm64 architecture for macOS created the need for…

The last post showed how to work with the macOS mdls command line XML output, but with {swiftr} we can avoid the command line round trip by bridging the low-level Spotlight API (which mdls uses) directly in R via Swift. If you've already…

(reminder: Quick Hits have minimal explanatory blathering, but I can elaborate on anything if folks submit a comment). I'm playing around with Screen Time on xOS again and noticed mdls (macOS command line utility for getting file metadata)…

Guillaume Pressiat (@GuillaumePressiat) did a solid post & video on using Selenium to scrape a paginated table from understat[.]com/league/EPL/2020 (I just cannot bring myself to provide an active link to any SportsBall site). He does a…

Greynoise helps security teams focus on potential threats by reducing the noise from logs, alerts, and SIEMs. They constantly watch for badly behaving internet hosts, keep track of the benign ones, and use this research to classify IP…

Apple M1/Apple Silicon/arm64 macOS can run x8664 programs via Rosetta and most M1 systems currently (~March 2021) very likely run a mix of x8664 and arm64 processes. Activity Monitor can show the architecture: but command line tools such…

Brim Security maintains a free, Electron-based desktop GUI for exploration of PCAPs and select cybersecurity logs: along with a broad ecosystem of tools which can be used independently of the GUI. The standalone or embedded zqd server, as…

Horrible puns aside, hopefully everyone saw the news, earlier this week, from @thomasp85 on the evolution of modern typographic capabilities in the R ecosystem. Thomas (and some cohorts) has been working on {systemfonts}, {ragg}, and…

readcsv( file = "~/Data/applehealthexport/electrocardiograms/ecg2020-09-24.csv", # this is extracted below skip = 12, colnames = "µV" ) %>% mutate( idx = 1:n() ) ->…

Presented without much commentary since I stopped once {ggrepel} and {graphlayouts} failed (RStudio doesn't support it yet, either, which I knew). The following steps will get you a fully working and STUPID FAST fully native ARM64 M1/Apple…

If you've been following me around the internets for a while you've likely heard me pontificate about the need to be aware of and reduce — when possible — your personal "cyber" attack surface. One of the ways you can do that is to install…

I was chatting with a fellow Amazon Athena user and the topic of using Presto functions such as approx\distinct() via {d\[b\]plyr} came up and it seems it might not be fully common knowledge that any non-already translated function is…

(this is an unrolled Twitter thread converted to the blog since one never knows how long content will be preserved anywhere…

The past two posts have (lightly) introduced how to use compiled Swift code in R, but they've involved a bunch of "scary" command line machinations and incantations. One feature of {Rcpp} I've always ? is the cppFunction() ("r-lib" zealots…

The previous post introduced the topic of how to compile Swift code for use in R using a useless, toy example. This one goes a bit further and makes a case for why one might want to do this by showing how to use one of Apple's machine…

I've been on a Swift + R bender for a while now, but have been envious of the pure macOS/iOS (et al) folks who get to use Apple's seriously ++good machine learning libraries, which are even more robust on the new M1 hardware (it's cool…

There was an org that didn't see The data exfil hacking spree. A patch went up, our guard was down, Oh blow, SolarWinds,…

Last week I introduced a new bookdown series on how to embed R into a macOS Swift application. The initial chapters focused on core concepts and showed how to build a macOS compiled, binary command line application that uses embedded R for…

(Leading this with the periodic warning/reminder that this blog occasionally breaks from technical content and has category-based RSS feeds which can be used to ensure one never see non-technical content.) Every decent human (which…

I went completely daft this week and broke my months-long Twitter break due to the domestic terror event in my nation's capitol. I'll likely be resuming the break starting today. Whilst keeping up with the final descent of the U.S. into a…

// handling dots in named calls is a WIP = try R.evalParse("options(tidyverse.quiet = TRUE…

While the future of the Apache Drill ecosystem is somewhat in-play (MapR — a major sponsoring org for the project — is kinda dead), I still use it almost daily (on my local home office cluster) to avoid handing over any more money to…

It's been a while since I've posted anything R-related and, while this one will be brief, it may be of use to some R folks who have taken the leap into Big Sur and/or Apple Silicon. Stay to the end for an early Christmas…

NOTE: There's a unique feed URL for R/tech stuff — https://rud.is/b/category/r/feed/. If you hit the generic "subscribe" button b/c the vast majority of posts have been on that, this isn't one of those posts and you should probably delete…

The CDC continues to "deliver" in 2020, this time by changing the JSON response of one of the hidden APIs that my {cdcfluview} package wraps. CDC: So helpful! It was a quick fix, and version 0.9.2 passed automated CRAN checks in ~9.42…

(This is part 2 of n "quick hit" posts, each walking through some approaches to speeding up components of an iterative operation. Go here for part 1). Thanks to the aforementioned previous post, we now have a super fast way of reading…

(This is part 1 of n posts using this same data; n will likely be 2-3, and the posts are more around optimization than anything else.) I recently had to analyze HTTP response headers (generated by a HEAD request) from around 74,000 sites…

The Marshall Project has a solid story and set of visualizations on the impact of COVID-19 in U.S. prisons. They keep the data (and vis) regularly updated. They do great work and this is an important topic, but this visualization breaks my…

The incredibly talented folks over at Bishop Fox were quite generous this week, providing a scanner for figuring out PAN-OS GlobalProtect versions. I've been using their decoding technique and date-based fingerprint table to keep an eye on…

I've changed my years-long avatar to the blue Cap' shield because the colors of our flag have no business being displayed in any venue until Donald Trump is no longer President (one way or another). The red/white/blue triad has been…

Tis been a long time coming, but a minor change to default S3 parameters in tibbles finally caused a push of {sergeant} — the R package that lets you use the Apache Drill REST API via {DBI}, {dplyr}, or directly — to CRAN. The CRAN…

I (and, apparently, Gandalf O\o) are pleased to announce that RSwitch version 1.7.0 has been released. (Direct Download) RSwitch is a macOS menubar utility…

R 4.0.0 has been out for a while, now, and — apart from a case where merge() was slower than dirt — it's been really stable for at least me (I use it daily on macOS, Linux, and Windows). Sure, it came with some headline-grabbing…

I caught this tweet by Terence Eden about using Twitter image alt-text to "PGP sign" tweet and my mind immediately went to "how can I abuse this for covert communications, malicious command-and-control, and embedding R code in tweets?".…

Oven @ 425°F/218°C ; Sheet pan lined with parchment paper. Sift dry bits together. Cut up shortening into small squares and mix into the dry bits with a pastry blender. Combine eggs and yogurt. Add to ^^. Fold and knead gently just until…

The United States Centers for Disease Control (CDC from now on) has setup two new public surveillance resources for COVID-19. Together, COVIDView and COVID-NET provide similar weekly surveillance data as FluView does for influenza-like…

Brown sausage in oil then remove. Sauté onion in same oil (add more if dry) til clear. Add garlic and sauté for 1-2 minutes. Add back sausage and add in everything else and simmer for 45 minutes. To thicken, remove and pulse a few…

Just a quick note that thanks to a gentle nudge an updated version of {uaparser} --- a package that processes User Agent strings web clients send to servers --- is making its way to all the CRAN mirrors and is also available on CINC. The…

Waffle House announced it was closing hundreds of stores this week due to SARS-Cov-2 (a.k.a COVID-19). This move garnered quite a bit of media attention since former FEMA Administrator Craig Fugate used the restaurant chain as both an…

Oven @ 375°F. Paddle sugar, shortening and salt. 3-5 mins. Whisk eggs, milk, yogurt, vanilla & oil. In three batches, mix/fold ^^ into the paddled mixture. Sift together dry ingredients and mix until moist. Don't over-mix. Fold in chips.…

Über Tuesday has come and almost gone (some state results will take a while to coalesce) and I'm relieved to say that {catchpole} did indeed work, with the example code from before producing this on first run: If we tweak the buffer space…

For folks who are smart enough not to go near Twitter, I've been on a hiatus from the platform insofar as reading the Twitter feed goes. "Why" isn't the subject of this post so I won't go into it, but I've broken this half-NYE resolution…

It seems that the need for MX, DKIM, SPF, and DMARC records for modern email setups were just not enough acronyms (and setup tasks) for some folks, resulting in the creation of yet-another-acronym --- BIMI, or, Brand Indicators for Message…

As the maintainer of RSwitch --- and developer of my own (for personal use) macOS, iOS, watchOS, iPadOS and tvOS apps --- I need the full Apple Xcode install around (more R-focused macOS folk can get away with just the command-line tools…

macOS R users who tend to work on the bleeding edge likely noticed some downtime at this past weekend. Part of the issue was an SSL/TLS certificate expiration situation. Moving forward, we can monitor this with R using the super spiffy…

Each year the World Economic Forum releases their Global Risk Report around the time of the annual Davos conference. This year's report is out and below are notes on the "cyber" content to help others speed-read through those sections (in…

UPDATE 2020-02-11 Apple now supports downloading transactions as CSV or OFX! (via…

Before we start wrapping foreign language code we need to make sure that basic R packages can be created. If you've followed along from the previous post you have everything you need to get started here. Just to make sure, you should be…

Offspring #4 eats pretty healthy, normally, but likes to indulge in sugary confections on occasion. He was a big help clearing away the massive amount of snow that accumulated in the last winter storm of 2019 so I decided to start the new…

The R language and RStudio IDE are a powerful combination for "getting stuff done", and one aspect of R itself that makes it especially useful is the ability to use it with other programming languages via a robust foreign language…

I woke up to Axios' "1 Big Thing" ridgeline chart showing the crazy that was the 2019 news cycle: and, I decided to reproduce it in…

The right jolly old elves over at Alteryx created a "Santalytics" challenge back in 2016 to see if their community members could help Santa deliver presents to kids all across the globe. They posted data for four challenges along with…

I had need to present a wall-of-text to show off a giant list of SSL certificate alternate names and needed the entire list to fit on one slide (not really for reading in full, but to show just how many there were in a way that a simple…

Apple has brought Screen Time to macOS for some time now and that means it has to store this data somewhere. Thankfully, Sarah Edwards has foraged through the macOS filesystem for us and explained where these bits of knowledge are in her…

I posted a visualization of email safety status (a.k.a. DMARC) of the Fortune 500 (2017 list) the other day on Twitter and received this spiffy request from…

An RSwitch user, lcolladotor filed a most-welcome issue letting me know that the core functionality of the switcher was busted ?. After testing out the 1.5.1 release candidate I had made a "harmless" & "clever" change to reduce some…

This past week @propublica linked to a really spiffy resource for getting an overview of a Twitter user's profile and activity called accountanalysis. It has a beautiful interface that works as well on mobile as it does in a real browser.…

RSwitch is a macOS menubar application that works on macOS 10.14+ and provides handy shortcuts for developing with R on macOS. Version 1.5.0 brings a reorganized menu system and the ability to manage and make connections to RStudio Server…

The latest round of the 2020 Democratic debates is over and the data from all the 2019 editions of the debates have been added to {ggchicklet}. The structure of the debates2019 built-in dataset has changed a…

A minor update to RSwitch has been released. Apart from some internal code reorganization there are three user-facing changes. First, RSwitch is now notarized! That means you won't get a notice about it being from an "unidentified…

Swift 5 has been so much fun to hack on that there's a new update to macOS R-focused mebubar utility RSwitch available. Along with the app comes a new dedicated RSwitch landing page and a new user's guide since it has enough features to…

It's only been a couple days since the initial version of my revamped take on RSwitch but there have been numerous improvements since then worth mentioning. For starters, there's a new app icon that uses the blue and gray from the official…

I caught this post on the The Surprising Number Of Programmers Who Can’t Program from the Hacker News RSS feed. Said post links to another, classic post on the same subject and you should read both before continuing. Back? Great! Let's dig…

At the bottom of the R for macOS Developer's Page there's mention of an "other binary" called "RSwitch" that is "a small GUI that allows you to switch between R versions quickly (if you have multiple versions of R framework installed)."…

It's been yet-another weirdly busy summer but I'm finally catching up on noting some recent-ish developments in the blog. First up is a full rewrite of the {wand} pacakge which still uses magic#Magicnumbersinfiles) but is 100% R code (vs a…

Despite being a full-on denizen of all things digital I receive a fair number of dead-tree print magazines as there's nothing quite like seeing an amazing, large, full-color print data-driven visualization up close and personal. I also…

The {waffle} package got some ? this week and now has a substantially improved geomwaffle() along with a brand new sibling function geompictogram() which has all the powerful new features of geomwaffle() but lets you use Font Awesome 5…

The first U.S. Democratic debates of the 2020 election season were held over two nights this past week due to the daft number of candidates running for POTUS. The spiffy @NYTgraphics folks took the tallies of time spent blathering by each…

The r-project.org domain had some temporary technical difficulties this week (2019-29) that made reaching R-related resources problematic for a bunch of folks for a period of time. Incidents like this underscore the need for regional and…

Despite being on holiday I'm getting in a bit of non-work R coding since the fam has a greater ability to sleep late than I do. Apart from other things I've been working on a PR into {lutz}, a package by @andyteucher that turns lat/lng…

I've become a big fan of the {logspline} package over the past ~6 months and decided to wrap up a manual ggplot2 plotting process (well, it was at least in an RStudio snippet) into a small {ggplot2} Stat to make it easier to visualize…

I mentioned EtherCalc in a previous post and managed to scrounge some time to put together a fledgling {ethercalc} package (it's also on GitLab, SourceHut, Bitbucket and GitUgh, just sub out the appropriate URL prefix). I'm creating a…

This morning, @kairyssdal tweeted out the following graphic from…

I did another twitter thread on the aforeblogged MowerPlus database as I explored the tables after a second mow to determine what identified a unique mowing "session" (using John Deere's terms). This is the…

I was pretty brutal to Apple earlier this week in a Twitter thread that I tried to craft so it occurred in-line with the WWDC live stream (which might be something you want to remember as/if you read on). I really don't care about…

A user of the {ggalt} package recently posted a question about how to add points to a geomdumbbell() plot. For now, this is not something you can do with geomdumbbell() but with a bit of data wrangling you can do this in a pretty…

In a recent previous post I brazenly talked over the "hard parts" of how I got to the target SQLite file that houses "mowing history" for what has become my weekend obsession. So, we'll cover just how to do that (find things in iOS…

For the first time ever we got a new riding mower this weekend. We've always haggled to keep the one sellers were using with any given house we've purchased over the years (that was big enough for a yard that "requires" a riding mower). We…

The Feedly category I have setup for git-stalking has indicated a fairly massive interest in Joshua Levy's The Art of the Command Line. What is "The Art of the Command Line"? To quote the…

A fair bit of time ago the {gdns} package made its way to CRAN to give R users the ability to use Google's (at that time) nascent support for DNS over HTTPS (DoH). A bit later on Cloudflare also provided a global DoH endpoint and that…

I've talked about the retailpocalypse before and this morning I was greeted with the news about Dressbarn closing all 650 stores as I fired up a browser. I tweeted some pix and data but not everyone is on Twitter so I'm just posting a…

So, we have to create a map of historical tornado frequency trends on deadline. We emailed researchers and received three txt files. One is a set of latitudes, another longitudes, and the final one is the trend value. It's gridded data.…

I'm using GitUgh links here b/c the issue was submitted there. Those not wishing to be surveilled by Microsoft can find the macOS QuickLook plugin project and {rdatainfo} project in SourceHut and GitLab (~hrbrmstr and hrbrmstr accounts…

I've mentioned {htmlunit} in passing before, but did not put any code in the blog post. Since I just updated {htmlunitjars} to the latest and greatest version, now might be a good time to do a quick demo of it. The {htmlunit}/{htmunitjars}…

UPDATE 2019-04-17 — The example at the bottom which shows that the, er, randomly chosen site has the offending tag present is an old result. As of this update timestamp, that robots noindex tag is not on the site. Since the presence status…

Like more posts than I care to admit, this one starts innocently enough with a tweet by…

Researching "the internet" (i.e. $DAYJOB) means having to deal with a ton of "unique" (I'm being kind) data formats. This is ultimately a tale of how I performed full-text searches across one of them. It all started off innocently enough.…

@ted\dunning recently updated the t-Digest algorithm he created back in 2013. What is this "t-digest"? Fundamentally, it is a probabilistic data structure for estimating any percentile of distributed/streaming data. Ted explains it quite…

I saw a second post on turning htmlwidgets into interactive Twitter Player cards and felt somewhat compelled to make creating said entities a bit easier so posited the…

There's been alot of talk about "dependencies" in the R universe of late. This is not really a post about that but more of a "really, don't do this" if you decide you want to poke the dependency bear by trying to build a deeply flawed…

The fine folks over at @PacketTotal bequeathed an API token on me so I cranked out an R package for it to enable more dynamic investigations work (RStudio makes for an amazing incident responder investigations console given that you can…

In the previous post I tried to explain what Content Security Policies (CSPs) are and how to work with them in R. In case you didn't RTFPost the TLDR is that CSPs give you control over what can be loaded along with your web content and can…

I blathered alot about HTTP headers in the last post. In the event you wanted to dig deeper I threw together a small package that will let you grab HTTP headers from a given URL and take a look at them. The README has examples for most…

Not flagging this with an "R" tag since I don't want to spam R-bloggers but I mentioned here that I'd be disabling logging on https://cinc.rud.is and https://git.rud.is and I wanted to follow up on that with an addendum that I've opted to…

In the "Changes on CRAN" section of the latest version of the The R Journal (Vol. 10/2, December 2018) had this short blurb entitled "CRAN mirror…

The in-dev htmlunit package for javascript-"enabled" web-scraping without the need for Selenium, Splash or headless Chrome relies on the HtmlUnit library and said library just released version 2.34.0 with a wide array of changes that…

I've got a work-in-progress drat\-ified CRAN-like repo for (eventually) all my packages over at CINC? ("CINC is not CRAN" and it also sounds like "sync"). This is in parallel with a co-location/migration of all my packages to SourceHut…

There are two fledgling rJava-based R packages that enable working with the AWS SDK for…

Those outside the Colonies may not know that Payless—a national chain that made footwear affordable for millions of 'Muricans who can't spare $100.00 USD for a pair of shoes their 7 year old will outgrow in a year— is closing. CNBC also…

WiGLE has been around a while and is a great site to explore the pervasiveness or sparsity of Wi-Fi (and cellular) networks around the globe. While interactive use is fun, WiGLE also has a free API (so long as you obey the EULA and aren't…

I must preface this post with the posit that if you're doing anything interactive() with Amazon Athena you should seriously consider just using their free ODBC drivers as it's the easiest way to wire them up to R DBI- and tidyverse-wise.…

I've got a post up over at $DAYJOB's blog on using the ropendata? package to access the ginormous and ever-increasing amount of internet telemetry (scan) data via the Rapid7 Open Data API. It's super-R-code-heavy but renders surprisingly…

Over at $DAYJOB's blog I've queued up a post that shows how to use our new ropendata? package to work with our Open Data portal's API. I'm not super-sure when it's going to be posted so keep an RSS reader fixed on https://blog.rapid7.com/…

We worked pretty hard over at $DAYJOB on helping to quantify and remediate a fairly significant configuration weakness in Ubiquiti network work gear attached to the internet. Ubiquiti network gear — routers, switches, wireless access…

The urlscan? package (an interface to the urlscan.io API) is now at version 0.2.0 and supports urlscan.io's authentication requirement when submitting a link for analysis. The service is handy if you want to learn about the details — all…

A major new release of Homebrew has landed and now includes support for Linux as well as Windows! via the Windows Subsystem for Linux. There are overall stability and speed improvements baked in as well. The aforelinked notification has…

Luke Whyte posted an article (apologies for a Medium link) over on Towards Data Science showing how to use a command line workflow involving curl, node and various D3 libraries and javascript source files to build a series of SVG static…

The seymour? Feedly API package has been updated to support subscribing to RSS/Atom feeds. Previously the package was intended to just treat your Feedly as a data source, but there was a compelling use case for enabling subscription…

(A reminder to folks expecting "R"/"data science" content: the feed for that is at https://rud.is/b/category/r/feed/ if you don't want to see the occasional non-R/datasci…

As someone who measures all kinds of things on the internet as part of his $DAYJOB, I can say with some authority that huge swaths of organizations are using cloud-services such as Google Apps, Dropbox and Office 365 as part of their…

Version 0.6.0 of the hrbrthemes? package should be hitting a CRAN mirror near you soon. Apart from some general documentation and code cleanup this release includes the dark theme folks have been seeing in blog posts and tweets over the…

The splashr package \[srht|GL|GH\] — an alternative to Selenium for javascript-enabled/browser-emulated web scraping — is now at version 0.6.0 (still in dev-mode but on its way to CRAN in the next 14 days). The major change from version…

By now, even remote villages on uncharted islands in the Pacific know that the U.S. is in the midst of a protracted partial government shutdown. It's having real impacts on the lives of Federal government workers but they aren't the only…

More than just sergeant has been hacked on recently, so here's a run-down of various ?…

It was probably not difficult to discern from my previous Drill-themed post that I'm fairly excited about the Apache Drill 1.15.0 release. I've rounded out most of the existing corners for it in preparation for a long-overdue CRAN update…

Today's RSS feeds picked up this article by Marianne Sullivan, Chris Sellers, Leif Fredrickson, and Sarah Lamdanon on the woeful state of enforcement actions by the U.S. Environmental Protection Agency (EPA). While there has definitely…

Apache Drill is an innovative distributed SQL engine designed to enable data exploration and analytics on non-relational datastores \[...\] without having to create and manage schemas. \[...\] It has a schema-free JSON document model…

Well, 2018 has flown by and today seems like an appropriate time to take a look at the landscape of R bloggerdom as seen through the eyes of readers of R-bloggers and R Weekly. We'll do this via a new package designed to make it easier to…

Phishing is \[still\] the primary way attackers either commit a primary criminal act (i.e. phish a target to, say, install ransomware) or is the initial vehicle used to gain a foothold in an organization so they can perform other criminal…

I can't seem to free my infrequently-viewed email inbox from "you might like!" notices by the content-lock-in site Medium. This one made it to the iOS notification screen (otherwise I'd've been blissfully unaware of it and would have saved…

The libcurl library (the foundational library behind the RCurl and curl packages) has switched to using OpenSSL's default ciphers since version 7.56.0 (October 4 2017). If you're a regular updater of curl/httr you should be fairly current…

Despite their now inherent evil status, GitHub has some tools other repository aggregators do not. One such tool is the free vulnerability alert service which will scan repositories for outdated+vulnerable dependencies. Now, "R" is nowhere…

If you're an R/RStudio user who has migrated to Mojave (macOS 10.14) or are contemplating migrating to it, you will likely eventually run into an issue where you're trying to access resources that are in Apple's new hardened filesystem…

The CBC covered the recent (as of the original post-time on this blog entry) Quebec elections and used a well-crafted hex grid map to display results: They have a great 'splainer on why they use this type of map. Thinking that it may be…

In my semi-daily run of brew update I noticed that proj4 had been updated to 5.2. I kinda "squeee"'d since (as the release notes show) the Equal Earth projection was added to it (+proj=eqearth). As the team who created the projection…

Despite having sailed through the core components of this year's Talk Like A Pirate Day R post a few months ago, time has been an enemy of late so this will be a short post that others can build off of, especially since there's lots more…

The wayback? package has had an update to more efficiently retrieve mementos and added support for working with the Internet Archive's advanced search+scrape…

I was chatting with some cyber-mates at a recent event and the topic of cyber attacks on the U.S. power-grid came up (as it often does these days). The conversation was brief, but the topic made its way into active memory and resurfaced…

The sergeant? package has a minor update that adds REST API coverage for two "new" storage endpoints that make it possible to add, update and remove storage configurations on-the-fly without using the GUI or manually updating a config…

Nowadays (I've seen that word used so much in journal articles lately that I could not resist using it) I'm using world tile grids more frequently as the need arises to convey the state of exposure of various services at a global (country)…

Apple has run the death bell on 32-bit macOS apps and, if you're running a recent macOS version on your Mac (which you should so you can get security updates) you likely see this alert from time-to-time: If you're like me, you click…

It's been over a year since Headless Chrome was introduced and it has matured greatly over that time and has acquired a pretty large user base. The TLDR on it is that you can now use Chrome as you would any command-line interface (CLI)…

If you've got a directory full of Bro NSM logs, it's easy to work with them in Apache Drill since they're just tab-separated values (TSV) files by default. The most tedious part is mapping the columns to proper types and hopefully this…

Apache Drill 1.14.0 was recently released, bringing with it many new features and a temporary incompatibility with the current rev of the MapR ODBC drivers. The Drill community expects new ODBC drivers to arrive shortly. The sergeant? is…

The development version of splashr now support authenticated connections to Splash API instances. Just specify user and pass on the initial splashr::splash() call to use your scraping setup a bit more safely. For those not familiar with…

A previous post showed how to use a different authentication provider to wire up Apache Zeppelin and Amazon Athena. As noted in that post, Zeppelin is a "notebook" alternative to Jupyter (and other) notebooks. Unlike Jupyter, I can…

Apache Zeppelin is a "notebook" alternative to Jupyter (and other) notebooks. It supports a plethora of kernels/interpreters and can do a ton of things that this post isn't going to discuss (perhaps future ones will, especially since it's…

I had to processes a bunch of emails for a $DAYJOB task this week and my "default setting" is to use R for pretty much everything (this should come as no surprise). Treating mail as data is not an uncommon task and many R packages exist…

After reading this interesting analysis of "How Often Are Americans' Accounts Breached?" by Gaurav Sood (which we need more of in cyber-land) I gave in to the impulse to do some gg-doodling with the "Have I Been Pwnd" JSON data he used.…

Continuing the blog's UDF theme of late, there are two new UDF kids in…

There are many ways to gather Twitter data for analysis and many R and Python (et al) libraries make full use of the Twitter API when building a corpus to extract useful metadata for each tweet along with the text of each tweet. However,…

A previous post explored how to deal with Amazon Athena queries asynchronously. The function presented is a beast, though it is on purpose (to provide options for folks). In reality, nobody really wants to use rJava wrappers much anymore…

I've blogged about how to use Amazon Athena with R before and if you are a regular Athena user, you've likely run into a situation where you prepare a dplyr chain, fire off a collect() and then wait. And, wait. And, wait. And, wait.…

Both my osqueryr and macthekinfe packages have had a few updates and I wanted to put together a fun example (it being Friday, and all) for what you can do with them. All my packages are now on GitHub and GitLab and I'll be maintaining them…

I've mentioned @stiles before on the blog but for those new to my blatherings, Matt is a top-notch data journalist with the @latimes and currently stationed in South Korea. I can only imagine how much busier his life has gotten since that…

You can catch a bit of the @rOpenSci 2018 Unconference experience at home w with this short-ish 'splainer video on how to use the new middlechild package (https://github.com/ropenscilabs/middlechild) & mitmproxy to automagically create…

The forthcoming RStudio 1.2 release has a new "Jobs" feature for running and managing background R tasks. I did a series of threaded screencaps on Twitter but that doesn't do the feature justice. So I threw together a quick 'splainer on…

A colleague asked if I would blog about how I crafted the grid of world tile grids in this post and I accepted the challenge. The technique isn't too hard as it just builds on the initial work by Jon Schwabish and a handy file made by…

After seeing Fira Sans in action in presentations at eRum 2018 I felt compelled to add hrbrthemes support for it so I made a firasans? extension to it that uses Fira Sans Condensed and Fira Code fonts for ggplot2 graphics. But I really…

Most modern operating systems keep secrets from you in many ways. One of these ways is by associating extended file attributes with files. These attributes can serve useful purposes. For instance, macOS uses them to identify when files…

Regular readers will recall the "utility belt" post from back in April of this year. This is a follow-up to a request made asking for a list of all the % infix functions in those files. We're going…

I'm going to (eventually) write a full post on the package I'm mentioning in this one : osqueryr?. The TLDR on osqueryr is that it is an R DBI wrapper (that has just enough glue to also be plugged into dbplyr) for osquery?. The TLDR on…

I pen this mini-tome on "GDPR Enforcement Day". The spirit of GDPR is great, but it's just going to be another Potempkin Village in most organizations much like PCI or SOX. For now, the only thing GDPR has done is made GDPR consulting…

The cloc Perl script (yes, Perl!) by Al Danial (https://github.com/AlDanial/cloc) has been one of the go-to tools for generating code metrics. Given a single file, directory tree, archive, or git repo, cloc can speedily give you metrics on…

One file that's in XML that I only occasionally have to process is the IANA IPv4 Address Space Registry. If you visited that link you may have uttered "Hey! That's not XML it's HTML!", to wit --- I would respond --- "Well, HTML is really…

RIPE 76 is going on this week and --- as usual --- there are scads of great talks. The selected ones below are just my (slightly) thinner slice at what may have broader appeal outside pure networking circles. Do not read anything more into…

The U.S. FBI Internet Crime Complaint Center was established in 2000 to receive complaints of Internet crime. They produce an annual report, just released 2017's edition, and I need the data from it. Since I have to wrangle it out, I…

but I wanted to do more than that since Noam took 20 minutes out his day this week (with no advance warning) to speak to my intro-to-stats class about his work and R. The Twitter request was (ultimately) a question on how to use R to post…

I spent some time this morning upgrading the JDBC driver (and changing up some supporting code to account for changes to it) for my metis package? which connects R up to Amazon Athena via RJDBC. I'm used to JDBC and have to deal with Java…

This week's edition of Data is Plural had two really fun data sets. One is serious fun (the first comprehensive data set on U.S. evictions, and the other I knew about but had forgotten: The Federal Register Executive Order (EO) data…

If you come here often you've noticed that I've been writing a semi-frequent series on using the Feedly API with R. A recent post was created to help someone use the API. It worked for them but — as you can see in the comment — an…

@mkjcktzn asked if one can access Feedly "Saved for Later" items via the API. The answer is "Yes!", and it builds off of that previous post. You'll need to read it and get your authentication key (still no package ?) before continuing.…

Since I just railed against Congress for being a bit two-faced about privacy I thought some rud.is site disclosure would be in order. At present, third-party tracking is limited…

I apologize up-front for using bad words in this post. Said bad words include "Facebook", "Mark Zuckerberg" and many referrals to entities within the U.S. Government. Given the topic, it cannot be helped. I've also left the R tag on this…

We'll use one of @hadleywickham's books since it's O'Reilly and they do epubs well. The archive package lets us treat the epub (which is really just a ZIP file) as a mini-filesystem and embraces "tidy" so we have lovely data frames to work…

Many R package authors (including myself) lump a collection of small, useful functions into some type of utils.R file and usually do not export the functions since they are (generally) designed to work on package internals rather than…

Modern MacBook Pros have a fairly useless (c'mon, admit it!) "Touch Bar" that did little more than cause severe ire in the developer community after turning a full-fledged, tactile Escape key into a hollow version if its former self.…

There's a yuge chance you're reading this post (at least initially) on R-Bloggers right now (though you should also check out R Weekly and add their live feed to your RSS reader pronto!). It's a central "watering hole" for R folks and is…

The 2018 IEEE Security & Privacy Conference is in May but they've posted their full proceedings and it's better to grab them early than to wait for it to become part of a paid journal offering. There are alot of papers. Not all match my…

A while back, Medium blogger 'Nykolas Z' posted results from a globally distributed DNS resolver test to find the speediest provider (NOTE: speed is not the only consideration when choosing an alternative DNS provider). While the test…

You have to have been living under a rock to not know about Cloudflare's new 1.1.1.1 DNS offering. I won't go into "privacy", "security" or "speed" concepts in this post since that's a pretty huge topic to distill for folks given the, now,…

The fs package makes it super quick and easy to find out just how much "package hoarding" you've been…

(R⁶ == brief, low-expository posts) @yoniceedee suggested I look at the Cambridge Analytics "whistleblower" testimony…

A quick Friday post to let folks know about three in-development R packages that you're encouraged to poke the tyres o\[fn\] and also jump in and file issues or PRs…

(FWIW I think I even caused myself pain due to the title of this blog post). Kaiser Fung (@junkcharts) did a makeover post on this chart about U.S. steel tariffs: Kaiser's makeover is good (Note: just because I said "good" does not mean…

I work with internet-scale data and do my fair share of macro-analyses on vulnerabilities. I use the R semver package for most of my work and wanted to blather on a bit about it since it's super-helpful for this work and doesn't get the…

Tis the season for finding out how well Maine fisherfolk did last year; specifically, Maine lobsterfolk. Most of the news sites in Maine do a feature on the annual landings (here's one from Bangor Daily News). There was a marked decline —…

The NPR Visuals Team created and maintains a javascript library that makes it super easy to embed iframes on web pages and have said documents still be responsive. The widgetframe R htmlwidget uses pym.js to bring this (much needed)…

Much of what I need to do for work-work involves using tools that are (for the moment) not in R. Today, I needed to test the validity of (and other processing on) DMARC records and I'm loathe to either reinvent the wheel or reticulate bits…

Time for another look at what's new and interesting in the #rstats world with the help of Peter Meissner's (@marvin\dpr) crossword.r?. The answers to last week's puzzle have been posted (it seemed to make more sense posting the answers a…

Peter Meissner (@marvin\dpr) released crossword.r? to CRAN today. It's a spiffy package that makes it dead simple to generate crossword puzzles. He also made a super spiffy javascript library to pair with it, which can turn crossword model…

We're doing some interesting studies (cybersecurity-wise, not finance-wise) on digital currency networks at work-work and --- while I'm loathe to create a geo-map from IPv4 geolocation data ---…

NOTE: The likelihood of this recipe being added to the recent practice bookdown book is slim, but I'll try to keep the same format for the blog…

The new year begins with me being on the hook to crank out a book on advanced web-scraping in R by July (more on that in a future blog post). The bookdown? package seemed to be the best way to go about doing this but I had only played with…

UPDATE 2018-01-01 --- this has been added to rtweet (GH version). A Twitter…

2017 is nearly at an end. We humans seem to need these cycles to help us on our path forward and have, throughout history, used these annual demarcation points as a time of reflection of what was, what is an what shall come next. To that…

First off, swatches has nothing to do with those faux-luxury brand Swiss-made timepieces. swatches is all about color. R/CRAN has plenty of color picking packages. The colourlovers? ? by @thosjleeper is one of my favs. But, color palettes…

GitHub (2017-12-21 post-time) started adding obnoxious boxes to their activity feed. I use that to discover new projects/developers. While I also have it in RSS and that's nice and compact, I do browse the activity feed directly. Those…

(For first-timers, R⁶ tagged posts are short & sweet with minimal expository; R⁶ feed) At work-work I mostly deal with medium-to-large-ish data. I often want to poke at new or existing data sets w/o working across billions of rows. I also…

I know some folks had a bit of fun with the previous post since it exposed the fact that I left out unique MQTT client id generation from the initial 0.1.0 release of the in-development package (client ids need to be unique). There have…

Most of us see the internet through the lens of browsers and apps on our laptops, desktops, watches, TVs and mobile devices. These displays are showing us --- for the most part --- content designed for human consumption. Sure, apps handle…

More sites are turning to services like Cloudflare due to just how stupid-easy it is to DDoS --- perform a (possibly Distributed) Denial of Service attack on -- a site. Sometimes the DDoS is intentional (malicious). Sometimes it's because…

This one was complex enough formatting-wise that I needed to iframe it below. Feel free to bust out of the iframe at any time. Some good follow-ups to this (if you're so inclined) would be to predict prices next year and/or clean up the…

Over the past few weeks, I had been noticing that some posts in the R-bloggers feed were getting truncated in Feedly. I don't remember when I noticed that since I usually click through immediately from the headline entry to the R-bloggers…

Our family has been reading, listening to and watching "A Christmas Carol" for just abt 30 years now. I got it into my crazy noggin to perform a sentiment analysis on it the other day and tweeted out the results, but a large chunk of the R…

Earlier this year, I made a package that riffed off of ProPublica's really neat voting cartograms (maps) for the U.S. House and Senate. You can see one for disaster relief spending in the House and one for the ACA "Skinny Repeal" in the…

NOTE: This is mainly for those of us in the Colonies, but some tips apply globally. Black Friday / Cyber Monday / Cyber November / Holiday ?hopping is upon us. You're going to buy stuff. You're going to use digital transactions to do so.…

(one more time: sub to the #rstats feed if you're only on the blog for #rstats/datasci…

A long time ago, in a github repo far, far away there lived a tiny package that made it possible to create equal area, square U.S. state cartograms in R dubbed statebins?. Three years have come and gone and --- truth be told --- I've never…

IBM has a new set of corporate typefaces --- dubbed "Plex" --- and has released them with a generous open license. IBM Plex Sans is not too shabby: (that image was grifted from a Font Squirrel preview page) The digit glyphs are especially…

By now, virtually every major media outlet has covered the "280 Apocalypse"™. For those still not "in the know", Twitter recently moved the tweet character cap to 280 after a "successful" beta test (some of us have different ideas of what…

Working remotely has many benefits, but if you work remotely in an area like, say, rural Maine, one of those benefits is not massively speedy internet connections. Being able to go fast and furious on the internet is one of the many things…

Unlike @noamross, I am not an epidemiologist (NOTE: Noam battles pandemics before breakfast, so be super nice to him) but I do like to find kindred methodologies in other disciplines to help foster the growth of cybersecurity into…

Despite my week-long Twitter consumption sabbatical (helped --- in part --- by the nigh week-long internet and power outage here in Maine), I still catch useful snippets from folks. My cow-orker @dabdine shunted a tweet by @terrencehart…

This past weekend, violent windstorms raged through New England. We — along with over 500,000 other Mainers — went "dark" in the wee hours of Monday morning and (this post was published on Thursday AM) we still have no utility-provided…

As many folks know, I live in semi-rural Maine and we were hit pretty hard with a wind+rain storm Sunday to Monday. The hrbrmstr compound had no power (besides a generator) and no stable/high-bandwidth internet (Verizon LTE was heavily…

Way back in July of 2009, the first version of the twitteR package was published by Geoff Jentry in CRAN. Since then it has seen 28 updates, finally breaking the 0.x.y barrier into 1.x.y territory in March of 2013 and receiving it's last…

I've blogged a bit about robots.txt --- the rules file that documents a sites "robots exclusion" standard that instructs web crawlers what they can and cannot do (and how frequently they should do things when they are allowed to). This is…

NOTE: If the usual aggregators are picking this up and there are humans curating said aggregators, this post is/was not intended as something to go into the "data science" aggregation sites. Just personal commentary with code in the event…

The Internet Archive recently announced a new service they've dubbed 'Third Eye'. This service scrapes the chyrons that annoyingly scroll across the bottom-third of TV news broadcasts. IA has a vast historical archive of TV news that…

Modern websites are complex beasts. They house photo galleries, interactive visualizations, web fonts, analytics code and other diverse types of content. Despite the potential for diversity, many web sites share similar "tech stacks" ---…

I occasionally hang out on StackOverflow and often use an answer as an opportunity to fill a package void for a particular need. docxtractr and qrencoder are two (of many) packages that were birthed from SO answers. I usually try to answer…

Is it fast? Listen bud, It's got C++ under the hood. Can you scrape, from a site? Test with canfetch(), TRUE == alright Hey, there There goes the…

International Code Talk Like A Pirate Day almost slipped by without me noticing (September has been a crazy busy month), but it popped up in the calendar notifications today and I was glad that I had prepped the meat of a post a few weeks…

I was socially engineered by @yoniceedee into creating today's post due to being prodded with this…

This is a post about the fictional company FredCo. If the context or details presented by the post seem familiar, it's purely coincidental. This is, again, a fictional story. Let's say FredCo had a pretty big breach that (fictionally)…

I've blathered about trust before 1 2 , but said blatherings were in a "what if" context. Unfortunately, the if has turned into a when, which begged for further blathering on a recent FOSS ecosystem cybersecurity incident. The gg\spiffy…

I've blogged about my in-development R package hgr a before and it's slowly getting to a CRAN release. There are two new features to it that are more useful in an interactive session than in a programmatic context. Since they build on each…

Putting this here to make it easier for others who try to Google this topic to find it w/o having to find and tediously search through other UDFs (user-defined functions). I was/am making a custom UDF for base64 decoding/encoding and ran…

Earlier this year, the GDELT Project released their Television Explorer that enabled API access to closed-caption tedt from television news broadcasts. They've done an incredible job expanding and stabilizing the API and just recently…

I recently posted about using a Python module to convert HTML to usable text. Since then, a new package has hit CRAN dubbed htm2txt that is 100% R and uses regular expressions to strip tags from text. I gave it a spin so folks could…

I'm pleased to announce that splashr is now on CRAN. (That image was generated with splashr::renderpng(url = "https://cran.r-project.org/web/packages/splashr/")). The package is an R interface to the Splash javascript rendering service. It…

I was about to embark on setting up a background task to sift through R package PDFs for traces of functions that "omit NA values" as a surprise present for Colin Fay and Sir…

There was a discussion on Twitter about the need to read in ".msg" files using R. The "MSG" file format is one of the many binary abominations created by Microsoft to lock folks and users into their platform and tools. Thankfully, they…

I needed to clean some web HTML content for a project and I usually use hgr::cleantext() for it and that generally works pretty well. The cleantext() function uses an XSLT stylesheet to try to remove all non-"main text content" from an…

I've blathered about my crawldelay project before and am just waiting for a rainy weekend to be able to crank out a follow-up post on it. Working on that project involved sifting through thousands of Web Archive (WARC) files. While I have…

(General reminder abt "R⁶" posts in that they are heavy on code-examples, minimal on expository. I try to design them with 2-3 "nuggets" embedded for those who take the time to walk through the code examples on their systems. I'll always…

The reticulate package provides a very clean & concise interface bridge between R and Python which makes it handy to work with modules that have yet to be ported to R (going native is always better when you can do it). This post shows how…

One of my tweets that referenced an excellent post about the ethics of web scraping garnered some…

It's no secret that I'm a fan of Apache Drill. One big strength of the platform is that it normalizes the access to diverse data sources down to ANSI SQL calls, which means that I can pull data from parquet, Hie, HBase, Kudu, CSV, JSON,…

Matt @stiles is a spiffy data journalist at the @latimes and he posted an interesting chart on U.S. Attorneys General longevity (given that the current US AG is on thin…

I'm extremely pleased to announce that the sergeant package is now on CRAN or will be hitting your local CRAN mirror soon. sergeant provides JDBC, DBI and dplyr/dbplyr interfaces to Apache Drill. I've also wrapped a few goodies into the…

list( Obama="http://m.rasmussenreports.com/publiccontent/politics/obamaadministration/obamaapprovalindexhistory", Trump="http://m.rasmussenreports.com/publiccontent/politics/trumpadministration/trumpapprovalindexhistory" ) %>% mapdf(~{…

The Apache Drill folks have a nice walk-through tutorial on how to analyze the Yelp Academic Dataset with Drill. It's a bit out of date (the current Yelp data set structure is different enough that the tutorial will error out at various…

I caught a mention of this project by Pete Warden on Four Short Links today. If his name sounds familiar, he's the creator of the DSTK, an O'Reilly author, and now works at Google. A decidedly clever and decent chap. The project goal is…

I caught the 0.7.0 release of dplyr on my home CRAN server early Friday morning and immediately set out to install it since I'm eager to finish up my sergeant package and get it on CRAN. "Tidyverse" upgrades aren't trivial for me as I…

I've been doing intermittent prep work for a follow-up to an earlier post on store closings and came across this CNN Money "article" on it. Said "article" is a deliberately obfuscated or lazily crafted series of GIF images that contain all…

I used reading a directory of CSVs as the foundational example in my recent post on idioms. During my exchange with Matt, Hadley and a few others -- in the crazy Twitter thread that spawned said post -- I mentioned that I'd personally…

Last year, I was able to sit back and lazily "RT" Julia Silge's excellent retrospective on her 2016 @rOpenSci "unconference" experience. Since Julia was not there this year, and the unconference experience is still in primary storage (LMD…

NOTE: I'll do my best to ensure the next post will have nothing to do with Twitter, and this post might not completely meet my R⁶ criteria. A single, altruistic, nigh exuberant R tweet about slurping up a directory of CSVs devolved quickly…

UPDATE: I was reminded that I made a more generic version of adobecolor to handle many types of swatch files which you can find on…

Most of the examples of working with most of the AWS services show basic username & password authentication. That's all well-and-good, but many shops use the AWS Security Token Service to provide temporary credentials and session tokens to…

If you follow me on Twitter or monitor @Rapid7's Community Blog you know I've been involved a bit in the WannaCry ransomworm triage. One thing I've been doing is making charts of the hourly contribution to the Bitcoin addresses that the…

Tagging this as #rstats\-related since many R coders use Travis-CI to automate package builds (and other things). Security researcher Ivan Vyshnevskyi did some ++gd responsible disclosure to the Travis-CI folks letting them know they were…

Political machinations are a tad insane in the U.S. these days & I regularly hit up @ProPublica & @GovTrack sites (& sub to the GovTrack e-mail updates) as I try to be an informed citizen, especially since I've got a Senator and…

As I've noted before, web scraping has some rules, even though they can be tough to find. This post made a very common mistake of not putting in a time delay between requests (a cardinal scraping rule) which we'll fix in a moment. There…

Once I realized that my planned, larger post would not come to fruition today I took the R⁶ post (i.e. "minimal expository, keen focus") route, prompted by a Twitter discussion with some R mates who needed to convert "lightly formatted"…

The intrepid @ma\salmon cranked out another blog post, remixing classical music schedule data from Radio Swiss Classic. It's a fun post and you should read it before continuing here. Seriously, click the link and go read it before…

I caught this "gem" in the Wall Street Journal tonight: It's pretty hard to compare store-to-store, even though it is fairly clear which ones are going-going-gone. If we want to see the relative percentage of each store closing and also…

It's likely you've seen the news regarding yet-another researcher showing off a phishing domain attack. The technique is pretty…

By now, word of the forcible deplanement of a medical professional by United has reached even the remotest of outposts in the #rstats universe. Since the news brought this practice to global attention, I found some aggregate U.S. Gov data…