Archives

eMudhra's breakable keys were the symptom: one firm signs off on over half the web's live CA intermediates. A CCADB dive into CA audits and weak keys.

A new {hrbrthemes} theme built on the semi-condensed cut of Archivo, plus humanize_duration() for putting computed time spans into chart subtitles.

rud.is/b left WordPress after 15 years. Why I moved the blog to Astro, how I converted the legacy posts, and how an ActivityPub sidecar keeps it federated.

Citrix published a security bulletin on September 27, 2026. It covers eight vulnerabilities in customer-managed NetScaler ADC and NetScaler Gateway. Two of them, CVE-2026-88771 and CVE-2026-88772, are under active exploitation. Both score…

Maine uses more heating oil per capita than any other state, which means every global distillate shock impacts Mainers more than most other states. The chart, above, captures three major spikes and some smaller…

Rapid7 is sunsetting AttackerKB, and the public API goes dark on August 18, 2026 . After that, the vulnerability discussions, community assessments, and Rapid7 analysis that accumulated on the platform become a dead…

I've written and said enough times that I'm tired of OpenAI and Anthropic pulling cybersecurity out as the headline AI risk. The framing is commercially convenient, and it flattens a more complex reality — these models can break plenty of…

PCAP forensics of the HMEFB worm: a self-propagating scanner with a Belarusian engineer's escape plea embedded in HTTP requests.

For the second time in a row, a post by cr0w on Mastodon regarding the Chrome release blog appearing to not render anything resulted in me firing up lynx to show a sub-second load and render, then finally doing something a bit more…

Cloudflare announced PACT this week — privacy-preserving tokens to separate humans from bots, backed by Google, Mozilla, Microsoft, and Shopify. The cryptography is solid. The governance model doesn't exist…

May 2026 dropped three critical Linux vulnerabilities on a near-weekly cadence, and the security discourse has mostly treated them as three separate bad days. They're not. Together they form a reliable, race-free, forensically quiet kill…

At our previous employer, the global deception and detection infrastructure generates tons of events that eventually make their way into an ever-growing data lake with (as of February 2026) 22 TB of PCAPs and 32 TB of session protocol…

In the past ~4 weeks I have personally observed some irrefutable things in "AI" that are very likely going to cause massive shocks to employment models in IT, software development, systems administration, and cybersecurity. I know some…

Back in 1997, a commercial airline captain noticed his fellow pilots had a problem: they'd gotten so used to following the magenta flight path lines on their fancy new navigation screens that they were forgetting how to actually fly the…

If you want to or do vote "yes" for question 1, you are not a real American, you are not a real Christian (if you profess to be one), you are not a decent human. You are at the very least a classist; you are also very likely a…

— We have no idea of why/who re: Wed's assassination. — Today is the anniversary of a tragic event that has enabled much of the harm caused by the GOP this year. — Trump is an adjudicated rapist & was involved at least in some way in the…

Details:…

ENISA published docs for their European Vulnerability Database (EUVD) — https://euvd.enisa.europa.eu/apidoc…

Meet Suriest — a new REST API service for validating Suricata rules, designed to be run by organizations to streamline rule validation workflows. Suriest supports Suricata 6.0 and later and offers features like secure configuration,…

MCP servers let you wire up external services/APIs in a standard way for LLM/GPT tool-calling and other forms of…

RSAC 2025 Sets A Dangerous Precedent for Cybersecurity Leadership

Introducing geolocus-cli: A fast, cross-platform command-line tool for IP enrichment using ONYPHE's free Geolocus database. This lightweight utility enables security professionals and network analysts to efficiently look up comprehensive IP metadata including geolocation, ASN information, organizati

Chris Krebs, the former director of the Cybersecurity and Infrastructure Security Agency (CISA), was fired by Donald Trump in 2020 for publicly affirming that the presidential election was secure and free from widespread fraud.…

A long, long time ago I can still remember How those CVEs would make me smile And I knew if I had my chance To patch a vuln or take a stance Maybe we’d be secure for a…

Just putting a marker out there, that I'm either fundamentally wrong, or we all are f'd this…

ICERAID (iceraid[.]us) presents itself as a "GovFi protocol" that purportedly delegates intelligence gathering tasks from law enforcement to citizens. According to their materials, it rewards users with cryptocurrency tokens for capturing…

Discover how Secretary Rubio's State Department determination classifies border, immigration, and trade policies as "foreign affairs functions," exempting them from the Administrative Procedure Act. Learn about potential consequences including surprise policy changes, expanded device searches, and e

On March 10, 2025, Xitter experienced major service disruptions throughout the day. Users couldn't access the platform on both mobile apps and the website. Here's what happened and why it…

Recent administrative changes at the Social Security Administration (SSA) reveal a concerning pattern of decisions that disproportionately impact vulnerable populations while being implemented in ways that limit public awareness and…

As a longtime Mainer and independent voter, I have watched Senator Susan Collins’ career with cautious optimism, hoping her self-branded image as a moderate willing to cross party lines might translate into principled leadership. Instead,…

Today, my Senator — Susan Collins — failed in her oath and duty to uphold the Constitution. She voted for the appointment of a traitor to head national intelligence, and is supporting someone for director of the Office of Management and…

Discover CVESky: A real-time tracker for CVE discussions on Bluesky, featuring daily vulnerability chatter cards, interactive heatmaps, and comprehensive CVE exploration tools. Built by GreyNoise Intelligence to monitor genuine security conversations beyond automated NVD feeds.

2024-08-30 UPDATE: Binary versions of this extension are available for amd64 Linux ( linuxamd64 & linuxamd64gcc4 ) and Apple Silicon. ( osxarm64…

100 Miles — rud.is blog archive

A concise Bash script for resolving Bluesky/ATproto handles to DID:PLC identifiers, simplifying user identification in the decentralized social network ecosystem.

I had not planned to blog this (this is an incredibly time-crunched week for me) but CERT/CC and CISA made a big deal out of a non-vulnerability in R, and it's making the round on socmed, so here we…

Get Command Line Access To Fantastical Schedules With An Apple Shortcuts And Bash Combo.

Learn how to work with VulnCheck's new community KEV+NVD CVE APIs in curl & Go, & checkout a new, small CLI tool to get both simultaneously.

A plea for fosstodon to continue its capricious and undocumented silliness.

Learn how to enable ligatures in RStudio Dailies using CSS and themes. Enhance your coding experience with granular control over ligatures for a more visually appealing and readable code.

Explore the impact of the libwebp supply chain disaster on Electron-based apps and discover Positron, a Golang macOS CLI tool designed to identify vulnerable apps. Stay informed and secure with our in-depth analysis and practical solutions.

Explore the impact of Rite-Aid's store closures on communities, learn about the company's bankruptcy, and discover how data collection scripts can track these changes. This analysis provides insights into the consequences of pharmacy deserts and the role of leadership in these developments."

The days are getting shorter and when we were visiting Down East Maine the other week, there was just a hint of some trees starting to change up their leaf palettes. It was a solid reminder to re-up my ~annual “foliage” plotting that I…

I'm just putting this here so the LLM/GPT overlords (and, mebbe even legacy search engines) can get it indexed and use the content from it to help…

While the future of Bluesky is nowhere near certain, it is most certainly growing. It's also the largest community of users for the AT…

HTTP Headers Hashing ( HHHash ) is a technique developed by Alexandre Dulaunoy to gen­erate a fingerprint of an HTTP server based on the headers it returns. It employs one-way hashing to generate a hash value from the list of header…

Lynn (of TITAA and general NLP wizardy fame) was gracious enough to lend me a Bluesky invite, so I could claim my handle on yet-another social media site. I'm still wary of it (as noted in one of this week's Drops ), but the AT protocol —…

Back in 2016, I did a post on {ggplot2} text annotations because it was a tad more challenging to do some of the things in that post back in the…

WebR has a template for React, but I'm not a fan of it or Vue (a fact longtime readers are likely tired of hearing right about now). It's my opinion and experience that Lit webcomponents come closer to “bare metal” webcomponents, which…

I was cranking out a blog post for work earlier this week that shows off just how many integrations our platform has. I won't blather about that content here, but as I was working on it, I really wanted to show off all the…

The official example WebR REPL is definitely cool and useful to get the feel for WebR. But, it is far from an ideal way to deal with it interactively, even as just a…

I won't wax long and poetic here since I've already posted the experiment that has all the…

The previous post brought lit-webr , to introduce Lit and basic…

My recent hack-y WebR experiments have been using Reef which is an even ligher -weight javascript web components-esque library, and it's a bit more (initially) accessible than Lit. Lit's focus on "Web Components-first" means that you are…

After writing the initial version of a tutorial on wrapping and binding R functions on the javascript side of WebR , I had a number of other WebR projects on the TODO list. But, I stuck around noodling on the whole "wrapping & binding"…

It's difficult to believe it has only been a couple of weeks since WebR has been around. But that might just be my perception. The spike protein invasion has significantly increased sedentary time, and that has enabled me to focus on this…

I have graphics working in Vanilla JS WebR, now, and I'll cover the path to that in two…

I have a post coming on using base and {ggplot2} plots in VanillaJS WebR, but after posting some bits on social media regarding how slow {ggplot2} is to deal with, I had some "performance"-related inquiries, which led me down a rabbit hole…

Let's walk through how to set up a ~minimal HTML/JS/CSS + WebR-powered "app" on a server you own. This will be vanilla JS (i.e. no React/Vue/npm/bundler) you can hack on…

WebR 0.1.0 was released ! I had been git-stalking George (the absolute genius who we all must thank for this) for a while and noticed the GH org and repos being updated earlier this week, So, I was already pretty…

I've been (slowly) making my way through FOSDEM 23 presentations and caught up to Peter Lowe 's " Bizarre and Unusual Uses of DNS • Rule 53: If you can think of it, someone's done it in the DNS " talk. DNS oddities are items I collect…

I'll follow up with a more detailed post in a ~week or so, but if you are considering purchasing a Kucht appliance, please, please reconsider your decision. They have repeatedly lied to us (I have proof) and are incapable of manufacturing…

Ref:…

Visiting #2 and doing some $WORK -work, but intrigued with Hanukkah of Data since Puzzle 0 was solvable with a ZIP password cracker (the calendar date math seemed too trivial to bother…

Cross-post to Substack where I dropped some details on the newest browser in town: Arc .…

So, you've committed to abandoning the bird site, joined a ? instance, or three, and are now a citizen of the Fediverse. This is ??! But, what if you want to dig into this brave new universe a bit and see how it works? Or, perhaps you…

This is more of a test post after enabling some new Fediverse features on the…

I've been (mostly) keeping up with annual updates for my R/{sf} U.S. foliage post which you can find on GH . This year, we have Quarto , and it comes with so many batteries included that you'd think it was Christmas. One of those batteries…

My previous post announced a Rust-based command line tool for generating Quarto projects from Observable…

The previous post had some hacky R code to grab seekrit JSON data in ObservableHQ (OHQ) Notebooks and spit out a directory with a Quarto qmd and any associated FileAttachment s. Holding firm to my "no more generic public R packages"…

Quarto is amazing! And, it's eating the world! OK. Perhaps not the entire world. But it's still amazing…

Apple is in the final stages of shuttering the DarkSky service/API. They've replaced it with WeatherKit , which has both an xOS framework version as well as a REST API. To use either, you need to be a member of the Apple Developer Program…

My { darksky } package has been around for years, now, and the service that powers it was purchased by Apple before the pandemic. The DarkSky API is slated to be shuttered in December of this year and is being replaced by Apple's…

I realize you have to be living under a rock in the U.S. to not know that yesterday, was Juneteenth (a portmanteau of "June Nineteenth"). Still, I feel compelled to explain that said date marks the day when federal troops arrived in…

In today's newsletter Leonardo , an open source project and free online too from Adobe that lets you make great and accessible color palettes for use in UX/UI design and data visualizations! You can read the one newsletter section to get a…

Friday, May 13, 2022 was my last day at my, now, former employer of nearly seven years. I'm not mentioning the company name 1 because this post is not about them 2…

In my M-F newsletter today I mentioned an awesome Rust-based HTML/JS/CSS minifier library that also include batteries for a few other…

Just a quick post to note that I'll still be doing long-form, single-topic posts here on the blog, but I've also started an M-F daily Substack newsletter [free], that introduces 1-3(ish) topics per-issue on stuff I find during my daily RSS…

I came across a neat site that uses a Golang wasm function called from javascript on the page to help you see if your GitHub public SSH keys are "safe". What does "safe" mean? This is what the function checks for (via that…

The morning before work was super productive and there is a nigh-complete DSL for ESC/POS commands along with the ability to just print {ggplot2}/{grid}…

At the end of March, I caught a fleeting tweet that showcased an Epson thermal receipt printer generating a new "ticket" whenever a new GitHub issue was filed on a repository. @aschmelyun documents it well in this blog post . It's a pretty…

RStudio's macOS Electron build is coming along quite nicely and is blazing fast on Apple…

Open this URL in Tor to access BBC (??):…

Here is the full transcript of Ukrainian President Volodymyr Zelenskyy's video address early on Feb. 24 local time warning that a Russian invasion could cause tens of thousands of deaths…

If you've been following my inane tweets and non-technical blog posts for any length of time since 2015, you likely know the 2016 election cycle broke me more than just a tad, with each subsequent month of the Trump presidency adding a bit…

After a Twitter convo about weather stations I picked up a WeatherFlow Tempest . Setup was quick, but the sensor package died within 24 hours. I was going to give up on it but I had written an R package (for the REST API & UDP broadcast…

The New York Times had a [tragic] story on Covid deaths today and one of their plots really stuck with me for how well it told that part of the…

The Moderna booster level drained me all day on Dec 1 and did what jab two did during the overnight period (achy enough to wake me up and not get back to slumber easily). To try to wear myself down, I decided to practice a bit of R with…

I've been wanting to create a custom MaxMind mmdb file for alternate IPv4 range classifications for a while, and finally had an opportunity to do so for the $DAYJOB. MaxMind mmdb files are small and easy to work with and provide lightning…

The 30-Day Map Challenge is on again, and I'm hoping to be able to scrounge some time to get an entry for each day. Day 2 is lines (Day 1 was posted on Twitter only) and — while I'm hoping to focus on saving U.S. democracy for the majority…

RStudio is continuing to refine their ?? new product version string scheme, with the most recent change using something like the following for daily…

Vox grabbed some data from the Kaiser Family Foundation and did a story a few days ago on it, then posted a different visualization of it that attracted some…

Version 2 of RSwitch — the macOS menubar utility that enables fast and seamless switching between R versions on macOS and also provides notifications for new versions of RStudio Dailies has had a hasty update to deal with an error…

Hot on the heels of the previous CyberDefenders Challenge Solution comes this noisy installment which solves their Acoustic…

It was a rainy weekend in southern Maine and I really didn't feel like doing chores, so I was skimming through RSS feeds and noticed a link to a PacketMaze challenge in the latest This Week In 4n6…

WWDC 2021 is on this week and many new fun things are being introduced, including some data science-friendly additions to the frameworks that come with Xcode 13 and available on macOS 12+ (and its OS…

My archinfo utility is a small macOS command line application that displays process id, name, and architecture (arm64 or…

My {cdcfluview} package started tossing erros on CRAN just over a week ago when the CDC added an extra parameter to one of the hidden API endpoints that the package wraps. After a fairly hectic set of days since said NOTE came, I had time…

The fine folks over at @ObservableHQ released a new javascript exploratory visualization library called Plot last week with great fanfare. It was primarily designed to be used in Observable notebooks and I quickly tested it out there (you…

On or about Friday evening (May 7, 2021) Edge notified me that the Feedly Mini extension (one of the only extensions I use as extensions are dangerous things) was remove from the store due to…

Rather than continue to generate daily images with R, I threw together an Observable notebook that takes advantage of the CDC COVID-19 county data datasette (provided by Simon Willison) and the new {Plot} library (by the @ObservableHQ…

There's a semi-infrequent-but-frequent-enough-to-be-annoying manual task at $DAYJOB that involves extracting a particular set of strings (identifiable by a fairly benign set of regular expressions) from various interactive text sources…

There are a plethora of amazingly useful Golang libraries, and it has been possible for quite some time to use Go libraries with Swift. The advent of the release of the new Apple Silicon/M1/arm64 architecture for macOS created the need for…

The last post showed how to work with the macOS mdls command line XML output, but with {swiftr} we can avoid the command line round trip by bridging the low-level Spotlight API (which mdls uses) directly in R via…

(reminder: Quick Hits have minimal explanatory blathering, but I can elaborate on anything if folks submit a…

Guillaume Pressiat (@GuillaumePressiat) did a solid post & video on using Selenium to scrape a paginated table from understat[.]com/league/EPL/2020 (I just cannot bring myself to provide an active link to any SportsBall site). He does a…

They also have a generous (10K calls/day), free community API which does not require credentialed access and returns a subset of information that the full API does. This is handy for folks who can't afford the service or who only need to…

Apple M1/Apple Silicon/arm64 macOS can run x8664 programs via Rosetta and most M1 systems currently (~March 2021) very likely run a mix of x8664 and arm64…

along with a broad ecosystem of tools which can be used independently of the…

Horrible puns aside, hopefully everyone saw the news, earlier this week, from @thomasp85 on the evolution of modern typographic capabilities in the R ecosystem. Thomas (and some cohorts) has been working on {systemfonts}, {ragg}, and…

readcsv( file = "~/Data/applehealthexport/electrocardiograms/ecg2020-09-24.csv", # this is extracted below skip = 12, colnames = "µV" ) %>% mutate( idx = 1:n() ) ->…

Presented without much commentary since I stopped once {ggrepel} and {graphlayouts} failed (RStudio doesn't support it yet, either, which I…

If you've been following me around the internets for a while you've likely heard me pontificate about the need to be aware of and reduce — when possible — your personal "cyber" attack surface. One of the ways you can do that is to install…

I was chatting with a fellow Amazon Athena user and the topic of using Presto functions such as approxdistinct() via {d[b]plyr} came up and it seems it might not be fully common knowledge that any non-already translated function is passed…

(this is an unrolled Twitter thread converted to the blog since one never knows how long content will be preserved anywhere…

The past two posts have (lightly) introduced how to use compiled Swift code in R, but they've involved a bunch of "scary" command line machinations and…

The previous post introduced the topic of how to compile Swift code for use in R using a useless, toy example. This one goes a bit further and makes a case for why one might want to do this by showing how to use one of Apple's machine…

I've been on a Swift + R bender for a while now, but have been envious of the pure macOS/iOS (et al) folks who get to use Apple's seriously ++good machine learning libraries, which are even more robust on the new M1 hardware (it's cool…

There was an org that didn't see The data exfil hacking spree. A patch went up, our guard was down, Oh blow, SolarWinds,…

Last week I introduced a new bookdown series on how to embed R into a macOS Swift…

Every decent human (which excludes 74,222,958 ?? who voted for this, now 100% undeniable, traitor) with knowledge of this past week's tragic events is likely still processing — and will be for a while — what happened; I am no exception.…

I went completely daft this week and broke my months-long Twitter break due to the domestic terror event in my nation's capitol. I'll likely be resuming the break starting…

of some almost-natural "R" looking code (this is a…

While the future of the Apache Drill ecosystem is somewhat in-play (MapR — a major sponsoring org for the project — is kinda dead ), I still use it almost daily (on my local home office cluster) to avoid handing over any more money to…

It's been a while since I've posted anything R-related and, while this one will be brief, it may be of use to some R folks who have taken the leap into Big Sur and/or Apple Silicon. Stay to the end for an early Christmas…

The last 4+ years — especially the last ~10 months — had taken a bigger personal toll than I realized. I spent much of President-Elect Joseph R. Biden Jr.'s and Vice President-elect Kamala Harris' first speeches as duly & honestly selected…

The CDC continues to "deliver" in 2020, this time by changing the JSON response of one of the hidden APIs that my {cdcfluview} package wraps. CDC: So…

(This is part 2 of n "quick hit" posts, each walking through some approaches to speeding up components of an iterative operation. Go here for part…

(This is part 1 of n posts using this same data; n will likely be 2-3, and the posts are more around optimization than anything…

The Marshall Project has a solid story and set of visualizations on the impact of COVID-19 in U.S. prisons. They keep the data (and vis) regularly updated. They do great work and this is an important topic, but this visualization breaks my…

The incredibly talented folks over at Bishop Fox were quite generous this week, providing a scanner for figuring out PAN-OS GlobalProtect versions. I've been using their decoding technique and date-based fingerprint table to keep an eye on…

I've changed my years-long avatar to the blue Cap' shield because the colors of our flag have no business being displayed in any venue until Donald Trump is no longer President (one way or another). The red/white/blue triad has been…

Tis been a long time coming, but a minor change to default S3 parameters in tibbles finally caused a push of {sergeant} — the R package that lets you use the Apache Drill REST API via {DBI}, {dplyr}, or directly — to…

I (and, apparently, Gandalf Oo) are pleased to announce that RSwitch version 1.7.0 has been released . ( Direct Download…

R 4.0.0 has been out for a while, now, and — apart from a case where merge() was slower than dirt — it's been really stable for at least me (I use it daily on macOS, Linux, and Windows). Sure, it came with some headline-grabbing…

I caught this tweet by Terence Eden about using Twitter image alt-text to "PGP sign" tweet and my mind immediately went to "how can I abuse this for covert communications, malicious command-and-control, and embedding R code in…

Cut up shortening into small squares and mix into the dry bits with a pastry…

The United States Centers for Disease Control (CDC from now on) has setup two new public surveillance resources for COVID-19. Together, COVIDView and COVID-NET provide similar weekly surveillance data as FluView does for influenza-like…

Add back sausage and add in everything else and simmer for 45…

Just a quick note that thanks to a gentle nudge an updated version of {uaparser} --- a package that processes User Agent strings web clients send to servers --- is making its way to all the CRAN mirrors and is also available on CINC . The…

Waffle House announced it was closing hundreds of stores this week due to SARS-Cov-2 (a.k.a COVID-19). This move garnered quite a bit of media attention since former FEMA Administrator Craig Fugate used the restaurant chain as both an…

Sift together dry ingredients and mix until moist. Don't…

Über Tuesday has come and almost gone (some state results will take a while to coalesce) and I'm relieved to say that {catchpole} did indeed work, with the example code from before producing this on first…

For folks who are smart enough not to go near Twitter, I've been on a hiatus from the platform insofar as reading the Twitter feed goes. "Why" isn't the subject of this post so I won't go into it, but I've broken this half-NYE resolution…

It seems that the need for MX, DKIM, SPF, and DMARC records for modern email setups were just not enough acronyms (and setup tasks) for some folks, resulting in the creation of yet-another-acronym --- BIMI, or, Brand Indicators for Message…

As the maintainer of RSwitch --- and developer of my own (for personal use) macOS, iOS, watchOS, iPadOS and tvOS apps --- I need the full Apple Xcode install around (more R-focused macOS folk can get away with just the command-line tools…

macOS R users who tend to work on the bleeding edge likely noticed some downtime at <mac.r-project.org> this past weekend. Part of the issue was an SSL/TLS certificate expiration situation. Moving forward, we can monitor this with R using…

Each year the World Economic Forum releases their Global Risk Report around the time of the annual Davos conference. This year's report is out and below are notes on the "cyber" content to help others speed-read through those sections (in…

I saw this CNBC article on an in-theory browser client-side-only conversion utility for taking Apple Card PDF statements and turning them into CSV…

Before we start wrapping foreign language code we need to make sure that basic R packages can be created. If you've followed along from the previous post you have everything you need to get started here. Just to make sure , you should be…

Offspring #4 eats pretty healthy, normally, but likes to indulge in sugary confections on occasion. He was a big help clearing away the massive amount of snow that accumulated in the last winter storm of 2019 so I decided to start the new…

The R language and RStudio IDE are a powerful combination for "getting stuff done", and one aspect of R itself that makes it especially useful is the ability to use it with other programming languages via a robust foreign language…

I woke up to Axios' "1 Big Thing" ridgeline chart showing the crazy that was the 2019 news…

The right jolly old elves over at Alteryx created a "Santalytics" challenge back in 2016 to see if their community members could help Santa deliver presents to kids all across the…

I had need to present a wall-of-text to show off a giant list of SSL certificate alternate names and needed the entire list to fit on one slide (not really for reading in full, but to show just how many there were in a way that a simple…

Apple has brought Screen Time to macOS for some time now and that means it has to store this data somewhere . Thankfully, Sarah Edwards has foraged through the macOS filesystem for us and explained where these bits of knowledge are in her…

I posted a visualization of email safety status (a.k.a. DMARC ) of the Fortune 500 (2017 list) the other day on Twitter and received this spiffy request from…

An RSwitch user, lcolladotor filed a most-welcome issue letting me know that the core functionality of the switcher was busted ?. After testing out the 1.5.1 release candidate I had made a "harmless" & "clever" change to reduce some…

This past week @propublica linked to a really spiffy resource for getting an overview of a Twitter user's profile and activity called accountanalysis . It has a beautiful interface that works as well on mobile as it does in a real browser.…

All books, links, and other reference resources are under a single submenu…

The latest round of the 2020 Democratic debates is over and the data from all the 2019 editions of the debates have been added to {ggchicklet} . The structure of the debates2019 built-in dataset has changed a…

A minor update to RSwitch has been released. Apart from some internal code reorganization there are three user-facing…

Swift 5 has been so much fun to hack on that there's a new update to macOS R-focused mebubar utility RSwitch available. Along with the app comes a new dedicated RSwitch landing page and a new user's guide since it has enough features to…

It's only been a couple days since the initial version of my revamped take on RSwitch but there have been numerous improvements since then worth…

I caught this post on the The Surprising Number Of Programmers Who Can’t Program from the Hacker News RSS feed. Said post links to another, classic post on the same subject and you should read both before…

At the bottom of the R for macOS Developer's Page there's mention of an "other binary" called "RSwitch" that is "a small GUI that allows you to switch between R versions quickly (if you have multiple versions of R framework installed)."…

It's been yet-another weirdly busy summer but I'm finally catching up on noting some recent-ish developments in the…

Despite being a full-on denizen of all things digital I receive a fair number of dead-tree print magazines as there's nothing quite like seeing an amazing, large, full-color print data-driven visualization up close and personal. I also…

The {waffle} package got some ? this week and now has a substantially improved geomwaffle() along with a brand new sibling function geompictogram() which has all the powerful new features of geomwaffle() but lets you use Font Awesome 5…

The first U.S. Democratic debates of the 2020 election season were held over two nights this past week due to the daft number of candidates running for POTUS. The spiffy @NYTgraphics folks took the tallies of time spent blathering by each…

The r-project.org domain had some temporary technical difficulties this week (2019-29) that made reaching R-related resources problematic for a bunch of folks for a period of time. Incidents like this underscore the need for regional and…

Despite being on holiday I'm getting in a bit of non-work R coding since the fam has a greater ability to sleep late than I do. Apart from other things I've been working on a PR into {lutz}, a package by @andyteucher that turns lat/lng…

I've become a big fan of the {logspline} package over the past ~6 months and decided to wrap up a manual ggplot2 plotting process (well, it was at least in an RStudio snippet) into a small {ggplot2} Stat to make it easier to visualize…

I mentioned EtherCalc in a previous post and managed to scrounge some time to put together a fledgling {ethercalc} package (it's also on GitLab, SourceHut, Bitbucket and GitUgh, just sub out the appropriate URL…

This morning, @kairyssdal tweeted out the following graphic from…

I did another twitter thread on the aforeblogged MowerPlus database as I explored the tables after a second mow to determine what identified a unique mowing "session" (using John Deere's…

I was pretty brutal to Apple earlier this week in a Twitter thread that I tried to craft so it occurred in-line with the WWDC live stream (which might be something you want to remember as/if you read on). I really don't care about…

A user of the {ggalt} package recently posted a question about how to add points to a geomdumbbell() plot. For now, this is not something you can do with geomdumbbell() but with a bit of data wrangling you can do this in a pretty…

In a recent previous post I brazenly talked over the "hard parts" of how I got to the target SQLite file that houses "mowing history" for what has become my weekend obsession. So, we'll cover just how to do that (find things in iOS…

For the first time ever we got a new riding mower this weekend. We've always haggled to keep the one sellers were using with any given house we've purchased over the years (that was big enough for a yard that "requires" a riding…

The Feedly category I have setup for git-stalking has indicated a fairly massive interest in Joshua Levy's The Art of the Command Line . What is "The Art of the Command Line"? To quote the…

A fair bit of time ago the {gdns} package made its way to CRAN to give R users the ability to use Google's (at that time) nascent support for DNS over HTTPS (DoH). A bit later on Cloudflare also provided a global DoH endpoint and that…

I've talked about the retailpocalypse before and this morning I was greeted with the news about Dressbarn closing all 650 stores as I fired up a…

Harry's thread and Observable post are great on their own and both show the power and utility of Observable javascript…

I'm using GitUgh links here b/c the issue was submitted there. Those not wishing to be surveilled by Microsoft can find the macOS QuickLook plugin project and {rdatainfo} project in SourceHut and GitLab ( ~hrbrmstr and hrbrmstr accounts…

I've mentioned {htmlunit} in passing before, but did not put any code in the blog post. Since I just updated {htmlunitjars} to the latest and greatest version, now might be a good time to do a quick demo of…

Say your organization has done something pretty terrible . Terrible enough that you really didn't want to acknowledge it initially but eventually blogged about it, and haven't added a blog post in a long time so that entry is at the top of…

Like more posts than I care to admit, this one starts innocently enough with a tweet by…

Researching "the internet" (i.e. $DAYJOB) means having to deal with a ton of "unique" (I'm being kind) data formats. This is ultimately a tale of how I performed full-text searches across one of…

@teddunning recently updated the t-Digest algorithm he created back in 2013. What is this "t-digest"? Fundamentally, it is a probabilistic data structure for estimating any percentile of distributed/streaming data. Ted explains it quite…

I saw a second post on turning htmlwidgets into interactive Twitter Player cards and felt somewhat compelled to make creating said entities a bit easier so posited the…

There's been alot of talk about "dependencies" in the R universe of late. This is not really a post about that but more of a "really, don't do this" if you decide you want to poke the dependency bear by trying to build a deeply flawed…

The fine folks over at @PacketTotal bequeathed an API token on me so I cranked out an R package for it to enable more dynamic investigations work (RStudio makes for an amazing incident responder investigations console given that you can…

In the previous post I tried to explain what Content Security Policies (CSPs) are and how to work with them in R. In case you didn't RTFPost the TLDR is that CSPs give you control over what can be loaded along with your web content and can…

In the event you wanted to dig deeper I threw together a small package that will let you grab HTTP headers from a given URL and take a look at them. The README has examples for most things but we'll go through a bit of them here as…

Not flagging this with an "R" tag since I don't want to spam R-bloggers but I mentioned here that I'd be disabling logging on https://cinc.rud.is and https://git.rud.is and I wanted to follow up on that with an addendum that I've opted to…

In the "Changes on CRAN" section of the latest version of the The R Journal (Vol. 10/2, December 2018) had this short blurb entitled "CRAN mirror…

The in-dev htmlunit package for javascript-"enabled" web-scraping without the need for Selenium, Splash or headless Chrome relies on the HtmlUnit library and said library just released version 2.34.0 with a wide array of changes that…

I've got a work-in-progress drat -ified CRAN-like repo for (eventually) all my packages over at CINC ? ("CINC is not CRAN" and it also sounds like "sync"). This is in parallel with a co-location/migration of all my packages to SourceHut…

There are two fledgling rJava-based R packages that enable working with the AWS SDK for…

Those outside the Colonies may not know that Payless—a national chain that made footwear affordable for millions of 'Muricans who can't spare $100.00 USD for a pair of shoes their 7 year old will outgrow in a year— is closing . CNBC also…

read the code (to ensure R package developers are pwning or tracking you),and …after putting your "Encoded for use" token in the WIGLEAPIKEY environment variable…start…

I must preface this post with the posit that if you're doing anything interactive() with Amazon Athena you should seriously consider just using their free ODBC drivers as it's the easiest way to wire them up to R DBI- and tidyverse-wise.…

I've got a post up over at $DAYJOB's blog on using the ropendata ? package to access the ginormous and ever-increasing amount of internet telemetry (scan) data via the Rapid7 Open Data API…

Over at $DAYJOB's blog I've queued up a post that shows how to use our new ropendata ? package to work with our Open Data portal's API. I'm not super-sure when it's going to be posted so keep an RSS reader fixed on https://blog.rapid7.com/…

We worked pretty hard over at $DAYJOB on helping to quantify and remediate a fairly significant configuration weakness in Ubiquiti network work gear attached to the internet…

The urlscan ? package (an interface to the urlscan.io API) is now at version 0.2.0 and supports urlscan.io's authentication requirement when submitting a link for analysis. The service is handy if you want to learn about the details — all…

A major new release of Homebrew has landed and now includes support for Linux as well as Windows! via the Windows Subsystem for Linux. There are overall stability and speed improvements baked in as well. The aforelinked notification has…

We can do all of that in R with the help of a couple packages and by using a free geocoding service which will also allow us to put more data on the map, albeit with some extra work due to it returning weird values for Hawaii…

The seymour ? Feedly API package has been updated to support subscribing to RSS/Atom feeds. Previously the package was intended to just treat your Feedly as a data source, but there was a compelling use case for enabling subscription…

Over at the $WORK blog we posted some research into the fairly horrible Cisco RV320/RV325 router vulnerability. The work blog is the work blog and this blog is my blog (i.e. opinions are my own, yada, yada, yada) and I felt compelled to…

As someone who measures all kinds of things on the internet as part of his $DAYJOB, I can say with some authority that huge swaths of organizations are using cloud-services such as Google Apps, Dropbox and Office 365 as part of their…

Version 0.6.0 of the hrbrthemes ? package should be hitting a CRAN mirror near you soon. Apart from some general documentation and code cleanup this release includes the dark theme folks have been seeing in blog posts and tweets over the…

The splashr package [ srht | GL | GH ] — an alternative to Selenium for javascript-enabled/browser-emulated web scraping — is now at version 0.6.0 (still in dev-mode but on its way to CRAN in the next 14…

By now, even remote villages on uncharted islands in the Pacific know that the U.S. is in the midst of a protracted partial government shutdown. It's having real impacts on the lives of Federal government workers but they aren't the only…

More than just sergeant has been hacked on recently, so here's a run-down of various ?…

It was probably not difficult to discern from my previous Drill-themed post that I'm fairly excited about the Apache Drill 1.15.0 release. I've rounded out most of the existing corners for it in preparation for a long-overdue CRAN update…

Today's RSS feeds picked up this article by Marianne Sullivan, Chris Sellers, Leif Fredrickson, and Sarah Lamdanon on the woeful state of enforcement actions by the U.S. Environmental Protection Agency (EPA). While there has definitely…

To ring in the new year the Drill team knocked out a new 1.15.0 release with a cadre of new functionality…

Well, 2018 has flown by and today seems like an appropriate time to take a look at the landscape of R bloggerdom as seen through the eyes of readers of R-bloggers and R Weekly . We'll do this via a new package designed to make it easier to…

Phishing is [still] the primary way attackers either commit a primary criminal act (i.e. phish a target to, say, install ransomware) or is the initial vehicle used to gain a foothold in an organization so they can perform other criminal…

I can't seem to free my infrequently-viewed email inbox from "you might like!" notices by the content-lock-in site Medium. This one made it to the iOS notification screen (otherwise I'd've been blissfully unaware of it and would have saved…

The libcurl library (the foundational library behind the RCurl and curl packages) has switched to using OpenSSL's default ciphers since version 7.56.0 (October 4 2017). If you're a regular updater of curl / httr you should be fairly…

Despite their now inherent evil status, GitHub has some tools other repository aggregators do not. One such tool is the free vulnerability alert service which will scan repositories for outdated+vulnerable…

If you're an R/RStudio user who has migrated to Mojave (macOS 10.14) or are contemplating migrating to it, you will likely eventually run into an issue where you're trying to access resources that are in Apple's new hardened filesystem…

The CBC covered the recent (as of the original post-time on this blog entry) Quebec elections and used a well-crafted hex grid map to display…

In my semi-daily run of brew update I noticed that proj4 had been updated to 5.2. I kinda " squeee "'d since (as the release notes show ) the Equal Earth projection was added to it ( +proj=eqearth…

Despite having sailed through the core components of this year's Talk Like A Pirate Day R post a few months ago, time has been an enemy of late so this will be a short post that others can build off of, especially since there's lots more…

The wayback ? package has had an update to more efficiently retrieve mementos and added support for working with the Internet Archive's advanced search+scrape API…

I was chatting with some cyber-mates at a recent event and the topic of cyber attacks on the U.S. power-grid came up (as it often does these days). The conversation was brief, but the topic made its way into active memory and resurfaced…

The sergeant ? package has a minor update that adds REST API coverage for two "new" storage endpoints that make it possible to add, update and remove storage configurations on-the-fly without using the GUI or manually updating a config…

Nowadays (I've seen that word used so much in journal articles lately that I could not resist using it) I'm using world tile grids more frequently as the need arises to convey the state of exposure of various services at a global (country)…

Apple has run the death bell on 32-bit macOS apps and, if you're running a recent macOS version on your Mac (which you should so you can get security updates) you likely see this alert from…

It's been over a year since Headless Chrome was introduced and it has matured greatly over that time and has acquired a pretty large user base. The TLDR on it is that you can now use Chrome as you would any command-line interface (CLI)…

If you've got a directory full of Bro NSM logs, it's easy to work with them in Apache Drill since they're just tab-separated values (TSV) files by default. The most tedious part is mapping the columns to proper types and hopefully this…

Apache Drill 1.14.0 was recently released , bringing with it many new features and a temporary incompatibility with the current rev of the MapR ODBC drivers. The Drill community expects new ODBC drivers to arrive shortly. The sergeant ? is…

The development version of splashr now support authenticated connections to Splash API instances. Just specify user and pass on the initial splashr::splash() call to use your scraping setup a bit more safely. For those not familiar with…

A previous post showed how to use a different authentication provider to wire up Apache Zeppelin and Amazon Athena. As noted in that post, Zeppelin is a "notebook" alternative to Jupyter (and other) notebooks. Unlike Jupyter, I can…

One really cool feature of Zeppelin is the ability for it to wire it up to databases via JDBC and use it interactive queries. A future post will provide instructions for Apache Drill, but this one's about wiring up Amazon Athena and Apache…

I had to processes a bunch of emails for a $DAYJOB task this week and my "default setting" is to use R for pretty much everything (this should come as no surprise). Treating mail as data is not an uncommon task and many R packages exist…

After reading this interesting analysis of "How Often Are Americans' Accounts Breached?" by Gaurav Sood (which we need more of in cyber-land) I gave in to the impulse to do some gg-doodling with the "Have I Been Pwnd" JSON data he…

Continuing the blog's UDF theme of late, there are two new UDF kids in…

There are many ways to gather Twitter data for analysis and many R and Python (et al) libraries make full use of the Twitter API when building a corpus to extract useful metadata for each tweet along with the text of each tweet. However,…

A previous post explored how to deal with Amazon Athena queries asynchronously. The function presented is a beast , though it is on purpose (to provide options for…

I've blogged about how to use Amazon Athena with R before and if you are a regular Athena user, you've likely run into a situation where you prepare a dplyr chain, fire off a collect() and then…

Both my osqueryr and macthekinfe packages have had a few updates and I wanted to put together a fun example (it being Friday, and all) for what you can do with them. All my packages are now on GitHub and GitLab and I'll be maintaining them…

I've mentioned @stiles before on the blog but for those new to my blatherings, Matt is a top-notch data journalist with the @latimes and currently stationed in South Korea. I can only imagine how much busier his life has gotten since that…

You can catch a bit of the @rOpenSci 2018 Unconference experience at home w with this short-ish 'splainer video on how to use the new middlechild package (https://github.com/ropenscilabs/middlechild) & mitmproxy to automagically create…

The forthcoming RStudio 1.2 release has a new "Jobs" feature for running and managing background R…

A colleague asked if I would blog about how I crafted the grid of world tile grids in this post and I accepted the challenge. The technique isn't too hard as it just builds on the initial work by Jon Schwabish and a handy file made by…

After seeing Fira Sans in action in presentations at eRum 2018 I felt compelled to add hrbrthemes support for it so I made a firasans ? extension to it that uses Fira Sans Condensed and Fira Code fonts for ggplot2…

Most modern operating systems keep secrets from you in many ways. One of these ways is by associating extended file attributes with files. These attributes can serve useful purposes. For instance, macOS uses them to identify when files…

Regular readers will recall the " utility belt " post from back in April of this year. This is a follow-up to a request made asking for a list of all the % infix functions in those…

I'm going to (eventually) write a full post on the package I'm mentioning in this one : osqueryr ? . The TLDR on osqueryr is that it is an R DBI wrapper (that has just enough glue to also be plugged into dbplyr ) for osquery ? . The TLDR…

I pen this mini-tome on "GDPR Enforcement Day". The spirit of GDPR is great, but it's just going to be another Potempkin Village in most organizations much like PCI or SOX . For now, the only thing GDPR has done is made GDPR consulting…

The cloc Perl script (yes, Perl !) by Al Danial ( https://github.com/AlDanial/cloc ) has been one of the go-to tools for generating code metrics. Given a single file, directory tree, archive, or git repo, cloc can speedily give you metrics…

The D.C. Universe magic hero Zatanna used spells (i.e. incantations) to battle foes and said spells were just sentences said backwards, hence the mixed up jumble in the title. But, now I'm regretting not naming the package zatanna and…

Do not read anything more into the order than the end-number of the "Main URL" since this was auto-generated from a script that processed my Firefox tab…

The U.S. FBI Internet Crime Complaint Center was established in 2000 to receive complaints of Internet crime. They produce an annual report, just released 2017's edition , and I need the data from it. Since I have to wrangle it out, I…

The answer was a 1:1 "file upload" curl to httr…

I spent some time this morning upgrading the JDBC driver (and changing up some supporting code to account for changes to it) for my metis package? which connects R up to Amazon Athena via RJDBC. I'm used to JDBC and have to deal with Java…

This week's edition of Data is Plural had two really fun data sets. One is serious fun (the first comprehensive data set on U.S. evictions , and the other I knew about but had forgotten: The Federal Register Executive Order (EO) data…

If you come here often you've noticed that I've been writing a semi-frequent series on using the Feedly API with…

@mkjcktzn asked if one can access Feedly "Saved for Later" items via the API. The answer is "Yes!", and it builds off of that previous post. You'll need to read it and get your authentication key (still no package ?) before…

Since I just railed against Congress for being a bit two-faced about privacy I thought some rud.is site disclosure would be in…

Said bad words include "Facebook", "Mark Zuckerberg" and many referrals to entities within the U.S. Government. Given the topic, it cannot be…

Here's one way to do that which doesn't rely on pandoc ( pandoc can easily do this and ships with RStudio but shelling out for this is cheating…

Many R package authors (including myself) lump a collection of small, useful functions into some type of utils.R file and usually do not export the functions since they are (generally) designed to work on package internals rather than…

Modern MacBook Pros have a fairly useless ( c'mon, admit it! ) "Touch Bar" that did little more than cause severe ire in the developer community after turning a full-fledged, tactile Escape key into a hollow version if its former…

There's a yuge chance you're reading this post (at least initially) on R-Bloggers right now (though you should also check out R Weekly and add their live feed to your RSS reader pronto!). It's a central "watering hole" for R folks and is…

The 2018 IEEE Security & Privacy Conference is in May but they've posted their full proceedings and it's better to grab them early than to wait for it to become part of a paid journal…

A while back, Medium blogger 'Nykolas Z' posted results from a globally distributed DNS resolver test to find the speediest provider (NOTE: speed is not the only consideration when choosing an alternative DNS provider). While the test…

You have to have been living under a rock to not know about Cloudflare's new 1.1.1.1 DNS offering. I won't go into "privacy", "security" or "speed" concepts in this post since that's a pretty huge topic to distill for folks given the, now,…

The fs package makes it super quick and easy to find out just how much "package hoarding" you've been…

@yoniceedee suggested I look at the Cambridge Analytics "whistleblower" testimony…

A quick Friday post to let folks know about three in-development R packages that you're encouraged to poke the tyres o[fn] and also jump in and file issues or PRs…

(FWIW I think I even caused myself pain due to the title of this blog…

I work with internet-scale data and do my fair share of macro-analyses on vulnerabilities. I use the R semver package for most of my work and wanted to blather on a bit about it since it's super-helpful for this work and doesn't get the…

Tis the season for finding out how well Maine fisherfolk did last year; specifically, Maine…

The NPR Visuals Team created and maintains a javascript library that makes it super easy to embed iframes on web pages and have said documents still be…

Much of what I need to do for work-work involves using tools that are (for the moment) not in R. Today, I needed to test the validity of (and other processing on) DMARC records and I'm loathe to either reinvent the wheel or reticulate bits…

Time for another look at what's new and interesting in the #rstats world with the help of Peter Meissner's (@marvindpr) crossword.r ?…

He also made a super spiffy javascript library to pair with it, which can turn crossword model output into an interactive…

We're doing some interesting studies (cybersecurity-wise, not finance-wise) on digital currency networks at work-work and --- while I'm loathe to create a geo-map from IPv4 geolocation data ---…

NOTE: The likelihood of this recipe being added to the recent practice bookdown book is slim, but I'll try to keep the same format for the blog…

The new year begins with me being on the hook to crank out a book on advanced web-scraping in R by July (more on that in a future blog post). The bookdown ? package seemed to be the best way to go about doing this but I had only played…

that spawned from Maëlle's recent look-back post turned into a quick function for capturing an image of a Tweet/thread using webshot , rtweet , magick and glue…

2017 is nearly at an end. We humans seem to need these cycles to help us on our path forward and have, throughout history, used these annual demarcation points as a time of reflection of what was, what is an what shall come…

It's been a long time coming, but swatches ? is now on CRAN…

GitHub (2017-12-21 post-time) started adding obnoxious boxes to their activity feed. I use that to discover new projects/developers. While I also have it in RSS and that's nice and compact, I do browse the activity feed directly. Those…

At work-work I mostly deal with medium-to-large-ish data. I often want to poke at new or existing data sets w/o working across billions of rows. I also use Apache Drill for much of my exploratory…

I know some folks had a bit of fun with the previous post since it exposed the fact that I left out unique MQTT client id generation from the initial 0.1.0 release of the in-development package (client ids need to be unique…

Most of us see the internet through the lens of browsers and apps on our laptops, desktops, watches, TVs and mobile devices. These displays are showing us --- for the most part --- content designed for human consumption. Sure, apps handle…

More sites are turning to services like Cloudflare due to just how stupid-easy it is to DDoS --- perform a (possibly Distributed) Denial of Service attack on -- a site. Sometimes the DDoS is intentional (malicious). Sometimes it's because…

I’m in the market for Christmas presents for my true love, @mrshrbrmstr, and thought I’d look to an age-old shopping list for inspiration. Just what would it set me back if I decided to mimic the 12 Days of Christmas in this modern day and…

Over the past few weeks, I had been noticing that some posts in the R-bloggers feed were getting truncated in Feedly. I don't remember when I noticed that since I usually click through immediately from the headline entry to the R-bloggers…

Our family has been reading, listening to and watching "A Christmas Carol" for just abt 30 years now. I got it into my crazy noggin to perform a sentiment analysis on it the other day and tweeted out the results, but a large chunk of the R…

Earlier this year, I made a package that riffed off of ProPublica's really neat voting cartograms (maps) for the U.S. House and Senate. You can see one for disaster relief spending in the House and one for the ACA "Skinny Repeal" in the…

NOTE: This is mainly for those of us in the Colonies, but some tips apply…

Daughter #2 got married to a wonderful chap yesterday. I wanted to preserve the text of my speech and blessing for them here. I know it's on their wedding video but I'd like to archive it into the historical record as…

A long time ago, in a github repo far, far away there lived a tiny package that made it possible to create equal area, square U.S. state cartograms in R dubbed statebins ? . Three years have come and gone and --- truth be told --- I've…

IBM has a new set of corporate typefaces --- dubbed "Plex" --- and has released them with a generous open license. IBM Plex Sans is not too…

By now, virtually every major media outlet has covered the "280 Apocalypse"™. For those still not "in the know", Twitter recently moved the tweet character cap to 280 after a "successful" beta test (some of us have different ideas of what…

Working remotely has many benefits, but if you work remotely in an area like, say, rural Maine, one of those benefits is not massively speedy internet connections. Being able to go fast and furious on the internet is one of the many things…

Unlike @noamross, I am not an epidemiologist (NOTE: Noam battles pandemics before breakfast, so be super nice to him) but I do like to find kindred methodologies in other disciplines to help foster the growth of cybersecurity into…

Despite my week-long Twitter consumption sabbatical (helped --- in part --- by the nigh week-long internet and power outage here in Maine), I still catch useful snippets from folks. My cow-orker @dabdine shunted a tweet by @terrencehart…

This past weekend, violent windstorms raged through New England. We — along with over 500,000 other Mainers — went "dark" in the wee hours of Monday morning and (this post was published on Thursday AM) we still have no utility-provided…

As many folks know, I live in semi-rural Maine and we were hit pretty hard with a wind+rain storm Sunday to Monday. The hrbrmstr compound had no power (besides a generator) and no stable/high-bandwidth internet (Verizon LTE was heavily…

Way back in July of 2009, the first version of the twitteR package was published by Geoff Jentry in CRAN. Since then it has seen 28 updates, finally breaking the 0.x.y barrier into 1.x.y territory in March of 2013 and receiving it's last…

I've blogged a bit about robots.txt --- the rules file that documents a sites "robots exclusion" standard that instructs web crawlers what they can and cannot do (and how frequently they should do things when they are allowed to). This is…

The "data science" feed URL is https://rud.is/b/category/r/feed/…

The Internet Archive recently announced a new service they've dubbed 'Third Eye'. This service scrapes the chyrons that annoyingly scroll across the bottom-third of TV news broadcasts. IA has a vast historical archive of TV news that…

Modern websites are complex beasts. They house photo galleries, interactive visualizations, web fonts, analytics code and other diverse types of content. Despite the potential for diversity, many web sites share similar "tech stacks" ---…

I occasionally hang out on StackOverflow and often use an answer as an opportunity to fill a package void for a particular need. docxtractr and qrencoder are two (of many) packages that were birthed from SO answers. I usually try to answer…

spiderbar, spiderbar Reads robots rules from afar. Crawls the web, any size; Fetches with respect, never lies. Look Out! Here comes the…

International Code Talk Like A Pirate Day almost slipped by without me noticing (September has been a crazy busy month), but it popped up in the calendar notifications today and I was glad that I had prepped the meat of a post a few weeks…

I was socially engineered by @yoniceedee into creating today's post due to being prodded with this…

insert(post, "{ 'standarddisclaimer' : 'My opinion, not my employer\'s'…

I've blathered about trust before 1 2 , but said blatherings were in a "what if" context. Unfortunately, the if has turned into a when , which begged for further blathering on a recent FOSS ecosystem cybersecurity…

I've blogged about my in-development R package hgr a before and it's slowly getting to a CRAN release. There are two new features to it that are more useful in an interactive session than in a programmatic context. Since they build on each…

Putting this here to make it easier for others who try to Google this topic to find it w/o having to find and tediously search through other UDFs (user-defined…

Earlier this year, the GDELT Project released their Television Explorer that enabled API access to closed-caption tedt from television news broadcasts. They've done an incredible job expanding and stabilizing the API and just recently…

I recently posted about using a Python module to convert HTML to usable text. Since then, a new package has hit CRAN dubbed htm2txt that is 100% R and uses regular expressions to strip tags from…

(That image was generated with splashr::renderpng(url = "https://cran.r-project.org/web/packages/splashr/")…

I was about to embark on setting up a background task to sift through R package PDFs for traces of functions that "omit NA values" as a surprise present for Colin Fay and Sir…

There was a discussion on Twitter about the need to read in ".msg" files using R. The "MSG" file format is one of the many binary abominations created by Microsoft to lock folks and users into their platform and tools. Thankfully, they…

I needed to clean some web HTML content for a project and I usually use hgr::cleantext() for it and that generally works pretty well. The cleantext() function uses an XSLT stylesheet to try to remove all non-"main text content" from an…

I've blathered about my crawldelay project before and am just waiting for a rainy weekend to be able to crank out a follow-up post on it. Working on that project involved sifting through thousands of Web Archive (WARC) files. While I have…

(General reminder abt "R⁶" posts in that they are heavy on code-examples, minimal on expository. I try to design them with 2-3 "nuggets" embedded for those who take the time to walk through the code examples on their systems. I'll always…

The reticulate package provides a very clean & concise interface bridge between R and Python which makes it handy to work with modules that have yet to be ported to R (going native is always better when you can do it). This post shows how…

One of my tweets that referenced an excellent post about the ethics of web scraping garnered some…

It's no secret that I'm a fan of Apache Drill . One big strength of the platform is that it normalizes the access to diverse data sources down to ANSI SQL calls, which means that I can pull data from parquet, Hie, HBase, Kudu, CSV, JSON,…

Matt @stiles is a spiffy data journalist at the @latimes and he posted an interesting chart on U.S. Attorneys General longevity (given that the current US AG is on thin…

I'm extremely pleased to announce that the sergeant package is now on CRAN or will be hitting your local CRAN mirror…

Rasmussen makes their Presidential polling data available for both ? & O. Why not compare their ratings from day 1 in office (skipping days that Rasmussen doesn't…

The Apache Drill folks have a nice walk-through tutorial on how to analyze the Yelp Academic Dataset with Drill . It's a bit out of date (the current Yelp data set structure is different enough that the tutorial will error out at various…

I caught a mention of this project by Pete Warden on Four Short Links today. If his name sounds familiar, he's the creator of the DSTK , an O'Reilly author, and now works at Google. A decidedly clever and decent…

I caught the 0.7.0 release of dplyr on my home CRAN server early Friday morning and immediately set out to install it since I'm eager to finish up my sergeant package and get it on CRAN. "Tidyverse" upgrades aren't trivial for me as I…

I've been doing intermittent prep work for a follow-up to an earlier post on store closings and came across this CNN Money "article" on it. Said "article" is a deliberately obfuscated or lazily crafted series of GIF images that contain all…

I used reading a directory of CSVs as the foundational example in my recent post on idioms…

Last year, I was able to sit back and lazily "RT" Julia Silge's excellent retrospective on her 2016 @rOpenSci "unconference" experience. Since Julia was not there this year, and the unconference experience is still in primary storage (LMD…

NOTE: I'll do my best to ensure the next post will have nothing to do with Twitter, and this post might not completely meet my R⁶…

Many of my posts seem to begin with a link to a tweet, and this one falls into that…

Most of the examples of working with most of the AWS services show basic username & password authentication. That's all well-and-good, but many shops use the AWS Security Token Service to provide temporary credentials and session tokens to…

If you follow me on Twitter or monitor @Rapid7's Community Blog you know I've been involved a bit in the WannaCry ransomworm…

Tagging this as #rstats -related since many R coders use Travis-CI to automate package builds (and other things). Security researcher Ivan Vyshnevskyi did some ++gd responsible disclosure to the Travis-CI folks letting them know they were…

Political machinations are a tad insane in the U.S. these days & I regularly hit up @ProPublica & @GovTrack sites (& sub to the GovTrack e-mail updates) as I try to be an informed citizen, especially since I've got a Senator and…

Mara is — without a doubt — the best data science promoter in the Twitterverse. She seems to have her finger on the pulse of everything that's happening in the data science world and is one of the most ardent amplifiers there…

Once I realized that my planned, larger post would not come to fruition today I took the R⁶ post (i.e. "minimal expository, keen focus") route, prompted by a Twitter discussion with some R mates who needed to convert "lightly formatted"…

The intrepid @masalmon cranked out another blog post , remixing classical music schedule data from Radio Swiss Classic . It's a fun post and you should read it before continuing…