Author: hrbrmstr
Everything written by hrbrmstr.
Teasing Out Top Daily Topics with GDELT's Television Explorer
Earlier this year, the GDELT Project released their Television Explorer that enabled API access to closed-caption tedt from television news broadcasts. They've done an incredible job expanding and stabilizing the API and just recently…
Readability Redux
I recently posted about using a Python module to convert HTML to usable text. Since then, a new package has hit CRAN dubbed htm2txt that is 100% R and uses regular expressions to strip tags from…
New CRAN Package Announcement: splashr
(That image was generated with splashr::renderpng(url = "https://cran.r-project.org/web/packages/splashr/")…
Rpad Domain Repurposed To Deliver Creepy (and potentially malicious) Content
I was about to embark on setting up a background task to sift through R package PDFs for traces of functions that "omit NA values" as a surprise present for Colin Fay and Sir…
Unbottling ".msg" Files in R
There was a discussion on Twitter about the need to read in ".msg" files using R. The "MSG" file format is one of the many binary abominations created by Microsoft to lock folks and users into their platform and tools. Thankfully, they…
Reticulating Readability
I needed to clean some web HTML content for a project and I usually use hgr::cleantext() for it and that generally works pretty well. The cleantext() function uses an XSLT stylesheet to try to remove all non-"main text content" from an…
Caching httr Requests? This means WAR[C]!
I've blathered about my crawldelay project before and am just waiting for a rainy weekend to be able to crank out a follow-up post on it. Working on that project involved sifting through thousands of Web Archive (WARC) files. While I have…
R⁶ — Exploring macOS Applications with codesign, Gatekeeper & R
(General reminder abt "R⁶" posts in that they are heavy on code-examples, minimal on expository. I try to design them with 2-3 "nuggets" embedded for those who take the time to walk through the code examples on their systems. I'll always…
R⁶ — Reticulating Parquet Files
The reticulate package provides a very clean & concise interface bridge between R and Python which makes it handy to work with modules that have yet to be ported to R (going native is always better when you can do it). This post shows how…
Analyzing "Crawl-Delay" Settings in Common Crawl robots.txt Data with R
One of my tweets that referenced an excellent post about the ethics of web scraping garnered some…