Tag:post
All the articles with the tag "post".
Caching httr Requests? This means WAR[C]!
I've blathered about my crawldelay project before and am just waiting for a rainy weekend to be able to crank out a follow-up post on it. Working on that project involved sifting through thousands of Web Archive (WARC) files. While I have…
R⁶ — Exploring macOS Applications with codesign, Gatekeeper & R
(General reminder abt "R⁶" posts in that they are heavy on code-examples, minimal on expository. I try to design them with 2-3 "nuggets" embedded for those who take the time to walk through the code examples on their systems. I'll always…
R⁶ — Reticulating Parquet Files
The reticulate package provides a very clean & concise interface bridge between R and Python which makes it handy to work with modules that have yet to be ported to R (going native is always better when you can do it). This post shows how…
Analyzing "Crawl-Delay" Settings in Common Crawl robots.txt Data with R
One of my tweets that referenced an excellent post about the ethics of web scraping garnered some…
Reading PCAP Files with Apache Drill and the sergeant R Package
It's no secret that I'm a fan of Apache Drill . One big strength of the platform is that it normalizes the access to diverse data sources down to ANSI SQL calls, which means that I can pull data from parquet, Hie, HBase, Kudu, CSV, JSON,…
R⁶ — General (Attys) Distributions
Matt @stiles is a spiffy data journalist at the @latimes and he posted an interesting chart on U.S. Attorneys General longevity (given that the current US AG is on thin…
Ten-HUT! The Apache Drill R interface package — sergeant — is now on CRAN
I'm extremely pleased to announce that the sergeant package is now on CRAN or will be hitting your local CRAN mirror…
R⁶ — Disproving Approval
Rasmussen makes their Presidential polling data available for both ? & O. Why not compare their ratings from day 1 in office (skipping days that Rasmussen doesn't…
Replicating the Apache Drill 'Yelp' Academic Dataset Analysis with sergeant
The Apache Drill folks have a nice walk-through tutorial on how to analyze the Yelp Academic Dataset with Drill . It's a bit out of date (the current Yelp data set structure is different enough that the tutorial will error out at various…
Keeping Users Safe While Collecting Data
I caught a mention of this project by Pete Warden on Four Short Links today. If his name sounds familiar, he's the creator of the DSTK , an O'Reilly author, and now works at Google. A decidedly clever and decent…