The development version of splashr now support authenticated connections to Splash API instances. Just specify user and pass on the initial splashr::splash() call to use your scraping setup a bit more safely. For those not familiar with splashr and/or Splash: the latter is a lightweight alternative to tools like Selenium and the former is an… Continue reading
Connecting Apache Zeppelin and Apache Drill, PostgreSQL, etc.
A previous post showed how to use a different authentication provider to wire up Apache Zeppelin and Amazon Athena. As noted in that post, Zeppelin is a “notebook” alternative to Jupyter (and other) notebooks. Unlike Jupyter, I can tolerate Zeppelin and it’s got some nifty features like plug-and-play JDBC access. Plus it can do some… Continue reading
Connecting Apache Zeppelin Up to Amazon Athena with an IAM Profile Name
Apache Zeppelin is a “notebook” alternative to Jupyter (and other) notebooks. It supports a plethora of kernels/interpreters and can do a ton of things that this post isn’t going to discuss (perhaps future ones will, especially since it’s the first “notebook” environment I’ve been able to tolerate for longer than a week). One really cool… Continue reading
Digging into mbox details: A tale of tm & reticulate
✨ I had to processes a bunch of emails for a $DAYJOB task this week and my “default setting” is to use R for pretty much everything (this should come as no surprise). Treating mail as data is not an uncommon task and many R packages exist that can reach out and grab mail from… Continue reading
ggplot “Doodling” with HIBP Breaches
After reading this interesting analysis of “How Often Are Americans’ Accounts Breached?” by Gaurav Sood (which we need more of in cyber-land) I gave in to the impulse to do some gg-doodling with the “Have I Been Pwnd” JSON data he used. It’s just some basic data manipulation with some heavy ggplot2 styling customization, so… Continue reading
Two new Apache Drill UDFs for Processing UR[IL]s and Internet Domain Names
Continuing the blog’s UDF theme of late, there are two new UDF kids in town: drill-url-tools🔗 for slicing & dicing URI/URLs (just going to use ‘URL’ from now on in the post) drill-domain-tools🔗 for slicing & dicing internet domain names (IDNs). Now, if you’re an Apache Drill fanatic, you’re likely thinking “Hey hrbrmstr: don’t you… Continue reading
New Apache Drill UDF for Processing Twitter Tweet Text
There are many ways to gather Twitter data for analysis and many R and Python (et al) libraries make full use of the Twitter API when building a corpus to extract useful metadata for each tweet along with the text of each tweet. However, many corpus archives are minimal and only retain a small portion… Continue reading
A new ‘boto3’ Amazon Athena client wrapper with dplyr async query support
A previous post explored how to deal with Amazon Athena queries asynchronously. The function presented is a beast, though it is on purpose (to provide options for folks). In reality, nobody really wants to use rJava wrappers much anymore and dealing with icky Python library calls directly just feels wrong, plus Python functions often return… Continue reading