More “Scraping Ethics Gone Awry” and “Why Do This When There’s a Free API?”

I can’t seem to free my infrequently-viewed email inbox from “you might like!” notices by the content-lock-in site Medium. This one made it to the iOS notification screen (otherwise I’d’ve been blissfully unaware of it and would have saved you the trouble of reading this). Today, they sent me this gem by @JeromeDeveloper: Scrapy and… Continue reading

Use GitHub Vulnerability Alerts to Keep Users of Your R Packages Safe

Despite their now inherent evil status, GitHub has some tools other repository aggregators do not. One such tool is the free vulnerability alert service which will scan repositories for outdated+vulnerable dependencies. Now, “R” is nowhere near a first-class citizen in the internet writ large, including software development tooling (e.g. the Travis-CI and GitLab continuous integration… Continue reading

Access the Internet Archive Advanced Search/Scrape API with wayback (+ links to a new vignette & pkgdown site)

The wayback🔗 package has had an update to more efficiently retrieve mementos and added support for working with the Internet Archive’s advanced search+scrape API. Search/Scrape The search/scrape interface lets you examine the IA collections and download what you are after (programmatically). The main function is ia_scrape() but you can also paginate through results with the… Continue reading