Skip navigation

I occasionally hang out on StackOverflow and often use an answer as an opportunity to fill a package void for a particular need. docxtractr and qrencoder are two (of many) packages that were birthed from SO answers. I usually try to answer with inline code first then expand the functionality into a package (if warranted). Some make it to CRAN (like those two), others stay on GitHub.

This (short) post is about two new ones: webhose? and pigeon?.

The webhose package is an API interface package to, which is an interesting service that scrapes the web & “dark web” and provides a short but handy API to retrieving the content using a fairly intuitive query language.

The pigeon package is a hastily-hacked-together wrapper around pgn-extract, a cross-platform utility written in C for working with chess game data in PGN format.

I’m not going to have any time to round out the corners on either of those packages but will gladly make time to help anyone who wants to jump on board to either (or both!) of them.

Working on either package will let you get your feet wet (or, wetter) with R package development. They both need:

  • more functions + docs!
  • test harness setup
  • Travis-CI, code coverage & AppVeyor configs

Working on webhose will give you experience dealing with HTTP APIs (and their API is super clean to work with) and possibly introduce you to an area of research you’re not already in.

Working on pigeon will give you experience integrating C[++] & R code (and the C-library I’ve hack-wrapped definitely needs some care & feeding to ensure no memory leaks are present).

Neither is “mission critical”. The world will gladly go on w/o either of them. But, if you wanted a judgement-free place to hone your R package skills or try/learn new things (and ask questions along the way), file an issue, drop a note in the comments or hit me up on Twitter. If you’re an experienced R package-r and want to “own” either of these, that’s ? as well!

I’d encourage all nascent R coders to adopt “SODD” and use SO as a place to hone your skills while you help others (and you don’t need to write a package for every answer :-).


  1. Thank you for the interesting article and packages.
    SODD sounds interesting. Can you reference the SO posts that influence webhose and pigeon? I’d like to see how they connect to the work you did. Thank you again!

    • Aye. I was thinking of adding a badge to the GH repos with them as well. Stay tuned!

  2. Perfect! Good day!

  3. I’ve been adding functions to my stackoverflow package for the SO answers that are somewhere in between ‘this should be a full package’ and ‘only one person would ever care about this’… it was also a somewhat futile effort to keep coworkers from copy-pasting from SO into our corporate code base.

  4. So, now a good thing would be to develop a package to collect ideas automatically from Stack Overflow :)

3 Trackbacks/Pingbacks

  1. […] article was first published on R –, and kindly contributed to […]

  2. […] article was first published on R –, and kindly contributed to […]

  3. […] point-shill with a direct link but it’ll be easy to find if you are interested in this latest SODD […]

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.