<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/exploring-r-bloggers-posts-with-the-feedly-api/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rud.is/b/2018/04/04/exploring-r-bloggers-posts-with-the-feedly-api/"> R – rud.is</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>There’s a <em>yuge</em> chance you’re reading this post (at least initially) on R-Bloggers right now (though you should also check out <a rel="nofollow" href="https://rweekly.org/" target="_blank">R Weekly</a> and add their live feed to your RSS reader pronto!). It’s a central “watering hole” for R folks and is read by many (IIRC over 20,000 Feedly users have it in their OPML).</p>
<p>I’m <em>addicted</em> to <a rel="nofollow" href="https://feedly.com/" target="_blank">Feedly</a> and waited <em>years</em> for them to publish their API. <a rel="nofollow" href="https://developer.feedly.com/" target="_blank">They have</a> and there will eventually be a package for it (go for it if you want to get’er done before me since I won’t have time to do it justice for a while). As just parenthetically noted, I’ve started work on one and have scaffolded <em>just enough</em> to give R folks a present: almost 5 years of R-Bloggers data — posts, engagement rates, authors, etc). <em>But</em>, you’ll have to put up with some expository, first.</p>
<h3>Digging In</h3>
<p>We’ll need some packages to help this expository and extraction. Plus, you’ll need to go to <a rel="nofollow" href="https://developer.feedly.com/" target="_blank">https://developer.feedly.com/</a> to get your developer token (NOTE: this requires a “Pro” account <em>or</em> a regular account and you manually doing the OAuth dance to get an access token; any final “Feedly package” by myself or others will likely use OAuth) and store it in your <code>~/.Renviron</code> in <code>FEEDLY_ACCESS_TOKEN</code>.</p>
<p>I’ve sliced and diced bits from the (non-published) fledgling package to give a peek behind the API covers. There’s <em>plenty</em> of exposition in the following code block comment header to describe what it does:</p>
<pre><code>#' Simplifying some example package setup for this non-pkg example
.pkgenv &lt;- new.env(parent=emptyenv())
.pkgenv$token &lt;- Sys.getenv(&quot;FEEDLY_ACCESS_TOKEN&quot;)

#' In reality, this is more complex since the non-toy example has to
#' refresh tokens when they expire.
.feedly_token &lt;- function() {
  return(.pkgenv$token)
}

#' Get a chunk of a Feedly &quot;stream&quot;
#'
#' For the purposes of this short example, consider a
#' &quot;stream&quot; to be all the historical items in a feed.
#' (Note: the definition is more complex than that)
#'
#' Max &quot;page size&quot; (mad numbner of items returned in a single call)
#' is 1,000. For example simplicity, there's a blanket assumption
#' that if `continuation` is actually present, the caller is
#' savvy and asked for a large number of items (e.g. 10,000).
#' Therefore, assume we're paging by the thousands.
#'
#' @md
#' @param feed_id the id of the stream (for this examplea feed id)
#' @param ct numnber of items to retrieve (API will only return 1,000
#'        items for a single response and populate `continuation`
#'        with a value that should be passed to subsequent calls
#'        to page through the results; `ct` will be reset to 1,000
#'        internally if this is the case)
#' @param continuation see `ct`
#' @references 
#' @return for this example, an ugly `list`
feedly_stream &lt;- function(stream_id, ct=100L, continuation=NULL) {

  ct &lt;- as.integer(ct)

  if (!is.null(continuation)) ct &lt;- 1000L

  httr::GET(
    url = &quot;https://cloud.feedly.com/v3/streams/contents&quot;,
    httr::add_headers(
      `Authorization` = sprintf(&quot;OAuth %s&quot;, .feedly_token())
    ),
    query = list(
      streamId = stream_id,
      count = ct,
      continuation = continuation
    )
  ) -> res

  httr::stop_for_status(res)

  res &lt;- httr::content(res, as=&quot;text&quot;)
  res &lt;- jsonlite::fromJSON(res)

  res

}</code></pre>
<p>We’ll grab 10,000 Feedly entries for the R-Bloggers feed stream:</p>
<pre><code>r_bloggers_feed_id &lt;- &quot;feed/http://feeds.feedburner.com/RBloggers&quot;

rb_stream &lt;- feedly_stream(r_bloggers_feed_id, 10000L)

# preallocate space
streams &lt;- vector(&quot;list&quot;, 10)
streams[1L] &lt;- list(rb_stream)

# gotta catch'em all!
idx &lt;- 2L
while(length(rb_stream$continuation) > 0) {
  cat(&quot;.&quot;, sep=&quot;&quot;) # poor dude's progress par
  feedly_stream(
    stream_id = r_bloggers_feed_id,
    ct = 1000L,
    continuation = rb_stream$continuation
  ) -> rb_stream
  streams[idx] &lt;- list(rb_stream)
  idx &lt;- idx + 1L
}
cat(&quot;\n&quot;)</code></pre>
<p>For those who aren’t used to piecing together bits from API’s like this (and for those who do not have a Pro account, those who didn’t want to write OAuth code or those who don’t use Feedly and cannot reproduce the post example), here’s some dissection:</p>
<pre><code>str(streams, 1)
## List of 12
##  $ :List of 7
##  $ :List of 7
##  $ :List of 7
##  $ :List of 7
##  $ :List of 7
##  $ :List of 7
##  $ :List of 7
##  $ :List of 7
##  $ :List of 7
##  $ :List of 7
##  $ :List of 7
##  $ :List of 6 # No &quot;continuation&quot; in this one

str(streams[[1]], 1)
## List of 7
##  $ id          : chr &quot;feed/http://feeds.feedburner.com/RBloggers&quot;
##  $ title       : chr &quot;R-bloggers&quot;
##  $ direction   : chr &quot;ltr&quot;
##  $ updated     : num 1.52e+12
##  $ alternate   :'data.frame':	1 obs. of  2 variables:
##  $ continuation: chr &quot;15f457e2b66:160d6e:8cbd7d4f&quot;
##  $ items       :'data.frame':	1000 obs. of  22 variables:

glimpse(streams[[1]]$items)
## Observations: 1,000
## Variables: 22
## $ id             </code></pre>