<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/about-risks-and-side-effects-consult-your-purrr-macist/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://www.statworx.com/de/blog/about-risks-and-side-effects-consult-your-purrr-macist/"> r-bloggers – STATWORX</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2>Capture errors, warnings and messages, but keep your list operations going</h2>
<p>In a recent post about text mining, I discussed some solutions to webscraping the contents of our STATWORX blog using the <em>purrr</em>-package. However, while preparing the next the episode of my series on text mining, I remembered a little gimmick that I found quite helpful along the way. Thus, a little detour: How do I capture side-effects and errors when I perform operations on lists with <em>purrr</em> rather than using a loop?</p>
<p><img alt="meme blog loop" src="https://i1.wp.com/www.statworx.com/wp-content/uploads/meme-blog-loop.jpg?w=450&ssl=1" class="wp-image-13863" data-recalc-dims="1"></p>
<p>First of all, a quick motivating example: Imagine we will build a parser for the blog-section of our STATWORX website. However, we have no idea how many entries were posted in the meantime (the hive of data scientists in our office is pretty quick with writing these things, usually). Thus, we need such a function to be more robust in the sense that it can endure the cruelties of &quot;404 – Not found&quot; error messages and still continues parsing after running into an error.</p>
<h2>How could this <em>possibly()</em> work?</h2>
<p>So let's use some beautiful <em>purrr</em> adverbs to fearlessly record all our outputs, errors and warnings, rather than stopping and asking the user to handle side-effects the exact moment errors turn up. These adverbs are reminiscent of <code>try()</code>, however, these are a little more convenient for operations on lists.</p>
<p>Let's consider a more complex motivating example first, but no worries – there are more obvious examples to help explain the nitty-gritties further down this page. The R-code below illustrates our use of <code>possibly()</code> for later use with <code>puurr::map()</code>. First, let us have a look at what we tried to achieve with our function. More specifically, what happens between the curly braces below: Our <code>robust_parse()</code> function will simply parse HTML-webpages for other links using URLs that we provide it with. In this case, we simply use <code>paste0()</code> to create a vector of links to our blog overview pages, extract the weblinks from these each of these pages using <code>XML::xpathSApply()</code>, pipe these weblinks into a data_frame and clean our results from duplicates using <code>dplyr::filter()</code> – there are various overview pages that group our blogs by category – and <code>dplyr::distinct()</code>.</p>
<pre><code>robust_parse &lt;- possibly(function(value){
  htmlParse(paste0(&quot;http://www.statworx.com/de/blog/page/&quot;,
                   value, &quot;/&quot;)) %>%
    xpathSApply(., &quot;//a/@href&quot;) %>%
    data_frame(.) %>%
    filter(., grepl(&quot;/blog&quot;, .)) %>%
    filter(., !grepl(&quot;/blog/$|/blog/page/|/data-science/|/statistik/&quot;, .)) %>%
    distinct()
  }, otherwise = NULL)
</code></pre>
<p>Second, let us inspect how we employ <code>possibly()</code> in this context. <code>possibly()</code> expects a function to be modified from us, as well as the argument <code>otherwise</code>, stating what it is supposed to do when things go south. In this case, we want <code>NULL</code> as an output value. Another popular choice would be <code>NA</code>, signaling that somewhere, we have not produced a string as intended. However, in our example we are happy with <code>NULL</code>, since we only want to parse the pages that exist and do not require a specific listing of pages that do not exist (or what happened when we did not find a page).</p>
<pre><code lang="python">webpages &lt;- map_df(0:100, ~robust_parse(.)) %>%
			unlist

webpages
</code></pre>
<pre><code lang="r">.1 
&quot;https://www.statworx.com/de/blog/strsplit-but-keeping-the-delimiter/&quot;
.2 
&quot;https://www.statworx.com/de/blog/data-science-in-python-vorstellung-von-nuetzlichen-datenstrukturen-teil-1/&quot;
.3 
&quot;https://www.statworx.com/de/blog/burglr-stealing-code-from-the-web/&quot;
.4 
&quot;https://www.statworx.com/de/blog/regularized-greedy-forest-the-scottish-play-act-i/&quot;
...
</code></pre>
<p>Third, we use our new function <code>robust_parse()</code> to operate on a vector or list of integers from 0 to 100 (possible numbers of subpages we want to parse) and have a quick look at the beautiful links we extracted. Just as a reminder, below you find the code to extract and clean the contents of the individual pages, using another <code>map_df()</code>-based loop – which is the focus of <a rel="nofollow" href="https://www.statworx.com/blog/furchtlose-grammatiker-textmining-im-tidyverse-teil-2/" target="_blank">another post</a>.</p>
<pre><code>tidy_statworx_blogs &lt;- map_df(webpages, ~read_html(.) %>% 
                              	htmlParse(., asText = TRUE) %>%
                                xpathSApply(., &quot;//p&quot;, xmlValue) %>%
                                paste(., collapse = &quot;\n&quot;) %>%
                                gsub(&quot;\n&quot;, &quot;&quot;, .) %>%
                                data_frame(text = .) %>%
                                unnest_tokens(word, text) %>%
                                anti_join(data_frame(word = stopwords(&quot;de&quot;))) %>% 
                                anti_join(data_frame(word = stopwords(&quot;en&quot;))) %>% 
                                mutate(author = .$word[2]))
</code></pre>
<p>However, we actually want to go back to our <em>purrr</em>-helpers and see what they can do for us. To be more specific, rather than <em>helpers</em>, these are actually called <em>adverbs</em> since we use them to modify the behavior of a function (i.e. a <em>verb</em>). Our current <code>robust_parse()</code> function does not produce entries when the loop does not successfully find a webpage to parse for links. Consider the situation where you intend to keep track of unsuccessfull operations and of errors that arise along the way. Instead of further exploring <em>purrr</em> adverbs using the above code, let us look at a much easier example to realise the possible contexts in which using <em>purrr</em> adverbs might help you out.</p>
<h2>A much easier example: Try dividing a character string by 2</h2>
<p>Suppose there is an element in our list where our amazing division powers are useless: We are going to try to divide all the elements in our list by 2 – but this time, we want <em>purrr</em> to note where the function <code>i_divide_things</code> resists dividing particular elements for us. Again, the <em>otherwise</em> argument helps us defining our output in situations that are beyond the scope of our function.</p>
<pre><code>i_divide_things &lt;- possibly(function(value){
				value /2},
                  		otherwise = &quot;I won't divide this for you.&quot;)

# Let's try our new function

> purrr::map(list(1, 2, &quot;a&quot;, 6), ~ i_divide_things(.))
</code></pre>
<pre><code lang="r">[[1]]
[1] 0.5

[[2]]
[1] 1

[[3]]
[1] &quot;I won't divide this for you.&quot;

[[4]]
[1] 3
</code></pre>
<p>However, consider the case where &quot;something did not work out&quot; might not suffice and you want to keep track of possible errors as well as warnings while still retaining the entire output. A job for <code>safely()</code>: As illustrated below, wrapping our function by <code>safely()</code>, helps us output a nested list. For each element of the input, the output provides two components – <code>$result</code> and <code>$error</code>. For all iterations where a list element is numeric, <code>$result</code> includes a numeric output and an empty (<code>= NULL</code>) error-element. Only for the third list element – where our function stumbled over a character input – we captured an error message, as well as the result we defined using <code>otherwise</code>.</p>
<pre><code>i_divide_things &lt;- safely(function(value){
                      value /2},
                      otherwise = &quot;This did not quite work out.&quot;)

purrr::map(list(1, 2, &quot;a&quot;, 6), ~ i_divide_things(.))
</code></pre>
<pre><code lang="r">[[1]]
[[1]]$result
[1] 0.5

[[1]]$error
NULL


[[2]]
[[2]]$result
[1] 1

[[2]]$error
NULL


[[3]]
[[3]]$result
[1] &quot;This did not quite work out.&quot;

[[3]]$error
</code></pre>