<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/snakes-in-a-package-combining-python-and-r-with-reticulate/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://www.mango-solutions.com/blog/snakes-in-a-package-combining-python-and-r-with-reticulate"> RBlog – Mango Solutions</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>When I first started working as a data scientist (or something like it) I was told to program in C++ and Java. Then R came along and it was liberating, my ability to do data analysis increased substantially. As my applications grew in size and complexity, I started to miss the structure of Java/C++. At the time, Python felt like a good compromise so I switched again. After joining Mango Solutions I noticed I was not an anomaly, most data scientists here know both Python and R.</p>
</div>
<p>Nowadays whenever I do my work in R there is a constant nagging voice in the back of my head telling me “you should do this in Python”. And when I do my work in Python it’s telling me “you can do this faster in R”. So when the <strong>reticulate</strong> package came out I was overjoyed and in this blogpost I will explain to you why.</p>
<div>
<h3>re-tic-u-late (rĭ-tĭkˈyə-lĭt, -lātˌ)</h3>
<p>So what exactly does <strong>reticulate</strong> do? It’s goal is to facilitate interoperability between Python and R. It does this by embedding a Python session within the R session which enables you to call Python functionality from within R. I’m not going to go into the nitty gritty of how the package works here; RStudio have done a great job in providing some excellent <a rel="nofollow" href="https://rstudio.github.io/reticulate/" target="_blank">documentation</a> and a <a rel="nofollow" href="https://resources.rstudio.com/webinars/r-rstudio-1-2-amp-python-a-love-story-sean-lopp" target="_blank">webinar</a>. Instead I’ll show a few examples of the main functionality.</p>
<p>Just like R, the House of Python was built upon packages. Except in Python you don’t load functionality from a package through a call to <code>library</code>but instead you import a module. <strong>reticulate</strong> mimics this behaviour and opens up all the goodness from the module that is imported.</p>
<pre><code><span>library</span>(reticulate)
np &lt;- import(<span>&quot;numpy&quot;</span>)
<span># the Kronecker product is my favourite matrix operation</span>
np$kron(c(<span>1</span>,<span>2</span>,<span>3</span>), c(<span>4</span>,<span>5</span>,<span>6</span>))</code></pre>
<pre><code>## [1]  4  5  6  8 10 12 12 15 18</code></pre>
<p>In the above code I import the <em>numpy</em> module which is a powerful package for all sorts of numerical computations. <strong>reticulate</strong> then gives us an interface to all the functions (and objects) from the <em>numpy</em> module. I can call these functions just like any other R function and pass in R objects, <strong>reticulate</strong> will make sure the R objects are converted to the appropriate Python objects.</p>
<p>You can also run Python code through <code>source_python</code> if it’s an entire script or <code>py_eval</code>/<code>py_run_string</code> if it’s a single line of code. Any objects (functions or data) created by the script are loaded into your R environment. Below is an example of using <code>py_eval</code>.</p>
<pre><code>data(<span>&quot;mtcars&quot;</span>)
py_eval(<span>&quot;r.mtcars.sum(axis=0)&quot;</span>)</code></pre>
<pre><code>## mpg      642.900
## cyl      198.000
## disp    7383.100
## hp      4694.000
## drat     115.090
## wt       102.952
## qsec     571.160
## vs        14.000
## am        13.000
## gear     118.000
## carb      90.000
## dtype: float64</code></pre>
<p>Notice the use of the <code>r.</code> prefix in front of the <code>mtcars</code> object in the python code. The <code>r</code> object exposes the R environment to the python session, it’s equivalent in the R session is the <code>py</code> object. The <code>mtcars</code> data.frame is converted to a <em>pandas</em> DataFrame to which I then applied the <code>sum</code>function on each column.</p>
<p>Clearly RStudio have put in a lot of effort to ensure a smooth interface to Python, from the easy conversion of objects to the <a rel="nofollow" href="https://rstudio.github.io/reticulate/articles/rstudio_ide.html" target="_blank">IDE integration</a>. Not only will <strong>reticulate</strong> enable R users to benefit from the wealth of functionality from Python, I believe it will also enable more collaboration and increased sharing of knowledge.</p>
</div>
<div>
<h3>Enter mailman</h3>
<p>So what is it exactly that you can do with Python that you can’t with R? I asked myself the same question until I came across the following use case.</p>
<p>While helping a colleague out with a blogpost it was suggested that I should publish it on a Tuesday. No rationale was given so naturally I wondered if I could provide one using data. The data would have to come from <a rel="nofollow" href="http://r-bloggers.com/" target="_blank">R-bloggers</a>. This is a great resource for reading blogposts about R (and related topics) and they also provide a daily newsletter with a link to the blogposts from that day. At the time the newsletter seemed the easiest way to collect data <a rel="nofollow" href="https://www.mango-solutions.com/blog/snakes-in-a-package-combining-python-and-r-with-reticulate#fn1" target="_blank"><sup>1</sup></a>. All I needed to do now is extract the data from my Gmail account.</p>
<p>Therein lies the problem as I want to avoid querying the Gmail server (it wouldn’t make it easy to reproduce). Fortunately, Google have made it easy to download your data (thanks to the Google Data Liberation Front) through <a rel="nofollow" href="https://takeout.google.com/" target="_blank">Google Takeout</a>. Unfortunately, all the e-mails are exported in the <em>mbox</em> format. Although this is a plain text based format it would take some effort to write a parser in R, something I wasn’t willing to do. And then came along Python, which has a built-in <em>mbox</em>-parser in the <em>mailbox</em> module.</p>
<p>Using <strong>reticulate</strong> I extracted the necessary information from each e-mail.</p>
<pre><code><span># import the module</span>
mailbox &lt;- import(<span>&quot;mailbox&quot;</span>)
<span># use the mbox function to open a file connection</span>
cnx &lt;- mailbox$mbox(<span>&quot;rblogs_box.mbox&quot;</span>)

<span># the messages are stored as key/value pairs</span>
<span># in this case they are indexed by an integer id</span>
message &lt;- cnx$get_message(<span>10L</span>)
<span># each message has a number of fields with meta-data</span>
message$get(<span>&quot;Date&quot;</span>)</code></pre>
<pre><code>## [1] &quot;Mon, 12 Dec 2016 23:56:19 +0000&quot;</code></pre>
<pre><code>message$get(<span>&quot;Subject&quot;</span>)</code></pre>
<pre><code>## [1] &quot;[R-bloggers] Building Shiny App exercises part 1 (and 7 more aRticles)&quot;</code></pre>
<p>And there we have it! I just read an e-mail from an <em>mbox</em>-file with very little effort. Of course I will need to do this for all messages, so I wrote a function to help me. And because we’re living in the Age of R I placed this function in an R package. You can find it on the MangoTheCat github repo, it is called <a rel="nofollow" href="https://github.com/MangoTheCat/mailman" target="_blank"><strong>mailman</strong></a>.</p>
</div>
<div>
<h3>To publish or not to publish?</h3>
<p>I have yet to provide a rationale for publishing a blogpost on a particular day so let’s quickly get to it. With the package all sorted I can now call the function <code>mailman::read_messages</code> to get a <code>tibble</code> with everything I need.</p>
<p>We can extract the number of blogposts on a particular date from the subject of each e-mail. Aggregating that to day of week will then give us a good overview of which day is popular.</p>
<pre><code><span>library</span>(dplyr)
<span>library</span>(mailman)
<span>library</span>(lubridate)
<span>library</span>(stringr)

messages &lt;- read_messages(<span>&quot;rblogs_box.mbox&quot;</span>, type=<span>&quot;mbox&quot;</span>) %>% 
  mutate(Date = as.POSIXct(Date, format=<span>&quot;%a, %d %b %Y %H:%M:%S %z&quot;</span>),
         Day_of_Week = wday(Date, label=<span>TRUE</span>, abbr=<span>TRUE</span>),
         Number_Articles = str_extract(Subject, <span>&quot;[0-9](?=[\\n]* more aRticles)&quot;</span>), 
         <span># Whenever a regex works you feel like a superhero!</span>
         Number_Articles = as.numeric(Number_Articles) + <span>1</span>,
         <span># Ok, sometimes it doesn't work but you're still a hero for trying!</span>
         Number_Articles = ifelse(is.na(Number_Articles), <span>1</span>, Number_Articles)) %>% 
  select(Date, Day_of_Week, Number_Articles)</code></pre>
<p><a rel="nofollow" href="https://i2.wp.com/www.mango-solutions.com/wp-content/uploads/sites/6/2018/09/download.png?ssl=1" target="_blank"><img sizes="(max-width: 672px) 100vw, 672px" src="https://i2.wp.com/www.mango-solutions.com/wp-content/uploads/sites/6/2018/09/download.png?w=450&ssl=1" alt="" class="wp-image-6212" data-recalc-dims="1"></a></p>
<p>Judging by the graph, weekends would be a good time to publish a blogpost as there is less competition. Then again, not many people might read blogposts in the weekend. The next candidate would then be Monday which has the lowest average among the weekdays. Coming back to my original quest, I can conclude that publishing on a Tuesday is not the best option.</p>
</div>
<div>
<h3>In summary</h3>
<p>In my opinion, the <strong>reticulate</strong> package is a ground-breaking development. It allows me to combine the good parts of R with the good parts of Python (it’s already in use by the <a rel="nofollow" href="http://tensorflow.rstudio.com/" target="_blank"><strong>tensorflow</strong></a> and <a rel="nofollow" href="http://keras.rstudio.com/" target="_blank"><strong>keras</strong></a> packages). Also, it allows the data science community to collaborate more easily and focus our energy on getting things done. This is the future, this is R and Python (Rython? PRython? PyR?).</p>
</div>
<div>
<hr>
<ol>
<li>After I had collected all the data, <a rel="nofollow" href="https://rud.is/b/2018/04/04/exploring-r-bloggers-posts-with-the-feedly-api/" target="_blank">Bob Rudis</a> wrote about the Feedly API and released a dataset of blogposts over a longer time period. I would say his solution is preferable even though my results are slightly different due to the more recent time horizon.<a rel="nofollow" href="https://www.mango-solutions.com/blog/snakes-in-a-package-combining-python-and-r-with-reticulate#fnref1" target="_blank"><img alt="↩" src="https://i2.wp.com/s.w.org/images/core/emoji/2.4/72x72/21a9.png?w=456&ssl=1" class="wp-smiley" data-recalc-dims="1"></a></li>
</ol>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://www.mango-solutions.com/blog/snakes-in-a-package-combining-python-and-r-with-reticulate"> RBlog – Mango Solutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/SmrLOY0JQLk" height="1"></a></div>