<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/machine-learning-pipelines-for-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://alexioannides.com/2017/05/08/machine-learning-pipelines-for-r/"> R – When Localhost Isn't Enough</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><img title="Pipelines!" alt="" src="https://alexioannides.files.wordpress.com/2017/05/pipelines1.png?w=456" data-recalc-dims="1"></p>
<p>Building machine learning and statistical models often requires pre- and post-transformation of the input and/or response variables, prior to training (or fitting) the models. For example, a model may require training on the logarithm of the response and input variables. As a consequence, fitting and then generating predictions from these models requires repeated application of transformation and inverse-transformation functions – to go from the domain of the original input variables to the domain of the original output variables (via the model). This is usually quite a laborious and repetitive process that leads to messy code and notebooks.</p>
<p>The <code>pipeliner</code> package aims to provide an elegant solution to these issues by implementing a common interface and workflow with which it is possible to:</p>
<ul>
<li>define transformation and inverse-transformation functions;</li>
<li>fit a model on training data; and then,</li>
<li>generate a prediction (or model-scoring) function that automatically applies the entire pipeline of transformations and inverse-transformations to the inputs and outputs of the inner-model and its predicted values (or scores).</li>
</ul>
<p>The idea of pipelines is inspired by the machine learning pipelines implemented in <a title="Pipelines in Apache Spark MLib" target="_blank" rel="nofollow" href="http://spark.apache.org/docs/latest/ml-pipeline.html">Apache Spark’s MLib library</a> (which are in-turn inspired by Python’s scikit-Learn package). This package is still in its infancy and the latest development version can be downloaded from <a title="Pipeliner on GitHub" target="_blank" rel="nofollow" href="https://github.com/AlexIoannides/pipeliner">this GitHub repository</a> using the <code>devtools</code> package (bundled with RStudio),</p>
<pre>
devtools::install_github(&quot;alexioannides/pipeliner&quot;)
</pre>
<h2>Pipes in the Pipleline</h2>
<p>There are currently four types of pipeline section – a section being a function that wraps a user-defined function – that can be assembled into a pipeline:</p>
<ul>
<li><code>transform_features</code>: wraps a function that maps input variables (or features) to another space – e.g.,</li>
</ul>
<pre>
transform_features(function(df) {
  data.frame(x1 = log(df$var1))
})
</pre>
<ul>
<li><code>transform_response</code>: wraps a function that maps the response variable to another space – e.g.,</li>
</ul>
<pre>
transform_response(function(df) {
  data.frame(y = log(df$response))
})
</pre>
<ul>
<li><code>estimate_model</code>: wraps a function that defines how to estimate a model from training data in a data.frame – e.g.,</li>
</ul>
<pre>
estimate_model(function(df) {
  lm(y ~ 1 + x1, df)
})
</pre>
<ul>
<li><code>inv_transform_features(f)</code>: wraps a function that is the inverse to <code>transform_response</code>, such that we can map from the space of inner-model predictions to the one of output domain predictions – e.g.,</li>
</ul>
<pre>
inv_transform_response(function(df) {
  data.frame(pred_response = exp(df$pred_y))
})
</pre>
<p>As demonstrated above, each one of these functions expects as its argument another unary function of a data.frame (i.e. it has to be a function of a single data.frame). With the <strong>exception</strong> of <code>estimate_model</code>, which expects the input function to return an object that has a <code>predict.object-class-name</code> method existing in the current environment (e.g. <code>predict.lm</code> for linear models built using <code>lm()</code>), all the other transform functions also expect their input functions to return data.frames (consisting entirely of columns <strong>not</strong> present in the input data.frame). If any of these rules are violated then appropriately named errors will be thrown to help you locate the issue.</p>
<p>If this sounds complex and convoluted then I encourage you to to skip to the examples below – this framework is <strong>very</strong> simple to use in practice. Simplicity is the key aim here.</p>
<h2>Two Interfaces to Rule Them All</h2>
<p>I am a great believer and protagonist for functional programming – especially for data-related tasks like building machine learning models. At the same time the notion of a ‘machine learning pipeline’ is well represented with a simple object-oriented class hierarchy (which is how it is implemented in <a title="Pipelines in Apache Spark MLib" target="_blank" rel="nofollow" href="http://spark.apache.org/docs/latest/ml-pipeline.html">Apache Spark’s</a>). I couldn’t decide which style of interface was best, so I implemented both within <code>pipeliner</code> (using the same underlying code) and ensured their output can be used interchangeably. To keep this introduction simple, however, I’m only going to talk about the functional interface – those interested in the (more) object-oriented approach are encouraged to read the manual pages for the <code>ml_pipeline_builder</code> ‘class’.</p>
<h3>Example Usage with a Functional Flavor</h3>
<p>We use the <code>faithful</code> dataset shipped with R, together with the <code>pipeliner</code> package to estimate a linear regression model for the eruption duration of ‘Old Faithful’ as a function of the inter-eruption waiting time. The transformations we apply to the input and response variables – before we estimate the model – are simple scaling by the mean and standard deviation (i.e. mapping the variables to z-scores).</p>
<p>The end-to-end process for building the pipeline, estimating the model and generating in-sample predictions (that include all interim variable transformations), is as follows,</p>
<pre>
library(pipeliner)

data &lt;- faithful

lm_pipeline &lt;- pipeline(
  data,

  transform_features(function(df) {
    data.frame(x1 = (df$waiting - mean(df$waiting)) / sd(df$waiting))
  }),

  transform_response(function(df) {
    data.frame(y = (df$eruptions - mean(df$eruptions)) / sd(df$eruptions))
  }),

  estimate_model(function(df) {
    lm(y ~ 1 + x1, df)
  }),

  inv_transform_response(function(df) {
    data.frame(pred_eruptions = df$pred_model * sd(df$eruptions) + mean(df$eruptions))
  })
)

in_sample_predictions &lt;- predict(lm_pipeline, data, verbose = TRUE)
head(in_sample_predictions)
##   eruptions waiting         x1 pred_model pred_eruptions
## 1     3.600      79  0.5960248  0.5369058       4.100592
## 2     1.800      54 -1.2428901 -1.1196093       2.209893
## 3     3.333      74  0.2282418  0.2056028       3.722452
## 4     2.283      62 -0.6544374 -0.5895245       2.814917
## 5     4.533      85  1.0373644  0.9344694       4.554360
## 6     2.883      55 -1.1693335 -1.0533487       2.285521
</pre>
<h3>Accessing Inner Models and Prediction Functions</h3>
<p>We can access the estimated inner models directly and compute summaries, etc – for example,</p>
<pre>
summary(lm_pipeline$inner_model)
##
## Call:
## lm(formula = y ~ 1 + x1, data = df)
##
## Residuals:
##      Min       1Q   Median       3Q      Max
## -1.13826 -0.33021  0.03074  0.30586  1.04549
##
## Residual standard error: 0.435 on 270 degrees of freedom
## Multiple R-squared:  0.8115, Adjusted R-squared:  0.8108
## F-statistic:  1162 on 1 and 270 DF,  p-value: &lt; 2.2e-16
</pre>
<p>Pipeline prediction functions can also be accessed directly in a similar way – for example,</p>
<pre>
pred_function &lt;- lm_pipeline$predict
predictions &lt;- pred_function(data, verbose = FALSE)

head(predictions)
##   pred_eruptions
## 1       4.100592
## 2       2.209893
## 3       3.722452
## 4       2.814917
## 5       4.554360
## 6       2.285521
</pre>
<h1>Turbo-Charged Pipelines in the Tidyverse</h1>
<p>The <code>pipeliner</code> approach to building models becomes even more concise when combined with the set of packages in the [tidyverse](<a target="_blank" rel="nofollow" href="http://tidyverse.org/">http://tidyverse.org</a> &quot;Welcome to The Tidyverse!&quot;). For example, the 'Old Faithful' pipeline could be rewritten as,</p>
<pre>
library(tidyverse)

lm_pipeline %
  pipeline(
    transform_features(function(df) {
      transmute(df, x1 = (waiting - mean(waiting)) / sd(waiting))
    }),

    transform_response(function(df) {
      transmute(df, y = (eruptions - mean(eruptions)) / sd(eruptions))
    }),

    estimate_model(function(df) {
      lm(y ~ 1 + x1, df)
    }),

    inv_transform_response(function(df) {
      transmute(df, pred_eruptions = pred_model * sd(eruptions) + mean(eruptions))
    })
  )

head(predict(lm_pipeline, data))
## [1] 4.100592 2.209893 3.722452 2.814917 4.554360 2.285521
</pre>
<p>Nice, compact and expressive (if I don’t say so myself)!</p>
<h3>Compact Cross-validation</h3>
<p>If we now introduce the <code>modelr</code> package into this workflow and adopt the the list-columns pattern described in Hadley Wickham’s <a title="R 4 Data Science - Many Models &amp; List Columns" target="_blank" rel="nofollow" href="http://r4ds.had.co.nz/many-models.html#list-columns-1">R for Data Science</a>, we can also achieve wonderfully compact end-to-end model estimation and cross-validation,</p>
<pre>
library(modelr)

# define a function that estimates a machine learning pipeline on a single fold of the data
pipeline_func &lt;- function(df) {
  pipeline(
    df,
    transform_features(function(df) {
      transmute(df, x1 = (waiting - mean(waiting)) / sd(waiting))
    }),

    transform_response(function(df) {
      transmute(df, y = (eruptions - mean(eruptions)) / sd(eruptions))
    }),

    estimate_model(function(df) {
      lm(y ~ 1 + x1, df)
    }),

    inv_transform_response(function(df) {
      transmute(df, pred_eruptions = pred_model * sd(eruptions) + mean(eruptions))
    })
  )
}

# 5-fold cross-validation using machine learning pipelines
cv_rmse %
  mutate(model = map(train, ~ pipeline_func(as.data.frame(.x))),
         predictions = map2(model, test, ~ predict(.x, as.data.frame(.y))),
         residuals = map2(predictions, test, ~ .x - as.data.frame(.y)$eruptions),
         rmse = map_dbl(residuals, ~ sqrt(mean(.x ^ 2)))) %>%
  summarise(mean_rmse = mean(rmse), sd_rmse = sd(rmse))

cv_rmse
## # A tibble: 1 × 2
##   mean_rmse    sd_rmse
##             
## 1 0.4877222 0.05314748
</pre>
<h1>Forthcoming Attractions</h1>
<p>I built <code>pipeliner</code> largely to fill a hole in my own workflows. Up until now I’ve used Max Kuhn’s excellent <a title="Caret" target="_blank" rel="nofollow" href="http://topepo.github.io/caret/index.html">caret package</a> quite a bit, but for in-the-moment model building (e.g. within a R Notebook) it wasn’t simplifying the code <em>that</em> much, and the style doesn’t quite fit with the tidy and functional world that I now inhabit most of the time. So, I plugged the hole by myself. I intend to live with <code>pipeliner</code> for a while to get an idea of where it might go next, but I am always open to suggestions (and bug notifications) – please <a title="Pipeliner Issues on GitHub" target="_blank" rel="nofollow" href="https://github.com/AlexIoannides/pipeliner/issues">leave any ideas here</a>.</p>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/alexioannides.wordpress.com/395/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/alexioannides.wordpress.com/395/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://alexioannides.com/2017/05/08/machine-learning-pipelines-for-r/"> R – When Localhost Isn't Enough</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/1ojPmLhUK_c"></a></div>