<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/announcing-the-simputation-package-make-imputation-simple/"></div></div><div><a data-url="https://www.r-bloggers.com/announcing-the-simputation-package-make-imputation-simple/" data-counturl="https://www.r-bloggers.com/announcing-the-simputation-package-make-imputation-simple/" data-count="horizontal" data-text="Announcing the simputation package: make imputation simple" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.markvanderloo.eu/yaRb/2016/09/13/announcing-the-simputation-package-make-imputation-simple/"> R – Mark van der Loo</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p>I am happy to announce that my <a target="_blank" href="https://cran.r-project.org/package=simputation">simputation</a> package has appeared on CRAN this weekend. This package aims to simplify missing value imputation. In particular it offers standardized interfaces that</p>
<ul>
<li>make it easy to define both imputation method and imputation model;</li>
<li>for multiple variables at once;</li>
<li>while grouping data by categorical variables;</li>
<li>all fitting in the <a target="_blank" href="https://cran.r-project.org/package=magrittr">magrittr</a> not-a-pipeline.</li>
</ul>
<h3>A few examples</h3>
<p>To start with an example, let us first create a data set with some missings.</p>
<pre><code>dat &lt;- iris
# empty a few fields
dat[1:3,1] &lt;- dat[3:7,2] &lt;- dat[8:10,5] &lt;- NA
head(dat,10)
</code></pre>
<pre><code>##    Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1            NA         3.5          1.4         0.2  setosa
## 2            NA         3.0          1.4         0.2  setosa
## 3            NA          NA          1.3         0.2  setosa
## 4           4.6          NA          1.5         0.2  setosa
## 5           5.0          NA          1.4         0.2  setosa
## 6           5.4          NA          1.7         0.4  setosa
## 7           4.6          NA          1.4         0.3  setosa
## 8           5.0         3.4          1.5         0.2    &lt;NA>
## 9           4.4         2.9          1.4         0.2    &lt;NA>
## 10          4.9         3.1          1.5         0.1    &lt;NA>
</code></pre>
<p>Below, we first impute <code>Sepal.Width</code> and <code>Sepal.Length</code> by regression on <code>Petal.Width</code> and <code>Species</code>. After this we impute <code>Species</code> using a <a target="_blank" href="https://en.wikipedia.org/wiki/Decision_tree_learning">decision tree</a> model (CART) using every other variable as a predictor (including the ones just imputed).</p>
<pre><code>library(magrittr)    # load the %>% operator
library(simputation) 
imputed &lt;- dat %>% 
  impute_lm(Sepal.Width + Sepal.Length ~ Petal.Width + Species) %>%
  impute_cart(Species ~ .)
head(imputed,10)
</code></pre>
<pre><code>##    Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1      4.979844    3.500000          1.4         0.2  setosa
## 2      4.979844    3.000000          1.4         0.2  setosa
## 3      4.979844    3.409547          1.3         0.2  setosa
## 4      4.600000    3.409547          1.5         0.2  setosa
## 5      5.000000    3.409547          1.4         0.2  setosa
## 6      5.400000    3.561835          1.7         0.4  setosa
## 7      4.600000    3.485691          1.4         0.3  setosa
## 8      5.000000    3.400000          1.5         0.2  setosa
## 9      4.400000    2.900000          1.4         0.2  setosa
## 10     4.900000    3.100000          1.5         0.1  setosa
</code></pre>
<p>The package is pretty lenient against failure of imputation. For example, if one of the predictors is missing, fields just remain unimputed and if one of the models cannot be fitted, only a warning is issued (not shown here).</p>
<pre><code>dat %>% impute_lm(Sepal.Length ~ Sepal.Width + Species) %>% head(3)
</code></pre>
<pre><code>##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1     5.076579         3.5          1.4         0.2  setosa
## 2     4.675654         3.0          1.4         0.2  setosa
## 3           NA          NA          1.3         0.2  setosa
</code></pre>
<p>So here, the third <code>Sepal.Length</code> value could not be imputed since the predictor <code>Sepal.Width</code> is missing.</p>
<p>It is possible to split data into groups before estimating the imputation model and predicting missing values. There are two ways. The first is to use the <code>|</code> operator to specify grouping variables.</p>
<pre><code># We first need to complete 'Species'. Here, we use sequential 
# hot deck after sorting by Petal.Length
dat %&lt;>% impute_shd(Species ~ Petal.Length) 
# Now impute Sepal.Length by regressing on 
# Sepal.Width, computing a model for each Species.
dat %>% impute_lm(Sepal.Length ~ Sepal.Width | Species) %>% head(3)
</code></pre>
<pre><code>##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1     5.067813         3.5          1.4         0.2  setosa
## 2     4.725677         3.0          1.4         0.2  setosa
## 3           NA          NA          1.3         0.2  setosa
</code></pre>
<p>The second way is to use the <code>group_by</code> command from <a target="_blank" href="https://cran.r-project.org/package=dplyr">dplyr</a></p>
<pre><code>dat %>% dplyr::group_by(Species) %>% 
    impute_lm(Sepal.Length ~ Sepal.Width) %>% 
    head(3)
</code></pre>
<pre><code>## Source: local data frame [3 x 5]
## Groups: Species [1]
## 
##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
##          &lt;dbl>       &lt;dbl>        &lt;dbl>       &lt;dbl>  &lt;fctr>
## 1     5.067813         3.5          1.4         0.2  setosa
## 2     4.725677         3.0          1.4         0.2  setosa
## 3           NA          NA          1.3         0.2  setosa
</code></pre>
<p>Note: by using <code>group_by</code>, we also transformed the data.frame to a <a target="_blank" href="https://blog.rstudio.org/2016/03/24/tibble-1-0-0/">tibble</a>, which not only sounds funny when you pronounce it (tibble, TIBBLE, tibble? tibbebbebbebble) but is also pretty useful.</p>
<h3>Supported methods and how to specify them</h3>
<p>Currently, the package supports the following methods:</p>
<ul>
<li>Model based (optionally add [non-]parametric random residual)
<ul>
<li>linear regression </li>
<li>robust linear regression</li>
<li>CART models</li>
<li>Random forest</li>
</ul>
</li>
<li>Donor imputation (including various donor pool specifications)
<ul>
<li>k-nearest neigbour (based on <a target="_blank" href="https://cran.r-project.org/package=gower">gower</a>‘s distance)</li>
<li>sequential hotdeck (LOCF, NOCB)</li>
<li>random hotdeck</li>
<li>Predictive mean matching</li>
</ul>
</li>
<li>Other
<ul>
<li>(groupwise) median imputation (optional random residual)</li>
<li>Proxy imputation (copy from other variable) </li>
</ul>
</li>
</ul>
<p>Any call to one of the <code>impute_</code> functions looks as follows:</p>
<pre><code>impute_&lt;method>(data, formula [, &lt;method-specific options>])
</code></pre>
<p>and the formula always has the following form:</p>
<pre><code>&lt;imputed variables> ~ &lt;model specification> [|&lt;grouping variables>]
</code></pre>
<p>The parts in square brackets are optional.</p>
<p>Please see the package <a target="_blank" href="https://cran.r-project.org/web/packages/simputation/vignettes/intro.html">vignette</a> for more examples and details, or <code>?simputation::impute_</code> for an overview of all imputation functions.</p>
<p>Happy imputing!</p>
</div>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/announcing-the-simputation-package-make-imputation-simple/"></div></div><div><a data-url="https://www.r-bloggers.com/announcing-the-simputation-package-make-imputation-simple/" data-counturl="https://www.r-bloggers.com/announcing-the-simputation-package-make-imputation-simple/" data-count="vertical" data-text="Announcing the simputation package: make imputation simple" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.markvanderloo.eu/yaRb/2016/09/13/announcing-the-simputation-package-make-imputation-simple/"> R – Mark van der Loo</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/wOeMgSXnIaM"></a></div>