<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/xray-the-r-package-to-have-x-ray-vision-on-your-datasets/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://blog.datascienceheroes.com/x-ray-vision-on-your-datasets/"> R - Data Science Heroes Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><img alt="xray: The R Package to Have X Ray Vision on your Datasets" src="https://i2.wp.com/blog.datascienceheroes.com/content/images/2017/11/xray.png?w=456" data-recalc-dims="1"></p>
<p><img alt="xray: The R Package to Have X Ray Vision on your Datasets" src="https://i1.wp.com/blog.datascienceheroes.com/content/images/2017/11/xray-1.png?w=456&ssl=1" data-recalc-dims="1"><br>
This package lets you analyze the variables of a dataset, to evaluate how is the shape of your data. Consider this the first step when you have your data for modeling, you can use this package to analyze all variables and check if there is anything weird worth transforming or even avoiding the variable altogether.</p>
<h2>Installation </h2>
<p>You can install xray from github with:</p>
<pre><code># install.packages(&quot;devtools&quot;)
devtools::install_github(&quot;sicarul/xray&quot;)  
</code></pre>
<h2>Usage </h2>
<h3>Anomaly detection</h3>
<p><code>xray::anomalies</code> analyzes all your columns for anomalies, whether they are NAs, Zeroes, Infinite, etc, and warns you if it detects variables with at least 80% of rows with those anomalies. It also warns you when all rows have the same value.</p>
<p>Example:</p>
<pre><code>data(longley)  
badLongley=longley  
badLongley$GNP=NA  
xray::anomalies(badLongley)  
#> Warning in xray::anomalies(badLongley): Found 1 possible problematic variables: 
#> GNP
#> $variables
#>       Variable  q qNA pNA qZero pZero qBlank pBlank qInf pInf qDistinct
#> 1          GNP 16  16   1     0     0      0      0    0    0         1
#> 2 GNP.deflator 16   0   0     0     0      0      0    0    0        16
#> 3   Unemployed 16   0   0     0     0      0      0    0    0        16
#> 4 Armed.Forces 16   0   0     0     0      0      0    0    0        16
#> 5   Population 16   0   0     0     0      0      0    0    0        16
#> 6         Year 16   0   0     0     0      0      0    0    0        16
#> 7     Employed 16   0   0     0     0      0      0    0    0        16
#>      type anomalous_percent
#> 1 Logical                 1
#> 2 Numeric                 0
#> 3 Numeric                 0
#> 4 Numeric                 0
#> 5 Numeric                 0
#> 6 Integer                 0
#> 7 Numeric                 0
#> 
#> $problem_variables
#>   Variable  q qNA pNA qZero pZero qBlank pBlank qInf pInf qDistinct
#> 1      GNP 16  16   1     0     0      0      0    0    0         1
#>      type anomalous_percent
#> 1 Logical                 1
#>                                                              problems
#> 1 Anomalies present in 100% of the rows. Less than 2 distinct values.
</code></pre>
<h3>Distributions</h3>
<p><code>xray::distributions</code> tries to analyze the distribution of your variables, so you can understand how each variable is statistically structured. It also returns a percentiles table of numeric variables as a result, which can inform you of the shape of the data.</p>
<pre><code>distrLongley=longley  
distrLongley$testCategorical=c(rep('One',7), rep('Two', 9))  
xray::distributions(distrLongley)  
</code></pre>
<p><img alt="xray: The R Package to Have X Ray Vision on your Datasets" src="https://i1.wp.com/blog.datascienceheroes.com/content/images/2017/11/README-example-distributions-1.png?w=456&ssl=1" data-recalc-dims="1"><br>
<img alt="xray: The R Package to Have X Ray Vision on your Datasets" src="https://i2.wp.com/blog.datascienceheroes.com/content/images/2017/11/README-example-distributions-2.png?w=456&ssl=1" data-recalc-dims="1"></p>
<pre><code>#>       Variable      p_1     p_10     p_25     p_50     p_75    p_90
#> 1 GNP.deflator    83.78    88.35   94.525    100.6   111.25  114.95
#> 2          GNP 237.8537   258.74  317.881  381.427 454.0855 510.387
#> 3   Unemployed   187.93   201.55  234.825   314.35   384.25   434.4
#> 4 Armed.Forces   147.61    160.3    229.8   271.75  306.075  344.85
#> 5   Population 107.7616 109.2025 111.7885 116.8035  122.304  126.61
#> 6         Year  1947.15   1948.5  1950.75   1954.5  1958.25  1960.5
#> 7     Employed  60.1938  60.7225  62.7125   65.504  68.2905 69.4475
#>       p_99
#> 1   116.72
#> 2 549.3859
#> 3  478.725
#> 4  358.695
#> 5 129.7466
#> 6  1961.85
#> 7   70.403
</code></pre>
<h3>Distributions along a time axis</h3>
<p><code>xray::timebased</code> also investigates into your distributions, but shows you the change over time, so if there is any change in the distribution over time (For example a variable stops or starts being collected) you can easily visualize it.</p>
<pre><code>dateLongley=longley  
dateLongley$Year=as.Date(paste0(dateLongley$Year,'-01-01'))  
dateLongley$Data='Original'  
ndateLongley=dateLongley  
ndateLongley$GNP=dateLongley$GNP+10  
ndateLongley$Data='Offseted'  
xray::timebased(rbind(dateLongley, ndateLongley), 'Year')  
</code></pre>
<p><img alt="xray: The R Package to Have X Ray Vision on your Datasets" src="https://i2.wp.com/blog.datascienceheroes.com/content/images/2017/11/README-example-timebased-1.png?w=456&ssl=1" data-recalc-dims="1"><br>
<img alt="xray: The R Package to Have X Ray Vision on your Datasets" src="https://i2.wp.com/blog.datascienceheroes.com/content/images/2017/11/README-example-timebased-2.png?w=456&ssl=1" data-recalc-dims="1"><br>
#> [1] “7 charts have been generated.”</p>
<p>If you are just starting to learn about Data Science or want to explore further, i encourage you to check this cool book made by my buddy Pablo Casas: <a rel="nofollow" href="https://livebook.datascienceheroes.com/" target="_blank">The Data Science Live Book</a></p>
<p><img alt="xray: The R Package to Have X Ray Vision on your Datasets" src="https://i2.wp.com/blog.datascienceheroes.com/content/images/2017/11/data-science-live-book-cover-2.png?w=456&ssl=1" data-recalc-dims="1"></p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://blog.datascienceheroes.com/x-ray-vision-on-your-datasets/"> R - Data Science Heroes Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/QiIU6qe8iZQ" height="1"></a></div>