<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/cleaning-up-and-combining-data-a-dataset-for-practice/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://rmhogervorst.nl/cleancode/blog/2018/03/12/a-dirty-dataset-for-you-to-clean.html"> Clean Code</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>tldr: I created an open dataset for the explicit practice of data munging.<br>
Feel free to use it in assignments,<br>
but do mention where you got it from (CC-by-4.0). Also unicorns are awesome.</p>
<p>Find the dataset at: https://github.com/RMHogervorst/unicorns_on_unicycles</p>
<h2>Data munging / cleaning / engineering</h2>
<p>At work I was working with a two excel files that were slightly different but<br>
could be combined into 1 dataset. This is very typical for day to day cleaning<br>
operations that analysts and data scientists do (statisticians too).</p>
<p>I realized cleaning, joining and enriching is something that statistics classes<br>
just take for granted. But if a student only works with perfectly prepared<br>
data, they are unable to work with real world data. Because the real world is<br>
someone handing you an excel file with weird values and beautiful colors, that<br>
you cannot use in your work. Or it is a webscraping exercise where some of the<br>
pages are missing and people can’t seem to spell right. Some people say 80% of<br>
a data scientists work is cleaning data, so let us teach students how to do that<br>
effectively. I made this dataset in R, but it does not really matter what tool<br>
you use to read and clean this set.</p>
<h3>What is the dataset about?</h3>
<p>All good datasets have a story, this dataset is thought to have been recorded by<br>
an amateur scientist, a natural philosopher by the name of Rudolphus<br>
in the 17th century in The Netherlands. This scientist<br>
recorded the annual population of unicorns in western Europe over a century and also recorded<br>
the sales of unicycles in that same time period. Although not much of the<br>
accompanying text remains of<br>
the original documents, what we can read is the tables and the idea that<br>
Rudolphus thought there was some sort of relationship between uncorns and<br>
unicycle sales.</p>
<h3>what kind of tasks would a student have to do?</h3>
<p>There are 2 files, that contain a total of 3 tables. The tables can be<br>
connected to each other with full joins on countryname and year (To make it<br>
more difficult, you could remove one row in one of the sets).</p>
<p>a student should:</p>
<ul>
<li>read in the data from excel</li>
<li>recognize that one file contains 2 tables</li>
<li>realize that countrynames are slightly different in 1 file</li>
<li>join the files together</li>
<li>create 1 ‘long’ tidy datafile (country, year, value1, value2, value3)</li>
</ul>
<p>During analysis:</p>
<ul>
<li>realize that some years are missing</li>
<li>perhaps impute, reason for reasons of missingness (disease epidemic in<br>
Austria, others are missing at random)</li>
<li>find some relation between unicycles and unicorns in the 17th century in<br>
western Europe</li>
</ul>
<h2>what can I do with this set?</h2>
<p>I’ve released the set under the creative commons cc-by-4.0 license. That means<br>
you may remix, share, use commercially, and modify the dataset. I just want<br>
you to mention where you got the dataset from.<br>
If you think the units are stupid, change it. if you think the assignment is<br>
too difficult, make it easier. Really, you can use it any way you want.</p>
<p>I hope this set will help in teaching students the art of data munging. I also<br>
thought it was very funny to have unicorns on unicycles, but searching the<br>
internet shows I was hardly the first one to think of that pun. Ah well, just<br>
goes to show that it was a good one.</p>
<p>Find the dataset at: https://github.com/RMHogervorst/unicorns_on_unicycles</p>
<p>Happy coding!</p>
<p><a rel="nofollow" href="http://rmhogervorst.nl/cleancode/blog/2018/03/12/a-dirty-dataset-for-you-to-clean.html" target="_blank">Cleaning up and combining data, a dataset for practice</a> was originally published by at <a rel="nofollow" href="http://rmhogervorst.nl/cleancode" target="_blank">Clean Code</a> on March 12, 2018.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://rmhogervorst.nl/cleancode/blog/2018/03/12/a-dirty-dataset-for-you-to-clean.html"> Clean Code</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/kypkz2pDqkg" height="1"></a></div>