<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/data-science-competitions-101-anatomy-and-approach/"></div></div><div><a data-url="http://www.r-bloggers.com/data-science-competitions-101-anatomy-and-approach/" data-counturl="http://www.r-bloggers.com/data-science-competitions-101-anatomy-and-approach/" data-count="horizontal" data-text="Data Science Competitions 101: Anatomy and Approach" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://techandmortals.wordpress.com/2016/07/27/data-science-competitions-101-anatomy-and-approach/"> R – Tech and Mortals</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I recently participated in a weekend-long data science hackathon, titled ‘The Smart Recruits’. Organized by the amazing folks at <a target="_blank" href="https://analyticsvidhya.com/">Analytics Vidhya</a>, it saw some serious competition. Although my performance can be classified as decent at best (47 out of 379 participants), it was among the more satisfying ones I have participated in on both AV <a target="_blank" href="http://datahack.analyticsvidhya.com/user/profile/data_vinci">(profile)</a> and Kaggle <a target="_blank" href="http://kaggle.com/akashtandon">(profile)</a> over the last few months. Thus, I decided it might be worthwhile to try and share some insights as a data science autodidact.</p>
<p><img alt="TheSmartRecruits_AnalyticsVidhya" class="wp-image-959" src="http://i2.wp.com/techandmortals.files.wordpress.com/2016/07/thesmartrecruits_analyticsvidhya.jpg?w=456&ssl=1" data-recalc-dims="1"></p>
<h4>The problem</h4>
<p>The competition required us to use historical data to create a model to help an organization pick out better recruits. The evaluation metric to be used for judging the predictions was <a target="_blank" href="https://www.kaggle.com/wiki/AreaUnderCurve">AUC</a> (area under the ROC curve). You can read the problem statement on the competition <a target="_blank" href="https://analyticsvidhya.com/">page</a>.</p>
<h4>The Data</h4>
<p>The hackathon itself was a weekend-long sprint and the data reflected this. The training set (1.2 MB) comprised of 9527 observations consisting of 23 variables including the target variable. The test set (~600 KB) comprised of 5045 observations and 22 predictor variables.</p>
<h4>The Science behind it all</h4>
<h5>Code</h5>
<p>My code for the competition can be found <a target="_blank" href="http://github.com/analyticalmonk/AnalyticsVidhya_SmartRecruits">here</a>. The code is in <a target="_blank" href="https://www.r-project.org/about.html">R</a> although the description of my approach towards the problem in this post is presented in a language-agnostic manner.<br>
As for the choice of language, I used R since I am relatively more proficient with it as compared to Python, my other language of choice for data-intensive work. And that was an important factor considering the sprint nature of the competition. Also, I find R more suited than Python for data manipulation and visualization. I prefer Python when dabbling in deep learning or working with image and textual data. Then again, it’s just my opinion.</p>
<p><em>Recommended read: <a target="_blank" href="http://www.kdnuggets.com/2015/05/r-vs-python-data-science.html">R vs Python for Data Science: The Winner is …</a></em></p>
<h5>Exploration</h5>
<p>I spent some time exploring the data and the nature of variables through statistical summaries and visualizations. Getting to know your data is extremely important and just throwing it in a model straightway without doing that is a really bad approach! That can’t be stressed enough.<br>
That being said, I didn’t spend as much time on this part as one normally would when participating in a month-long Kaggle competition or when working with real-life data. It’s a sprint after all.</p>
<p><em>Recommended read: <a target="_blank" href="http://www.analyticsvidhya.com/blog/2016/01/guide-data-exploration/">A Comprehensive Guide to Data Exploration</a></em></p>
<h5>Cross-validation</h5>
<p>If I was forced to choose just one significant learning I have acquired from participating in such competitions, it has to be the importance of cross-validation.<br>
So the next thing I did after playing around with data for a while was to try and set up a good CV framework. There are all sorts of complicated frameworks that are used by experienced campaigners; peer into an active Kaggle forum and you will know.<br>
But <a target="_blank" href="https://en.wikipedia.org/wiki/Cross-validation_(statistics)#k-fold_cross-validation">k-fold cross validation</a>, which I used is, in general, a good enough start for most competitions. The decision regarding how to perform the split is critical. Random splits might be good enough at times. Other times the classes (0/1) are unbalanced so you might need to do <a target="_blank" href="http://stats.stackexchange.com/questions/117643/why-use-stratified-cross-validation-why-does-this-not-damage-variance-related-b">stratified sampling</a> or sometimes time-based splits (month, quarter, year, etc.) will have to be made.</p>
<p><em>Recommended read: <a target="_blank" href="http://www.analyticsvidhya.com/blog/2015/11/improve-model-performance-cross-validation-in-python-r/">Improve Your Model Performance using Cross Validation</a></em></p>
<h5>Feature engineering</h5>
<p>As a beginner, I used to stumble across this term quite a lot and was unable to straightaway find some good resource. Over time, I have learnt what it represents and why it’s an indispensable part of any machine learning problem. A quote from this gem of an article, <a target="_blank" href="http://machinelearningmastery.com/discover-feature-engineering-how-to-engineer-features-and-how-to-get-good-at-it/">Discover Feature Engineering</a>, sums it up nicely:</p>
<blockquote>
<p>Feature engineering is the process of transforming raw data into features that better represent the underlying problem to the predictive models, resulting in improved model accuracy on unseen data.</p>
</blockquote>
<p>Bulk of my time was spent on this part. Briefly, I encoded the categorical variables (such as Gender, Occupation, Qualification), performed <a target="_blank" href="https://en.wikipedia.org/wiki/Imputation_(statistics)">imputation</a> to deal with missing/NA values , created new variables and removed some variables.<br>
<em>The encoding part</em> is fairly straightforward; a glance through the code and one can understand the process.<br>
<em>Dealing with missing values</em> is tricky for beginners and experienced campaigners alike. There are various methodologies one might employ. You can do away with incomplete observations altogether. That isn’t a good idea and in a case like ours when we don’t have a lot of data points, it’s a very bad approach. Other usual approach is to use the mean/median/mode of observations for numeric variables. That is what I have used. There are also sophisticated algorithms for imputation. <a target="_blank" href="http://www.r-bloggers.com/imputing-missing-data-with-r-mice-package/">MICE</a> had given me good results in the past but after trying it out, I chose not to implement it in the final model.<br>
<em>Creating new useful variables </em>was key in this competition. I managed to come up with some useful new features. For example, I split Applicant_DOB into three separate columns for date, month and year and further used the year column to create Applicant_Age.</p>
<p><em>Recommended read: <a target="_blank" href="http://machinelearningmastery.com/discover-feature-engineering-how-to-engineer-features-and-how-to-get-good-at-it/">Discover Feature Engineering, How to Engineer Features and How to Get Good at It</a></em></p>
<h5>The Algorithm</h5>
<p>The final model employs <a target="_blank" href="https://github.com/dmlc/xgboost">XGBoost</a>, the most popular algorithm in data science competitions. I also tried out a couple of models from the H2O package but went with XGBoost eventually since it provided better performance both locally and on the public leaderboard.</p>
<h5>Hyperparameter tuning</h5>
<p>Simply put, hyperparameters can be thought of as cogs which can be turned to fine-tune the machine that is your algorithm. In case of XGBoost, nrounds (number of iterations performed during training) and max_depth (maximum depth of a tree created during training) are examples of hyperparameters.<br>
There are automated methods such as <a target="_blank" href="http://scikit-learn.org/stable/modules/grid_search.html">GridSearch</a> and <a target="_blank" href="http://scikit-learn.org/stable/modules/generated/sklearn.grid_search.RandomizedSearchCV.html">RandomizedSearch</a> which can be used. I used a manual approach, as described <a target="_blank" href="https://www.kaggle.com/c/bnp-paribas-cardif-claims-management/forums/t/19083/best-practices-for-parameter-tuning-on-models/108783#post108783">here</a> by Davut Polat, a Kaggle master.</p>
<p><em>Recommended read: <a target="_blank" href="http://blog.turi.com/how-to-evaluate-machine-learning-models-part-4-hyperparameter-tuning">How to Evaluate Machine Learning Models: Hyperparameter Tuning</a></em></p>
<h5>The grind</h5>
<p>Solving a problem such as this takes time and perseverance, among other things. Analytics Vidhya’s hackathons have progressively become better in terms of the quality of competition on offer. Having Kaggle grandmasters and masters (including the eventual winner, <a target="_blank" href="https://www.kaggle.com/rohanrao">Rohan Rao</a>) in the discussion forums, and the competition itself, helped.<br>
It’s fun so if you are looking to get your hands dirty in a data science competition, don’t think too much. Dive right in. Even if it seems overwhelming at first, you will only end up having fun and learn something along the way.</p>
<p><strong>The code is available as a <a target="_blank" href="https://github.com/analyticalmonk/AnalyticsVidhya_SmartRecruits/tree/master">Github repo</a>.<br>
If you read and liked the article, sharing it would be a good next step.<br>
Drop me a <a target="_blank" href="mailto:akashtndn.acm@gmail.com">mail</a>, or hit me up on <a target="_blank" href="http://twitter.com/AkashTandon">Twitter</a> or <a target="_blank" href="https://www.quora.com/profile/Akash-Tandon-1">Quora</a> in case you want to get in touch.<br>
</strong></p>
<p> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/techandmortals.wordpress.com/701/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/techandmortals.wordpress.com/701/" border="0"></a> <img alt="" src="http://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/data-science-competitions-101-anatomy-and-approach/"></div></div><div><a data-url="http://www.r-bloggers.com/data-science-competitions-101-anatomy-and-approach/" data-counturl="http://www.r-bloggers.com/data-science-competitions-101-anatomy-and-approach/" data-count="vertical" data-text="Data Science Competitions 101: Anatomy and Approach" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://techandmortals.wordpress.com/2016/07/27/data-science-competitions-101-anatomy-and-approach/"> R – Tech and Mortals</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/CvnOd-5Zdhg"></a></div>