<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/hr-analytics-using-machine-learning-to-predict-employee-turnover/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.business-science.io/business/2017/09/18/hr_employee_attrition.html"> business-science.io - Articles</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><strong>Employee turnvover (attrition) is a major cost to an organization, and predicting turnover is at the forefront of needs of Human Resources (HR) in many organizations</strong>. Until now the mainstream approach has been to use logistic regression or survival curves to model employee attrition. However, with advancements in machine learning (ML), we can now get both better predictive performance and better explanations of what critical features are linked to employee attrition. In this post, we’ll use two cutting edge techniques. First, we’ll use the <strong><code>h2o</code> package’s new FREE automatic machine learning algorithm, <code>h2o.automl()</code></strong>, to develop a predictive model that is in the same ballpark as commercial products in terms of ML accuracy. Then we’ll use the <strong>new <code>lime</code> package that enables breakdown of complex, black-box machine learning models into variable importance plots</strong>. We can’t stress how excited we are to share this post because it’s a much needed step towards machine learning in business applications!!! Enjoy.</p>
<h2>Employee Attrition: A Major Problem</h2>
<p>Bill Gates was once quoted as saying,</p>
<blockquote>
<p>“You take away our top 20 employees and we [Microsoft] become a mediocre company”.</p>
</blockquote>
<p>His statement cuts to the core of a <strong>major problem: employee attrition</strong>. An organization is only as good as its employees, and these people are the true source of its competitive advantage.</p>
<p><strong>Organizations face huge costs resulting from employee turnover</strong>. Some costs are tangible such as training expenses and the time it takes from when an employee starts to when they become a productive member. However, the most important costs are intangible. Consider what’s lost when a productive employee quits: new product ideas, great project management, or customer relationships.</p>
<p>With advances in machine learning and data science, its possible to <strong>not only predict employee attrition but to understand the key variables that influence turnover</strong>. We’ll take a look at two cutting edge techniques:</p>
<ol>
<li>
<p><strong>Machine Learning with <code>h2o.automl()</code> from the <code>h2o</code> package</strong>: This function takes automated machine learning to the next level by testing a number of advanced algorithms such as random forests, ensemble methods, and deep learning along with more traditional algorithms such as logistic regression. The main takeaway is that <strong>we can now easily achieve predictive performance that is in the same ball park (and in some cases even better than) commercial algorithms and ML/AI software</strong>.</p>
</li>
<li>
<p><strong>Feature Importance with the <code>lime</code> package</strong>: The problem with advanced machine learning algorithms such as deep learning is that it’s near impossible to understand the algorithm because of its complexity. This has all changed with the <code>lime</code> package. The major advancement with <code>lime</code> is that, by recursively analyzing the models locally, it can extract feature importance that repeats globally. What this means to us is that <code>lime</code> has opened the door to understanding the ML models regardless of complexity. <strong>Now the best (and typically very complex) models can also be investigated and potentially understood as to what variables or features make the model tick</strong>.</p>
</li>
</ol>
<h2>Employee Attrition: Machine Learning Analysis</h2>
<p>With these new automated ML tools combined with tools to uncover critical variables, we now have <strong>capabilities for both extreme predictive accuracy and understandability</strong>, which was previously impossible! We’ll investigate an HR Analytic example of employee attrition that was evaluated by IBM Watson.</p>
<h4>IBM Watson (Where we got the data)</h4>
<p>The example comes from <a rel="nofollow" href="https://www.ibm.com/analytics/us/en/watson-data-platform/" target="_blank">IBM Watson Analytics</a> website. You can download the data and read the analysis here:</p>
<ul>
<li><a rel="nofollow" href="https://www.ibm.com/communities/analytics/watson-analytics-blog/hr-employee-attrition/" target="_blank">Get data used in this post here</a>.</li>
<li><a rel="nofollow" href="https://www.ibm.com/communities/analytics/watson-analytics-blog/watson-analytics-use-case-for-hr-retaining-valuable-employees/" target="_blank">Read IBM Watson Analytics article here</a>.</li>
</ul>
<p>To summarize, the article makes a usage case for IBM Watson as an automated ML platform. The article shows that using Watson, the analyst was able to detect features that led to increased probability of attrition.</p>
<h4>Automated Machine Learning (What we did with the data)</h4>
<p>In this example we’ll show how we can use the combination of <strong>H2O for developing a complex model with high predictive accuracy on unseen data</strong> and then how we can use <strong>LIME to understand important features related to employee attrition</strong>.</p>
<h2>Packages</h2>
<p>Load the following packages.</p>
<figure>
<pre><code data-lang="r"><span># Load the following packages
</span><span>library</span><span>(</span><span>tidyquant</span><span>)</span><span> </span><span># Loads tidyverse and several other pkgs </span><span>library</span><span>(</span><span>readxl</span><span>)</span><span> </span><span># Super simple excel reader
</span><span>library</span><span>(</span><span>h</span><span>2</span><span>o</span><span>)</span><span> </span><span># Professional grade ML pkg
</span><span>library</span><span>(</span><span>lime</span><span>)</span><span> </span><span>#</span><span> </span><span>Explain</span><span> </span><span>complex</span><span> </span><span>black</span><span>-</span><span>box</span><span> </span><span>ML</span><span> </span><span>models</span></code></pre>
</figure>
<h2>Data</h2>
<p>Download the <a rel="nofollow" href="https://www.ibm.com/communities/analytics/watson-analytics-blog/hr-employee-attrition/" target="_blank">data here</a>. You can load the data using <code>read_excel()</code>, pointing the path to your local file.</p>
<figure>
<pre><code data-lang="r"><span># Read excel data
</span><span>hr_data_raw</span><span> </span><span>&lt;-</span><span> </span><span>read_excel</span><span>(</span><span>path</span><span> </span><span>=</span><span> </span><span>&quot;data/WA_Fn-UseC_-HR-Employee-Attrition.xlsx&quot;</span><span>)</span></code></pre>
</figure>
<p>Let’s check out the raw data. It’s 1470 rows (observations) by 35 columns (features). The “Attrition” column is our target. We’ll use all other columns as features to our model.</p>
<figure>
<pre><code data-lang="r"><span># View first 10 rows
</span><span>hr_data_raw</span><span>[</span><span>1</span><span>:</span><span>10</span><span>,]</span><span> </span><span>%>%</span><span>
</span><span>knitr</span><span>::</span><span>kable</span><span>(</span><span>caption</span><span> </span><span>=</span><span> </span><span>&quot;First 10 rows&quot;</span><span>)</span></code></pre>
</figure>
<table>
<thead>
<tr>
<th>Age</th>
<th>Attrition</th>
<th>BusinessTravel</th>
<th>DailyRate</th>
<th>Department</th>
<th>DistanceFromHome</th>
<th>Education</th>
<th>EducationField</th>
<th>EmployeeCount</th>
<th>EmployeeNumber</th>
<th>EnvironmentSatisfaction</th>
<th>Gender</th>
<th>HourlyRate</th>
<th>JobInvolvement</th>
<th>JobLevel</th>
<th>JobRole</th>
<th>JobSatisfaction</th>
<th>MaritalStatus</th>
<th>MonthlyIncome</th>
<th>MonthlyRate</th>
<th>NumCompaniesWorked</th>
<th>Over18</th>
<th>OverTime</th>
<th>PercentSalaryHike</th>
<th>PerformanceRating</th>
<th>RelationshipSatisfaction</th>
<th>StandardHours</th>
<th>StockOptionLevel</th>
<th>TotalWorkingYears</th>
<th>TrainingTimesLastYear</th>
<th>WorkLifeBalance</th>
<th>YearsAtCompany</th>
<th>YearsInCurrentRole</th>
<th>YearsSinceLastPromotion</th>
<th>YearsWithCurrManager</th>
</tr>
</thead>
<tbody>
<tr>
<td>41</td>
<td>Yes</td>
<td>Travel_Rarely</td>
<td>1102</td>
<td>Sales</td>
<td>1</td>
<td>2</td>
<td>Life Sciences</td>
<td>1</td>
<td>1</td>
<td>2</td>
<td>Female</td>
<td>94</td>
<td>3</td>
<td>2</td>
<td>Sales Executive</td>
<td>4</td>
<td>Single</td>
<td>5993</td>
<td>19479</td>
<td>8</td>
<td>Y</td>
<td>Yes</td>
<td>11</td>
<td>3</td>
<td>1</td>
<td>80</td>
<td>0</td>
<td>8</td>
<td>0</td>
<td>1</td>
<td>6</td>
<td>4</td>
<td>0</td>
<td>5</td>
</tr>
<tr>
<td>49</td>
<td>No</td>
<td>Travel_Frequently</td>
<td>279</td>
<td>Research &amp; Development</td>
<td>8</td>
<td>1</td>
<td>Life Sciences</td>
<td>1</td>
<td>2</td>
<td>3</td>
<td>Male</td>
<td>61</td>
<td>2</td>
<td>2</td>
<td>Research Scientist</td>
<td>2</td>
<td>Married</td>
<td>5130</td>
<td>24907</td>
<td>1</td>
<td>Y</td>
<td>No</td>
<td>23</td>
<td>4</td>
<td>4</td>
<td>80</td>
<td>1</td>
<td>10</td>
<td>3</td>
<td>3</td>
<td>10</td>
<td>7</td>
<td>1</td>
<td>7</td>
</tr>
<tr>
<td>37</td>
<td>Yes</td>
<td>Travel_Rarely</td>
<td>1373</td>
<td>Research &amp; Development</td>
<td>2</td>
<td>2</td>
<td>Other</td>
<td>1</td>
<td>4</td>
<td>4</td>
<td>Male</td>
<td>92</td>
<td>2</td>
<td>1</td>
<td>Laboratory Technician</td>
<td>3</td>
<td>Single</td>
<td>2090</td>
<td>2396</td>
<td>6</td>
<td>Y</td>
<td>Yes</td>
<td>15</td>
<td>3</td>
<td>2</td>
<td>80</td>
<td>0</td>
<td>7</td>
<td>3</td>
<td>3</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<td>33</td>
<td>No</td>
<td>Travel_Frequently</td>
<td>1392</td>
<td>Research &amp; Development</td>
<td>3</td>
<td>4</td>
<td>Life Sciences</td>
<td>1</td>
<td>5</td>
<td>4</td>
<td>Female</td>
<td>56</td>
<td>3</td>
<td>1</td>
<td>Research Scientist</td>
<td>3</td>
<td>Married</td>
<td>2909</td>
<td>23159</td>
<td>1</td>
<td>Y</td>
<td>Yes</td>
<td>11</td>
<td>3</td>
<td>3</td>
<td>80</td>
<td>0</td>
<td>8</td>
<td>3</td>
<td>3</td>
<td>8</td>
<td>7</td>
<td>3</td>
<td>0</td>
</tr>
<tr>
<td>27</td>
<td>No</td>
<td>Travel_Rarely</td>
<td>591</td>
<td>Research &amp; Development</td>
<td>2</td>
<td>1</td>
<td>Medical</td>
<td>1</td>
<td>7</td>
<td>1</td>
<td>Male</td>
<td>40</td>
<td>3</td>
<td>1</td>
<td>Laboratory Technician</td>
<td>2</td>
<td>Married</td>
<td>3468</td>
<td>16632</td>
<td>9</td>
<td>Y</td>
<td>No</td>
<td>12</td>
<td>3</td>
<td>4</td>
<td>80</td>
<td>1</td>
<td>6</td>
<td>3</td>
<td>3</td>
<td>2</td>
<td>2</td>
<td>2</td>
<td>2</td>
</tr>
<tr>
<td>32</td>
<td>No</td>
<td>Travel_Frequently</td>
<td>1005</td>
<td>Research &amp; Development</td>
<td>2</td>
<td>2</td>
<td>Life Sciences</td>
<td>1</td>
<td>8</td>
<td>4</td>
<td>Male</td>
<td>79</td>
<td>3</td>
<td>1</td>
<td>Laboratory Technician</td>
<td>4</td>
<td>Single</td>
<td>3068</td>
<td>11864</td>
<td>0</td>
<td>Y</td>
<td>No</td>
<td>13</td>
<td>3</td>
<td>3</td>
<td>80</td>
<td>0</td>
<td>8</td>
<td>2</td>
<td>2</td>
<td>7</td>
<td>7</td>
<td>3</td>
<td>6</td>
</tr>
<tr>
<td>59</td>
<td>No</td>
<td>Travel_Rarely</td>
<td>1324</td>
<td>Research &amp; Development</td>
<td>3</td>
<td>3</td>
<td>Medical</td>
<td>1</td>
<td>10</td>
<td>3</td>
<td>Female</td>
<td>81</td>
<td>4</td>
<td>1</td>
<td>Laboratory Technician</td>
<td>1</td>
<td>Married</td>
<td>2670</td>
<td>9964</td>
<td>4</td>
<td>Y</td>
<td>Yes</td>
<td>20</td>
<td>4</td>
<td>1</td>
<td>80</td>
<td>3</td>
<td>12</td>
<td>3</td>
<td>2</td>
<td>1</td>
<td>0</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<td>30</td>
<td>No</td>
<td>Travel_Rarely</td>
<td>1358</td>
<td>Research &amp; Development</td>
<td>24</td>
<td>1</td>
<td>Life Sciences</td>
<td>1</td>
<td>11</td>
<td>4</td>
<td>Male</td>
<td>67</td>
<td>3</td>
<td>1</td>
<td>Laboratory Technician</td>
<td>3</td>
<td>Divorced</td>
<td>2693</td>
<td>13335</td>
<td>1</td>
<td>Y</td>
<td>No</td>
<td>22</td>
<td>4</td>
<td>2</td>
<td>80</td>
<td>1</td>
<td>1</td>
<td>2</td>
<td>3</td>
<td>1</td>
<td>0</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<td>38</td>
<td>No</td>
<td>Travel_Frequently</td>
<td>216</td>
<td>Research &amp; Development</td>
<td>23</td>
<td>3</td>
<td>Life Sciences</td>
<td>1</td>
<td>12</td>
<td>4</td>
<td>Male</td>
<td>44</td>
<td>2</td>
<td>3</td>
<td>Manufacturing Director</td>
<td>3</td>
<td>Single</td>
<td>9526</td>
<td>8787</td>
<td>0</td>
<td>Y</td>
<td>No</td>
<td>21</td>
<td>4</td>
<td>2</td>
<td>80</td>
<td>0</td>
<td>10</td>
<td>2</td>
<td>3</td>
<td>9</td>
<td>7</td>
<td>1</td>
<td>8</td>
</tr>
<tr>
<td>36</td>
<td>No</td>
<td>Travel_Rarely</td>
<td>1299</td>
<td>Research &amp; Development</td>
<td>27</td>
<td>3</td>
<td>Medical</td>
<td>1</td>
<td>13</td>
<td>3</td>
<td>Male</td>
<td>94</td>
<td>3</td>
<td>2</td>
<td>Healthcare Representative</td>
<td>3</td>
<td>Married</td>
<td>5237</td>
<td>16577</td>
<td>6</td>
<td>Y</td>
<td>No</td>
<td>13</td>
<td>3</td>
<td>2</td>
<td>80</td>
<td>2</td>
<td>17</td>
<td>3</td>
<td>2</td>
<td>7</td>
<td>7</td>
<td>7</td>
<td>7</td>
</tr>
</tbody>
</table>
<p>The only pre-processing we’ll do in this example is change all character data types to factors. This is needed for H2O. We could make a number of other numeric data that is actually categorical factors, but this tends to increase modeling time and can have little improvement on model performance.</p>
<figure>
<pre><code data-lang="r"><span>hr_data</span><span> </span><span>&lt;-</span><span> </span><span>hr_data_raw</span><span> </span><span>%>%</span><span>
</span><span>mutate_if</span><span>(</span><span>is.character</span><span>,</span><span> </span><span>as.factor</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>Attrition</span><span>,</span><span> </span><span>everything</span><span>())</span></code></pre>
</figure>
<p>Let’s take a <code>glimpse</code> at the processed dataset. We can see all of the columns. Note our target (“Attrition”) is the first column.</p>
<figure>
<pre><code data-lang="r"><span>glimpse</span><span>(</span><span>hr_data</span><span>)</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## Observations: 1,470
## Variables: 35
## $ Attrition                </code></pre></figure>