<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/getting-started-with-dplyr-in-r-using-titanic-dataset/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://datascienceplus.com/getting-started-with-dplyr-in-r-using-titanic-dataset/"> R Programming – DataScience+</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p><code>dplyr</code> is one of the most popular r-packages and also part of <code>tidyverse</code> that’s been developed by Hadley Wickham. The mere fact that <code>dplyr</code> package is very famous means, it’s one of the most frequently used. Being a data scientist is not always about creating sophisticated models but Data Analysis (Manipulation) and Data Visualization play a very important role in BAU of many us – in fact, a very important part before any modeling exercise since Feature Engineering and EDA are the most important differentiating factors of your model and someone else’s.<br>
Hence, this post aims to bring out some well-known and not-so-well-known applications of <code>dplyr</code> so that any data analyst could leverage its potential using a much familiar – <a rel="nofollow" href="https://www.kaggle.com/c/titanic/data" target="_blank">Titanic Dataset</a>. </p>
<p><code>dplyr</code> library can be installed <a rel="nofollow" href="https://cran.r-project.org/web/packages/dplyr/index.html" target="_blank">directly from CRAN</a> and loaded into R session like any other R package.</p>
<pre>
#install.packages('dplyr')
library(dplyr) # Loading Dplyr package
</pre>
<p>Let us start by reading the input training file using the base r function <code>read.csv</code></p>
<pre>
train &lt;- read.csv('../input/train.csv',stringsAsFactors = F, header = T)
</pre>
<p>Getting the total number of rows in the given data frame (even though it’s been very straightforward with <code>nrow()</code> in base-r, this being a <code>dplyr</code> starter-kit, we’ll start with that.</p>
<pre>train%>% count()
<em>n
891
</em>
</pre>
<p>The above code just gives the row count of the data frame that’s been passed with the pipe %>% operator. The pipe operator works very similar to the | (pipe) operator in Unix environment where the output of the current operation is fed as the input of the following operation. Similarly, in <code>dplyr</code> or any other package that supports pipe operator, the functions in it will always take only data frame as the first argument hence the function can be called in two ways like below:</p>
<pre>count(train) #Without pipe, passing the df as the first argument
train %>% count() #with pipe, more convenient and more readability
<em>n
891
n
891</em>
</pre>
<p>But dplyr’s real flavor starts with the following 5 functions (or as most people call, verbs of dplyr):</p>
<ul>
<li>select()</li>
<li>filter() </li>
<li>arrange() </li>
<li>mutate() </li>
<li>summarise() </li>
<li>group_by() </li>
</ul>
<p>And let us see what every one of these does!</p>
<h2>select</h2>
<p><code>select()</code> as the name suggests selects the columns that are required from a given dataframe and if multiple columns are required or not required, then <code>one_of()</code> could be used within select.</p>
<pre>select(train,Age) #without pipe
<em>Age
22
38
26
35
35
NA
54
2
27
14
4
</em>
</pre>
<pre>#multicolumn selection
train %>% select(one_of('Sex','Age'))
<em>Sex	Age
male	22
female	38
female	26
female	35
male	35
male	NA
male	54
male	2
female	27
female	14
female	4
</em>
</pre>
<pre>#multicolumn rejection
train %>% select(-one_of('Age','Sex'))
<em>PassengerId	Survived	Pclass	Name	SibSp	Parch	Ticket	Fare	Cabin	Embarked
1	0	3	Braund, Mr. Owen Harris	1	0	A/5 21171	7.2500	S
2	1	1	Cumings, Mrs. John Bradley (Florence Briggs Thayer)	1	0	PC 17599	71.2833	C85	C
3	1	3	Heikkinen, Miss. Laina	0	0	STON/O2. 3101282	7.9250	S
4	1	1	Futrelle, Mrs. Jacques Heath (Lily May Peel)	1	0	113803	53.1000	C123	S
5	0	3	Allen, Mr. William Henry	0	0	373450	8.0500	S
6	0	3	Moran, Mr. James	0	0	330877	8.4583	Q
7	0	1	McCarthy, Mr. Timothy J	0	0	17463	51.8625	E46	S
8	0	3	Palsson, Master. Gosta Leonard	3	1	349909	21.0750	S
9	1	3	Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)	0	2	347742	11.1333	S
10	1	2	Nasser, Mrs. Nicholas (Adele Achem)	1	0	237736	30.0708	C
</em>
</pre>
<p>Like selecting a column with entire column name (or multiple column names with one_of()), select could also be used with a few more string ops.</p>
<pre>train %>% select(starts_with('P'))
<em>PassengerId	Pclass	Parch
1	3	0
2	1	0
3	3	0
4	1	0
5	3	0
6	3	0
7	1	0
8	3	1
9	3	2
10	2	0
</em>
</pre>
<pre>train %>% select(ends_with('e'))
<em>Name	Age	Fare
Braund, Mr. Owen Harris	22	7.2500
Cumings, Mrs. John Bradley (Florence Briggs Thayer)	38	71.2833
Heikkinen, Miss. Laina	26	7.9250
Futrelle, Mrs. Jacques Heath (Lily May Peel)	35	53.1000
Allen, Mr. William Henry	35	8.0500
Moran, Mr. James	NA	8.4583
McCarthy, Mr. Timothy J	54	51.8625
Palsson, Master. Gosta Leonard	2	21.0750
Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)	27	11.1333
Nasser, Mrs. Nicholas (Adele Achem)	14	30.0708
Sandstrom, Miss. Marguerite Rut	4	16.7000
</em>
</pre>
<h2>group_by</h2>
<p><code>group_by</code> is a lot similar to SQL Group by but more versatile. It is related to concept of “split-apply-combine”. Let us understand group_by with a starter example of finding out number of male and number of female – which logically could be the count of each Sex Type (once grouped by Sex).</p>
<pre>train %>% group_by(Sex) %>% count()
<em>Sex	n
female	314
male	577
</em>
</pre>
<p>Aha! That seems simple and now let us do a two level grouping to understand how many of survived of each gender.</p>
<pre>train %>% group_by(Survived, Sex) %>% count()
train %>% group_by(Sex, Survived) %>% count()
<em>Survived	Sex	n
0	female	81
0	male	468
1	female	233
1	male	109
Sex	Survived	n
female	0	81
female	1	233
male	0	468
male	1	109
</em>
</pre>
<h2>mutate and summarise</h2>
<p>That’s minimally <code>group_by</code>, but the true power of group_by is unveiled only when it is coupled with <code>mutate</code> and <code>summarise</code> functions.<br>
Mutate function adds a new column based on the given expression while summarise function summarises the dataset based on the given function and let us see the difference in action with the following example.</p>
<p>Let us get the average age of all survivors (and non-survivors): so this must be group_by -ed based on Survived while summarised by Age so that we will get a summarised mean value.for two groups.</p>
<pre>train %>% group_by(Survived) %>% summarise(mean(Age))
#Remember we have got NAs, so mean() wouldn't work and to bypass NAs, na.rm = T must be passed. 
train %>% group_by(Survived) %>% summarise(average_age = mean(Age,na.rm=T))
<em>Survived	mean(Age)
0	NA
1	NA
Survived	average_age
0	30.62618
1	28.34369
</em>
</pre>
<p>That’s <code>summarise()</code> giving us the summary of the dataframe. If we need to create a new column, values filled for all 891 datapoints, that’s where mutate plays its role. Let us create a new column, <code>Age_Bracket</code> containing value <code>Minor</code> if Age is less than 18 else <code>Major</code></p>
<pre>train %>% mutate(Age_Bracket = ifelse(Age &lt; 18, 'Minor','Major')) %>% select(starts_with('Age'))
#In fact this can be coupled with Survivor list to see the impact of this Age_bracket
train %>% 
mutate(Age_Bracket = ifelse(Age &lt; 18, 'Minor','Major')) %>% 
group_by(Survived,Age_Bracket) %>% 
summarise(pnt = (n()/nrow(train))*100)
<em>Age	Age_Bracket
22	Major
38	Major
26	Major
35	Major
35	Major
NA	NA
54	Major
2	Minor
27	Major
14	Minor
Survived	Age_Bracket	pnt
0	Major	41.750842
0	Minor	5.836139
0	NA	14.029181
1	Major	25.701459
1	Minor	6.846240
1	NA	5.836139
</em>
</pre>
<p>That’s how <code>dplyr</code> can get more powerful with group_by coupled with <code>mutate</code> or summarise for feautre engineering and for better data visualization. But this doesn’t stop here, because one of the most important function a dataanalyst would require is sorting and that’s what <code>arrange()</code> does.</p>
<h2>arrange</h2>
<p>Extracting last 4 results after sorting the fare in asending order:</p>
<pre>train %>% arrange(Fare) %>%  tail(4) 
<em>	PassengerId	Survived	Pclass	Name	Sex	Age	SibSp	Parch	Ticket	Fare	Cabin	Embarked
870	319	1	1	Wick, Miss. Mary Natalie	female	31	0	2	36928	164.8667	C7	S
871	857	1	1	Wick, Mrs. George Dennick (Mary Hitchcock)	female	45	1	1	36928	164.8667	S
872	690	1	1	Madill, Miss. Georgette Alexandra	female	15	0	1	24160	211.3375	B5	S
873	731	1	1	Allen, Miss. Elisabeth Walton	female	29	0	0	24160	211.3375	B5	S
874	780	1	1	Robert, Mrs. Edward Scott (Elisabeth Walton McMillan)	female	43	0	1	24160	211.3375	B3	S
</em>
</pre>
<p>Arrange in descending order:</p>
<pre>
train %>% arrange(desc(Age)) %>% head(5)
<em>PassengerId	Survived	Pclass	Name	Sex	Age	SibSp	Parch	Ticket	Fare	Cabin	Embarked
631	1	1	Barkworth, Mr. Algernon Henry Wilson	male	80.0	0	0	27042	30.0000	A23	S
852	0	3	Svensson, Mr. Johan	male	74.0	0	0	347060	7.7750	S
97	0	1	Goldschmidt, Mr. George B	male	71.0	0	0	PC 17754	34.6542	A5	C
494	0	1	Artagaveytia, Mr. Ramon	male	71.0	0	0	PC 17609	49.5042	C
117	0	3	Connors, Mr. Patrick	male	70.5	0	0	370369	7.7500	Q
</em>
</pre>
<h2>filter</h2>
<p><code>filter</code> does <code>row_wise</code> filter ( similar to what select did with columns). <code>filter()</code> takes a logical expression and evaluates them and results the only_true datapoints. So to be clear, all that matters to filter() function is if the expression evaluates to TRUE.<br>
Let us start with filtering (extracting) only male and getting their Embarked station count.</p>
<pre>train %>% 
filter(Sex == 'male') %>%
group_by(Embarked) %>%
count()
<em>Embarked	n
C	95
Q	41
S	441
</em>
</pre>
<pre>#Getting the count of everyone whose age is lesser than 18
train %>% filter(Age &lt; 18) %>% count()
<em>n
113
</em>
</pre>
<pre>train %>% filter(grepl('wick',train$Name))
<em>PassengerId	Survived	Pclass	Name	Sex	Age	SibSp	Parch	Ticket	Fare	Cabin	Embarked
344	0	2	Sedgwick, Mr. Charles Frederick Waddington	male	25	0	0	244361	13	S
</em>
</pre>
<p>And this is dplyr in a nut shell and hope you get a decent start with this article, if you are a beginner. Please share your thoughts and suggestions in comments!. The <a rel="nofollow" href="https://nbviewer.jupyter.org/github/amrrs/datascienceplus_codes/blob/master/dplyr%20in%20R%20using%20Titanic%20Dataset.ipynb" target="_blank">notebook</a> used here is available on <a rel="nofollow" href="https://github.com/amrrs/datascienceplus_codes/blob/master/dplyr%20in%20R%20using%20Titanic%20Dataset.ipynb" target="_blank">my github</a>. </p>
<h2>References</h2>
<ul>
<a rel="nofollow" href="https://www.kaggle.com/c/titanic/data" target="_blank">Titanic Dataset from Kaggle</a><br>
<a rel="nofollow" href="https://www.kaggle.com/nulldata/dplyr-starter-kit-with-titanic-dataset/" target="_blank">Kaggle Kernel of the above Notebook</a><br>
<a rel="nofollow" href="https://github.com/amrrs/datascienceplus_codes/blob/master/dplyr%20in%20R%20using%20Titanic%20Dataset.ipynb" target="_blank">Github Code</a><br>
<a rel="nofollow" href="https://nbviewer.jupyter.org/github/amrrs/datascienceplus_codes/blob/master/dplyr%20in%20R%20using%20Titanic%20Dataset.ipynb" target="_blank">Notebook Viewer</a></ul>
<ul></ul>
<p><strong>
<p>Related Post</p>
<p></p></strong>
<ol>
<li><a rel="nofollow" href="https://datascienceplus.com/how-to-apply-monte-carlo-simulation-to-forecast-stock-prices-using-python/" title="Permanent Link toHow to apply Monte Carlo simulation to forecast Stock prices using Python" target="_blank">How to apply Monte Carlo simulation to forecast Stock prices using Python</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/analysing-ios-app-store-itunes-reviews-in-r/" title="Permanent Link toAnalysing iOS App Store iTunes Reviews in R" target="_blank">Analysing iOS App Store iTunes Reviews in R</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/handling-happy-vs-not-happy-better-sentiment-analysis-with-sentimentr-in-r/" title="Permanent Link toHandling ‘Happy’ vs ‘Not Happy’: Better sentiment analysis with sentimentr in R" target="_blank">Handling ‘Happy’ vs ‘Not Happy’: Better sentiment analysis with sentimentr in R</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/creating-reporting-template-with-glue-in-r/" title="Permanent Link toCreating Reporting Template with Glue in R" target="_blank">Creating Reporting Template with Glue in R</a></li>
<li><a rel="nofollow" href="https://datascienceplus.com/predict-employee-turnover-with-python/" title="Permanent Link toPredict Employee Turnover With Python" target="_blank">Predict Employee Turnover With Python</a></li>
</ol>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://datascienceplus.com/getting-started-with-dplyr-in-r-using-titanic-dataset/"> R Programming – DataScience+</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/kOHSR_POUv4" height="1"></a></div>