<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/r-code-to-accompany-real-world-machine-learning-chapter-6-exploring-nyc-taxi-data/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://feedproxy.google.com/~r/DataProne-R/~3/Oen6IUvob5c/rwml-R-chapter-6-nyc-taxi-eda.html"> data prone - R</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2>Abstract</h2>
<p>The rwml-R Github repo is updated with R code for exploratory data analysis of New York City taxi data from Chapter 6 of the book “Real-World Machine Learning” by Henrik Brink, Joseph W. Richards, and Mark Fetherolf. Examples given include reading large data files with the <code>fread</code> function from <code>data.table</code>, joining data frames by multiple variables with <code>inner_join</code>, and plotting categorical and numerical data with <code>ggplot2</code>.</p>
<h2>Data for NYC taxi example</h2>
<p>The data files for the examples in Chapter 6 of the book are available at<br>
<a target="_blank" rel="nofollow" href="http://www.andresmh.com/nyctaxitrips/">http://www.andresmh.com/nyctaxitrips/</a>.<br>
They are compressed as a 7-Zip file archive<br>
(e.g. with <a target="_blank" rel="nofollow" href="http://p7zip.sourceforge.net/">p7zip</a>), so you will<br>
need to have the <code>7z</code> command available in your path to decompress and load<br>
the data.<br>
(On a mac, you can use <a target="_blank" rel="nofollow" href="https://brew.sh/">Homebrew</a> to install p7zip with<br>
the command <code>brew install p7zip</code>.)</p>
<h2>Using fread (and dplyr…again)</h2>
<p>As in Chapter 5, the <code>fread</code> function from the<br>
<code>data.table</code> library is used to quickly read in a sample of the rather large<br>
data files. It is similar to <code>read.table</code> but faster and more convenient.<br>
The following code reads in the first 50k lines of data from one of the<br>
trip data files and one of the fare data files. The <code>mutate</code> and <code>filter</code><br>
functions from <code>dplyr</code> are used to clean up the data (e.g. remove data<br>
with unrealistic latitude and longitude values). The trip and fare data are<br>
combined with the <code>inner_join</code> function from the <code>dplyr</code> package.</p>
<figure>
<pre><code data-lang="r"><span>tripFile1</span><span> </span><span>&lt;-</span><span> </span><span>&quot;../data/trip_data_1.csv&quot;</span><span>
</span><span>fareFile1</span><span> </span><span>&lt;-</span><span> </span><span>&quot;../data/trip_fare_1.csv&quot;</span><span>
</span><span>npoints</span><span> </span><span>&lt;-</span><span> </span><span>50000</span><span>
</span><span>tripData</span><span> </span><span>&lt;-</span><span> </span><span>fread</span><span>(</span><span>tripFile1</span><span>,</span><span> </span><span>nrows</span><span>=</span><span>npoints</span><span>,</span><span> </span><span>stringsAsFactors</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>store_and_fwd_flag</span><span> </span><span>=</span><span> </span><span>replace</span><span>(</span><span>store_and_fwd_flag</span><span>,</span><span> </span><span>which</span><span>(</span><span>store_and_fwd_flag</span><span> </span><span>==</span><span> </span><span>&quot;&quot;</span><span>),</span><span> </span><span>&quot;N&quot;</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>trip_distance</span><span> </span><span>></span><span> </span><span>0</span><span> </span><span>&amp;</span><span> </span><span>trip_time_in_secs</span><span> </span><span>></span><span> </span><span>0</span><span> </span><span>&amp;</span><span> </span><span>passenger_count</span><span> </span><span>></span><span> </span><span>0</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>pickup_longitude</span><span> </span><span>&lt;</span><span> </span><span>-70</span><span> </span><span>&amp;</span><span> </span><span>pickup_longitude</span><span> </span><span>></span><span> </span><span>-80</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>pickup_latitude</span><span> </span><span>></span><span> </span><span>0</span><span> </span><span>&amp;</span><span> </span><span>pickup_latitude</span><span> </span><span>&lt;</span><span> </span><span>41</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>dropoff_longitude</span><span> </span><span>&lt;</span><span> </span><span>0</span><span> </span><span>&amp;</span><span> </span><span>dropoff_latitude</span><span> </span><span>></span><span> </span><span>0</span><span>)</span><span>
</span><span>tripData</span><span>$</span><span>store_and_fwd_flag</span><span> </span><span>&lt;-</span><span> </span><span>factor</span><span>(</span><span>tripData</span><span>$</span><span>store_and_fwd_flag</span><span>)</span><span>
</span><span>fareData</span><span> </span><span>&lt;-</span><span> </span><span>fread</span><span>(</span><span>fareFile1</span><span>,</span><span> </span><span>nrows</span><span>=</span><span>npoints</span><span>,</span><span> </span><span>stringsAsFactors</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span>
</span><span>dataJoined</span><span> </span><span>&lt;-</span><span> </span><span>inner_join</span><span>(</span><span>tripData</span><span>,</span><span> </span><span>fareData</span><span>)</span><span>
</span><span>remove</span><span>(</span><span>fareData</span><span>,</span><span> </span><span>tripData</span><span>)</span></code></pre>
</figure>
<h2>Exploring the data</h2>
<p>In <a target="_blank" rel="nofollow" href="https://padamson.github.io/rwml-R/Chapter6.html">the complete code-through</a>, plots of categorical and numerical<br>
features of the data are made using<br>
<code>ggplot2</code>, including a visualization of the pickup locations in latitude and<br>
longitude space which is shown below. With slightly less than 50,000 data<br>
points, we can clearly see the street layout of downtown Manhatten.<br>
Many of the trips originate in the other boroughs of New York, too.</p>
<p><img alt="The latitude/longitude of pickup locations. Note that the x-axis is flipped, compared to a regular map." src="https://i2.wp.com/padamson.github.io/assets/figure6.4-1.png?w=456" data-recalc-dims="1"></p>
<h2>Feedback welcome</h2>
<p>If you have any feedback on the <a target="_blank" rel="nofollow" href="https://github.com/padamson/rwml-R">rwml-R</a> project, please<br>
leave a comment below or use the Tweet button.<br>
As with any of my projects, feel free to <a target="_blank" rel="nofollow" href="https://github.com/padamson/rwml-R/fork">fork the rwml-R repo</a><br>
and submit a pull request if you wish to contribute.<br>
For convenience, I’ve created a <a target="_blank" rel="nofollow" href="https://padamson.github.io/rwml-R/">project page for rwml-R</a> with<br>
the generated HTML files from <code>knitr</code>, including a page with<br>
<a target="_blank" rel="nofollow" href="https://padamson.github.io/rwml-R/Chapter6.html">all of the event-modeling examples from chapter 6</a>.</p>
<p><a data-style="mega" target="_blank" rel="nofollow" href="https://github.com/padamson/rwml-R/archive/master.zip" data-icon="octicon-cloud-download">Download</a><br>
<a data-style="mega" data-count-href="http://feedproxy.google.com/padamson/rwml-R/network" data-count-aria-label="# forks on GitHub" target="_blank" data-count-api="/repos/padamson/rwml-R#forks_count" rel="nofollow" href="https://github.com/padamson/rwml-R/fork" data-icon="octicon-repo-forked">Fork</a></p>
<p><img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/DataProne-R/~4/Oen6IUvob5c"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://feedproxy.google.com/~r/DataProne-R/~3/Oen6IUvob5c/rwml-R-chapter-6-nyc-taxi-eda.html"> data prone - R</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/QTBxnx7rEM4"></a></div>