<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/extracting-tables-from-pdfs-in-r-using-the-tabulizer-package/"></div></div><div><a data-url="https://www.r-bloggers.com/extracting-tables-from-pdfs-in-r-using-the-tabulizer-package/" data-counturl="https://www.r-bloggers.com/extracting-tables-from-pdfs-in-r-using-the-tabulizer-package/" data-count="horizontal" data-text="Extracting Tables from PDFs in R using the Tabulizer Package" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://datascienceplus.com/extracting-tables-from-pdfs-in-r-using-the-tabulizer-package/"> DataScience+</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Recently I wanted to extract a table from a pdf file so that I could work with the table in R. Specifically, I wanted to get data on layoffs in California from the <a target="_blank" rel="nofollow" href="http://www.edd.ca.gov/">California Employment Development Department</a>. The EDD publishes a list of all of the layoffs in the state that fall under the WARN act <a target="_blank" rel="nofollow" href="http://www.edd.ca.gov/jobs_and_training/Layoff_Services_WARN.htm">here</a>. Unfortunately, the tables are available only in pdf format. I wanted an interactive version of the data that I could work with in R and export to a csv file. Fortunately, the tabulizer package in R makes this a cinch. In this post, I will use this scenario as a working example to show how to extract data from a pdf file using the tabulizer package in R.</p>
<p>The link to the pdf gets updated often, so here I’ve provided the pdf (link is below) as downloaded from the site on November 29, 2016:</p>
<p>Preview of the PDF (link is below):<br>
<a title="" data-rel="lightbox-0" target="_blank" rel="nofollow" href="https://i0.wp.com/datascienceplus.com/wp-content/uploads/2016/12/pdf-datascreenshot.png"><img sizes="(max-width: 490px) 100vw, 490px" alt="pdf-datascreenshot" class="wp-image-4923" src="https://i1.wp.com/datascienceplus.com/wp-content/uploads/2016/12/pdf-datascreenshot-490x303.png?resize=456%2C282" data-recalc-dims="1"></a></p>
<p>First, we will need to load the tabulizer package as well as dplyr.</p>
<pre>library(tabulizer)
library(dplyr)</pre>
<p>Next we will use the <code>extract_tables()</code> function from <code>tabulizer</code>. First, I specify the url of the pdf file from which I want to extract a table. This pdf link includes the most recent data, covering the period from July 1, 2016 to November 25, 2016. I am using the default parameters for <code>extract_tables</code>. These are guess and method. I’ll leave guess set to TRUE, which tells tabulizer that we want it to figure out the locations of the tables on its own. We could set this to <code>FALSE</code> if we want to have more granular control, but for this application we don’t need to. We leave the method argument set to “matrix”, which will return a list of matrices (one for each pdf page). This could also be set to return data frames instead.</p>
<pre># Location of WARN notice pdf file
location &lt;- 'http://www.edd.ca.gov/jobs_and_training/warn/WARN-Report-for-7-1-2016-to-10-25-2016.pdf'

# Extract the table
out &lt;- extract_tables(location)</pre>
<p>Now we have a list object called out, with each element a matrix representation of a page of the pdf table. We want to combine these into a single data matrix containing all of the data. We can do so most elegantly by combining <code>do.call</code> and <code>rbind</code>, passing it our list of matrices. Notice that I am excluding the last page here. The final page is the totals and summary information. We don’t need that.</p>
<pre>final &lt;- do.call(rbind, out[-length(out)])</pre>
<p>After doing so, the first three rows of the matrix contain the headers, which have not been formatted well since they take up multiple rows of the pdf table. Let’s fix that. Here I turn the matrix into a <code>data.frame</code> dropping the first three rows. Then I create a character vector containing the formatted headers and use that as the column names.</p>
<pre># table headers get extracted as rows with bad formatting. Dump them.
final &lt;- as.data.frame(final[3:nrow(final), ])

# Column names
headers &lt;- c('Notice.Date', 'Effective.Date', 'Received.Date', 'Company', 'City', 
             'No.of.Employees', 'Layoff/Closure')

# Apply custom column names
names(final) &lt;- headers</pre>
<p>We now have a data.frame of all of the California layoffs. A quick glance at the first few rows:</p>
<pre>head(final)<em>
Notice.Date Effective.Date Received.Date Company City No.of.Employees Layoff/Closure
1 2016-06-30 2016-08-31 2016-07-01 DCS Facility Services Antelope 55 Closure Permanent
2 2016-06-30 2016-08-31 2016-07-01 DCS Facility Services Bakersfield 22 Closure Permanent
3 2016-06-30 2016-08-31 2016-07-01 DCS Facility Services Brentwood 22 Closure Permanent
4 2016-06-30 2016-08-31 2016-07-01 DCS Facility Services Chico 22 Closure Permanent
5 2016-06-30 2016-08-31 2016-07-01 DCS Facility Services Clovis 33 Closure Permanent
6 2016-06-30 2016-08-31 2016-07-01 DCS Facility Services Fontana 33 Closure Permanent</em></pre>
<p>In order to manipulate the data properly , we will probably want to change the date column to a Date object as well as convert the No.of.Employees column to numeric. Here I do so using <code>dplyr</code>.</p>
<pre># These dplyr steps are not strictly necessary for dumping to <code>csv</code>, but useful if further data 
# manipulation in R is required. 
final &lt;- final %>%
    # Convert date columns to date objects
    mutate_each(funs(as.Date(., format='%m/%d/%Y')), Notice.Date, Effective.Date, Received.Date) %>%
    # Convert No.of.Employees to numeric
    mutate(No.of.Employees = as.numeric(levels(No.of.Employees)[No.of.Employees]))</pre>
<p>Last of all, I finish up by writing the final table to csv so that I can load it for later use.</p>
<pre># Write final table to disk
write.csv(final, file='CA_WARN.csv', row.names=FALSE)</pre>
<p>I have found the tabulizer package to be wonderfully easy to use. Much of the process of extracting the data and tables from pdfs is abstracted away from the user. This was a very simple example, however if one requires more finely-tuned control of how tables are extracted, the <code>extract_tables</code> function has a lot of additional arguments to tweak to one’s liking. I encourage you to take a look for yourself.</p>
<p>You can find the code for this post on my <a target="_blank" rel="nofollow" href="https://github.com/capt-calculator/CA_WARN_data">Github</a>.</p>
<ul></ul>
<p><strong>
<p>Related Post</p>
<p>
<ol>
<li><a title="Permanent Link toExtract Twitter Data Automatically using Scheduler R package" target="_blank" rel="nofollow" href="http://datascienceplus.com/extract-twitter-data-automatically-using-scheduler-r-package/">Extract Twitter Data Automatically using Scheduler R package</a></li>
<li><a title="Permanent Link toAn Introduction to Time Series with JSON Data" target="_blank" rel="nofollow" href="http://datascienceplus.com/introduction-to-time-series-with-json-data/">An Introduction to Time Series with JSON Data</a></li>
<li><a title="Permanent Link toGet Your Data into R: Import Data from SPSS, Stata, SAS, CSV or TXT" target="_blank" rel="nofollow" href="http://datascienceplus.com/get-your-data-into-r/">Get Your Data into R: Import Data from SPSS, Stata, SAS, CSV or TXT</a></li>
<li><a title="Permanent Link toExporting Data from R to TXT, CSV, SPSS or Stata" target="_blank" rel="nofollow" href="http://datascienceplus.com/exporting-data-from-r/">Exporting Data from R to TXT, CSV, SPSS or Stata</a></li>
<li><a title="Permanent Link toFirst Things to Do After You Import the Data into R" target="_blank" rel="nofollow" href="http://datascienceplus.com/first-things-to-do-after-you-import-the-data-into-r/">First Things to Do After You Import the Data into R</a></li>
</ol></strong>
<div>
<a target="_blank" rel="nofollow" href="http://feeds.feedburner.com/~ff/datascienceplus?a=rfOlmUlPRW8:XuRpobLtL_g:D7DqB2pKExk"><img src="http://feeds.feedburner.com/~ff/datascienceplus?i=rfOlmUlPRW8:XuRpobLtL_g:D7DqB2pKExk" border="0"></a> <a target="_blank" rel="nofollow" href="http://feeds.feedburner.com/~ff/datascienceplus?a=rfOlmUlPRW8:XuRpobLtL_g:qj6IDK7rITs"><img src="http://feeds.feedburner.com/~ff/datascienceplus?d=qj6IDK7rITs" border="0"></a> <a target="_blank" rel="nofollow" href="http://feeds.feedburner.com/~ff/datascienceplus?a=rfOlmUlPRW8:XuRpobLtL_g:yIl2AUoC8zA"><img src="http://feeds.feedburner.com/~ff/datascienceplus?d=yIl2AUoC8zA" border="0"></a>
</div>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/extracting-tables-from-pdfs-in-r-using-the-tabulizer-package/"></div></div><div><a data-url="https://www.r-bloggers.com/extracting-tables-from-pdfs-in-r-using-the-tabulizer-package/" data-counturl="https://www.r-bloggers.com/extracting-tables-from-pdfs-in-r-using-the-tabulizer-package/" data-count="vertical" data-text="Extracting Tables from PDFs in R using the Tabulizer Package" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://datascienceplus.com/extracting-tables-from-pdfs-in-r-using-the-tabulizer-package/"> DataScience+</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/qeWbYUS0l-Y"></a></div>