<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/comparing-2017-maine-lobster-landings-to-historical-landings/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://rud.is/b/2018/03/02/comparing-2017-maine-lobster-landings-to-historical-landings/"> R – rud.is</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Tis the season for finding out how well Maine fisherfolk did last year; specifically, Maine lobsterfolk.</p>
<p>Most of the news sites in Maine do a feature on the annual landings (here’s one from <a rel="nofollow" href="https://bangordailynews.com/2018/03/02/fisheries/maines-annual-lobster-landings-drop-by-nearly-100m-in-value-largest-decline-ever/" target="_blank">Bangor Daily News</a>). There was a marked decline — the largest ever — in both poundage and revenue in 2017 and many sources point to the need to improve fishery management to help ensure both the environmental and economic health of the state.</p>
<p>My preferred view for this annual catch comparison is a connected scatterplot, tracing a path along the years. That way you get the feel of a time-series with the actual poundage-to-value without having to resort to two charts or (heaven forbid) a dual-geom/dual-axis chart. </p>
<p>The State of Maine Department of Marine Resources makes the <a rel="nofollow" href="https://www.maine.gov/dmr/commercial-fishing/landings/historical-data.html" target="_blank">data available</a> but it’s in a PDF:</p>
<p><a rel="nofollow" href="https://i2.wp.com/rud.is/b/wp-content/uploads/2018/03/Screen-Shot-2018-03-02-at-6.17.32-PM.png?ssl=1" target="_blank"><img data-orig-file="https://i2.wp.com/rud.is/b/wp-content/uploads/2018/03/Screen-Shot-2018-03-02-at-6.17.32-PM.png?fit=1442%2C1164&amp;ssl=1" data-orig-size="1442,1164" data-medium-file="https://i2.wp.com/rud.is/b/wp-content/uploads/2018/03/Screen-Shot-2018-03-02-at-6.17.32-PM.png?fit=300%2C242&amp;ssl=1" data-attachment-id="8356" data-image-meta="{" data-image-description="" src="https://i2.wp.com/rud.is/b/wp-content/uploads/2018/03/Screen-Shot-2018-03-02-at-6.17.32-PM.png?resize=780%2C630&ssl=1" data-permalink="https://rud.is/b/2018/03/02/comparing-2017-maine-lobster-landings-to-historical-landings/screen-shot-2018-03-02-at-6-17-32-pm/" data-large-file="https://i2.wp.com/rud.is/b/wp-content/uploads/2018/03/Screen-Shot-2018-03-02-at-6.17.32-PM.png?fit=530%2C428&amp;ssl=1" alt="" data-image-title="Screen Shot 2018-03-02 at 6.17.32 PM" width="450" data-comments-opened="1" class="wp-image-8356" data-recalc-dims="1"></a></p>
<p>Thankfully, the PDF is not obfuscated and is just a plain table so it’s easy to parse and turn into:</p>
<p><a rel="nofollow" href="https://i1.wp.com/rud.is/b/wp-content/uploads/2018/03/2017-lobster-landings.png?ssl=1" target="_blank"><img data-orig-file="https://i1.wp.com/rud.is/b/wp-content/uploads/2018/03/2017-lobster-landings.png?fit=2082%2C1652&amp;ssl=1" data-orig-size="2082,1652" data-medium-file="https://i1.wp.com/rud.is/b/wp-content/uploads/2018/03/2017-lobster-landings.png?fit=300%2C238&amp;ssl=1" data-attachment-id="8357" data-image-meta="{" data-image-description="" src="https://i1.wp.com/rud.is/b/wp-content/uploads/2018/03/2017-lobster-landings.png?resize=780%2C619&ssl=1" data-permalink="https://rud.is/b/2018/03/02/comparing-2017-maine-lobster-landings-to-historical-landings/2017-lobster-landings/" data-large-file="https://i1.wp.com/rud.is/b/wp-content/uploads/2018/03/2017-lobster-landings.png?fit=530%2C421&amp;ssl=1" alt="" data-image-title="2017-lobster-landings" width="450" data-comments-opened="1" class="wp-image-8357" data-recalc-dims="1"></a></p>
<p>The code to retrieve the PDF, parse it and produce said connected scatterplot is below.</p>
<pre><code>library(stringi)
library(pdftools)
library(hrbrthemes)
library(tidyverse)

lobster_by_county_url &lt;- &quot;https://www.maine.gov/dmr/commercial-fishing/landings/documents/lobster.county.pdf&quot;
lobster_by_county_fil &lt;- basename(lobster_by_county_url)

if (!file.exists(lobster_by_county_fil)) download.file(lobster_by_county_url, lobster_by_county_fil)

# read in the PDF
lobster_by_county_pgs &lt;- pdftools::pdf_text(lobster_by_county_fil)

map(lobster_by_county_pgs, stri_split_lines) %>% # split each page into lines
  flatten() %>%
  flatten_chr() %>%
  keep(stri_detect_fixed, &quot;$&quot;) %>% # keep only lines with &quot;$&quot; in them
  stri_trim_both() %>% # clean up white space
  stri_split_regex(&quot;\ +&quot;, simplify = TRUE) %>% # get the columns
  as_data_frame() %>%
  mutate_at(c(&quot;V3&quot;, &quot;V4&quot;), lucr::from_currency) %>% # turn the formatted text into numbers
  set_names(c(&quot;year&quot;, &quot;county&quot;, &quot;pounds&quot;, &quot;value&quot;)) %>% # better column names
  filter(county != &quot;TOTAL&quot;) %>% # we'll calculate our own, thank you
  mutate(year = as.Date(sprintf(&quot;%s-01-01&quot;, year))) %>% # I like years to be years for plotting
  mutate(county = stri_trans_totitle(county)) -> lobster_by_county_df

arrange(lobster_by_county_df, year) %>%
  mutate(value = value / 1000000, pounds = pounds / 1000000) %>% # easier on the eyes
  group_by(year) %>%
  summarise(pounds = sum(pounds), value = sum(value)) %>%
  mutate(year_lab = lubridate::year(year)) %>%
  mutate(highlight = ifelse(year_lab == 2017, &quot;2017&quot;, &quot;Other&quot;)) %>% # so we can highlight 2017
  ggplot(aes(pounds, value)) +
  geom_path() +
  geom_label(
    aes(label = year_lab, color = highlight, size = highlight),
    family = font_ps, show.legend = FALSE
  ) +
  scale_x_comma(name = &quot;Pounds (millions) →&quot;, limits = c(0, 150)) +
  scale_y_comma(name = &quot;$ USD (millions) →&quot;, limits = c(0, 600)) +
  scale_color_manual(values = c(&quot;2017&quot; = &quot;#742111&quot;, &quot;Other&quot; = &quot;#2b2b2b&quot;)) +
  scale_size_manual(values = c(&quot;2017&quot; = 6, &quot;Other&quot; = 4)) +
  labs(
    title = &quot;Historical Maine Fisheries Landings Data — Lobster (1964-2017)&quot;,
    subtitle = &quot;All counties combined; Not adjusted for inflation&quot;,
    caption = &quot;The 2002 &amp; 2003 landings may possibly reflect the increased effort by DMR to collect voluntary landings from some lobster dealers;\nLobster reporting became mandatory in 2004 for all Maine dealers buying directly from harvesters.\nSource: </code></pre>