<div>
<p>I just found a fun food themed dataset that I’d never heard about and that I thought I’d share. It’s from a project called <a target="_blank" href="http://menus.nypl.org/"><em>What’s on the menu</em></a> where the New York Public Library has crowdsourced a digitization of their collection of historical restaurant menus. The collection stretches all the way back to the 19th century and well into the 1990’s, and on the home page it is stated that there are “1,332,271 dishes transcribed from 17,545 menus”. Here is one of those menus, from a turn of the (old) century Chinese-American restaurant: </p>
<p><img alt="" src="https://i2.wp.com/www.sumsar.net/images/posts/2016-09-21-whats-on-the-menu/oriental_restaurant_mann_fang_lowe.jpeg?w=456" data-recalc-dims="1"></p>
<p>The data is freely available in csv format (yay!) and here I ‘ll just show how to the get the data into R and I’ll use it to plot the popularity of some foods over time.</p>
<p></p>
<p>First we’re going to download the data, “unzip” csv files into a temporary directory, and read them into R. </p>
<pre><code><div><pre>library(tidyverse)
library(stringr)
library(curl)

<span># This url changes every month, check what's the latest at http://menus.nypl.org/data</span>
menu_data_url <span>&lt;-</span> <span>&quot;https://s3.amazonaws.com/menusdata.nypl.org/gzips/2016_09_16_07_00_30_data.tgz&quot;</span>
temp_dir <span>&lt;-</span> tempdir()
curl_download(menu_data_url, file.path(temp_dir, <span>&quot;menu_data.tgz&quot;</span>))
untar(file.path(temp_dir, <span>&quot;menu_data.tgz&quot;</span>), exdir <span>=</span> temp_dir)
dish <span>&lt;-</span> read_csv(file.path(temp_dir, <span>&quot;Dish.csv&quot;</span>))
menu <span>&lt;-</span> read_csv(file.path(temp_dir, <span>&quot;Menu.csv&quot;</span>))
menu_item <span>&lt;-</span> read_csv(file.path(temp_dir, <span>&quot;MenuItem.csv&quot;</span>))
menu_page <span>&lt;-</span> read_csv(file.path(temp_dir, <span>&quot;MenuPage.csv&quot;</span>))
</pre>
</div>
<p></p>
<p>The resulting tables together describe the contents of the menus, but in order to know which dish was on which menu we need to join together the four tables. While doing this we’re also going to remove some uninteresting columns and remove some records that were not coded correctly.</p>
<pre><code><div><pre>d <span>&lt;-</span> menu_item <span>%>%</span> select( id, menu_page_id, dish_id, price) <span>%>%</span>
  left_join(dish <span>%>%</span> select(id, name) <span>%>%</span> rename(dish_name <span>=</span> name),
            by <span>=</span> c(<span>&quot;dish_id&quot;</span> <span>=</span> <span>&quot;id&quot;</span>)) <span>%>%</span>
  left_join(menu_page <span>%>%</span> select(id, menu_id),
            by <span>=</span> c(<span>&quot;menu_page_id&quot;</span> <span>=</span> <span>&quot;id&quot;</span>)) <span>%>%</span>
  left_join(menu <span>%>%</span> select(id, date, place, location),
            by <span>=</span> c(<span>&quot;menu_id&quot;</span> <span>=</span> <span>&quot;id&quot;</span>)) <span>%>%</span>
  mutate(year <span>=</span> lubridate<span>::</span>year(date)) <span>%>%</span>
  filter(<span>!</span>is.na(year)) <span>%>%</span>
  filter(year <span>></span> <span>1800</span> <span>&amp;</span> year <span>&lt;=</span> <span>2016</span>) <span>%>%</span>
  select(year, location, menu_id, dish_name, price, place)
</pre>
</div>
<p></p>
<p>What we are left with in the <code>d</code> data frame is a table of what dishes were served, where they were served and when. Here is a sampler: </p>
<pre><code><div><pre>d[sample(<span>1:</span>nrow(d), <span>10</span>), ]
</pre>
</div>
<p></p>
<pre><code><div><pre><span># A tibble: 10 × 6</span>
    year                      location menu_id                         dish_name price
   <span>&lt;</span>dbl<span>></span>                         <span>&lt;</span>chr<span>></span>   <span>&lt;</span>int<span>></span>                             <span>&lt;</span>chr<span>></span> <span>&lt;</span>dbl<span>></span>
<span>1</span>   <span>1900</span>            Fifth Avenue Hotel   <span>25394</span>            Broiled Mutton Kidneys    <span>NA</span>
<span>2</span>   <span>1971</span>                 Tadlich Grill   <span>26670</span>                       Mixed Green  <span>0.85</span>
<span>3</span>   <span>1939</span>                Maison Prunier   <span>30325</span>                  Entrecote Minute    <span>NA</span>
<span>4</span>   <span>1914</span>          The Beekman Café Co.   <span>33898</span>                  Camembert cheese  <span>0.10</span>
<span>5</span>   <span>1900</span>         Carlton Hotel Company   <span>21865</span>                        Pork Chops  <span>0.15</span>
<span>6</span>   <span>1914</span> Gutmann<span>'</span><span>s Café and Restaurant 33982 Cold Boiled Ham with Potato Salad 0.40</span>
<span>7</span>   <span>1912</span>               Waldorf<span>-</span>Astoria   <span>34512</span>            Stuffed Figs and Dates  <span>0.30</span>
<span>8</span>   <span>1933</span>                   Hotel Astor   <span>31262</span>              Assorted Small Cakes  <span>0.25</span>
<span>9</span>   <span>1933</span>              Ambassador Grill   <span>31291</span>                    Stuffed celery  <span>0.55</span>
<span>10</span>  <span>1901</span>            Del Coronado Hotel   <span>14512</span>                           peaches    <span>NA</span>
<span># ... with 1 more variables: place &lt;chr></span>
</pre>
</div>
<p></p>
<p>Personally I’d go for the <em>Stuffed Figs and Dates</em> at the Waldorf-Astoria followed by some <em>Assorted Small Cakes</em> 21 years later at the Astor. If you want to download this slightly processed version of the dataset <a target="_blank" href="http://www.sumsar.net/files/posts/2016-09-21-whats-on-the-menu/whats_on_the_menu_2016.csv.zip">it’s available here in csv format</a>. We can also see which are the most common menu items in the dataset:</p>
<pre><code><div><pre>d <span>%>%</span> count(tolower(dish_name)) <span>%>%</span> arrange(desc(n)) <span>%>%</span> head(<span>10</span>)
</pre>
</div>
<p></p>
<pre><code><div><pre><span># A tibble: 10 × 2</span>
   <span>`tolower(dish_name)`</span>     n
                  <span>&lt;</span>chr<span>></span> <span>&lt;</span>int<span>></span>
<span>1</span>                coffee  <span>8532</span>
<span>2</span>                celery  <span>4865</span>
<span>3</span>                olives  <span>4737</span>
<span>4</span>                   tea  <span>4682</span>
<span>5</span>              radishes  <span>3426</span>
<span>6</span>       mashed potatoes  <span>2999</span>
<span>7</span>       boiled potatoes  <span>2502</span>
<span>8</span>     vanilla ice cream  <span>2379</span>
<span>9</span>         chicken salad  <span>2306</span>
<span>10</span>                 milk  <span>2218</span>
</pre>
</div>
<p></p>
<p>That coffee is king isn’t that surprising, but the popularity of celery seems weird. My current hypothesis is that “celery” often refers to some kind of celery salad, or maybe it was common as a snack in the New York area in the 1900s. It should be remembered that the dataset does not represent what people ate in general, but is based on what menus were collected by the New York public library (presumably from the New York area). Also the bulk of the menus are from between 1900 and 1980:</p>
<pre><code><div><pre>ggplot(d, aes(year)) <span>+</span>
  geom_histogram(binwidth <span>=</span> <span>5</span>, center <span>=</span> <span>1902.5</span>, color <span>=</span> <span>&quot;black&quot;</span>, fill <span>=</span> <span>&quot;lightblue&quot;</span>) <span>+</span>
  scale_y_continuous(<span>&quot;N.o. menu items&quot;</span>)
</pre>
</div>
<p></p>
<p><img alt="" src="https://i1.wp.com/www.sumsar.net/figures/2016-09-21-whats-on-the-menu/unnamed-chunk-7-1.png?w=456" data-recalc-dims="1"></p>
<p>Even though it’s not completely clear what the dataset represents we could still have a look at some food trends over time. Below I’m going to go through a couple of common foodstuffs and, for each decennium, calculate what proportion of menus includes that foodstuff.</p>
<pre><code><div><pre>d<span>$</span>decennium <span>=</span> floor(d<span>$</span>year <span>/</span> <span>10</span>) <span>*</span> <span>10</span>
foods <span>&lt;-</span> c(<span>&quot;coffee&quot;</span>, <span>&quot;tea&quot;</span>, <span>&quot;pancake&quot;</span>, <span>&quot;ice cream&quot;</span>, <span>&quot;french frie&quot;</span>,
           <span>&quot;french peas&quot;</span>, <span>&quot;apple&quot;</span>, <span>&quot;banana&quot;</span>, <span>&quot;strawberry&quot;</span>)
<span># Above I dropped the &quot;d&quot; in French fries in order </span>
<span># to also match &quot;French fried potatoes.&quot;</span>
food_over_time <span>&lt;-</span> map_df(foods, <span>function</span>(food) {
  d <span>%>%</span>
    filter(year <span>>=</span> <span>1900</span> <span>&amp;</span> year <span>&lt;=</span> <span>1980</span>) <span>%>%</span>
    group_by(decennium, menu_id) <span>%>%</span>
    summarise(contains_food <span>=</span>
      any(str_detect(dish_name, regex(food, ignore_case <span>=</span> <span>TRUE</span>)),
          na.rm <span>=</span> <span>TRUE</span>)) <span>%>%</span>
    summarise(prop_food <span>=</span> mean(contains_food, na.rm <span>=</span> <span>TRUE</span>)) <span>%>%</span>
    mutate(food <span>=</span> food)
})
</pre>
</div>
<p></p>
<p>First up, Coffee vs. Tea:</p>
<pre><code><div><pre><span># A reusable list of ggplot2 directives to produce a lineplot</span>
food_time_plot <span>&lt;-</span> list(
  geom_line(),
  geom_point(),
  scale_y_continuous(<span>&quot;% of menus include&quot;</span>,labels <span>=</span> scales<span>::</span>percent,
                     limits <span>=</span> c(<span>0</span>, <span>NA</span>)),
  scale_x_continuous(<span>&quot;&quot;</span>),
  facet_wrap(<span>~</span> food),
  theme_minimal(),
  theme(legend.position <span>=</span> <span>&quot;none&quot;</span>))

food_over_time <span>%>%</span> filter(food <span>%in%</span> c(<span>&quot;coffee&quot;</span>, <span>&quot;tea&quot;</span>)) <span>%>%</span>
  ggplot(aes(decennium, prop_food, color <span>=</span> food)) <span>+</span> food_time_plot
</pre>
</div>
<p></p>
<p><img alt="" src="https://i0.wp.com/www.sumsar.net/figures/2016-09-21-whats-on-the-menu/unnamed-chunk-9-1.png?w=456" data-recalc-dims="1"></p>
<p>Both pretty popular menu items, but I’m not sure what to make of the trends… Next up Ice cream vs. Pancakes:</p>
<pre><code><div><pre>food_over_time <span>%>%</span> filter(food <span>%in%</span> c(<span>&quot;pancake&quot;</span>, <span>&quot;ice cream&quot;</span>)) <span>%>%</span>
  ggplot(aes(decennium, prop_food, color <span>=</span> food)) <span>+</span> food_time_plot
</pre>
</div>
<p></p>
<p><img alt="" src="https://i2.wp.com/www.sumsar.net/figures/2016-09-21-whats-on-the-menu/unnamed-chunk-10-1.png?w=456" data-recalc-dims="1"></p>
<p>Ice cream wins, but again I’m not sure what to make of how ice cream varies over time. Maybe it’s just an artifact of how the data was collected or maybe it actually reflects the icegeist somehow. What about French fries vs. French peas:</p>
<pre><code><div><pre>food_over_time <span>%>%</span> filter(food <span>%in%</span> c(<span>&quot;french frie&quot;</span>, <span>&quot;french peas&quot;</span>)) <span>%>%</span>
  ggplot(aes(decennium, prop_food, color <span>=</span> food)) <span>+</span> food_time_plot
</pre>
</div>
<p></p>
<p><img alt="" src="https://i2.wp.com/www.sumsar.net/figures/2016-09-21-whats-on-the-menu/unnamed-chunk-11-1.png?w=456" data-recalc-dims="1"></p>
<p>Seems like the heyday of French peas are over, but French fries also seemed to peak in the 40s… Finally let’s look at some fruit:</p>
<pre><code><div><pre>food_over_time <span>%>%</span> filter(food <span>%in%</span> c(<span>&quot;apple&quot;</span>, <span>&quot;banana&quot;</span>, <span>&quot;strawberry&quot;</span>)) <span>%>%</span>
  ggplot(aes(decennium, prop_food, color <span>=</span> food)) <span>+</span> food_time_plot
</pre>
</div>
<p></p>
<p><img alt="" src="https://i0.wp.com/www.sumsar.net/figures/2016-09-21-whats-on-the-menu/unnamed-chunk-12-1.png?w=456" data-recalc-dims="1"></p>
<p>Banana has really dropped in menu popularity since the early 1900s…</p>
<p>Anyway, this is a really cool dataset and I barely scratched the surface of what could be done with it. If you decide to explore this dataset further, and you make some plots and/or analyses, do send me a link and I will link to it here. </p>
<p>To finish off let’s look at this elegant cocktail menu from 1937 which, among cocktails and fizzes, advertises tiny cocktail tamales:</p>
<p><img alt="" src="https://i1.wp.com/www.sumsar.net/images/posts/2016-09-21-whats-on-the-menu/tortola_1937.jpeg?w=456" data-recalc-dims="1"></p>
<p></p>

<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/a-fun-gastronomical-dataset-whats-on-the-menu/"></div></div><div><a data-url="https://www.r-bloggers.com/a-fun-gastronomical-dataset-whats-on-the-menu/" data-counturl="https://www.r-bloggers.com/a-fun-gastronomical-dataset-whats-on-the-menu/" data-count="vertical" data-text="A Fun Gastronomical Dataset: What’s on the Menu?" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>

<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.sumsar.net/blog/2016/09/whats-on-the-menu/"> Publishable Stuff</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/Yffl0hGROcM"></a></div></code></pre></code></pre></code></pre></code></pre></code></pre></code></pre></code></pre></code></pre></code></pre></code></pre></code></pre></code></pre></div>