<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/bad-stock-photos-of-my-job-data-science-on-pexels/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.masalmon.eu/2018/05/06/dsstock/"> Maëlle</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I couldn’t miss the fun Twitter hashtag <a rel="nofollow" href="https://twitter.com/hashtag/BadStockPhotosOfMyJob?src=hash" target="_blank">#BadStockPhotosOfMyJob</a> thanks to a <a rel="nofollow" href="https://twitter.com/juliasilge/status/992952211201130497" target="_blank">tweet by Julia Silge</a> and <a rel="nofollow" href="https://twitter.com/_ColinFay/status/993055416266436608" target="_blank">another one by Colin Fay</a>. The latter inspired me to actually go and look for what makes a data science photo… What characterizes “data science” stock photos?</p>
<p><span></span></p>
<h1>My (not bad) stock photo source</h1>
<h2>Pexels metadata for the win</h2>
<p>Where to find information related to stock photos? In <a rel="nofollow" href="http://www.masalmon.eu/2018/01/04/bubblegumpuppies/" target="_blank">two</a> <a rel="nofollow" href="http://www.masalmon.eu/2018/01/07/rainbowing/" target="_blank">previous</a> blog posts of mine I used Pexels, a website providing CC0 pictures which is quite nice. My goal was to obtain the titles and the tags of stock photos of “data science”: for instance if you look at <a rel="nofollow" href="https://www.pexels.com/photo/black-laptop-beside-black-computer-mouse-inside-room-669996/" target="_blank">this picture</a>, its tags are “business”, “contemporary”, “computer”, etc. Pexels tags are very useful metadata, saving me the effort to use machine learning methods to analyse images.</p>
<h2>Responsible webscraping</h2>
<p>When researching this post I discovered that Pexels has an API, documented <a rel="nofollow" href="https://www.pexels.com/api/documentation/" target="_blank">here</a> but this API does not get you the title nor the tags associated to a picture so only webscraping could get me what I needed.</p>
<p>Webscraping is a powerful tool allowing one to <a rel="nofollow" href="https://speakerdeck.com/jennybc/data-rectangling" target="_blank">rectangle</a> webpages but with great power comes great responsability. Being <em>able</em> to scrape a webpage does not mean you are <em>allowed</em> to. You could get sued or your IP could get blocked. I am far from being an expert but I often read <a rel="nofollow" href="https://rud.is/b/" target="_blank">Bob Rudis’ blog</a> where I learnt about rOpenSci’s <a rel="nofollow" href="https://github.com/ropenscilabs/robotstxt" target="_blank"><code>robotstxt</code> package</a> that does “robots.txt file parsing and checking for R” which in plain language means it checks for you what a webpage legally allows you to do. See below,</p>
<div>
<div>
<pre><code><span># how I'll find pictures</span><span>
</span><span>robotstxt</span><span>::</span><span>paths_allowed</span><span>(</span><span>&quot;https://www.pexels.com/search&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] TRUE
</code></pre>
</div>
</div>
<div>
<div>
<pre><code><span># where tags live</span><span>
</span><span>robotstxt</span><span>::</span><span>paths_allowed</span><span>(</span><span>&quot;https://www.pexels.com/photo&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## [1] TRUE
</code></pre>
</div>
</div>
<p>robots.txt files often also tell you how often you can hit a page by defining a “crawling delay”. Sadly Pexels robots.txt doesn’t:</p>
<div>
<div>
<pre><code><span>robotstxt</span><span>::</span><span>get_robotstxt</span><span>(</span><span>&quot;https://www.pexels.com&quot;</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<div>
<div>
<pre><code>## Sitemap: https://s3.amazonaws.com/pexels/sitemaps/sitemap.xml.gz
</code></pre>
</div>
</div>
<p>But Bob Rudis, who was patient and nice enough to answer my questions, told me that I should probably respect the rate limit defined in <a rel="nofollow" href="https://www.pexels.com/api/documentation/" target="_blank">Pexels API docs</a>. “Do not abuse the API. The API is rate-limited to 200 requests per hour and 20,000 requests per month.” As I recently explained in <a rel="nofollow" href="https://itsalocke.com/blog/some-web-api-package-development-lessons-from-hibpwned/" target="_blank">a post on Locke Data’s blog</a>, these days to limit rate of a function I use <a rel="nofollow" href="https://github.com/tarakc02/ratelimitr" target="_blank">the very handy <code>ratelimitr</code> package</a> by <a rel="nofollow" href="https://tarakc02.github.io/" target="_blank">Tarak Shah</a>.</p>
<div>
<div>
<pre><code><span>limited_get</span><span> </span><span>&lt;-</span><span> </span><span>ratelimitr</span><span>::</span><span>limit_rate</span><span>(</span><span>httr</span><span>::</span><span>GET</span><span>,</span><span>
</span><span>ratelimitr</span><span>::</span><span>rate</span><span>(</span><span>200</span><span>,</span><span> </span><span>60</span><span>*</span><span>60</span><span>),</span><span># not more than 200 times an hour</span><span>
</span><span>ratelimitr</span><span>::</span><span>rate</span><span>(</span><span>1</span><span>,</span><span> </span><span>5</span><span>))</span><span>#not more than 1 time every 5 seconds</span><span>
</span></code></pre>
</div>
</div>
<h2>Elegant webscraping</h2>
<p>At the time of the two aforelinked blog posts I had used <a rel="nofollow" href="https://github.com/ropensci/RSelenium" target="_blank"><code>RSelenium</code></a> to scroll down and get the download link of many pictures, but Bob Rudis wrote <a rel="nofollow" href="https://gist.github.com/hrbrmstr/4cabe4af87bd2c5fe664b0b44a574366" target="_blank">an elegant and cool alternative</a> using query parameters, on which I’ll build in this post.</p>
<p>I first re-wrote the function to get all 15 pictures of each page of results.</p>
<div>
<div>
<pre><code><span>get_page</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>num</span><span> </span><span>=</span><span> </span><span>1</span><span>,</span><span> </span><span>seed</span><span> </span><span>=</span><span> </span><span>1</span><span>)</span><span> </span><span>{</span><span>
</span><span>message</span><span>(</span><span>num</span><span>)</span><span>
</span><span>limited_get</span><span>(</span><span>
</span><span>url</span><span> </span><span>=</span><span> </span><span>&quot;https://www.pexels.com/search/data science/&quot;</span><span>,</span><span>
</span><span>query</span><span> </span><span>=</span><span> </span><span>list</span><span>(</span><span>
</span><span>page</span><span>=</span><span>num</span><span>,</span><span>
</span><span>seed</span><span>=</span><span>seed</span><span>
</span><span>)</span><span>
</span><span>)</span><span> </span><span>-></span><span> </span><span>res</span><span>
</span><span>httr</span><span>::</span><span>stop_for_status</span><span>(</span><span>res</span><span>)</span><span>
</span><span>pg</span><span> </span><span>&lt;-</span><span> </span><span>httr</span><span>::</span><span>content</span><span>(</span><span>res</span><span>)</span><span>
</span><span>tibble</span><span>::</span><span>tibble</span><span>(</span><span>
</span><span>url</span><span> </span><span>=</span><span> </span><span>rvest</span><span>::</span><span>html_attr</span><span>(</span><span>rvest</span><span>::</span><span>html_nodes</span><span>(</span><span>pg</span><span>,</span><span> </span><span>xpath</span><span> </span><span>=</span><span> </span><span>&quot;//a[@class='js-photo-link']&quot;</span><span>),</span><span> </span><span>&quot;href&quot;</span><span>),</span><span>
</span><span>title</span><span> </span><span>=</span><span> </span><span>rvest</span><span>::</span><span>html_attr</span><span>(</span><span>rvest</span><span>::</span><span>html_nodes</span><span>(</span><span>pg</span><span>,</span><span> </span><span>xpath</span><span> </span><span>=</span><span> </span><span>&quot;//a[@class='js-photo-link']&quot;</span><span>),</span><span> </span><span>&quot;title&quot;</span><span>),</span><span>
</span><span>tags</span><span> </span><span>=</span><span> </span><span>purrr</span><span>::</span><span>map</span><span>(</span><span>url</span><span>,</span><span> </span><span>get_tags</span><span>)</span><span>
</span><span>)</span><span>
</span><span>}</span><span> </span></code></pre>
</div>
</div>
<p>I re-wrote it because I needed the “href” and because it seems that the structure of each page changed a bit since the day on which the gist was written. To find out I had to write “a[@class=’js-photo-link’]” I inspected the source of a page.</p>
<p>Then I wrote a function getting tags for each picture.</p>
<div>
<div>
<pre><code><span>get_tags</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>url</span><span>){</span><span>
</span><span>message</span><span>(</span><span>url</span><span>)</span><span>
</span><span>url</span><span> </span><span>&lt;-</span><span> </span><span>paste0</span><span>(</span><span>&quot;https://www.pexels.com&quot;</span><span>,</span><span> </span><span>url</span><span>)</span><span>
</span><span>res</span><span> </span><span>&lt;-</span><span> </span><span>limited_get</span><span>(</span><span>url</span><span>)</span><span>
</span><span>httr</span><span>::</span><span>stop_for_status</span><span>(</span><span>res</span><span>)</span><span>
</span><span>pg</span><span> </span><span>&lt;-</span><span> </span><span>httr</span><span>::</span><span>content</span><span>(</span><span>res</span><span>)</span><span>
</span><span>nodes</span><span> </span><span>&lt;-</span><span> </span><span>rvest</span><span>::</span><span>html_nodes</span><span>(</span><span>pg</span><span>,</span><span> </span><span>xpath</span><span> </span><span>=</span><span> </span><span>'//a[@data-track-label=&quot;tag&quot; ]'</span><span>)</span><span>
</span><span>rvest</span><span>::</span><span>html_text</span><span>(</span><span>nodes</span><span>)</span><span>
</span><span>}</span><span>
</span></code></pre>
</div>
</div>
<p>And finally I got results for 20 pages. I chose 20 without thinking too much. It seemed enough for my needs, and each of these pages had pictures.</p>
<div>
<div>
<pre><code><span>ds_stock</span><span> </span><span>&lt;-</span><span> </span><span>purrr</span><span>::</span><span>map_df</span><span>(</span><span>1</span><span>:</span><span>20</span><span>,</span><span> </span><span>get_page</span><span>)</span><span>
</span><span>ds_stock</span><span> </span><span>&lt;-</span><span> </span><span>unique</span><span>(</span><span>ds_stock</span><span>)</span><span>
</span><span>ds_stock</span><span> </span><span>&lt;-</span><span> </span><span>tidyr</span><span>::</span><span>unnest</span><span>(</span><span>ds_stock</span><span>,</span><span> </span><span>tags</span><span>)</span><span>
</span></code></pre>
</div>
</div>
<p>I got 300 unique pictures.</p>
<h1>What’s in a data science stock photo?</h1>
<p>Now that I have all this information at hand, I can describe data science stock photos!</p>
<h2>Data science tags</h2>
<div>
<div>
<pre><code><span>library</span><span>(</span><span>&quot;ggplot2&quot;</span><span>)</span><span>
</span><span>library</span><span>(</span><span>&quot;ggalt&quot;</span><span>)</span><span>
</span><span>library</span><span>(</span><span>&quot;hrbrthemes&quot;</span><span>)</span><span>
</span><span>tag_counts</span><span> </span><span>&lt;-</span><span> </span><span>dplyr</span><span>::</span><span>count</span><span>(</span><span>ds_stock</span><span>,</span><span> </span><span>tags</span><span>,</span><span> </span><span>sort</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)[</span><span>1</span><span>:</span><span>10</span><span>,]</span><span>
</span><span>dplyr</span><span>::</span><span>mutate</span><span>(</span><span>tag_counts</span><span>,</span><span>
</span><span>tags</span><span> </span><span>=</span><span> </span><span>reorder</span><span>(</span><span>tags</span><span>,</span><span> </span><span>n</span><span>))</span><span> </span><span>%>%</span><span> </span><span>ggplot</span><span>()</span><span> </span><span>+</span><span>
</span><span>geom_lollipop</span><span>(</span><span>aes</span><span>(</span><span>tags</span><span>,</span><span> </span><span>n</span><span>),</span><span>
</span><span>size</span><span> </span><span>=</span><span> </span><span>2</span><span>,</span><span> </span><span>col</span><span> </span><span>=</span><span> </span><span>&quot;salmon&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>hrbrthemes</span><span>::</span><span>theme_ipsum</span><span>(</span><span>base_size</span><span> </span><span>=</span><span> </span><span>16</span><span>,</span><span>
</span><span>axis_title_size</span><span> </span><span>=</span><span> </span><span>16</span><span>)</span><span> </span><span>+</span><span>
</span><span>xlab</span><span>(</span><span>&quot;Tag&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>ggtitle</span><span>(</span><span>&quot;300 data science stock photos&quot;</span><span>,</span><span>
</span><span>subtitle</span><span> </span><span>=</span><span> </span><span>&quot;Most frequent tags. Source: https://www.pexels.com&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>coord_flip</span><span>()</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="plot of chunk unnamed-chunk-4" src="https://i0.wp.com/www.masalmon.eu/figure/source/2018-05-06-dsstock/unnamed-chunk-4-1.png?w=456" data-recalc-dims="1"></p>
<p>So the most common tags are data, technology, business and computer. Not too surprising!</p>
<h2>Data science scenes</h2>
<p>Now, let’s have a look at <em>titles</em> that are in general more descriptive of what’s happening/present on the photo (i.e. is the computer near a cup of coffee or is someone working on it). I tried using a technique described in <a rel="nofollow" href="https://juliasilge.com/" target="_blank">Julia Silge</a>’s and <a rel="nofollow" href="http://varianceexplained.org/" target="_blank">David Robinson</a>’s <a rel="nofollow" href="https://www.tidytextmining.com/" target="_blank">Tidy text mining book</a>: “Counting and correlating pairs of words with the <code>widyr</code> package” described in <a rel="nofollow" href="https://www.tidytextmining.com/ngrams.html#counting-and-correlating-pairs-of-words-with-the-widyr-package" target="_blank">this section of the book</a> but it wasn’t too interesting because most correlation values were too low. One issue was probably my having too few titles: only half of pictures have titles! So I’ll resort to plotting most common bigrams, which I learnt in the <a rel="nofollow" href="https://www.tidytextmining.com/" target="_blank">Tidy text mining book</a> as well.</p>
<div>
<div>
<pre><code><span>stopwords</span><span> </span><span>&lt;-</span><span> </span><span>rcorpora</span><span>::</span><span>corpora</span><span>(</span><span>&quot;words/stopwords/en&quot;</span><span>)</span><span>$</span><span>stopWords</span><span>
</span><span>ds_stock</span><span> </span><span>%>%</span><span>
</span><span>dplyr</span><span>::</span><span>filter</span><span>(</span><span>!</span><span>is.na</span><span>(</span><span>title</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>dplyr</span><span>::</span><span>select</span><span>(</span><span>title</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>unique</span><span>()</span><span> </span><span>%>%</span><span>
</span><span>tidytext</span><span>::</span><span>unnest_tokens</span><span>(</span><span>bigram</span><span>,</span><span> </span><span>title</span><span>,</span><span>
</span><span>token</span><span> </span><span>=</span><span> </span><span>&quot;ngrams&quot;</span><span>,</span><span> </span><span>n</span><span> </span><span>=</span><span> </span><span>2</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>tidyr</span><span>::</span><span>separate</span><span>(</span><span>bigram</span><span>,</span><span> </span><span>c</span><span>(</span><span>&quot;word1&quot;</span><span>,</span><span> </span><span>&quot;word2&quot;</span><span>),</span><span> </span><span>sep</span><span> </span><span>=</span><span> </span><span>&quot; &quot;</span><span>,</span><span>
</span><span>remove</span><span> </span><span>=</span><span> </span><span>FALSE</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>dplyr</span><span>::</span><span>filter</span><span>(</span><span>!</span><span>word1</span><span> </span><span>%in%</span><span> </span><span>stopwords</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>dplyr</span><span>::</span><span>filter</span><span>(</span><span>!</span><span>word2</span><span> </span><span>%in%</span><span> </span><span>stopwords</span><span>)</span><span>%>%</span><span>
</span><span>dplyr</span><span>::</span><span>count</span><span>(</span><span>bigram</span><span>,</span><span> </span><span>sort</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>dplyr</span><span>::</span><span>mutate</span><span>(</span><span>bigram</span><span> </span><span>=</span><span> </span><span>reorder</span><span>(</span><span>bigram</span><span>,</span><span> </span><span>n</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>head</span><span>(</span><span>n</span><span> </span><span>=</span><span> </span><span>10</span><span>)</span><span>%>%</span><span>
</span><span>ggplot</span><span>()</span><span> </span><span>+</span><span>
</span><span>geom_lollipop</span><span>(</span><span>aes</span><span>(</span><span>bigram</span><span>,</span><span> </span><span>n</span><span>),</span><span>
</span><span>size</span><span> </span><span>=</span><span> </span><span>2</span><span>,</span><span> </span><span>col</span><span> </span><span>=</span><span> </span><span>&quot;salmon&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>hrbrthemes</span><span>::</span><span>theme_ipsum</span><span>(</span><span>base_size</span><span> </span><span>=</span><span> </span><span>16</span><span>,</span><span>
</span><span>axis_title_size</span><span> </span><span>=</span><span> </span><span>16</span><span>)</span><span> </span><span>+</span><span>
</span><span>ggtitle</span><span>(</span><span>&quot;300 data science stock photos&quot;</span><span>,</span><span>
</span><span>subtitle</span><span> </span><span>=</span><span> </span><span>&quot;Most frequent bigrams in titles. Source: https://www.pexels.com&quot;</span><span>)</span><span>+</span><span>
</span><span>coord_flip</span><span>()</span><span>
</span></code></pre>
</div>
</div>
<p><img alt="plot of chunk unnamed-chunk-5" src="https://i2.wp.com/www.masalmon.eu/figure/source/2018-05-06-dsstock/unnamed-chunk-5-1.png?w=456" data-recalc-dims="1"></p>
<p>So there’s a lot of holding computer happening, and these laptops are either black or white… And well Macbook Pro probably looks more professional?</p>
<h1>Hold my laptop and watch…</h1>
<p>my trying to find a good post conclusion! In this post, I tried to responsibly and elegantly scrape rich photo metadata from Pexels to characterize stock photos of data science. Using tags, and most common bigrams in titles, I found that data science stock photos are associated with data, business and computers; and that they often show people holding computers. Now, you’ll excuse me while I try and comfort my poor pink laptop that feels a bit too un-data-sciency.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.masalmon.eu/2018/05/06/dsstock/"> Maëlle</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/Pbf8FMwqtVM" height="1"></a></div>