<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/tweeting-wikidata-info/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://blog.rmhogervorst.nl/blog/2018/11/19/tweeting-wikidata-info/"> Category R on Roel's R-tefacts</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>In this explainer I walk you through the steps I took to create a twitter bot<br>
that tweets daily about people who died on that date.</p>
<p>I created a script that queries wikidata, takes that information and creates<br>
a sentence. That sentence is then tweeted.</p>
<p>For example:</p>
<div>
<img alt="A tweet I literally just send out from the docker container" src="https://i1.wp.com/blog.rmhogervorst.nl/post/2018-11-19-tweeting-wikidata-info_files/Screenshot%20from%202018-11-19%2021-32-11.png?w=456&ssl=1" data-recalc-dims="1">
<p>A tweet I literally just send out from the docker container</p>
</div>
<p>I hope you are has excited as I am about this project. Here it comes!</p>
<p>There are 3 parts:</p>
<ol>
<li>Talk to wikidata and retrieve information about 10 people that died today</li>
<li>Grab one of the deaths and create a sentence</li>
<li>Post that sentence to twitter in the account <a rel="nofollow" href="https://twitter.com/WikidataB" target="_blank">wikidatabot</a></li>
<li>Throw it all into a docker container so it can run on the computer of someone else (AKA: THA CLOUD)</li>
</ol>
<p>You might wonder, why people who died? To which I answer, emphatically but not<br>
really helpfully: ‘valar morghulis’.</p>
<div>
<h1>1. Talk to wikidata and retrieve information</h1>
<p>I think wikidata is one of the coolest knowledge bases in the world, it contains<br>
facts about people, other animals, places, and the world. It powers many boxes<br>
you see in Wikipedia pages. For instance <a rel="nofollow" href="https://en.wikipedia.org/wiki/Charles_I_of_England" target="_blank">this random page about Charles the first</a> has a box on the<br>
right that says something about his ancestors, successors and coronation.<br>
The same information can be displayed in <a rel="nofollow" href="https://nl.wikipedia.org/wiki/Karel_I_van_Engeland" target="_blank">Dutch</a>.<br>
This is very cool and saves Wikipedia a lot of work. However, we can also use it!</p>
<p>You can create your own query about the world in <a rel="nofollow" href="https://query.wikidata.org/" target="_blank">the query editor</a>. But it is quite hard to figure out how to do that. These queries need to made in<br>
a specific way. I just used an example from wikidata: ‘who’s birthday is it today?’<br>
and modified it to search for people’s death (that’s how I learn, modify<br>
something and see if I broke it). It looks a lot like SQL, but is slightly different.</p>
<p>Of course this editor is nice for us humans, but we want the computer to do it<br>
so we can send a query to wikidata. I was extremely lazy and used the<br>
<code>WikidataQueryServiceR</code> created by wiki-guru Mikhail Popov <span>[@bearlogo]</span>(<a rel="nofollow" href="https://twitter.com/bearloga" target="_blank">https://twitter.com/bearloga</a>).</p>
<p>This is the query I ended up using (It looks very much like the birthdays one<br>
but with added information):</p>
<pre><code>querystring &lt;- 
'SELECT # what variables do you want to return (defined later on)
  ?entityLabel (YEAR(?date) AS ?year) 
  ?cause_of_deathLabel 
  ?place_of_deathLabel 
  ?manner_of_deathLabel  
  ?country_of_citizenshipLabel 
  ?country_of_birth
  ?date_of_birth
WHERE {
  BIND(MONTH(NOW()) AS ?nowMonth) # this is a very cool trick
  BIND(DAY(NOW()) AS ?nowDay)
  ?entity wdt:P570 ?date.
  SERVICE wikibase:label { bd:serviceParam wikibase:language &quot;en&quot;. }
  ?entity wdt:P509 ?cause_of_death.
  OPTIONAL { ?entity wdt:P20 ?place_of_death. }
  OPTIONAL { ?entity wdt:P1196 ?manner_of_death. }
  FILTER(((MONTH(?date)) = ?nowMonth) &amp;&amp; ((DAY(?date)) = ?nowDay))
  OPTIONAL { ?entity wdt:P27 ?country_of_citizenship. }
  OPTIONAL { ?entity wdt:p19 ?country_of_birth}
  OPTIONAL { ?entity wdt:P569 ?date_of_birth.}
}
LIMIT 10'</code></pre>
<p>Try this in <a rel="nofollow" href="https://query.wikidata.org/" target="_blank">the query editor</a></p>
<p>When I created this blog post (every day the result will be different)<br>
the result looked like this:</p>
<pre><code>library(WikidataQueryServiceR)
result &lt;- query_wikidata(querystring)</code></pre>
<pre><code>## 10 rows were returned by WDQS</code></pre>
<pre><code>result[1:3,1:3]# first 3 rows, first 3 columsn</code></pre>
<pre><code>##                  entityLabel year cause_of_deathLabel
## 1 Rafael García-Plata y Osma 1918           influenza
## 2        Dobroslava Menclová 1978       traffic crash
## 3             Alan J. Pakula 1998       traffic crash</code></pre>
<p>The query returns name, year, cause of death, manner of death<br>
(didn’t know which one to use), place of death, country of citizenship, country<br>
of birth and date of birth.<br>
I can now glue all these parts together to create a sentence of sorts</p>
</div>
<div>
<h1>2. grab one of the deaths and create a sentence</h1>
<p>I will use glue to make text, but the paste functions from base R is also fine.</p>
<p>These are the first lines for instance:</p>
<pre><code>library(glue)
glue_data(result[1:2,], &quot;Today in {year} in the place {place_of_deathLabel} died {entityLabel} with cause: {cause_of_deathLabel}. {entityLabel} was born on {as.Date(date_of_birth, '%Y-%m-%d')}. Find more info on this cause of death on www.en.wikipedia.org/wiki/{cause_of_deathLabel}.  #wikidata&quot;)</code></pre>
<pre><code>## Today in 1918 in the place Cáceres died Rafael García-Plata y Osma with cause: influenza. Rafael García-Plata y Osma was born on 1870-03-04. Find more info on this cause of death on www.en.wikipedia.org/wiki/influenza.  #wikidata
## Today in 1978 in the place Plzeň died Dobroslava Menclová with cause: traffic crash. Dobroslava Menclová was born on 1904-01-02. Find more info on this cause of death on www.en.wikipedia.org/wiki/traffic crash.  #wikidata</code></pre>
</div>
<div>
<h1>Post that sentence to twitter in the account wikidatabot</h1>
<p>I created the twitter account <a rel="nofollow" href="https://twitter.com/WikidataB" target="_blank">wikidatabot</a> and<br>
added pictures 2fa and some bio information. I wanted to make it clear that it<br>
was a bot. To post something on your behalf on twitter requires a developers<br>
account. Go to <a rel="nofollow" href="https://developer.twitter.com/" target="_blank">https://developer.twitter.com</a> and create that account. In my case<br>
I had to manually verify twice because apparently everything I did screamed bot activity<br>
to twitter (they were not entirely wrong). You have to sign some boxes,<br>
acknowledge the code of conduct and understand twitter’s terms.</p>
<p>The next step is to create a twitter app but I will leave that explanation to<br>
rtweet, because <a rel="nofollow" href="https://rtweet.info/articles/auth.html" target="_blank">that vignette</a> is very<br>
very helpful.</p>
<p>When you’re done, you can post to twitter on your account with the help of<br>
a consumer key, access key, consumer token and access token. You will need them<br>
all and you will have to keep them a secret (or other people can post on your<br>
account, and that is something you really don’t want).</p>
<p>With those secrets and the rtweet package you can create a token that enables<br>
you to post to twitter.</p>
<p>And it is seriously as easy as:</p>
<pre><code>rtweet::post_tweet(status = tweettext, token = token )</code></pre>
<div>
<img alt="Again the same tweet" src="https://i1.wp.com/blog.rmhogervorst.nl/post/2018-11-19-tweeting-wikidata-info_files/Screenshot%20from%202018-11-19%2021-32-11.png?w=456&ssl=1" data-recalc-dims="1">
<p>Again the same tweet</p>
</div>
</div>
<div>
<h1>4 Throw it all into a docker container</h1>
<p>I want to post this every day but to make it run in the cloud it would be nice<br>
if R and the code would be nicely packed together. That is where docker comes in,<br>
you can define what packages you want and a mini operating system is created<br>
that will run for everyone on ever computer (if they have docker).<br>
The whole example script and docker file can be found <a rel="nofollow" href="https://github.com/RMHogervorst/tweetwikidatadeaths" target="_blank">here on github</a>.</p>
<p>And that’s it. If you have suggestions on how to run it every day in the cloud<br>
for cheap, let me know by <a rel="nofollow" href="https://twitter.com/RoelMHogervorst" target="_blank">twitter</a> or by opening an issue on <a rel="nofollow" href="https://github.com/RMHogervorst/tweetwikidatadeaths/issues/new" target="_blank">github</a>.</p>
<div>
<h2>Things that could be done better:</h2>
<ul>
<li>I can run the container, but I don’t know how to make it run in the cloud</li>
<li>I ask for 10 deaths and pick one randomly, I don’t know if there is a random function in sparql</li>
<li>I put the (twitter) keys into the script, it would be better to use environment variables for that</li>
<li>rtweet and WikidataQueryServiceR have lots of dependencies that make the docker container difficult to build (mostly time consuming)</li>
<li>I guess I could just build the query and post to wikidata, but using WikidataQueryServiceR was much faster</li>
<li>I wish I knew how to use the rocker:tidyverse container to run a script, but I haven’t figured that out yet</li>
</ul>
<div>
<h3>State of the machine</h3>
<details>
<p>
</p>
<pre><code>sessioninfo::session_info()</code></pre>
<pre><code>## ─ Session info ──────────────────────────────────────────────────────────
##  setting  value                       
##  version  R version 3.5.1 (2018-07-02)
##  os       Ubuntu 16.04.5 LTS          
##  system   x86_64, linux-gnu           
##  ui       X11                         
##  language en_US                       
##  collate  en_US.UTF-8                 
##  tz       Europe/Amsterdam            
##  date     2018-11-19                  
## 
## ─ Packages ──────────────────────────────────────────────────────────────
##  package               * version date       source         
##  backports               1.1.2   2017-12-13 CRAN (R 3.5.0) 
##  blogdown                0.8     2018-07-15 CRAN (R 3.5.1) 
##  bookdown                0.7     2018-02-18 CRAN (R 3.5.0) 
##  clisymbols              1.2.0   2017-05-21 CRAN (R 3.5.0) 
##  crayon                  1.3.4   2017-09-16 CRAN (R 3.5.0) 
##  curl                    3.2     2018-03-28 CRAN (R 3.5.0) 
##  digest                  0.6.15  2018-01-28 CRAN (R 3.5.0) 
##  evaluate                0.11    2018-07-17 CRAN (R 3.5.1) 
##  glue                  * 1.3.0   2018-07-17 CRAN (R 3.5.1) 
##  htmltools               0.3.6   2017-04-28 CRAN (R 3.5.0) 
##  httr                    1.3.1   2017-08-20 CRAN (R 3.5.0) 
##  knitr                   1.20    2018-02-20 CRAN (R 3.5.0) 
##  magrittr                1.5     2014-11-22 CRAN (R 3.5.0) 
##  R6                      2.2.2   2017-06-17 CRAN (R 3.5.0) 
##  Rcpp                    0.12.18 2018-07-23 cran (@0.12.18)
##  rmarkdown               1.10    2018-06-11 CRAN (R 3.5.0) 
##  rprojroot               1.3-2   2018-01-03 CRAN (R 3.5.0) 
##  sessioninfo             1.0.0   2017-06-21 CRAN (R 3.5.1) 
##  stringi                 1.2.4   2018-07-20 cran (@1.2.4)  
##  stringr                 1.3.1   2018-05-10 CRAN (R 3.5.0) 
##  WikidataQueryServiceR * 0.1.1   2017-04-28 CRAN (R 3.5.1) 
##  withr                   2.1.2   2018-03-15 CRAN (R 3.5.0) 
##  xfun                    0.3     2018-07-06 CRAN (R 3.5.1) 
##  yaml                    2.2.0   2018-07-25 CRAN (R 3.5.1)</code></pre>
</details>
</div>
</div>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://blog.rmhogervorst.nl/blog/2018/11/19/tweeting-wikidata-info/"> Category R on Roel's R-tefacts</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/A9O-fV2SvyA" height="1"></a></div>