<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/web-data-acquisition-the-structure-of-rcurl-request-part-2/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://r-posts.com/web-data-acquisition-the-structure-of-rcurl-request-part-2/"> R-posts.com</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>The acquisition of data in json structure presented in <a target="_blank" rel="nofollow" href="http://www.linkedin.com/pulse/web-data-acquisition-understanding-rcurl-from-command-roberto-palloni">part 1</a> clearly showed the functioning of the client-server connection and the possibility to collect the data of interest. However, the json output appeares as a set of raw data in a json string that needs to be structured and stored in a suitable form for data processing and statistical analysis.</p>
<p>For this reason, it makes sense to develop the entire process using #R in order to have the data directly queried, collected, parsed, structured and made usable in a unique environment. Of course, this will be the one used in the process “last mile”, i.e. data analysis.</p>
<p>The <em>curl </em>library adopted in the command line process described in the previous post has its alter ego in the <em>RCurl</em> library. Together with <em>jsonlite</em> for ‘R-JSON translation’ these are the necessay packages for the development of the request as presented in the following code.</p>
<pre>
# before loading the libraries rememeber to install them - install.packages('library here')
library(RCurl)
library(jsonlite)

# save the url of the request in an object (same as -X POST in the curl request)

url &lt;- 'https://www.googleapis.com/qpxExpress/v1/trips/search?key={SERVER_KEY}&amp;alt=json'
# headers (same as -H)
headers &lt;- list('Accept' = 'application/json', 'Content-Type' = 'application/json', 'charset' = 'UTF-8')

# R structure of the input for the request (same as -d + JSON)
x = list(
  request = list(
    slice = list(
      list(origin = 'FCO', destination = 'LHR', date = '2017-06-30')),
    passengers = list(adultCount = 1, infantInLapCount = 0, infantInSeatCount = 0, childCount = 0, seniorCount = 0),
    solutions = 500,
    refundable = F))

# url, headers and x are the parameters to be used in R functions to send the request
# and save the output data in the datajson object
# postForm is the RCurl function to send the request using the POST method
# toJSON is the jsonlite function to convert the R structure of the request in JSON input

datajson &lt;- postForm(url, .opts=list(postfields=toJSON(x), httpheader=headers))
datajson

</pre>
<p>After few seconds from the POST request necessary to send the request and collect the response, all the information related to the flights with origin FCO (Fiumicino – Rome) and destination LHR (London Heathrow) will be hosted in the datajson object, similarly to the command line procedure. The json string holds and hides all the observations and variables of interest for the statistical analysis inlcuding the most important, i.e. the flight prices.</p>
<p>The next post will explain how to <strong>parse the json</strong> object and structure the information in a suitable dataframe for analysis using the powerful library #<em>tidyjson</em>.</p>
<p>#R #rstats #maRche #json #curl #qpxexpress #Rbloggers</p>
<p><em>This post is also shared in </em><a target="_blank" rel="nofollow" href="http://www.r-bloggers.com/"><em>www.r-bloggers.com</em></a><em> and </em><a target="_blank" rel="nofollow" href="http://www.linkedin.com/pulse/web-data-acquisition-structure-rcurl-request-part-2-roberto-palloni"><em>LinkedIn</em></a></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://r-posts.com/web-data-acquisition-the-structure-of-rcurl-request-part-2/"> R-posts.com</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/5JJOy1Mo1eE"></a></div>