<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/take-screenshot-of-webpage-using-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.listendata.com/2018/05/take-screenshot-of-webpage-using-r.html"> ListenData</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div dir="ltr">Programmatically taking screenshots of a web page is very essential in a testing environment to see about the web page. But the same can be used for automation like getting the screenshot of the news website every morning into your Inbox or generating a report of candidates’ github activities. But this wasn’t possible in command line until the rise of headless browsers and javascript libraries supporting them. Even when such JavaScript libraries where made available, R programmers did not have any option to integrate such functionality in their code.<br>That is when <b>webshot </b>an R package that helps R programmers take web screenshots programmatically with the help of <b>phantomJS </b>running in the backend.
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i2.wp.com/1.bp.blogspot.com/-47nkifV2JLs/WwxSwVrYX1I/AAAAAAAAHOM/x2HCmsnVuVQL8Yxdi3zWGVwXka_9EmKiACLcBGAs/s1600/Capture.PNG?ssl=1" target="_blank"><img border="0" data-original-height="224" src="https://i2.wp.com/1.bp.blogspot.com/-47nkifV2JLs/WwxSwVrYX1I/AAAAAAAAHOM/x2HCmsnVuVQL8Yxdi3zWGVwXka_9EmKiACLcBGAs/s1600/Capture.PNG?resize=450%2C224&ssl=1" data-recalc-dims="1" data-original-width="450"></a></td>
</tr>
<tr>
<td>Take Screenshot from R</td>
</tr>
</tbody>
</table>
<h2><b><br>What is PhantomJS?</b></h2>
<p>PhantomJS is a headless webkit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.</p>
<p>PhantomJS is an optimal solution for the following:</p>
<ul>
<li>Headless website testing</li>
<li>Screen Capture</li>
<li>Page Automation</li>
<li>Network Monitoring</li>
</ul>
<p></p>
<h3><b><span>Webshot : R Package </span></b></h3>
<p>The webshot package allows users to take screenshots of web pages from R with the help of PhantomJS. It also can take screenshots of R Shiny App and R Markdown Documents (both static and interactive).</p>
<h3><b><span>Install and Load Package</span></b></h3>
<p>The stable version of webshot is available on CRAN hence can be installed using the below code:</p>
<blockquote><p>install.packages(‘webshot’)<br>library(‘webshot’)</p></blockquote>
<p>Also, the latest development version of webshot is hosted on github and can be installed using the below code:</p>
<blockquote><p>#install.packages(‘devtools’)<br>devtools::install_github(‘wch/webshot’)</p></blockquote>
<p><b><span>Initial Setup</span></b></p>
<p>As we saw above, the R package webshot works with PhantomJS in the backend, hence it is essential to have PhantomJS installed on the local machine where webshot package is used. To assist with that, webshot itself has an easy function to get PhantomJS installed on your machine.</p>
<blockquote><p>webshot::install_phantomjs()</p></blockquote>
<p>The above function automatically downloads PhantomJS from its website and installs it. Please note this is only a first time setup and once both webshot and PhantomJS are installed these above two steps can be skipped for using the package as mentioned in the below sections.</p>
<p>Now, webshot package is installed and setup and is ready to use. To start with let us take a PDF copy of a web page.</p>
<p><b><span>Screenshot Function</span></b></p>
<p>webshot package provides one simple function webshot() that takes a webpage url as its first argument and saves it in the given file name that is its second argument. It is important to note that the filename includes the file extensions like ‘.jpg’, ‘.png’, ‘.pdf’ based on which the output file is rendered. Below is the basic structure of how the function goes:</p>
<blockquote><p>library(webshot)</p>
<p>#webshot(url, filename.extension)<br>webshot(“https://www.listendata.com/”, “listendata.png”) </p></blockquote>
<p>If no folder path is specified along with the filename, the file is downloaded in the current working directory which can be checked with <i><b>getwd()</b></i>.</p>
<p>Now that we understood the basics of the <b>webshot()</b> function, It is time for us to begin with our cases – starting with downloading/converting a webpage as a PDFcopy.</p>
<p><b><span>Case #1: PDF Copy of WebPage</span></b><br><b><br></b>Let us assume, we would like to download <a rel="nofollow" href="https://www.gatesnotes.com/About-Bill-Gates/Best-Books-2017" target="_blank">Bill Gates’ notes on Best Books of 2017</a> as a PDF copy.</p>
<blockquote><p>#loading the required library<br> library(webshot)</p>
<p>#PDF copy of a web page / article<br> webshot(“https://www.gatesnotes.com/About-Bill-Gates/Best-Books-2017”,<br> “billgates_book.pdf”,<br> delay = 2) </p></blockquote>
<p>The above code generates a PDF whose (partial) screenshot is below:</p>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i1.wp.com/4.bp.blogspot.com/-JaY4y_R-u0Y/WwwjRyF1c2I/AAAAAAAAAPk/lBxXb7RjiVM4vZ7VsCylaWka4MX4OHkNwCLcBGAs/s1600/billgates_book.PNG?ssl=1" target="_blank"><img border="0" src="https://i1.wp.com/4.bp.blogspot.com/-JaY4y_R-u0Y/WwwjRyF1c2I/AAAAAAAAAPk/lBxXb7RjiVM4vZ7VsCylaWka4MX4OHkNwCLcBGAs/s400/billgates_book.PNG?resize=450%2C589&ssl=1" data-original-width="450" width="450" data-original-height="589" data-recalc-dims="1"></a></td>
</tr>
<tr>
<td>Snapshot of PDF Copy</td>
</tr>
</tbody>
</table>
<p>Dissecting the above code, we can see that the <b>webshot( )</b> function has got 3 arguments supplied with it.</p>
<ol>
<li><b>URL </b>from which the screenshot has to be taken. </li>
<li><b>Output Filename</b> along with its file extensions. </li>
<li><b>Time to wait</b> before taking screenshot, in seconds. Sometimes a longer delay is needed for all assets to display properly.</li>
</ol>
<p>Thus, a webpage can be converted/downloaded as a PDF programmatically in R.</p>
<p><span><b>Case #2: </b><b>Webpage Screenshot (Viewport Size)</b></span><br><b><br></b>Now, I’d like to get an automation script running to get screenshot of a News website and probably send it to my inbox for me to see the headlines without going to the browser. Here we will see how to get a simple screenshot of <b>livemint.com</b> an Indian news website.</p>
<blockquote><p>#Screenshot of Viewport<br>webshot(‘https://www.livemint.com/’,’livemint.png’, cliprect = ‘viewport’)</p></blockquote>
<div>While the first two arguments are similar to the above function, there’s a new third argument <i><b>cliprect </b></i>which specifies the size of the Clipping rectangle.
<p>If <b><i>cliprect</i> </b>is unspecified, the screenshot of the complete web page is taken (like in the above case). Since we are updated in only the latest news (which is usually on the top of the website), we use <i><b>cliprect </b></i>with the value <b>‘<i>viewport</i>‘ </b>which clips only the viewport part of the browser, as below.</div>
<p></p>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i0.wp.com/2.bp.blogspot.com/-fVBbyvjzIN0/WwwjWEwkvVI/AAAAAAAAAPo/li-EuV8mxrIS4eTTVG7vZXzoOIRdkGh0ACLcBGAs/s1600/livemint.png?ssl=1" target="_blank"><img border="0" src="https://i1.wp.com/2.bp.blogspot.com/-fVBbyvjzIN0/WwwjWEwkvVI/AAAAAAAAAPo/li-EuV8mxrIS4eTTVG7vZXzoOIRdkGh0ACLcBGAs/s320/livemint.png?resize=450%2C744&ssl=1" data-original-width="450" width="450" data-original-height="744" data-recalc-dims="1"></a></td>
</tr>
<tr>
<td>Screenshot of Viewport of Browser</td>
</tr>
</tbody>
</table>
<p><span><b>Case #3: </b><b>Multiple Selector Based Screenshots</b></span></p>
<p>All the while we have seen taking simple screenshots of the whole pages and we dealt with one screenshot and one file, but that is not what usually happens when you are dealing with automation or perform something programmatically. In most of the cases we end up performing more than one action, hence this case deals with <b>taking multiple screenshots</b> <b>and saving multiple files</b>. But instead of taking multiple screenshots of different urls (which is quite straightforward), we will screenshots of different sections of the same web page with different <b>CSS selector</b> and save them in respective files.</p>
<blockquote><p>#Multiple Selector Based Screenshots<br>webshot(“https://github.com/hadley”,<br> file = c(“organizations.png”,”contributions.png”),<br> selector = list(“div.border-top.py-3.clearfix”,”div.js-contribution-graph”))</p></blockquote>
<div>In the above code, we take screenshot of two CSS Selectors from the github profile page of  <a rel="nofollow" href="https://github.com/hadley" target="_blank"><b>Hadley Wickham</b></a> and save them in two PNG files – <b>organizations.png</b> and <b>contributions.png.</b>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i2.wp.com/2.bp.blogspot.com/-psctzk6IJI8/WwwpOtE68bI/AAAAAAAAAQE/AGycYY07GeMtaRKcTd_kss5-fakgXMz3QCLcBGAs/s1600/contributions.png?ssl=1" target="_blank"><img border="0" src="https://i1.wp.com/2.bp.blogspot.com/-psctzk6IJI8/WwwpOtE68bI/AAAAAAAAAQE/AGycYY07GeMtaRKcTd_kss5-fakgXMz3QCLcBGAs/s400/contributions.png?resize=450%2C188&ssl=1" data-original-width="450" width="450" data-original-height="188" data-recalc-dims="1"></a></td>
</tr>
<tr>
<td>Contributions.png</td>
</tr>
</tbody>
</table>
<p></p>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i0.wp.com/4.bp.blogspot.com/-PFs1YkDW54Q/WwwpPs_ZevI/AAAAAAAAAQI/Y6eFzJmqugMTjBeLXfIbBo6epveokdqawCLcBGAs/s1600/organizations.png?ssl=1" target="_blank"><img border="0" data-original-height="137" src="https://i0.wp.com/4.bp.blogspot.com/-PFs1YkDW54Q/WwwpPs_ZevI/AAAAAAAAAQI/Y6eFzJmqugMTjBeLXfIbBo6epveokdqawCLcBGAs/s1600/organizations.png?resize=229%2C137&ssl=1" data-recalc-dims="1" data-original-width="229"></a></td>
</tr>
<tr>
<td>Organizations.png</td>
</tr>
</tbody>
</table>
</div>
<p>Thus, we have seen how to use the R package webshot for taking screenshots programmatically in R. Hope, this post helps fuel your automation needs and helps your organisation improve its efficiency.<br><b><br></b><b><span>References</span></b></p>
<ul>
<li><a rel="nofollow" href="https://github.com/amrrs/blogpost_codes/blob/master/web_page_screenshots_in_R.R" target="_blank"><b>Code used above on Github</b></a></li>
<li><a rel="nofollow" href="https://github.com/wch/webshot" target="_blank"><b>webshot – Github Repo</b></a></li>
<li><a rel="nofollow" href="http://phantomjs.org/" target="_blank"><b>PhantomJS Website</b></a></li>
</ul>
</div>
<div><span> About Author:</span> 
<p>Deepanshu founded ListenData with a simple objective – Make analytics easy to understand and follow. He has over 7 years of experience in data science and predictive modeling. During his tenure, he has worked with global clients in various domains.</p>
<p>Let's Get Connected: <a rel="nofollow" href="https://www.linkedin.com/in/deepanshubhalla" target="_blank">LinkedIn</a></p>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.listendata.com/2018/05/take-screenshot-of-webpage-using-r.html"> ListenData</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/cnV1fcyqvdg" height="1"></a></div>