<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/interacting-with-aws-from-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://www.daeconomist.com/post/2018-06-27-aws/"> Digital Age Economist on Digital Age Economist</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2>Getting set up</h2>
<p>If there is one realisation in life, it is the fact that you will never have enough CPU or RAM available for your analytics. Luckily for us, cloud computing is becoming cheaper and cheaper each year. One of the more established providers of cloud services is AWS. If you don’t know yet, they provide a free, yes free, option. Their <code>t2.micro</code> instance is a 1 CPU, 500MB machine, which doesn’t sound like much, but I am running a Rstudio and Docker instance on one of these for a small project.</p>
<p>The management console has the following interface:</p>
<p><img src="https://i2.wp.com/www.daeconomist.com/img/blogs/2018-06-30-AWS/AWS.png?w=450&ssl=1" data-recalc-dims="1"></p>
<p>So, how cool would it be if you could start up one of these instances from <code>R</code>? Well, with the <a rel="nofollow" href="https://github.com/cloudyr" target="_blank">cloudyr</a> project it makes <code>R</code> a lot better at interacting with cloud based computing infrastructure. With this in mind, I have been playing with the <a rel="nofollow" href="https://github.com/cloudyr/aws.ec2" target="_blank">aws.ec2</a> package which is a simple client package for the Amazon Web Services (‘AWS’) Elastic Cloud Compute <a rel="nofollow" href="https://aws.amazon.com/ec2/" target="_blank">EC2</a><code>API</code>. There is some irritating setup that has to be done, so if you want to use this package, you need to follow the instructions on the <a rel="nofollow" href="https://github.com/cloudyr/aws.ec2" target="_blank">github</a> page to create <code>AWS_ACCESS_KEY_ID</code>, <code>AWS_SECRET_ACCESS_KEY</code> and <code>AWS_DEFAULT_REGION</code> parameters in the <code>ENV</code>. But once you have figured out this step, the fun starts.</p>
<p>I always enjoy getting the development version of a package, so I am going to install the package straight from github:</p>
<pre><code>devtools::install_github(&quot;cloudyr/aws.ec2&quot;)
</code></pre>
<p>Next we are going to use a Amazon Machine Images (AMI) which is a pre-build image that already contains all the necessary installations such as <code>R</code> and <code>RStudio</code>. You can build your own AMI and I suggest you build your own if you comfortable with Linux CLI.</p>
<h2>Release the beast</h2>
<pre><code>library(aws.ec2)
# Describe the AMI (from: http://www.louisaslett.com/RStudio_AMI/)
aws.signature::locate_credentials()
image &lt;- &quot;ami-3b0c205e&quot;
describe_images(image)
</code></pre>
<p>In the code snippet above you will notice I call a function <code>aws.signature::locate_credentials()</code>. I use this function to confirm my credentials. You will need to populate your own credentials after creating a user profile on <a rel="nofollow" href="https://aws.amazon.com/" target="_blank">IAM management console</a> and have generated an <code>ACCESS_KEY</code> for the use of the API. My preferred method of implementing the credentials, is to add the information to the environment using <code>usethis::edit_r_environ()</code>.</p>
<p>Here is my (fake) <code>.Renviron</code>:</p>
<pre><code>AWS_ACCESS_KEY_ID=F8D6E9131F0E0CE508126
AWS_SECRET_ACCESS_KEY=AAK53148eb87db04754+f1f2c8b8cae222a2
AWS_DEFAULT_REGION=us-east-2
</code></pre>
<p>Now we are almost ready to test out the package and its functions, but first, I recommend you source a handy function I wrote that helps to tidy the outputs from selected functions from the <code>aws.ec2</code> package.</p>
<pre><code>source(&quot;https://bit.ly/2KnkdzV&quot;)
</code></pre>
<p>I found the <code>list</code> object returned from functions such as <code>describe_images()</code>, <code>describe_instance()</code> and <code>instance_status()</code> very verbose and difficult to work with. The <code>tidy_describe()</code> function is there to clean up the outputs and only return the most important information. The function also implements a <code>pretty_print</code> option which<br>
will <code>cat</code> the output in a table to the screen for a quick overview of the information contained in the object.</p>
<p>Lets use this function to see the output from the <code>describe_images()</code> as a <code>pretty_print</code>. Print the <code>aws_describe</code> object without this handy function at your own peril.</p>
<pre><code>image &lt;- &quot;ami-3b0c205e&quot;
aws_describe &lt;- describe_images(image)
aws_describe %>% tidy_describe(.)
</code></pre>
<pre><code>--------------------------------------
                Summary               
--------------------------------------
imageId : ami-3b0c205e 
imageOwnerId : 732690581533 
creationDate : 2017-10-17T09:28:45.000Z 
name : RStudio-1.1.383_R-3.4.2_Julia-0.6.0_CUDA-8_cuDNN-6_ubuntu-16.04-LTS-64bit 
description : Ready to run RStudio + Julia/Python server for statistical computation (www.louisaslett.com). Connect to instance public DNS in web brower (standard port 80), username rstudio and password rstudio 

To return as tibble: pretty_print = FALSE
</code></pre>
<p>Once we have confirmed that we are happy with the image, we need to save the <a rel="nofollow" href="https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Subnets.html" target="_blank">subnet</a> information as well as the <a rel="nofollow" href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html" target="_blank">security group</a> information.</p>
<pre><code>s &lt;- describe_subnets()
g &lt;- describe_sgroups()
</code></pre>
<p>Now that you have specified those two things, you have all the pieces to spin up the machine of your choice. To have a look at what machines are available, visit the <a rel="nofollow" href="https://aws.amazon.com/ec2/instance-types/" target="_blank">instance type</a> webpage to choose your machine. <strong>Warning:</strong> <em>choosing big machines with lots of CPU and a ton of RAM can be addictive. Winners know when to stop</em></p>
<p>In this example I spin up a <code>t2.micro</code> instance, which is part of the free tier from which Amazon provides.</p>
<pre><code># Launch the instance using appropriate settings
i &lt;- run_instances(image = image, 
                   type = &quot;t2.micro&quot;, # &lt;- you might want to change this to something like x1e.32xlarge ($26.688 p/h) if you feeling adventurous
                   subnet = s[[1]], 
                   sgroup = g[[1]])
</code></pre>
<p>Once I have executed the code above, I can check on the instance using <code>instance_status</code> to see if the machine is ready, or <code>describe_instance</code> to get the meta information on the machine such as <code>ip</code>. Again, I use the custom <code>tidy_describe</code></p>
<pre><code>aws_instance &lt;- describe_instances(i)
aws_instance %>% tidy_describe()
</code></pre>
<pre><code>--------------------------------------
                Summary               
--------------------------------------
ownerId : 748485365675 
instanceId : i-007fd9116488691fe 
imageId : ami-3b0c205e 
instanceType : t2.micro 
launchTime : 2018-06-30T13:15:50.000Z 
availabilityZone : us-east-2b 
privateIpAddress : 172.31.16.198 
ipAddress : 18.222.174.186 
coreCount : 1 
threadsPerCore : 1 
To return as tibble: pretty_print = FALSE
</code></pre>
<pre><code>aws_status &lt;- instance_status(i)
aws_status %>% tidy_describe()
</code></pre>
<pre><code>--------------------------------------
                Summary               
--------------------------------------
instanceId : i-007fd9116488691fe 
availabilityZone : us-east-2b 
code : 16 
name : running 

To return as tibble: pretty_print = FALSE
</code></pre>
<p>The final bit of code (which is VERY important when running large instance), is to stop the instance and confirm that it has been terminated:</p>
<pre><code># Stop and terminate the instances
stop_instances(i[[1]])
terminate_instances(i[[1]])
</code></pre>
<h2>Final comments</h2>
<p>Working with <code>AWS</code>-instances for a while now has really been a game changer in the way I conduct/approach any analytical project. Having the capability to switch on large machines on demand and quickly run any of my analytical scripts opened up new opportunities on what I can do as a consultant who has very limited budget to spend on hardware – also, where will I keep my 96 core 500GB RAM machine once I have scraped enough cash together to actually build such a machine?</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://www.daeconomist.com/post/2018-06-27-aws/"> Digital Age Economist on Digital Age Economist</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/2a6rj76XR00" height="1"></a></div>