<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/time-averages-of-netcdf-files-from-ecmwf-in-arcgis-with-r-bridge/"></div></div><div><a data-url="http://www.r-bloggers.com/time-averages-of-netcdf-files-from-ecmwf-in-arcgis-with-r-bridge/" data-counturl="http://www.r-bloggers.com/time-averages-of-netcdf-files-from-ecmwf-in-arcgis-with-r-bridge/" data-count="horizontal" data-text="Time Averages of NetCDF files from ECMWF in ArcGIS with R-Bridge" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://r-video-tutorial.blogspot.com/2016/07/time-averages-of-netcdf-files-from.html"> R tutorial for Spatial Statistics</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>With this post I would like to talk again about R-Bridge, which allows a direct communication between ArcGIS and R.<br>In the <a target="_blank" href="http://r-video-tutorial.blogspot.ch/2016/07/combine-arcgis-and-r-clustering-toolbox.html">previous post</a>, I presented a very simple application of R-Bridge where I built a toolbox to perform k-means clustering on point shapefiles. The purpose of that post was to explain the basics of the technology, but its scientific scope was limited. However, I would like to try step by step to translate more of my R scripts in ArcGIS, so that more people can use them even if they are not experts in R.<br>In this post I will start presenting a toolbox to handle NetCDF files downloaded from the European Centre for Medium-Range Weather Forecasts (ECMWF).</p>
<p><span>ECMWF</span><br>The European Centre for Medium-Range Weather Forecasts provides free access to numerous weather data through their website. You can go directly to this page to take a look at the data available: <a target="_blank" href="http://www.ecmwf.int/en/research/climate-reanalysis/browse-reanalysis-datasets">http://www.ecmwf.int/en/research/climate-reanalysis/browse-reanalysis-datasets</a><br>The data are freely accessible and downloadable (for research purposes!!) but you need to register to the website to be able to do so.</p>
<p><span>My Research</span><br>For a research I’m doing right now I downloaded the ERA Interim dataset from 2010 to 2015 from this access page: <a target="_blank" href="http://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=sfc/">http://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=sfc/</a></p>
<p>The data are provided in large NetCDF files, which include all the weather variables I selected and for the entire time frame. In R, NetCDF files can be easily imported, using the packages <code>raster</code> and <code>ncdf4</code>, as a raster brick. This will have a X and Y dimensions, plus a time dimension for each of the variables I decided to download. Since I wanted 5 years and the ERA dataset include four datasets per day, I had quite a lot of data.<br>I decided that for the research I was planning to do I did not need each and every one of these rasters, but rather some time averages, for example the monthly averages for each variable. Therefore I created an R script to do the job.</p>
<p>Now I decided to use R-Bridge to implement the R script I developed in ArcGIS. This should allow people not familiar with R to easily create time averages of weather reanalysis data provided by ECMWF.</p>
<p><span><br></span><span>Toolbox</span><br>I already covered in the previous post the installation process of R-Bridge and explained how to create a new toolbox with a script, so if you do not know how to do these thing please refer to <a target="_blank" href="http://r-video-tutorial.blogspot.ch/2016/07/combine-arcgis-and-r-clustering-toolbox.html">this post</a>.<br>For this script I created a simple GUI with two inputs and one output:</p>
<div><a target="_blank" href="http://i1.wp.com/3.bp.blogspot.com/-MdDbyUL62CQ/V39MINx3WbI/AAAAAAAAR-Q/uLQcL0IJH2QWRGUg86LcrHYJadD1qx-qACLcB/s1600/Fig1.jpg?ssl=1"><img src="http://i1.wp.com/3.bp.blogspot.com/-MdDbyUL62CQ/V39MINx3WbI/AAAAAAAAR-Q/uLQcL0IJH2QWRGUg86LcrHYJadD1qx-qACLcB/s640/Fig1.jpg?resize=456%2C279&ssl=1" border="0" data-recalc-dims="1"></a></div>
<p>The first is used to select the NetCDF file, downloaded from the ECMWF website, on the users’ computer. The second is a list of values from which the user can select what type of time average to perform:</p>
<div><a target="_blank" href="http://i2.wp.com/1.bp.blogspot.com/-felPHz4rNoU/V39MbwMQpTI/AAAAAAAAR-U/gCaC7nSRnTwD2VmBplm3Y5qKwAHMx8isgCLcB/s1600/Fig2.jpg?ssl=1"><img src="http://i2.wp.com/1.bp.blogspot.com/-felPHz4rNoU/V39MbwMQpTI/AAAAAAAAR-U/gCaC7nSRnTwD2VmBplm3Y5qKwAHMx8isgCLcB/s640/Fig2.jpg?resize=456%2C278&ssl=1" border="0" data-recalc-dims="1"></a></div>
<p>Users can select between four types: hourly, daily, monthly and yearly averages. In each case, R will select the unique values for each of these categories and create average rasters. Please remember that ECMWF does not provide hourly data, but only observations at specific time of day, every 6 hours or so; therefore, do not expect the script to generate hourly rasters, but only averages for these time slots.</p>
<p>The final thing that users need to provide is the output folder, where R will save all the rasters. This needs to be a new folder!! R will then create the new folder on disk and then save the rasters in there.<br>For the time being I do not have a way to plot these rasters onto ArcGIS after the script is completed. In theory with the function <code>writeRaster</code> in the <code>raster</code> package it is possible to export a raster to ArcGIS directly, but users would need to provide the name of this output raster in the Toolbox GUI and in this case this is not possible because many rasters are created at once. I also tried to create another toolbox in Model Builder where the R script was followed by an iterator that should have opened the rasters directly from the output folder, but it does not work. If you have any suggestion for doing this I would like to hear them. In any case this is not an issue, the important thing is being able to produce average rasters from NetCDF files.</p>
<p><span>R Script</span><br>In the final part of the post I will present the R script I used for this Toolbox. Here is the code:</p>
<div>
<div>
<pre><span>### NetCDF Time Average Toolbox</span><br><span>##Author: Fabio Veronesi</span><br>tool_exec &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/function"><span>function</span></a><span>(</span>in_params<span>,</span> out_params<span>)</span><br><span>{</span><br> <span>if</span> <span>(</span>!requireNamespace<span>(</span><span>&quot;ncdf4&quot;</span><span>,</span> quietly = <span>TRUE</span><span>)</span><span>)</span><br>  <a target="_blank" href="http://inside-r.org/r-doc/utils/install.packages"><span>install.packages</span></a><span>(</span><span>&quot;ncdf4&quot;</span><span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/base/require"><span>require</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/ncdf4">ncdf4</a><span>)</span><br> <br> <span>if</span> <span>(</span>!requireNamespace<span>(</span><span>&quot;reshape2&quot;</span><span>,</span> quietly = <span>TRUE</span><span>)</span><span>)</span><br>  <a target="_blank" href="http://inside-r.org/r-doc/utils/install.packages"><span>install.packages</span></a><span>(</span><span>&quot;reshape2&quot;</span><span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/base/require"><span>require</span></a><span>(</span>reshape2<span>)</span><br> <br> <span>if</span> <span>(</span>!requireNamespace<span>(</span><span>&quot;sp&quot;</span><span>,</span> quietly = <span>TRUE</span><span>)</span><span>)</span><br>  <a target="_blank" href="http://inside-r.org/r-doc/utils/install.packages"><span>install.packages</span></a><span>(</span><span>&quot;sp&quot;</span><span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/base/require"><span>require</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/sp">sp</a><span>)</span><br> <br> <span>if</span> <span>(</span>!requireNamespace<span>(</span><span>&quot;raster&quot;</span><span>,</span> quietly = <span>TRUE</span><span>)</span><span>)</span><br>  <a target="_blank" href="http://inside-r.org/r-doc/utils/install.packages"><span>install.packages</span></a><span>(</span><span>&quot;raster&quot;</span><span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/base/require"><span>require</span></a><span>(</span>raster<span>)</span><br> <br> <span>if</span> <span>(</span>!requireNamespace<span>(</span><span>&quot;rgdal&quot;</span><span>,</span> quietly = <span>TRUE</span><span>)</span><span>)</span><br>  <a target="_blank" href="http://inside-r.org/r-doc/utils/install.packages"><span>install.packages</span></a><span>(</span><span>&quot;rgdal&quot;</span><span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/base/require"><span>require</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/rgdal">rgdal</a><span>)</span><br> <br> <br> <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><span>&quot;Time Averages of ECMWF Datasets&quot;</span><span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><span>&quot;Author: Fabio Veronesi&quot;</span><span>)</span><br> <br> source_nc = in_params<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><br> time_average = in_params<span>[</span><span>[</span><span>2</span><span>]</span><span>]</span><br> <br> out_folder = out_params<span>[</span><span>[</span><span>1</span><span>]</span><span>]</span><br>        <br> <a target="_blank" href="http://inside-r.org/r-doc/base/dir.create"><span>dir.create</span></a><span>(</span>out_folder<span>)</span><br> <br> <span>### Read Data</span><br> arc.progress_label<span>(</span><span>&quot;Reading the NetCDF Dataset...&quot;</span><span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><span>&quot;Opening NC...&quot;</span><span>)</span><br> nc &lt;- nc_open<span>(</span>source_nc<span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/stats/var"><span>var</span></a> &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/names"><span>names</span></a><span>(</span>nc$var<span>)</span><br> <br> <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;NetCDF Variable: &quot;</span><span>,</span><a target="_blank" href="http://inside-r.org/r-doc/stats/var"><span>var</span></a><span>)</span><span>)</span><br> <br> <br> <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><span>&quot;Creating Average Rasters ...&quot;</span><span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><span>&quot;Please note that this process can be time-consuming.&quot;</span><span>)</span><br> <br> <br> <span>for</span><span>(</span>VAR1 <span>in</span> <a target="_blank" href="http://inside-r.org/r-doc/stats/var"><span>var</span></a><span>)</span><span>{</span><br> <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;Executing Script for Variable: &quot;</span><span>,</span> VAR1<span>)</span><span>)</span><br> var.nc &lt;- brick<span>(</span>source_nc<span>,</span> varname=VAR1<span>,</span> layer=<span>&quot;time&quot;</span><span>)</span><br> <br> <span>#Divide by Month</span><br> <a target="_blank" href="http://inside-r.org/packages/cran/time">TIME</a> &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/as.POSIXct"><span>as.POSIXct</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/substr"><span>substr</span></a><span>(</span>var.nc@<a target="_blank" href="http://inside-r.org/r-doc/utils/data"><span>data</span></a>@<a target="_blank" href="http://inside-r.org/r-doc/base/names"><span>names</span></a><span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/stats/start"><span>start</span></a>=<span>2</span><span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/stop"><span>stop</span></a>=<span>20</span><span>)</span><span>,</span> <a target="_blank" href="http://inside-r.org/r-doc/base/format"><span>format</span></a>=<span>&quot;%Y.%m.%d.%H.%M.%S&quot;</span><span>)</span><br> <a target="_blank" href="http://inside-r.org/r-doc/stats/df"><span>df</span></a> &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/data.frame"><span>data.frame</span></a><span>(</span>INDEX = <span>1</span>:<a target="_blank" href="http://inside-r.org/r-doc/base/length"><span>length</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/time">TIME</a><span>)</span><span>,</span> <a target="_blank" href="http://inside-r.org/packages/cran/time">TIME</a>=<a target="_blank" href="http://inside-r.org/packages/cran/time">TIME</a><span>)</span><br> <br> <span>if</span><span>(</span>time_average==<span>&quot;Daily Averages&quot;</span><span>)</span><span>{</span><br>  days &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/unique"><span>unique</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/format"><span>format</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/time">TIME</a><span>,</span> <span>&quot;%d&quot;</span><span>)</span><span>)</span><br> <br>  <span>#LOOP</span><br>  <span>for</span><span>(</span>DAY <span>in</span> days<span>)</span><span>{</span><br>   <a target="_blank" href="http://inside-r.org/r-doc/base/subset"><span>subset</span></a> &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/df"><span>df</span></a><span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/format"><span>format</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/stats/df"><span>df</span></a>$TIME<span>,</span> <span>&quot;%d&quot;</span><span>)</span> == DAY<span>,</span><span>]</span><br>   sub.var &lt;- var.nc<span>[</span><span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/subset"><span>subset</span></a>$INDEX<span>]</span><span>]</span><br> <br>   <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;Executing Average for Day: &quot;</span><span>,</span>DAY<span>)</span><span>)</span><br>   av.var &lt;- calc<span>(</span>sub.var<span>,</span> fun=<a target="_blank" href="http://inside-r.org/r-doc/base/mean"><span>mean</span></a><span>,</span> filename=paste0<span>(</span>out_folder<span>,</span><span>&quot;/&quot;</span><span>,</span>VAR1<span>,</span><span>&quot;_Day&quot;</span><span>,</span>DAY<span>,</span><span>&quot;.tif&quot;</span><span>)</span><span>)</span><br>   <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;Raster for Day &quot;</span><span>,</span>DAY<span>,</span><span>&quot; Ready in the Output Folder&quot;</span><span>)</span><span>)</span><br>  <span>}</span><br> <span>}</span> <span>else</span> <span>if</span><span>(</span>time_average==<span>&quot;Monthly Averages&quot;</span><span>)</span> <span>{</span><br>  <a target="_blank" href="http://inside-r.org/r-doc/base/months"><span>months</span></a> &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/unique"><span>unique</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/format"><span>format</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/time">TIME</a><span>,</span> <span>&quot;%m&quot;</span><span>)</span><span>)</span><br> <br>  <span>#LOOP</span><br>  <span>for</span><span>(</span>MONTH <span>in</span> <a target="_blank" href="http://inside-r.org/r-doc/base/months"><span>months</span></a><span>)</span><span>{</span><br>   <a target="_blank" href="http://inside-r.org/r-doc/base/subset"><span>subset</span></a> &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/df"><span>df</span></a><span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/format"><span>format</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/stats/df"><span>df</span></a>$TIME<span>,</span> <span>&quot;%m&quot;</span><span>)</span> == MONTH<span>,</span><span>]</span><br>   sub.var &lt;- var.nc<span>[</span><span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/subset"><span>subset</span></a>$INDEX<span>]</span><span>]</span><br> <br>   <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;Executing Average for Month: &quot;</span><span>,</span>MONTH<span>)</span><span>)</span><br>   av.var &lt;- calc<span>(</span>sub.var<span>,</span> fun=<a target="_blank" href="http://inside-r.org/r-doc/base/mean"><span>mean</span></a><span>,</span> filename=paste0<span>(</span>out_folder<span>,</span><span>&quot;/&quot;</span><span>,</span>VAR1<span>,</span><span>&quot;_Month&quot;</span><span>,</span>MONTH<span>,</span><span>&quot;.tif&quot;</span><span>)</span><span>)</span><br>   <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;Raster for Month &quot;</span><span>,</span>MONTH<span>,</span><span>&quot; Ready in the Output Folder&quot;</span><span>)</span><span>)</span><br>  <span>}</span><br> <span>}</span> <span>else</span> <span>if</span><span>(</span>time_average==<span>&quot;Yearly Averages&quot;</span><span>)</span> <span>{</span><br>  years &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/unique"><span>unique</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/format"><span>format</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/time">TIME</a><span>,</span> <span>&quot;%Y&quot;</span><span>)</span><span>)</span><br> <br>  <span>#LOOP</span><br>  <span>for</span><span>(</span>YEAR <span>in</span> years<span>)</span><span>{</span><br>   <a target="_blank" href="http://inside-r.org/r-doc/base/subset"><span>subset</span></a> &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/df"><span>df</span></a><span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/format"><span>format</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/stats/df"><span>df</span></a>$TIME<span>,</span> <span>&quot;%Y&quot;</span><span>)</span> == YEAR<span>,</span><span>]</span><br>   sub.var &lt;- var.nc<span>[</span><span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/subset"><span>subset</span></a>$INDEX<span>]</span><span>]</span><br> <br>   <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;Executing Average for Year: &quot;</span><span>,</span>YEAR<span>)</span><span>)</span><br>   av.var &lt;- calc<span>(</span>sub.var<span>,</span> fun=<a target="_blank" href="http://inside-r.org/r-doc/base/mean"><span>mean</span></a><span>,</span> filename=paste0<span>(</span>out_folder<span>,</span><span>&quot;/&quot;</span><span>,</span>VAR1<span>,</span><span>&quot;_Year&quot;</span><span>,</span>YEAR<span>,</span><span>&quot;.tif&quot;</span><span>)</span><span>)</span><br>   <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;Raster for Year &quot;</span><span>,</span>YEAR<span>,</span><span>&quot; Ready in the Output Folder&quot;</span><span>)</span><span>)</span><br>  <span>}</span> <br> <span>}</span> <span>else</span> <span>{</span><br>  hours &lt;- <a target="_blank" href="http://inside-r.org/r-doc/base/unique"><span>unique</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/format"><span>format</span></a><span>(</span><a target="_blank" href="http://inside-r.org/packages/cran/time">TIME</a><span>,</span> <span>&quot;%H&quot;</span><span>)</span><span>)</span><br> <br>  <span>#LOOP</span><br>  <span>for</span><span>(</span>HOUR <span>in</span> hours<span>)</span><span>{</span><br>   <a target="_blank" href="http://inside-r.org/r-doc/base/subset"><span>subset</span></a> &lt;- <a target="_blank" href="http://inside-r.org/r-doc/stats/df"><span>df</span></a><span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/format"><span>format</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/stats/df"><span>df</span></a>$TIME<span>,</span> <span>&quot;%H&quot;</span><span>)</span> == HOUR<span>,</span><span>]</span><br>   sub.var &lt;- var.nc<span>[</span><span>[</span><a target="_blank" href="http://inside-r.org/r-doc/base/subset"><span>subset</span></a>$INDEX<span>]</span><span>]</span><br> <br>   <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;Executing Average for Hour: &quot;</span><span>,</span>HOUR<span>)</span><span>)</span><br>   av.var &lt;- calc<span>(</span>sub.var<span>,</span> fun=<a target="_blank" href="http://inside-r.org/r-doc/base/mean"><span>mean</span></a><span>,</span> filename=paste0<span>(</span>out_folder<span>,</span><span>&quot;/&quot;</span><span>,</span>VAR1<span>,</span><span>&quot;_Hour&quot;</span><span>,</span>HOUR<span>,</span><span>&quot;.tif&quot;</span><span>)</span><span>)</span><br>   <a target="_blank" href="http://inside-r.org/r-doc/base/print"><span>print</span></a><span>(</span><a target="_blank" href="http://inside-r.org/r-doc/base/paste"><span>paste</span></a><span>(</span><span>&quot;Raster for Hour &quot;</span><span>,</span>HOUR<span>,</span><span>&quot; Ready in the Output Folder&quot;</span><span>)</span><span>)</span><br>  <span>}</span> <br> <span>}</span><br> <br> <span>}</span><br><span>}</span></pre>
</div>
</div>
<p><a title="Created by Pretty R at inside-R.org" target="_blank" href="http://www.inside-r.org/pretty-r">Created by Pretty R at inside-R.org</a></p>
<p>As I described in my <a target="_blank" href="http://r-video-tutorial.blogspot.ch/2016/07/combine-arcgis-and-r-clustering-toolbox.html">previous post</a>, the R code for an ArcGIS Toolbox needs to be included in a function that takes inputs and outputs from the ArcGIS GUI.<br>In this function the very first thing we need to do is load the required packages, with an option to install them if necessary. Then we need to assign object names to the inputs and output parameters.<br>As you can see I included quite a few <code>print</code> calls so that ArcGIS users can easily follow the process.</p>
<p>At this point we can move to the NetCDF file. As I mentioned, I will be using the <code>raster</code> package to import the NetCDF file, but first I need to open it directly with the package <code>ncdf4</code> and the function <code>nc_open</code>. This is necessary to obtain the list of variables included in the file. In my tests I downloaded the temperature at 2m above ground and the albedo, therefore the variables’ names were d2m and alb. Since these names are generally not known by end users, we need a way to extract them from the NetCDF file, which is provided by these lines.</p>
<p>Once we have that, we can start a <code>for</code> loop when we iterate through the variables. As you can see the first line within the loop needs to import the nc file with the function <code>brick</code> in the package <code>raster</code>. Within this function we need to specify the name of the variable to use. The raster names include the temporal information we are going to need to create the time averages. For this reason I created an object called <code>TIME</code> with <code>POSIXct</code> values created starting from these names, and then I collected these value into a <code>data.frame</code>. This will be used later on to extract only the indexes of the rasters with the correct date/time.</p>
<p>Now I set up a series of <code>if</code> statements that trigger certain actions depending on what the user selected in the Time Average list on the GUI. Let us assume that the user selected “Daily Averages”.<br>At this point R first uses the function <code>format</code> to extract the days from the <code>data.frame</code> with date/time, named <code>df</code>. Then extracts the <code>unique</code> values from this list. The next step involves iterating through these days and create an average raster for each of these. This can be done with the function <code>calc</code> in <code>raster</code>. This function takes a series of rasters, a function (in this case mean) and can also save a raster object on disk. For the correct address for the output file I simply used the function <code>paste</code> to name the file according to the variable and day. The exact same process is performed for the other time averages.</p>
<p><span>Download</span><br>The R script and Toolbox to perform the time average on ECMWF datasets is available on my GitHub page at:<br><a target="_blank" href="https://github.com/fveronesi/ECMWF_Toolbox/tree/v0.1">https://github.com/fveronesi/ECMWF_Toolbox/tree/v0.1</a></p>
<p>As I said I have other ideas to further enhance this toolbox, thus I created a branch named v0.1. I hope to have time to do these other scripts.</p>
<p></p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/time-averages-of-netcdf-files-from-ecmwf-in-arcgis-with-r-bridge/"></div></div><div><a data-url="http://www.r-bloggers.com/time-averages-of-netcdf-files-from-ecmwf-in-arcgis-with-r-bridge/" data-counturl="http://www.r-bloggers.com/time-averages-of-netcdf-files-from-ecmwf-in-arcgis-with-r-bridge/" data-count="vertical" data-text="Time Averages of NetCDF files from ECMWF in ArcGIS with R-Bridge" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://r-video-tutorial.blogspot.com/2016/07/time-averages-of-netcdf-files-from.html"> R tutorial for Spatial Statistics</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/z6bxIFMKe0Q"></a></div>