<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/imagenet-needs-more-wild-boar-photos/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://dgarmat.github.io/Heatmaps-For-Deep-Learning/"> Dan Garmat's Blog -- R</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Is your deep convolutional network misclassifying images? You can find out why with a heatmap of class activation overlaid on its misclassified pictures.</p>
<p>A heatmap overlay shows parts of an image most activated in a neural network’s last convolutional layer. In this African elephant picture, the top-most convolutional layer of the VGG16 architecture turns the photo into a 14×14 grid highlighting blocks with strongest African_elephant activation:</p>
<p><img alt="af_el_1" title="af_el_1" src="https://i2.wp.com/dgarmat.github.io/images/African_elephant_1_overlay.jpg?w=456&ssl=1" data-recalc-dims="1"></p>
<p><span></span></p>
<p>Original image source:<br>
<a rel="nofollow" href="https://www.elephants.com/" target="_blank">elephants.com</a> – <a rel="nofollow" href="https://elephants-media.s3.amazonaws.com/images/399/original/Flora%20ears%20out%20carrying%20limb_0002AA.jpg" target="_blank">African elephant Flora</a></p>
<p>What it’s saying with a yellow-green splotch is “Look! There’s an African elephant here!” The learner returns a score of 46%, quite high for a blink-of-an-eye judgment with 1000 objects to choose from and even locates that object in the picture correctly. Impressive.</p>
<div>
<div>
<pre><code><span>imagenet_decode_predictions</span><span>(</span><span>preds</span><span>,</span><span> </span><span>top</span><span> </span><span>=</span><span> </span><span>3</span><span>)[[</span><span>1</span><span>]]</span><span>
</span><span># class_name class_description score</span><span>
</span><span>#1 n02504458 African_elephant 0.46432969</span><span>
</span><span>#2 n02437312 Arabian_camel 0.29539737</span><span>
</span><span>#3 n01871265 tusker 0.07210348</span><span>
</span></code></pre>
</div>
</div>
<p>Shaded parts of this photo have at least some activation to class African_elephant. These show the elephant’s face and nearby foliage are what distinguish it from an Indian elephant and other classes, like a strawberry or an aircraft carrier. Parts of the photo that have 0 activation on the corresponding heatmap show up as non-shaded, which can be verified from a visualization of the activation heatmap:</p>
<p><img alt="af_el_1_hm" title="af_el_1_hm" src="https://i1.wp.com/dgarmat.github.io/images/African_elephant_1_heatmap.png?w=456&ssl=1" data-recalc-dims="1"></p>
<p>or printing it out as a numeric matrix:</p>
<div>
<div>
<pre><code><span>round</span><span>(</span><span>heatmap</span><span>,</span><span> </span><span>2</span><span>)</span><span>
</span><span># [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]</span><span>
</span><span># [1,] 0.00 0.00 0.02 0.02 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00</span><span>
</span><span># [2,] 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00</span><span>
</span><span># [3,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00</span><span>
</span><span># [4,] 0.00 0.00 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00</span><span>
</span><span># [5,] 0.00 0.00 0.09 0.07 0.26 0.00 0.11 0.13 0.07 0.16 0.00 0.00 0.00 0.00</span><span>
</span><span># [6,] 0.00 0.00 0.01 0.04 0.30 0.24 0.69 0.63 0.41 0.12 0.04 0.00 0.00 0.00</span><span>
</span><span># [7,] 0.00 0.00 0.01 0.04 0.14 0.14 0.55 0.72 0.92 0.23 0.06 0.00 0.00 0.00</span><span>
</span><span># [8,] 0.00 0.00 0.00 0.01 0.00 0.03 0.61 0.98 1.00 0.22 0.00 0.00 0.00 0.00</span><span>
</span><span># [9,] 0.00 0.00 0.02 0.01 0.00 0.00 0.30 0.27 0.31 0.00 0.00 0.00 0.02 0.00</span><span>
</span><span>#[10,] 0.00 0.00 0.04 0.04 0.01 0.00 0.01 0.00 0.02 0.00 0.00 0.02 0.04 0.01</span><span>
</span><span>#[11,] 0.00 0.00 0.10 0.09 0.07 0.06 0.10 0.00 0.00 0.00 0.00 0.03 0.05 0.04</span><span>
</span><span>#[12,] 0.01 0.14 0.13 0.13 0.11 0.11 0.10 0.00 0.00 0.08 0.09 0.12 0.08 0.08</span><span>
</span><span>#[13,] 0.13 0.13 0.15 0.14 0.12 0.10 0.08 0.00 0.03 0.11 0.11 0.15 0.12 0.11</span><span>
</span><span>#[14,] 0.04 0.06 0.06 0.04 0.00 0.00 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00</span><span>
</span></code></pre>
</div>
</div>
<h2>Detecting sources of errors</h2>
<p>Here is another African elephant with huge ears above its neck, but this time the learner has misclassified it as a tusker with a score of 55% as opposed to 17% for African elephant. Tusker isn’t a terrible judgment. It’s a more generic group that includes wild boars but the classification is not as accurate as African elephant. What threw it off from making a more precise call? Let’s see.</p>
<p><img alt="af_el_2" title="af_el_2" src="https://i1.wp.com/dgarmat.github.io/images/African_elephant_2_overlay.jpg?w=456&ssl=1" data-recalc-dims="1"></p>
<p>Original image source:<br>
By <a rel="nofollow" href="https://www.komar.de/en/elephant.html" target="_blank">Komar.de</a> – <a rel="nofollow" href="https://www.komar.de/en/media/catalog/product/cache/5/image/780x/17f82f742ffe127f42dca9de82fb58b1/x/x/xxl4-529_elephant_ma.jpg" target="_blank">Non-woven photomural Elephant</a></p>
<p>Looks like the top of the head and the back. Surprising it’s not the tusks. If we take <a rel="nofollow" href="http://imagenet.stanford.edu/synset?wnid=n01871265" target="_blank">a sample of ImageNet tusker training images</a>, it quickly becomes obvious most tusker images are of elephants. In the first 25 tusker examples shown here, none look like wild boars.</p>
<p><img alt="tuskers" title="tuskers" src="https://i1.wp.com/dgarmat.github.io/images/tusker_imagenet.JPG?w=456&ssl=1" data-recalc-dims="1"></p>
<p>So the cause of our misclassification is understandable, and a training set limitation error. A great first recourse would be to add to ImageNet other kinds of tuskers to better train that class.</p>
<div>
<div>
<pre><code>imagenet_decode_predictions(preds, top = 3)[[1]]
#  class_name class_description     score
#1  n01871265            tusker 0.5496630
#2  n02504013   Indian_elephant 0.2749955
#3  n02504458  African_elephant 0.1732897
</code></pre>
</div>
</div>
<h2>R Code</h2>
<div>
<div>
<pre><code><span>library</span><span>(</span><span>keras</span><span>)</span><span>
</span><span>library</span><span>(</span><span>magick</span><span>)</span><span>
</span><span>library</span><span>(</span><span>viridis</span><span>)</span><span>
</span><span>model</span><span> </span><span>&lt;-</span><span> </span><span>application_vgg16</span><span>(</span><span>weights</span><span> </span><span>=</span><span> </span><span>&quot;imagenet&quot;</span><span>)</span><span> </span><span># keeping top</span><span>
</span><span>model</span><span> </span><span># assumes input picture of size 224 x 224</span><span>
</span><span>img_path</span><span> </span><span>&lt;-</span><span> </span><span>&quot;images/African_elephant_1.jpg&quot;</span><span>
</span><span>img</span><span> </span><span>&lt;-</span><span> </span><span>image_load</span><span>(</span><span>img_path</span><span>,</span><span> </span><span>target_size</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>224</span><span>,</span><span> </span><span>224</span><span>))</span><span> </span><span>%>%</span><span> </span><span>image_to_array</span><span>()</span><span> </span><span>%>%</span><span> </span><span>array_reshape</span><span>(</span><span>dim</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>1</span><span>,</span><span> </span><span>224</span><span>,</span><span> </span><span>224</span><span>,</span><span> </span><span>3</span><span>))</span><span> </span><span>%>%</span><span> </span><span># for batch of this size</span><span>
</span><span>imagenet_preprocess_input</span><span>()</span><span> </span><span># channelwise color normalization</span><span>
</span><span>preds</span><span> </span><span>&lt;-</span><span> </span><span>model</span><span> </span><span>%>%</span><span> </span><span>predict</span><span>(</span><span>img</span><span>)</span><span>
</span><span>imagenet_decode_predictions</span><span>(</span><span>preds</span><span>,</span><span> </span><span>top</span><span> </span><span>=</span><span> </span><span>3</span><span>)[[</span><span>1</span><span>]]</span><span>
</span><span># get least likely classes for fun</span><span>
</span><span>tail</span><span>(</span><span>imagenet_decode_predictions</span><span>(</span><span>preds</span><span>,</span><span> </span><span>top</span><span> </span><span>=</span><span> </span><span>1000</span><span>)[[</span><span>1</span><span>]])</span><span>
</span><span>max_class_nbr</span><span> </span><span>&lt;-</span><span> </span><span>which.max</span><span>(</span><span>preds</span><span>[</span><span>1</span><span>,</span><span> </span><span>])</span><span> </span><span># is the class index</span><span>
</span><span># if want to see second most class activations, get at which index</span><span>
</span><span>second_class_nbr</span><span> </span><span>&lt;-</span><span> </span><span>which.max</span><span>((</span><span>preds</span><span>[</span><span>1</span><span>,</span><span> </span><span>])[</span><span>-</span><span>max_class_nbr</span><span>])</span><span> </span><span># should be second </span><span>
</span><span># add +1 if above the previous index number</span><span>
</span><span>second_class_nbr</span><span> </span><span>&lt;-</span><span> </span><span>ifelse</span><span>(</span><span>second_class_nbr</span><span> </span><span>>=</span><span> </span><span>max_class_nbr</span><span>,</span><span> </span><span>second_class_nbr</span><span> </span><span>+</span><span> </span><span>1</span><span>,</span><span> </span><span>second_class_nbr</span><span>)</span><span>
</span><span># visualize which parts of the image are most class 1 using Grad-CAM</span><span>
</span><span>elephant_output</span><span> </span><span>&lt;-</span><span> </span><span>model</span><span>$</span><span>output</span><span>[,</span><span> </span><span>max_class_nbr</span><span>]</span><span>
</span><span>elephant_output</span><span> </span><span>&lt;-</span><span> </span><span>model</span><span>$</span><span>output</span><span>[,</span><span> </span><span>second_class_nbr</span><span>]</span><span>
</span><span>last_conv_layer</span><span> </span><span>&lt;-</span><span> </span><span>model</span><span> </span><span>%>%</span><span> </span><span>get_layer</span><span>(</span><span>&quot;block5_conv3&quot;</span><span>)</span><span>
</span><span>grads</span><span> </span><span>&lt;-</span><span> </span><span>k_gradients</span><span>(</span><span>elephant_output</span><span>,</span><span> </span><span>last_conv_layer</span><span>$</span><span>output</span><span>)[[</span><span>1</span><span>]]</span><span>
</span><span>pooled_grads</span><span> </span><span>&lt;-</span><span> </span><span>k_mean</span><span>(</span><span>grads</span><span>,</span><span> </span><span>axis</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>1</span><span>,</span><span> </span><span>2</span><span>,</span><span> </span><span>3</span><span>))</span><span>
</span><span>iterate</span><span> </span><span>&lt;-</span><span> </span><span>k_function</span><span>(</span><span>list</span><span>(</span><span>model</span><span>$</span><span>input</span><span>),</span><span>
</span><span>list</span><span>(</span><span>pooled_grads</span><span>,</span><span> </span><span>last_conv_layer</span><span>$</span><span>output</span><span>[</span><span>1</span><span>,,,]))</span><span>
</span><span>c</span><span>(</span><span>pooled_grads_value</span><span>,</span><span> </span><span>conv_layer_output_value</span><span>)</span><span> </span><span>%&lt;-%</span><span> </span><span>iterate</span><span>(</span><span>list</span><span>(</span><span>img</span><span>))</span><span>
</span><span>for</span><span>(</span><span>i</span><span> </span><span>in</span><span> </span><span>1</span><span>:</span><span>dim</span><span>(</span><span>conv_layer_output_value</span><span>)[</span><span>3</span><span>]){</span><span>
</span><span>conv_layer_output_value</span><span>[,,</span><span>i</span><span>]</span><span> </span><span>&lt;-</span><span> </span><span>conv_layer_output_value</span><span>[,,</span><span>i</span><span>]</span><span> </span><span>*</span><span> </span><span>pooled_grads_value</span><span>[[</span><span>i</span><span>]]</span><span>
</span><span>}</span><span>
</span><span>heatmap</span><span> </span><span>&lt;-</span><span> </span><span>apply</span><span>(</span><span>conv_layer_output_value</span><span>,</span><span> </span><span>c</span><span>(</span><span>1</span><span>,</span><span> </span><span>2</span><span>),</span><span> </span><span>mean</span><span>)</span><span>
</span><span># normalize heatmap between 0 and 1</span><span>
</span><span>heatmap</span><span> </span><span>&lt;-</span><span> </span><span>pmax</span><span>(</span><span>heatmap</span><span>,</span><span> </span><span>0</span><span>)</span><span>
</span><span>heatmap</span><span> </span><span>&lt;-</span><span> </span><span>heatmap</span><span> </span><span>/</span><span> </span><span>max</span><span>(</span><span>heatmap</span><span>)</span><span>
</span><span>round</span><span>(</span><span>heatmap</span><span>,</span><span> </span><span>2</span><span>)</span><span>
</span><span>write_heatmap</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>heatmap</span><span>,</span><span> </span><span>filename</span><span>,</span><span> </span><span>width</span><span> </span><span>=</span><span> </span><span>224</span><span>,</span><span> </span><span>height</span><span> </span><span>=</span><span> </span><span>224</span><span>,</span><span>
</span><span>bg</span><span> </span><span>=</span><span> </span><span>&quot;white&quot;</span><span>,</span><span> </span><span>col</span><span> </span><span>=</span><span> </span><span>terrain.colors</span><span>(</span><span>12</span><span>)){</span><span>
</span><span>png</span><span>(</span><span>filename</span><span>,</span><span> </span><span>width</span><span> </span><span>=</span><span> </span><span>width</span><span>,</span><span> </span><span>height</span><span> </span><span>=</span><span> </span><span>height</span><span>,</span><span> </span><span>bg</span><span> </span><span>=</span><span> </span><span>bg</span><span>)</span><span>
</span><span>op</span><span> </span><span>=</span><span> </span><span>par</span><span>(</span><span>mar</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>0</span><span>,</span><span> </span><span>0</span><span>,</span><span> </span><span>0</span><span>,</span><span> </span><span>0</span><span>))</span><span>
</span><span>on.exit</span><span>({</span><span>par</span><span>(</span><span>op</span><span>);</span><span> </span><span>dev.off</span><span>()},</span><span> </span><span>add</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span>
</span><span>rotate</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>x</span><span>)</span><span> </span><span>t</span><span>(</span><span>apply</span><span>(</span><span>x</span><span>,</span><span> </span><span>2</span><span>,</span><span> </span><span>rev</span><span>))</span><span>
</span><span>image</span><span>(</span><span>rotate</span><span>(</span><span>heatmap</span><span>),</span><span> </span><span>axes</span><span> </span><span>=</span><span> </span><span>FALSE</span><span>,</span><span> </span><span>asp</span><span> </span><span>=</span><span> </span><span>1</span><span>,</span><span> </span><span>col</span><span> </span><span>=</span><span> </span><span>col</span><span>)</span><span>
</span><span>}</span><span>
</span><span>write_heatmap</span><span>(</span><span>heatmap</span><span>,</span><span> </span><span>paste0</span><span>(</span><span>substr</span><span>(</span><span>img_path</span><span>,</span><span> </span><span>1</span><span>,</span><span> </span><span>nchar</span><span>(</span><span>img_path</span><span>)</span><span> </span><span>-</span><span> </span><span>4</span><span>),</span><span> </span><span>&quot;_heatmap.png&quot;</span><span>))</span><span>
</span><span>image</span><span> </span><span>&lt;-</span><span> </span><span>image_read</span><span>(</span><span>img_path</span><span>)</span><span>
</span><span>info</span><span> </span><span>&lt;-</span><span> </span><span>image_info</span><span>(</span><span>image</span><span>)</span><span>
</span><span>geometry</span><span> </span><span>&lt;-</span><span> </span><span>sprintf</span><span>(</span><span>&quot;%dx%d!&quot;</span><span>,</span><span> </span><span>info</span><span>$</span><span>width</span><span>,</span><span> </span><span>info</span><span>$</span><span>height</span><span>)</span><span>
</span><span>pal</span><span> </span><span>&lt;-</span><span> </span><span>col2rgb</span><span>(</span><span>viridis</span><span>(</span><span>20</span><span>),</span><span> </span><span>alpha</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span>
</span><span>alpha</span><span> </span><span>&lt;-</span><span> </span><span>floor</span><span>(</span><span>seq</span><span>(</span><span>0</span><span>,</span><span> </span><span>255</span><span>,</span><span> </span><span>length</span><span> </span><span>=</span><span> </span><span>ncol</span><span>(</span><span>pal</span><span>)))</span><span>
</span><span>pal_col</span><span> </span><span>&lt;-</span><span> </span><span>rgb</span><span>(</span><span>t</span><span>(</span><span>pal</span><span>),</span><span> </span><span>alpha</span><span> </span><span>=</span><span> </span><span>alpha</span><span>,</span><span> </span><span>maxColorValue</span><span> </span><span>=</span><span> </span><span>255</span><span>)</span><span>
</span><span>write_heatmap</span><span>(</span><span>heatmap</span><span>,</span><span> </span><span>&quot;elephant_overlay.png&quot;</span><span>,</span><span>
</span><span>width</span><span> </span><span>=</span><span> </span><span>dim</span><span>(</span><span>heatmap</span><span>)[</span><span>1</span><span>],</span><span> </span><span>height</span><span> </span><span>=</span><span> </span><span>dim</span><span>(</span><span>heatmap</span><span>)[</span><span>2</span><span>],</span><span> </span><span>bg</span><span> </span><span>=</span><span> </span><span>NA</span><span>,</span><span> </span><span>col</span><span> </span><span>=</span><span> </span><span>pal_col</span><span>)</span><span>
</span><span>image_read</span><span>(</span><span>&quot;elephant_overlay.png&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>image_resize</span><span>(</span><span>geometry</span><span>,</span><span> </span><span>filter</span><span> </span><span>=</span><span> </span><span>&quot;quadratic&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>image_composite</span><span>(</span><span>image</span><span>,</span><span> </span><span>operator</span><span> </span><span>=</span><span> </span><span>&quot;blend&quot;</span><span>,</span><span> </span><span>compose_args</span><span> </span><span>=</span><span> </span><span>&quot;20&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>plot</span><span>()</span><span>
</span><span># then save output</span><span>
</span><span>image_read</span><span>(</span><span>&quot;elephant_overlay.png&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>image_resize</span><span>(</span><span>geometry</span><span>,</span><span> </span><span>filter</span><span> </span><span>=</span><span> </span><span>&quot;quadratic&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>image_composite</span><span>(</span><span>image</span><span>,</span><span> </span><span>operator</span><span> </span><span>=</span><span> </span><span>&quot;blend&quot;</span><span>,</span><span> </span><span>compose_args</span><span> </span><span>=</span><span> </span><span>&quot;20&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>image_scale</span><span>(</span><span>&quot;x480&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>image_convert</span><span>(</span><span>format</span><span> </span><span>=</span><span> </span><span>&quot;jpg&quot;</span><span>)</span><span> </span><span>%>%</span><span> </span><span>image_write</span><span>(</span><span>paste0</span><span>(</span><span>substr</span><span>(</span><span>img_path</span><span>,</span><span> </span><span>1</span><span>,</span><span> </span><span>nchar</span><span>(</span><span>img_path</span><span>)</span><span> </span><span>-</span><span> </span><span>4</span><span>),</span><span> </span><span>&quot;_overlay.jpg&quot;</span><span>))</span><span>
</span><span># reset the image to second elephant</span><span>
</span><span>img_path</span><span> </span><span>&lt;-</span><span> </span><span>&quot;images/African_elephant_2.jpg&quot;</span><span>
</span><span># then rerun the above from img &lt;-</span><span>
</span></code></pre>
</div>
</div>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://dgarmat.github.io/Heatmaps-For-Deep-Learning/"> Dan Garmat's Blog -- R</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/gIBAdcvNKwY" height="1"></a></div>