<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/reinforcementlearning-a-package-for-replicating-human-behavior-in-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.rblog.uni-freiburg.de/2017/04/08/reinforcementlearning-a-package-for-replicating-human-behavior-in-r/"> R Blog</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h4><em>Nicolas Proellochs and Stefan Feuerriegel</em></h4>
<h4><em>2017-04-06</em></h4>
<div>
<h1>Introduction</h1>
<p>Reinforcement learning has recently gained a great deal of traction in studies that call for human-like learning. In settings where an explicit teacher is not available, this method teaches an agent via interaction with its environment without any supervision other than its own decision-making policy. In many cases, this approach appears quite natural by mimicking the fundamental way humans learn. However, implementing reinforcement learning is programmatically challenging, since it relies on continuous interactions between an agent and its environment. In fact, there is currently no package available that performs model-free reinforcement learning in <em>R</em>. As a remedy, we introduce the <code>ReinforcementLearning</code> <em>R</em> package, which allows an agent to learn optimal behavior based on sample experience consisting of states, actions and rewards. The result of the learning process is a highly interpretable reinforcement learning policy that defines the best possible action in each state.</p>
<p>In the following sections, we present multiple step-by-step examples to illustrate how to take advantage of the capabilities of the <code>ReinforcementLearning</code> package. Moreover, we present methods to customize the learning and action selection behavior of the agent. Main features of <code>ReinforcementLearning</code> include, but are not limited to,</p>
<ul>
<li>Learning an optimal policy from a fixed set of a priori known transition samples</li>
<li>Predefined learning rules and action selection modes</li>
<li>A highly customizable framework for model-free reinforcement learning tasks</li>
</ul>
</div>
<div>
<h1>Reinforcement learning</h1>
<p>Reinforcement learning refers to the problem of an agent that aims to learn optimal behavior through trial-and-error interactions with a dynamic environment. All algorithms for reinforcement learning share the property that the feedback of the agent is restricted to a reward signal that indicates how well the agent is behaving. In contrast to supervised machine learning methods, any instruction concerning how to improve its behavior is absent. Thus, the goal and challenge of reinforcement learning is to improve the behavior of an agent given only this limited type of feedback.</p>
<div>
<h2>The reinforcement learning problem</h2>
<p>In reinforcement learning, the decision-maker, i.e. the agent, interacts with an environment over a sequence of observations and seeks a reward to be maximized over time. Formally, the model consists of a finite set of environment states <em><strong><span>S</span></strong></em>, a finite set of agent actions <em><strong><span>A</span></strong></em>, and a set of scalar reinforcement signals (i.e. rewards) <em><strong><span>R</span></strong></em>. At each iteration<em><strong> <span>i</span></strong></em>, the agent observes some representation of the environment’s state <span>s<sub>i</sub></span> ∈ <em><strong>S</strong></em>. On that basis, the agent selects an action <span><em><strong>a<span><sub>i</sub></span></strong></em> ∈ <em><strong>A(<span>s<sub>i</sub></span>)</strong></em></span>, where <span><em><strong>A(<span>s<sub>i</sub></span>)</strong></em> ⊆ <em><strong>A</strong></em></span> denotes the set of actions available in state<em><strong> <span><span>s<sub>i</sub></span></span></strong></em>. After each iteration, the agent receives a numerical reward <span><em><strong><span>r<sub>i+1</sub></span></strong></em> ∈ <em><strong>R</strong></em></span> and observes a new state <em><strong><span>s<sub>i+1</sub></span></strong></em>.</p>
</div>
<div>
<h2>Policy learning</h2>
<p>In order to store current knowledge, the reinforcement learning method introduces a so-called state-action function <em><strong><span>Q(<span>s<sub>i</sub></span>,<span>a<sub>i</sub></span>)</span></strong></em>, which defines the expected value of each possible action <em><strong><span><span>a<sub>i</sub></span></span></strong></em> in each state<em><strong> <span><span>s<sub>i</sub></span></span></strong></em>. If <em><strong>Q(<span>s<sub>i</sub></span>,<span>a<sub>i</sub></span>)</strong></em> is known, then the optimal policy <em><strong><span>π*(<span>s<sub>i</sub></span>,<span>a<sub>i</sub></span>)</span> </strong></em>is given by the action <em><strong><span><span>a<sub>i</sub></span></span></strong></em>, which maximizes <em><strong><span>Q(<span>s<sub>i</sub></span>,<span>a<sub>i</sub></span>)</span></strong></em> given the state<em><strong> <span><span>s<sub>i</sub></span></span></strong></em>. Consequently, the learning problem of the agent is to maximize the expected reward by learning an optimal policy function <em><strong><span>π*(<span>s<sub>i</sub></span>,<span>a<sub>i</sub></span>)</span>.</strong></em></p>
</div>
<div>
<h2>Experience replay</h2>
<p>Experience replay allows reinforcement learning agents to remember and reuse experiences from the past. The underlying idea is to speed up convergence by replaying observed state transitions repeatedly to the agent, as if they were new observations collected while interacting with a system. Hence, experience replay only requires input data in the form of sample sequences consisting of states, actions and rewards. These data points can be, for example, collected from a running system without the need for direct interaction. The stored training examples then allow the agent to learn a state-action function and an optimal policy for every state transition in the input data. In a next step, the policy can be applied to the system for validation purposes or to collect new data points (e.g. in order to iteratively improve the current policy). As its main advantage, experience replay can speed up convergence by allowing for the back-propagation of information from updated states to preceding states without further interaction.</p>
</div>
</div>
<div>
<h1>Setup of the ReinforcementLearning package</h1>
<p>Even though reinforcement learning has recently gained a great deal of traction in studies that perform human-like learning, the available tools are not living up to the needs of researchers and practitioners. The <code>ReinforcementLearning</code> package is intended to partially close this gap and offers the ability to perform model-free reinforcement learning in a highly customizable framework.</p>
<div>
<h2>Installation</h2>
<p>Using the <code>devtools</code> package, one can easily install the latest development version of <code>ReinforcementLearning</code> as follows.</p>
<div>
<pre><code><span><em>#</em> <em>install.packages(&quot;devtools&quot;)</em>
<em># Option 1: download and install latest version from GitHub</em></span>
<strong>devtools::<span>install_github</span></strong>(<span>&quot;nproellochs/ReinforcementLearning&quot;</span>)

<span><em># Option 2: install directly from bundled archive</em></span>
<strong>devtoos::<span>install_local</span></strong>(<span>&quot;ReinforcementLearning_1.0.0.tar.gz&quot;</span>)</code></pre>
</div>
</div>
<div>
<h2>Package loading</h2>
<p>Afterwards, one merely needs to load the <code>ReinforcementLearning</code> package as follows.</p>
<div>
<pre><code><strong><span>library</span></strong>(ReinforcementLearning)</code></pre>
</div>
</div>
</div>
<div>
<h1>Usage</h1>
<p>The following sections present the usage and main functionality of the <code>ReinforcementLearning</code> package.</p>
<div>
<h2>Data format</h2>
<p>The <code>ReinforcementLearning</code> package uses experience replay to learn an optimal policy based on past experience in the form of sample sequences consisting of states, actions and rewards. Here each training example consists of a state transition tuple <em>(s,a,r,s_new)</em>, as described ibelow.</p>
<ul>
<li><strong>s</strong> The current environment state.</li>
<li><strong>a</strong> The selected action in the current state.</li>
<li><strong>r</strong> The immediate reward received after transitioning from the current state to the next state.</li>
<li><strong>s_new</strong> The next environment state.</li>
</ul>
<p>Note:</p>
<ul>
<li>The input data must be a dataframe in which each row represents a state transition tuple <em>(s,a,r,s_new)</em>.</li>
</ul>
</div>
<div>
<h2>Read-in sample experience</h2>
<p>The state transition tuples can be collected from an external source and easily read-in into <em>R</em>. The sample experience can then be used to train a reinforcement learning agent without requiring further interaction with the environment. The following example shows a representative dataset containing game states of 100,000 randomly sampled Tic-Tac-Toe games.</p>
<div>
<pre><code><strong><span>data</span></strong>(<span>&quot;tictactoe&quot;</span>)
<strong><span>head</span></strong>(tictactoe, <span>5</span>)</code></pre>
</div>
<pre><code>##       State Action NextState Reward
## 1 .........     c7 ......X.B      0
## 2 ......X.B     c6 ...B.XX.B      0
## 3 ...B.XX.B     c2 .XBB.XX.B      0
## 4 .XBB.XX.B     c8 .XBBBXXXB      0
## 5 .XBBBXXXB     c1 XXBBBXXXB      0</code></pre>
</div>
<div>
<h2>Experience sampling using an environment function</h2>
<p>The <code>ReinforcementLearning</code> package is shipped with the built-in capability to sample experience from a function that defines the dynamics of the environment. If the the dynamics of the environment are known a priori, one can set up an arbitrary complex environment function in <em>R</em> and sample state transition tuples. This function has to be manually implemented and must take a state and an action as input. The return value must be a list containing the name of the next state and the reward. As a main advantage, this method of experience sampling allows one to easily validate the performance of reinforcement learning, by applying the learned policy to newly generated samples.</p>
<pre><code>environment &lt;- function(state, action) {
  ...
  return(list(&quot;NextState&quot; = newState,
              &quot;Reward&quot; = reward))
}</code></pre>
<p>The following example illustrates how to generate sample experience using an environment function. Here we collect experience from an agent that navigates from a random starting position to a goal position on a simulated 2×2 grid (see figure below).</p>
<p>|———–|<br>
| s1  | s4  |<br>
| s2     s3 |<br>
|———–|</p>
<p>Each cell on the grid represents one state, which yields a total of 4 states. The grid is surrounded by a wall, which makes it impossible for the agent to move off the grid. In addition, the agent faces a wall between s1 and s4. At each state, the agent randomly chooses one out of four possible actions, i. e. to move up, down, left, or right. The agent encounters the following reward structure: Crossing each square on the grid leads to a reward of -1. If the agent reaches the goal position, it earns a reward of 10.</p>
<div>
<pre><code><span><em><span># Load exemplary environment (gridworld)</span></em></span>
env &lt;- gridworldEnvironment
<strong><span>print</span></strong>(env)</code></pre>
</div>
<pre><code>## function(state, action) {
##   next_state &lt;- state
##   if(state == state(&quot;s1&quot;) &amp;&amp; action == &quot;down&quot;) next_state &lt;- state(&quot;s2&quot;)
##   if(state == state(&quot;s2&quot;) &amp;&amp; action == &quot;up&quot;) next_state &lt;- state(&quot;s1&quot;)
##   if(state == state(&quot;s2&quot;) &amp;&amp; action == &quot;right&quot;) next_state &lt;- state(&quot;s3&quot;)
##   if(state == state(&quot;s3&quot;) &amp;&amp; action == &quot;left&quot;) next_state &lt;- state(&quot;s2&quot;)
##   if(state == state(&quot;s3&quot;) &amp;&amp; action == &quot;up&quot;) next_state &lt;- state(&quot;s4&quot;)
## 
##   if(next_state == state(&quot;s4&quot;) &amp;&amp; state != state(&quot;s4&quot;)) {
##     reward &lt;- 10
##   } else {
##     reward &lt;- -1
##   }
## 
##   out &lt;- list(&quot;NextState&quot; = next_state, &quot;Reward&quot; = reward)
##   return(out)
## }
## &lt;environment: namespace:ReinforcementLearning></code></pre>
<div>
<pre><code><span><em><span># Define state and action sets</span></em></span>
states &lt;- <strong><span>c</span></strong>(<span>&quot;s1&quot;</span>, <span>&quot;s2&quot;</span>, <span>&quot;s3&quot;</span>, <span>&quot;s4&quot;</span>)
actions &lt;-<strong> <span>c</span></strong>(<span>&quot;up&quot;</span>, <span>&quot;down&quot;</span>, <span>&quot;left&quot;</span>, <span>&quot;right&quot;</span>)

<span><em><span># Sample N = 1000 random sequences from the environment</span></em></span>
data &lt;- <strong><span>sampleExperience</span></strong>(<span><span>N</span> <span>=</span></span> <span>1000</span>, <span>env =</span> env, <span>states =</span> states, <span>actions =</span> actions)
<strong><span>head</span></strong>(data)</code></pre>
</div>
<pre><code>##   State Action Reward NextState
## 1    s4   left     -1        s4
## 2    s2  right     -1        s3
## 3    s2  right     -1        s3
## 4    s3   left     -1        s2
## 5    s4     up     -1        s4
## 6    s1   down     -1        s2</code></pre>
</div>
<div>
<h2>Performing reinforcement learning</h2>
<p>The following example shows how to teach a reinforcement learning agent using input data in the form of sample sequences consisting of states, actions and rewards. The ‘data’ argument must be a dataframe object in which each row represents a state transition tuple <em>(s,a,r,s_new)</em>. Moreover, the user is required to specify the column names of the individual tuple elements in ‘data’.</p>
<div>
<pre><code><span><em><span># Define reinforcement learning parameters</span></em></span>
control &lt;- <strong><span>list</span></strong>(<span>alpha =</span><span> 0.1</span>, <span>gamma =</span> <span>0.5</span>, <span>epsilon =</span> <span>0.1</span>)

<span><em><span># Perform reinforcement learning</span></em></span>
model &lt;- <strong><span>ReinforcementLearning</span></strong>(data,<span> s =</span> <span>&quot;State&quot;</span>,<span> a =</span> <span>&quot;Action&quot;</span>, <span>r =</span> <span>&quot;Reward&quot;</span>, <span>s_new =</span> <span>&quot;NextState&quot;</span>, <span>control =</span> control)

<span><em><span># Print result</span></em></span>
<strong><span>print</span></strong>(model)</code></pre>
</div>
<pre><code>## State-Action function Q
##         right         up       down       left
## s1 -0.6633782 -0.6687457  0.7512191 -0.6572813
## s2  3.5806843 -0.6893860  0.7760491  0.7394739
## s3  3.5702779  9.1459425  3.5765323  0.6844573
## s4 -1.8005634 -1.8567931 -1.8244368 -1.8377018
## 
## Policy
##      s1      s2      s3      s4 
##  &quot;down&quot; &quot;right&quot;    &quot;up&quot; &quot;right&quot; 
## 
## Reward (last iteration)
## [1] -263</code></pre>
<p>The result of the learning process is a state-action table and an optimal policy that defines the best possible action in each state.</p>
<div>
<pre><code><span><em><span># Print policy</span></em></span>
<strong><span>policy</span></strong>(model)</code></pre>
</div>
<pre><code>##      s1      s2      s3      s4 
##  &quot;down&quot; &quot;right&quot;    &quot;up&quot; &quot;right&quot;</code></pre>
</div>
<div>
<h2>Updating an existing policy</h2>
<p>Specifying an environment function to model the dynamics of the environment allows one to easily validate the performance of the agent. In order to do this, one simply applies the learned policy to newly generated samples. For this purpose, the <code>ReinforcementLearning</code> package comes with an additional predefined action selection mode, namely ‘epsilon-greedy’. In this strategy, the agent explores the environment by selecting an action at random with probability <em><strong><span>1-ε</span></strong></em>. Alternatively, the agent exploits its current knowledge by choosing the optimal action with probability <em><strong><span>1-ε</span></strong></em>.</p>
<p>The following example shows how to sample new experience from an existing policy using ‘epsilon-greedy’ action selection. The result is new state transition samples (‘data_new’) with significantly higher rewards compared to the the original sample (‘data’).</p>
<div>
<pre><code><span><em><span># Define reinforcement learning parameters</span></em></span>
control &lt;- <strong><span>list</span></strong>(<span>alpha =</span> <span>0.1</span>, <span>gamma =</span> <span>0.5</span>, <span>epsilon =</span> <span>0.1</span>)

<em><span><span># Sample N = 1000 sequences from the environment using epsilon-greedy</span> <span>action selection</span></span></em>
data_new &lt;- <strong><span>sampleExperience</span></strong>(<span>N =</span><span> 1000</span>, <span>env =</span> env, <span>states =</span> states, <span>actions =</span> actions, <span>model =</span> model, <span>actionSelection =</span> <span>&quot;epsilon-greedy&quot;</span>, <span>control =</span> control)
<strong><span>head</span></strong>(data_new)</code></pre>
</div>
<pre><code>##   State Action Reward NextState
## 1    s2  right     -1        s3
## 2    s4  right     -1        s4
## 3    s4  right     -1        s4
## 4    s4  right     -1        s4
## 5    s2  right     -1        s3
## 6    s1   down     -1        s2</code></pre>
<div>
<pre><code><span><em><span># Update the existing policy using new training data</span></em></span>
model_new &lt;- <strong><span>ReinforcementLearning</span></strong>(data_new, <span>s = </span><span>&quot;State&quot;</span>,<span> a =</span> <span>&quot;Action&quot;</span>, <span>r = </span><span>&quot;Reward&quot;</span>, <span>s_new =</span><span> &quot;NextState&quot;</span>, <span>control =</span> control, <span>model =</span> model)

<span><em># Print resul</em>t</span>
<strong><span>print</span></strong>(model_new)</code></pre>
</div>
<pre><code>## State-Action function Q
##        right         up       down       left
## s1 -0.643587 -0.6320560  0.7657318 -0.6314927
## s2  3.530829 -0.6407675  0.7714129  0.7427914
## s3  3.548196  9.0608344  3.5521760  0.7382102
## s4 -1.939574 -1.8922783 -1.8835278 -1.8856132
## 
## Policy
##      s1      s2      s3      s4 
##  &quot;down&quot; &quot;right&quot;    &quot;up&quot;  &quot;down&quot; 
## 
## Reward (last iteration)
## [1] 1211</code></pre>
<div>
<pre><code><strong><span>summary</span></strong>(model_new)</code></pre>
</div>
<pre><code>## 
## Model details
## Learning rule:           experienceReplay
## Learning iterations:     2
## Number of states:        4
## Number of actions:       4
## Total Reward:            1211
## 
## Reward details (per iteration)
## Min:                     -263
## Max:                     1211
## Average:                 474
## Median:                  474
## Standard deviation:      1042.275</code></pre>
<div>
<pre><code><span><em><span># Plot reinforcement learning curve</span></em></span>
<strong><span>plot</span></strong>(model_new)</code></pre>
</div>
<p><img sizes="(max-width: 480px) 85vw, 480px" alt="reinforcement_learning_curve" class="wp-image-112" src="https://i1.wp.com/www.rblog.uni-freiburg.de/wp-content/uploads/2017/04/reinforcement_learning_curve.png?fit=450%2C600" data-recalc-dims="1"></p>
</div>
<h1>Parameter configuration</h1>
<p>The <code>ReinforcementLearning</code> package allows for the adjustment of the following parameters in order to customize the learning behavior of the agent.</p>
<ul>
<li><strong>alpha</strong> The learning rate, set between 0 and 1. Setting it to 0 means that the Q-values are never updated and, hence, nothing is learned. Setting a high value, such as 0.9, means that learning can occur quickly.</li>
<li><strong>gamma</strong> Discount factor, set between 0 and 1. Determines the importance of future rewards. A factor of 0 will render the agent short-sighted by only considering current rewards, while a factor approaching 1 will cause it to strive for a greater reward over the long term.</li>
<li><strong>epsilon</strong> Exploration parameter, set between 0 and 1. Defines the exploration mechanism in <em><strong>ε</strong></em>-greedy action selection. In this strategy, the agent explores the environment by selecting an action at random with probability <em><strong>ε</strong></em>. Alternatively, the agent exploits its current knowledge by choosing the optimal action with probability <em><strong><span>1-ε</span></strong></em>. This parameter is only required for sampling new experience based on an existing policy.</li>
<li><strong>iter</strong> Number of repeated learning iterations. Iter is an integer greater than 0. The default is set to 1.</li>
</ul>
<div>
<pre><code><span><em><span># Define control object</span></em></span>
control &lt;- <strong><span>list</span></strong>(<span>alpha =</span><span> 0.1</span>, <span>gamma =</span> <span>0.1</span>, <span>epsilon =</span> <span>0.1</span>)

<span><em><span># Pass learning parameters to reinforcement learning function</span></em></span>
model &lt;- <strong><span>ReinforcementLearning</span></strong>(data, <span>iter =</span> <span>10</span>, <span>control =</span> control)</code></pre>
</div>
</div>
<div>
<h1>Working example: Learning Tic-Tac-Toe</h1>
<p>The following example shows the use of <code>ReinforcementLearning</code> in an applied setting. More precisely, we utilize a dataset containing 406,541 game states of Tic-Tac-Toe to learn the optimal actions for each state of the board.</p>
<div>
<pre><code><span><em><span># Load dataset</span></em></span>
<strong><span>data</span></strong>(<span>&quot;tictactoe&quot;</span>)

<span><em><span># Define reinforcement learning parameters</span></em></span>
control &lt;- <strong><span>list</span></strong>(<span>alpha =</span> <span>0.2</span>,<span> gamma =</span> <span>0.4</span>, <span>epsilon =</span><span> 0.1</span>)

<span><em><span># Perform reinforcement learning</span></em></span>
model &lt;- <strong><span>ReinforcementLearning</span></strong>(tictactoe,<span> s =</span> <span>&quot;State&quot;</span>, <span>a =</span> <span>&quot;Action&quot;</span>, <span>r =</span> <span>&quot;Reward&quot;</span>, <span>s_new = </span><span>&quot;NextState&quot;</span>, <span>iter = </span><span>1</span>, <span>control =</span> control)

<span><em><span># Print optimal policy</span></em></span>
<strong><span>policy</span></strong>(model)</code></pre>
</div>
<p>Notes:</p>
<ul>
<li>All states are observed from the perspective of player X, who is also assumed to have played first.</li>
<li>The player who succeeds in placing three of their marks in a horizontal, vertical, or diagonal row wins the game. Reward for player X is +1 for ‘win’, 0 for ‘draw’, and -1 for ‘loss’.</li>
</ul>
<p><strong>Download</strong>: <a target="_blank" rel="nofollow" href="https://github.com/nproellochs/ReinforcementLearning">https://github.com/nproellochs/ReinforcementLearning</a></p>
</div>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.rblog.uni-freiburg.de/2017/04/08/reinforcementlearning-a-package-for-replicating-human-behavior-in-r/"> R Blog</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/-V2KDYDxSCo"></a></div>