<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/beyond-univariate-single-sample-data-with-mcht/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://ntguardian.wordpress.com/2018/11/05/beyond-univariate-single-sample-data-mcht/"> R – Curtis Miller's Personal Website</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2>Introduction</h2>
<p>I’ve spent the past few weeks writing about <a rel="nofollow" href="https://wp.me/p27liZ-TD" target="_blank"><strong>MCHT</strong></a>, my new package for Monte Carlo and bootstrap hypothesis testing. After discussing how to use <strong>MCHT</strong> <a rel="nofollow" href="https://wp.me/p27liZ-TG" target="_blank">safely</a>, I discussed how to use it for <a rel="nofollow" href="https://wp.me/p27liZ-TQ" target="_blank">maximized Monte Carlo (MMC) testing</a>, then <a rel="nofollow" href="https://wp.me/p27liZ-TU" target="_blank">bootstrap testing</a>. One may think I’ve said all I want to say about the package, but in truth, I’ve only barely passed the halfway point!</p>
<p><span></span></p>
<p>Today I’m demonstrating how general <strong>MCHT</strong> is, allowing one to use it for multiple samples and on non-univariate data. I’ll be doing so with two examples: a permutation test and the <img alt="F" title="F" src="https://s0.wp.com/latex.php?latex=F&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> test for significance of a regression model.</p>
<h2>Permutation Test</h2>
<p>The idea of the permutation test dates back to Fisher (see [1]) and it forms the basis of computational testing for difference in mean. Let’s suppose that we have two samples with respective means <img alt="mu_X" title="mu_X" src="https://s0.wp.com/latex.php?latex=%5Cmu_X&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> and <img alt="mu_Y" title="mu_Y" src="https://s0.wp.com/latex.php?latex=%5Cmu_Y&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">, respectively. Suppose we wish to test</p>
<p><img alt="H_0: mu_X = mu_Y" title="H_0: mu_X = mu_Y" src="https://s0.wp.com/latex.php?latex=H_0%3A+%5Cmu_X+%3D+%5Cmu_Y&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p>against</p>
<p><img alt="H_A: mu_X &gt; mu_Y" title="H_A: mu_X &gt; mu_Y" src="https://s0.wp.com/latex.php?latex=H_A%3A+%5Cmu_X+%3E+%5Cmu_Y&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p>using samples <img alt="X_1, ldots, X_m" title="X_1, ldots, X_m" src="https://s0.wp.com/latex.php?latex=X_1%2C+%5Cldots%2C+X_m&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> and <img alt="Y_1, ldots, Y_n" title="Y_1, ldots, Y_n" src="https://s0.wp.com/latex.php?latex=Y_1%2C+%5Cldots%2C+Y_n&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">, respectively.</p>
<p>If the null hypothesis is true and we also make the stronger assumption that the two samples were drawn from distributions that could differ only in their means, then the labelling of the two samples is artificial, and if it were removed the two samples would be indistinguishable. Relabelling the data and artificially calling one sample the <img alt="X" title="X" src="https://s0.wp.com/latex.php?latex=X&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> sample and the other the <img alt="Y" title="Y" src="https://s0.wp.com/latex.php?latex=Y&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> sample would produce highly similar statistics to the one we actually observed. This observation suggests the following procedure:</p>
<ol>
<li>Generate <img alt="N" title="N" src="https://s0.wp.com/latex.php?latex=N&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> new datasets by randomly assigning labels to the combined sample of <img alt="X_1, ldots, X_m" title="X_1, ldots, X_m" src="https://s0.wp.com/latex.php?latex=X_1%2C+%5Cldots%2C+X_m&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> and <img alt="Y_1, ldots, Y_n" title="Y_1, ldots, Y_n" src="https://s0.wp.com/latex.php?latex=Y_1%2C+%5Cldots%2C+Y_n&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">.</li>
<li>Compute copies of the test statistic on each of the new samples; suppose that the test statistic used is the difference in means, <img alt="bar{X} -  bar{Y}" title="bar{X} -  bar{Y}" src="https://s0.wp.com/latex.php?latex=%5Cbar%7BX%7D+-++%5Cbar%7BY%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">.</li>
<li>Compute the test statistic on the actual sample and compare to the simulated statistics. If the actual statistic is relatively large compared to the simulated statistics, then reject the null hypothesis in favor of the alternative; otherwise, don’t reject.</li>
</ol>
<p>In practice step 3 is done by computing a <img alt="p" title="p" src="https://s0.wp.com/latex.php?latex=p&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">-value representing the proportion of simulated statistics larger than the one actually computed.</p>
<h2>Permutation Tests Using MCHT</h2>
<p>The permutation test is effectively a bootstrap test, so it is supported by <strong>MCHT</strong>, though one may wonder how that’s the case when the parameters <code>test_stat</code>, <code>stat_gen</code>, and <code>rand_gen</code> only accept one parameter, <code>x</code>, representing the dataset (as opposed to, say, <code>t.test()</code>, which has an <code>x</code> and an optional <code>y</code> parameter). But <code>MCHTest()</code> makes very few assumptions about what object <code>x</code> actually is; if your object is either a vector or tabular, then the <code>MCHTest</code> object should not have a problem with it (it’s even possible a loosely structured <code>list</code> would be fine, but I have not tested this; tabular formats should cover most use cases).</p>
<p>In this case, putting our data in <a rel="nofollow" href="https://www.theanalysisfactor.com/wide-and-long-data/" target="_blank">long-form format</a> makes doing a permutation test fairly simple. One column will contain the group an observation belongs to while the other contains observation values. The <code>test_stat</code> function will split the data according to group, compute group-wise means, and finally compute the test statistic. <code>rand_gen</code> generates new dataset by permuting the labels in the data frame. <code>stat_gen</code> merely serves as the glue between the two.</p>
<p>The result is the following test.</p>
<pre>
library(MCHT)
library(doParallel)

registerDoParallel(detectCores())

ts &lt;- function(x) {
  grp_means &lt;- aggregate(value ~ group, data = x, FUN = mean)

  grp_means$value[1] - grp_means$value[2]
}

rg &lt;- function(x) {
  x$group &lt;- sample(x$group)
  x
}

sg &lt;- function(x) {
  test_stat(x)
}

permute.test &lt;- MCHTest(ts, sg, rg, seed = 123, N = 1000,
                        localize_functions = TRUE)

df &lt;- data.frame(&quot;value&quot; = c(rnorm(5, 2, 1), rnorm(10, 0, 1)),
                 &quot;group&quot; = rep(c(&quot;x&quot;, &quot;y&quot;), times = c(5, 10)))

permute.test(df)
</pre>
<pre>
## 
## 	Monte Carlo Test
## 
## data:  df
## S = 1.3985, p-value = 0.036
</pre>
<h2>Linear Regression F Test</h2>
<p>Suppose for each observation in our dataset there is an outcome of interest, <img alt="y_i" title="y_i" src="https://s0.wp.com/latex.php?latex=y_i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">, and there are variables <img alt="x_{1,i}, ldots, x_{p,i}" title="x_{1,i}, ldots, x_{p,i}" src="https://s0.wp.com/latex.php?latex=x_%7B1%2Ci%7D%2C+%5Cldots%2C+x_%7Bp%2Ci%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> that could together help predict the value of <img alt="y_i" title="y_i" src="https://s0.wp.com/latex.php?latex=y_i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> if they are known. Consider then the following linear regression model (with <img alt="E[epsilon_i] = 0" title="E[epsilon_i] = 0" src="https://s0.wp.com/latex.php?latex=E%5B%5Cepsilon_i%5D+%3D+0&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">):</p>
<p><img alt="y_i = beta_0 + beta_1 x_{1,i} + ldots + beta_p x_{p,i} + epsilon_i" title="y_i = beta_0 + beta_1 x_{1,i} + ldots + beta_p x_{p,i} + epsilon_i" src="https://s0.wp.com/latex.php?latex=y_i+%3D+%5Cbeta_0+%2B+%5Cbeta_1+x_%7B1%2Ci%7D+%2B+%5Cldots+%2B+%5Cbeta_p+x_%7Bp%2Ci%7D+%2B+%5Cepsilon_i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p>The first question someone should asked when considering a regression model is whether it’s worth anything at all. An alternative approach to predicting <img alt="y_i" title="y_i" src="https://s0.wp.com/latex.php?latex=y_i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> is simply to predict its mean value. That is, the model</p>
<p><img alt="y_i = beta_0 + epsilon_i" title="y_i = beta_0 + epsilon_i" src="https://s0.wp.com/latex.php?latex=y_i+%3D+%5Cbeta_0+%2B+%5Cepsilon_i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p>is much simpler and should be preferred to the more complicated model listed above if it’s just as good at explaining the behavior of <img alt="y_i" title="y_i" src="https://s0.wp.com/latex.php?latex=y_i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> for all <img alt="i" title="i" src="https://s0.wp.com/latex.php?latex=i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">. Notice the second model is simply the first model with all the coefficients <img alt="beta_1, ldots, beta_p" title="beta_1, ldots, beta_p" src="https://s0.wp.com/latex.php?latex=%5Cbeta_1%2C+%5Cldots%2C+%5Cbeta_p&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> identically equal to zero.</p>
<p>The <img alt="F" title="F" src="https://s0.wp.com/latex.php?latex=F&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">-test (described in more detail <a rel="nofollow" href="https://en.wikipedia.org/wiki/F-test" target="_blank">here</a>) can help us decide between these two competing models. Under the null hypothesis, the second model is the true model.</p>
<p><img alt="H_0: beta_1 = ldots = beta_p = 0" title="H_0: beta_1 = ldots = beta_p = 0" src="https://s0.wp.com/latex.php?latex=H_0%3A+%5Cbeta_1+%3D+%5Cldots+%3D+%5Cbeta_p+%3D+0&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p>The alternative says that at least one of the regressors is helpful in predicting <img alt="y_i" title="y_i" src="https://s0.wp.com/latex.php?latex=y_i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">.</p>
<p><img alt="H_A: H_0 text{ is false}" title="H_A: H_0 text{ is false}" src="https://s0.wp.com/latex.php?latex=H_A%3A+H_0+%5Ctext%7B+is+false%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p>We can use the <img alt="F" title="F" src="https://s0.wp.com/latex.php?latex=F&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> statistic to decide between the two models:</p>
<p><img alt="F = frac{frac{text{RSS}_2 - text{RSS}_1}{p - 1}}{frac{text{RSS}_1}{n - p}}" title="F = frac{frac{text{RSS}_2 - text{RSS}_1}{p - 1}}{frac{text{RSS}_1}{n - p}}" src="https://s0.wp.com/latex.php?latex=F+%3D+%5Cfrac%7B%5Cfrac%7B%5Ctext%7BRSS%7D_2+-+%5Ctext%7BRSS%7D_1%7D%7Bp+-+1%7D%7D%7B%5Cfrac%7B%5Ctext%7BRSS%7D_1%7D%7Bn+-+p%7D%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p><img alt="text{RSS}_1" title="text{RSS}_1" src="https://s0.wp.com/latex.php?latex=%5Ctext%7BRSS%7D_1&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> and <img alt="text{RSS}_2" title="text{RSS}_2" src="https://s0.wp.com/latex.php?latex=%5Ctext%7BRSS%7D_2&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> are the <a rel="nofollow" href="https://en.wikipedia.org/wiki/Residual_sum_of_squares" target="_blank">residual sum of squares</a> of models 1 and<br>
2, respectively.</p>
<p>This test is called the <img alt="F" title="F" src="https://s0.wp.com/latex.php?latex=F&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">-test because usually the <a rel="nofollow" href="https://en.wikipedia.org/wiki/F-distribution" target="_blank">F-distribution</a> is used to compute <img alt="p" title="p" src="https://s0.wp.com/latex.php?latex=p&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex">-values (as this is the distributiont the <img alt="F" title="F" src="https://s0.wp.com/latex.php?latex=F&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> statistic should follow when certain conditions hold, at least asymptotically if not exactly). What then would a bootstrap-based procedure look like?</p>
<p>If the null hypothesis is true then the best model for the data is this:</p>
<p><img alt="y_i = bar{y} + hat{epsilon}_i" title="y_i = bar{y} + hat{epsilon}_i" src="https://s0.wp.com/latex.php?latex=y_i+%3D+%5Cbar%7By%7D+%2B+%5Chat%7B%5Cepsilon%7D_i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"></p>
<p><img alt="bar{y}" title="bar{y}" src="https://s0.wp.com/latex.php?latex=%5Cbar%7By%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> is the sample mean of <img alt="y" title="y" src="https://s0.wp.com/latex.php?latex=y&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> and <img alt="hat{epsilon}_i = y_i - bar{y}" title="hat{epsilon}_i = y_i - bar{y}" src="https://s0.wp.com/latex.php?latex=%5Chat%7Bepsilon%7D_i+%3D+y_i+-+%5Cbar%7By%7D&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> is the residual. This suggests the following procedure:</p>
<ol>
<li>Shuffle <img alt="y_i" title="y_i" src="https://s0.wp.com/latex.php?latex=y_i&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> over all rows of the input dataset, with replacement, to generate new datasets.</li>
<li>Compute <img alt="F" title="F" src="https://s0.wp.com/latex.php?latex=F&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> statistics for each of the generated datasets.</li>
<li>Compare the <img alt="F" title="F" src="https://s0.wp.com/latex.php?latex=F&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> statistic of the actual dataset to the generated datasets’ statistics.</li>
</ol>
<h2>F Test Using MCHT</h2>
<p>Let’s perform the <img alt="F" title="F" src="https://s0.wp.com/latex.php?latex=F&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> test on a subset of the <a rel="nofollow" href="https://en.wikipedia.org/wiki/Iris_flower_data_set" target="_blank"><code>iris</code></a> dataset. We will see if there is a relationship between the sepal length and sepal width among <a rel="nofollow" href="https://en.wikipedia.org/wiki/Iris_setosa" target="_blank"><em>iris setosa</em></a> flowers. Below is an initial split and visualization:</p>
<pre>
library(dplyr)

setosa &lt;- iris %>% filter(Species == &quot;setosa&quot;) %>%
            select(Sepal.Length, Sepal.Width)
plot(Sepal.Width ~ Sepal.Length, data = setosa)
</pre>
<p><img alt="plot of chunk unnamed-chunk-2" src="https://ntguardian.files.wordpress.com/2018/10/unnamed-chunk-2-1.png?w=456" data-recalc-dims="1"></p>
<p>There is an obvious relationship between the variables. Thus we should expect the test to reject the null hypothesis. That is what we would conclude if we were to run the conventional <img alt="F" title="F" src="https://s0.wp.com/latex.php?latex=F&bg=ffffff&%23038;fg=444444&%23038;s=0" class="latex"> test:</p>
<pre>
res &lt;- lm(Sepal.Width ~ Sepal.Length, data = setosa)
summary(res)
</pre>
<pre>
## 
## Call:
## lm(formula = Sepal.Width ~ Sepal.Length, data = setosa)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.72394 -0.18273 -0.00306  0.15738  0.51709 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   -0.5694     0.5217  -1.091    0.281    
## Sepal.Length   0.7985     0.1040   7.681 6.71e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.2565 on 48 degrees of freedom
## Multiple R-squared:  0.5514,	Adjusted R-squared:  0.542 
## F-statistic: 58.99 on 1 and 48 DF,  p-value: 6.71e-10
</pre>
<p>Let’s now implement the procedure I described with <code>MCHTest()</code>.</p>
<pre>
ts &lt;- function(x) {
  res &lt;- lm(Sepal.Width ~ Sepal.Length, data = x)
  summary(res)$fstatistic[[1]]  # Only way I know to automatically compute the
                                # statistic
}

# rand_gen's function can use both x and n, and n will be the number of rows of
# the dataset
rg &lt;- function(x, n) {
  x$Sepal.Width &lt;- sample(x$Sepal.Width, replace = TRUE, size = n)
  x
}

b.f.test.1 &lt;- MCHTest(ts, ts, rg, seed = 123, N = 1000)

b.f.test.1(setosa)
</pre>
<pre>
## 
## 	Monte Carlo Test
## 
## data:  setosa
## S = 58.994, p-value &lt; 2.2e-16
</pre>
<p>Excellent! It reached the correct conclusion.</p>
<h2>Conclusion</h2>
<p>One may naturally ask whether we can write functions a bit more general than what I’ve shown here at least in the regression context. For example, one may want parameters specifying a formula so that the regression model isn’t hard-coded into the test. In short, the answer is yes; <code>MCHTest</code> objects try to pass as many parameters to the input functions as they can.</p>
<p>Here is the revised example that works for basically any formula:</p>
<pre>
ts &lt;- function(x, formula) {
  res &lt;- lm(formula = formula, data = x)
  summary(res)$fstatistic[[1]]
}

rg &lt;- function(x, n, formula) {
  dep_var &lt;- all.vars(formula)[1]  # Get the name of the dependent variable
  x[[dep_var]] &lt;- sample(x[[dep_var]], replace = TRUE, size = n)
  x
}

b.f.test.2 &lt;- MCHTest(ts, ts, rg, seed = 123, N = 1000)

b.f.test.2(setosa, formula = Sepal.Width ~ Sepal.Length)
</pre>
<pre>
## 
## 	Monte Carlo Test
## 
## data:  setosa
## S = 58.994, p-value &lt; 2.2e-16
</pre>
<p>This shows that you can have a lot of control over how <code>MCHTest</code> objects handle their inputs, giving you considerable flexibility.</p>
<p><a rel="nofollow" href="https://wp.me/p27liZ-U5" target="_blank">Next post: time series and <strong>MCHT</strong></a></p>
<h2>References</h2>
<ol>
<li>R. A. Fisher, <em>The design of experiments</em> (1935)</li>
</ol>
<hr>
<p><a rel="nofollow" href="https://www.packtpub.com/" target="_blank">Packt Publishing</a> published a book for me entitled <a rel="nofollow" href="https://ntguardian.wordpress.com/books/hands-on-data-analysis-with-numpy-and-pandas/" target="_blank"><em>Hands-On Data Analysis with NumPy and Pandas</em></a>, a book based on my video course <a rel="nofollow" href="https://ntguardian.wordpress.com/video-courses/unpacking-numpy-and-pandas/" target="_blank"><em>Unpacking NumPy and Pandas</em></a>. This book covers the basics of setting up a Python environment for data analysis with Anaconda, using Jupyter notebooks, and using NumPy and pandas. If you are starting out using Python for data analysis or know someone who is, please consider <a rel="nofollow" href="https://www.packtpub.com/big-data-and-business-intelligence/hands-data-analysis-numpy-and-pandas" target="_blank">buying my book</a> or at least spreading the word about it. You can buy the book directly or purchase a <a rel="nofollow" href="https://mapt.packtpub.com/?gclid=Cj0KCQjwuuHdBRCvARIsAELQRQGrXDoR4ox216bUlSD6mTQYibN24CA5bYF7QikiQQ6j3cGm63HRXsEaAjJjEALw_wcB" target="_blank">subscription to Mapt</a> and read it there.</p>
<p>If you like my blog and would like to support it, spread the word (if not get a copy yourself)!</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://ntguardian.wordpress.com/2018/11/05/beyond-univariate-single-sample-data-mcht/"> R – Curtis Miller's Personal Website</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/bkhTMa-z-Sk" height="1"></a></div>