<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/pushing-ordinary-least-squares-to-the-limit-with-xy/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://www.statworx.com/de/blog/pushing-ordinary-least-squares-to-the-limit-with-xy/"> r-bloggers – STATWORX</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<h2> Introduction to <code>Xy()</code> </h2>
<p>Simulation is mostly about answering particular research questions. Whenever the word simulation appears somewhere in a discussion, everyone knows that this means additional effort. At <a rel="nofollow" href="https://www.statworx.com/" target="_blank">STATWORX</a> we are using simulations as a first step to proof concepts we are developing. Sometimes such a simulation is simple, in other cases a simulation is plenty of work. Though, research questions are always pretty unique, which complicates the construction of a simulation framework. With the <code>Xy()</code> function I try to establish such a simulation framework for supervised learning tasks by trying to find the right balance between capability and simplicity. </p>
<p><img src="https://i2.wp.com/www.statworx.com/wp-content/uploads/Xy.png?resize=250%2C250&ssl=1" alt="xy hexagon logo" width="250" class="wp-image-14540" data-recalc-dims="1" height="250"></p>
<p>I have already introduced this function in a recent <a rel="nofollow" href="https://www.statworx.com/blog/simulating-regression-data-with-xy/" target="_blank">blog post</a>. Since then more and more functionalities were implemented and the overall interface was improved. If you are interested in the motivation behind the function, I strongly encourage you to read my last <a rel="nofollow" href="https://www.statworx.com/blog/simulating-regression-data-with-xy/" target="_blank">blog post</a>. In the following we are going to analyze OLS coefficients under weak signal to noise ratios and get to know all revisions of the function. </p>
<h2> Key Functionalities </h2>
<p>The purpose of <code>Xy()</code> is to simulate supervised learning data with maximum degrees of freedom. The user can independently choose how features generate the target by choosing interaction depths, user-defined nonlinearity transformations, categorical variables and all that jazz.<br>
In the recent update I wrapped a package around the function which can be conveniently forked on <a rel="nofollow" href="http://www.github.com/andrebleier" target="_blank">GitHub</a> or installed via <code>library(devtools)</code>: </p>
<pre><code># Install the package 
devtools::install_github(&quot;andrebleier/Xy&quot;) 
# load the package 
library(Xy) 
# start hacking 
my_first_simulation &lt;- Xy::Xy() 
</code></pre>
<p>By default, the function simulates regression learning data with 1000 observations, two linear and two nonlinear features as well as five randomly generated variables, which are not used to generate the target. One main goal of the function is to challenge feature selection algorithms, thus there is an option to generate random variables. Let us create a simple example: </p>
<pre><code># Simulating regression data with four numeric effects,
# one categorical effect and five irrelevant features 

reg_sim = Xy(n = 1000,  
             numvars = c(2, 2), 
             catvars = c(1, 2), 
             noisevars = 5, 
             family = Xy_task(), 
             nlfun = function(x) {x^2}, 
             interactions = 1, 
             sig = c(1,10),  
             cor = c(0.1,0.3), 
             weights = c(5,10), 
             sigma = NULL, 
             stn = 4, 
             noise.coll = FALSE, 
             intercept = TRUE) 

print(reg_sim)   
</code></pre>
<pre><code lang="r">Xy Simulation
  |  
  | + task regression 
  | + observations 1000 
  | + interactions 1D 
  | + signal to noise ratio 4 
  | + effects  
    | - linear 2 
    | - nonlinear 2 
    | - categorical 1 
    | - noise 5 
  | + intervals  
    | - correlation [0.1,0.3] 
    | - weights [5,10] 
    | - sd [1,10] 

Target generating process:  
y = 416.07 + 9.81NLIN_1 + 5.23NLIN_2 + 5.53LIN_1 + 5.69LIN_2 + 6.75DUMMY_1__2 + e ~ N(0,487.15) 
</code></pre>
<p>The function comes with a <code>print()</code>, <code>plot()</code> and <code>coef()</code> method to further analyze the simulation object. The print method summarizes the settings that were used for the simulation. On the bottom you can see the target generating process, where you can see the coefficients and their respective weights as well as the noise. There are four key arguments in the function. The <code>n</code> argument controls the number of observations. The user can choose a desired amount of linear and nonlinear features with the <code>numvars</code> argument. Further, categorical variables and their respective levels can be determined with <code>catvars</code>. Of course there are many other functionalities, however going through all these would be a little dry, so why not use a toy example? </p>
<h2> Pushing Ordinary Least Squares to the limit </h2>
<p>Suppose we are interested how OLS can cope with different signal strengths. The signal (strength) is defined as</p>
<p><span>   </span><span>   </span><img src="https://i2.wp.com/www.statworx.com/wp-content/ql-cache/quicklatex.com-e93c59dc711064cfb83c071729781a67_l3.png?resize=152%2C43&ssl=1" width="152" alt="[newcommand{Var}{operatorname{Var}} signal = frac{Var[f(X)]}{Var[epsilon]}]" title="Rendered by QuickLaTeX.com" data-recalc-dims="1" height="43"></p>
<p>Hence the signal strength controls how many information about the target can be extracted from our features. Higher values mean that there is relatively more variation in our features opposed to the noise. In such cases we can be more confident about the estimation results of our statistical model. In the following simulation we will try to find the sweet spot for the signal strength. In the <code>Xy()</code> function the signal strength can be manipulated with the <code>stn</code> argument. </p>
<pre><code># Simulating one supervised learning task with signal to noise ratio of 0.01 

sig_sim = Xy(n = 1000,
             numvars = c(2, 0),
             catvars = 0,
             noisevars = 0,
             family = Xy_task(),
             sig = c(1,30),
             cor = 0,
             weights = c(-10,10),
             stn = 0.01,
             intercept = TRUE)

# extract the training data
training &lt;- sig_sim$data

# underlying effects
coef(sig_sim)
</code></pre>
<pre><code lang="r">(Intercept)       LIN_1       LIN_2  
   44.68159    -6.07000     3.50000
</code></pre>
<pre><code>coef(lm(y ~ ., data = training))
</code></pre>
<pre><code lang="r">`(Intercept)`         LIN_1         LIN_2  
   -47.455572     -4.807992      2.936850 
</code></pre>
<p>So in this setup we simulate a regression target which is generated by 2 linear features and dominating noise, since the signal to noise ratio is very weak. We can see that the OLS estimators are pretty off, which is not surprising with such a weak ratio.<br>
In the following we are going to analyze the accuracy of the estimator by repeating the above simulation a thousand times for a single signal to noise ratio. We are going to analyze a total of nine different signal to noise ratios. </p>
<pre><code lang="pyhton"># create a signal to noise ratio 
ratios &lt;- c(rev(1/seq(1, 5, 1)), seq(2, 5, 1)) 
</code></pre>
<pre><code lang="r">[1] 0.2000000 0.2500000 0.3333333 0.5000000 1.0000000 2.0000000 3.0000000 4.0000000 
[9] 5.0000000 
</code></pre>
<p>You can find the complete code of this simulation on my <a rel="nofollow" href="http://www.github.com/andrebleier/Xy" target="_blank">GitHub</a>. I’ve summarized the results in the following figure. </p>
<p><img alt="" src="https://i2.wp.com/www.statworx.com/wp-content/uploads/mse.png?w=450&ssl=1" class="wp-image-14507" data-recalc-dims="1"></p>
<p>You can observe the mean squared error of estimated coefficients against the real underlying coefficients of the <code>Xy()</code> simulation. The dotted line marks the break-even point of a one to one signal to noise ratio. The marginal decrease in the MSE is saturates from a signal of <img src="https://i0.wp.com/www.statworx.com/wp-content/ql-cache/quicklatex.com-e584dd0bab4e6c8efc164939c28db757_l3.png?resize=8%2C12&ssl=1" alt="2" width="8" title="Rendered by QuickLaTeX.com" data-recalc-dims="1" height="12"> onwards. Out of curiosity I ran this simulation with a signal to noise ratio of <img src="https://i1.wp.com/www.statworx.com/wp-content/ql-cache/quicklatex.com-1bf8f6fff8256386df3f8eca006e6c48_l3.png?resize=35%2C13&ssl=1" alt="1000" width="35" title="Rendered by QuickLaTeX.com" data-recalc-dims="1" height="13">, to get a sense of the absolute differences in terms of MSE between a signal of <img src="https://i0.wp.com/www.statworx.com/wp-content/ql-cache/quicklatex.com-8995e2084120c1c1f4b53f490d281bc4_l3.png?resize=8%2C13&ssl=1" alt="5" width="8" title="Rendered by QuickLaTeX.com" data-recalc-dims="1" height="13"> and <img src="https://i1.wp.com/www.statworx.com/wp-content/ql-cache/quicklatex.com-1bf8f6fff8256386df3f8eca006e6c48_l3.png?resize=35%2C13&ssl=1" alt="1000" width="35" title="Rendered by QuickLaTeX.com" data-recalc-dims="1" height="13">. The overall MSE of the OLS coefficients estimated with such a strong signal lies within the fourth decimal place, whereas it is only in the second decimal place at a signal to noise ratio of <img src="https://i0.wp.com/www.statworx.com/wp-content/ql-cache/quicklatex.com-8995e2084120c1c1f4b53f490d281bc4_l3.png?resize=8%2C13&ssl=1" alt="5" width="8" title="Rendered by QuickLaTeX.com" data-recalc-dims="1" height="13">. </p>
<p>I hope you got a sense of the function and by now identified some applications for yourself. Feel free to drop me some feedback on the functionalities and on the overall interface. I am glad to improve the function and I will try to implement new functionalities on request as soon as possible. </p>
<p>In my next blog post I will introduce an upcoming feature of <code>Xy()</code> which will be especially useful in the field of machine learning. </p>
<div>
<h6>Über den Autor</h6>
<p><img src="https://i2.wp.com/www.statworx.com/wp-content/uploads/andre-1-300x300.jpg?resize=180%2C180&ssl=1" width="180" alt="André Bleier" class="wp-user-avatar wp-user-avatar-180" data-recalc-dims="1" height="180"></p>
<div>
<h4>André Bleier</h4>
<p>André ist im Data Science Team und organisiert intern unsere Afterwork Veranstaltungen. In seiner Freizeit treibt er Sport und programmiert gerne kleine und große R Codes.</p>
</div>
</div>
<p>Der Beitrag <a rel="nofollow" href="https://www.statworx.com/de/blog/pushing-ordinary-least-squares-to-the-limit-with-xy/" target="_blank">Pushing Ordinary Least Squares to the limit with Xy()</a> erschien zuerst auf <a rel="nofollow" href="https://www.statworx.com/de/" target="_blank">STATWORX</a>.</p>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://www.statworx.com/de/blog/pushing-ordinary-least-squares-to-the-limit-with-xy/"> r-bloggers – STATWORX</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/uPq4tsPbYq0" height="1"></a></div>