by Jonathan Regenstein In a previous post, we built an R Notebook that pulled in data on sector ETFs and allowed us to calculate the rolling correlation between a sector ETF and the S&amp;P 500 ETF, whose ticker is SPY. Today, we’ll wrap that into a Shiny app that allows the user to choose a […]
suppressPackageStartupMessages(library(tidyverse))
# devtools::install_github(&quot;dgrtwo/gganimate&quot;)
# install.packages(&quot;cowplot&quot;)
library(gganimate)
# install image magick in terminal >> &quot;brew install image magick&quot;
If the Weatherman says that t...
A major new update of the RPushbullet package is now on CRAN. RPushbullet interfacing the neat Pushbullet service for inter-device messaging, communication, and more. It lets you easily send alerts like the one to the to your browser, phone, tablet,...
  Over the year 2016, I manually gathered press clippings announcing Venture Capital (VC) deals from various online or newsletters public sources each time I bumped into something that caught my attention. Early January, I then put together and cleaned the data and made it R-usable as a csv dataset <a href="http://r-blog.salvaggio.net/?p=369">Read More ...</a>
A recent question on X validated ended up being quite interesting! The model under consideration is made of parallel Markov chains on a finite state space, all with the same Markov transition matrix, M, which turns into a hidden Markov model when the only summary available is the number of chains in a given state […]
Like many data nerds, I’m a big fan of Tyler Vigen’s Spurious Correlations, a humourous illustration of the old adage “correlation does not equal causation”. Technically, I suppose it should be called “spurious interpretations” since the correlations themselves are quite real, but then good marketing is everything. There is, however, a more formal definition of … <a href="https://nsaunders.wordpress.com/2017/02/03/the-real-meaning-of-spurious-correlations/">Continue reading <span>The real meaning of spurious correlations</span></a>
If you want to get data out of R and into another application or system, simply copying the data as it resides in memory generally isn't an option. Instead you have to serialize the data (into a file, usually), which the other application can then deserialize to recreate the original data. R has several options to serialize data frames: You can serialize (export) data to comma-separated files (CSVs), which can be imported by just about any application. R has several packages to read and write CSVs, including fwrite and fread from the data.table package. The downside is that as an...
I was working on to expand the disease transmission model from my first post but with the currents events, I felt compelled to work on something different. The new United States presidential administration formulated an executive order the end of last week, if issued, will suspend visas of internationals in the United States (either for school or work) from select countries, prevent … <a href="https://eugejjoh.wordpress.com/2017/02/02/unhcr-refugee-data-visualized/">Continue reading<span> &quot;UNHCR Refugee Data Visualized&quot;</span></a>
Every student learns how to look up areas under the normal curve using Z-Score tables in their first statistics class. But what is less commonly covered, especially in courses where calculus is […]
In many situations, we require that several plots are placed in the same figure as subplots. R has various ways of doing it. Base Graphics has three different ways to draw subplots, i.e. mfrow, layout and split.screen, with increasing degree of complexity, and, at the same time, with increased control over the plot elements. This […]<div>
<h3>Related exercise sets:</h3><ol><li><a title="Advanced Base Graphics Exercises" rel="bookmark" href="http://r-exercises.com/2016/09/23/advanced-base-graphics-exercises/">Advanced Base Graphics Exercises </a></li>
<li><a title="Graphics parameters exercises" rel="bookmark" href="http://r-exercises.com/2016/03/24/graphics-parameters-exercises/">Graphics parameters exercises </a></li>
<li><a title="Getting started with Plotly: basic Plots" rel="bookmark" href="http://r-exercises.com/2017/01/10/getting-started-with-plotly-basic-plots/">Getting started with Plotly: basic Plots </a></li>
<li><a href="http://r-exercises.com/start-here-to-learn-r/">Explore all our (>1000) R exercises</a></li><li><a href="http://r-exercises.com/r-courses/">Find an R course using our R Course Finder</a> directory</li></ol></div>
On occasion of the 10,000th R package on CRAN, eoda operated an automated Twitter account which regularly posted the current number of packages available on CRAN until the 10,000-package milestone has been passed on January 28, 2017. #Rstatsgoes10k – Hello World, it’s 2017-01-28 01:59:03 and currently there are 10000 packages on CRAN. — eodacelebratesR (@Rstatsgoes10k) … <a href="https://blog.eoda.de/2017/02/02/tutorial-how-to-set-up-a-twitter-bot-using-r/"><span>„Tutorial: How to set up a Twitter bot using R“ </span>weiterlesen</a>
The folks at RStudio have a segment on their RViews blog, “Reproducible Finance with R”, one that we at Business Science are very fond of! In the spirit of reproducibility, we thought that it would be appropriate to recreate the RViews post, “Rep...
<p>A solution in the R language for statistical computing to find the solution the Euler Problem 10: Find the sum of all primes below two million. <a href="http://r.prevos.net/euler-problem-10/">Continue reading <span>→</span></a></p>
<p>The post <a rel="nofollow" href="http://r.prevos.net/euler-problem-10/">Euler Problem 10: Summation of Primes</a> appeared first on <a rel="nofollow" href="http://r.prevos.net/">The Devil is in the Data</a>.</p>
<p>A summary of my essay on data science strategy for water utilities. This strategy is based on the principles of the science of data. <a href="http://r.prevos.net/data-science-strategy-water-utilities/">Continue reading <span>→</span></a></p>
<p>The post <a rel="nofollow" href="http://r.prevos.net/data-science-strategy-water-utilities/">Lifting the Big Data Veil: Data Science Strategy for Water Utilities</a> appeared first on <a rel="nofollow" href="http://r.prevos.net/">The Devil is in the Data</a>.</p>
Thomas Dinsmore's ML/DL blog recently concluded a look back on significant advancements in data science, machine learning and deep learning — many of which involved R and/or Microsoft. Here are those highlights (reproduced with permission): The R Project R and Python maintained their leadership as primary tools for open data science. The Python versus R debate continued amid an emerging consensus that data scientists should consider learning both. R has a stronger library of statistics and machine learning techniques and is agiler when working with small data. Python is better suited to developing applications, and the Python open source license...
May the best bot win - Halite is an artificial intelligence programming challenge. Players control a bot using the programming language of their choice. Bots fight for control of a 2D grid and the bot with the most territory at the end wins.
...
A while back I wrote this post about how I stay current in bioinformatics &amp; genomics. That was nearly five years ago. A lot has changed since then. A few links are dead. Some of the blogs or Twitter accounts I mentioned have shifted focus or haven?...
Here is the link to my new course at PACKT publishing. I apologize in advance for some of their video editing choices but you will definitely learn a lot and be able to work through a variety of practical examples to meet your bioinformatic needs. I will upload the R code on GitHub and post the links to the files for all the videos in the course section of my website rjbioinformatics.com. So be sure to stay tuned!
<p>3-way Anova with R: how to find which factors influence a variable Y, analysing the difference between the group means defined by factors’ levels. A new chapter of Raccoon, a Quantide's free web book about Statistical Models with R. </p>
<p>The post <a rel="nofollow" href="http://www.quantide.com/raccoon-ch-2-4-3-way-anova/">Raccoon | Ch 2.4 – 3-way Anova</a> appeared first on <a rel="nofollow" href="http://www.quantide.com/">Quantide - R training &amp; consulting</a>.</p>
roxygen2 6.0.0 is now available on CRAN. roxygen2 helps you document your packages by turning specially formatted inline comments into R’s standard Rd format. It automates everything that can be automated, and provides helpers for sharing documentation between topics. Learn more at http://r-pkgs.had.co.nz/man.html. Install the latest version with: install.packages(&quot;roxygen2&quot;) There are two headline features in this […]
by Max Kuhn Introduction The formula interface to symbolically specify blocks of data is ubiquitous in R. It is commonly used to generate design matrices for modeling function (e.g. lm). In traditional linear model statistics, the design matrix is the two-dimensional representation of the predictor set where instances of data are in rows and variable […]
<p>Shiny is awesome, but can be a bit daunting and easy to make mistakes in. I recently came back to Shiny after a hiatus of a few years and it was much more challenging than I feel comfortable admitting. I was making bonehead mistakes like writing something instead of output$something, confusing where to put Output commands vs Render commands, etc. I would… <a href="http://amitkohli.com/announcing-shinytester-a-package-that-helps-you-build-shiny-apps/">Read more →</a></p>
Note, at the time of this writing using the packages ggplot2 and ggmap from CRAN will result in an error. To avoid the error be sure to install both packages from GitHub with the package devtools and restart R if the problem persists. devtools::install_github(&quot;dkahle/ggmap&quot;) devtools::install_github(&quot;hadley/ggplot2&quot;) Note, at the time this writing the package gganimate requires […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toMetro Systems Over Time: Part 2" rel="bookmark" href="https://datascienceplus.com/metro-systems-over-time-part-2/">Metro Systems Over Time: Part 2</a></li><li><a title="Permanent Link toMetro Systems Over Time: Part 1" rel="bookmark" href="https://datascienceplus.com/metro-systems-over-time-part-1/">Metro Systems Over Time: Part 1</a></li><li><a title="Permanent Link toOutlier App: An Interactive Visualization of Outlier Algorithms" rel="bookmark" href="https://datascienceplus.com/outlier-app-an-interactive-visualization-of-outlier-algorithms/">Outlier App: An Interactive Visualization of Outlier Algorithms</a></li><li><a title="Permanent Link toCreating an animation using R" rel="bookmark" href="https://datascienceplus.com/creating-an-animation-using-r/">Creating an animation using R</a></li><li><a title="Permanent Link toThe importance of Data Visualization" rel="bookmark" href="https://datascienceplus.com/the-importance-of-data-visualization/">The importance of Data Visualization</a></li></ol>
  To post your R job on the next post Just visit this link and post a new R job to the R community. You can post a job for free (and there is also a “featured job” option for extra exposure). Current R jobs Job seekers: please follow the links below to learn more and apply for your R job of interest: Featured Jobs Full-Time Quantitative/Data Scientist Signal Maritime – Posted by s.malle Glifada, Glyfada Attica, Greece 13 Jan2017 Full-Time Software Engineer Icahn School of Medicine at Mt. Sinai – Posted by dtelad11 New York New York, United States 11 Jan2017 Full-Time Data Analyst in survey design and quantitative fisheries Sampling Design And Analysis Group @ Institute of Marine Research – Posted by Sampling.Design.And.Analysis.Group Lysekil Västra Götalands län, Sweden 11 Jan2017 Full-Time Customer Success Rep RStudio – Posted by jclemens1 Anywhere 3 Jan2017 More New Jobs Full-Time 1-year postdoctoral position in Bioinformatics/Statistics is available @ Madrid, Spain. rdiaz02 Madrid Comunidad de Madrid, Spain 17 Jan2017 Full-Time Transportation Market Research Analyst @ Burlington, Vermont, United States RSG – Posted by patricia.holland@rsginc.com Burlington Vermont, United States 13 Jan2017 Full-Time Data Scientist @ New York, United States 7Park Data – Posted by Derek Darves New York New York, United States […]
Go ahead stomp your feet on the floorboards Clap your hands if that’s really what you came here for (Heaven, The Milk Carton Kids) Inspired by curves created by the harmonograph, I have done a Shiny app to generate random images that you can personalize and use as an Exlibris.  You can try the App here. … <a href="https://fronkonstin.com/2017/02/01/the-ex-libris-generator/">Continue reading <span>The Ex Libris Generator</span> <span>→</span></a>
For the (Advanced) R Crash Course of the Data Science for Actuaries program, we will use the Ubuntu virtual machine. There might be some issues when installing some packages… One trick can be to open a terminal and then to use the sudo command, to install some packages, (after entering the password). Just type (or copy/paste) sudo apt-get install libcurl4-openssl-dev sudo apt-get install libxml2-dev sudo apt-get install openjdk-7-* update-alternatives --config java sudo apt-get install aptitude sudo aptitude install libgdal-dev sudo aptitude install libproj-dev sudo … <a href="http://freakonometrics.hypotheses.org/50127">Continue reading <span>Install R Packages on the Ubuntu Virtual Machine</span> <span>→</span></a>
So the votes are in: Your established blog is mostly about your work. Your work changes. Do you continue at the current blog or start a new one? — Neil Saunders (@neilfws) January 23, 2017 I thank you, kind readers. So here’s the plan: (1) keep blogging here as frequently as possible (perhaps monthly), (2) … <a href="https://nsaunders.wordpress.com/2017/02/01/taking-steps-in-xml/">Continue reading <span>Taking steps (in XML)</span></a>
The Windows edition of the Data Science Virtual Machine (DSVM) was recently updated on the Azure Marketplace. This update upgrades some existing components and adds some new ones as well. You now have your choice of integrated development environment to use with R. RStudio Desktop is now included in the Data Science Virtual Machine image — no need to install it manually. And R Tools for Visual Studio has been upgraded to Version 0.5. Microsoft R Server has also been upgraded to version 9.0. This includes the latest R 3.2.2 language engine, the RevoScaleR package for big-data support in R,...
This week we are launching four new courses as part of our Introduction to Statistics curriculum. We are taking a modern approach to teaching statistics with the use of simulations and randomization rather than a more traditional theoretical one. We ha...
Sometimes it is helpful to simulate gene expression data to test code or to see how your results look with simulated values from a particular probability distribution. Here I am going to show you how to simulate RNAseq expression data counts from a uniform distribution with a mininum = 0 and maximum = 1200.
RDSTK is a very versatile package. It includes functions to help you convert IP address to geo locations and derive statistics from them. It also allows you to input a body of text and convert it into sentiments. This package provides an R interface to Pete Warden’s Data Science Toolkit. See www.datasciencetoolkit.org for more information. […]<div>
No related exercise sets.</div>
I recently downloaded the latest version of R Studio and noticed that their import dataset functionality had changed significantly. I had previously written about this HERE and wanted to provide an update for the current version of RStudio. When you go to import data using R Studio, you get a menu like this. If you’re […]
BMC Bioinformatics has published a paper by colleagues of mine, about calculating exact p-values for pairwise comparison of Friedman rank sums. The paper provides fast and easy-to-use R code, making it an interesting read for ...
<p>The last few posts at Sharp Sight have been fairly intense.</p>
<p>Let’s make a quick map ...</p>
<p>The post <a rel="nofollow" href="http://sharpsightlabs.com/blog/map-unemployment-nov-2016/">Mapping unemployment data, 2016</a> appeared first on <a rel="nofollow" href="http://sharpsightlabs.com/">SHARP SIGHT LABS</a>.</p>
The R code that accompanies this post is found on GitHub: you will find R, Rmd, and HTML files there that were used during the first Open Data R Meetup held in Belgrade, 31 January 2017, organized by Data Science Serbia in Startit Center, Savska 5, Bel...
After months and months of working trying to get R on 64 bit Windows to work with link-time-optimization (LTO), I’ve come to the conclusion that GCC 4.9.3’s implementation of LTO just isn’t polished enough to work across the entire R infrastructure. I have been able to get base R, nloptr, Rblas, and Rlapack to compile<a href="http://www.avrahamadler.com/2017/01/31/enough-trying-to-use-lto/"> Read the full article...</a>
An update to the gunsales package is now on the CRAN network. As in the last update, some changes are mostly internal. We removed the need to import two extra packages with were used in one line each -- easy enough to be replaced with base R. We also...
For 8 years now, we've maintained a list of local R user groups here at the Revolutions blog. This is a list that began with a single group (the Bay Area RUG, the first and still one of the largest groups), and now includes 360 user groups worldwide (including 27 specifically for women). As the list has grown in size, it's become harder to manage. Thankfully, Colin Gillespie of Jumping Rivers Consulting has risen to the task, by creating a new website based on a GitHub repository that anyone can contribute to. I've updaed the Local R User Group Directory...
<p>R has become an essential tool in oceanography and marine ecology. For instance, R is specifically used to read, process and represent in situ oceanographic data and to manage satellite data in order to produce high temporal and spatial resolution maps useful to synoptically explore and monitoring vast areas of the world oceans. In this post we briefly describe a practical use of R in conjunction with satellite data to identify marine bioregions of the Labrador Sea with different patters in the phytoplankton seasonal cycle.</p>
<p>The post <a rel="nofollow" href="http://www.milanor.net/blog/r-satellite-data-to-identify-marine-bioregions/">Using R and satellite data to identify marine bioregions</a> appeared first on <a rel="nofollow" href="http://www.milanor.net/blog">MilanoR</a>.</p>
An introduction to package exams - Part of my job as a researcher and teacher is to periodically apply and
grade exams in my classroom. Being constantly in the shoes of an
examiner, you soon quickly realize that students are cleve...
In my last post, I built a shiny app to explore World Gender Statistics.
To make it a bit nicer and more convenient, I added a few more features:
The drop-down menu for Years is now reactive, i.e. it only shows options with data (all NA years are ...
I have just finished and released a new R video lecture demonstrating how to use the “Bizarro pipe” to debug magrittr pipelines. I think R dplyr users will really enjoy it. Please read on for the link to the video lecture. In this video lecture I use the “Bizarro pipe” to debug the example pipeline … <a href="http://www.win-vector.com/blog/2017/01/using-the-bizarro-pipe-to-debug-magrittr-pipelines-in-r/">Continue reading <span>Using the Bizarro Pipe to Debug magrittr Pipelines in R</span></a>
<p>SCADA data is essential in operating water treatment plants. Spikes in the data can indicate problems. How can you detect SCADA spikes using the R language? <a href="http://r.prevos.net/scada-spikes/">Continue reading <span>→</span></a></p>
<p>The post <a rel="nofollow" href="http://r.prevos.net/scada-spikes/">SCADA spikes in Water Treatment Data</a> appeared first on <a rel="nofollow" href="http://r.prevos.net/">The Devil is in the Data</a>.</p>
This is Part 3 of our “Getting Started with R Programming” series. For previous articles in the series, click here: Part 1, Part 2. This week, we are going to talk about using git and GitHub with RStudio to manage […]
Acquiring inauguration speeches
Though not about Greenville especially, it might be interesting to quantitatively analyze inauguration speeches. This analysis will be done using two paradigms: the tm package and the tidytext package. We will read the speeches in such a way that we use the tidytext package; later on we will use some tools from that package to make analyses traditionally done by tm.
I looked around for inauguration speeches, and finally found them at www.bartelby.com. They are in a format more for human consumption, but with the use of the rvest (harvest?) package, we can read them in relatively easily. However, we need to do a mapping from speech IDs to speakers (newly inaugurated presidents), which is a little ugly and tedious.
A draft of my book Empirical Software Engineering using R is now available for download. The book essentially comes in two parts: statistical techniques that are useful for analyzing software engineering data. This draft release contains most of the techniques I plan to cover. I am interested in hearing about any techniques you think ought […]
Data science enhances people’s decision making. Doctors and researchers are making critical decisions every day, so it is obvious that those people must have a substantial knowledge of data science. This series aims to help people that are around medical field to enhance their data science skills. We will work with a health related database […]<div>
<h3>Related exercise sets:</h3><ol><li><a title="Descriptive Analytics-Part 5: Data Visualisation  (Continuous variables)" rel="bookmark" href="http://r-exercises.com/2016/12/04/descriptive-analytics-part-5-data-visualisation-continuous-variables/">Descriptive Analytics-Part 5: Data Visualisation (Continuous variables) </a></li>
<li><a title="Basic Tree 2 Exercises" rel="bookmark" href="http://r-exercises.com/2016/12/15/basic-tree-2/">Basic Tree 2 Exercises </a></li>
<li><a title="Start plotting data!" rel="bookmark" href="http://r-exercises.com/2016/03/11/start-plotting-data-3/">Start plotting data! </a></li>
<li><a href="http://r-exercises.com/start-here-to-learn-r/">Explore all our (>1000) R exercises</a></li><li><a href="http://r-exercises.com/r-courses/">Find an R course using our R Course Finder</a> directory</li></ol></div>
Summary In this blog post I will show you how to make your own personal assistant (think Siri, Cortana, Alexa) in R, very quickly. This will be done in three steps: Get R to recognise your voice and convert it to text Set up a system which us...
This week I explored the World Gender Statistics dataset. You can look at 160 measurements over 56 years with my Shiny app here.
I prepared the data as follows:
Data.csv
Country.Name: the name of the country
Country.Code: the country’s co...
Lately I was getting a little bored with genomic data (and then TCGA2STAT started to give me a segfault on my university’s high performance computing facility too :stuck_out_tongue:). So I decided to analyze some brain imaging data that I had lying a...
In this post, I’m sharing a brand-new analysis! The reason for this is my blog being added to R-bloggers by Tal Galili after I filled this form. R-bloggers is a collection of blogs about R, whose new posts get added to the website via the magic of RS...
I am happy to announce a couple of exciting upcoming Win-Vector LLC public speaking engagements. BARUG Meetup Tuesday, Tuesday February 7, 2017 ~7:50pm, Intuit, Building 20, 2600 Marine Way, Mountain View, CA. Win-Vector LLC’s John Mount will be giving a “lightning talk” (15 minutes) on R calling conventions (standard versus non-standard) and showing how to … <a href="http://www.win-vector.com/blog/2017/01/upcoming-win-vector-llc-public-speaking-engagements/">Continue reading <span>Upcoming Win-Vector LLC public speaking engagements</span></a>
Data science enhances people’s decision making. Doctors and researchers are making critical decisions every day, so it is obvious that those people must have a substantial knowledge of data science. This series aims to help people that are around medical field to enhance their data science skills. We will work with a health related database […]<div>
<h3>Related exercise sets:</h3><ol><li><a title="Reading delimited data" rel="bookmark" href="http://r-exercises.com/2016/01/07/reading-delimited-data/">Reading delimited data </a></li>
<li><a title="Descriptive Analytics-Part 5: Data Visualisation  (Continuous variables)" rel="bookmark" href="http://r-exercises.com/2016/12/04/descriptive-analytics-part-5-data-visualisation-continuous-variables/">Descriptive Analytics-Part 5: Data Visualisation (Continuous variables) </a></li>
<li><a title="Merging Dataframes Exercises" rel="bookmark" href="http://r-exercises.com/2016/04/14/merging-dataframes-exercises/">Merging Dataframes Exercises </a></li>
<li><a href="http://r-exercises.com/start-here-to-learn-r/">Explore all our (>1000) R exercises</a></li><li><a href="http://r-exercises.com/r-courses/">Find an R course using our R Course Finder</a> directory</li></ol></div>
This is just a test page for forecastersblog.org. Ignore it, please. es() allows selecting between AIC (Akaike Information Criterion), AICc (Akaike Information Criterion corrected) and BIC (Bayesian Information Criterion, also known as Schwarz IC). The very basic information criterion is AIC. It is calculated for a chosen model using formula: \begin{equation} \label{eq:AIC} \text{AIC} = -2 […]
R just passed another milestone: 10,000 packages on CRAN. From A3 to zyp, from ABC analysis to zero-inflated models – 10,000 R packages mean great variety and methods for almost every use case. On occasion of this event we collected the top 10 R packages in collaboration with the ones who should know best: our … <a href="https://blog.eoda.de/2017/01/28/on-occasion-of-the-10000th-r-package-the-eoda-top-10/"><span>„On occasion of the 10,000th R package: The eoda Top 10“ </span>weiterlesen</a>
“Ten Simple Rules for Reproducible Computational Research” is a
freely available paper on PLOS computational biology.
As I’m currently very interested on the subject of reproducible data analysis, I will these ten rules and the possible implem...
BiBitR enters the fray!
Ewoud De Troyer, University of Hasselt (CenStat)
Introduction
The latest patch of the BiclustGUI includes the implementation of BiBitR, a simple R wrapper which directly calls the original Java code for applying the BiBit algo...
CRAN, the global repository of open-source packages that extend the capabiltiies of R, reached a milestone today. There are now more than 10,000 R packages available for download*. (Incidentally, that count doesn't even include all the R packages out there. There are also another 1294 packages for genomic analysis in the BioConductor repository, hundreds of R packages published only on GitHub, commercial R packages from vendors such as Microsoft and Oracle, and an unknowable number of private, unpublished packages.) Why so many packages? R has a very active developer community, who contribute new packages to CRAN on a daily basis....
After the last post building on feedback from readers, the blog is back to the regular program of recycling old Github repos. Today’s project was waiting for its turn here and will involve a Catan card game. Nearly a year ago, I played Catan with my ...
by Sean Lopp This month’s collection of Tips and Tricks comes from an excellent talk given at the 2017 RStudio::Conf in Orlando by RStudio Software Engineer Kevin Ushey. The slides from his talk are embedded below and cover features from autocompletion to R Markdown shortcuts. Use the left and right arrow keys to change slides. […]
A new release, now at version 0.6.12, of the digest package is now on CRAN and in Debian.
The digest creates hash digests of arbitrary R objects (using the 'md5', 'sha-1', 'sha-256', 'crc32', 'xxhash' and 'murmurhash' algorithms) permitting easy comp...
A good fraction of R users use Apple computers. Apple machines historically have sat at a sweet spot of convenience, power, and utility: Convenience: Apple machines are available at retail stores, come with purchasable support, and can run a lot of common commercial software. Power: R packages such as parallel and Rcpp work better on … <a href="http://www.win-vector.com/blog/2017/01/upgrading-to-macos-sierra-nee-osx-for-r-users/">Continue reading <span>Upgrading to macOS Sierra (nee OSX) for R users</span></a>
From my email today You use an illustration of a seasonal arima model: ARIMA(1,1,1)(1,1,1)4 I would like to simulate data from this process then fit a model… but I am unable to find any information as to how this can be conducted… if I set phi1, Phi1, theta1, and Theta1 it would be reassuring that […]
Multiple Regression is one of the most widely used methods in statistical modelling. However, despite its many benefits, it is oftentimes used without checking the underlying assumptions. This can lead to results which can be misleading or even completely wrong. Therefore, applying diagnostics to detect any strong violations of the assumptions is important. In the […]
Note, at the time of this writing using the packages ggplot2 and ggmap from CRAN will result in an error. To avoid the error be sure to install both packages from GitHub with the package devtools and restart R if the problem persists. devtools::install_github(&quot;dkahle/ggmap&quot;) devtools::install_github(&quot;hadley/ggplot2&quot;) Introduction In Part 1 of this series we collected geodata […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toMetro Systems Over Time: Part 1" rel="bookmark" href="https://datascienceplus.com/metro-systems-over-time-part-1/">Metro Systems Over Time: Part 1</a></li><li><a title="Permanent Link toOutlier App: An Interactive Visualization of Outlier Algorithms" rel="bookmark" href="https://datascienceplus.com/outlier-app-an-interactive-visualization-of-outlier-algorithms/">Outlier App: An Interactive Visualization of Outlier Algorithms</a></li><li><a title="Permanent Link toCreating an animation using R" rel="bookmark" href="https://datascienceplus.com/creating-an-animation-using-r/">Creating an animation using R</a></li><li><a title="Permanent Link toThe importance of Data Visualization" rel="bookmark" href="https://datascienceplus.com/the-importance-of-data-visualization/">The importance of Data Visualization</a></li><li><a title="Permanent Link toggplot2 themes examples" rel="bookmark" href="https://datascienceplus.com/ggplot2-themes-examples/">ggplot2 themes examples</a></li></ol>
A great way to hone your skills as a data scientist is to pick a topic you're passionate about, find some data related to it, and analyze the heck out of it. Jim Vallandingham is clearly passionate about old Kung Fu movies — particularly those from the Shaw Brothers Studio — and has used R to analyze data the studio's oeuvre: 260 films over 22 years. The complete R code behind the analysis is included in the post (and you can also find it as an R Markdown document here). Some interesting notes include: Use of the tidyjson package to...
<p>Image compression with principal component analysis is a frequently occurring application of the dimension reduction technique. Recall from a previous post that employed singular value decomposition to compress an image, that an image is a matrix of pixels represented by RGB color values. Thus, principal component analysis can be used...</p>
<p>The post <a rel="nofollow" href="http://www.aaronschlegel.com/image-compression-principal-component-analysis/">Image Compression with Principal Component Analysis</a> appeared first on <a rel="nofollow" href="http://www.aaronschlegel.com/">Aaron Schlegel</a>.</p>
I was lucky enough to get some nice and interesting feedback on my last post. One comment was really useful and pretty embarrassing: I had written “see” instead of “sea” in the whole post… Thanks Steve Dempsey for the correction! I also got s...
In an earlier post of mine, I carried out an analysis on ski jumping data for Zakopane, Poland and attempted to predict which athletes would end up on the podium. I also created a classification tree and tested it on the 2017 competition data with good results. For this side project of mine, I hope… <a href="https://sw23993.wordpress.com/2017/01/26/predicting-the-willingen-2017-mens-ski-jumping-competition/">Continue reading <span>Predicting the Willingen 2017 men’s ski jumping competition</span> <span>→</span></a>
Three courses in R are starting tomorrow and two next week on Statistics.com. You may use the code “1qR-blogger-17″ when registering (offering a reduced rate of $399 instead of,  around, $589 a course).  List of R courses: Mar 17:  SQL and R – Introduction to Database Queries, taught by Ben Baumer, (Director of the Statistical &amp; Data Sciences Program at Smith College, and served previously as the Statistical Analyst for the New York Mets) – for those seeking to query (not build) databases Mar 24:  Bayesian Statistics in R, taught by Peter Congdon, Research Professor in Quantitative Geography and Health Statistics at Queen Mary Univ. of London and the author of “Bayesian Statistical Modeling, Applied Bayesian Modeling,” and “Bayesian Models for Categorical Data.” Mar 31:  R Programming Intermediate, taught by Dr. Oliva C. Lau (self-described R ninja at Google) – systematic overview of R as a programming language, emphasizing good programming practices and the development of clear, concise code.  After completing the course, you’ll be able to manipulate data programmatically using R functions of your own design. There are no set hours when you must be online, and we estimate you will need about 15 hours per week.  The instructor will be available […]
<p>This Friday at 9am PT I will be running a live training titled Using Email for your Data Science Portfolio. Within the world of online communication, email is the...</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2017/01/25/live-training-using-email-data-science-portfolio/">Live Training: Using Email for your Data Science Portfolio</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
A chance occurrence on X validated: a question on an incomprehensible formula for Bayesian model choice: which, most unfortunately!, appeared in Bayesian Essentials with R! Eeech! It looks like one line in our LATEX file got erased and the likelihood part in the denominator altogether vanished. Apologies to all readers confused by this nonsensical formula!Filed […]
by Edgar Ruiz In the corporate world, spreadsheets and PowerPoint presentations still dominate as the tools used for analyzing and sharing information. So, it is not at all surprising that even when business analysts use R for the analytical heavy lifting, they frequently revert to using spreadsheets and slide decks to share their results. This […]
<p>Solution to Euler Problem 9 in the R Language: Find the Pythagorean triple for which a+b+c equals 1000. <a href="http://r.prevos.net/euler-problem-9/">Continue reading <span>→</span></a></p>
<p>The post <a rel="nofollow" href="http://r.prevos.net/euler-problem-9/">Euler Problem 9 : Special Pythagorean Triple</a> appeared first on <a rel="nofollow" href="http://r.prevos.net/">The Devil is in the Data</a>.</p>
Heartland Bank, a rapidly growing bank in New Zealand, has adopted a data-driven approach to analyzing risk, evaluating credit lines, and understanding cash flows. But they found their legacy SAS system to be labor-intensive and time consuming when it came to updating financial models, and it was expensive to boot. (Being licensed on a per-user basis, it was available only to a small group staff in IT.) The bank wanted an analytics platform that could support future innovation, and so Heartland Bank replaced SAS with Microsoft R Server and SQL Server. It's fitting that a bank in New Zealand (the...
A couple of weeks ago, I had a discussion with a co-worker regarding a project I was involved in, I felt that there was no clear understanding of the daily challenges data scientists face. Few days later, I was at Rstudio::Conf 2017 where I met lots of data scientists from academia and industry. Later on, I described one of the conference’s positive side effects as “group therapy”, where one could see how others face the same challenges and struggle with similar issues. So it was like a personal sanity check!
Hey R fans! We've just released a new episode of our DataChats video series.
In this episode, we interview Mine Çetinkaya-Rundel, Ph.D.. Mine is Director of Undergraduate Studies and an Associate Professor of the Practice in the Department of St...
This week we are launching Bond Valuation and Analysis in R taught by Clifford S. Ang, CFA, Vice President at Compass Lexecon. Bonds are securities issued by governments or corporations that pay interest over a fixed schedule and are the most well-kno...
Over the next few months we’re running a number of R, Stan and Scala courses around the UK. Feburary Mon 13 – Introduction to R (London) Tue 14 – Programming with R (London) Wed 15 – Advanced Graphics with R (London) Thur 16 (2-day course) – Predictive Analysis (London) March Tue Mar 21 – R for […]
Guest post by Jonathan Sidi Cheatsheets are currently built and used exclusivley as a teaching tool. We want to try and change this and produce a cheat sheet that gives a roadmap to build a known product, but also is built as a function so users can input data into it to make the cheatsheet … <a href="https://www.r-statistics.com/2017/01/data-driven-cheatsheets/">Continue reading<span> &quot;Data Driven Cheatsheets&quot;</span></a>
Sort of introduction to this post and hopefully the next ones
I usually don’t have any New Year resolution. However, recent tweets about productivity – from people I actually find productive and inspiring – made me ponder a bit on my unfinished...
<p>In several recent blog posts, I've emphasized the importance of data analysis.</p>
<p>My main point has been, that if you want to learn data science, you need to learn <i>data analysis</i>. Data analysis is the foundation of practical data science.</p>
<p>With that statement in mind, I want to show you step-by-step what an analysis looks like in R ...</p>
<p>The post <a rel="nofollow" href="http://sharpsightlabs.com/blog/shipping-analysis-r-data-visualization/">How to do an analysis in R (part 2, visualization and analysis)</a> appeared first on <a rel="nofollow" href="http://sharpsightlabs.com/">SHARP SIGHT LABS</a>.</p>
Microsoft R Server 9 includes a new R package for machine learning: MicrosoftML. (So do the Data Science Virtual Machine and the free Microsoft R Client edition, incidentally.) This package includes a suite of fast predictive modeling functions implemented by Microsoft Research, including: Linear (rxFastLinear) and logistic (rxLogisticRegression) model functions based on the Stochastic Dual Coordinate Ascent method; Classification/regression trees (rxFastTrees) and random forests (rxFastForests) based on FastRank, an efficient implementation of the MART gradient boosting algorithm; A neural network algorithm (rxNeuralNet) with support for custom, multilayer network topologies; and One-class anomaly detection (rxOneClassSvm) based on support vector machines. As...
During one of my summers, I had the opportunity to conduct some research on the prevalence of methicillin-resistant Staphylococcus aureus (MRSA) in vulnerable populations and examining US emergency department data and I thought this would be a pretty interesting topic to expand on for my thesis in lieu of the increasing concerns of antimicrobial resistance, … <a href="https://eugejjoh.wordpress.com/2017/01/24/descriptive-analysis-of-mlst-data-for-mrsa/">Continue reading<span> &quot;Descriptive Analysis of MLST Data for MRSA&quot;</span></a>
RENDER FUNCTIONS In the fourth part of our series we just “scratched the surface” of reactivity by analyzing some of the properties of the renderTable function. Now it is time to get deeper and learn how to use the rest of the render functions that shiny provides. As you were told in part 4 these […]
Since a few new events have been announced recently, I thought I'd give a run-down on some major R conferences coming up in the next six months. February 18: satRdays, Cape Town (South Africa). This is the second in a series of one-day conferences inspired by an R Consortium proposal. The first event in Budapest was a great success, and the lineup (including an all-women keynote program) looks equally awesome this time around. I'm sad I can't make this one myself! April 21-22: New York City R Conference (USA). New York is a hub for R developers, and people come...
tidyquant, version 0.3.0, is a pretty sizable release that includes a little bit for everyone, including new financial charting and moving average geoms for use with ggplot2, a new tq_get get option called &quot;key.stats&quot; for retrieving real-time stock inf...
I’m an avid R user and rarely use anything else for data analysis and visualisations. But while R is my go-to, in some cases, Python might actually be a better alternative.
That’s why I wanted to see how R and Python fare in a one-on-one compariso...
<p>I’ve recently spoken to several people who Have decided to create a portfolio of their data science projects Are new to online publishing They frequently have...</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2017/01/23/choosing-software-publish-data-science-portfolio/">Choosing Software to Publish your Data Science Portfolio</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
Principal component analysis (PCA) is routinely employed on a wide range of problems. From the detection of outliers to predictive modeling, PCA has the ability of projecting the observations described by variables into few orthogonal components defined at where the data ‘stretch’ the most, rendering a simplified overview. PCA is particularly powerful in dealing with multicollinearity and variables that … <a href="https://poissonisfish.wordpress.com/2017/01/23/principal-component-analysis-in-r/">Continue reading <span>Principal Component Analysis in R</span> <span>→</span></a>
Bioinformatics is an interdisciplinary field of study that combines the field of biology with computer science to understand biological data. Bioinformatics is generally used in laboratories as an initial or final step to get the information. This information can subsequently be utilized for the wet lab practices. However, it can also be used as a […]
<p>Analysis of the ownership relationships between organisations associated with Donald J. Trump. A social network analysis of Trumpland using the igraph package in R. <a href="http://r.prevos.net/trumpworld-analysis/">Continue reading <span>→</span></a></p>
<p>The post <a rel="nofollow" href="http://r.prevos.net/trumpworld-analysis/">Trumpworld Analysis : Ownership Relations in his Business Network</a> appeared first on <a rel="nofollow" href="http://r.prevos.net/">The Devil is in the Data</a>.</p>
To quantify the impact of the CPU on an analysis, I created the package benchmarkme. The idea is simple. If everyone runs the same R script, we can easily compare machines. One of the benchmarks in the package is for comparing read/write speeds; we write a large CSV file (using write.csv) and read it back in […]
Tal Galili, empowering bloggers to empower other R users - Hooray! I am now part of the R-bloggers community!
R-bloggers is a blog aggregator for content created by the Rstats community. It was founded by Tal Galili towards the end of 2009, i...
The Knoxville R Users Group is presenting a workshop on text analysis using R by Bob Muenchen. The workshop is free and open to the public. You can join the group at https://www.meetup.com/Knoxville-R-Users-Group. A description of the workshop follows. R for … <a href="http://r4stats.com/2017/01/19/knoxville-tn-r-for-text-analysis-workshop/">Continue reading <span>→</span></a>
Many of the problems we encounter in Econometrics can be formulated as a linear or a quadratic problem. In this post, I want to approach two traditional problems: Quantile Regression and Ordinary Least Squares as convex problems and how to implement them in R using the package RMosek.
<p>Often, it is not helpful or informative to only look at all the variables in a dataset for correlations or covariances. A preferable approach is to derive new variables from the original variables that preserve most of the information given by their variances. Principal component analysis is a widely used...</p>
<p>The post <a rel="nofollow" href="http://www.aaronschlegel.com/principal-component-analysis/">Principal Component Analysis</a> appeared first on <a rel="nofollow" href="http://www.aaronschlegel.com/">Aaron Schlegel</a>.</p>
Note: Cross-posted with the Stack Overflow blog
The 2017 Stack Overflow Developer Survey opened last week, and we on the Data Team are looking forward to analyzing the survey results to better understand our developer community. I am particularly inte...
Using package BatchGetSymbols - One of the great things of working in finance is that financial datasets
are freely available from sources such as Google and Yahoo Finance. This
is an excelent feature for building up to date conte...
A little bit about me and this blog - Hello everyone! This is the first (ever) post for a new blog about
Finance and R. My name is Marcelo Perlin and my day job is assistant
professor of Finance in Federal University of Rio Gran...
The goal of reproducible data analysis is, according to the CRAN task view, to tie specific instructions to data analysis and experimental data so that scholarship can be recreated, better understood and verified.
In order to prepare a small talk on t...
RcmdrPlugin.KMggplot2 v0.2-4 (an Rcmdr plug-in; a GUI for 'ggplot2') was released on Dec. 20, 2016. RcmdrPlugin.KMggplot2 (CRAN) NEWS Changes in version 0.2-4 (2016-12-20) ggplot2 2.2.0 was supported. Added ggplot2's colour/fill scales (scale_hue, scale_grey). New plot: Beeswarm plot in Box plot (Th…
In a previous post, I described the principles of my basic reproducible data analysis workflow. Today, let’s be more practical and see how to implement it.
Be noted that it is a basic workflow. The goal is to find a good balance between a minimal re...
<p>The antipode of any place on Earth is the point on the Earth's surface that is diametrically opposite to it. How can you use ggmap to find antipodes? <a href="http://r.prevos.net/antipodes/">Continue reading <span>→</span></a></p>
<p>The post <a rel="nofollow" href="http://r.prevos.net/antipodes/">Finding antipodes using the globe and ggmap packages</a> appeared first on <a rel="nofollow" href="http://r.prevos.net/">Data Science with R</a>.</p>
How do you measure “The Best”?
It’s not immediately clear. Because, “The Best” is incredibly vague and subjective. My “Best” is not the same as your “Best”. And our “Best”s can converge and diverge depending on what we are measur...
Movies of late haven’t impressed (this viewer anyhow) and I keep finding myself returning to classics of the 1980’s and 1990’s to get my movie fix. While browsing through Kaggle datasets, I came across the IMDB 5000 Movie Dataset which contains data on over 5000 movies scraped from the IMDB website. I dove into the… <a href="https://sw23993.wordpress.com/2017/01/19/imdb-movie-analysis/">Continue reading <span>IMDB Movie Analysis</span> <span>→</span></a>
In the follow-up to the useR! conference in Stanford last year, the Women in R Task force took the opportunity to survey the 900-or-so participants about their backgrounds, experiences and interests. With 455 responses, the recently-published results provide an interesting snapshot about the R community (or at least that subset able to travel to the US and who were able to register before the conference sold out). Among the findings (there are summaries; check the report for the detailed breakdowns): 33% of attendees identified as women 26% of attendees identified as other than White or Caucasian 5% of attendees identified...
We have been working towards a revision of our reparameterisation paper for quite a while now and too advantage of Kate Lee visiting Paris this fortnight to make a final round: we have now arXived (and submitted) the new version. The major change against the earlier version is the extension of the approach to a […]
by Jonathan Regenstein Welcome to the first installation of reproducible finance for 2017. It’s a new year, a new President takes office soon, and we could be entering a new political-economic environment. What better time to think about a popular topic over the last few years: equity correlations. Elevated correlations are important for several reasons […]
What makes an open science workshop effective or successful*? Over the last 15 years, I have had the good fortune to participate in workshops as a student and sometimes as an instructor. Consistently, there were beneficial discovery experiences, and at times, some of the processes highlighted have been transformative. Last year, I had the good fortune […]
The results of the study are interesting from an astrological point of view. Astrological signs are divided into groups by order. The first grouping is by alternating order, with the first sign (Aries) positive and the next sign negative, the third positive, and so on through the circle. The second grouping is in groups of […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toRegression model with auto correlated errors – Part 2, the models" rel="bookmark" href="https://datascienceplus.com/regression-model-with-auto-correlated-errors-part-2-the-models/">Regression model with auto correlated errors – Part 2, the models</a></li><li><a title="Permanent Link toRegression model with auto correlated errors – Part 1, the data" rel="bookmark" href="https://datascienceplus.com/fitting-a-simple-regression-model-with-auto-correlated-errors-an-astrological-example-part-1-the-data/">Regression model with auto correlated errors – Part 1, the data</a></li><li><a title="Permanent Link toLinear Regression from Scratch in R" rel="bookmark" href="https://datascienceplus.com/linear-regression-from-scratch-in-r/">Linear Regression from Scratch in R</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 5 – Analysis of Variance (ANOVA)" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-5-analysis-of-variance-anova/">R for Publication by Page Piccinini: Lesson 5 – Analysis of Variance (ANOVA)</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 4 – Multiple Regression" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-4-multiple-regression/">R for Publication by Page Piccinini: Lesson 4 – Multiple Regression</a></li></ol>
DataCamp recently launched two new online R courses on time series analysis. Introduction to Time Series Analysis What You’ll Learn: Chapter One: Exploratory Time Series Data Analysis (FREE) Learn how to organize and visualize time series data in R. Chapter Two: Predicting the Future Conduct trend spotting, learn the white noise model, the random walk model, and the definition of stationary processes. Chapter Three: Correlation Analysis and the Autocorrelation Function Review the correlation coefficient, then practice estimating and visualizing autocorrelations for time series data. Chapter Four: Autoregression Discover the autoregressive model and several of its basic properties. Chapter Five: A Simple Moving Average Learn about the simple moving average model, then compare the performance of several models. Play now… ARIMA Modeling with R What You’ll Learn: Chapter One: Time Series Data and Models Investigate time series data and learn the basics of ARMA models, which can explain the behavior of such data. Chapter Two: Fitting ARMA Models Discover the wonderful world of ARMA models and learn how to fit these models to time series data. Chapter Three: ARIMA Models Learn about integrated ARMA (ARIMA) models for nonstationary time series. Chapter Four: Seasonal ARIMA Learn how to fit and forecast seasonal time series […]
I am happy to introduce the padr package, which is now available on CRAN. If you frequently work with data containing a timestamp, especially automatically created data, you might find this package helpful. It solves two problems that you can be confronted with when preparing datetime data for analysis. First, data is often recorded on too low a level for your analysis. For instance the timestamp records the moment up to the second, where you want to do the analysis on an hourly level. Second, when no events toke place there are typically no data records. This is sensible from a storage perspective, but often unhelpful for analyzing the data. When calculating a moving average for example, you want missing observations to have the value 0. You don’t want them to be lacking from your set.
I had started a “52 Vis” initiative back in 2016 to encourage folks to get practice making visualizations since that’s the only way to get better at virtually anything. Life got crazy, 52 Vis fell to the wayside and now there are more visible alternatives such as Makeover Monday and Workout Wednesday. They’re geared towards... <a href="https://rud.is/b/2017/01/18/workout-wednesday-redux-2017-week-3/">Continue reading <span>→</span></a>
[…]
<p>by Angelika Stefan &amp; Felix Schönbrodt This is the second part of “Two meanings of priors”. The first part explained a first meaning – “priors as subjective probabilities of models”. While the first meaning of priors refers to a global appraisal of existing hypotheses, the second meaning of priors refers to specific assumptions which are […]</p>
<p>The post <a rel="nofollow" href="http://www.nicebread.de/two-meanings-of-priors-2/">Two meanings of priors, part II: Quantifying uncertainty about model parameters</a> appeared first on <a rel="nofollow" href="http://www.nicebread.de/">Nicebread</a>.</p>
mapply() works with multivariate arrays, and applys a function to a set of vector or list arguments. mapply() also simplifies the output. Structure of the mapply() function: mapply(FUN, ..., MoreArgs = NULL, SIMPLIFY = TRUE, USE.NAMES = TRUE) Answers to the exercises are available here. Exercise 1 Beginning level Required dataframe: PersonnelData &lt;- data.frame(Representative=c(1:4), Sales=c(95,110,115,90), […]
If you're doing any kind of in-depth programming in the R language (say, creating a report in Rmarkdown, or developing a package) you might want to consider using a version-control system. And if you collaborate with another person (or a team) on the work, it makes things infinitely easier when it comes to coordinating changes. Amongst other benefits, a version-control system: Saves you from the worry of making irrevocable changes. Instead of keeping multiple versions of files around (are filenames like Report.Rmd; Report2.Rmd; Report-final.Rmd; Report-final-final.Rmd familiar?) you just keep the latest version of the file, knowing that the older versions...
Guest post by Jonathan Sidi, Metrum Research Group Last week the updated version of ggedit was presented in RStudio::conf2017. First, a BIG thank you to the whole RStudio team for a great conference and being so awesome to answer the insane amount of questions I had (sorry!). For a quick intro to the package see … <a href="https://www.r-statistics.com/2017/01/ggedit-0-0-2-a-gui-for-advanced-editing-of-ggplot2-objects/">Continue reading<span> &quot;ggedit 0.0.2: a GUI for advanced editing of ggplot2 objects&quot;</span></a>
Share your knowledge – speak at EARL 2017 Mango Solutions are delighted to announce the call for abstracts for EARL London and EARL San Francisco 2017. EARL (Effective Applications of the R Language) focuses on the commercial usage and business … <a href="http://www.mango-solutions.com/wp/2017/01/earl-2017-abstract-submissions-open-for-san-francisco-and-london-san-francisco-dates-and-venue-announced/">Continue reading <span>→</span></a>
We have uploaded a new version of WEC, an R package to apply ‘weighted effect coding’ to your dummy variables. With weighted effect coding, your dummy variables represent the deviation of their respective category from ...
R for beginners: Some basic graphics code to produce informative graphs, part two, working with big data A tutorial by D. M. Wiig In part one of this tutorial I discussed the use of R code to produce 3d scatterplots. This is a useful way to produce visual results of multi- variate linear regression models. … <a href="https://dmwiig.net/2017/01/16/r-for-beginners-basic-graphics-code-to-produce-informative-graphs-part-two-working-with-big-data/">Continue reading <span>R For Beginners: Basic Graphics Code to Produce Informative Graphs, Part Two, Working With Big Data</span> <span>→</span></a>
  To post your R job on the next post Just visit this link and post a new R job to the R community. You can post a job for free (and there is also a “featured job” option for extra exposure). Current R jobs Job seekers: please follow the links below to learn more and apply for your R job of interest: Featured Jobs Full-Time Quantitative/Data Scientist Signal Maritime – Posted by s.malle Glifada, Glyfada Attica, Greece 13 Jan2017 Full-Time Software Engineer Icahn School of Medicine at Mt. Sinai – Posted by dtelad11 New York New York, United States 11 Jan2017 Full-Time Data Analyst in survey design and quantitative fisheries Sampling Design And Analysis Group @ Institute of Marine Research – Posted by Sampling.Design.And.Analysis.Group Lysekil Västra Götalands län, Sweden 11 Jan2017 Full-Time Customer Success Rep RStudio – Posted by jclemens1 Anywhere 3 Jan2017 Full-Time Analyst Newell Brands – Posted by hotesforclod Hoboken New Jersey, United States 30 Dec2016 Full-Time Senior Machine Learning / Statistician / Data Scientist Outcome Referrals, Inc. – Posted by ORIJobs17 Framingham Massachusetts, United States 14 Dec2016 More New Jobs Full-Time Data Scientist @ New York, United States 7Park Data – Posted by Derek Darves New York New York, United States 13 Jan2017 Full-Time […]
It’s Davos time again. Each year the World Economic Forum (WEF) gathers the global elite together to discuss how they’re going to shape our collective future. WEF also releases their annual Global Risks Report at the same time. I read it every year and have, in the past, borrowed some risk communication visualization idioms from... <a href="https://rud.is/b/2017/01/16/the-devils-in-the-davos-details-a-quick-look-at-this-years-wef-global-risks-report/">Continue reading <span>→</span></a>
Andrew Flowers, quantitiative editor of FiveThirtyEight.com, announced at last weeks' RStudio conference the availability of a new R package containing data and analyses from some of their data journalism features: the fivethirtyeight package. (Andrew's talk isn't yet online, but you can see him discuss several of these stories in his UseR!2016 presentation.) While not an official product of the FiveThirtyEight editorial team, it was developed by Albert Y. Kim, Chester Ismay and Jennifer Chunn under their guidance. Their motivation for producing the package was to provide a resource for teaching data science: We are involved in statistics and data science...
<p>by Angelika Stefan &amp; Felix Schönbrodt When reading about Bayesian statistics, you regularly come across terms like “objective priors“, “prior odds”, “prior distribution”, and “normal prior”. However, it may not be intuitively clear that the meaning of “prior” differs in these terms. In fact, there are two meanings of “prior” in the context of Bayesian […]</p>
<p>The post <a rel="nofollow" href="http://www.nicebread.de/two-meanings-of-priors-1/">Two meanings of priors, part I: The plausibility of models</a> appeared first on <a rel="nofollow" href="http://www.nicebread.de/">Nicebread</a>.</p>
<p>A new version of choroplethr is now on CRAN. You can get it by typing the following: Motivation This new version was motivated by a few...</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2017/01/16/choroplethr-v3-5-3-now-cran/">Choroplethr v3.5.3 is now on CRAN</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
This note lists a few of the organizations that are pushing the R language forward, as of early 2017. R is happy language right now.
Historically, the R Project for Statistical Computing has been supported by the R Foundation since its inception in 2...
Our newest course, Object-Oriented Programming in R: S3 and R6 taught by Richie Cotton is now available! Object-oriented programming (OOP) lets you specify relationships between functions and the objects that they can act on, helping you manage comple...
Multilevel regression with poststratification (MrP) is a useful technique to predict a parameter of interest within small domains through modeling the mean of the variable of interest conditional on poststratification counts. This method (or methods) w...
Hi Hillary, It’s Donald, would you like to have a beer with me in La Cabra Brewing, in Berwyn, Pensilvania? (Hypothetical utilization of The Meeting Point Locator) Finding a place to have a drink with someone may become a difficult task. It is quite common that one of them does not want to move to the other’s … <a href="https://fronkonstin.com/2017/01/16/the-meeting-point-locator/">Continue reading <span>The Meeting Point Locator</span> <span>→</span></a>
After the success of last January’s “Shiny Dev Con“, our friends at RStudio are back with their first full-blown conference, “rstudio::conf“. Like our own EARL Conference, rstudio::conf was preceeded by training courses from 3 of RStudio’s big names, Hadley Wickham, … <a href="http://www.mango-solutions.com/wp/2017/01/the-wizard-world-of-rstudioconf-2017/">Continue reading <span>→</span></a>
My cousin Eli is way cooler than me. Well, to be fair, I don’t exactly set a high bar! Eli is a skydiving enthusiast with over 200 jumps to his name. And I read books and write nonsense like this…
Eli is meticulous about collecting metadata from e...
In the exercises below we cover some material on multiple regression in R. Answers to the exercises are available here. If you obtained a different (correct) answer than those listed on the solutions page, please feel free to post your answer as a comment on that page. We will be using the dataset state.x77, which […]
A strategy for gathering scattered spreadsheet data with R - A simple request
Recently I was tasked with extracting data from multiple spreadsheets so that it could be included in a report. Seemed like a fairly innocuous request, but it tur...
We are celebrating the 1st anniversary of the survminer’s release on CRAN! Due to that fact I have prepared the most (uber platinum) customized survival plot that I could imagine. I went too far because it took over 30 parameters to create a graph.....
Guest post by Ayala S. Allon, School of Psychological Sciences, Tel-Aviv University prepdat is an R package that helps researchers to optimize and speedup their analysis, providing various cross sections of the data in order to better understand the results. prepdat was created by Ayala S. Allon and Roy Luria. The full papaer about prepdat was published in Journal of Open Research Software on Nov 25, 2016, and can be downloaded here. To better understand the abilities of the package let’s look at an example. Look at the two distributions in the figure below. The difference between the means of these two distributions is significant (t(20) = 8.65, p &lt; 0.001). As can be seen, the “mdvc1” distribution is right skewed, and the “mdvc5” distribution is not skewed. It could be that the “mdvc1” distribution, because of it’s skeweness, is not best characterized by the mean, but rather by another dependent measure such as the 25th percentile. As such, one should consider looking and testing different cross sections and dependent measures of the data because it can provide information about the source of the effect in question. Yet, in many studies the comparision between experimental conditions in the statistical inference […]
Machine learning uses so called features (i.e. variables or attributes) to generate predictive models. Using a suitable combination of features is essential for obtaining high precision and accuracy. Because too many (unspecific) features pose the prob...
Over the last two years I’ve occasionally been giving a very basic tutorial to Bayesian statistics using R and Stan. At the end of the tutorial I hand out an exercise for those that want to flex their newly acquired skills. I call this exercise Bay...
In the first part I created the data for testing the Astronomical/Astrological Hypotheses. In this part, I started by fitting a simple linear regression model. mod.lm = lm(div.a.ts~vulcan.ts) summary(mod.lm) Call: lm(formula = div.a.ts ~ vulcan.ts) Residuals: Min 1Q Median 3Q Max -159.30 -53.88 -10.37 53.48 194.05 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 621.23955 […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toRegression model with auto correlated errors – Part 1, the data" rel="bookmark" href="https://datascienceplus.com/fitting-a-simple-regression-model-with-auto-correlated-errors-an-astrological-example-part-1-the-data/">Regression model with auto correlated errors – Part 1, the data</a></li><li><a title="Permanent Link toLinear Regression from Scratch in R" rel="bookmark" href="https://datascienceplus.com/linear-regression-from-scratch-in-r/">Linear Regression from Scratch in R</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 5 – Analysis of Variance (ANOVA)" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-5-analysis-of-variance-anova/">R for Publication by Page Piccinini: Lesson 5 – Analysis of Variance (ANOVA)</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 4 – Multiple Regression" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-4-multiple-regression/">R for Publication by Page Piccinini: Lesson 4 – Multiple Regression</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 3 – Logistic Regression" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-3-logistic-regression/">R for Publication by Page Piccinini: Lesson 3 – Logistic Regression</a></li></ol>
The first ever RStudio conference was held January 11-14, 2017 in Orlando, FL. For anyone else like me who spends hours each working day staring into an RStudio session, the conference was truly excellent. The speaker lineup was diverse and covered lot...
<p>The Cross Industry Standard Process for Data Mining (CRISP-DM) was a concept developed 20 years ago now. I’ve read about it in various data mining and related books and it’s come in very handy over the years. In this post, I’ll outline what the model is and why you should know about it, even if … </p>
<p><a href="http://itsalocke.com/crisp-dm/">Continue reading<span> &quot;CRISP-DM and why you should know about it&quot;</span></a></p>
<p>The post <a rel="nofollow" href="http://itsalocke.com/crisp-dm/">CRISP-DM and why you should know about it</a> appeared first on <a rel="nofollow" href="http://itsalocke.com/">Locke Data</a>.</p>
If you’re serious about learning, you probably need to read a book at some point.
These days if you want to learn applied statistics and data science tools, you have amazing options in the form of blogs, Q&amp;A sites, and massive open online course...
We recently launched a new course, Exploratory Data Analysis in R: Case Study taught by David Robinson. Dave is a Data Scientist at Stack Overflow and his interests include statistics, data analysis, education, and programming in R. In his course, yo...
The Microsoft R Server Tiger Team assists customers around the world to implement large-scale analyytic solutions. Along the way, they discover useful tips and best practices, and share them on the Tiger Team blog. Here are a few recent tips from the Tiger Team on using Microsoft R Server: Gather metadata and exlore numeric summaries of large data sets held in XDF files Filter XDF files with regular expression matching using the rxDataStep function in the RevoScaleR package Import DBase .dbf files into Microsoft R Server as an XDF file Optimize performance when using rxExec to parallelize R code across...
by Joseph Rickert Here are a few things of some interest in the R world: The Gordon and Betty Moore Foundation has joined the R Consortium! This is a very positive development for the R Consortium and the R Community in general. The Moore Foundation has been a positive force for supporting technology that can […]
It might happen that you will need a animated graph of any kind. For purposes of plotting demographic data and changes through the years, Google Maps and plotting maps, merging and converting jpg files into a animated gif, sure will give a nice visualization effect. Here is a sample of changes over the time period … <a href="https://tomaztsql.wordpress.com/2017/01/13/animated-barplot-and-google-map-with-r/">Continue reading <span>Animated barplot and google map with R</span></a>
A few months ago we launched R Course Finder, an online directory that helps you to find the right R course quickly. With so many R courses available online, we thought it was a good idea to offer a tool that helps people to compare these courses, before they decide where to spend their valuable […]
<p>Raccoon is a free web-book about Statistical Models with R. It's the collection of twenty years of notes, exercises and concepts working with statistics and R. This is the third section of the second chapter, about 1-way Anova.</p>
<p>The post <a rel="nofollow" href="http://www.quantide.com/raccoon-ch-2-3-1-way-anova/">Raccoon | Ch 2.3 – 1-way Anova</a> appeared first on <a rel="nofollow" href="http://www.quantide.com/">Quantide - R training &amp; consulting</a>.</p>
By Fang Zhou, Microsoft Data Scientist; Hong Ooi, Microsoft Senior Data Scientist; and Graham Williams, Microsoft Director of Data Science Education is a relatively late adopter of predictive analytics and machine learning as a management tool. A keen desire for improving educational outcomes for society is now leading universities and governments to perform student predictive analytics to provide better-informed and timely decision making. Student predictive analytics often aims to solve two key problems: Predict student academic outcomes so as to better target support. Predict students at risk of dropping out so as to prevent attrition. Education systems face enormous diversity...
The assumptions of simple linear regression include the assumption that the errors are independent with constant variance. Fitting a simple regression when the errors are auto-correlated requires techniques from the field of time series. If you are interested in fitting a model to an evenly spaced series where the terms are auto-correlated, I have given […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toLinear Regression from Scratch in R" rel="bookmark" href="https://datascienceplus.com/linear-regression-from-scratch-in-r/">Linear Regression from Scratch in R</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 5 – Analysis of Variance (ANOVA)" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-5-analysis-of-variance-anova/">R for Publication by Page Piccinini: Lesson 5 – Analysis of Variance (ANOVA)</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 4 – Multiple Regression" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-4-multiple-regression/">R for Publication by Page Piccinini: Lesson 4 – Multiple Regression</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 3 – Logistic Regression" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-3-logistic-regression/">R for Publication by Page Piccinini: Lesson 3 – Logistic Regression</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 2 – Linear Regression" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-2-linear-regression/">R for Publication by Page Piccinini: Lesson 2 – Linear Regression</a></li></ol>
A new version of the Discrete-Event Simulator for R was released a few days ago on CRAN. The most interesting new feature is the implementation of the subsetting operators [ and [[ for trajectory objects. Basically, think about trajectories as lists of...
The dplyr package by Hadley Wickham is a very useful package that provides “A Grammar of Data Manipulation”. It aims to simplify common data manipulation tasks, and provides “verbs”, i.e. functions that correspond to the most common data manipulation tasks. Have fun playing with dplyr in the exercises below! Answers to the exercises are available […]
A Tutorial by D. M. Wiig The R programming language has a multitude of packages that can be used to display various types of graph. For a new user looking to display data in a meaningful way graphing functions can look very intimidating. When using a statistics package such as SPSS, Stata, Minitab or even … <a href="https://dmwiig.net/2017/01/11/r-for-beginners-some-simple-code-to-produce-informative-graphs-part-one/">Continue reading <span>R for Beginners: Some Simple Code to Produce Informative Graphs, Part One</span> <span>→</span></a>
The shinydashboard package provides functions like valueBox that conveniently display basic information like summary statistics. In addition to presenting a value and subtitle on a colored background, an icon may be included as well. However, the icon must come from either the Font Awesome or Glyphicon icon libraries and cannot be image files. I’ve provided […]
In case you missed them, here are some articles from December of particular interest to R users. Power BI now has a gallery of custom visualizations built with R. Chicago's Department of Public Health uses R to prioritize health inspections at restaurants. A beautiful map of Switzerland municipalities combined with a relief map of the mountains, created with R. Using the Azure Interface Tool to parallelize the problem of optimizing an R model across the hyperparameter space. A primer on Bayesian Statistics. Animating Voronoi tesselations in R to create a greeting card. The Linux Data Science Virtual Machine, which includes...
by Joseph Rickert Last month, 217 new packages were submitted to CRAN. By my count this was the highest monthly value recorded for the past nine years. Below, are brief descriptions of fifty-two of these new-for-December packages grouped into six categories: Data, Data Science, Financial Analysis, Statistics, Utilities and Visualizations. The Financial Analysis category is […]
A colleague of mine sent me the following R question:
I have a function that takes a list and does some stuff to it and then returns
it. I then take that output and run it through the same function again. But I
obviously don’t want to repeatedly ...
A post on how to use the randomForest package in R to classify data. This focuses on classifying mushrooms as edible or poisonous.
The Delaware River Basin Commission (DRBC) just released a new app for visualizing water quality in the Delaware River Basin.  The DRBC Special Protection Waters Monitoring Program Explorer is a Shiny app that allows users to generate a boxplot and summary table of selected water quality data. The boxplot and summary table are updated each time the user selects monitoring parameter, range of years, and site type. In addition, users can display outliers (or not) and switch to logarithmic scale. The app includes links to project information and definitions.<br><br>You can view the app here (<a href="https://elainepanuccio.shinyapps.io/specialprotectionwatersexplorer/">https://elainepanuccio.shinyapps.io/specialprotectionwatersexplorer/</a>)<br><br><div><a href="https://1.bp.blogspot.com/-FkB3-Himytk/WHVm8Mv_-nI/AAAAAAAAA44/iF3-A1WHb-A8ai_JIY6YX6gqYipNsMZwwCLcB/s1600/SPWshinyapp.PNG"></a></div><br>DRBC’s Special Protection Waters (SPW) program, adopted in 1992 and expanded in 2008, prevents degradation of the Delaware River and its non-tidal tributaries where existing water quality is better than water quality standards. SPW applies from Hancock, NY to Trenton, NJ. DRBC partners with the National Park Service for monitoring and assessment. Click (<a href="http://www.nj.gov/drbc/programs/quality/spw.html">http://www.nj.gov/drbc/programs/quality/spw.html</a>) to learn more about SPW.<br><br>Many thanks and to my colleague Elaine.Panuccio@drbc.nj.gov for sharing.
We’re thrilled to officially introduce the newest product in RStudio’s product lineup: RStudio Connect. You can download a free 45-day trial of it here. RStudio Connect is a new publishing platform for all the work your teams do in R. It provides a single destination for your Shiny applications, R Markdown documents, interactive HTML widgets, […]
Plotly is a d3 based graphing library used to produce interactive and high quality graphs in R. In the following exercises, we will look at the basic plots’ syntax and some basic features in the plotly functions. We will use datasets available in base R packages. Refer to the documentation of the plotly packages when […]
With thanks to NOAA's incredible data gathering and forecasting activities, I've been obsessed with this chart for the past few days: We used to live near the Napa river where this river gage is located, and still have many friends in the area. We were in the area last weekend, when a &quot;pineapple express&quot; weather event brought an atmospheric river over much of California, with much rain and some flooding in low-lying areas. This was just before the first peak in the chart above, which shows the water level in the Napa river (in blue) along with a NOAA forecast...
Did you know that you can completely replace the “knitting” engine in R Markdown documents? Well, you can! Why would you want to do this? Well, in the case of this post, to commit the unpardonable sin of creating a clunky jupyter notebook from a pristine Rmd file. I’m definitely not “a fan” of “notebook-style”... <a href="https://rud.is/b/2017/01/10/knit-directly-to-jupyter-notebooks-from-rstudio/">Continue reading <span>→</span></a>
TL;DR Konrad Więcko and Krzysztof Słomczyński (with tiny help from my side) have created a system that is tracing what skills are currently in demand among job offers for data scientists in Poland. What skills, how frequent and how the demand is changing over time. The full description how this was done. static, +shiny. Here: … <a href="http://smarterpoland.pl/index.php/2017/01/is-it-a-job-offer-for-a-data-scientist/">Czytaj dalej <span>Is it a job offer for a Data Scientist?</span></a>
With the support of ACTINFO Chair and the (French) Institute of Actuaries, our Third Actuarial Pricing Game starts today ! There is a toolbox file available online, with a description of the game : the rules, the dates, and a description of the datasets 3 datasets : one underwriting and one claims databases, for year 0 (training data) and one underwriting dataset to enter the game Anyone can play. Students from various programs around the world, as well as practitioners are welcome to play. … <a href="http://freakonometrics.hypotheses.org/50078">Continue reading <span>Third Actuarial Pricing Game</span> <span>→</span></a>
<p>As a beginning data scientist, you’ll have quite a few subject areas that you need to learn (and eventually master). While you’ll certainly need to learn some math and statistics, math and stats are not the first things I recommend to most beginners. Almost always, I recommend that people start with data visualization. The reason […]</p>
<p>The post <a rel="nofollow" href="http://sharpsightlabs.com/blog/r-package-think-about-visualization/">The best R package for learning to “think about visualization”</a> appeared first on <a rel="nofollow" href="http://sharpsightlabs.com/">SHARP SIGHT LABS</a>.</p>
StackOverflow, the popular Q&amp;A site for programmers, provides useful information to nearly 5 million programmers worldwide with its database of questions and answers — not to mention the additional comments that other programmers provide. (You might be interested in the architecture, based SQL Server 2016, required to deliver the 8.5 billion pages Stack Overflow served last year.) Since its inception, StackOverflow has has a policy of sharing all of this content under a Creative Commons license. This represents a rich trove of unstructured data for analysis, especially given that the database of 13 million questions, 21 million answers and 54...
<p>This Wednesday at 10am I will be hosting a webinar with Philip Morgan on Positioning Your Data Science Portfolio. If you are interested in creating a Data...</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2017/01/09/upcoming-webinar-positioning-data-science-portfolio/">Upcoming Webinar: Positioning Your Data Science Portfolio</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
<span>Yesterday, I received an email from<a href="https://uk.linkedin.com/in/robert-hillman-b760a989"> Robert Hillman</a>.</span><br><span><br></span><span>Robert wrote:</span><br><blockquote><span>&quot;I’ve thoroughly enjoyed your recent posts and associated links on distributed lags. I’d like to throw in a slightly different perspective.</span></blockquote><blockquote><span> To give you some brief background on myself: I did a PhD in econometrics 1993-1998 at Southampton University. ............ I now manage capital and am heavily influenced by my study of econometrics and in particular exploring the historical foundations of many things that today that look new and funky but are probably old but no less funky!</span></blockquote><blockquote><span>I wanted to draw attention to the fact that many finance practitioners have long used ‘models’ that in my view are robust and heuristic versions of nonlinear ADL models. I’m not sure this interpretation is as widely recognised as it could be.&quot;</span></blockquote><div><span>With Robert's permission, you can access the full contents of what Robert had to say, <a href="http://web.uvic.ca/~dgiles/blog/Hillman.pdf">here</a></span><br><span><br></span><div><span>Robert provides some interesting and useful insights into the connections between certain trading models and ARDL models, and I thought that these would be useful to readers of this blog.</span></div><div><span><br></span></div><div><span>Thanks, Robert!</span></div></div><div><span><br></span></div><span>© 2017, David E. Giles </span>
When discussing how to speed up slow R code, my first question is what is your computer spec? It always surprises me when complex biological experiments, costing a significant amount of money, are analysed using a six year old laptop. A new desktop machine costs around £1000 and that money would be saved within a […]
An Example of Using nimble‘s Particle Filtering Algorithms This example shows how to construct and conduct inference on a state space model using particle filtering algorithms. nimble currently has versions of the bootstrap filter, the auxiliary particle filter, the ensemble Kalman filter, and the Liu and West filter implemented. Additionally, particle MCMC samplers are available […]
You’ve just made an amazing plot in R, and the only thing remaining is finding the right colours to use. Arghhh this part is never fun… You’re probably familiar with this loop: try some colour values -> plot -> try different colou...
An updated version of my package fancycut is on CRAN. The package has one function, fancycut(), that is more versatile than cut() for certain applications. cut() takes a numeric vector and turns it into a factor or categorical vector. It works best when they are no gaps in the intervals you have defined and when … <a href="https://adamleerich.com/2017/01/09/fancycut/">Continue reading<span> &quot;fancycut&quot;</span></a>
JSON has become part of the SQL Server in the same version as R. Both were very highly anticipated and awaited from the community. JSON has very powerful statements for converting to and from JSON for storing into / from SQL Server engine (FOR JSON and JSON VALUE, etc).  And since it is gaining popularity … <a href="https://tomaztsql.wordpress.com/2017/01/09/using-r-sp_execute_external_script-with-json/">Continue reading <span>Using R sp_execute_external_script with JSON</span></a>
A new version, now at 0.2.0, of RcppCCTZ is now on CRAN. And it brings a significant change: windows builds! Thanks to Dan Dillon who dug deep enough into the libc++ sources from LLVM to port the std::get_time() function that is missing from the 4.* ...
The rest of the month is going to be super-hectic and it’s unlikely I’ll be able to do any more to help the push to CRAN 10K, so here’s a breakdown of CRAN and GitHub new packages &amp; package updates that I felt were worth raising awareness on: epidata I mentioned this one last week... <a href="https://rud.is/b/2017/01/08/2017-01-authored-package-updates/">Continue reading <span>→</span></a>
tidyquant, version 0.2.0, is now available on CRAN. If your not already familiar, tidyquant integrates the best quantitative resources for collecting and analyzing quantitative data, xts, zoo, quantmod and TTR, with the tidy data infrastructure of the ...
Introduction
I was in need of using stock Python (Homebrew provided version) from RStudio and I found that the user path is ignored by GUI applications. I discovered that because Python packages installed via pip were not recognised from RStudio. Here'...
Due to the incrising interest in the Internet and due to the its rising number of users, one can notice the surprising growth in the demand for analyzing data and information in the Internet that were left by users and for users. Many companies and in...
When query execution performance analysis is to be done, there are many ways to find which queries might cause any unwanted load or cause stall on the server. By encouraging DBA community to start practicing the advantage or R Language and world of data science, I have created a demo to show, how statistics on … <a href="https://tomaztsql.wordpress.com/2017/01/08/clustering-executed-sql-server-queries-using-r-as-tool-for/">Continue reading <span>Clustering executed SQL Server queries using R as tool for</span></a>
This post was written by Sara Robinson and originally published on Medium, and is being re-published here since it is particularly relevant to the R community. The post analyzes all Stack Overflow comments using Google BigQuery to determine which developers are happiest and angriest.  If you have comments or ideas for future analysis, find me on Twitter @SRobTweets. It’s officially winter, so what could be better than drinking hot chocolate while querying the new Stack Overflow dataset in BigQuery? It has every Stack Overflow question, answer, comment, and more — which means endless possibilities of data crunching. Inspired by Felipe Hoffa’s post on how response time varies by tag, I wanted to look at the comments table (53 million rows!). The happiest Stack Overflow tags 🙂 To measure happy comments I looked at comments with “thank you”, “thanks”, “awesome” or “:)” in the body. I limited the analysis to tags with more than 500,000 comments. Here’s the query: #standardSQL SELECT tag, ROUND((COUNT(case when comment_text like '%thanks%' or comment_text like '%:)%' or comment_text like '%thank you%' or comment_text like '%awesome%' then 1 end) / COUNT(*)) * 100,2) as percent_happy, COUNT(*) total_comments FROM ( SELECT LOWER(a.text) as comment_text, SPLIT(b.tags, '|') as tags FROM `bigquery-public-data.stackoverflow.comments` a […]
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”.As this series of exercises comes to an end, the last part is going to be the development of a data product. Not everybody is able to code in R, so it is useful to be able to […]
Today, Rcpp passed another milestone as 900 packages on CRAN now depend on it (as measured by Depends, Imports and LinkingTo declarations). The graph is on the left depicts the growth of Rcpp usage over time.
The easiest way to compute this is to us...
In two weeks, the third year of the Data Science for Actuaries program will start. I will be there for the introduction to R. The slides are available online (created with slidify) A markdown is also available. I have to thank Ewen for his help on slidify (especially for the online quizz, and the integration of leaflet maps or the rgl animated graph….)
I’m currently writing a book as a hobby. It’s titled Functional programming and unit testing for data munging with R and you can get it for free here. You can also read it online for free on my webpage What’s the book about?
Here’s the teaser...
Last year eoda.de
launched a contest to predict when the R packages will be 10k. So this
is a really good opportunity to use (finally) the forecastHybrid package developed by Peter Ellis and David Shaub.
This will be a really KISS-naive-sim...
by Joseph Rickert The R package system continues to expand as the number of packages on CRAN is about to blow through the 10,000 mark sometime soon. This is astonishing! Not only are new packages arriving at a rate of about 190 per month, but CRAN itself continues to tick along like a well-oiled machine, […]
If you're just getting started with data science, the Sharp Sight Labs blog argues that R is the best data science language to learn today. The blog post gives several detailed reasons, but the main arguments are: R is an extremely popular (arguably the most popular) data progamming language, and ranks highly in several popularity surveys. Learning R is a great way of learning data science, with many R-based books and resources for probability, frequentist and Bayesian statistics, data visualization, machine learning and more. Python is another excellent language for data science, but with R it's easier to learn the...
In this article we will discuss the machine learning method called “decision trees”, moving quickly over the usual “how decision trees work” and spending time on “why decision trees work.” We will write from a computational learning theory perspective, and hope this helps make both decision trees and computational learning theory more comprehensible. The goal … <a href="http://www.win-vector.com/blog/2017/01/why-do-decision-trees-work/">Continue reading <span>Why do Decision Trees Work?</span></a>
In the run-up to the election last year, Ben Heubl from The Economist used the Emotion API to chart the emotions portrayed by the candidates during the debates (note: auto-play video in that link). In his walkthrough of the implementation, Ben used Python to process the video files, and R to create the charts from the sentiment scores generated by the API. Now, the learn dplyr blog has recreated the analysis using R. A detailed walkthrough steps through the process of creating a free Emotion API key, submitting a video to the API using the httr package, and retrieving the...
One of the very first learning algorithms that you’ll encounter when studying data science and machine learning is least squares linear regression. Linear regression is one of the easiest learning algorithms to understand; it’s suitable for a wide array of problems, and is already implemented in many programming languages. Most users are familiar with the […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 5 – Analysis of Variance (ANOVA)" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-5-analysis-of-variance-anova/">R for Publication by Page Piccinini: Lesson 5 – Analysis of Variance (ANOVA)</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 4 – Multiple Regression" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-4-multiple-regression/">R for Publication by Page Piccinini: Lesson 4 – Multiple Regression</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 3 – Logistic Regression" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-3-logistic-regression/">R for Publication by Page Piccinini: Lesson 3 – Logistic Regression</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 2 – Linear Regression" rel="bookmark" href="https://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-2-linear-regression/">R for Publication by Page Piccinini: Lesson 2 – Linear Regression</a></li><li><a title="Permanent Link toHow to detect heteroscedasticity and rectify it?" rel="bookmark" href="https://datascienceplus.com/how-to-detect-heteroscedasticity-and-rectify-it/">How to detect heteroscedasticity and rectify it?</a></li></ol>
In this post I introduce “GooglyPlus” another more evolved version of my earlier Shiny app “Googly”. My R package ‘yorkr’,  on which both these Shiny apps are based, has the ability to output either a dataframe or plot, depending on a parameter plot=TRUE or FALSE. My initial version of the app only included plots and … <a href="https://gigadom.wordpress.com/2017/01/05/googlyplus-yorkr-analyzes-ipl-players-teams-matches-with-plots-and-tables/">Continue reading <span>GooglyPlus: yorkr analyzes IPL players, teams, matches with plots and tables</span></a>
This is a continuation of the intermediate decision tree exercise. Answers to the exercises are available here. If you obtained a different (correct) answer than those listed on the solutions page, please feel free to post your answer as a comment on that page. Exercise 1 use the predict() command to make predictions on the […]
Which genes have homologs in many species?
In Part 1 and Part 2 I have already explored gene homology between humans and other species. But there I have only considered how many genes where shared between the species.
In this post I want to have a cl...
A few weeks back I posted an article looking at the biggest values of the 2016 fantasy season through week 13. Now that the season is over, let’s rerun the analysis and see what happens.The methodology is the same as before. Simply taking the ratio of the average draft position (ADP) to the player’s Value Over <a rel="nofollow" href="http://jessepiburn.com/2017/01/end-of-season-fantasy-round-up-draft-values-busts-and-best-free-agents-of-the-2016-season/"><span>Read more about End of Season Fantasy Round Up: Draft Values, Busts, and Best Free Agents of the 2016 Season</span>[…]</a>
We are happy to announce the MSCJ-LIFE spring school on Statistical analysis of hyperspectral and high-dimensional remote-sensing data. The spring school will take place in Jena (Germany) at the GIScience department of the Friedrich-Schiller University on March 13-17 2017.
Last week all the rstatsosphere see the beautiful Timo Grossenbacher
work.
The last month, yep, the past year I’ve working on create maps
easily with highcharter.
When I saw this chart I took as challege to replicate this nice
map in using...
Despite being in cybersecurity nigh forever (a career that quickly turns one into a determined skeptic if you’re doing your job correctly) I have often trusted various (not to be named) news sources, reports and data sources to provide honest and as-unbiased-as-possible information. The debacle in the U.S. in late 2016 has proven (to me)... <a href="https://rud.is/b/2017/01/04/the-most-important-commodity-in-2017-is-data/">Continue reading <span>→</span></a>
by Sean Wells, Senior Software Engineer, Microsoft The mrsdeploy R package facilitates Remote Execution and Web Service interactions from your local R IDE command line against a remote Microsoft R Server instance. Both core features can be used independently of one another or combined to support different convenient workflows. These different workflows composed together can produce some creative R development and operationalization solutions. mrsdeploy comes bundled with installations of Microsoft R Client and Microsoft R Server. Before using any mrsdeploy API you must first authenticate against the R Server and before we authenticate we should become familiar with the different...
Determining the number of “topics” in a corpus of documents
In my last post I finished by topic modelling a set of political blogs from 2004. I made a passing comment that it’s a challenge to know how many topics to set; the R topicmodels pack...
This blog post is about my recently released package on CRAN, textTinyR. The following notes and examples are based mainly on the package Vignette.
The advantage of the textTinyR package lies in its ability to process big text data files in ...
by Joseph Rickert Recently, I had the opportunity to interview RStudio’s Joe Cheng. Joe, the inventor and lead developer for Shiny, was the first person that J.J. Allaire invited to join the RStudio IDE project. We talked about those early days, how Shiny got started, Joe’s background as a software developer, his take on the […]
A new minor release with version number 0.6.11 of the digest package is now on CRAN and in Debian.
It is mostly a maintenance release. Sometime last spring we were asked to consider changing the license GPL-2 to GPL (>= 2). Having gotten agreement...
Mango Solutions are delighted to announce that the Effective Applications of the R Language Conference (EARL) will be held in three locations this year: San Francisco Bay Area – June 2017 London – 12-14 September 2017 Boston – November 2017 … <a href="http://www.mango-solutions.com/wp/2017/01/event-announcement-earl-conferences-2017/">Continue reading <span>→</span></a>
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”.As this series of exercises comes to an end, the last part is going to be the development of a data product. Not everybody is able to code in R, so it is useful to be able to […]
In the last post I created a gif summarizing the HadCRUT4 sea-surface temperature (HadSST3) data as its data collection evolved over time.
The goal of the last post was simply to visualize presence of any temperature measurement in a given matrix gr...
Markdown (and Rmarkdown) are great ways to quickly develop material without worrying about the formatting. The documents can then be compiled using the knitr or rmarkdown packages to output formats such as HTML, latex, or even word. The main drawback o...
It's been another great year for the R project and the R community. Let's look at some of the highlights from 2016. The R 3.3 major release brought some significant performance improvements to R, along with a spiffy new logo. There were also two updates in 2016: R 3.3.1 and R 3.3.2. (The R 3.2 series also received an update with R 3.2.4.) The R Foundation accepted five new members, and R-core accepted a new contributor as well. R’s popularity continued to grow in 2016, as measured by the 2016 O'Reilly Data Science Salary Survey (57%), IEEE Spectrum (5th rank),...
Nicolas Chopin (CREST) just posted an entry on Statisfaction about the comparison of truncated Normal algorithms run by Alan Rogers, from the University of Utah. Nicolas wrote a paper in Statistics and Computing about a simulation method, which proposes a Ziggurat type of algorithm for this purpose, and which I do not remember reading, thanks […]
<p>In last week’s blog, I explained why you should Master R (even if it may eventually become obsolete). I wrote that article to address people who claim mastering R is a bit of a waste of time (because it will eventually become obsolete). But when I suggested that R may eventually become obsolete, this seemed […]</p>
<p>The post <a rel="nofollow" href="http://sharpsightlabs.com/blog/r-recommend-data-science/">Why R is the best data science language to learn today</a> appeared first on <a rel="nofollow" href="http://sharpsightlabs.com/">SHARP SIGHT LABS</a>.</p>
I’ve been playing with Django for a while now and I’m loving it. Don’t get me wrong, I still love Shiny, but Django is pretty tough to beat for data-heavy projects and managing user sessions. For deployment, I’ve been using Heroku and am very h...
(This article was first published on Blog - Flavio Azevedo, and kindly contributed to R-bloggers) To leave a comment for the author, please follow the link and comment on their blog: Blog - Flavio Azevedo. R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
In my last post, I went over how weighting and sampling methods can help to improve predictive performance in the case of imbalanced classes. I also included an applied example with a simulated dataset that used the area under the ROC curve (AUC) as th...
If you cache code chunks when using
knitr to generate reproducible
documents then you’ve likely had the issue arrise of needing to load the results
of cached chunks into an active interactive R session. The functions
lazyload_cache_dir and lazyload_...
  To post your R job on the next post Just visit this link and post a new R job to the R community. You can post a job for free (and there is also a “featured job” option for extra exposure). Current R jobs Job seekers: please follow the links below to learn more and apply for your R job of interest: Featured Jobs Full-Time Analyst Newell Brands – Posted by hotesforclod Hoboken New Jersey, United States 30 Dec2016 Full-Time Senior Machine Learning / Statistician / Data Scientist Outcome Referrals, Inc. – Posted by ORIJobs17 Framingham Massachusetts, United States 14 Dec2016 Full-Time Data Scientist – Machine Learning St Michael’s Hospital – Posted by jmurph25 Toronto Ontario, Canada 12 Dec2016 Freelance R developer Alastair Bowman – Posted by alastairbowman@hotmail.com Anywhere 6 Dec2016 More New Jobs Full-Time Analyst Newell Brands – Posted by hotesforclod Hoboken New Jersey, United States 30 Dec2016 Full-Time Data Analyst @ Southfield, Michigan, U.S. ashley.dempsey Southfield Michigan, United States 29 Dec2016 Full-Time R package developer with statistics training @ Los Angeles, California, U.S. UCLA Department of Political Science – Posted by graemeblair Los Angeles California, United States 22 Dec2016 Full-Time Manager Biostatistics – Machine Learning (m/f) @ Rheinland-Pfalz, Germany BASF SE – Posted by karl-heinz.arendt@basf.com […]
(This article was first published on Blog - Flavio Azevedo, and kindly contributed to R-bloggers) To leave a comment for the author, please follow the link and comment on their blog: Blog - Flavio Azevedo. R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
The family got hit pretty hard with the flu right as the Christmas festivities started and we were all pretty much bed-ridden zombies up until today (2017-01-02). When in the throes of a very bad ILI it’s easy to imagine that you’re a victim of a severe outbreak, especially with ancillary data from others that... <a href="https://rud.is/b/2017/01/02/removing-personal-bias-from-flu-severity-estimation-a-k-a-misery-loves-data/">Continue reading <span>→</span></a>
Following my previous post, I wanted to spend more time, on the time series with “global weather-related disaster losses as a proportion of global GDP” over the time period 1990-2016 that Roger Pilke sent me last night. db=data.frame(year=1990:2016, ratio=c(.23,.27,.32,.37,.22,.26,.29,.15,.40,.28,.14,.09,.24,.18,.29,.51,.13,.17,.25,.13,.21,.29,.25,.2,.15,.12,.12)) In my previous post, I spend some time explaining that we should provide some sort of ‘confidence interval’ when we try to predict a pattern. That was what we call ‘model uncertainty’. But there are two (important) issues that I did not mention. (1) it … <a href="http://freakonometrics.hypotheses.org/50057">Continue reading <span>Forcasting Natural Catastrophes (is rather difficult)</span> <span>→</span></a>
#Rstatsgoes10k: Join eoda in the celebration of R and take the chance to win with your prediction! The easiest way to stay up-to-date regarding the current number of R packages is to follow eoda’s automated Twitter bot @Rstatsgoes10k. This bot regularly informs its followers of the status quo so that participants of the contest will … <a href="https://blog.eoda.de/2017/01/02/eoda-celebrates-r-dont-miss-the-date-and-win/"><span>„eoda celebrates R – don’t miss the date and win!“ </span>weiterlesen</a>
The R code for this tutorial on Methods of Distributional Semantics in R is found in the respective GitHub repository. You will find .R, .Rmd, and .html files corresponding to each part of this tutorial (e.g. DistSemanticsBelgradeR-Part2.R, DistSemant...
With my Telegram account connected to @TeleR, the force of R can always be with me, where I have data. The following is a screenshot of my mobile:
If you want to have R where you are too, you will only need a Telegram account, then search for the pu...
APPLICATION LAYOUT &amp; REACTIVITY The fourth part of our series is “separated” into two “sub-parts”. In the first one we will start building the skeleton of our application by using tabsetPanel. This is how we will separate the sections of our app and also organize its structure better. In the second part you will learn […]
My new package, tidyquant, is now available on CRAN. tidyquant integrates the best quantitative resources for collecting and analyzing quantitative data, xts, quantmod and TTR, with the tidy data infrastructure of the tidyverse allowing for seamless in...
I had a very stranger discussion on twitter (yes, another one), about regression curves. I think it started with a tweet based on some xkcd picture (just for fun, because it was New Year’s Day) “don’t trust linear regressions” https://t.co/exUCvyRd1G pic.twitter.com/O6rBJfkULa — Arthur Charpentier (@freakonometrics) 1 janvier 2017 There were comments on that picture, by econometricians, mainly about ‘significant’ trends when datasets are very noisy. And I mentioned a graph that I saw earlier, that day @AndyHarless @mileskimball actually, all that reminds me of … <a href="http://freakonometrics.hypotheses.org/50003">Continue reading <span>What is a Linear Trend, by the way?</span> <span>→</span></a>
[Reader’s Note. Some of our articles are applied and some of our articles are more theoretical. The following article is more theoretical, and requires fairly formal notation to even work through. However, it should be of interest as it touches on some of the fine points of cross-validation that are quite hard to perceive or … <a href="http://www.win-vector.com/blog/2017/01/a-theory-of-nested-cross-simulation/">Continue reading <span>A Theory of Nested Cross Simulation</span></a>
You can use R's Shiny to create an interactive dashboard with this application, make your database explorable without knowing SQL or R!
As I currently am the NPM for PISA in Colombia, I must assist to several meetings dealing with the proper implementation of this assessment in my country. Few of them are devoted to the analysis of this kind of data (coming from IRT models). As usual, ...
Like last year, lets have some fun with the Plotly package. We’ll try out Plotly’s new animation capabilities. [crayon-586941b022674144350101/] You should now have something like this: For mode details visit: Plotly for R by Carson Seivert.
A year in review
And yet again, it’s that time of the year when one eats too much and gets in a reflective mood! 2016 is nearly over, and us bloggers here at opiateforthemass.es thought it would be nice to argue endlessly which R package was th...
Daily Kos bags of words from the time of the 2004 Presidential election
This is a bit of a rambly blog entry today. My original motivation was to just explore moving data around from R into the H2O machine learning software. While successful on this,...
It’s no secret that Google Big Brothers most of us. But at least they allow us to access quite a lot of the data they have collected on us. Among this is the Google location history.
If you want to see a few ways how to quickly and easily visualize ...
I was recently trying various outlier detection algorithms. For me, the best way to understand an algorithm is to tinker with it. I built a shiny app that allows you to play around with various outlier algorithms and wanted to share it with everyone. The shiny app is available on my site, but even better, […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toCreating an animation using R" rel="bookmark" href="http://datascienceplus.com/creating-an-animation-using-r/">Creating an animation using R</a></li><li><a title="Permanent Link toThe importance of Data Visualization" rel="bookmark" href="http://datascienceplus.com/the-importance-of-data-visualization/">The importance of Data Visualization</a></li><li><a title="Permanent Link toggplot2 themes examples" rel="bookmark" href="http://datascienceplus.com/ggplot2-themes-examples/">ggplot2 themes examples</a></li><li><a title="Permanent Link toMap the Life Expectancy in United States with data from Wikipedia" rel="bookmark" href="http://datascienceplus.com/map-the-life-expectancy-in-united-states-with-data-from-wikipedia/">Map the Life Expectancy in United States with data from Wikipedia</a></li><li><a title="Permanent Link toWhat can we learn from the statistics of the EURO 2016 – Application of factor analysis" rel="bookmark" href="http://datascienceplus.com/what-can-we-learn-from-the-statistics-of-the-euro-2016-application-of-factor-analysis/">What can we learn from the statistics of the EURO 2016 – Application of factor analysis</a></li></ol>
You've been able to include user-defined charts using R in Power BI dashboards for a while now, but a recent update to Power BI includes seven new custom charts based on R in the customs visuals gallery. You can see the new chart types by visiting the Power BI Custom Visuals Gallery and clicking on the &quot;R-powered visuals&quot; tab. The new custom visuals are listed below. Click on the visual names to see an example, and click on the &quot;GitHub&quot; link at the bottom of the pop-up to see the actual R code used. Clustering, based on the NbClust package...
New Year – a new chapter, new verse, or just the same old story ? Ultimately we write it. The choice is ours. ― Alex Morritt The Analytics Lab and Cmotions wish everybody a happy year. A year full of challenges, new experiences and new knowledge. Github library(ggplot2) rm(list = ls()) # First rocket t1 […]
The Riddler of this week has an extinction riddle which summarises as follows: One observes a population of N individuals, each with a probability of 10⁻⁴ to kill the observer each day. From one day to the next, the population decreases by one individual with probability K√N 10⁻⁴ What is the value of K that […]
For GSEA analysis, we are familar with the above figure which shows the running enrichment score. But for most of the software, it lack of visualization method to summarize the whole enrichment result.
In DOSE (and related tools including clusterP...
If you followed through the Basic Decision Tree exercise, this should be useful for you. This is like a continuation but we add so much more. We are working with a bigger and badder datasets. We will be also using techniques we learned from model evaluation and work with ROC, accuracy and other metrics. Answers […]
Abstract
I present a method for reactively updating a table of acronyms from a Shiny interactive timeline using renderDataTable and timevis. The method is used in the new Shiny app, stratvis.
The stratvis app
The stratvis Shiny app provides a rich a...
There are more than 15,000 restaurants in Chicago, but fewer than 40 inspectors tasked with making sure they comply with food-safety standards. To help prioritize the facilities targeted for inspection, the City of Chicago used R to create a model that predicts which restaurants are most likely to fail an inspection. Using this model to deploy inspectors, the City is able to detect unsafe restaurants more than a week sooner than by using traditional selection methods, and cite 37 additional restaurants per month. Chicago's Department of Public Health used the R language to build and deploy the model, and made...
Hey R fans! A new episode of DataCamp's DataChats video series is out! 
In this episode, we interview Jo Hardin. Jo is a Professor of Mathematics at Pomona College with many years of R experience.  She has a pure passion for education and has been w...
Currently the news are filled with articles about the rise of machine intelligence, artificial intelligence and deep learning.
For the average reader it seems that there was this single technical breakthrough that made AI possible. While I strongly bel...
In this post, I will show you how to optimize your Rcpp loops so that they are 2 to 3 times faster than a standard implementation.
Context
Real data example
For this post, I will use a big.matrix which represents genotypes for 15,283 individuals, corresponding to the number of mutations (0, 1 or 2) at 287,155 different loci. Here, I will use only the first 10,000 loci (columns).
What you need to know about the big.matrix format:
you can easily and quickly access matrice-like objects stored on disk,
you can use different types of storage (I use type char to store each element on only 1 byte),
it is column-major ordered as standard R matrices,
you can access elements of a big.matrix using X[i, j] in R,
you can access elements of a big.matrix using X[j][i] in Rcpp,
you can get a RcppEigen or RcppArmadillo view of a big.matrix (see Appendix).
for more details, go to the GitHub repo.
Peek at the data:
print(dim(X))
## [1] 15283 10000
print(X[1:10, 1:12])
## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]
## [1,] 2 0 2 0 2 2 2 1 2 2 2 2
## [2,] 2 0 1 2 1 1 1 1 2 1 2 2
## [3,] 2 0 2 2 2 2 1 1 2 1 2 2
## [4,] 2 2 0 2 0 0 0 2 2 2 0 2
## [5,] 2 1 2 2 2 2 2 1 2 2 2 2
## [6,] 2 1 2 1 2 2 1 1 2 2 2 2
## [7,] 2 0 2 0 2 2 2 0 2 1 2 2
## [8,] 2 1 1 2 1 1 1 1 2 1 2 2
## [9,] 2 1 2 2 2 2 2 2 2 2 2 2
## [10,] 2 0 2 1 2 2 2 0 2 1 2 1
What I needed
I needed a fast matrix-vector multiplication between a big.matrix and a vector. Moreover, I could not use any RcppEigen or RcppArmadillo multiplication because I needed some options of efficiently subsetting columns or rows in my matrix (see Appendix).
Writing this multiplication in Rcpp is no more than two loops:
// [[Rcpp::depends(RcppEigen, bigmemory, BH)]]
#include &lt;RcppEigen.h>
#include &lt;bigmemory/MatrixAccessor.hpp>
<p>7 Visualizations You Should Learn in R With ever increasing volume of data, it is impossible to tell stories without visualizations. Data visualization is an art of how to turn numbers into useful knowledge. R Programming lets you learn this art by offering a set of inbuilt functions and libraries to build visualizations and present... <a href="http://www.tatvic.com/blog/7-visualizations-learn-r-2/">Read More</a></p><p>The post <a href="http://www.tatvic.com/blog/7-visualizations-learn-r-2/">7 Visualizations You Should Learn in R</a> appeared first on <a href="http://www.tatvic.com/blog">Tatvic Blog</a>.</p>
Switzerland is a country with lots of mountains, and several large lakes. While the political subdivisions (called municipalities) cover the high mountains and lakes, nothing much of economic interest happens in these places. (Raclette and sailing are wonderful, but don't count for our purposes.) For this reason, the Swiss Federal Statistical Office publishes the boundaries of the &quot;productive&quot; parts of the municipalities, and as this choropleth of average age in Swiss municpalities created by Timo Grossenbacher shows, leaving out the non-productive parts leaves us with a very different-looking Switzerland. The choropleth would be more recognizable by filling in the non-productive...
<p>A year ago i wrote about a way to authenticate shiny with Auth0, using Apache: <a href="http://blog.datascienceheroes.com/adding-authentication-to-shiny-open-source-edition/">http://blog.datascienceheroes.com/adding-authentication-to-shiny-open-source-edition/</a></p>
<p>This method works but has some issues, Sebastian Peyrott has written an excellent new blogpost that explains how to add authentication to the Open Source edition of Shiny from scratch,</p>
Yesterday we published our 100th set of exercises on R-exercises. Kudos and many thanks to Avi, Maria Elisa, Euthymios, Francisco, Imtiaz, John, Karolis, Mary Anne, Matteo, Miodrag, Paritosh, Sammy, Siva, Vasileios, and Walter for contributing so much great material to practice R programming! Even more thanks to Onno, who is working (largely) behind the scenes […]
The greatest value of a picture is when it forces us to notice what we never expected to see. — John W. Tukey. Exploratory Data Analysis.Why do we use exploratory graphs in data analysis?Understand data propertiesFind patterns in dataSuggest mod...
Abstract
I updated the R code to accompany Chapter 2-4 of the book “Real-World Machine Learning” by Henrik Brink, Joseph W. Richards, and Mark Fetherolf to be more consistent with the listings and figures as presented in the book.
rwml-R Chapters...
(Just from my point of view as a package maintainer.)New users of R might not appreciate the full benefit of CRAN and new package maintainers may not appreciate the importance of keeping their packages updated and free of warnings and errors. This is something I only came to realize myself in the last few years […]
An R tutorial by D. M. Wiig This posting contains an embedded Word document. To view the document full screen click on the icon in the lower right hand corner of the embedded document.    
by Le Zhang (Data Scientist, Microsoft) and Graham Williams (Director of Data Science, Microsoft) In data science, to develop a model with optimal performance, exploratory experiments on different sets of hyper-parameters are often performed. Preliminary analyses on smaller data can be performed on a single machine, while the experimental one on large-scale data by sweeping multi-sets of parameters can be run on a cluster to boost the computational efforts. Scalable computation resources which can be easily managed are desired for such an application scenario. This blog post shares a walk-through using the Azure Interface tool that operates and manages Azure...
Test
Answers to the exercises are available here. Exercise 1 Consider 3 vectors, day, month and year: year=c(2005:2016) month=c(1:12) day=c(1:31) Define a list Date such as: Date= $year [1] 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 $month [1] 1 2 3 4 5 6 7 8 9 10 11 12 $day […]
<p>In last week’s blog post I asked How much data science do you actually remember? It’s a critical question. If you study data science, but forget everything that you learn, you’ll be in big trouble when you go in for an interview. Or, you’ll be in big trouble if you actually get a data science […]</p>
<p>The post <a rel="nofollow" href="http://sharpsightlabs.com/blog/master-r-obsolete/">Why you should master R (even if it might eventually become obsolete)</a> appeared first on <a rel="nofollow" href="http://sharpsightlabs.com/">SHARP SIGHT LABS</a>.</p>
Data preparation and cleaning are some of the most important steps of predictive analytic and data science tasks. They are laborious, where most of the errors are made, your last line of defense against a wild data, and hold the biggest opportunities for outcome improvement. No matter how much time you spend on then, they … <a href="http://www.win-vector.com/blog/2016/12/data-preparation-long-form-and-tldr-form/">Continue reading <span>Data Preparation, Long Form and tl;dr Form</span></a>
Bayesian Inference is a way of combining information from data with things we think we already know. For example, if we wanted to get an estimate of the mean height of people, we could use our prior knowledge that people are generally between 5 and 6 feet tall to inform the results from the data we collect. If our prior is informative and we don't have much data, this will help us to get a better estimate. If we have a lot of data, even if the prior is wrong (say, our population is NBA players), the prior won't change...
ADD CONTROL WIDGETS Welcome to the third part of our series. In this part you will learn how to build and place inside your app the rest of the widgets which were mentioned in part 2. More specifically we will analyze: 1) helptext, 2) numericInput, 3) radioButtons, 4) selectInput, 5) sliderInput and 6) textInput. As […]
Data on the shadow economy?
I’m reading Kenneth Rogoff’s The Curse of Cash. It was one of Bloomberg’s Best Books of 2016 and the Financial Times’ Best Economics Books of 2016, and I recommend it. It’s an excellent and convincing book, makin...
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”. In order to be able to solve this set of exercises you should have solved the part 0, part 1, part 2,part 3, and part 4 of this series but also you should run this script which […]
Presenting ‘Googly’, a cool Shiny app that I developed over the last couple of days. This interactive Shiny app was on my mind for quite some time, and I finally got down to implementing it. The Googly Shiny app is based on my R package ‘yorkr’ which is now available in CRAN. The R package … <a href="https://gigadom.wordpress.com/2016/12/25/googly-an-interactive-app-for-analyzing-ipl-players-matches-and-teams-using-r-package-yorkr/">Continue reading <span>Googly: An interactive app for analyzing IPL players, matches and teams using R package yorkr</span></a>
# create data x &lt;- c(8,7,6,7,6,5,6,5,4,5,4,3,4,3,2,3,2,1,0.5,0.1) dat1 &lt;- data.frame(x1 = 1:length(x), x2 = x) dat2 &lt;- data.frame(x1 = 1:length(x), x2 = -x) dat1$xvar &lt;- dat2$xvar &lt;- NA dat1$yvar &lt;- dat2$yvar &lt;- NA dat1$siz &lt;- dat2$siz &lt;- NA dat1$col &lt;- dat2$col dec_threshold){ dat1$xvar[row] &lt;- row #sample(1:dat1$x1[row],1) dat1$yvar[row] &lt;- sample(1:dat1$x2[row]-1,1) dat1$siz[row] &lt;- runif(1,0.5,1.5) dat1$col[row] dec_threshold){ dat2$xvar[row] &lt;- […]
The R package samplesize4surveys contains functions that allow to calculate sample sizes for estimating proportions, means, difference of proportions and even difference of two means. It also permits the calculation of sample error and power level for ...
I’ve been asked if the adapter “let” from our R package replyr works with data.table. My answer is: it does work. I am not a data.table user so I am not the one to ask if data.table benefits a from a non-standard evaluation to standard evaluation adapter such as replyr::let. Using replyr::let with data.table looks … <a href="http://www.win-vector.com/blog/2016/12/does-replyrlet-work-with-data-table/">Continue reading <span>Does replyr::let work with data.table?</span></a>
Based on feedback ggguitar has been updated and released on CRAN.  The <a href="https://cran.r-project.org/web/packages/ggguitar/vignettes/my-vignette.html">updated vignette</a> includes more examples of how the package can be used.  Example below as well:<br><br><br><div><a href="https://3.bp.blogspot.com/-0xJV8_q0pYg/WF7R3iZxSRI/AAAAAAAAA2o/A5g0gYCRhHsEZ1cm8vtUjm7gJiSNR0oJgCLcB/s1600/stille_nacht_guitar.png"></a> </div><br><br>Update - Blogger was reformatting the R code - so made it available in <a href="https://gist.github.com/ezgraphs/2a398363152e3aaf4036abbafefc8684">this gist</a> instead.<br><br>Merry Christmas.
The book I’ve been working on these pasts months (you can read about it here, and read it for free here) is now available on Leanpub! You can grab a copy and read it on your ebook reader or on your computer, and what’s even better is that it is av...
Christmas day is soon upon us, so here's a greeting made with R: Each frame is a Voronoi Tesselation: about 1,000 points are chosen across the plane, which each generate a polygon comprising the region closer to it than any other selected point. These process is repeated for three designs (a heart, the word &quot;Merry&quot;, and the word &quot;Xmas&quot;), and the Voronoi tiles are colored according to whether the generating point is within the design. In the transitions between the designs, the points migrate from one selected point in one design to a selected counterpoint in the next. The counterpoint...
Make it easy to make ensemble time series forecast
forecastHybrid is an R package to make it easier to use the average predictions of ‘ensembles’ (or ‘combinations’) of time series models from Rob Hyndman’s forecast package. It looks after t...
Introduction My last blog post in 2016 on SQL Server 2016….. Some years ago, I have heard predictions from ‘experts‘ that within a few years Hadoop / Spark systems would take over traditional RDBMS’s like SQL Server. I don’t think … <a href="https://longhowlam.wordpress.com/2016/12/23/did-you-say-sql-server-yes-i-did/">Continue reading <span>→</span></a>
by Joseph Rickert The 2017 R Conference season will get off to an early start on January 13th and 14th with RStudio::Conf 2017 in Orlando, Florida. The schedule promises an intense but collegial experience with plenty of hands-on practice working with R and the RStudio tool chain of packages and products. To prepare for the […]
I saw Simon Jackson’s recent blog post regarding ordering categories within facets. He proposed a way of dealing with the problem of ordering variables shared across facets within facets. This problem becomes apparent in text analysis where words are shared … <a href="https://trinkerrstuff.wordpress.com/2016/12/23/ordering-categories-within-ggplot2-facets/">Continue reading <span>→</span></a>
The startup package makes it easy to control your R startup processes and to share part of your startup settings with others (e.g. as a public Git repository) while keeping secret parts to yourself. Instead of having long and windy .Renviron and .Rpro...
You are a consultant who has been hired by a business that sells one commodity. The business owner wants to know what to expect by the end of January.
With the upcoming holidays, I thought it fitting to finally explore the ttbbeer package. It contains data on beer ingredients used in US breweries from 2006 to 2015 and on the (sin) tax rates for beer, champagne, distilled spirits, wine and various tob...
An R user asked a question regarding whether it’s possible to have the RStudio pipe (%>%) shortcut (Cmd-Shift-M) available in other macOS applications. If you’re using Alfred then you can use this workflow for said task (IIRC this requires an Alfred license which is reasonably cheap). When you add it to Alfred you must edit... <a href="https://rud.is/b/2016/12/22/pipes-everywhere/">Continue reading <span>→</span></a>
Consider the problem of “parametric programming” in R. That is: simply writing correct code before knowing some details, such as the names of the columns your procedure will have to be applied to in the future. Our latest version of replyr::let makes such programming easier. Archie’s Mechanics #2 (1954) copyright Archie Publications (edit: great news! … <a href="http://www.win-vector.com/blog/2016/12/comparative-examples-using-replyrlet/">Continue reading <span>Comparative examples using replyr:let</span></a>
We are committed to bringing you 100% authentic exercise sets. We even try to include as different datasets as possible to give you an understanding of different problems. No more classifying Titanic dataset. R has tons of datasets in its library. This is to encourage you to try as many datasets as possible. We will […]
2017 is bound to be an exciting year in Data Science. Here's DataCamp's list of conferences that we're most excited about in the new year. Whether you're an R user, a Python hacker, or just a general data science fan - you're sure to find a great confe...
If you've been thinking about trying out the Data Science Virtual Machine on Linux, but don't yet have an Azure account, you can now take a free test drive -- no credit card required! Just visit the Linux DSVM Marketplace page and click the blue button: The Linux Data Science Virtual Machine includes all of the tools a modern data scientist needs, in one easy-to-launch package. With it, you can try exploring data with Apache Drill, train deep neural networks for computer vision with MXNet, develop AI applications with the Cognitive Toolkit, or create statistical models with big data in...
This is another post of my series about how my packages integrate into a pipe-friendly workflow. The post focusses on my sjmisc-package, which was just updated on CRAN, and highlights some of the new features. Examples are based on data from the European Social Survey, which are freely available. Steps of the data analysis process […]<img height="1" alt="" width="1" src="https://pixel.wp.com/b.gif?host=strengejacke.wordpress.com&blog=10592659&post=1204&subd=strengejacke&ref=&feed=1" border="0">
Dear All, We are pleased to announce that the useR! 2017 Conference will take place July 04-08, Brussels, Belgium.  As always, we aim to bring together many participants from the R community. 16 tutorials will be given on July 04, 2017 followed by the main conference onJuly 05, 06 and 07, 2017. More details about the topics, program, social events, accommodation and invited speakers can  be found at http://user2017.brussels/ and on Twitter @UseR_Brussels. We hope to see you in Brussels in July. Best regards, Ziv Shkedy, Tobias Verbeke, Heather Turner.
by Sean Lopp At RStudio, we work with many companies interested in scaling R. They typically want to know: How can R scale for big data or big computation? How can R scale for a growing team of data scientists? This post provides a framework for answering both questions. Scaling R for Big Data or […]
<p>Introduction Tree roots growing under sidewalks often cause cracking or lifting of the pavement once the tree surpasses a certain size. This creates significant tripping hazards […]</p>
<p>The post <a rel="nofollow" href="http://blog.nycdatascience.com/student-works/capstone/tree-troubles-predicting-sidewalk-damage-resulting-trees-nyc/">Tree Troubles -- Predicting Sidewalk Damage Resulting From Trees In NYC</a> appeared first on <a rel="nofollow" href="http://blog.nycdatascience.com/">NYC Data Science Academy Blog</a>.</p>
by Alan Weaver, Advanced Analytics Specialist at Microsoft Very often data scientists and analysts require access to back-end resources on Azure. For example, they may need to start a virtual machine or resize a Hadoop cluster. This typically requires making a request to the IT department and patiently waiting. AzureSMR is a simple R package that enables those users to do many of those operations themselves. It’s very easy to script commonly-used functions which can be run without having to navigate the portal or wizards. AzureSMR uses the Azure Systems Management API and leverages standard packages such as httr, so...
Where should you buy a house to get the most value for your money? Your first step might be to make a map, but spatial data can be intimidating in R because of the complicated objects it often lives in. Our new course Working with Geospatial Data in R ...
I recently gave a presentation for analysts and data modellers at the Department for Work and Pensions (DWP) introducing the spatial microsimulation technique (specifically the IPF flavour), and below are the slides I used (use spacebar to navigate through the slides): Much of the content is based on material from Spatial Microsimulation with R by […]
R will soon reach a magic milestone – but how soon exactly? Predict the date and win! Over the past years, the popularity of R has unstoppably increased – and so has the number of R packages available on CRAN. As a result, the international R community will soon reach the next big milestone: 10,000 … <a href="https://blog.eoda.de/2016/12/21/rstatsgoes10k-eoda-is-hosting-a-contest-in-celebration-of-r/"><span>„#Rstatsgoes10k – eoda is hosting a contest in celebration of R“ </span>weiterlesen</a>
Today I spent a few hours debugging a hang R process that left a zombie sh which so far suggests bug (race condition?) in R’s system2() call. Anyway, it soon turned out that the only way to see what’s happening with R is to use gdb, which I personally dread. It is so because I haven’t found … <a href="https://random-remarks.net/2016/12/21/inspecting-r-in-gdb-with-python/">More <span>Inspecting R in GDB (with Python)</span></a><img height="1" alt="" width="1" src="https://pixel.wp.com/b.gif?host=random-remarks.net&blog=72810971&post=441&subd=randomremarksblog&ref=&feed=1" border="0">
<p>Today I am happy to announce a new Facebook group: Data Science Portfolios. This group is for anyone interested in creating a portfolio of data science...</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2016/12/20/new-facebook-group-data-science-portfolios/">New Facebook Group: Data Science Portfolios</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
Hey R fans! A new episode of DataCamp's DataChats video series is out! 
In this episode, we interview Ronald Pearson. Ron holds a Ph.D. in Electrical Engineering and Computer Science from M.I.T. and has written or co-written five books. Furthermore...
[For this exercise, before proceeding, first read the rvest package help and the selectorgadget help.] Answers to the exercises are available here. Exercise 1 Consider the url ‘http://statbel.fgov.be/en/statistics/figures/economy/indicators/prix_prod_con/’ Extract all the information load on table ‘Third Quarter 2016’. Exercise 2 Consider the url ‘http://www2.sas.com/proceedings/sugi30/toc.html’ Extract all the papers names, from 001-30 to 268-30 Exercise 3 […]
Even though ensembles of trees (random forests and the like) generally have better predictive power and robustness, fitting a single decision tree to data can often be very useful for: understanding the important variables in a data set exploring unusual subsegments of the data (and the explanatory variables that define them) presenting a simple, decision-based model to management to explain behaviors in data illustrating a model graphically But to get the best out of a decision tree, you need to be able to look at it, interact with it, and able to present it attractively. This blog post by Longhow...
I recently mentioned that I’ve been working on a development version of an Apache Drill R package called sergeant. Here’s a lifted “TLDR” on Drill: Drill supports a variety of NoSQL databases and file systems, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift, NAS and local files. A... <a href="https://rud.is/b/2016/12/20/sergeant-a-r-boot-camp-for-apache-drill/">Continue reading <span>→</span></a>
<p>How many data science books have you read? 5? 10? A few dozen? How many free online courses have you taken? A few? How many blog posts have you read? (I’d be willing to bet: you’ve read dozens.) If you’re like most budding data scientists, you’ve probably consumed a lot of material. You probably even […]</p>
<p>The post <a rel="nofollow" href="http://sharpsightlabs.com/blog/data-learning-vs-remembering/">How much data science do you actually <i>remember</i>?</a> appeared first on <a rel="nofollow" href="http://sharpsightlabs.com/">SHARP SIGHT LABS</a>.</p>
Numerical optimization is an important tool in the data scientist's toolbox. Many classical statistical problems boil down to finding the highest (or lowest) point on a multi-dimensional surface: the base R function optim provides many techniques for solving such maximum likelihood problems. Counterintuitively, numerical optimizations are easiest (though rarely actually easy) when all of the variables are continuous and can take any value. When integer variables enter the mix, optimization becomes much, much harder. This typically happens when the optimization is constrained by a limited selection of objects, for example packages in a weight-limited cargo shipment, or stocks in a...
It’s a common situation to have data from multiple processes in a “long” data format, for example a table with columns measurement and process_that_produced_measurement. It’s also natural to split that data apart to analyze or transform it, per-process — and then to bring the results of that data processing together, for comparison. Such a work … <a href="http://www.win-vector.com/blog/2016/12/a-simple-example-of-using-replyrgapply/">Continue reading <span>A Simple Example of Using replyr::gapply</span></a>
Almost every biomedical research paper requires a “Table 1: baseline patient
characteristics.” Many developers have published tools to help streamline the
construction of such tables. The qwraps2::summary_table function is my
contribution to the ...
The longurl package has been updated to version 0.3.0 as a result of a bug report noting that the URL expansion API it was using went pay-for-use. Since this was the second time a short URL expansion service either went belly-up or had breaking changes the package is now completely client-side-based and a very thin,... <a href="https://rud.is/b/2016/12/18/package-update-longurl-0-3-0-is-hitting-cran-mirrors/">Continue reading <span>→</span></a>
ADD CONTROL WIDGETS In the second part of our series you will see how to add control widgets in your Shiny app. Widget is a web element that your users can interact with. The widgets provided by Shiny are: FUNCTIONS actionButton: Action Button checkboxGroupInput: A group of check boxes checkboxInput: A single check box dateInput: […]
Earlier this week, Conrad released Armadillo 7.600.1. The corresponding RcppArmadillo release 0.7.600.1.0 is now on CRAN and in Debian. This follows several of rounds testing at our end with a full reverse-dependency of a pre-release version followe...
There’s nothing more frustrating than waiting for long-running R scripts to iteratively run. I’ve recently come across a new-ish package for parallel processing that plays nicely with the tidyverse: multidplyr. The package has saved me countless ho...
Power BI Desktop enable users to use R script visual for adding custom visualization generated with R language – regardless of R package used. Before using R script visual, you will need to enable it by setting path to R Engine on your client in the global options. Once this is done, you will be … <a href="https://tomaztsql.wordpress.com/2016/12/18/r-graphs-and-tables-in-power-bi-desktop/">Continue reading <span>R graphs and tables in Power BI Desktop</span></a>
This app is based on the gwascat R package and its ebicat38 database and shows trait-associated SNP locations of the human genome. You can visualize and compare the genomic locations of up to 8 traits simultaneously.
The National Human Genome Research...
I missed it first time around but apparently, back in October, Nature published a somewhat-controversial article: Evidence for a limit to human lifespan. It came to my attention in a recent tweet: Just wow https://t.co/fupXIOAC43 pic.twitter.com/vsxT3VyTg6 — Nick Loman (@pathogenomenick) December 11, 2016 The source: a fact-check article from Dutch news organisation NRC titled “Nature […]
A bit more on our replyr R package. library(&quot;replyr&quot;) help(let, package='replyr') let {replyr} R Documentation Prepare expr for execution with name substitutions specified in alias. Description replyr::let implements a mapping from desired names (names used directly in the expr code) to names used in the data. Mnemonic: &quot;expr code symbols are on the left, external … <a href="http://www.win-vector.com/blog/2016/12/helplet-packagereplyr/">Continue reading <span>help(let, package=’replyr’)</span></a>
It was just one of our hobby projects we do on the side. Just one of those projects we do to try out R-code, visualization techniques (i.e. ggplot2 code, mostly), and efficient collaborative reporting via RMarkdown. But then Lukas couldn’t sleep and instead decided to work on the project a little, creating a particular plot […]
PLOTTING TREES + DATA is difficult. @DrKatHolt developed plotTree (R and Python scripts) to visualize associated data with trees, e.g. heatmap, horizontal bar etc.
I reproduced the examples presented in the plotTree repo using ggtree. Source code is...
Pat Bagley is easily my favorite political cartoonist, period. For the politically aware in Utah, he is almost legendary, enjoying superstar status. I’ve been aware of him since I was a kid, and I always loved his cartoons. Not only does his artistic style appeal to me, he has a way of illustrating a situation…<a href="https://ntguardian.wordpress.com/2016/12/16/political-cartoon-markov-chain/">Read more <span>A Political Cartoon and a Markov Chain</span></a>
This post is part of a series of &quot;learning everything with R: An R book list&quot;. You can clink on this link to see other relevant posts. RStudio probably is the most popular interface to use R. But it definitely does not just serve as a code editor (than...
It is interesting to try and figure out what picture emerges from a join-the-dots puzzle (connect-the-dots in some parts of the world). Let’s have a go at some lightweight automatic generation such a puzzle (some heavy-weight techniques). If an image is available, expressed as an boolean matrix, R’s sample function can be used to select […]
I’ve been drafting a new R package — sergeant — to work with Apache Drill and have found it much easier to manage having Drill operating in a single node cluster vs drill-embedded mode (esp when I need to add a couple more nodes for additional capacity). That means running Apache Zookeeper and I’ve had... <a href="https://rud.is/b/2016/12/16/minding-the-zookeeper-with-r/">Continue reading <span>→</span></a>
<p>
For a while now, I’ve been wanting to experiment with rOpenSci’s <strong>pangaear</strong> package <span>(Chamberlain et al. 2016)</span>, which allows you to search, and download data from, the Pangaea, a major data repository for the earth and environmental sciences. Earlier in the year, as a member of the editorial board of <a href="http://www.nature.com/sdata/">Scientific Data</a>, Springer Nature’s open data journal I was handling a data descriptor submission that described a new 2,200-year foraminiferal δ18O record from the Gulf of Taranto in the Ionian Sea <span>(Taricco et al. 2016)</span>. The data descriptor was recently <a href="http://doi.org/10.1038/sdata.2016.42">published</a> and as part of the submission Carla Taricco deposited the data set in Pangaea. So, what better opportunity to test out <strong>pangaear</strong>? (Oh and to fit a GAM to the data while I’m at it!)
</p>
<div>
<div>
<p>
Chamberlain, Scott, Kara Woo, Andrew MacDonald, Naupaka Zimmerman, and Gavin Simpson. 2016. Pangaear: Client for the ’Pangaea’ Database. <a href="https://cran.r-project.org/package=pangaear">https://CRAN.R-project.org/package=pangaear</a>.
</p>
</div>
<div>
<p>
Taricco, Carla, Silvia Alessio, Sara Rubinetti, Gianna Vivaldo, and Salvatore Mancuso. 2016. “A Foraminiferal <span>()</span>18O Record Covering the Last 2,200 Years.” Scientific Data 3 (June). Nature Publishing Group: 160042. doi:<a href="https://doi.org/10.1038/sdata.2016.42">10.1038/sdata.2016.42</a>.
</p>
</div></div>
by Jonathan Regenstein In a previous post, we built an R Notebook that laid the groundwork for a Shiny app that allows users to graph country ETFs by clicking on a world map. In today’s Fun Friday post, we’ll charge forth to build that app, again using a flexdashboard so that we can stay in […]
As Google learned, predicting the spread of influenza, even with mountains of data, is notoriously difficult. Nonetheless, bioinformatician and R user Shirin Glander has created a two-part tutorial about predicting flu deaths with R (part 2 here). The analysis is based on just 136 cases of influenza A H7N9 in China in 2013 (data provided in the outbreaks package) so the intent was not to create a generally predictive model, but by providing all of the R code and graphics Shirin has created a useful example of real-word predictive modeling with R. The tutorial covers loading and cleaning the data...
The problem of weighting the type 1,2 errors on binary classification came up in a forum I visit. My solution: Here we can see the different loss behaviours We will try to classify the Virginica species from the iris dataset   And now the loss functions:
This is a continuation of the exercise Basic Tree 1 Answers to the exercises are available here. If you obtained a different (correct) answer than those listed on the solutions page, please feel free to post your answer as a comment on that page. Exercise 1 load the tree library. If it is not installed […]
&quot;Big Data&quot; generates a lot of news these days, but sometimes small data still means big computation. Indiana's Department of Workforce Development has the responsibility to forecast future employment rates in the State of Indiana. And not just the number of jobs available: the department also needs to forecast the types of jobs that will be available, so the administration can link up future job requirements with training and education policies and make sure a skilled workforce can fill the future demand. The State of Indiana contracted with analytics professional services firm Inquidia Consulting to develop a system to generate...
Help? I set an exam question with 10 boxes to be filled from 13 possible answers. If randomly completed, what should be the average mark?
- @KeithRMatthews, 12:13 PM - 15 Dec 2016
library(ggplot2)
answers &lt;- letters[1:10]
options &lt;- letters[...
Consider the common following problem: compute for a data set (say the infamous iris example data set) per-group ranks. Suppose we want the rank of iris Sepal.Lengths on a per-Species basis. Frankly this is an “ugh” problem for many analysts: it involves all at the same time grouping, ordering, and window functions. It also is … <a href="http://www.win-vector.com/blog/2016/12/organize-your-data-manipulation-in-terms-of-grouped-ordered-apply/">Continue reading <span>Organize your data manipulation in terms of “grouped ordered apply”</span></a>
In the spirit of Seinfeld, this post will also &quot;be about nothing.&quot; Data visualizations in R may shed some light on how these characters are related.
On the other side of the Holidays are sunny January days in Florida, home of rstudio::conf 2017 and the exclusive Wizarding World of Harry Potter experience reserved for our guests! The speakers and sponsors are set and the Gaylord Resort is ready. The entire RStudio team, led by masters of ceremony Hadley Wickham and Joe Cheng, […]
<p>
Eighteen months ago I <a href="http://www.fromthebottomoftheheap.net/2014/06/16/simultaneous-confidence-intervals-for-derivatives/">wrote a post</a> in which I described the use of simulation from the posterior distribution of a fitted GAM to derive simultaneous confidence intervals for the derivatives of a penalised spline. It was a nice post that attracted some interest. It was also wrong. I have no idea what I was thinking when I thought the intervals described in that post were simultaneous. Here I hope to rectify that past mistake.
</p>
<p>
I’ll tackle the issue of simultaneous intervals for the derivatives of penalised spline in a follow-up post. Here, I demonstrate one way to compute a simultaneous interval for a penalised spline in a fitted GAM. As example data, I’ll use the strontium isotope data set included in the <strong>SemiPar</strong> package, and which is extensively analyzed in the monograph Semiparametric Regression <span>(Ruppert, Wand, and Carroll 2003)</span>. First, load the packages we’ll need as well as the data, which is data set <code>fossil</code>. If you don’t have <strong>SemiPar</strong> installed, install it using <code>install.packages(“SemiPar”)</code> before proceeding
</p>
<div>
<div>
<p>
Ruppert, David, M P Wand, and R J Carroll. 2003. Semiparametric Regression. Cambridge University Press.
</p>
</div></div>
Gábor Csárdi — Mango Solutions Joe Cheng — RStudio Introduction It is not uncommon that the user interface of a Shiny application needs to be generated dynamically, based on data or program state. One typical use case that we encounter … <a href="http://www.mango-solutions.com/wp/2016/12/dynamically-generated-shiny-ui/">Continue reading <span>→</span></a>
This is pure nerdism. There is a project to create a shapfile
from a fictional world, the Middle Earth. The data is in this
https://github.com/jvangeld/ME-GIS repository. The author of the r-chart.com web site made a ggplot version of this map ...
by Ali Zaidi, Data Scientist at Microsoft In previous post we showcased the use of the sparklyr package for manipulating large datasets using a familiar dplyr syntax on top of Spark HDInsight Clusters. In this post, we will take a look at the RxSpark API for R, part of the RevoScaleR package and the Microsoft R Server distribution of R on HDInsight. We'll use RxSpark to visualize a dataset of 140M taxi rides between boroughs in New York City. Dealing with data in distributed storage and programming with concurrent systems often requires learning complicated new paradigms and techniques. Statisticans and...
by Jonathan Regenstein It’s the holiday season, and that can mean only one thing: time to build a leaflet map as an interface to country Exchange Traded Fund (ETF) data! In previous posts, we examined how to import stock data and then calculate and display the Sharpe Ratio of a portfolio. Today, we’re going to […]
Grouping objects into clusters is a frequent task in data analysis. In this set of exercises we will use hierarchical clustering to cluster European capitals based on their latitude and longitude. Before trying out this exercise please make sure that you are familiar with the following functions: dist, hlcust, cutree, rect.hclust We will be using […]
In my last post I created a gene homology network for human genes. In this post I want to extend the network to include edges for other species.
First, I am loading biomaRt and extract a list of all available datasets.
library(biomaRt)
ensembl = useM...
We just launched Data Visualization in R taught by Ronald Pearson! R supports four different graphics systems: base graphics, grid graphics, lattice graphics, and ggplot2. Base graphics is the default graphics system in R, the easiest of the four syst...
  To post your R job on the next post Just visit this link and post a new R job to the R community. You can post a job for free (and there is also a “featured job” option for extra exposure). Current R jobs Job seekers: please follow the links below to learn more and apply for your R job of interest: Featured Jobs Full-Time Senior Machine Learning / Statistician / Data Scientist Outcome Referrals, Inc. – Posted by ORIJobs17 FraminghamMassachusetts, United States 14 Dec2016 Full-Time Data Scientist – Machine Learning St Michael’s Hospital – Posted by jmurph25 TorontoOntario, Canada 12 Dec2016 Freelance R developer Alastair Bowman – Posted by alastairbowman@hotmail.com Anywhere 6 Dec2016 Full-Time Data Scientist and R Programmer Metrum Research Group – Posted by billk SimsburyConnecticut, United States 28 Nov2016 Full-Time Principal Big Data Architect Carbon Black Inc. – Posted by CarbonBlackInc Anywhere 18 Nov2016 Freelance Part time statistician / data scientist Gradient – Posted by tvladeck Anywhere 17 Nov2016 More New Jobs Full-Time Senior Machine Learning / Statistician / Data Scientist Outcome Referrals, Inc. – Posted by ORIJobs17 FraminghamMassachusetts, United States 14 Dec2016 Full-Time Data Analyst/R Shiny Programmer for Magid’s “EmotionalDNA™” Frank N. Magid Associates – Posted by nclayton Anywhere 13 Dec2016 Full-Time APS6 […]
A while back, I created the small package Roxford to access Microsoft’s Cognitive Services API in order to easily recognize objects in images.
Back then Microsoft called the service “Project Oxford”, hence the name “Roxford”. Since then Micro...
Often I find myself needing to search for .R or or .Rmd files in which I have used a specific function. There are a variety of searches that you can do; however, I wanted something that would work recursively at the command line of a Linux or Mac termi...
R picked up a nifty way to organize sequential calculations in May of 2014: magrittr by Stefan Milton Bache and Hadley Wickham. magrittr is now quite popular and also has become the backbone of current dplyr practice. If you read my last article on assignment carefully you may have noticed I wrote some code that … <a href="http://www.win-vector.com/blog/2016/12/magrittrs-doppelganger/">Continue reading <span>magrittr’s Doppelgänger</span></a>
There are some graphs that you cannot forget. One graph that I found puzzling was mentioned on Andrew Gelman’s blog, a few years back, and was related to rupture detection What I remember from this graph is that if you want to get a rupture, you can easily find one… Recently, I had to review a paper, and Imbens &amp; Lemieux (2008) was mentioned (the paper on Regression Discontinuity). It is a great reference and a major contribution, undoublty. But now, each time I … <a href="http://freakonometrics.hypotheses.org/49848">Continue reading <span>Rupture Detection</span> <span>→</span></a>
If you're looking to get started with data science in R, a great place to start is OnePageR by Graham Williams. (Graham is the creator of Rattle, author of Data Mining with Rattle and R, and Director of Data Science at Microsoft.) This free (CC-licensed) resource is a series of hands-on mini-chapters and associated R code, organized into four main topic areas: Data Science: introductions to data science, data mining, literate programming, and the R language Dealing with Data: Reading data files and open access data, basic explorations and visualizations, and two case studies Building Models: with tutorials for many...
[For this exercise, we will work using the package rpart. This is a beginner level exercise. Please refer to the help of rpart package] Answers to the exercises are available here. Exercise 1 Consider the Kyphosis data frame(type help(‘kyphosis’) for more details), that contains: -Kyphosis:a factor with levels absent present indicating if a kyphosis (a […]
This is the fourth part of the of the post where are going to create figure 4 Plot of Inventory Completeness against sample size for grid cells. Part 3 of this series we created chronohorogram for understanding seasonality by year of the data records. If you have not already done so, please follow steps in Part 1 of the post […]
Hey R fans! A new episode of DataCamp's DataChats video series is out! 
In this episode, we interview David Robinson. David is a Data Scientist at Stack Overflow and well known for his blog Variance Explained. He talks about his passion for bloggin...
Here is a short post which ranks IPL batsmen and bowlers post the 2016 IPL season. These are based on match data from Cricsheet. I had already ranked IPL players in my post yorkr ranks IPL batsmen and bowlers, but that was mid IPL 2016 season. This post will be final ranking post 2016 season … <a href="https://gigadom.wordpress.com/2016/12/13/yorkr-ranks-ipl-players-post-2016-season/">Continue reading <span>yorkr ranks IPL Players post 2016 season</span></a>
In answering yet another question on X validated about the numerical approximation of the marginal likelihood, I suggested using an harmonic mean estimate as a simple but worthless solution based on an MCMC posterior sample. This was on a toy example with a uniform prior on (0,π) and a “likelihood” equal to sin(θ) [really a […]<img height="1" alt="" width="1" src="https://pixel.wp.com/b.gif?host=xianblog.wordpress.com&blog=5051449&post=32177&subd=xianblog&ref=&feed=1" border="0">
R has a number of assignment operators (at least “&lt;-“, “=“, and “->“; plus “&lt;&lt;-” and “->>” which have different semantics). The R-style guides routinely insist on “&lt;-” as being the only preferred form. In this note we are going to try to make the case for “->” when using magrittr pipelines. Don Quijote and … <a href="http://www.win-vector.com/blog/2016/12/the-case-for-using-in-r/">Continue reading <span>The Case For Using -> In R</span></a>
I just wrote a new chapter for my students describing how to subset a data frame in R. The full text is available at https://docs.google.com/document/d/1K5U11-IKRkxNmitu_lS71Z6uLTQW_fp6QNbOMMwA5J8/edit?usp=sharing but here’s a preview: Let’s load in ChickWeight, […]
Len Kiefer, Deputy Chief Economist at Freddie Mac, recently posted an analysis of global housing price trends based on the international house price database (from the Dallas Fed). Using those data, Kiefer compared housing trends price increases (and in a couple of places like Spain and Ireland, decreases) across 24 countries. He also looks at the change in affordability of houses, comparing housing increases with disposable income growth. (Belgium is a surprising outlier in this category, where housing cost has inreased 1.8 times faster than income.) But I thought the most interesting chart was the animation of house prices below:...
INTRODUCTION TO SHINY Shiny is a package from RStudio that can be used to build interactive web pages with RStudio which is is an open source set of integrated tools designed to help you be more productive with R and you can download it from here. Use the examples in this tutorial to “take a […]
The environmental Kuznets curve hypothesizes an inverted u-shape relationship between income and environmental degradation: while richer countries tend to invest more ressources into environmental protection as their income rises, income growth in poor...
The html widget I produced with networksD3 seems to create problems with loading the site for some people, so here is the same blog post as yesterday just without the rendered html widget in it.
This week I want to explore how many of our human ge...
I was pleased to see today’s post, “(A Very) Experimental Threading in R,” by Lukasz Bartnik, as this is a long-standing interest of mine. My own effort in this direction has been my package Rdsm. The notion of threading, for those who may not have this background, refers to several instances of a program, in this … <a href="https://matloff.wordpress.com/2016/12/11/threading-in-r/">Continue reading <span>Threading in R?</span> <span>→</span></a>
Well, I tried to write a blog post using the <a target="_blank" href="http://www.rstudio.com/">RStudio</a> Rmarkdown system, and utterly failed. Thus, I set up a system where I could write from RStudio. So I set up a Github pages blog at <a href="http://randomjohn.github.io/">randomjohn.github.io</a>. There I can easily write and publish posts involving data analysis.
Mathematics knows no races or geographic boundaries; for mathematics, the cultural world is one country (David Hilbert) One of the best decisions I took this year related with this blog was to move it to my own self-hosted domain using WordPress.org. It allows to me, for example, to embed dynamic JavaScript visualizations like this one. … <a href="https://fronkonstin.com/2016/12/12/three-shiny-apps-to-celebrate-the-beauty-of-maths/">Continue reading <span>Three Shiny Apps to Celebrate the Beauty of Maths</span> <span>→</span></a>
We are working on a series of comic books that introduce statistical thinking and could be used as activity booklets in primary schools. Stories are built around adventures of siblings: Beta (skilled mathematician) and Bit (data hacker). What is the connection between these comic books and R? All plots are created with ggplot2. The first … <a href="http://smarterpoland.pl/index.php/2016/12/how-to-weigh-a-dog-with-a-ruler-looking-for-translators/">Czytaj dalej <span>How to weigh a dog with a ruler? (looking for translators)</span></a>
The Type-I Pareto distribution has a probability function shown as below f(y; a, k) = k * (a ^ k) / (y ^ (k + 1)) In the formulation, the scale parameter 0 &lt; a &lt; y and the shape parameter k > 1 . The positive lower bound of Type-I Pareto distribution is particularly […]
<p>Coloring edges in ggtree is quite easy, as we can map the color to numerical or categorical values via the <code>aes(color=VAR)</code> syntax. For user’s own data, it is also easy as <code>ggtree</code> provide the <code>%&lt;+%</code> operator to attach user data.</p>
<p>But as it seems not so obviously for <code>ggtree</code> users, see question <a href="https://groups.google.com/forum/#!topic/bioc-ggtree/4GgivKqVjB8">1</a>, <a href="https://github.com/GuangchuangYu/ggtree/issues/76">2</a>, and <a href="https://github.com/GuangchuangYu/ggtree/issues/105">3</a>, I will demonstrate how to color edges using user data here.</p>
<p>There are several solutions to this question. Let’s first start with a hard one by using <code>phylo4d</code> class defined by <code>phylobase</code> package:</p>
<pre><code>require(ggtree)
require(phylobase)

set.seed(123)
tr = rtree(30)
g1 = as(tr, 'phylo4')
d = data.frame(color=sample(c('red', 'blue', 'green'), 30, replace=T))
rownames(d) = tr$tip.label
g2 = phylo4d(g1, d)

rNodeData &lt;- data.frame(randomTrait = rnorm(nNodes(g1)),
                        color = sample(c('purple', 'yellow', 'black'), nNodes(g1), replace=T),
                        row.names = nodeId(g1, &quot;internal&quot;))

nodeData(g2) &lt;- rNodeData
</code></pre>
<p>We can store the information in a <code>phylo4d</code> object and as this object is supported by <code>ggtree</code>. We can directly coloring the tree by mapping the color to specific variable.</p>
<pre><code>ggtree(g2, aes(color=I(color)))
</code></pre>
<p></p>
<p><code>ggtree</code> itself also provide solution by using the operator, <code>%&lt;+%</code> to attach user data. After attaching, we can re-scale edge color by <code>aes</code>.</p>
<pre><code>d = data.frame(node=1:59, color=sample(c('red', 'blue', 'green'), 59, replace=T))
ggtree(tr) %&lt;+% d + aes(color=I(color))
</code></pre>
<p></p>
Luca Martino pointed out to me my own and forgotten review of a 2012 paper of his, “On the Generalized Ratio of Uniforms as a Combination of Transformed Rejection and Extended Inverse of Density Sampling” that obviously discusses a generalised version of Kinderman and Monahan’s (1977) ratio-of-uniform method. And further points out the earlier 1991 paper […]
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”. In order to be able to solve this set of exercises you should have solved the part 0, part 1, part 2,part 3, and part 4 of this series but also you should run this script which […]
I’ve been trying to find a way to introduce threads to R. I guess there can be many reasons to do that, among which I could mention simplified input/output logic, sending tasks to the background (e.g. building a model asynchronously), running computation-intensive tasks in parallel (e.g. parallel, chunk-wise var() on a large vector). Finally, it’s … <a href="https://random-remarks.net/2016/12/11/a-very-experimental-threading-in-r/">More <span>(A Very) Experimental Threading in R</span></a>
Introduction A few days ago Microsoft announced their new Microsoft R Server 9.0 version. Among a lot of new things, it includes some new and improved machine learning algorithms in their MicrosoftML package. Fast linear learner, with support for L1 … <a href="https://longhowlam.wordpress.com/2016/12/10/dont-give-up-on-single-trees-yet-an-interactive-tree-with-microsoft-r/">Continue reading <span>→</span></a>
Statisticians and data scientists want a neat world where data is arranged in a table such that every row is an observation or instance, and every column is a variable or measurement. Getting to this state of “ready to model format” (often called a denormalized form by relational algebra types) often requires quite a bit … <a href="http://www.win-vector.com/blog/2016/12/the-case-for-index-free-data-manipulation/">Continue reading <span>The case for index-free data manipulation</span></a>
When faced with classification tasks in the real world, it can be challenging to deal with an outcome where one class heavily outweighs the other (a.k.a., imbalanced classes). The following will be a two-part post on some of the techniques that can hel...
I had wanted to post this before the US Election, but time constraints didn't allow.  With the potential for new Supreme Court Justices in the next four years, many voters and namely single-issue voters rallied behind Donald Trump for his seeming support for a conservative justice.  Most of the people I spoke with were primarily concerned with the potential appointment of a justice who could help in concentrating efforts in overturning abortion.  For a more in-depth look on Trump, his stances and commentary on overturning abortion I found <a target="_blank" href="http://fivethirtyeight.com/features/trump-probably-cant-get-roe-v-wade-overturned/">this article</a> to be helpful. <br><br>I was curious in the past how the political leanings of the Justices has changed over time and if opportunities like a conservative bench with a republican president have occurred in the past.  I was curious because I wondered if an appointment during the Trump administration would change anything. <br><br>The graph below shows the amount of abortions over time during different presidents and each point shows the political split of the bench (Democrat-Republican).<br><br><div><a href="https://2.bp.blogspot.com/--ZBGCj_ZPIg/WEvkGIqGpbI/AAAAAAAACqo/QZbdRX-LVjcImJnXLQWoyaPG3QMs9pzzgCLcB/s1600/abortions.png"></a></div><br>Political leanings on a bench aren't indicative of a pro-life vote.  However, learning more conservative politically does provide potential for a favorable pro-life vote.  I found this information interesting since the split on the court has predominantly been conservative for the last 40 years and only recently has it become more liberal.  The graph speaks for itself and the code for it is <a target="_blank" href="http://justlebeau.github.io/abortiongraphcode.html">available here</a>.
Using the knowledge you acquired in the previous exercises on sampling and selecting(here), we will now go through an entire data analysis process. You will be using what you know as crutches to solve the problems. Don’t worry. It might look intimidating but follow the sequence and you will see that modeling a decision tree […]
[…]
Since I am already knocked out of the playoffs (our league playoffs start in week 12), I thought it would be interesting to see which players ended up being the best and worst values of this years draft. It’s only week 13 so these numbers will change by the end of the season and I <a rel="nofollow" href="http://jessepiburn.com/2016/12/fantasy-value-so-far-this-year-adp-vs-vor/"><span>Read more about Fantasy Value So Far This Year: ADP vs VOR</span>[…]</a>
A user of my Introductory Fisheries Analyses with R book recently asked me how to compute standard errors (SE) for the various PSD indices by using the usual equation for the SE of a proportion (square root of p times (1-p) divided by n). B...
Outliers in data can distort predictions and affect the accuracy, if you don’t detect and handle them appropriately especially in regression models. Why outliers treatment is important? Because, it can drastically bias/change the fit estimates and predictions. Let me illustrate this using the cars dataset. To better understand the implications of outliers better, I am […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toR for Publication: Lesson 6, Part 2 – Linear Mixed Effects Models" rel="bookmark" href="http://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-6-part-2-linear-mixed-effects-models-lmem/">R for Publication: Lesson 6, Part 2 – Linear Mixed Effects Models</a></li><li><a title="Permanent Link toR for Publication: Lesson 6, Part 1 – Linear Mixed Effects Models" rel="bookmark" href="http://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-6-part-1-linear-mixed-effects-models-lmem/">R for Publication: Lesson 6, Part 1 – Linear Mixed Effects Models</a></li><li><a title="Permanent Link toCross-Validation: Estimating Prediction Error" rel="bookmark" href="http://datascienceplus.com/cross-validation-estimating-prediction-error/">Cross-Validation: Estimating Prediction Error</a></li><li><a title="Permanent Link toInteractive Performance Evaluation of Binary Classifiers" rel="bookmark" href="http://datascienceplus.com/interactive-performance-evaluation-of-binary-classifiers/">Interactive Performance Evaluation of Binary Classifiers</a></li><li><a title="Permanent Link toPredicting wine quality using Random Forests" rel="bookmark" href="http://datascienceplus.com/predicting-wine-quality-using-random-forests/">Predicting wine quality using Random Forests</a></li></ol>
I was thrilled to be invited to speak at the Monktoberfest conference, held this past October in Portland, Maine. Not only have I been a great fan of the analysis from the Redmonk team for many years, I'd heard that it was one of the most interesting and diverse tech conferences around. (Also, beer.) And indeed, it turned out to be all of those things, and one of the most memorable and interesting conferences I've ever been to. You can find many of the talks on Redmonk's Youtube channel. There were so many great talks it's hard to single any...
by Sean Lopp Here is this month’s collection of RStudio Tips and Tricks. Thank you to those who responded to last month’s post; many of your tips are included below! Be sure to subscribe to @rstudiotips on Twitter for more. This month’s tips fall into two categories: Keyboard Shortcuts and Easier R Markdown Keyboard Shortcuts […]
At the University of Utah, I teach the R lab that accompanies MATH 3070, “Applied Statistics I.”” None of my students are presumed to have any programming experience, and they never hesitate to remind me of that fact, especially when they are starting out. When I create assignments and pick problems, I often can write…<a href="https://ntguardian.wordpress.com/2016/12/08/where-to-go-from-here-tips-r-experience/">Read more <span>Where to Go from Here? Tips for Building Up R Experience</span></a>
An overview on error analysis through cross-validation, variance, bias, bootstrapping, accuracy and time dependent predictive models.
The R Consortium has already funded 8 projects (and 3 more just in July) proposed by the R community, and the call for proposals for yet more projects is now open. If you have an idea for a projects that would advance R or the R Community, get your submission in by February 10, 2017. Meanwhile, the already-funded projects are making good progress. R-hub, the build service for R packages, has been running a successful public beta for a couple of months now. The SatRDays mini-conferences project has already had one very successful sold-out meeting in Budapest (follow that link...
Below I will expand on previous posts on bayesian regression modelling using STAN (see previous instalments here, here, and here). Topic of the day is modelling crossed and nested design in hierarchical models using STAN in R. Crossed design appear when we have more than one grouping variable and when data are recorded for each … <a href="https://biologyforfun.wordpress.com/2016/12/08/crossed-and-nested-hierarchical-models-with-stan-and-r/">Continue reading <span>Crossed and Nested hierarchical models with STAN and R</span></a>
The team here at DataCamp is thrilled to announce that we now offer free Italian (thanks to Quantide) and German (thanks to eoda) translations of our most popular course, Introduction to R. Best of all, the courses are free as a part of our open cours...
[For this exercise, first write down your answer, without using R. Then, check your answer using R.] Answers to the exercises are available here. Exercise 1 If M=matrix(c(1:10),nrow=5,ncol=2, dimnames=list(c('a','b','c','d','e'),c('A','B'))) What is the value of: M Exercise 2 Consider the matrix M, What is the value of: M[1,] M[,1] M[3,2] M['e','A'] Exercise 3 Consider the matrix […]
A new version of RcppAPT -- our interface from R to the C++ library behind the awesome apt, apt-get, apt-cache, ... commands and their cache powering Debian, Ubuntu and the like -- is now on CRAN.
We changed the package to require C++11 compilation a...
Microsoft R Server 9.0, Microsoft's R distribution with added big-data, in-database, and integration capabilities, was released today and is now available for download to MSDN subscribers. This latest release is built on Microsoft R Open 3.3.2, and adds new machine-learning capabilities, new ways to integrate R into applications, and additional big-data support for Spark 2.0. This release includes a brand new R package for machine learning: MicrosoftML. This package provides state-of-the-art, fast and scalable machine learning algorithms for common data science tasks including featurization, classification and regression. Some of the functions provided include: Fast linear and logistic model functions based...
How to write Structured Query Language (SQL) code in R. Well there are many packages on CRAN that relate to databases. In the exercises below we cover some of the important data manipulation operations using SQL in R. We will use a ‘sqldf’ package, an R package for running SQL statements on data frames. Answers […]
Yesterday OECD has published results and data from PISA 2015 study (Programme for International Student Assessment). It’s a very cool study – over 500 000 pupils (15-years old) are examined every 3 years. Raw data is publicly available and one can easily access detailed information about pupil’s academic performance and detailed data from surveys for … <a href="http://smarterpoland.pl/index.php/2016/12/pisa-2015-how-to-readprocessplot-the-data-with-r/">Czytaj dalej <span>PISA 2015 – how to read/process/plot the data with R</span></a>
An old riddle found on X validated asking for Monte Carlo resolution  but originally given on Project Euler: A 30×30 grid of squares contains 30² fleas, initially one flea per square. When a bell is rung, each flea jumps to an adjacent square at random. What is the expected number of unoccupied squares after 50 […]
This document describes a proof-of-concept for producing R demonstration videos in a fully-automated manner. The “script” for the video consists of a text file containing code chunks paired with text commentary. The video is produced by running the code while … <a href="http://stattech.wordpress.fos.auckland.ac.nz/2016-15-automating-r-demonstration-videos/">Continue reading <span>→</span></a>
Plusses and Arrows and Percents, oh my!. - Do you continually substitute “%>%” for “+” when switching between data wrangling and data visualization? I’ve got just the solution for you!
Count myself as one of those ...
Many data analysts in Riyadh (or as much as I have met) still valiantly use Excel and/or SQL to perform analyses.
Some engage in SQL gymnastics for the simple operation of pivoting, whilst others wait for purchase of SAS licences, affirming that SAS ...
by Joseph Rickert November was a prolific month for R developers: 189 new packages landed in CRAN. I have selected more than a quarter of them for this post, but I haven’t listed everything that is worth a look. My November 2016 picks are organized into four categories: Biotech (4 picks), Data (6 picks), Machine […]
Last year I made a blog post about a Secret Santa picker HERE, but to use it required quite a bit of messing around with the code. So this year I decided to improve the whole thing by making it a function rather than a script. The function take two inputs, a list of names […]
Imagine that in the course of your analysis, you regularly require summaries of numerical values. For some applications you want the mean of that quantity, plus/minus a standard deviation; for other applications you want the median, and perhaps an interval around the median based on the interquartile range (IQR). In either case, you may want … <a href="http://www.win-vector.com/blog/2016/12/using-replyrlet-to-parameterize-dplyr-expressions/">Continue reading <span>Using replyr::let to Parameterize dplyr Expressions</span></a>
Five different sources of error
When it comes to time series forecasts from a statistical model we have five sources of error:
Random individual errors
Random estimates of parameters (eg the coefficients for each autoregressive term)
Uncertain...
It’s been about a month since the U.S. presidential election, with Donald Trump’s victory over Hillary Clinton coming as a surprise to most. Reddit user Jason Baumgartner collected and published every submission and comment posted to Reddit on the ...
The last riddle from the Riddler [last before The Election] summed up to find the probability of a Binomial B(2N,½) draw ending up at the very middle, N. Which is If one uses the standard Stirling approximation to the factorial function, log(N!)≈Nlog(N) – N + ½log(2πN) the approximation to ℘ is 1/√πN, which is not […]
Hey R fans! We've just released a new episode of DataCamp's DataChats video series. 
In this third episode, we interview Charlotte Wickham. Charlotte is an Assistant Professor in the Department of Statistics at Oregon State University and an avid R...
In this exercise we cover the basics on selecting and extracting data using queries. We add a few other materials into it. This should prepare you for the next exercise: Basic Decision Tree. The purpose of this is to give you the 20 percent of the tools to get 80 percent of work done. You […]
by Hong Ooi, Sr. Data Scientist, Microsoft Version 0.90 of the dplyrXdf package has just been released. dplyrXdf is a package that brings dplyr pipelines and data transformation verbs to Microsoft R Server’s xdf files. This version includes several changes, mostly to address performance and efficiency concerns, which I’ll detail these below. The .outFile argument All dplyrXdf verbs now support a special argument .outFile, which determines how the output data is handled. If you don’t specify a value for this argument, the data will be saved to a tbl_xdf which will be managed by dplyrXdf. This supports the default behaviour,...
The ASA (American Statistical Association) has been working in collaboration with the ACM (Association for Computing Machinery) on developing a data science curriculum for Two Year Colleges. Part of this development is the need to understand the private-sector demand for two-year college data science graduates and the prevalence of the need to invest in the... <a href="https://rud.is/b/2016/12/06/survey-on-data-science-in-two-year-colleges/">Continue reading <span>→</span></a>
In case you missed them, here are some articles from November of particular interest to R users. Microsoft R Open 3.3.2, based on R 3.3.2, has been released for Windows, Mac and Linux. A new, free course on EdX focuses on the big-data extensions of Microsoft R Server. Using ggplot2 to create a calendar heat map of city bike usage in Chicago. A tutorial on creating an in-database rental prediction service for a ski shop with SQL Server R Services. Detailed R code to calculate AUC, the area under a ROC curve. Rankings of the 5 most popular R packages...
This is a short post for those looking to test out Amazon Athena with R. Amazon makes Athena available via JDBC, so you can use RJDBC to query data. All you need is their JAR file and some setup information. Here’s how to get the JAR file to the current working directory: To avoid putting... <a href="https://rud.is/b/2016/12/05/interacting-with-amazon-athena-from-r/">Continue reading <span>→</span></a>
[For this exercise, first write down your answer, without using R. Then, check your answer using R.] Answers to the exercises are available here. Exercise 1 Create a function that given a data frame and a vector, will add a the vector (if the vector length match with the rows number of the data frame) […]
The R Community is powered by open course contributions from a wide variety of individuals. Anyone can create and distribute R packages that benefit the R community and make it easier for R users all over the world to work with their code. Some R use...
I’ve added a couple of new functions to the forecast package for R which implement two types of cross-validation for time series. K-fold cross-validation for autoregression The first is regular k-fold cross-validation for autoregressive models. Although cross-validation is sometimes not valid for time series models, it does work for autoregressions, which includes many machine learning […]
After a successful EARL London in September of this year (see here and here) it was Boston’s turn to shine. This is only the second time this conference has run, yet its quality of speakers is at par with London. … <a href="http://www.mango-solutions.com/wp/2016/12/earl-boston-revisited/">Continue reading <span>→</span></a>
As you know, the composition of an Index changes over time and this scipt takes the Yahoo Finance page and retrieves all individual stock symbols for you and stores it in the tickers object.<br><br>I had to update my R-Version to use the RCurl package - this is required as it is a https:// connection we would like to use.<br><br>I was surprised how easy it is to parse the xml page and retrieve all the tables on the page.<br><br><a name="more"></a><br><br>In this little example we are interested in retrieving all symbols of the DAX (30 largest listed companies in Germany) from this link https://de.finance.yahoo.com/q/cp?s=^GDAXI<br><br>It is the table 5 which holds the relevant data for us  - it is completely transparent to the user and I enjoyed using the XML - Package. <br><br><span>library(XML)</span><br><span>library(RCurl)</span><br><span><br></span><span>universum &lt;-function()</span><br><span>{</span><br><span>  world=c(&quot;^GDAXI&quot;,&quot;^MDAXI&quot;,&quot;^SDAXI&quot;,&quot;^TECDAX&quot;)</span><br><span>  my=NA</span><br><span>  for(i in 1:NROW(world)) {</span><br><span>    url=paste(&quot;https://de.finance.yahoo.com/q/cp?s=&quot;,world[[i]],sep = &quot;&quot;)</span><br><span>    s &lt;- getURL(url)  #rcurl package</span><br><span>    t=readHTMLTable(s)</span><br><span>    t=t[[5]]</span><br><span>    t = t[-(1:5), ]</span><br><span>    newtickers=t[,1]</span><br><span>    a=NROW(my)</span><br><span>    b=NROW(newtickers)</span><br><span>    my[a+1:b]=as.character(newtickers)</span><br><span>    </span><br><span>  }</span><br><span>  my = my[-1]</span><br><span>  tickers=my</span><br><span>  rm(t)</span><br><span>  rm(a)</span><br><span>  rm(b)</span><br><span>  rm(i)</span><br><span>  rm(my)</span><br><span>  rm(newtickers)</span><br><span>  return(tickers)</span><br><span>}</span>
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”. In order to be able to solve this set of exercises you should have solved the part 0, part 1, part 2,part 3, and part 4 of this series but also you should run this script which […]
A look into Denver crime data surrounding marijuana, alcohol, and other drugs using R programming.
Imagine you bought solar PVC arrays today, sold the power generated to your local utility, and used the proceeds to add new solar arrays each year.  What generating capacity would you expect in 100 years?  What would be the impact of uncertainties in inflation and power generation?  I've been obsessed with this idea for a while.  I developed a new Shiny App <a target="_blank" href="https://johnyagecic.shinyapps.io/SolarEnergyMC/">available here</a> to help play around with these questions.<br><br><div><a href="https://2.bp.blogspot.com/-Xwt3k8QUZbs/WERvswamI4I/AAAAAAAAA30/bNg4HH9lp1w0u8OBwueyZfO1jCUP21-igCLcB/s1600/SolarMC.PNG"></a></div><br>There is a link in the app to obtain the R script.  Since this is not my area, I am hoping that people with domain specific knowledge will tinker and improve.<br><br><u>Opportunities for Improvement</u>  There are several areas where I think some improvement is possible.  The app assumes a uniform distribution (defined by the user) of annual power generation over the 100 years.  Users can estimate localized power generation ranges using the <a target="_blank" href="http://pvwatts.nrel.gov/pvwatts.php">PVWatts Calculator</a>.  However,  as the <a target="_blank" href="http://www.nrel.gov/pv/assets/images/efficiency_chart.jpg">NREL PV efficiency chart</a> shows, higher efficiency systems are in the pipeline.  Some sort of forecast trajectory seems like a possible improvement.  Similar trajectories could be applied to inflation rates too.  The app only uses 100 Monte Carlo iterations, which is a very small number.  But the app also uses looping, and loops are slow.  I ignored cost associated with real estate.  My assumption here is that <a target="_blank" href="https://www.epa.gov/brownfields">brownfield sites</a> close to the grid would be available and suitable, but explicit real estate costs could be included.
This post is in reply to a request: How did I set up this R blog?
I have wanted to have my own R blog for a while before I actually went ahead and realised this page. I had seen all the cool things people do with R by following R-bloggers and reading ...
When writing reusable code or packages you often do not know the names of the columns or variables you need to work over. This is what I call “parametric treatment of variables.” This can be a problem when using R libraries that assume you know the variable names. The R data manipulation library dplyr currently … <a href="http://www.win-vector.com/blog/2016/12/parametric-variable-names-and-dplyr/">Continue reading <span>Parametric variable names and dplyr</span></a>
When blogging about statistics and R, it is very useful to differentiate the body text to R code. I used to manage this issue by highlighting the code and pretty-R was a valuable instrument from Revolutions Analytics to accomplish this. However, as you...
When modeling any phenomena by including explanatory variables that highly relates the variable of interest, one question arises: which of the auxiliary variables have a higher influence on the response? I am not writing about significance testing or s...
A few months ago we launched R course finder, an online directory that helps you to find the right R course quickly. With so many R courses available online, we thought it was a good idea to offer a tool that helps people to compare these courses, before they decide where to spend their valuable […]
One thing I teach is: when evaluating the performance of regression models you should not use correlation as your score. This is because correlation tells you if a re-scaling of your result is useful, but you want to know if the result in your hand is in fact useful. For example: the Mars Climate Orbiter … <a href="http://www.win-vector.com/blog/2016/12/be-careful-evaluating-model-predictions/">Continue reading <span>Be careful evaluating model predictions</span></a>
Few people expect politicians to write every word they utter themselves; reliance on speechwriters and spokepersons is a long-established political practice. Still, it's interesting to know which statements are truly the politician's own words, and which are driven primarily by advisors or influencers. Recently, David Robinson established a way of figuring out which tweets from Donald Trump's Twitter account came from him personally, as opposed to from campaign staff, whcih he verified by comparing the sentiment of tweets from Android vs iPhone devices. Now, Ali Arsalan Kazmi has used stylometric analysis to investigate the provenance of speeches by the Prime...
I'm happy to introduce an add-on package, pdfsearch, that adds the ability to do keyword searches on pdf files. This add-on package uses the excellent pdftools package from the ropensci project to read in pdf files and perform keyword searches based ch...
<a href="https://1.bp.blogspot.com/-wwPe_7mlg4E/WEHTYduGGTI/AAAAAAAAA2M/SvqA6colfHQBrUKYppG5Cyy2mUZWVVNJACLcB/s1600/UnderstandingSQLandR.png"></a><br><br>Just had a video series released by O'Reilly Media on infiniteskills.com:<br><br><a href="https://www.infiniteskills.com/training/understanding-sql-and-r.html">https://www.infiniteskills.com/training/understanding-sql-and-r.html</a><br><br>This course shows you how to combine two of the most popular technologies used in data analytics: SQL and R.  Complete this course and you'll learn the basics of a skill set highly valued by employers around the globe.<br><br><ul><li>Gain experience combining SQL and R for data analytics and visualization purposes</li><li>Learn to write SQL queries that can retrieve and summarize data stored in databases</li><li>Explore the statistical analysis and visualization capabilities of R</li><li>See how R uses SQL to interact with databases</li><li>Discover how to create graphics using R, based on data in databases</li><li>Learn to use features of RStudio to streamline the analysis process</li></ul><br>Please consider picking up a copy.  You might even be able to obtain educational reimbursement from an employer.  Thanks!
We are committed to bringing you 100% authentic exercise sets. We even try to include as different datasets as possible to give you an understanding of different problems. No more classifying Titanic dataset. R has tons of datasets in its library. This is to encourage you to try as many datasets as possible. We will […]
We have released the R package bookdown (v0.3) to CRAN. It may be old news to some users, but we are happy to make an official announcement today. To install the package from CRAN, you can install.packages(&quot;bookdown&quot;) The bookdown package provides an easier way to write books and technical publications than traditional tools such as LaTeX and Word. It inherits the […]
by Joseph Rickert If you are aspiring to write good R code, you may find it helpful to occasionally spend some time reading about writing: reading about writing R code, and reading about writing about R code that you’ve written. (If you write some excellent R code, you will likely have the opportunity to write […]
So much time since my last post so I want to post something, no matter what it is, but I hope this will be somehow helpfull
In this post I will show some new features for the next version of
highcharter package. The main feature added is hc_ad...
Yesterday, the new version of D3partitionR was uploaded on CRAN, this version contains some major improvements. 1. General layout: Before this update, the height and width of the widget were somehow messy and so was the global layout. The layout is now the following: Area A: Title area (exists only if the user entered a […]<img height="1" alt="" width="1" src="https://pixel.wp.com/b.gif?host=antoineguillot.wordpress.com&blog=113378294&post=806&subd=antoineguillot&ref=&feed=1" border="0">
In last week’s post I explored whether machine learning models can be applied to predict flu deaths from the 2013 outbreak of influenza A H7N9 in China. There, I compared random forests, elastic-net regularized generalized linear models, k-nearest ne...
[For this exercise, first write down your answer, without using R. Then, check your answer using R.] Answers to the exercises are available here. Exercise 1 Using the following variables: x=1 i=c(1:10) For this exercise, write a for() loop that increments x by two for each i. Exercise 2 Using the following variables: x=1 y=40 […]
by Wee Hyong Tok, Senior Data Scientist Manager at Microsoft How do you read a novel in record time, and gain insights into the emotional journey of main characters, as they go through various trials and tribulations, as an exciting story unfolds from chapter to chapter? I remembered my experiences when I start reading a novel, and I get intrigued by the story, and simply cannot wait to get to the last chapter. I also recall many conversations with friends on some of the interesting novels that I have read awhile back, and somehow have only vague recollection of what...
I recently saw a tweet floating by which included a link to some recent statistics from PubMed Commons, the NCBI service for commenting on scientific articles in PubMed. Perhaps it was this post at their blog. So I thought now would be a good time to write some code to analyse PubMed Commons data. The […]
About two years ago we published a quick and easy guide to setting up your own RStudio server in the cloud using the Docker service and Digital Ocean. The process is incredibly easy-- about the only cumbersome part is retyping a random password. Toda...
Recently I wanted to extract a table from a pdf file so that I could work with the table in R. Specifically, I wanted to get data on layoffs in California from the California Employment Development Department. The EDD publishes a list of all of the layoffs in the state that fall under the WARN […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toExtract Twitter Data Automatically using Scheduler R package" rel="bookmark" href="http://datascienceplus.com/extract-twitter-data-automatically-using-scheduler-r-package/">Extract Twitter Data Automatically using Scheduler R package</a></li><li><a title="Permanent Link toAn Introduction to Time Series with JSON Data" rel="bookmark" href="http://datascienceplus.com/introduction-to-time-series-with-json-data/">An Introduction to Time Series with JSON Data</a></li><li><a title="Permanent Link toGet Your Data into R: Import Data from SPSS, Stata, SAS, CSV or TXT" rel="bookmark" href="http://datascienceplus.com/get-your-data-into-r/">Get Your Data into R: Import Data from SPSS, Stata, SAS, CSV or TXT</a></li><li><a title="Permanent Link toExporting Data from R to TXT, CSV, SPSS or Stata" rel="bookmark" href="http://datascienceplus.com/exporting-data-from-r/">Exporting Data from R to TXT, CSV, SPSS or Stata</a></li><li><a title="Permanent Link toFirst Things to Do After You Import the Data into R" rel="bookmark" href="http://datascienceplus.com/first-things-to-do-after-you-import-the-data-into-r/">First Things to Do After You Import the Data into R</a></li></ol>
Possibly the last post on random number generation by Kinderman and Monahan’s (1977) ratio-of-uniform method. After fiddling with the Gamma(a,1) distribution when a&lt;1 for a while, I indeed figured out a way to produce a bounded set with this method: considering an arbitrary cdf Φ with corresponding pdf φ, the uniform distribution on the set […]
We just launched Joining Data in R with dplyr taught by Garrett Grolemund, the author of Hands-On Programming with R and R for Data Science from O'Reilly Media. This course builds on what you learned in Data Manipulation in R with dplyr by showi...
How wary we are of something warm and unborn. Something calmly by zero will divide (Unbegotten, The Somnambulist) Some time ago, I assumed the mission to draw a plot for the cover of the new album of The Somnambulist, a music band from Berlin. They wanted a circlization of Pi, which is a graphic where … <a href="https://fronkonstin.com/2016/12/01/the-somnambulist-and-pi/">Continue reading <span>The Somnambulist and Pi</span> <span>→</span></a>
a quick follow up to my last post - I’ve had a few folk get in touch following my last post, all commenting on the last plot :
Most of these enquiries went along the following lines:
“I hadn’t thought of doing that. It looks really ...
Christmas can be a stressful and expensive time of year, with many gifts to buy for many people. My family have tried to eliminate some of this stress by having a Secret Santa gifting system where we all buy one … <a href="https://amywhiteheadresearch.wordpress.com/2016/12/01/santas-little-helper/">Continue reading <span>→</span></a>
Get 50% off DataCamp to Celebrate 2017 (only 1 day left) DataCamp is now offering R-bloggers readers a 50% discount on unlimited access to their course curriculum. They have over 40 exceptional courses (and counting) taught by expert instructors like Hadley Wickham (RStudio), Matt Dowle (data.table), Garrett Grolemund (RStudio), and Max Kuhn (caret). Check out this link to get the discount! Here are some highlights from their course list: R Programming Introduction to R Intermediate R Writing Functions in R (Hadley &amp; Charlotte Wickham) Importing and Cleaning Importing Data in R (part 1) Importing Data in R (part 2) Cleaning Data in R Data Manipulation Data Manipulation with dplyr (Garrett Grolemund) Joining Data in R with dplyr (Garrett Grolemund) Data Analysis the data.table way (Matt Dowle &amp; Arun Srinivasan) Manipulating Time Series Data in R with xts and zoo (Jeffrey Ryan) Data Visualization Data Visualization with ggplot2 (Rick Scavetta) Data Visualization with ggvis (Garrett Grolemund) Machine Learning &amp; Statistics Introduction to Machine Learning (DataCamp) Machine Learning Toolbox (Max Kuhn &amp; Zachary Dean-Mayer) Kaggle: R tutorial on Machine Learning (Kaggle) A Hands-On Introduction to Statistics with R (Andrew Conway, Princeton) And much more – but don’t wait, this offer ends January […]
by Joseph Rickert One great advantages of working in R is the quantity and sophistication of the statistical functions and techniques available. For example, R’s quantile() function allows you to select one of the nine different methods for computing quantiles. Who would have thought there could be so many ways to do something that seems […]
In this exercise, we cover some basics on frequency tables. We also briefly look at chi-square test for independence to find relationships of two variables. Before proceeding, it might be helpful to look over the help pages for the table, summary, and margin.table functions. Answers to the exercises are available here. If you obtained a […]
[Following my post of lastTuesday, Matt Graham commented on the paper with force détails. Here are those comments. A nicer HTML version of the Markdown reply below is also available on Github.] Thanks for the comments on the paper! A few additional replies to augment what Amos wrote: This however sounds somewhat intense in that […]
When working on data science problems, you might want to set up an interactive environment to work and share your code for a project with others. You can easily set this up with a notebook. 
In other cases, you’ll just want to communicate about t...
Nina Zumel and I are happy to announce a formal article discussing data preparation and cleaning using the vtreat methodology is now available from arXiv.org as citation arXiv:1611.09477 [stat.AP]. vtreat is an R data.frame processor/conditioner that prepares real-world data for predictive modeling in a statistically sound manner. It prepares variables so that data has fewer … <a href="http://www.win-vector.com/blog/2016/11/vtreat-data-cleaning-and-preparation-article-now-available-on-arxiv/">Continue reading <span>vtreat data cleaning and preparation article now available on arXiv</span></a>
The following R-script is for download the data automatically from twitter using the package SchedulerR. As first step we download the data using the OAuth protocol and store the data using as a name the date from the download. As second we used the package SchedulerR for to specify how long the script runs. This […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toAn Introduction to Time Series with JSON Data" rel="bookmark" href="http://datascienceplus.com/introduction-to-time-series-with-json-data/">An Introduction to Time Series with JSON Data</a></li><li><a title="Permanent Link toGet Your Data into R: Import Data from SPSS, Stata, SAS, CSV or TXT" rel="bookmark" href="http://datascienceplus.com/get-your-data-into-r/">Get Your Data into R: Import Data from SPSS, Stata, SAS, CSV or TXT</a></li><li><a title="Permanent Link toExporting Data from R to TXT, CSV, SPSS or Stata" rel="bookmark" href="http://datascienceplus.com/exporting-data-from-r/">Exporting Data from R to TXT, CSV, SPSS or Stata</a></li><li><a title="Permanent Link toFirst Things to Do After You Import the Data into R" rel="bookmark" href="http://datascienceplus.com/first-things-to-do-after-you-import-the-data-into-r/">First Things to Do After You Import the Data into R</a></li></ol>
<p>The second satRday (and first satRday on African soil) will happen in Cape Town on 18 August 2017. It’s going to be a one day celebration of R. We have a trio of phenomenal keynote speakers (Hilary Parker, Jenny Bryan and Julia Silge) who will be giving inspiring talks at the conference and also conducting …</p>
<p>The post <a rel="nofollow" href="http://www.exegetic.biz/blog/2016/11/satrday-in-cape-town/">satRday in Cape Town</a> appeared first on <a rel="nofollow" href="http://www.exegetic.biz/blog">Exegetic Analytics</a>.</p>
<p>When working with ggplot2 you don't have to care about legend, it is automatically produced according to the mapped aesthetics. But producing separate legends for the same aesthetic is not easy. This article proposes a solution!</p>
<p>The post <a rel="nofollow" href="http://www.quantide.com/ggplot-multiple-legends-for-the-same-aesthetic/">ggplot2: multiple legends for the same aesthetic</a> appeared first on <a rel="nofollow" href="http://www.quantide.com/">Quantide - R training &amp; consulting</a>.</p>
A few months ago a reader point me out this new way of connecting R and Excel. I don’t know for how long this has been around, but I never came across it and I’ve never seen any blog post or article about it. So I decided to write a post as the tool is really […]
<a href="https://4.bp.blogspot.com/-2D0KYYAYubc/WD7SN8qLfQI/AAAAAAAAA1k/ZEzb5QrW6Qc6Im_PfWPe-E4yJkBTguGuwCLcB/s1600/C_Major.png"></a><br>The initial release of ggguitar is available on CRAN!  This package allows you to create guitar tablature in the style of ggplot2.  As a quick example, the C Major chord shown above can be created as follows.<br><br><span>library(ggguitar)</span><br><span>C_M -></span><br><span>tablature('C Major', C_M)</span><br><br>The tablature function takes the name of the chord for the first argument and a six element vector representing the six strings of the guitar.  An NA indicates a string that is not played, zero represents an open string, and a number indicates a fret where a finger is to be placed on the specified string.<br><br>The &quot;String&quot; and &quot;Fret&quot; labels and corresponding axis ticks are optional.<br><br><span>tablature('C Major', C_M, FALSE, FALSE)</span><br><br><div><a href="https://2.bp.blogspot.com/-4piKwpiH3d4/WD7TSYuFSfI/AAAAAAAAA1s/k97O4ZndRJUxInlCcI40lmGeKdFw0-dZgCLcB/s1600/C_Major_plain.png"></a></div><br>See the <a href="https://cran.r-project.org/web/packages/ggguitar/index.html">package at CRAN</a>, the <a href="https://cran.r-project.org/web/packages/ggguitar/vignettes/my-vignette.html">vignette</a>, or the <a href="https://github.com/ezgraphs/ggguitar">code at Github</a> for more information.<br><br><br><br><br><br>
Microsoft R Open 3.3.2, Microsoft's enhanced distribution of open source R, is now available for download for Windows, Mac, and Linux. This update upgrades the R language engine to version 3.3.2, adds new bundled packages and updates others, and upgrades the Intel Math Kernel Libraries. The updated R 3.3.2 engine includes some performance improvements (particularly in calculation of eigenvalues), better handling of date axes in graphics, and improved documentation for the methods package. (See here for a complete list of fixes.) There are no user-visible changes in the language itself, which means that scripts and packages should work without changes...
About the book
This is a technical book. The scope of the book is to go straight to the point and the writing style is similar to a recipe with detailed instructions. It is assumed that you know the basics of R and that you want to learn how to c...
Themes are an convenient way to give ggplot charts an individualized, sometimes stylish look. Most of the time, I rely on the ggthemes package and the Economist style. Last week colleagues asked me to change the look of my charts. We joked around and I...
If you're already familiar with R, but struggling with out-of-memory or performance problems when attempting to analyze large data sets, you might want to check out this new EdX course, Analyzing Big Data with Microsoft R Server, presented by my colleague Seth Mottaghinejad. In the course, you'll learn how to build models using the RevoScaleR package, and deploy those models to production environments like Spark and SQL Server. The course is self-paced with videos, tutorials and tests, and is free to audit. (By the way, if you don't already know R, you might want to check out the courses Introduction...
Hey R fans! We've just released a new episode of DataCamp's DataChats video series. Our first episode with Max Kuhn has already been watched over 700 times and this new episode promises to be a blockbuster as well. 
In this second episode we inte...
One way, two way and n way ANOVA are used to test difference in means when we have one, two and n factor variables. A key assumption when performing these ANOVAs is that the measurements are independent. When we have repeated measures this assumption is violated, so we have to use repeated measures ANOVA. Repeated […]
Here’s a new package that brings to R new API to handle child processes – similar to how Python handles them. Unlike the already available system() and system2() calls from the base package or the mclapply() function from the parallel package, this new API is aimed at handling long-lived child processes that can be controlled by the parent R process in a … <a href="https://random-remarks.net/2016/11/29/a-new-subprocess-package-for-r/">More <span>A new `subprocess` package for R</span></a><img height="1" alt="" width="1" src="https://pixel.wp.com/b.gif?host=random-remarks.net&blog=72810971&post=190&subd=randomremarksblog&ref=&feed=1" border="0">
elasticsearchr: a Lightweight Elasticsearch Client for R Elasticsearch is a distributed NoSQL document store search-engine and column-oriented database, whose fast (near real-time) reads and powerful aggregation engine make it an excellent choice as an ‘analytics database’ for R&amp;D, production-use or … <a href="https://alexioannides.com/2016/11/28/elasticsearchr-a-lightweight-elasticsearch-client-for-r/">Continue reading <span>→</span></a>
Chicago's a great city for a bike-sharing service. It's pretty flat, and there are lots of wide roads with cycle lanes. I love Divvy and use it all the time. Not so much in the winter though: it gets very cold here. Nonetheless, this heat map of Divvy riders, created in R by Austin Wehrwein, reveals a hardcore set of riders that use the service even in the bitter cold. For example on January 27 2014 when the temperature was °F (-18°C), there were still 406 riders that day. There's no calendar heatmap extension to ggplot2 (that I know of),...
I've joined Hadley's team at RStudio. Unsurprisingly, I’ll be working on some modeling related R packages ...
Make a simulation and function using R. I continue to teach some R programming in my AP Stats class because this is an essential skill for them to have. We are now in chapter five of our textbook, Probability and Simulation. I had my students review the parking problem on page 290. The problem asked […]
Modeling the frequency is one of the most important aspects in operational risk models. In the previous post (https://statcompute.wordpress.com/2016/05/13/more-flexible-approaches-to-model-frequency), the importance of flexible modeling approaches for both under-dispersion and over-dispersion has been discussed. In addition to the quasi-poisson regression, three flexible frequency modeling techniques, including generalized poisson, double poisson, and Conway-Maxwell poisson, with their implementations […]
[In the exercises below we cover the basics of data frames.] Answers to the exercises are available here. Exercise 1 Consider two vectors: x=seq(1,43,along.with=Id) y=seq(-20,0,along.with=Id) Create a data.frame df: >df Id Letter x y 1 1 a 1.000000 -20.000000 2 1 b 4.818182 -18.181818 3 1 c 8.636364 -16.363636 4 2 a 12.454545 -14.545455 5 […]
After rewatching the thanksgiving classic, Bill and Ted’s Excellent Adventure, it reminded me of the history of #Rstats and its current status as the defacto software for general data programming. The most excellent thing about R, is the literate programming options you have. As a data analyst, you are Bill S. Preston Esquire (or Ted […]
In this set of exercises we will practice functions that enable us to manipulate strings.You can find more about string manipulation functions in Handling and Processing Strings in R e-book. Answers to the exercises are available here. If you have different solution, feel free to post it. Exercise 1 Load text from the file and […]
In my role I have to produce a lot of run charts and control charts for monthly data, but sometimes there are processes behind the run charts that merit looking at the data on a daily basis.
It can become impractical to start producing run c...
Among the many R packages, there is the outbreaks package. It contains datasets on epidemics, on of which is from the 2013 outbreak of influenza A H7N9 in China, as analysed by Kucharski et al. (2014):
A. Kucharski, H. Mills, A. Pinsent, C. Fraser,...
A broad family of fast and effective forecast methods
Exponential smoothing state space methods constitute a broad family of approaches to univariate time series forecasting that have been around for many decades and only in the twenty-first century pl...
I recently attempted a Stylistic Analysis of the Pakistani Prime Minister’s speeches, in order to identify if his speeches exhibit different Stylistic patterns. Large differences would indicate possibilities of different writers.
Background to this ...
Maybe the following sounds familiar. You have a large data set with many, many columns of which the most are irrelevant to you. Typically, a dump from a database or the full set extracted from an API. Several times I found myself the better part of an afternoon going back and forth between a view of the data where I tried to figure out which columns to keep, and an R session where I wrote the code for creating the subset of columns. Wouldn’t it be nice to have an app in which you could just click the columns you would like to keep? This seemed a perfect opportunity to get my feet wet with Shiny gadgets, which I still wanted to do since I first heard about it on useR2016.
I spend a considerable time of my working hours with data processing where I often use the %in% R function as x %in% y. Whenever I need the negation of that, I used to write !(x %in% y). Not much of a hassle, but still, wouldn’t it be nicer to have x %notin% y instead? So I decided to code it for my mefa4 package that I maintain primarily to make my data munging time shorter and more efficient. Coding a %special% function was no big deal. But I had to do quite a bit of research and trial-error until I figured out the proper documentation. So here it goes.
The riddle set by The Riddler of last week sums up as follows: Within a population of size N, each individual in the population independently selects another individual. All individuals selected at least once are removed and the process iterates until one or zero individual is left. What is the probability that there is zero […]
Downloading a weather model and using it to predict stuff is all well and good – but what are the sources of error?  Specifically, how accurate is the atmospheric parametrization?   Luckily, NOMADS provides sets of models with slightly different initialization values.  The result is a set of realizations that captures the variability of the model […]
.fusion-fullwidth-1 {
padding-left: px !important;
padding-right: px !important;
}by Joseph Rickert
I am ever-optimistic that on the weekend I will have enough time and bra...
<p>There are a lot of times when it would be great if your computer talked to you.
<a href="https://cran.r-project.org/package=pushoverr">pushoverr</a> allows you to send yourself or your group notifications from R.
Instead of compulsively checking the status of a running job, you can just have R tell you when it’s done, if it failed, or what the results are.</p>
Using R to add data to maps has been pretty straightforward for a few years now. That said, it seems easier than ever to do things like use map APIs (e.g. Google, Open Street Map), overlay quite complex data visualisations (e.g. “heatmap-style” densities) and even generate animations. A couple of key R packages in this […]
It's Thanksgiving day here in the US, so we're taking the rest of the week off to reflect on what we're thankful for. And even if you're not in the US, today is a great day to send thanks to the R Core Group for providing their dedication, time, and expertise to make the R Project what it is today. (Sadly, cowsay doesn't feature a Thanksgiving turkey.) We'll be back next week. For those that are celebrating, have a great holiday. See you on Monday!
How can I write R codes that other people can understand and use? In the exercises below we cover some of the best practices while writing a small piece of R code or a full automated script. These are some of the practices which should be kept in mind while coding, trust me it will […]
R is an open-source software package and rapidly increases its popularity in both industry and academics. Google trend is probably the best tool to show you how popular R is since it allows us to rank the search interest among five major statistical so...
Part 2: Create an interactive web map with geotagged photos
See the other parts in this series of blog posts.
In part 1 we looked at making a simple interactive map using locations from our geotagged photos. Those photos have data associated with the...
Answers to the exercises are available here. Exercise 1 What is the output of: ifelse(sqrt(9)&lt;2,sqrt(9),0) Exercise 2 What is the output of: ifelse(sqrt(100)>9,sqrt(100),0) Exercise 3 What is the output y of: x=12 if(is.numeric(x)) y=x*2 Exercise 4 What is the output x,y of: z=-1 if(z&lt;0){x=abs(z);y=z*3} Exercise 5 What is the output y of: z=6 if(z&lt;0) y=z*3 […]
Want to Master R? There’s no better time or place than Hadley Wickham’s workshop on December 12th and 13th at the Cliftons in Melbourne, VIC, Australia. Register here: https://www.eventbrite.com/e/master-r-developer-workshop-melbourne-tickets-22546200292   (Note: Prices are in $US and VAT is not collected) Discounts are still available for academics (students or faculty) and for 5 or more attendees […]
by Joseph Rickert In my August Package Picks post, I explained that my selection criteria favor packages with vignettes. (I find skimming through a package’s vignettes to be an effective method of “grokking” what a package is all about.) I also questioned why a person would go to all of the trouble to develop a […]
A great way to learn is by doing, so if you've been thinking about how to enable R-based computations within SQL Server, a new tutorial will take you through all the steps of building an intelligent application. In a few simple steps, you'll set up all the necessary software and code to build a live service that predicts demand for a ski rental shop. The tutorial &quot;Create a predictive model in SQL Server R Services&quot; will step you through: Installing SQL Server, Microsoft R Client, and an IDE for R (RTVS or RStudio) on your Windows machine. All of the...
<p>The histogram is a very useful visualization tool, and you need to master it. </p>
<p>The post <a rel="nofollow" href="http://sharpsightlabs.com/blog/annotated-histogram/">A simple histogram (and why you need to practice it)</a> appeared first on <a rel="nofollow" href="http://sharpsightlabs.com/">SHARP SIGHT LABS</a>.</p>
At DataCamp, we’re big fans of learning via expert interviews, fireside chats with key community contributors and all good data science talks in general. That is why we are proud to launch a new video series: DataChats &lt;/>
DataChats is a new s...
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”. In order to be able to solve this set of exercises you should have solved the part 0, part 1, part 2 ,and part 3 of this series but also you should run this script which contain […]
Reading data from google sheets into R is something you imagine should be really simple, but often is anything but. However, package googlesheets goes a long way to solving this problem. Let’s crack on with an example. First, install the software: install.packages(&quot;googlesheets&quot;) We then need an example sheet to work with, and I’m going to […]
In this post, we are going to see how to deploy R, RStudio, and Shiny apps on a virtual server. The post won’t focus on Shiny programming aspect, neither on Unbuntu programming, however, the tutorial is self-sufficient. Only by following the different steps, you’ll get a working server and Shiny app. 1. Choosing your virtual […]<img height="1" alt="" width="1" src="https://pixel.wp.com/b.gif?host=antoineguillot.wordpress.com&blog=113378294&post=549&subd=antoineguillot&ref=&feed=1" border="0">
I am very pleased to announce the first release of the ggforce package. ggforce is a general ggplot2 extension package in the same vein as ggalt with no overarching goal other than to provide additional functionality to the ggplot2 universe. The inc...
It has been over 2 months since the first satRday conference ended and I shared some quick...
On my trip to Japan, I took this photo of the stairs leading to the “Rucker Park of Tokyo.” I crossed up some Tokyo cats, they were garbage. That one girl behind the blue pillar was practicing her hip hop power moves. She thought no one could see, but I saw. I’ve been traveling. I’ve been starring […]
In an article called A Paradox in the Interpretation of Group Comparisons published in Psychological Bulletin, Lord (1967) made famous the following controversial story:A university is interested in investigating the effects of the nutritional diet its...
A decision tree for function minimization What R calls “optimization” is more generally known as function minimization. The tools in the stats package function optim() are all essentially function mimizers, as are those in the package optimrx found at https://r-forge.r-project.org/projects/optimizer/. optimrx tries to make it easier for users to call quite a few methods without having to learn many different sets of command…
Last week’s post showed how to create a Gilmore Girls character network.
In this week’s short post, I want to explore the Gilmore Girls’ famous coffee addiction by analysing the same episode transcripts that were also used last week.
I am also ...
Basis and principle of D3partitionR D3partitionR is to plot sequential and hierarchical data using treemap (and circle Tree Map), sunburst and partition chart, collapsible trees (indented or not). The package only has one all-in function called D3partitionR(…) to create a partition chart. Two others functions as renderD3partitionR(…) and D3partitionROutput(…) are used to render partition charts in […]
Taking a look at the Denver crime data. Starting at exploratory and hoping to get some deeper insights later on.
Theoretical stuff Last time we talked about pure additive models, today I want to discuss multiplicative ones. There is a general scepticism about pure multiplicative exponential smoothing models in the forecasters society, because it is not clear why level, trend, seasonality and error term should be multiplied. Well, when it comes to seasonality, then there […]
Using R to get a better view into crime trends in the city of Denver.
The fourth, the fifth, the minor fall and the major lift (Hallelujah, Leonard Cohen) Next problem is extracted from MacKay’s Information Theory, Inference and Learning Algorithms: Two people have left traces of their own blood at the scene of a crime. A suspect, Oliver, is tested and found to have type ‘O’ blood. The blood … <a href="https://fronkonstin.com/2016/11/17/bayesian-blood/">Continue reading <span>Bayesian Blood</span> <span>→</span></a>
As we prepare for the upcoming release of Microsoft R Open, I've been preparing the list of new and updated packages for the spotlights page. This involves scanning the CRANberries feed (with gracious thanks to Dirk Eddelbuettel) for newly-released packages and significant updates to existing ones. This is a lot of data to process. For context, in October 2016 alone there were 174 new packages and 629 updated packages. For the 4 months since the last MRO release, I reviewed all of the new packages and some of the updates (those with a 1.0 or 2.0 release) by hand, and...
  To post your R job on the next post Just visit this link and post a new R job to the R community. You can post a job for free (and there is also a “featured job” option for extra exposure). Current R jobs Job seekers: please follow the links below to learn more and apply for your R job of interest: Featured Jobs Freelance Part time statistician / data scientist Gradient – Posted by tvladeck Anywhere 17 Nov2016 Full-Time Senior Data Analyst, Business Insight – European Tyre Enterprise Ltd Charlie Hall Letchworth Garden City England, United Kingdom 7 Nov2016 Full-Time Investment Data Analyst Risk Premium Investment Management Company LLC – Posted by tnederlof New York New York, United States 27 Oct2016 Full-Time Research Data Manager Brown University – Posted by sfsousa Providence Rhode Island, United States 25 Oct2016 More New Jobs Freelance Part time statistician / data scientist Gradient – Posted by tvladeck Anywhere 17 Nov2016 Freelance Data Analytics Consultant @ Riyadh Province, Saudi Arabia Talmix – Posted by Charlotte Fraisse Riyadh Province Saudi Arabia 15 Nov2016 Full-Time Senior Data Analyst, Business Insight – European Tyre Enterprise Ltd Charlie Hall Letchworth Garden City England, United Kingdom 7 Nov2016 Full-Time Senior Data Analyst, Business Insights @ Avenue […]
A theme that comes up fairly regularly on X validated is the production of a sample with given moments, either for calibration motives or from a misunderstanding of the difference between a distribution mean and a sample average. Here are some entries on that topic: How to sample from a distribution so that mean of […]
Practical Data Science with R, Zumel, Mount; Manning 2014 is a book Nina Zumel and I are very proud of. I have written before how I think this book stands out and why you should consider studying from it. Please read on for a some additional comments on the intent of different sections of the … <a href="http://www.win-vector.com/blog/2016/11/teaching-practical-data-science-with-r/">Continue reading <span>Teaching Practical Data Science with R</span></a>
This is a great time to have tools to do data analysis.  Microsoft is now <a href="https://www.microsoft.com/en-us/evalcenter/evaluate-sql-server-vnext-ctp">allowing evaluations</a> of its new SQL Server.  This new SQL Server can now be deployed to a Linux OS environment.  This is big news especially for those that have followed this blog through the years.<br><br>The new SQL Server version is also carries new analytic tools such as R.  From this Techcrunch <a href="https://techcrunch.com/2016/11/16/microsofts-sql-server-for-linux-is-now-available-for-testing/">announcement</a><br><br><blockquote><span>The new version of SQL Server will include improved support for R Services and a number of new machine learning and deep neural networking features.</span></blockquote><br>These are exciting times in deed.  I hope Microsoft continues this path to bringing new tools to the market.
.fusion-fullwidth-1 {
padding-left: px !important;
padding-right: px !important;
}by Nathan Stephens
How R Enters Through the Back Door
In many organizations, R enters thro...
Introduction In a recent blog post, I introduced the new R package, manhattanly, which creates interactive manhattan and Q-Q plots using the plotly.js engine. In the latest CRAN release, you can also create volcano plots. In this post, I describe how to create interactive volcano plots using the manhattanly package. Volcano plots are the negative […]
Microsoft Cognitive Services is a set of cloud-based machine-intelligence APIs that you can use to extract structured data from complex sources (unstructured text, images, video and audio), and add &quot;AI&quot; type features to applications. A good example is the &quot;Seeing AI&quot; glasses in the video below: the image descriptions, emotion inference, and text recognition are all driven by Cognitive Services APIs. You can call these APIs from any application, including R. Here are a few examples: In a blog post a few months back, I called the Face API from R to determine the apparent gender of programmers according to...
Guest post by Jonathan Sidi, Metrum Research Group ggplot2 has become the standard of plotting in R for many users. New users, however, may find the learning curve steep at first, and more experienced users may find it challenging to keep track of all the options (especially in the theme!). ggedit is a package that … <a href="https://www.r-statistics.com/2016/11/ggedit-interactive-ggplot-aesthetic-and-theme-editor/">Continue reading<span> &quot;ggedit – interactive ggplot aesthetic and theme editor&quot;</span></a>
mapmate has now been updating from version 0.1.0 to 0.2.0 on Github. The key change is the incorporation of new functions, help docs and code examples focused on network maps, which is a more complex map type not previously covered. The new tutorial content below provides a a couple basic code examples for making network […]
In the last post, I mapped gas stations and gas prices in Germany. After posting it, I started to look at the dataset from a different angle. The starting question was; “How can I model gas prices? What are the influencing factors?”
One well know...
In case you missed them, here are some articles from October of particular interest to R users. A brief summary of the R 3.3.2 release. &quot;Data Science with SQL Server 2016&quot;, a free E-book featuring several in-depth R examples, is now available for download. The ReporterRs package makes it easy to insert R output, tables and graphics into Word and Powerpoint templates. R-hub, an online service to build and check R packages on multiple platforms, is now in public beta test. A style guide for R programs from Graham Williams, creator of rattle. The Economist used R and the Emotion...
Curious which R packages your colleagues and the rest of the R community are using? Thanks to Rdocumentation.org you can now see for yourself! Rdocumentation.org aggregates R documentation and download information from popular repositories like CRAN, B...
Today we are pleased to release a new version of svglite. This release fixes many bugs, includes new documentation vignettes, and improves fonts support. You can install svglite with: install.packages(&quot;svglite&quot;) Font handling Fonts are tricky with SVG because they are needed at two stages: When creating the SVG file, the fonts are needed in order […]
[For this exercise, first write down your answer, without using R. Then, check your answer using R.] Answers to the exercises are available here. Exercise 1 Consider two vectors, x, y x=c(4,6,5,7,10,9,4,15) y=c(0,10,1,8,2,3,4,1) What is the value of: x*y Exercise 2 Consider two vectors, a, b a=c(1,2,4,5,6) b=c(3,2,4,1,9) What is the value of: cbind(a,b) Exercise […]
So Tuesday happened. I have a lot to say about our President-elect, and you’ll probably see a couple posts about him in the next few weeks. But that’s for later. Today, I’m writing about a more pressing issue, one that has been subject to debate for years without resolution, and I plan to settle this…<a href="https://ntguardian.wordpress.com/2016/11/14/using-microsofts-emotion-api-r-self-portrait/">Read more <span>Using Microsoft’s Emotion API to Settle an Old Argument</span></a>
I’m very pleased to announce ggplot2 2.2.0. It includes four major new features: Subtitles and captions. A large rewrite of the facetting system. Improved theme options. Better stacking. It also includes as numerous bug fixes and minor improvements, as described in the release notes. The majority of this work was carried out by Thomas Pederson, who I […]
<p>Next week I will be traveling to Washington to meet with someone at the US Census Bureau. While I’ve had several phone calls with people...</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2016/11/14/questions-census-bureau/">Questions for the Census Bureau?</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
The usual response from statisticians and data professionals to pie charts ranges from lofty disdain to outright snobbery. But sometimes I think they’re the right tool for communication with a particular audience. Like others I was struck by this i...
Seth Flaxman (Oxford), Dougal J. Sutherland (UCL), Yu-Xiang Wang (CMU), and Yee Whye Teh (Oxford), published on arXiv this morning an analysis of the US election, in what they called most appropriately a post-mortem. Using ecological inference already employed after Obama’s re-election. And producing graphs like the following one:Filed under: pictures, R, Statistics, Travel, University […]
Recent development in R packages are increasingly focussing on the philosophy of tidy data and a common package design and api. Tidy data is an important part of data exploration and analysis, as shown in the following figure: Tidying data not only includes data cleaning, but also data transformation, both being necessary to perform the […]
by Lixun Zhang (Data Scientist), Ye Xing (Senior Data Scientist) and Tao Wu (Principal Data Scientist Manager), all at Microsoft Editor's Note: To learn more about migrating from SAS to R, there will be a live webinar presented by Lixun and Ye tomorrow (Tuesday, November 15). Register to attend the webinar here. R has been gaining in popularity among data professionals in recent years in industries such as financial services, as shown for example in this survey from executive search firm Burtch Works. In this blog post we share some key considerations in migrating SAS to R for a financial...
In this Exercise, we will dive quickly through some basic sampling methods. Follow along this series to use these methods later for our decision trees modelling exercise. We will sample using the package caTools and caret. This is a beginner level exercise. Please refer to the help section for set.seed(), sample.split(),createDataPartition(), and createFolds() functions. You […]
A new minor release 0.1.4 of pkgKitten just hit on CRAN this morning.
One small change is that the package manual page it creates now makes use of the (still new-ish and under-documented and under-used) Rd macros described at the end of Section 2.13...
I have come several times upon cases of scientists [I mean, real, recognised, publishing, senior scientists!] from other fields blindly copying MCMC code from a paper or website, and expecting the program to operate on their own problem… One illustration is from last week, when I read a X Validated question [from 2013] about an […]
One of the most appealing data visualisation charts are maps.
I love maps as they combine an incredible information density with intuitive readability.
Also I feel that most people prefer maps over other visualisations. (Is there research on this?) So...
A rather obscure question on Metropolis-Hastings algorithms on X Validated ended up being about our first illustration in Introducing Monte Carlo methods with R. And exposing some inconsistencies in the following example… Example 7.2 is based on a [toy] joint Beta x Binomial target, which leads to a basic Gibbs sampler. We thought this was […]
<p>About a couple of years ago, I rolled out the flock package to help me synchronize R processes. I have used it ever since, but it wasn’t until recently that I found the time to move the source code to GitHub, and to add the package to CRAN.</p>
<p>The post <a rel="nofollow" href="http://www.quintuitive.com/2016/11/13/flock-package-cran/">The flock Package is on CRAN</a> appeared first on <a rel="nofollow" href="http://www.quintuitive.com/">Quintuitive</a>.</p>
Microsoft provides some sophisticated Machine Learning tools via its Cognitive Services: Microsoft enables us to access several of its APIs in order to leverage Advanced Analyses. One of these is the Emotion API which can be fed pictures or videos that...
Friday 11 November 2016 - 19:06
ShinyProxy is a novel, open source platform to deploy Shiny apps for the enterprise or larger organizations.
Our previous post on the how and why of ShinyProxy triggered a lot of encouraging reactions. Here's our...
About a decade or so photomosaics were all the rage: a near-recreation of a famous image by using many smaller images as elements. Here, for example, is the Mona Lisa, created using the Metapixel program by overlaying 32x32 images of vehicles and animals. An image like this presents an interesting computer vision challenge: can you use deep learning techniques to find the pictures of boats and cars embedded in the image, amongst all the noise and clutter of the other images around and often on top of them? This is the challenge that Max Kaznady and his colleagues in the...
.fusion-fullwidth-1 {
padding-left: px !important;
padding-right: px !important;
}by Sean Lopp
The RStudio IDE reached version 1.0 this month. The IDE has come a long way s...
Welcome! We are making this post today to address a burning question in our minds. And you can help us with! We just expanded our author team for creating exercise sets once more. One of our goals is to deliver content that you really want to practice with and incorporate in your daily R work […]
Nina Zumel and I have been doing a lot of writing on the (important) details of re-encoding high cardinality categorical variables for predictive modeling. These are variables that essentially take on string-values (also called levels or factors) and vary through many such levels. Typical examples include zip-codes, vendor IDs, and product codes. In a sort … <a href="http://www.win-vector.com/blog/2016/11/you-should-re-encode-high-cardinality-categorical-variables/">Continue reading <span>You should re-encode high cardinality categorical variables</span></a>
My adopted home state of Utah has been a weird place this election cycle. For the unfamiliar, Utah is extremely conservative when it comes to politics; it is one of the reddest of the red states and has backed the Republican candidate for president for...
Real-world data sometime show complex structure that call for the use of special models. When data are organized in more than one level, hierarchical models are the most relevant tool for data analysis. One classic example is when you record student performance from different schools, you might decide to record student-level variables (age, ethnicity, social … <a href="https://biologyforfun.wordpress.com/2016/11/10/hierarchical-models-with-rstan-part-1/">Continue reading <span>Hierarchical models with RStan (Part 1)</span></a>
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”. In order to be able to solve this set of exercises you should have solved the part 0 and’part 1 , and part 2 of this series but also you should run this script which contain some […]
Written by Frank E Harrell Jr (first published here) A major new version of the rms package is now on CRAN.  The most user-visible changes are: – interactive plotly graphic methods for model fits.  The best example of this is survplot for npsurv (Kaplan-Meier) estimates where the number of risk pop up as you hover over the curves, and you can click to bring up confidence bands for differences in survival curves – html methods for model fit summaries especially when using Rmarkdown html notebooks – instead of running print(fit, latex=TRUE) use options(prType=’latex’) print(fit) Or: options(prType=’html’) A complete list of changes is below.  See http://biostat.mc.vanderbilt.edu/Rrms for an html notebook showing examples of new features. Changes in version 5.0-0 (2016-10-31) * plot.summary.rms: implemented plotly interactive plots if options(grType=’plotly’) in effect * plot.anova.rms: implemented plotly interactive plots if options(grType=’plotly’) in effect; remove psmall argument; changed margin default to chisq and P * ggplot.Predict: implemented plotly interactive plots if options(grType=’plotly’) in effect * print(fit, md=TRUE), prModFit, prStats: added latex/html methods using htmlTable package and MathJax for latex math * html.anova.rms, html.validate, html.summary.rms: new functions for use with html and MathJax/knitr/RStudio * latex methods for model fits: added md=TRUE argument to produce MathJax-compatible latex and […]
This animation of AirBnB host locations from 2011-2014, presented by Ricardo Bion (data scientist manager at AirBnb) at the EARL Boston conference earlier this week, shows the dramatic growth in properties to rent through the service along with the most common routes of travellers. (You can find the R code that created this animation here.) How did AirBnb achieve such rapid growth? According to Ricardo, it's by being a &quot;data-informed company&quot;. One of the first eight hires at AirBnb was a data scientist. And as we've noted here before, R is widely used at AirBnb along with other data science...
Nina Zumel recently mentioned the use of Laplace noise in “count codes” by Misha Bilenko (see here and here) as a known method to break the overfit bias that comes from using the same data to design impact codes and fit a next level model. It is a fascinating method inspired by differential privacy methods, … <a href="http://www.win-vector.com/blog/2016/11/laplace-noising-versus-simulated-out-of-sample-methods-cross-frames/">Continue reading <span>Laplace noising versus simulated out of sample methods (cross frames)</span></a>
This report describes changes in version 0.4 of the ‘DOM’ package for R. The main change in this version is the addition of new functions that allow control over the Cascading Style Sheet (CSS) content of a web page. This … <a href="http://stattech.wordpress.fos.auckland.ac.nz/2016-14-dom-version-0-4/">Continue reading <span>→</span></a>
This afternoon, one of my Monte Carlo students at ENSAE came to me with an exercise from Monte Carlo Statistical Methods that I did not remember having written. And I thus “charged” George Casella with authorship for that exercise! Exercise 3.3 starts with the usual question (a) about the (Binomial) precision of a tail probability […]
Did you know last month over 23,000 (!) R enthusiasts directly downloaded the Plotly R package?
In case you were not one of them, the Plotly package is one of the best R tools to create beautiful, interactive data visualizations as proven by these exa...
[For this exercise, first write down your answer, without using R. Then, check your answer using R.] Answers to the exercises are available here. Exercise 1 if x &lt;- c(a = 1, b = 2,c=3,d=4) What is the output for the code: seq(5,11,along.with =x) Exercise 2 If x= seq(4,12,4) , what is the output for […]
.fusion-fullwidth-1 {
padding-left: px !important;
padding-right: px !important;
}by Jonathan Regenstein
Financial applications were an early driving force behind the adopt...
On October 12, RStudio launched R Views with great enthusiasm. R Views is a new blog for R users about the R Community and the R Language. Under the care of editor-in-chief and new RStudio ambassador-at-large, Joseph Rickert, R Views provides a new perspective on R and RStudio that we like to think will become […]
In this post we’ll create a Filled Chord Diagram using plotly. The post is inspired by Plotly’s Python documentation. Install / update packages Just to ensure we are working with the latest dev version of plotly. [crayon-5822c30650161761327802/] Dataset The dataset we’ll use consists of the number of comments a person made on her own facebook […]
by Ali Zaidi, Data Scientist at Microsoft # Apache Spark and a Tale of APIs Spark is an exceptionally popular processing engine for distributed data. Dealing with data in distributed storage and programming with concurrent systems often requires learning complicated new paradigms and techniques. Statisticans and data scientists familiar wtih R are unlikely to have much experience with such systems. Fortunately, Spark is a very intuitive distributed computing platform, and there are APIs available in R and Python that are more familiar to the average data scientist than scala. One of the most intriguing APIs available for Spark for R...
A minor release 0.0.3 of gettz arrived on CRAN two days ago.
gettz provides a possible fallback in situations where Sys.timezone() fails to determine the system timezone. That can happen when e.g. the file /etc/localtime somehow is not a link into th...
Continuing from Part 1, in case you have not done so, please set up the data as described before we try to make this temporal polar plot. To create Figure 1b. Graph showing accumulation of records through time (during the year) we need  use function tempolar. This name ‘tempolar’ is simply a short of ‘temporal polar’. For this […]
Weighted effect coding is a variant of dummy coding to include categorical variables in regression analyses, in which the estimate for each category represents the deviation of that category from the sample mean. The ‘wec’ ...
Interested in a career as a quantitive analyst? Working on a computational finance team? Well then, we've got great news! At DataCamp you can now enroll yourself and your team in an entirely new series of courses focused on Applied Finance with R. Lea...
When the RStudio first came on the scene in February 2011, there wasn't much in the way of tools for developers using the R language. The R GUI on Windows and Mac had a basic code-editing window, and there were a couple of menu-based GUIs like Rattle and R Commander that were aimed more at those who would prefer not to program in the R language. But at the time, the only real integrated development environments (IDEs) for R were ESS for Emacs, the Revolution R IDE (which has since been superseded by R Tools for Visual Studio). While RStudio...
In my posts to this blog (less frequent than I would like, hopefully more frequent in the future), I’ve often mentioned my R package regtools, which contains a number of functions useful for regression and classification. None of them duplicate what is available in the excellent packages on CRAN, so I will dare characterize regtools … <a href="https://matloff.wordpress.com/2016/11/07/my-regtools-package-is-now-on-cran/">Continue reading <span>My regtools Package Is Now on CRAN</span> <span>→</span></a>
Background There are a few ways to approach the problem of a conditionally formatted table in R. You can use the ReporteRs package's FlexTable() function, the formattable package, or the condformat package. These allow you to produce a conditionally formatted tables in HTML. You can also use xtable package and essentially program what you want … <a href="http://labrtorian.com/2016/11/07/conditional-formatting-of-a-table-in-r/">Continue reading <span>Conditional Formatting of a Table in R</span> <span>→</span></a>
Naturally, both Hillary Clinton and Donald Trump have been accused of lying; if I had told you in 2012 that both candidates from both political parties were being accused of lies, you would likely have given me a blank, disinterested stare; this alone is not shocking. What is shocking, though, is the level of deceit…<a href="https://ntguardian.wordpress.com/2016/11/07/deceit-politics-analysis-politifact-data/">Read more <span>Deceit in Politics; An Analysis of PolitiFact Data</span></a>
Following a question on X Validated, I became aware of the following descriptions of the pros and cons of Bayesian analysis, as perceived by whoever (Tim Arnold?) wrote SAS/STAT(R) 9.2 User’s Guide, Second Edition. I replied more specifically on the point It [Bayesian inference] provides inferences that are conditional on the data and are exact, […]
<p>Ghana has always been getting flooded, but it seems that only floods in Accra are getting attention. I wrote about it here, and the key visualization was an animated map showing the floods in Ghana, and built in R. In this post, I document how I did it, hopefully you can do one also! Before we start, there are a…
</p><div><a href="http://datascience-africa.org/2016/11/ghana-floods-animation-created/">Read more</a></div>
<p>The post <a rel="nofollow" href="http://datascience-africa.org/2016/11/ghana-floods-animation-created/">How the Ghana Floods animation was created</a> appeared first on <a rel="nofollow" href="http://datascience-africa.org/">Data Science Africa</a>.</p>
<p>Raccoon is a free web-book about Statistical Models with R. It's the collection of twenty years of notes, exercises and concepts working with statistics and R. This is the second section of the second chapter, about 2-sample t-test and paired t.</p>
<p>The post <a rel="nofollow" href="http://www.quantide.com/raccoon-ch-2-2-2-sample-t-test-and-paired-t/">Raccoon | Ch 2.2 – 2 sample t-test and paired t</a> appeared first on <a rel="nofollow" href="http://www.quantide.com/">Quantide - R training &amp; consulting</a>.</p>
<p>We have collected all notes and materials of the 7th MilanoR Meeting and now they're available online. </p>
<p>The post <a rel="nofollow" href="http://www.milanor.net/blog/7th-milanor-meeting-download-presentations/">7th MilanoR Meeting: download presentations</a> appeared first on <a rel="nofollow" href="http://www.milanor.net/blog">MilanoR</a>.</p>
shinyjs is a package that helps you easily improve the user interaction and user experience in your shiny apps. It has come a long way in the 1.5 years since it was first released, and today I’m happy to announce that version 0.8 is now on...
After using R for a few years people come to realize how inefficient loops are (in all languages), even after pre-allocating memory. Then you begin to use the apply functions and revel at their speed. However, once you have datasets > 10-15 million the apply family can begin to creep along (relative to the amount […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toR Markdown: How to number and reference tables" rel="bookmark" href="http://datascienceplus.com/r-markdown-how-to-number-and-reference-tables/">R Markdown: How to number and reference tables</a></li><li><a title="Permanent Link toRDBL – manipulate data in-database with R code only" rel="bookmark" href="http://datascienceplus.com/rdbl-manipulate-data-in-database-with-r-code-only/">RDBL – manipulate data in-database with R code only</a></li><li><a title="Permanent Link toR Markdown: How to format tables and figures in .docx files" rel="bookmark" href="http://datascienceplus.com/r-markdown-how-to-format-tables-and-figures-in-docx-files/">R Markdown: How to format tables and figures in .docx files</a></li><li><a title="Permanent Link toR Markdown: How to insert page breaks in a MS Word document" rel="bookmark" href="http://datascienceplus.com/r-markdown-how-to-insert-page-breaks-in-a-ms-word-document/">R Markdown: How to insert page breaks in a MS Word document</a></li><li><a title="Permanent Link toWorking on Data-Warehouse (SQL) with R" rel="bookmark" href="http://datascienceplus.com/working-on-data-warehouse-sql-with-r/">Working on Data-Warehouse (SQL) with R</a></li></ol>
<p>sparklyr is a new R front-end for Apache Spark, developed by the good people at RStudio. It offers much more functionality compared to the existing SparkR interface by Databricks, allowing both dplyr-based data transformations, as well as access to the machine learning libraries of both Spark and H2O Sparkling Water. Moreover, the latest RStudio IDE v1.0 now offers native support ...
</p><div><a href="http://www.nodalpoint.com/sparklyr-on-yarn/">Read More</a></div>
<p>The post <a rel="nofollow" href="http://www.nodalpoint.com/sparklyr-on-yarn/">sparklyr: a test drive on YARN</a> appeared first on <a rel="nofollow" href="http://www.nodalpoint.com/">Nodalpoint</a>.</p>
A few months ago I wrote about my first solo author submission to the R Journal entitled, “A Simple Guide to S3 Methods”. Unfortunately the article didn’t make it to publication on the R Journal, but admittedly that might have been a bit of a lon...
In the last few years there have been more attempts at a fresh approach to statistical timeseries forecasting using the increasingly accessible tools of machine learning. This means methods like neural networks and extreme gradient boosting, as supple...
Shiny Server 1.5.1.834 and Shiny Server Pro 1.5.1.760 are now available. The Shiny Server 1.5.x release family upgrades our underlying Node.js engine from 0.10.47 to 6.9.1. The impetus for this change was not stability or performance, but because the 0.10.x release family has reached the end of its life. We highly recommend that you test […]
I have started writing a ‘book’ using the awesome bookdown package. In the book I explain and show why using functional programming and putting your functions in your own packages is the way to go when you want to clean, prepare and transform lar...
On the 1st of september we launched R course finder, an online directory that helps you to find the right R course quickly. With so many R courses available online, we thought it was a good idea to offer a tool that helps people to compare these courses, before they decide where to spend their […]
I’m pleased to announce the CRAN release of packagedocs which provides a mechanism for simple generation and automated deployment of nice-looking online R package documentation that plugs into the traditional R package vignette system.
You can see...
.fusion-fullwidth-1 {
padding-left: px !important;
padding-right: px !important;
}by Joseph Rickert
Here we offer ephemera, a little light reading and some more challenging...
Many economists would agree that the most efficient way to fight global warming would be a world-wide tax or an emmission trading system for greenhouse gases. Yet, if only a part of the world implements such a scheme, a reasonable concern is that firms...
Being still puzzled (!) by the ratio-of-uniform approach, mostly failing to catch its relevance for either standard distributions in a era when computing a cosine or an exponential is negligible, or non-standard distributions for which computing bounds and boundaries is out-of-reach, I kept searching for solutions that would include unbounded densities and still produce compact […]
Rmazon The goal of Rmazon is to help you download product information and reviews from Amazon.com easily. Installation You can install Rmazon from github with: # install.packages(&quot;devtools&quot;) devtools::install_github(&quot;56north/Rmazon&quot;) Example – product information This is a basic example which shows you how ro get product information: # Get product information for 'The Art of R Programming:... <a title="Read Download product information and reviews from Amazon.com" href="http://www.56n.dk/download-product-information-and-reviews-from-amazon-com/">Read more »</a>
The rbokeh package version 0.5.0 was recently released. For those not familiar with rbokeh, it is a plotting library based on BokehJS with the goal of making it easy to flexibly create declarative interactive web-based visualizations in R. To get an o...
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”. In order to be able to solve this set of exercises you should have solved the part 0 and’part 1 of this series, in case you haven’t you can find the solutions to run them in your […]
Next summer of 2017, the biennial International Conference on Monte Carlo Methods and Applications (MCM) will take place in Montréal, Québec, Canada, on July 3-7. This is a mathematically-oriented meeting that works in alternance with MCqMC and that is “devoted to the study of stochastic simulation and Monte Carlo methods in general, from the theoretical […]
If the media coverage is anything to go by, people are desperate to know who will win the US election on November 8. Polls give us some indication of what's likely to happen, but any single poll isn't a great guide (despite the hype that accompanies some of them). One poll is subject to any number of possible errors, statistical and otherwise: the sample, the methodology, the analysis, or even deliberate bias. But put a whole bunch of polls together, and you can assemble a more realistic picture of the likely outcome, and the uncertainty associated with it. That's what...
Someone contacted me recently about the map projections used in my blscrapeR package. After a bit of web searching, I couldn’t find a really good list of map projections for the continental U.S. that could be used in R. This list is as much for my own reference as anyone else, but hope you find […]
.fusion-fullwidth-1 {
padding-left: px !important;
padding-right: px !important;
}by Joseph Rickert
I found my way into data science and machine learning relatively late in...
Every now and again someone comes along and writes an R package that I consider to be a ‘game changer’ for the language and it’s application to Data Science. For example, I consider dplyr one such package as it has … <a href="https://alexioannides.com/2016/11/02/asynchronous-and-distributed-programming-in-r-with-the-future-package/">Continue reading <span>→</span></a>
Recently Steph Locke asked on twitter if there was a list of R conferences. After some googling, all that I came up was a list of useR groups maintained by Microsoft. While the list was lengthy, it was missing a few groups (and twitter handles). So the other night I created a github repository listing […]
The glmnetUtils package provides a collection of tools to streamline the process of fitting elastic net models with glmnet. I wrote the package after a couple of projects where I found myself writing the same boilerplate code to convert a data frame into a predictor matrix and a response vector. In addition to providing a formula interface, it also has a function (cvAlpha.glmnet) to do crossvalidation for both elastic net parameters α and λ, as well as some utility functions. The formula interface The interface that glmnetUtils provides is very much the same as for most modelling functions in R....
Today we’re very pleased to announce the availability of RStudio Version 1.0! Version 1.0 is our 10th major release since the initial launch in February 2011 (see the full release history below), and our biggest ever! Highlights include: Authoring tools for R Notebooks. Integrated support for the sparklyr package (R interface to Spark). Performance profiling via […]
mapmate has now been updating from version 0.0.2 to 0.1.0 on Github. The biggest addition is a number of plotting options for making different kinds of maps. The new tutorial content below provides a number of code examples for making a variety of maps and also highlights current limitations associated with certain map types and […]
NIMBLE version 0.6-1 has been released on CRAN and at  r-nimble.org.   NIMBLE is a system that allows you to: Write general hierarchical statistical models in BUGS code and create a corresponding model object to use in R. Build Markov chain Monte Carlo (MCMC), particle filters, Monte Carlo Expectation Maximization (MCEM), or write generic algorithms […]
We have already written quite a few times about our vtreat open source variable treatment package for R (which implements effects/impact coding, missing value replacement, and novel value replacement; among other important data preparation steps), but we thought we would take some time to describe some of the principles behind the package design. Introduction vtreat … <a href="http://www.win-vector.com/blog/2016/11/some-vtreat-design-principles/">Continue reading <span>Some vtreat design principles</span></a>
I was attending a course of Bayesian Statistics where this problem showed up: There is a number of individuals, say 12, who take a pass/fail test 15 times. For each individual we have recorded the number of passes, which can go from 0 to 15. Because of confidentiality issues, we are presented with rounded-to-the-closest-multiple-of-3 data […]
R 3.3.2, the latest update to the R language, was released today. Binary releases for Linux and Mac are available now from your local CRAN mirror, and the Windows builds will be available shortly. As a minor update to the R 3.3 series, this update focuses mainly on fixing bugs and doesn't make any major changes to the langauge. As a result, you can expect existing scripts and packages to continue to work if you're upgrading from R 3.3.1. This update includes some performance improvements (particularly in calculation of eigenvalues), better handling of date axes in graphics, and improved documentation...
Network models have become a popular way to abstract complex systems and gain insights into relational patterns among observed variables in almost any area of science. The majority of these applications focuses on analyzing the structure of the network...
R 3.3.2 (codename “Sincere Pumpkin Patch”) was released yesterday You can get the latest binaries version from here. (or the .tar.gz source code from here). The full list of bug fixes and new features is provided below. Upgrading to R 3.3.2 on Windows If you are using Windows you can easily upgrade to the latest version of R using the installr … <a href="https://www.r-statistics.com/2016/11/r-3-3-2-is-released/">Continue reading<span> &quot;R 3.3.2 is released!&quot;</span></a>
<p>Today I am happy to announce my newest course: Learn Shiny in an Afternoon! To learn more, watch this video: Shiny has revolutionized the way that...</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2016/10/31/learn-shiny-afternoon/">Learn Shiny in an Afternoon!</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
During the eRum 2016, Adam Zagdański gave a very good tutorial about time series modeling. Among other things I’ve learned that the forecast package (created by Rob Hyndman) got cool new plots based on the ggplot2 package. Let’s use it to play with mailbox statistics for my gmail account! 1. Get the data Follow this … <a href="http://smarterpoland.pl/index.php/2016/10/ggmail-ggplot2-data-from-gmail-forecast/">Czytaj dalej <span>ggmail + forecast = how many emails I will get tomorrow?</span></a>
We prepared a poster/cheatsheet for the bioconductor package <a href="https://bioconductor.org/packages/release/bioc/html/genomation.html">genomation</a>, which is a package for summary and annotation of genomic intervals. Users can visualize and quantify genomic intervals over pre-defined functional regions, such as promoters, exons, introns, etc. The genomic intervals represent regions with a defined chromosome position, which may be associated with a score, such as aligned reads from HT-seq experiments, TF binding sites, methylation scores, etc. The package can use any tabular genomic feature data as long as it has minimal information on the locations of genomic intervals. In addition, It can use BAM or BigWig files as input. [download from slideshare for a better resolution]<div><br></div><div><br></div><div> <div> <strong> <a title="Poster/cheatsheet for R/BioC package genomation" target="_blank" href="https://www.slideshare.net/altunaakalin/postercheatsheet-for-rbioc-package-genomation">Poster/cheatsheet for R/BioC package genomation</a> </strong> from <strong><a target="_blank" href="https://www.slideshare.net/altunaakalin">Altuna Akalin</a></strong> </div></div>
Detecting outliers and fraudulent behaviour (transactions, purchases, events, actions, triggers, etc.) takes a large amount of experiences and statistical/mathetmatical background. One of the samples Microsoft provided with release of new SQL Server 2016 was using simple logic of Benford’s law. This law works great with naturally occurring numbers and can be applied across any kind … <a href="https://tomaztsql.wordpress.com/2016/10/31/detecting-outliers-and-fraud-with-r-and-sql-server-on-my-bank-account-data-part-1/">Continue reading <span>Detecting outliers and fraud with R and SQL Server on my bank account data – Part 1</span></a>
Monday 31 October 2016 - 15:58
ShinyProxy is a novel, open source platform to deploy Shiny apps for the enterprise or larger organizations.
Why is this needed?
There is currently no valid open source alternative that offers this functionality.
...
If your regression model contains a categorical predictor variable, you commonly test the significance of its categories against a preselected reference category. If all categories have (roughly) the same number of observations, you can also ...
Way back in 2012, I posted a simple R script for trying to work out the finishing combinations in the last two races of that year’s F1 season for Fernando Alonso and Sebastien Vettel to explore the circumstances under which Alonso could take the championship (Paths to the F1 2012 Championship Based on How They […]
Following my earlier post on Kinderman’s and Monahan’s (1977) ratio-of-uniform method, I must confess I remain quite puzzled by the approach. Or rather by its consequences. When looking at the set A of (u,v)’s in R⁺×X such that 0≤u²≤ƒ(v/u), as discussed in the previous post, it can be represented by its parameterised boundary u(x)=√ƒ(x),v(x)=x√ƒ(x)    x […]
A common task performed during data preparation or data analysis is the manipulation of strings. Regular expressions are meant to assist in such and similar tasks. A regular expression is a pattern that describes a set of strings. Regular expressions can range from simple patterns (such as finding a single number) thru complex ones (such […]
CRAN requested a release updating any URLs for Omegahat to the (actually working) omegahat.net URL. So that caused this 0.1.2 release which arrived on CRAN yesterday. It contains the requested change along with one or two other mostly minor changes w...
In a previous post, we demonstrated that ridge regression (a form of regularized linear regression that attempts to shrink the beta coefficients toward zero) can be super-effective at combating overfitting and lead to a greatly more generalizable model. This approach… <p><a href="http://www.onthelambda.com/2016/10/30/the-bayesian-approach-to-ridge-regression/">Continue reading <span>→</span></a></p>
It’s nearly summeRtime in Australia! Join RStudio Chief Data Scientist Hadley Wickham for his popular Master R workshop in Melbourne. Register here:  https://www.eventbrite.com/e/master-r-developer-workshop-melbourne-tickets-22546200292 Melbourne will be Hadley’s first and only scheduled Master R workshop in Australia. Whether you live or work nearby or you just need one more good reason to visit Melbourne in the […]
3,000+ voting intention surveys
Like many others around the world, I have been watching with interest the democratic process in the United States of America. One of the most influential and watched websites is FiveThirtyEight.com, headed by Nate Silv...
The package ReporteRs has been getting some play on the interwebs this week, though it’s actually been around for a while. The nice thing about this package is that it allows writing Word and PowerPoint documents in an OS-independent fashion unlike some earlier packages. It also allows the editing of documents by using bookmarks within […]
There's a new e-book available to download free from Microsoft Academy: Data Science with Microsoft SQL Server 2016. This 90-page e-book is aimed at data scientists who already have some experience in R, but want to learn how to use R wirth SQL Server. The book was written by some of my most experienced colleagues from Microsoft's data science team at Microsoft: Buck Woody, Danielle Dean, Debraj GuhaThakurta, Gagan Bansal, Matt Conners, and Wee-Hyong Tok, and begins with an introduction by Joseph Sirosh. It includes everything you need to know to use R and SQL Server: How to install and...
Lazy reader’s guide: skip to the pretty pictures, skim the conclusions section, ignore the rest.
Background
I think a lot about eigenvalue and singular value decompositions. I won’t get into it right now, but I have been quoted in the past as sa...
I am so pleased to announce that tidytext 0.1.2 is now available on CRAN. This release of tidytext, a package for text mining using tidy data principles by Dave Robinson and me, includes some bug fixes and performance improvements, as well as some new ...
<p>Ok, this post is about one of my latest crazy/harebrained/whacky ideas. I’m fed up of having to carry my Galaxy Note 3 in my hand. I can’t stand handbags and most women’s clothing items don’t have pockets or the pockets are insufficient. Given how easy it is to build a website these days, I thought […]</p>
<p>The post <a rel="nofollow" href="http://itsalocke.com/girlswithdeeppockets-com/">GirlswithDeepPockets.com</a> appeared first on <a rel="nofollow" href="http://itsalocke.com/">It's a Locke</a>.</p>
.fusion-fullwidth-1 {
padding-left: px !important;
padding-right: px !important;
}by Joseph Rickert
Computer Age Statistical Inference: Algorithms, Evidence and Data Scienc...
In order to get the discount, simply click choose a link below and when paying use the promo code: JULY20310 Udemy is offering readers of R-bloggers access to its global online learning marketplace for only $10 per course! This deal (offering over 50%-90% discount) is for hundreds of their courses – including many R-Programming, data science, machine learning etc. Click here to browse ALL (R and non-R) courses Advanced R courses:  The Comprehensive Programming in R Course (25 Hours of video) Bayesian Computational Analyses with R (12 Hours of video) R Programming for Simulation and Monte Carlo Methods (12 Hours of video) Applied Multivariate Analysis with R (13 Hours of video) Linear Mixed-Effects Models with R (11 Hours of video) Graphs in R (ggplot2, plotrix, base R) – Data Visualization with R Programming Language (5 Hours of video) Multivariate Data Visualization with R (7 Hours of video) More Data Mining with R (11 Hours of video) Text Mining, Scraping and Sentiment Analysis with R (4 Hours of video) Programming Statistical Applications in R (12 Hours of video) Comprehensive Linear Modeling with R (15 Hours of video) Time Series Analysis and Forecasting in R (3 Hours of video) Introductory R courses:  Introduction to R (15 Hours of video) Applied Data Science with R (11 Hours of video) R Level 1 […]
To stay on top of R in the news, we’re sharing some stories related to R published last week. The R Graph Gallery Has Made a Combeback(Yan Holtz) I must admit, the first edition of this gallery must have been before my time! Announced here, I was immediately most intrigued by what obscure visualizations were […]
We are proud to present simmer 3.5.0, a new release on CRAN of the Discrete-Event Simulator for R with a bunch of exciting features:
Simpler logging. So far, if we wanted arrivals to print something into the console, we had, for instance, to make a...
<p>Raccoon is a free web-book about Statistical Models with R. It's the collection of twenty years of notes, exercises and concepts working with statistics and R. This is the first section of the second chapter, about 1-sample t-test.</p>
<p>The post <a rel="nofollow" href="http://www.quantide.com/raccoon-ch-2-1-1-sample-t-test-with-r/">Raccoon | Ch. 2.1 – T-test, Anova: 1-sample t-test</a> appeared first on <a rel="nofollow" href="http://www.quantide.com/">Quantide - R training &amp; consulting</a>.</p>
As much as we data scientists would like to work solely in R, there inevitably comes the time when our managers or customers want to see the fruits of our labours as a Word document or PowerPoint presentation. And not just any old document: it needs to be in the official corporate template, and will no doubt go through a series of back-and-forth revisions — marked up directly in the document — before it's considered final. R has several options for generating Word and PowerPoint documents, notably RMarkdown and Slidify. But in both these cases, R is in the driver's...
Recently I got to read the paper about Completeness of Digital Accessible Knowledge DAK by Alex Asase and A. Townsend Peterson. I really enjoyed reading the paper and liked the way the figures are presented. There is a lot of overlap of this with my work on package bdvis (of course under guidance of Town Peterson). So I […]
mapmate is an R package for map- and globe-based data animation pre-production. Specifically, mapmate functions are used to generate and save to disk a series of global map graphics that make up a still image sequence, which can then be used in video editing and rendering software of the user’s choice. This package does not […]
<p>A reader just asked me a fascinating question: Could you give a rough estimate of how much time you had to invest until your portfolio...</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2016/10/26/long-r-portfolio-makes-money/">How long until an R portfolio makes money?</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”. In order to be able to solve this set of exercises you should have solved the ‘part 0’ of this series, in case you haven’t you can find the solutions to run them in your machine here. […]
by Joseph Rickert September was a bit of a slow month for new R packages. Only 96 new packages showed up un CRAN. Nevertheless, I have picked out 23 for special mention which I have listed in 5 categories. I used the same selection criteria as I described in the post for August picks. Data […]
If you're developing a package for R to share with others — on CRAN, say — you'll want to make sure it works for others. That means testing it on various platforms (Windows, Mac, Linux, and all the versions thereof), and on various versions of R (current, past, and future). But it's likely you only have access to one platform, and installing and managing multiple R versions can be a pain. R-hub, the online package-building service now in public beta, aims to solve this problem by making it easy to build and test your package on a variety of platforms...
Ten days ago eRum2016 conference (European R Users Meeting 2016) has finished. It was a huge event that attracted over 250 attenders, both from academia and business. Meeting was a great opportunity to listen to amazing keynotes like Heather Turner, K...
<p>satRday Cape Town will happen on 18 February 2017 at Workshop 17, Victoria &amp; Alfred Waterfront, Cape Town, South Africa. Keynotes and Workshops We have a trio of fantastic keynote speakers: Hilary Parker, Jennifer Bryan and Julia Silge, who’ll be dazzling you on the day as well as presenting workshops on the two days prior …</p>
<p>The post <a rel="nofollow" href="http://www.exegetic.biz/blog/2016/10/satrday-cape-town-call-submissions/">satRday Cape Town: Call for Submissions</a> appeared first on <a rel="nofollow" href="http://www.exegetic.biz/blog">Exegetic Analytics</a>.</p>
A new release of Rblpapi is now on CRAN. Rblpapi provides a direct interface between R and the Bloomberg Terminal via the C++ API provided by Bloomberg Labs (but note that a valid Bloomberg license and installation is required).
This is the sixth rel...
<p>On 27th of October I’m going to attend the <a href="http://www.milanor.net/blog/7th-milanor-meeting-october-27/">7th MilanoR meeting</a> featuring the following two talks:</p> <p>1. <strong>Interactive big data analysis with R: SparkR and MongoDB: a friendly walkthrough  </strong>by  Thimoty Barbieri and Marco Biglieri</p> <p>2. <strong>Power consumption prediction based on statistical learning techniques </strong>by Davide Pandini</p> <p>This is my first official R event and I’m very much looking forward to it. There has been a strong positive reply from a growing number of people interested in R confirmed by the fact that the tickets were sold out pretty much a few hours after the event was announced.</p> <p>However, due to this unexpected sudden success in the allocation of the tickets, many people will not be able to attend. Fear not though, the MilanoR staff has just decided to live stream the event on its brand new <a href="https://www.facebook.com/MilanoRcommunity/?fref=ts">Facebook page</a>. The event will start around 6.30 PM local time. Feel free to leave a comment and share any thoughts you may have on the topic.</p> <p>If you would like to know more about the event, the talks and the speakers, check out the following articles:</p> <p>1. <a href="http://www.milanor.net/blog/7th-milanor-meeting-october-27/">7th MilanoR Meeting: October 27</a></p> <p>2. <a href="http://www.milanor.net/blog/7th-milanor-meeting-live-on-facebook/">7h MilanoR Meeting live on Facebook</a></p>
by Graham Williams, Director of Data Science, Microsoft Programming is an art and a way we express ourselves. As we write our programs we should keep in mind that someone else is very likely to be reading it. We can facilitate the accessibility of our programs through a clear presentation of the messages we are sharing. As data scientists we also practice this art of programming. Indeed even more so we aim to share the narrative of our discoveries through our living and breathing of data through programming over the data. Writing programs so that others understand why and how...
Data Science is sometimes a stressful and responsible job… so we spontaneously organize mini R quizzes in our R team at the company!
Chaining R packages
One of my favourite fast 5-minutes long R game is chaining R packages. One person says the nam...
<p>You would like to attend the meeting but could not get a ticket? Follow the live event online!</p>
<p>The post <a rel="nofollow" href="http://www.milanor.net/blog/7th-milanor-meeting-live-on-facebook/">7th MilanoR Meeting live on Facebook</a> appeared first on <a rel="nofollow" href="http://www.milanor.net/blog">MilanoR</a>.</p>
There was once a time where only the most extreme leftists would accuse Hillary Clinton of not being a true progressive, prior to, say, 2008. Even after 2008, Hillary Clinton was seen as perhaps being a more moderate Democrat, but still, ultimately, a progressive. Republicans certainly would call Clinton a leftist and still continue to…<a href="https://ntguardian.wordpress.com/2016/10/25/hillary-clinton-progressive-statistics/">Read more <span>Is Hillary Clinton a Progressive? An Investigation Using Statistical Methods</span></a>
Digital transformation or digital business transformation is apparently currently one of the hot topics in the German business world.
What puzzles me slightly, is why just now? The digitization trend has been around for ~20 years. Established buisness ...
In the commercial environment money makes the world go around, while in academia recognition (e.g., number of times your work is cited, being fawned over at conferences, impressive job titles) is the coin of the realm (there are a few odd balls who do it out of love for the subject or a desire to […]
If you didn't have a chance to catch my presentation at the Machine Learning and Data Science Summit, I'll be reprising an updated version of the talk in a live webinar on Tuesday, November 1. I'll also be taking questions from the audience after the webinar. You can register here, and the details of the webinar are below. Changing Lives with Data Science and R at Microsoft Whether it’s called data science, machine learning, or predictive analytics, the combination of new data sources and statistical modeling has produced some truly revolutionary applications. Many of these applications incorporate open source technologies...
I recently had the opportunity to listen to some great minds in the area of high-frequency data and trading. While I won’t go into the details about what has been said, I wanted to illustrate the importance of proper out-of-sample testing and proper variable lags in potential trade algorithms or arbitrage models that has been […]
<p>I recently ran an office hours on the topic of building R Portfolios. The conversation was fascinating to me. One issue that came up was monetization....</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2016/10/24/monetizing-r-portfolios/">Monetizing R Portfolios</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
Welcome to the second in a series of posts of maps-to-places-I-am-unlikely-to-visit generated by R (the previous post is <a href="http://www.r-chart.com/2016/10/map-of-middle-earth-map-above-was.html">here</a>).<br><br>A bit-o NASA data, the <a href="http://bwlewis.github.io/rthreejs/">R threejs library</a> and a few lines of code can be used to create an interactive map that identifies the landing locations of Apollo lunar landing missions.<br><br><div><a href="https://3.bp.blogspot.com/-40GVKaSh6a4/WA05ut3y2MI/AAAAAAAAA0Y/B90cEYGp7xQ4sUXJ-WdWWK-a8zG4BKSRACLcB/s1600/moon2.png"></a></div><div><br></div><br>The lunar map above is better viewed using the interactive version on <a href="http://rpubs.com/Cas/apollo-moon-landings1">RPubs</a>.  The data was obtained from <a href="http://nssdc.gsfc.nasa.gov/planetary/lunar/apollo_tables.html#lunar_sites.">a NASA page</a>.  For comparison, see Google's <a href="https://www.google.com/moon/">Map of Apollo mission landing locations</a>.  Additional information is included on this <a href="https://en.wikipedia.org/wiki/List_of_Apollo_missions#/media/File:Moon_landing_map.jpg">map on Wikipedia</a>.<br><br>The <a href="https://github.com/ezgraphs/R-Programs/blob/master/moon/moon.csv">data</a> and <a href="https://github.com/ezgraphs/R-Programs/blob/master/moon/moon.R">code</a> are on Github.  If you can believe it, all the R code required is as follows:<br><br><span>library(threejs)</span><br><span>moon_data = read.csv(&quot;moon.csv&quot;, stringsAsFactors=FALSE)</span><br><span>moon = system.file(&quot;images/moon.jpg&quot;, package=&quot;threejs&quot;)</span><br><span>globejs(img=moon, bodycolor=&quot;#555555&quot;, emissive=&quot;#444444&quot;, </span><br><span>        lightcolor=&quot;#555555&quot;, lat=moon_data$Latitude.N, </span><br><span>        long=moon_data$Longitude.E)</span><br><br><br><br>
One approach to random number generation that had always intrigued me is Kinderman and Monahan’s (1977) ratio-of-uniform method. The method is based on the result that the uniform distribution on the set A of (u,v)’s in R⁺xX such that 0≤u²≤ƒ(v/u) induces the distribution with density proportional to ƒ on V/U. Hence the name. The proof […]
This set of exercises will help you to learn and test your skill with basic arithmetical operations and logic functions. Before proceeding, it might be helpful to look over the help pages for the **, %/%, %%, and the logical operators such as !=, ==, >=, isTRUE . Answers to the exercises are available here. […]
Abstract
In the latest update to the rwml-R Github repo, I provide R code to accompany Chapter 4 of the book “Real-World Machine Learning” by Henrik Brink, Joseph W. Richards, and Mark Fetherolf. Topics covered include optimization of model parame...
Guest post by AVNER KANTOR I used the wonderful tutorial of Dean Attall to set my machine in Google cloud. After I finished to configure it successfully I wanted to redirect my domain to the Shiny application URL. This is a short description how you can do it. The first step is changing the domain … <a href="https://www.r-statistics.com/2016/10/set-application-domain-name-with-shiny-server/">Continue reading<span> &quot;Set Application Domain Name with Shiny Server&quot;</span></a>
A quick demo of RStudio’s R Notebooks shown by John Mount (of Win-Vector LLC, a statistics, data science, and algorithms consulting and training firm). (see http://rmarkdown.rstudio.com/r_notebooks.html and https://www.rstudio.com/products/rstudio/download/preview/ )
A few days ago, Conrad released Armadillo 7.500.0. The corresponding RcppArmadillo release 0.7.500.0.0 is now on CRAN (and will get into Debian shortly).
Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a...
Nina Zumel recently announced upcoming speaking appearances. I want to promote the upcoming sessions at ODSC West 2016 (11:15am-1:00pm on Friday November 4th, or 3:00pm-4:30pm on Saturday November 5th) and invite executives, managers, and other data science consumers to attend. We assume most of the Win-Vector blog audience is made of practitioners (who we hope … <a href="http://www.win-vector.com/blog/2016/10/data-science-for-executives-and-managers/">Continue reading <span>Data science for executives and managers</span></a>
Temperament has been a key issue in the 2016 presidential election between Hillary Clinton and Donald Trump, and an issue highlighted in the series of three debates that concluded this week. Quantifying &quot;temperament&quot; isn't an easy task, but The Economist used the Microsoft Emotion API to chart the anger, contempt, sadness and surprised expressed in the faces of the candidates during key sequences of the debates, like this from the third debate: Economist Data Journalist Ben Heubl explains how you can analyze emotions in a video file using Python and R. The Emotion API provides scores for eight attributes of...
<p>OK, if you haven’t heard of it before Slack is kinda like IRC, kinda like Dropbox, kinda like a lot of things – it’s a neat place to bring together communications between your team or community, and the integrations allow you to pipe in external feeds like twitter activity or RSS. It’s a great way […]</p>
<p>The post <a rel="nofollow" href="http://itsalocke.com/slack-all-the-things/">Slack all the things!</a> appeared first on <a rel="nofollow" href="http://itsalocke.com/">It's a Locke</a>.</p>
.fusion-fullwidth-1 {
padding-left: px !important;
padding-right: px !important;
}by Joseph Rickert
141 new packages landed on CRAN in August. The following are my picks f...
To stay on top of R in the news, we’re sharing some stories related to R published last week. Interactive Data Visualization Using the Shiny App!(Saurav Kaushik) This is one of the best tutorials on Shiny that I have seen, and anything that says interactive next to data visualization really should get the top story […]
First Warsaw R-Ladies Workshops were held yesterday! Over 100 R-Ladies have registered for the event! In this post I present notes from this meeting and provide some pictures of the beauty part of R. You can also learn a bit about the gender gap in th...
I was recently asked to do a panel of grouped boxplots of a continuous variable, with each panel representing a categorical grouping variable. This seems easy enough with ggplot2 and the facet_wrap function, but then my collaborator wanted p-values on the graphs! This post is my approach to the problem. First of all, one caveat. I’m a […]
Last Friday the Cologne R user group came together for two talks and a quiz at Eye/o, the company behind Adblock Plus, in Köln-Ehrenfeld. Eye/o were a great host, offering nibbles and drinks to warm up the event and pizza in the end.Cologne R user mee...
This is the third set of exercises on networks in which we practice the functions for graph structure, using package igraph. The first and second part are available here: Part 1 Part 2 If you don’t have package already installed, install it using the following code: install.packages(&quot;igraph&quot;) and load it into the session using the […]
So, I just finished building a R wrapper for FOAAS (F*ck Off As A Service).  FOAAS (F*ck Off As A Service) provides a modern, RESTful, scalable solution to the common problem of telling people to f*ck off. I wanted to share the package with you and also tell you how I automated the build. You... <a title="Read Building a package automatically" href="http://www.56n.dk/building-a-package-automatically/">Read more »</a>
Once upon a time, there was the original R Graph Gallery, by Romain François. Sadly, it's been unavailable for several years. Now there's a new R Graph Gallery to fill the void, created by Yan Holtz. It contains more than 200 data visualizations categorized by type, along with the R code that created them. You can browse the gallery by types of chart (boxplots, maps, histograms, interactive charts, 3-D charts, etc), or search the chart descriptions. Once you've found a chart you like, you can admire it in the gallery (and interact with it, if possible), and also find the...
This report describes changes in version 0.3 of the ‘DOM’ package for R. This version represents a major refactoring of the package code, including its user-facing API. These changes were made in order to facilitate the addition of new features … <a href="http://stattech.wordpress.fos.auckland.ac.nz/2016-13-dom-version-0-3/">Continue reading <span>→</span></a>
<p>We are happy to announce that we can finally answer the question on how to cite <em>mlr</em> properly in publications.</p>
<p>Our paper on <em>mlr</em> has been published in the open-access Journal of Machine Learning Research (JMLR) and can be downloaded on the <a href="http://www.jmlr.org/papers/v17/15-066.html">journal home page</a>.</p>
As discussed in the previous entry, there are two interpretations to this question from The Riddler: “…how long is the longest path a knight can travel on a standard 8-by-8 board without letting the path intersect itself?” as to what constitutes a path. As a (terrible) chess player, I would opt for the version on […]
Our new Financial Trading in R course is here! Learn from Ilya Kipnis, a professional quantitative analyst and co-author of Introduction to Quantitative Trading With R.  Master the basics of financial trading, and learn how to use quantstrat to build ...
R Tools for Visual Studio, the open-source Visual Studio add-in for R programmers, has a new update available for download. RTVS 0.5 makes it easier to run R within SQL Server 2016 as a stored procedure, and supports multiple windows for R graphics. RTVS 0.5 makes use of Visual Studio's polyglot nature so you can manage and edit SQL queries within R projects. You can now add a SQL query to an R project, add a database connection, and easily check the output from the query. (You can even inspect the execution plan to optimize the query.) Once you've got...
  To post your R job on the next post Just visit this link and post a new R job to the R community. You can post a job for free (and there is also a “featured job” option for extra exposure). Current R jobs Job seekers: please follow the links below to learn more and apply for your R job of interest: Featured Jobs Full-Time Data Engineer Future of Insurance – Posted by Future of Insurance New York New York, United States 17 Oct2016 Full-Time Data Scientist / Bioinformatician / Statistician at University of California UCSF – Posted by clif.duhn San Francisco California, United States 16 Oct2016 Full-Time Data Scientist – Analytics Booking.com – Posted by work_at_booking Anywhere 30 Sep2016 More New Jobs Full-Time Statistical Analyst @ Rostock, Mecklenburg-Vorpommern, Germany Max Planck Institute for Demographic Research – Posted by MPIDR Rostock Mecklenburg-Vorpommern, Germany 19 Oct2016 Full-Time Data Engineer Future of Insurance – Posted by Future of Insurance New York New York, United States 17 Oct2016 Full-Time Data Scientist / Bioinformatician / Statistician at University of California UCSF – Posted by clif.duhn San Francisco California, United States 16 Oct2016 Full-Time Data Scientist – Post-Graduate Programme @ Nottingham, England Capital One – Posted by olwen.merchant@capitalone.com Nottingham England, United Kingdom […]
By Mine Cetinkaya-Rundel Sometimes great ideas come from trying to solve simple problems. This seems to be especially true for software developers who are willing to put in an unreasonable amount of effort to solve a simple problem to their satisfaction. So the story goes that Torstein Hønsi, the founder and Chief Product Officer of […]
Descriptive Analytics is the examination of data or content, usually manually performed, to answer the question “What happened?”. This is the first set of exercise of a series of exercises that aims to provide a descriptive analytics solution to the ‘2008’ data set from here. Download it and save it as a csv file. This […]
Last Friday together with <a target="_blank" href="https://www.linkedin.com/in/przemys%C5%82aw-szufel-720a4411b">Przemysław Szufel</a> and <a target="_blank" href="https://www.linkedin.com/in/jakuczunwit">Wit Jakuczun</a> we were giving a live demo on introduction to deep learning at <a target="_blank" href="http://www.digitalchampions.pl/">Digital Champions</a> conference.<br><br>The objective of the workshop was to show how to build a simple predictive model using MXNet library in a few minutes. In the example, we train a model on GPUs using a standard <span>MNIST database of handwritten digits.</span><br><span><br></span><span>Our post-presentation thoughts are that many people did not have experience with using GPUs in the cloud. Our point is that it is actually simpler and cheaper than having the infrastructure on-premises. Running of the whole tutorial cost us under 1$. Using </span>bitfusion.io <a target="_blank" href="https://aws.amazon.com/marketplace/seller-profile?id=3b372560-86bf-4e3d-9ec0-016892a64bed">Scientific Computing AMI</a> we were able to painlessly get the whole computing environment running on AWS in a few minutes.<br><span><br></span><span>On <a target="_blank" href="https://github.com/WLOGSolutions/digital_champions_deeplearning_r_mxnet">GitHub</a> you can find the instructions how to set up computations in the cloud and the R source code allowing to build the model using MXNnet.</span>
Introduction Many people know that nasty feeling when buying a brand new car. The minute that you have left the dealer, your car has lost a substantial amount of value. Unfortunately this depreciation is inevitable, however, the amount depends heavily … <a href="https://longhowlam.wordpress.com/2016/10/19/dont-buy-a-brand-new-porsche-911-or-audi-q7/">Continue reading <span>→</span></a>
With the release of Plotly 4.0 using the pipe %>% operator is a lot more intuitive when using plot_ly(). Quick Introduction For those new to the pipe operator from the magrittr package here’s a quick introduction. In essence, the pipe operator takes the argument on the left hand side of the operator and inserts it […]
Following a request from one of the reviewers of our chapter Likelihood-free model choice, I tried to run EP-ABC on a toy problem and to compare it with the outcome of a random forest ABC. Literally starting from scratch, namely from the description found in Simon and Nicolas’ JASA paper.  To run my test, I […]
Clara Schartner As part of my summer internship at Mango I got to help out creating training materials. Learning new methods by preparing them in a teachable way is definitely my new favourite way of learning! But once I started … <a href="http://www.mango-solutions.com/wp/2016/10/how-long-do-i-have-to-survive-without-cake/">Continue reading <span>→</span></a>
<p>This is the first chapter of our new web book, Raccoon - Statistical Models with R: it's an introduction to Linear models, with theoratical explanation and lots of examples + two summary tables with linear model formulae and functions in R </p>
<p>The post <a rel="nofollow" href="http://www.quantide.com/raccoon-ch-1-introduction-to-linear-models-with-r/">Raccoon | Ch. 1 – Introduction to Linear Models with R</a> appeared first on <a rel="nofollow" href="http://www.quantide.com/">Quantide - R training &amp; consulting</a>.</p>
by Srini Kumar, Director of Data Science at Microsoft We tend to think of R and other such ML tools only in the context of the workplace, to do “weighty” things aimed at saving millions. A little judicious use of R may help us hugely in our personal lives too. The ideas of regression, classification trees etc. can be powerful tools in valuation, as I found out. Recently, I was in a five-car accident on the infamous 101 in the San Francisco bay area. Luckily, none of us required an ambulance and all of us walked away. However, my car...
Tourism competition data
The tourism forecasting competition described in Athanasopoulos et al (2011) was an important investigation into domain-specific time-series forecasting; a different approach from the broader-scope “M” series forecasting co...
<p>Recently a few people in my membership site have asked me the following question: How can I get good at R? This has come up enough...</p>
<p>The post <a rel="nofollow" href="http://www.arilamstein.com/blog/2016/10/18/get-good-r/">How to “get good at R”</a> appeared first on <a rel="nofollow" href="http://www.arilamstein.com/">AriLamstein.com</a>.</p>
Last week, I gave an overview of bunch of tidyverse packages (tibble, dplyr, tidyr, ggplot, readr, purrr) to the Davis R-Users’ Group. Here is that talk (and since videos don’t display everywhere this blog is syndicated, here is the YouTube link).
...
I’d like to share some tips and recommendations on building htmlwidgets, based on my own learning experience while creating timevis. These tips are mostly concerned with making your htmlwidget more useful and user friendly. In this post, ev...
We are excited to announce that submissions for lightning talks at rstudio::conf are now open! Lightning talks are short (5 minute) high energy presentations that give you the chance to talk about an interesting project that you’ve tackled with R. Short talks, or demos of your R code, R packages, and shiny apps are great […]
Release 0.0.2 of gettz is now on CRAN.
gettz provides a possible fallback in situations where Sys.timezone() fails to determine the system timezone. That can happen when e.g. the file /etc/localtime somehow is not a link into the corresponding file w...
Data Science Live Book - Scoring, Model Performance &amp; profiling - Update!
One way analysis of variance helps us understand the relationship between one continuous dependent variable and one categorical independent variable. When we have one continuous dependent variable and more than one independent categorical variable we cannot use one way ANOVA. When we have two independent categorical variable we need to use two way ANOVA. When […]
.fusion-fullwidth-1 {
padding-left: px !important;
padding-right: px !important;
}by Joseph Rickert
Welcome to “R Views”, the new R Community blog from RStudio....
As more and more organizations are setting up teams of data scientists to make sense of the massive amounts of data they collect, the need grows for a standardized process for managing the work of those teams. To help with this, the data science team at Microsoft has drawn on their experience with large-scale data science projects to develop the Team Data Science Process. The process is built around this data science lifecycle: The Team Data Science Process proposes a standardised directory structure for managing the data, code and documents for a data science project, and provides for tracking of...
For those of us who received statistical training outside of statistics departments, it often emphasized procedures over principles. This entailed that we learned about various statistical techniques and how to perform analysis in a particular statistical software, but glossed over the mechanisms and mathematical statistics underlying these practices. While that training methodology (hereby referred to […]
Microsoft has been really increasing their development of tools in the predictive analytics and machine learning space.  Another such tool they released recently is <a href="https://github.com/Microsoft/LightGBM">LightGBM</a>.  From the Github site...<br><br><blockquote><span>LightGBM is a fast, distributed, high performance gradient boosting (GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.</span></blockquote><br>Microsoft is definitely increasing their attempts to capitalize on the machine learning and big data movement.  I hope they continue to develop tools such as LightGBM and <a href="https://msdn.microsoft.com/en-us/library/mt604845.aspx">R with SQL Server</a>.
A moment ago, Rcpp hit another milestone: 800 packages on CRAN now depend on it (as measured by Depends, Imports and LinkingTo declarations). The graph is on the left depicts the growth of Rcpp usage over time.
The easiest way to compute this is to ...
David Rossell and Francisco Rubio (both from Warwick) arXived a month ago a paper on non-normal variable selection. They use two-piece error models that preserve manageable inference and allow for simple computational algorithms, but also characterise the behaviour of the resulting variable selection process under model misspecification. Interestingly, they show that the existence of asymmetries […]
To stay on top of R in the news, we’re sharing some stories related to R published last week. A great interview with JJ Allaire, creator of RStudio.(Joseph Rickert) The man who build RStudio now 13 years ago shares some insight on the company and his own motivation. Or was it a company? we are […]
Abstract
The rwml-R Github repo is updated with R code to accompany Chapter 3 of the book “Real-World Machine Learning” by Henrik Brink, Joseph W. Richards, and Mark Fetherolf.
Survivors on the Titanic
The Titanic Passengers dataset is used to i...
As a testament to my obsession with progress bars in R, here is
a quick investigation about the overhead cost of drawing a progress bar during computations in R.
I compared several approaches including
my pbapply and Hadley Wickham’s plyr.
In this post we’ll create some simple functions to generate and chart a Receiver Operator (ROC) curve and visualize it using Plotly. See Carson’s plotly book for more details around changes in syntax. We’ll do this from a credit risk perspective i.e. validating a bank’s internal rating model (we’ll create a sample dataset keeping this in […]
anytime arrived on CRAN with releases 0.0.1 and 0.0.2 about a month ago. anytime aims to convert anything in integer, numeric, character, factor, ordered, ... format to POSIXct (or Date) objects.
Release 0.0.3 brings a bugfix for Windows (where for d...
The apply() functions in R are a utilization of the Split-Apply-Combine strategy for Data Analysis, and are a faster alternative to writing loops. The sapply() function applies a function to individual values of a dataframe, and simplifies the output. Structure of the sapply() function: sapply(data, function, ...) The dataframe used for these exercises: dataset1 &lt;- […]
Converting mouse gene names to the human equivalent and vice versa is not always as straightforward as it seems, so I wrote a function to simplify the task. The function takes advantage of the getLDS() function from the biomaRt to get the hgnc symbol equivalent from the mgi symbol.
In this busy election season (here in the US, at least), we're seeing a lot of maps. Some states are red, some states are blue. But there's a problem: voters are not evenly distributed throughout the United States. In this map (the firethirtyeight.com US election forecast on October 13) Montana (MT) is a large state shaded red, but only represents 3 of the 538 Electoral College votes. In the big scheme of things, the outcome in Montana doesn't have much impact on the election. Contrast that with the much smaller state New Jersey with 14 electoral votes: a state so...
Association rules are very handy for analyzing Retail data. And WWI database has really neat set of invoices that can be used to make a primer. Starting with following T-SQL query: USE WideWorldIMportersDW; GO ;WITH PRODUCT AS ( SELECT   [Stock Item Key]  ,[WWI Stock Item ID]  ,[Stock Item]  ,LEFT([Stock Item], 8) AS L8DESC  ,ROW_NUMBER() … <a href="https://tomaztsql.wordpress.com/2016/10/14/association-rules-on-wideworldimporters-and-sql-server-r-services/">Continue reading <span>Association Rules on WideWorldImporters and SQL Server R Services</span></a>
Shiny Server 1.4.7.815 and Shiny Server Pro 1.4.7.736 are now available! This release includes new features to support Shiny 0.14. It also updates our Node.js to 0.10.47, which includes important security fixes for SSL/TLS. Connection robustness (a.k.a. grey-outs) Shiny’s architecture is built on top of websockets, which are long-lived network connections between the browser and an R […]
by Andy Nicholls When speaking with clients and other R users at events such as LondonR and EARL I’ve noticed an increasing trend in people looking to learn some python as the next step in their data science journey.  At … <a href="http://www.mango-solutions.com/wp/2016/10/eight-not-10-things-an-r-user-will-find-frustrating-when-trying-to-learn-python/">Continue reading <span>→</span></a>
Over the past couple months there have been datavizs showing global average temp trends. They have gotten a lot of media attention, so I got to see examples fairly regularly. Several of them (e.g., 1 and 2) used the same HadCRUT global temperature a...
rstudio::conf 2017, the conference on all things R and RStudio, is only 90 days away. Now is the time to claim your spot or grab one of the few remaining seats at Training Days – including the new Tidyverse workshop. REGISTER NOW Whether you’re already registered or still working on it, we’re delighted today to […]
If you'd like to learn how to run R within Azure Machine Learning and SQL Server, you may be interested in these upcoming 4-day Practical Data Science courses, presented by Rafal Lukawiecki from Project Botticelli. In this classroom-based course, you will learn machine learning, data mining, some statistics, data preparation, and how to interpret the results. You will also learn how to formulate business questions in terms of data science hypotheses and experiments, and how to prepare inputs to answer those questions. Rafal will share his decade of hands-on experience while teaching you about Azure Machine Learning (Azure ML) which...
The Riddler of this week had a riddle that is a variation of the knight tour problem, namely “…how long is the longest path a knight can travel on a standard 8-by-8 chessboard without letting the path intersect itself?” the riddle being then one of a self-avoiding random walk [kind]… As I could not get […]
The Consumer Price Index (CPI) is the main standard for tracking the inflation of the U.S. dollar. The various CPI measures are published monthly by the Bureau of Labor Statistics. For this walk-through, we will be using the blcsrapeR package to download our data from the BLS and perform the calculation. The blscrapeR package can […]
I was supplied spatial point data this summer where the locations were detailed in Ordnance Survey’s National Grid (NGR) format. If you’re not familiar with it, rather than using eastings and northings (in… <a href="https://scottishsnow.wordpress.com/2016/10/13/converting-bng/">Continue reading <span>→</span></a>
If there’s one word that could describe the default styling of Shiny applications, it might be “minimalist.” Shiny’s UI components are built using the Bootstrap web framework, and unless the appearance is customized, the application will be mostly white and light gray. Fortunately, it’s easy to add a bit of flavor to your Shiny application, […]
If you couldn't tell already, we are pretty big fans of the R language here at DataCamp. From its ability to create complex statistical models with a few lines of code to its robust graphical capabilities and stunning data visualizations - R is a...
Following my earlier post on Delyon and Portier’s proposal to replacing the true importance distribution ƒ with a leave-one-out (!) kernel estimate in the importance sampling estimator, I ran a simple one-dimensional experiment to compare the performances of the traditional method with this alternative. The true distribution is a N(0,½) with an importance proposal a N(0,1) […]
<p>Two years ago in <a href="http://dirk.eddelbuettel.com/blog/2014/10/12#github_oct2013_oct2014">this post</a> I reference the <a href="http://lifehacker.com/281626/jerry-seinfelds-productivity-secret">Seinfeld Streak</a> used in an even <a href="http://dirk.eddelbuettel.com/blog/2013/02/03/">earlier post</a> of regular updates to to the <a href="http://gallery.rcpp.org/">Rcpp Gallery</a>:</p>
<blockquote>
<p>This is sometimes called Jerry Seinfeld's secret to productivity: Just keep at it. Don't break the streak.</p>
</blockquote>
<p>and showed the this first chart of GitHub streaking</p>
<blockquote>
</blockquote>
<p>Last year a follow-up appeared in <a href="http://dirk.eddelbuettel.com/blog/2015/10/12#github_oct2014_oct2015">this post</a>:</p>
<blockquote>
</blockquote>
<p>And as it is that time again, here is this year's version:</p>
<blockquote>
</blockquote>
<p>Special thanks go to <a href="https://naramsim.github.io/">Alessandro Pezzè</a> for the Chrome add-on <a href="https://github.com/Naramsim/GithubOriginalStreak">GithubOriginalStreak</a>.</p>
<p>
This post by <a href="http://dirk.eddelbuettel.com/">Dirk Eddelbuettel</a> originated on his <a href="http://dirk.eddelbuettel.com/blog/">Thinking inside the box</a> blog. Please report excessive re-aggregation in third-party for-profit settings.
</p>
If you'd like to manipulate and analyze very large data sets with the R language, one option is to use R and Apache Spark together. R provides the simple, data-oriented language for specifying transformations and models; Spark provides the storage and computation engine to handle data much larger than R alone can handle. At the KDD 2016 conference last October, a team from Microsoft presented a tutorial on Scalable R on Spark, and made all of the materials available on Github. The materials include an 80-slide presentation covering several tutorials (you can download the 13Mb PowerPoint file here). Slides 1-29...
What’s new
A new version (0.5.1) of my ggseas R package is now available on CRAN. ggseas is a small package that provides several tools to make it easier to do seasonal adjustment or decomposition of time series on the fly in a ggplot2 pipeline. N...
In my previous blog post, I was focusing on data manipulation tasks with RevoScaleR Package in comparison to other data manipulation packages and at the end conclusions were obvious; RevoScaleR can not (without the help of dplyrXdf) do piping (or chaining) and storing temporary results take time and on top of that, data manipulation can … <a href="https://tomaztsql.wordpress.com/2016/10/12/performance-comparison-between-kmeans-and-revoscaler-rxkmeans/">Continue reading <span>Performance comparison between kmeans and RevoScaleR rxKmeans</span></a>
Due to anthropogenic climate change, the average global temperature has increased steadily over the past decade or so. While we're all familiar with the hockey-stick line chart of rising temperature, the change is even more dramatic on this animated globe showing the local effects of climate change. The first half of the animation shows the monthly local change compared to historic averages (blue is cooler; red is warmer). The second half of the animation repeats the cycle, but introduces a moving 10-year smoother to reduce the variability in the temperature changes, making the global temperature increase much more apparent in...
Every time I download subtitles for a movie or a series episode, I cannot help thinking about all this text that could be analyzed with text mining methods. As my PhD came to its end in April and I started a postdoc in September, I could use some time of my looong summer break to work on a new R package, subtools which aims to provide a toolbox to read, manipulate and write subtitles files in R. In this post...<p><a href="http://www.pieceofk.fr/?p=437">Lire la suite<span> Lire la suite</span></a></p>
I recently published a technical note in Fisheries Research that described an algorithm for implementing the “seasonal cessation in growth model” of @Paulyetal1992. Describing an algorithm for a growth model does not seem like a useful...
We are delighted to announce the full agenda for the EARL2016 Conference which takes place at the fabulous Museum of Science, Boston  on 7-9th November. The conference will feature : 3 pre-conference workshops 2 Keynote Presentations 21 Presentations from leading R and industry practitioners 1 Evening … <a href="http://www.mango-solutions.com/wp/2016/10/earl-boston-full-agenda-released/">Continue reading <span>→</span></a>
<p>MilanoR is a free event, open to all R users and enthusiasts or those who wish to learn more about R. The meeting consists of two talks (this time one about big data and and one about statistical learning) + a free buffet and networking time</p>
<p>The post <a rel="nofollow" href="http://www.milanor.net/blog/7th-milanor-meeting-october-27/">7th MilanoR Meeting: October 27</a> appeared first on <a rel="nofollow" href="http://www.milanor.net/blog">MilanoR</a>.</p>
The 19th Cologne R user group meeting is scheduled for this Friday, 14 October 2016. We have three talks, followed by networking drinks. Introduction to the tidyverse tools - Jiddu AlexanderPerformance profiling and improvement in R - Nils GlückBatch ...
Tomorrow the European R users meeting (eRum) 2016 starts in Poznan, Poland. About 250 participants from business and science are expected at the three-day conference that aims at connecting users of the R language. As R specialist and silver sponsor of the event, eoda will be attending with a team of data scientists and participating … <a href="http://blog.eoda.de/2016/10/11/r-heroes-unite-the-starting-gun-for-the-erum-2016-will-be-fired-tomorrow/"><span>„R heroes unite – The starting gun for the eRum 2016 will be fired tomorrow“ </span>weiterlesen</a>
Introduction to mapmate mapmate (map animate) is an R package for map animation. It is used to generate and save a sequence of plots to disk as a still image sequence intended for later use in data animation production. The mapmate package is used for map- and globe-based data animation pre-production. Specifically, mapmate functions are […]
A key question of environmental economics is to economically quantify damages from different types of pollution and benefits from environmental policy that attempts to reduce pollution. Often only indirect measures are available. An important measure i...
Joe Russell If you were to ask any R-user for the reason for R’s success, you’re almost guaranteed to hear the words “open source”.  As the second most popular open source language (behind Python) R has exploded in popularity in … <a href="http://www.mango-solutions.com/wp/2016/10/rs-testing-predicament/">Continue reading <span>→</span></a>
We all know R is the first choice for statistical analysis and data visualisation, but what about big data munging? tidyverse (or we’d better say hadleyverse) has been doing a lot in this field, nevertheless it is often the case this kind of activities being handled from some other coding language. Moreover, sometimes you get […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toR Programming – Pitfalls to avoid (Part 1)" rel="bookmark" href="http://datascienceplus.com/r-programming-pitfalls-to-avoid-part-1/">R Programming – Pitfalls to avoid (Part 1)</a></li><li><a title="Permanent Link toEclipse – an alternative to RStudio – part 2" rel="bookmark" href="http://datascienceplus.com/eclipse-an-alternative-to-rstudio-part-2/">Eclipse – an alternative to RStudio – part 2</a></li><li><a title="Permanent Link toEclipse – an alternative to RStudio – part 1" rel="bookmark" href="http://datascienceplus.com/eclipse-an-alternative-to-rstudio-part-1/">Eclipse – an alternative to RStudio – part 1</a></li><li><a title="Permanent Link toHow to use R for matching samples (propensity score)" rel="bookmark" href="http://datascienceplus.com/how-to-use-r-for-matching-samples-propensity-score/">How to use R for matching samples (propensity score)</a></li><li><a title="Permanent Link toR for Publication by Page Piccinini: Lesson 1 – R Basics" rel="bookmark" href="http://datascienceplus.com/r-for-publication-by-page-piccinini-lesson-1-r-basics/">R for Publication by Page Piccinini: Lesson 1 – R Basics</a></li></ol>
This tutorial was first published in “Geocomputation a Practical Primer“. Here is a more complex example showing how to produce a map of 18th Century Shipping flows. The data have been obtained from the CLIWOC project and they represent a sample of digitised ships’ logs from the 18th Century. We are using a very small […]
R's ggplot2 package is a well-known tool for producing beautiful static data visualizations that you can include in a printed report. But what if you want to include a ggplot2 graphic on a webpage and provide the ability for the user to interact with the data? The ggiraph package by David Gohel (available for installation via CRAN). WIth ggiraph, you can take an existing ggplot2 bar chart, scatterplot, boxplot, map, or many other types of chart and add one or both of the following iteractions: Display a tooltip of your choice (e.g. data values or labels) when the cursor hovers...
Great news R pirates! The yarrr package, which contains the pirateplot, has now been updated to version 0.0.8 and is up on CRAN (after hiding in plain sight on GitHub). Let’s install the latest version (0.0.8) and go over some of the updates: The most important function in the yarrr package is pirateplot(). What the heck ... <span><a href="http://nathanieldphillips.com/2016/10/the-yarrr-package-is-now-on-cran/">[Read more...]</a></span>
To stay on top of R in the news, we’re sharing some stories related to R published last week. Why Data Science ‘Rock Stars’ Earn Big Salaries (Dennis McCafferty) Recent post and slide deck related to the 2016 Data Science Salary Survey (O’Reilly Media), with R mentioned as one of the high-demand programming languages (next […]
Long I wanted to test a simple data manipulation tasks and compare the execution time, ease of writing the code and simplicity between T-SQL and R package for data manipulation. Couple of packages I will mention for data manipulations are plyr, dplyr and data.table and compare the execution time, simplicity and ease of writing with … <a href="https://tomaztsql.wordpress.com/2016/10/10/comparing-performance-on-dplyr-package-revoscaler-package-and-t-sql-on-simple-data-manipulation-tasks/">Continue reading <span>Comparing performance on dplyr package, RevoScaleR package and T-SQL on simple data manipulation tasks</span></a>
Purpose I recently completed a set of data science for biostatistics training exercises for graduate students. I extensively used R for Data Science and Efficient R programming to develop a set of Adventure Time R-statistics slide decks. Whilst I recognize that they are very minimal in terms of text, I hope that the general visual flow can […]
Getting introduced to Google and their API
Previously we constructed a data frame that consisted out of the following variables that we might want to perform analysis on:
## [1] &quot;wine_farms&quot; &quot;wine_year&quot; &quot;wine_name&quot; &quot;points&quot; &quot;price&quot; ## ...
Hey there! We’ve been sharing R exercise sets for about a year, and think this is a good moment to reflect and ask for your feedback. So here is your opportunity to have a say in where we take R-exercises next! We’d like to hone in a bit on the degree of difficulty of the […]
It’s always nice when 12-month old code runs without a hitch. Not sure why this did not become a Github repo first time around, but now it is: my RMarkdown code to generate a report using data from the Nobel Prize API. Now you too can generate a “gee, it’s all old white men” chart […]
I have released a new version of my “pretty quick” R interpreter, pqR-2016-10-05. One major change with this version is that pqR, which was based on R-2.15.0, is now compatible with R-2.15.1.  This allows for an increased number of packages in the pqR repository. This release also has some significant speed improvements, a new form of the “for” statement, for […]
Extending accessibility of open-source statistical software to the masses: A shiny case study
Brandon LeBeau
University of Iowa
R
R is an open source statistical programming language.
Pros:
Common statistical procedures are found...
Recently Microsoft Data Scientist Bob Horton wrote a very nice article on ROC plots. We expand on this a bit and discuss some of the issues in computing “area under the curve” (AUC). R has a number of ROC/AUC packages; for example ROCR, pROC, and plotROC. But it is instructive to see how ROC plots … <a href="http://www.win-vector.com/blog/2016/10/on-calculating-auc/">Continue reading <span>On calculating AUC</span></a>
The tint package is now on CRAN. Its name stands for Tint Is Not Tufte and it offers a fresh take on the excellent Tufte-style html (and now also pdf) presentations.
As a little teaser, here is what the html variant looks like:
and the full under...
In this post, I will talk about the presentation of R that I did today, in the first week of my PhD. Usually, it is a team-only presentation. Yet, other people came because they were interested in learning more about R.
How I get this idea?
I get the idea of doing an R presentation while reading Getting Your Colleagues Hooked on R on R-bloggers. I began by following the 7 tips of this post to make my presentation, which was a good starting point.
After a while, I feared that a general presentation would not get my team interested in R. So, I decided to set up a google form and ask them what they wanted to learn about R. It was the way to get sure that they would care.
Get results automatically
Because I was writing my R Markdown presentation while they were answering the google form, I decided that I should get (and show) the results automatically (only by re-knitting my presentation).
To get the results
I used the gsheet package (one could also use the googlesheets package):
library(pacman)
p_load(magrittr, longurl, gsheet)
In case you missed them, here are some articles from September of particular interest to R users. The R-Ladies meetups and the Women in R Taskforce support gender diversity in the R community. Highlights from the Microsoft Data Science Summit include recordings of many presentations about R, and the keynote &quot;The Future of Data Analysis&quot; by Edward Tufte. An R-based fraud detection model scores credit card transactions in SQL Server at a rate of 1 million records per second. The Financial Times uses R for quantitative journalism (and made some lovely animations comparing European football teams). Part 3 in a...
Creating sample data is a common task performed in many different scenarios. R has several base functions that make the sampling process quite easy and fast. Below is an explanation of the main functions used in the current set of exercices: 1. set.seed() – Although R executes a random mechanism of sample creation, set.seed() function […]
<p>Raccoon is a free web-book about Statistical Models with R. Raccoon is the collection of twenty years of notes, exercises and concepts working with statistics and R, and it is part of our web-book project, together with Rabbit: Introduction to R and Ramarro: R for Developers.</p>
<p>The post <a rel="nofollow" href="http://www.quantide.com/raccoon-statistical-models-r-free-web-book/">Raccoon: Statistical Models with R free web-book</a> appeared first on <a rel="nofollow" href="http://www.quantide.com/">Quantide - R training &amp; consulting</a>.</p>
Plentiful high-quality data is the key to great machine learning models. But good data doesn’t grow on trees, and that …<p><a href="https://cartesianfaith.com/2016/10/06/what-you-need-to-know-about-data-augmentation-for-machine-learning/">Continue reading <span>→</span></a></p>
<p>Whether you are an environmental scientist, a pollution expert or just concerned about the air you breathe when cycling in the United Kingdom, the ropensci <a href="https://cran.r-project.org/package=rdefra">rdefra</a> package can help find the information you need. This package gives you access to the UK-AIR database, hosted by the Department for Environment, Food &amp; Rural Affairs in the United Kingdom, directly from R. The database comprises hundreds of air quality monitoring sites and each provides time series of concentration for different pollutants such as ozone, carbon dioxide and nitrogen oxides amongst others. It allows you to efficiently download, plot and compare data for multiple years/stations so that you can easily carry out spatio-temporal analysis or simply plan an healthier route for you bike ride. The package is fully documented and the <a href="https://github.com/ropenscilabs/rdefra">vignette</a> contains examples for a number of applications, such as how to generate the interactive map of the monitoring stations (see screenshot below).</p>
<p><a href="https://github.com/ropenscilabs/rdefra"></a></p>
<h1>Installation</h1>
<p>The stable version of the package is available from CRAN:</p>
<div><pre><code>install.packages<span>(</span><span>&quot;rdefra&quot;</span><span>)</span>
</code></pre></div>
<p>While the development version can be installed from github using <a href="https://github.com/hadley/devtools">devtools</a>:</p>
<div><pre><code>devtools<span>::</span>install_github<span>(</span><span>&quot;ropensci/rdefra&quot;</span><span>)</span>
</code></pre></div>
<p>This package depends on the Geospatial Data Abstraction Library (GDAL) library, which should be previously installed on your machine.</p>
Regardless of the tool you use to analyse data, you'll often have to access data living in file formats generated by other tools. The &quot;haven&quot; package from RStudio allows you to import and export data in SAS, SPSS and Stata formats. Version 1.0 was released on October 4, and is now available on CRAN. Haven is also installed as part of the tidyverse. Haven augments the base R foreign package with additional formats. The core read/write engine is the ReadStat package, and it provides support for: SAS binary files (SAS7BDAT), including compressed files SPSS .sav and .por files Stata files...
Part of the R course offering of BNOSAC which you can find at http://bnosac.be/images/activities/bnosac_courses_r.pdf, we offer several 2-day hands-on courses covering the use of text mining tools for the purpose of data analysis. It covers basic text ...
  To post your R job on the next post Just visit this link and post a new R job to the R community. You can either post a job for free (which works great), or pay $50 to have your job featured (and get extra exposure). Current R jobs Job seekers: please follow the links below to learn more and apply for your R job of interest: Featured Jobs Full-Time Data Scientist – Analytics Booking.com – Posted by work_at_booking Anywhere 30 Sep2016 Full-Time Data AnalystThe Freshwater Trust – Posted by The Freshwater Trust Portland Oregon, United States 26 Sep2016 Full-Time Data Manager II, CRP Boston Children’s Hospital – Posted by katiekerr7 Boston Massachusetts, United States 14 Sep2016 Full-Time Research and Analytics Associate Hodges Ward Elliott – Posted by tkiely@hwerealestate.com New York New York, United States 13 Sep2016 Full-Time Data Mining Analyst Fellow – fixed term 1 year Boston Scientific – Posted by glynburtt Hemel Hempstead England, United Kingdom 5 Sep2016 More New Jobs Full-Time Health Data Scientist @ Boston, Massachusetts, U.S. Harvard T.H. Chan School of Public Health – Posted byChristine Choirat Boston Massachusetts, United States 4 Oct2016 Full-Time Data Science Consultant @ Notre Dame, Indiana, United States Notre Dame Center for Social Research – Posted bykate.mueller […]
English summary: On 2016-09-28 I presented an application of methods of estimating Exponential-family Random Graph Models from ego-centrically sampled data. Conference was in Polish. See the program and my slides for further details. W zeszłym tygodniu (28-30.09.2016) miała miejsce konferencja Metodologiczne Inspiracje 2016. Badania ilościowe w naukach społecznych: wyzwania i problemy organizowana przez Instytut Filozofii […]
You want to get started on text mining, but most of the tutorials you start, get pretty complex very quickly? Or you can’t find a proper data set to work on?
DataCamp’s latest post will walk you through 8 tips and tricks that will help you to star...
The results of many scientific papers are wrong. There are many reasons for this, including p-hacking, publication bias, and the general inability to replicate results. But there's another, more mundane cause: incorrect calculation of p-values in statistical tests. This could be caused by simple transcription errors when plugging numbers into a statistical tool, incorrect rounding, or misapplication of the test itself (say, applying a two-sided test when a 1-sided p-value is appropriate). Such errors should be picked up in the peer review process, but given that even expert statisicians sometimes struggle to explain p-values, it's not surprising that some errors...
Today we’re excited to announce R Notebooks, which add a powerful notebook authoring engine to R Markdown. Notebook interfaces for data analysis have compelling advantages including the close association of code and output and the ability to intersperse narrative with computation. Notebooks are also an excellent tool for teaching and a convenient way to share […]
<p>The new ropensci <a href="https://cran.r-project.org/web/packages/graphql/">graphql</a> package is now on CRAN. It implements R bindings to the <a href="https://github.com/graphql/libgraphqlparser">libgraphqlparser C++ library</a> to parse GraphQL syntax and export the syntax tree in JSON format:</p>
<div><pre><code>graphql2json<span>(</span><span>&quot;{ field(complex: { a: { b: [ $var ] } }) }&quot;</span><span>)</span>
</code></pre></div>
<p>A syntax parser is perhaps not super useful to most end-users, but can be used to validate graphql queries or implement a GraphQL API in R. We hope to add more related functionality later on.</p>
<h2>What is GraphQL</h2>
<p>From the <a href="http://graphql.org/">GraphQL website</a>:</p>
<blockquote>
<p>GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.</p>
</blockquote>
<p>GraphQL is a new approach and syntax to structuring API queries. It provides a modern, and perhaps more natural alternative to e.g. REST or SQL (which were designed for hypertext and relational data, respectively). A GraphQL query on the other hand resembles the structure of the data. For example the query:</p>
<div><pre><code>{
  me {
    name
  }
}
</code></pre></div>
<p>Could produce the JSON result:</p>
<div><pre><code><span>{</span>
  <span>&quot;me&quot;</span><span>:</span> <span>{</span>
    <span>&quot;name&quot;</span><span>:</span> <span>&quot;Luke Skywalker&quot;</span>
  <span>}</span>
<span>}</span>
</code></pre></div>
<p>The Github API <a href="https://developer.github.com/early-access/graphql/">supports GraphQL</a> and illustrates how GraphQL reduces the number of requests and data in comparison with traditional REST queries.</p>
In the last few years, I’ve been using ggplot2 quite a lot. While the
ggplot2 default theme is quite slick, I started tweaking the theme to
fit my own taste and/or that of the target audience.
artyfarty is a collection of a few themes that I have us...
I was never that much into JavaScript until I was introduced to D3.js. This open source JS library provides the features for dynamic data manipulation and visualization and allows users to become active participants in data visualization process. As such, … <a href="http://www.vesnam.com/Rblog/viznets6/">Continue reading <span>→</span></a>
A new version of RcppGSL in now on CRAN and in Debian. The RcppGSL package provides an interface from R to the GNU GSL using our Rcpp package.
This release is one that entirely focused on maintenance. CRAN asked us to change on aspect relative to vig...
This post will be about attempting to use the Depmix package for online state prediction. While the depmix package performs … <span><a href="https://quantstrattrader.wordpress.com/2016/10/05/the-problem-with-depmix-with-online-prediction/">Continue reading <span>→</span></a></span>
I’m pleased to announce the release of haven. Haven is designed to faciliate the transfer of data between R and SAS, SPSS, and Stata. It makes it easy to read SAS, SPSS, and Stata file formats in to R data frames, and makes it easy to save your R data frames in to SAS, SPSS, […]
When we teach “R for statistics” to groups of scientists (who tend to be quite well informed in statistics, and just need a bit of help with R) we take the time to re-work some tests of model quality with the appropriate significance tests. We organize the lesson in terms of a larger and more … <a href="http://www.win-vector.com/blog/2016/10/adding-polished-significance-summaries-to-papers-using-r/">Continue reading <span>Adding polished significance summaries to papers using R</span></a>
Welcome to the third and last part of the “lubridate” exercises. If you missed Part 1 and 2 then please refer to the links below: Part 1 Part 2 In this part, I’ll cover the following topics: 1. Durations (exact spans of time) 2. Periods (relative spans of time) 3. Rounding dates As always, in […]
by Udayan Kumar, Data Scientist at Microsoft Beginning in 2016, Microsoft rolled out a preview of Microsoft R Server (MRS) for Azure HDInsight clusters. This service provides a preconfigured instance of R server with Spark/Hadoop that can be provisioned within minutes. Recent blog posts (by Max Kaznady and David Smith) have highlighted how to use and tune this service for large scale machine learning tasks. In this post, we push the envelope and show how to build an end-to-end fully operationalized analytics pipeline using Azure Data Factory (ADF) and MRS with HDInsight (specifically Apache Spark). Herein, we provide a walk-through...
Guest post by Jonathan Sidi, Metrum Research Group Overview D3js is a great tool to visualize complex data in a dynamic way. But how can the visualization be part of the natural workflow? Creating new reactive elements through the integration of Shiny with d3js objects allows us to solve this problem. Through Shiny we let the server observe the d3 collapsible tree library and its real-time layout. The data transferred back to Shiny can be mapped to a series of logial expressions to create reactive filters. This allows for complex data structures, such as heirarchal simulations, complex design of clinical trials and results from polycompartmental structural models to be visually represented and filtered in a reactive manner through an intuitive and simple tool. Examples Running the App through Github #check to see if libraries need to be installed libs=c(&quot;rstan&quot;,&quot;shiny&quot;,&quot;shinyAce&quot;,&quot;reshape2&quot;,&quot;stringr&quot;,&quot;DT&quot;,&quot;plyr&quot;,&quot;dplyr&quot;) x=sapply(libs,function(x)if(!require(x,character.only = T,warn.conflicts = F,quietly = T)) install.packages(x));rm(x,libs) #run App #shiny::runGitHub(&quot;metrumresearchgroup/SearchTree&quot;) Titanic Let’s start with a popular data.frame, the Titanic data The structure of the data consists of 4 levels (Class, Sex, Age, Survived) To slice the data we can set up a number of filters on it and slice it many to many outcomes. But this is iterative and can get […]
<p>When performing some kind of analysis in R it is usually common to use data stored in basic formats such as .csv or .Rdata, and then present the analysis using a .Rmd file. Linpe package allows to send a single file instead of two. It has been created by Andrea Spanò, managing partner at Quantide.</p>
<p>The post <a rel="nofollow" href="http://www.quantide.com/linpe-make-sending-receiving-data-analysis-faster-easier/">Linpe: make sending and receiving data analysis faster and easier</a> appeared first on <a rel="nofollow" href="http://www.quantide.com/">Quantide - R training &amp; consulting</a>.</p>
Lay your dreams, little darling, in a flower bed; let that sunshine in your hair (Where the skies are blue, The Lumineers) I discovered this nice visualization some days ago. The author is also the creator of Highcharter, an incredible R wrapper for Highcharts javascript libray and its modules. I am a big fan of him. … <a href="https://fronkonstin.com/2016/10/04/visualizing-the-daily-variability-of-bitcoin-with-quandl-and-highcharts/">Continue reading <span>Visualizing the Daily Variability of Bitcoin with Quandl and Highcharts</span> <span>→</span></a>
The european R users meeting [eRum] is going to start in just 7 days. We expect over 250 participants, 10 invited talks, 47 regular talks, 13 lightning talks and 12 posters. In order to handle that much content we scheduled 18 sessions [+ workshops]. Find the program of the conference here or here. In the … <a href="http://smarterpoland.pl/index.php/2016/10/program-of-the-european-r-users-meeting-only-7-days-to-go/">Czytaj dalej <span>Program of the european R users meeting [only 7 days to go]</span></a>
On Wednesday, Mathias Drton and I will be presenting a read paper on Bayesian model choice for singular models at the Royal Statistical Society in London. You can read more about it on the RSS web site , where you … <a href="https://martynplummer.wordpress.com/2016/10/03/a-bayesian-information-criterion-for-singular-models/">Continue reading <span>→</span></a>
It's been a long time since I watched the The Simpsons, but I was always under the impression that Bart was the primary character. Perhaps it was all the Do the Bartman and &quot;Cowabunga!&quot; nonsense from the 90s. Anyway, data scientist Todd W Schneider used R to analyze the scripts of the first 26 seasons and found that Homer speaks twice as much as next most represented character, Marge. Bart comes a close third. Marge and Lisa are represented in orange (the color of Lisa's dress, in fact) as the only 2 female characters that make the top 10. Female...
A week ago, I received a request for refereeing a paper for the Journal of Open Source Software, which I have never seen (or heard of) before. The concept is quite interesting with a scope much broader than statistical computing (as I do not know anyone in the board and no-one there seems affiliated with […]
A significant chunk of the data that we encounter on a daily basis is available in an unstructured, free text format. Hence, the ability to glean useful bits of information from this unstructured pile can be quite valuable. In this post, we will attempt a basic analysis of the text from the first Presidential debate […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toGoodReads: Machine Learning (Part 3)" rel="bookmark" href="http://datascienceplus.com/goodreads-machine-learning-part-3/">GoodReads: Machine Learning (Part 3)</a></li><li><a title="Permanent Link toMachine Learning for Drug Adverse Event Discovery" rel="bookmark" href="http://datascienceplus.com/machine-learning-for-drug-adverse-event-discovery/">Machine Learning for Drug Adverse Event Discovery</a></li><li><a title="Permanent Link toGoodReads: Exploratory data analysis and sentiment analysis (Part 2)" rel="bookmark" href="http://datascienceplus.com/exploratory-data-analysis-and-sentiment-analysis/">GoodReads: Exploratory data analysis and sentiment analysis (Part 2)</a></li><li><a title="Permanent Link toGoodReads: Webscraping and Text Analysis with R (Part 1)" rel="bookmark" href="http://datascienceplus.com/goodreads-webscraping-and-text-analysis-with-r-part-1/">GoodReads: Webscraping and Text Analysis with R (Part 1)</a></li><li><a title="Permanent Link toEuro 2016 analytics: Who’s playing the toughest game?" rel="bookmark" href="http://datascienceplus.com/euro-2016-analytics-whos-playing-the-toughest-game/">Euro 2016 analytics: Who’s playing the toughest game?</a></li></ol>
What is R? I was asked at the end of my presentation on the 10th Cracow R Users Meetup that was held last Friday (30.09.2016). I felt strange but absolutely confirmed that R is the language of Data Science and is designed to performed the statistical ...
(Guest post by ODSC West Team) These are good days to be an R programmer. There’s been a slew of big players announcing R integration including IBM and Microsoft.  These and similar announcements are accelerating R’s move out of the lab and into the enterprise.  Couple this development with the fact that demand for data scientists is still white hot especially for with programming skills in languages like R. The Open Data Science Conference (ODSC) in Santa Clara on November, 4th – 6th is there to help you accelerate your R learning by covering a lot of ground on a lot topics quickly.  This year’s ODSC conference includes more than 90 talks and 50 workshops on a wide range of data science tools, topics and languages, including R. Here are some great reasons why thousands of R enthusiasts, just like you, will be attending ODSC West 2016. ODSC conferences are well known for their high-level speakers. Presenters are hand-picked and only the best and brightest in the data science field make the cut. That’s why we invited two of the top R experts this year to train you. Jared Lander, Author of R for Everyone, will be giving 2 premium training sessions: Modeling &amp; […]
Recently saw a really fun article making the rounds: The prevalence of statistical reporting errors in psychology (1985–2013) Nuijten, M.B., Hartgerink, C.H.J., van Assen, M.A.L.M. et al. Behav Res (2015). doi:10.3758/s13428-015-0664-2. The authors built an R package to check psychology papers for statistical errors. Please read on for how that is possible, some tools, and … <a href="http://www.win-vector.com/blog/2016/10/proofing-statistics-in-papers/">Continue reading <span>Proofing statistics in papers</span></a>
In this set of exercises we shall practice the functions for network statistics, using package igraph.If you don’t have package already installed, install it using the following code: install.packages(&quot;igraph&quot;) and load it into the session using the following code: library(&quot;igraph&quot;) before proceeding. You can find more info about the package and graphs in general here […]
Here is a representation of the precision of a kernel density estimate (second axis) against the true value of the density (first axis), which looks like a lasso of sorts, hence the title. I am not sure this tells much, except that the estimated values are close to the true values and that a given […]
An new version 0.0.8 of RcppAnnoy, our Rcpp-based R integration of the nifty Annoy library by Erik, is now on CRAN. Annoy is a small, fast, and lightweight C++ template header library for approximate nearest neighbours.
This release pulls in a few su...
Abstract
Introduces my Github repo providing R code to accompany the book “Real-World Machine Learning”.
Introducing rwml-R
The book “Real-World Machine Learning” attempts to prepare the reader for the realities of machine learning. ...
A month ago we launched R course finder, an online directory that helps you to find the right R course quickly. With so many R courses available online, we thought it was a good idea to offer a tool that helps people to compare these courses, before they decide where to spend their valuable time […]
I’m planning to release ggplot2 2.2.0 in early November. In preparation, I’d like to announce that a release candidate is now available: version 2.1.0.9001. Please try it out, and file an issue on GitHub if you discover any problems. I hope we can find and fix any major issues before the official release. Install the pre-release version […]
Two groups are making and impact in improving the gender diversity of R users worldwide. The R-Ladies organization is creating chapters worldwide to facilitate female R programmers meeting and working together, and the Taskforce on Women in the R Community is working to improve the participation and experience of women in the R community. R has more participation by women than many programming communities, but there's still a long way to go towards equity: the R Foundation's Women in R Taskforce estimates that between 11% and 15% of R package authors are women. (The count is based on package author...
In the first installment of this series, we scraped reviews from Goodreads. In the second one, we performed exploratory data analysis and created new variables. We are now ready for the “main dish”: machine learning! Setup and general data prep Let’s start by loading the libraries and our dataset. library(data.table) library(dplyr) library(caret) library(RTextTools) library(xgboost) library(ROCR) […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toMachine Learning for Drug Adverse Event Discovery" rel="bookmark" href="http://datascienceplus.com/machine-learning-for-drug-adverse-event-discovery/">Machine Learning for Drug Adverse Event Discovery</a></li><li><a title="Permanent Link toGoodReads: Exploratory data analysis and sentiment analysis (Part 2)" rel="bookmark" href="http://datascienceplus.com/exploratory-data-analysis-and-sentiment-analysis/">GoodReads: Exploratory data analysis and sentiment analysis (Part 2)</a></li><li><a title="Permanent Link toGoodReads: Webscraping and Text Analysis with R (Part 1)" rel="bookmark" href="http://datascienceplus.com/goodreads-webscraping-and-text-analysis-with-r-part-1/">GoodReads: Webscraping and Text Analysis with R (Part 1)</a></li><li><a title="Permanent Link toEuro 2016 analytics: Who’s playing the toughest game?" rel="bookmark" href="http://datascienceplus.com/euro-2016-analytics-whos-playing-the-toughest-game/">Euro 2016 analytics: Who’s playing the toughest game?</a></li><li><a title="Permanent Link toIntegrating R with Apache Hadoop" rel="bookmark" href="http://datascienceplus.com/integrating-r-with-apache-hadoop/">Integrating R with Apache Hadoop</a></li></ol>
When we are interested in finding if there is a statistical difference in the mean of two groups we use the t test. When we have more than two groups we cannot use the t test, instead we have to use analysis of variance (ANOVA). In one way ANOVA we have one continuous dependent variable […]
H2O has recently released its steam AI engine, a fully open source engine that support the management and deployment of machine learning models. Both H2O on R and H2O steam are easy to set up and use. And both complement each … <a href="https://longhowlam.wordpress.com/2016/09/30/danger-caution-h2o-steam-is-very-hot/">Continue reading <span>→</span></a>
My last post about <a href="http://rsnippets.blogspot.com/2016/09/telco-churn-prediction-with-rh2o.html">telco churn prediction with R+H2O</a> attracted unexpectedly high response. It seems that R+H2O combo has currently a very good momentum :). Therefore <a href="https://www.linkedin.com/in/jakuczunwit">Wit Jakuczun</a> decided to publish a case study that he uses in his <i>R boot camps</i> that is based on the same technology stack.<br><br>You can find the shortened version of the case study material along with the source codes and data sets on <a href="https://github.com/WLOGSolutions/retail-segmentation-based-marketing-campaing-in-r-and-h2o/">GitHub</a>.<br><br>I think it is nice not only because it is another example how to use H2O in R, but it is also a basic introduction to how to combine segmentation and prediction modeling for marketing campaign targeting.
I just got back from Atlanta, the host of the Microsoft Machine Learning and Data Science Summit. This was the first year for this new conference, and it was a blast: the energy from the 1,000 attendees was palpable. I covered Joseph Sirosh's keynote presentation yesterday, but today I wanted to highlight a few other talks from the program now that the recordings are available to stream. On deep learning and machine intelligence: CNTK: Microsoft’s Open Source Deep Learning Toolkit, by Frank Seide (CNTK architect), on the engine behind many of Microsoft's AI and predictive services Cognitive Services: Making AI...
As a follow-up to my Primer On Universal Function Approximation with Deep Learning, I’ve created a project on Github that …<p><a href="https://cartesianfaith.com/2016/09/29/a-simple-workflow-for-deep-learning/">Continue reading <span>→</span></a></p>
A new version of the RcppCNPy package arrived on CRAN a few days ago.
RcppCNPy provides R with read and write access to NumPy files thanks to the cnpy library by Carl Rogers.
This new release reflects all the suggestions and comments I received durin...
A pure maintenance release 0.2.6 of the gcbd package is now on CRAN. The gcbd proposes a benchmarking framework for LAPACK and BLAS operations (as the library can exchanged in a plug-and-play sense on suitable OSs) and records result in local databas...
Warsaw R and Data Analytics Enthusiast group is an effort that aims at integrating users of the R language in Warsaw, Poland. Our group has over 970 members at its meetup page. In this post I provide a summary of our group, our last 2 meetups before t...
In Joseph Sirosh's keynote presentation at the Data Science Summit on Monday, Wee Hyong Took demonstrated using R in SQL Server 2016 to detect fraud in real-time credit card transactions at a rate of 1 million transactions per second. The demo (which starts at the 17:00 minute mark) used a gradient-boosted tree model to predict the probability of a credit card transaction being fraudulent, based on attributes like the charge amount and the country of origin. Then, a stored procedure in SQL Server 2016 was used to score transactions streaming into the database at a rate of 3.6 billion per...
<p>Organize your data manipulation tasks in a standard way, write clean and efficient code, and build reproducible data management processes, using the most modern R tools: tidyr, dplyr and lubridate.</p>
<p>The post <a rel="nofollow" href="http://www.milanor.net/blog/efficient-data-manipulation-r/">Efficient Data Manipulation with R Course | Milan</a> appeared first on <a rel="nofollow" href="http://www.milanor.net/blog">MilanoR</a>.</p>
This year’s conference was a great opportunity to catch up with friends and meet new people who are eager to talk about their work with R. We enjoyed the sessions we attended and thought that the presenters were entertaining as … <a href="http://www.mango-solutions.com/wp/2016/09/earl-london-2016-revisited-day-2/">Continue reading <span>→</span></a>
Guest post by Maciej Beręsewicz. It has been a very active semester in Europe regarding R meetings. After two successful main events: satRday in Budapest, and EARL in London, now is the turn for eRum 2016, the first European R users meeting. More than 250 data scientists will meet in Poznan, Poland, from 12th to 14th of October, to discuss and present innovative uses of the statistical language R in different industries, academic world, and government. A crucial part of an event like this is the support of sponsors for many reasons: In first place they make possible to keep registration fees low, so more people can attend; they also help to validate the meeting inside the R global community; they contribute to the sessions with innovative applications in real business environments; and they add more value to the conference by the direct interaction with the rest of the attendants on coffee breaks and social events, or directly on their exhibition stands. As organizers of eRum 2016 we are very proud and grateful to have well known and innovative companies who believe in our contribution to the R community and will make eRum 2016 a great meeting of useRs. Let […]
We are thrilled to announce our first batch of speakers for EARL Boston in November 2016. Keynotes Jenny Bryan and Ricardo Bion Jenny is the Associate Professor of Statistics at the University of British Columbia and part of the leadership … <a href="http://www.mango-solutions.com/wp/2016/09/first-10-speakers-announced-for-earl-boston-in-november/">Continue reading <span>→</span></a>
At the 2016 EARL London conference senior data-visualisation journalist John Burn-Murdoch, described how the Financial Times uses R to produce high-quality, striking data visualisations. Until recently, charts were the realm of an information designer using tools like Adobe Illustrator: the output was beautiful, but the process was a long and winding one. The FT needed to be able to &quot;audition&quot; several different visual treatments quickly, to be able to create stunning visuals before deadline. That's where R and the ggplot2 package come in. John presented a case study (you can see the slides with animations here) on creating this FT...
sjmisc-package: Working with labelled data A major update of my sjmisc-package was just released an CRAN. A major change (see changelog for all changes )is the support of the latest release from the haven-package, a package to import and export SPSS, SAS or Stata files. The sjmisc-package mainly addresses three domains: reading and writing data […]
Zachary Deane-Mayer, who collaborates on caret, has put together a DataCamp course on Machine Learning in R. Zach and DataCamp did a great job of developing a course that is just right for people who are ...
We’re excited today to announce sparklyr, a new package that provides an interface between R and Apache Spark. Over the past couple of years we’ve heard time and time again that people want a native dplyr interface to Spark, so we built one! sparklyr also provides interfaces to Spark’s distributed machine learning algorithms and much more. Highlights include: Interactively manipulate Spark data using […]
BiclustGUI in R
Ewoud De Troyer, University of Hasselt (CenStat)
Biclustering in a GUI &amp; Growing a GUI together!
Introduction
We are happy to announce the first release of the BiclustGUI on CRAN (RcmdrPlugin.BiclustGUI)!
This GUI will you enable...
Last week the French National Institute of Health and Medical Research (Inserm) organised with the Stan Group a training programme on Bayesian Inference with Stan for Pharmacometrics in Paris. Daniel Lee and Michael Betancourt, who run the course over ...
I'll be in the UK next week doing three talks in three days: First, I'll be giving a talk at the London R-Ladies meetup on Monday October 3rd with perhaps the best title yet: Whose Scat Is That? An 'Easil...
We can use unsupervised machine learning to identify which drugs are associated with which adverse events. Specifically, machine learning can help us to create clusters based on gender, age, outcome of adverse event, route drug was administered, purpose the drug was used for, body mass index, etc. This can help for quickly discovering hidden associations […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toGoodReads: Exploratory data analysis and sentiment analysis (Part 2)" rel="bookmark" href="http://datascienceplus.com/exploratory-data-analysis-and-sentiment-analysis/">GoodReads: Exploratory data analysis and sentiment analysis (Part 2)</a></li><li><a title="Permanent Link toGoodReads: Webscraping and Text Analysis with R (Part 1)" rel="bookmark" href="http://datascienceplus.com/goodreads-webscraping-and-text-analysis-with-r-part-1/">GoodReads: Webscraping and Text Analysis with R (Part 1)</a></li><li><a title="Permanent Link toEuro 2016 analytics: Who’s playing the toughest game?" rel="bookmark" href="http://datascienceplus.com/euro-2016-analytics-whos-playing-the-toughest-game/">Euro 2016 analytics: Who’s playing the toughest game?</a></li><li><a title="Permanent Link toIntegrating R with Apache Hadoop" rel="bookmark" href="http://datascienceplus.com/integrating-r-with-apache-hadoop/">Integrating R with Apache Hadoop</a></li><li><a title="Permanent Link toPlotting App for ggplot2" rel="bookmark" href="http://datascienceplus.com/plotting-app-for-ggplot2/">Plotting App for ggplot2</a></li></ol>
Wordclouds are one of the most visually straightforward, compelling ways of displaying text info in a graph. Of course, we have a lot of web pages (and even apps) that, given an input text, will plot you some nice tagclouds. However, when … <a href="https://talesofr.wordpress.com/2016/09/26/quick-wordclouds-from-pubmed-abstracts-using-pmid-lists-in-r/">Sigue leyendo <span>→</span></a>
By Carson Sievert, lead Plotly R developer I’m excited to announce that plotly’s R package just sent its first CRAN update in nearly four months. To install the update, run install.packages(&quot;plotly&quot;). This update has breaking changes, enables new features, fixes numerous bugs, and takes us from version 3.6.0 to 4.5.2. To see all the changes, […]
<p>Wonderful, indeed, to have an RStan book in Japanese: Kentarou Matsuura. 2016. Bayesian Statistical Modeling Using Stan and R. Wonderful R Series, Volume 2. Kyoritsu Shuppan Co., Ltd. Google translate makes the following of the description posted on Amazon Japan (linked from the title above): In recent years, understanding of the phenomenon by fitting a […]</p>
<p>The post <a rel="nofollow" href="http://andrewgelman.com/2016/09/26/rstan-book-in-japanese-wonderful-r/">A book on RStan in Japanese: <i>Bayesian Statistical Modeling Using Stan and R</i> (Wonderful R, Volume 2)</a> appeared first on <a rel="nofollow" href="http://andrewgelman.com/">Statistical Modeling, Causal Inference, and Social Science</a>.</p>
R’s boxplot function has a lot of useful parameters allowing us to change the behaviour and appearance of the boxplot graphs. In this exercise we will try to use those parameters in order to replicate the visual style of Matlab’s boxplot. Before trying out this exercise please make sure that you are familiar with the […]
<p>For quite some time now I have been using R’s caret package to choose the model for forecasting time series data. The approach is satisfactory as long as the model is not an evolving model (i.e. is not re-trained), or if it evolves rarely. If the model is re-trained often – the approach has significant […]</p>
<p>The post <a rel="nofollow" href="http://www.quintuitive.com/2016/09/25/better-model-selection-evolving-models/">Better Model Selection for Evolving Models</a> appeared first on <a rel="nofollow" href="http://www.quintuitive.com/">Quintuitive</a>.</p>
Anyone that follows US politics will be aware of the tremendous changes and volatility that has struck the US political landscape in the past year. In this post, I leverage third-party data to surface who are the most frequent liars, and show how to build a containerized Shiny app to visualize direct comparisons between individuals. […]
Radiant is a platform-independent browser-based interface for business analytics in R. I first introduced Radiant through R-bloggers on 5/2/2015 and, according to Dean Attali, the post was reasonably popular. So I decided to write a post about the chan...
FileTable has been around now for quite some time and and it is useful  for storing files, documents, pictures and and binary files in a designated SQL Server table – FileTable. The best part of FileTable is the fact one can access it from windows or other application as if it were stored on file … <a href="https://tomaztsql.wordpress.com/2016/09/25/filetable-and-storing-graphs-from-microsoft-r-server/">Continue reading <span>FileTable and storing graphs from Microsoft R Server</span></a>
A new experimental package is now on the ghrr drat. It is named tint which stands for Tint Is Not Tufte. It provides an alternative for Tufte-style html presentation. I wrote a bit more on the package page and the README in the repo -- so go read thi...
If you are on Windows 10, no doubt you have heard that Microsoft included the bash shell in its 2016 Windows 10 anniversary update. What you may not know is that this is much, much more than just the bash shell. This is a whole Linux layer that enables you to use Linux tools, and does away with a further layer like Cygwin (which requires a special dll). However, you will only get the bash shell out of the box. To enable the whole Linux layer, follow instructions <a target="_blank" href="http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/">here</a>. Basically, this involves enabling developer mode then enabling the Linux layer feature. In the process, you will download some further software from the Windows store.<br><br>Why is this big news? To me, this installs a lot of the Linux tools that have proven useful over the years, such as <span>wc</span><span>, <span>sed</span>, <span>awk</span>, <span>grep</span>, and so forth. In some cases, these tools work much better than software packages such as R or SAS, and their power comes in combining these tools through pipes. You also get <span>apt-get</span>, which enables you to install and manage packages such as SQLite, octave, and gnuplot. You can even install R through this method, though I don't know if RStudio works with R installed in this way.</span><br><br>If you're a Linux buff who uses Windows, you can probably think of many more things  you can do with this. The only drawback is that I haven't tried using any sort of X Windows or other graphical interfaces.
After having carefully followed the online official Shiny tutorial, I decided to make a quick try at making my very first Shiny App. I should say that I found myself very well with the explanation given and Shiny was definitely one of the libraries tha...
In this post we’ll quickly show how to create radial stacked ara charts in plotly. We’ll use the AirPassengers dataset. Inspired by Mike Bostocks post: http://bl.ocks.org/mbostock/3048740 [crayon-57e6a72085a64536913742/] Radial Stacked Area Chart
Nobody knows how to say goodbye, It seems so easy until you try (Nobody Knows, The Lumineers) I have been user of WordPress.com for (almost) three years and it has been a very good experience. But I want to take more control of my blog: install plugins, manage ads (someone interested out there to publicize in my … <a href="https://aschinchon.wordpress.com/2016/09/23/bye-ripples-hi-fronkonstin/">Continue reading <span>Bye Ripples, Hi Fronkonstin</span> <span>→</span></a>
Slides have now been posted for many of the talks given at the recent Effective Applications of the R Language (London) conference, and I thought I'd highlight a few that featured Microsoft R. Chris Cole manages the deployment of R at Investec, supporting investment and risk teams worldwide. Despite some initial grumbling about having to use &quot;old&quot; versions of software, the R developers ultimately appreciated the improvements in collaboration and IT support derived from a standardized &quot;R Suite&quot; company-wide. Investec also supports R deployment for other analysts who don't necessarily use R, by integrating R into desktop applications via a...
Being able to visualize information through plots is essential for a statistic analysis. A simple and clean graph  can explain much more than words.  In this set of exercises you will test and learn advanced graphic arguments. Before you start check the documentation for the following functions: plot, points, abline, title, legend ,par (including all the arguments), mfrow and layout For […]
Python, R, Qt, peewee, bokeh, pandas, SQLite plus couple of sleepless nights and here you are a сute app for the environmental monitoring needs )))Main window of the application
We’ve just released Shiny Server and Shiny Server Pro 1.4.6. Relative to 1.4.2, our previously blogged-about version, the 1.4.6 release primarily includes bug fixes, and mitigations for low-severity security issues found by penetration testing. The full list of changes is after the jump. If you’re running a Shiny Server Pro release that is older than 1.4.3 and are configured to use SSL/TLS, […]
A straightforward but probabilistic riddle this week in the Riddler, which is to find the expected order of integer i when the sequence {1,2,…,n} is partitioned at random into two sets, A and B, each of which is then sorted before both sets are merged. For instance, if {1,2,3,4} is divided in A={1,4} and B={2,3}, […]
It came to my attention that size of XDF (external data frame) file can change drastically based on the compute context and environment. When testing the output of a dataset I was working on in SQL Server Management Studio I was simultaneously testing R code in RTVS or  R Studio and I have noticed a … <a href="https://tomaztsql.wordpress.com/2016/09/22/size-of-xdf-files-using-revoscaler-package/">Continue reading <span>Size of XDF files using RevoScaleR package</span></a>
<p>Have you ever watched a livestream? Have you ever wondered what the actual minute to minute of doing data science looks like? Do you wonder if other R users have the same frustrations as you? If yes -- then read on!</p><p>I'm off on a new professional adventure where I am doing public facing work for the first time in years. While working at home the other day I thought it would be a great idea to keep myself on-task and document my decisions if I recorded myself working with my webcam. Then, I thought, why stop there -- why not livestream my work? </p><p>And thus, the <a target="_blank" href="https://jknowles.github.io/DataScienceLivestream/">R Data Science Livestream</a> was born. The idea is that every day for an hour or two I will livestream myself doing some data science tasks related to my current project -- which is to analyze dozens of years of FBI Uniform Crime reports (<a target="_blank" href="https://jknowles.github.io/DataScienceLivestream/pages/project.html">read more</a>). I haven't done much R coding in the last 4 months, so it's also a good way to shake off the rust of being out of the game for so long.</p><p>So if you are at all interested or curious why someone would do this, check out <a href="https://jknowles.github.io/DataScienceLivestream/">the landing page I put up to document the project</a> and if you are really curious, maybe even <a target="_blank" href="https://www.youtube.com/c/jaredknowles">tune in or watch the archives on YouTube</a>!</p><p> </p>
by Brandon Rohrer, Principal Data Scientist, Microsoft R or Python? Torch or TensorFlow? (or MXNet or CNTK)? Spark or map-reduce? When we're getting started on a project, the mountain of tools to choose from can be overwhelming. Sometimes it makes me feel small and bewildered, like Alice in Wonderland. Luckily, the Cheshire Cat cut to the heart of the problem: “Would you tell me, please, which way I ought to go from here?” “That depends a good deal on where you want to get to,” said the Cat. “I don’t much care where–” said Alice. “Then it doesn’t matter which...
R Markdown is a great tool to make research results reproducible. However, in scientific research papers or reports, tables and figures usually need to be numbered and referenced. Unfortunately, R Markdown has no “native” method to number and reference table and figure captions. The recently published bookdown package makes it very easy to number and […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toRDBL – manipulate data in-database with R code only" rel="bookmark" href="http://datascienceplus.com/rdbl-manipulate-data-in-database-with-r-code-only/">RDBL – manipulate data in-database with R code only</a></li><li><a title="Permanent Link toR Markdown: How to format tables and figures in .docx files" rel="bookmark" href="http://datascienceplus.com/r-markdown-how-to-format-tables-and-figures-in-docx-files/">R Markdown: How to format tables and figures in .docx files</a></li><li><a title="Permanent Link toR Markdown: How to insert page breaks in a MS Word document" rel="bookmark" href="http://datascienceplus.com/r-markdown-how-to-insert-page-breaks-in-a-ms-word-document/">R Markdown: How to insert page breaks in a MS Word document</a></li><li><a title="Permanent Link toWorking on Data-Warehouse (SQL) with R" rel="bookmark" href="http://datascienceplus.com/working-on-data-warehouse-sql-with-r/">Working on Data-Warehouse (SQL) with R</a></li><li><a title="Permanent Link toImplementing Apriori Algorithm in R" rel="bookmark" href="http://datascienceplus.com/implementing-apriori-algorithm-in-r/">Implementing Apriori Algorithm in R</a></li></ol>
Hadley Wickham, co-author (with Garrett Grolemund) of R for Data Science and RStudio's Chief Scientist, has focused much of his R package development on the un-sexy but critically important part of the data science process: data management. In the Tidy Tools Manifesto, he proposes four basic principles for any computer interface for handling data: Reuse existing data structures. Compose simple functions with the pipe. Embrace functional programming. Design for humans. Those principles are realized in a new collection of his R packages: the tidyverse. Now, with a simple call to library(tidyverse) (after installing the package from CRAN), you can load...
I just found a fun food themed dataset that I’d never heard about and that I thought I’d share. It’s from a project called What’s on the menu where the New York Public Library has crowdsourced a digitization of their collection of historical ...
The paired samples t test is used to check if there are any differences in the mean of the same sample at two different time points. For example a medical researcher collects data on the same patients before and after a therapy. A paired t test will show if the therapy improves patient outcomes. There […]
<p>Info, dates and topics of our five Autumn R Courses: from programming to data manipulation, from statistics to data mining, everything with R starting with R for Beginners course.</p>
<p>The post <a rel="nofollow" href="http://www.milanor.net/blog/r-beginners-r-courses-milan/">R for Beginners and other R courses | Milan</a> appeared first on <a rel="nofollow" href="http://www.milanor.net/blog">MilanoR</a>.</p>
The Linux edition of the Data Science Virtual Machine on Microsoft Azure was recently upgraded. The Linux DSVM includes Microsoft R, Anaconda Python, Jupyter, CNTK and many other data science and machine learning tools, new or upgraded for this release. This eWeek story gives an overview of the improvements, but the highlights are: Microsoft R Server (developer edition) is now included. This includes the complete R distribution from CRAN, plus additional data-analysis functions with big-data capabilities, and the DeployR framework for integrating R into applications as a web service. (The developer edition is identical to the enterprise Microsoft R Server...
  To post your R job on the next post Just visit this link and post a new R job to the R community. You can either post a job for free (which works great), or pay $50 to have your job featured (and get extra exposure). Current R jobs Job seekers: please follow the links below to learn more and apply for your R job of interest: Featured Jobs Full-Time Data Manager II, CRP Boston Children’s Hospital – Posted by katiekerr7 Boston Massachusetts, United States 14 Sep2016 Full-Time Research and Analytics Associate Hodges Ward Elliott – Posted by tkiely@hwerealestate.com New York New York, United States 13 Sep2016 Full-Time Data Mining Analyst Fellow – fixed term 1 year Boston Scientific – Posted by glynburtt Hemel Hempstead England, United Kingdom 5 Sep2016 Full-Time Statistician USC Center for Applied Molecular Medicine – Posted byNaim Matasci Los Angeles California, United States 2 Sep2016 Full-Time Algorithm Developer @ Toronto Crescendo Technology Ltd – Posted by Crescendo Toronto Ontario, Canada 23 Aug2016 Full-Time Enterprise Advocate for RStudio RStudio Inc. – Posted by jclemens1 Anywhere 23 Aug2016 More New Jobs Full-Time Help with finding similarities between XY scatter plots and clustering of time series RNA Seq. data needed UALR – Posted by tfh4 Anywhere […]
Short Course on R and Data Mining Information Technology and Engineering, University of Canberra Fees: There is no fees for the short course but seats are limited to 60 – so register early through http://www.meetup.com/CanberraDataSci/events/234168862/ Presenters: Dr Yanchang Zhao (Adjunct … <a href="https://rdatamining.wordpress.com/2016/09/20/short-course-on-r-and-data-mining-university-of-canberra-fri-7-oct-2016/">Continue reading <span>→</span></a>
Nina Zumel prepared an excellent article on the consequences of working with relative error distributed quantities (such as wealth, income, sales, and many more) called “Living in A Lognormal World.” The article emphasizes that if you are dealing with such quantities you are already seeing effects of relative error distributions (so it isn’t an exotic … <a href="http://www.win-vector.com/blog/2016/09/relative-error-distributions-without-the-heavy-tail-theatrics/">Continue reading <span>Relative error distributions, without the heavy tail theatrics</span></a>
Today is Talk Like A Pirate Day, the perfect day to learn R, the programming language of pirates (arrr, matey!). If you have two-and-a-bit hours to spare, Nathaniel Phillips has created a video tutorial YaRrr! The Pirate's Guide to R which will take you through the basics: installation, basic R operations, and the matrix and data frame obects. For a more in-depth study of R, there's also a 250-page e-book YaRrr! The Pirate’s Guide to R which goes into the basics in more depth, and covers more advanced topics including data visualization, statistical analysis, and writing your own functions. There's...
The lapply() function applies a function to individual values of a list, and is a faster alternative to writing loops. Structure of the lapply() function: lapply(LIST, FUNCTION, ...) The list variable used for these exercises: list1 &lt;- list(observationA = c(1:5, 7:3), observationB=matrix(1:6, nrow=2)) Answers to the exercises are available here. Exercise 1 Using lapply(), find […]
<p>Quantide in partnership with DataCamp , offers an introductory online course to R completely in Italian. The course is free and open to anyone who wishes to participate. There is no requirement, you need only a Pc or a Mac with an Internet access.</p>
<p>The post <a rel="nofollow" href="http://www.milanor.net/blog/introduction-r-free-basic-r-course-datacamp-italian/">Introduction to R: free basic R course on DataCamp in italian</a> appeared first on <a rel="nofollow" href="http://www.milanor.net/blog">MilanoR</a>.</p>
Quite often I want to share simple (static) web pages with other colleagues or clients. For example, I may have written a report using R Markdown and rendered it to HTML. AWS S3 can easily host such a simple web … <a href="https://alexioannides.com/2016/09/19/an-r-function-for-generating-authenticated-urls-to-private-web-sites-hosted-on-aws-s3/">Continue reading <span>→</span></a>
The guys at Stack Overflow have recently released a very interesting dataset containing the entire history of questions made by users since the beginning of the site, back in 2008. It's called...<br><br>
&lt;&lt;Keep reading>>
Here’s a problem I’ve had again and again: let’s say you’ve defined a statistical model with several parameters. One of them is a scalar. Another is a matrix. The third one is a vector, and so on. When fitting the model the natural thing to do is to write a likelihood function that takes as […]
Individual level data is essential to understand voting behaviour
My previous analysis has occasionally come up against the problem “only individual level data could resolve that,”. Since I last wrote that, the New Zealand Election Study data for ...
Recently I was surfing the web, when I came across a real cool post New R package to access World Bank data, by Markus Gesmann on using googleVis and motion charts with World Bank Data. The post also introduced me to Hans Rosling, Professor of Sweden’s Karolinska Institute. Hans Rosling, the creator of the famous … <a href="https://gigadom.wordpress.com/2016/09/18/analyzing-world-bank-data-with-wdi-googlevis-motion-charts/">Continue reading <span>Analyzing World Bank data with WDI, googleVis Motion Charts</span></a>
As of today, there are 20 R packages that reverse depend/import/suggest (3/14/3)
the pbapply package. Current and future package developers
who decide to incorporate the progress bar using pbapply
might want to customize the type and style of the progress bar
in their packages to better suit the needs of certain functions
or to create a distinctive look.
Here is a quick guide to help in setting up and customizing the progress bar.
A few weeks ago I worked on some old code parallelization. The whole process made me think about how efficient parallelization of the existing code in R can really be and what should be considered efficient. There is a lot … <a href="http://www.vesnam.com/Rblog/existing-code-parallelization-yes-or-no/">Continue reading <span>→</span></a>
I recently had an email exchange with a seasoned, well respected analytical professional which included the following (from them, not me): “… my versioning is to have multiple versions of files and to use naming conventions… it works really well....
<p>I’m currently getting the following error on a Windows machine: The machine in question is sitting behind a gnarly firewall and proxy, which I suspect are the source of the problem. I also need to use --ignore-certificate-errors when running chromium-browser, which points to the same issue. This seems to resolve the issue:</p>
<p>The post <a rel="nofollow" href="http://www.exegetic.biz/blog/2016/09/peer-certificate/">Fixing “Peer certificate cannot be authenticated”</a> appeared first on <a rel="nofollow" href="http://www.exegetic.biz/blog">Exegetic Analytics</a>.</p>
anytime arrived on CRAN via release 0.0.1 a good two days ago. anytime aims to convert anything in integer, numeric, character, factor, ordered, ... format to POSIXct (or Date) objects.
This new release 0.0.2 adds two new functions to gather conversi...
As R users we mostly perform analysis, produce reports and create interactive shiny applications. Those are rather one-time performances. Sometimes, however, the R developer enters the world of the real software development, where R applications shoul...
The tidyverse is a set of packages that work in harmony because they share common data representations and API design. The tidyverse package is designed to make it easy to install and load core packages from the tidyverse in a single command. The best place to learn about all the packages in the tidyverse and how […]
I am pleased to announced lubridate 1.6.0. Lubridate is designed to make working with dates and times as pleasant as possible, and is maintained by Vitalie Spinu. You can install the latest version with: install.packages(&quot;lubridate&quot;) This release includes a range of bug fixes and minor improvements. Some highlights from this release include: period() and duration() constructors now accept character […]
In this set of exercises we shall create an empty graph and practice the functions for basic manipulation with vertices and edges, using the package igraph. If you don’t have the package already installed, install it using the following code: install.packages(&quot;igraph&quot;) and load it into the session using the following code: library(&quot;igraph&quot;) before proceeding. You […]
<p>Haveibeenpwned.com is a fantastic service that helps people find out if they’ve been involved in a data breach. HIBPwned is an R wrapper for that service. Recently, due to abuse of the system, Troy Hunt had to add a limit of one request per 1.5s. The new version published on CRAN last night adds a […]</p>
<p>The post <a rel="nofollow" href="http://itsalocke.com/hibpwned-updated-cran/">HIBPwned updated on CRAN</a> appeared first on <a rel="nofollow" href="http://itsalocke.com/">It's a Locke</a>.</p>
As I’m in the industry now I figured I needed some business cards and as it seems the 90s never left us and Japanese monsters are hip again, I decided to make them Pokémon themed. I think they turned out pretty well, and here I’m just going ...
I am happy to announce that my simputation package has appeared on CRAN this weekend. This package aims to simplify missing value imputation. In particular it offers standardized interfaces that make it easy to define both imputation method and imputation … <a href="http://www.markvanderloo.eu/yaRb/2016/09/13/announcing-the-simputation-package-make-imputation-simple/">Continue reading <span>→</span></a>
<p>The previous post in this series, showed a way to identify trading opportunities. The approach I implemented used time series daily data to identify good entry points in terms of risk-reward. The natural next step is to try to make use of these opportunities using machine learning. To refresh: the output of the previous post […]</p>
<p>The post <a rel="nofollow" href="http://www.quintuitive.com/2016/09/13/forecasting-opportunities/">Forecasting Opportunities</a> appeared first on <a rel="nofollow" href="http://www.quintuitive.com/">Quintuitive</a>.</p>
This report describes changes in version 0.2 of the ‘DOM’ package for R. Version 0.1 of ‘DOM’ allowed HTML content to be added to a web page (or removed or modified); version 0.2 adds the ability to append SVG content … <a href="http://stattech.wordpress.fos.auckland.ac.nz/2016-12-dom-version-0-2/">Continue reading <span>→</span></a>
A new version of the OpenStreetMap package has been released to CRAN. OpenStreetMap 0.3.3 contains several minor improvements. I've removed the CloudMade tile set types, as they seem to have gone out of business. MapQuest has also been removed as they have moved to a new API. The mapbox type has been updated to use their […]
The independent t test is used to test if there is any statistically significant difference between two means. Use of an independent t test requires several assumptions to be satisfied. The assumptions are listed below The variables are continuous and independent The variables are normally distributed The variances in each group are equal When these […]
anytime just arrived on CRAN as a very first release 0.0.1.
So why (yet another) package dealing with dates and times? R excels at computing with dates, and times. By using typed representation we not only get all that functionality but also of the a...
imager is an R package for image processing, based on CImg. This new release brings many new features, including: Support for automatic parallel processing using OpenMP. A new S3 class, imlist, which makes it easy to work with image lists New functions for interactively selecting image regions (grabRect,grabPoint,grabLine) Experimental support for CImg’s byte-compiled DSL via […]
In this post, I will show you how to create an animation using R and ffmpeg. The idea to do so is pretty simple: Generate a number of snapshots Combine them in a video file using ffmpeg The best way to learn about the art of animation is by doing it ourselves, so our work […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toThe importance of Data Visualization" rel="bookmark" href="http://datascienceplus.com/the-importance-of-data-visualization/">The importance of Data Visualization</a></li><li><a title="Permanent Link toggplot2 themes examples" rel="bookmark" href="http://datascienceplus.com/ggplot2-themes-examples/">ggplot2 themes examples</a></li><li><a title="Permanent Link toMap the Life Expectancy in United States with data from Wikipedia" rel="bookmark" href="http://datascienceplus.com/map-the-life-expectancy-in-united-states-with-data-from-wikipedia/">Map the Life Expectancy in United States with data from Wikipedia</a></li><li><a title="Permanent Link toWhat can we learn from the statistics of the EURO 2016 – Application of factor analysis" rel="bookmark" href="http://datascienceplus.com/what-can-we-learn-from-the-statistics-of-the-euro-2016-application-of-factor-analysis/">What can we learn from the statistics of the EURO 2016 – Application of factor analysis</a></li><li><a title="Permanent Link toVisualizing obesity across United States by using data from Wikipedia" rel="bookmark" href="http://datascienceplus.com/visualizing-obesity-across-united-states-by-using-data-from-wikipedia/">Visualizing obesity across United States by using data from Wikipedia</a></li></ol>
At the conference of the New Zealand Association of Economists (NZAE) in late June 2016 I gave a paper on Modelled Territorial Authority Gross Domestic Product, a new dataset my team developed last year in my day job. See the official website of the M...
This is the third and final blog post in my series on income inequality. This post discusses the detachment of compensation from productivity that occured around 1973. I look at the data and use R for exploring this break, along with why it may have occured. R code is with the analysis, in the spirit of reproducible research.
<p>A new version of the ropensci <a href="https://cran.r-project.org/package=hunspell">hunspell</a> package has been released to CRAN. Hunspell is the spell checker library used by LibreOffice, OpenOffice, Mozilla Firefox, Google Chrome, Mac OS-X, InDesign, Opera, RStudio and many others. It provides a system for tokenizing, stemming and spelling in almost any language or alphabet. The R package exposes both the high-level spell-checker as well as low-level stemmers and tokenizers which analyze or extract individual words from various formats (text, html, xml, latex).</p>
<h1>New Vignette</h1>
<p>This new version now includes a beautiful <a href="https://cloud.r-project.org/web/packages/hunspell/vignettes/intro.html">vignette</a> which gives an overview of the main functionality to get you started! It demonstrates the tokenizer, stemmer and spell-checker and has an example how to use the stemmer and tokenizer to create a word cloud from a large body of text.</p>
<p><a href="https://cloud.r-project.org/web/packages/hunspell/vignettes/intro.html"></a></p>
<h1>Installing and Updating</h1>
<p>The package is most easily installed from CRAN:</p>
<div><pre><code>install.packages<span>(</span><span>&quot;hunspell&quot;</span><span>)</span>
</code></pre></div>
<p>Or to get the latest version from Github:</p>
<div><pre><code>devtools<span>::</span>install_github<span>(</span><span>&quot;ropensci/hunspell&quot;</span><span>)</span>
</code></pre></div>
<p>This package does not require any system dependencies (libhunspell is now bundled with the package).</p>
A new Shiny release is upon us! There are many new exciting features, bug fixes, and library updates. We’ll just highlight the most important changes here, but you can browse through the full changelog for details. This will likely be the last release before shiny 1.0, so get out your party hats! To install it, […]
Introduction Although more than 20 years old, Market Basket Analysis (MBA) (or association rules mining) can still be a very useful technique to gain insights in large transactional data sets. The classical example is transactional data in a supermarket. For … <a href="https://longhowlam.wordpress.com/2016/09/12/some-insights-in-soccer-transfers-using-market-basket-analysis/">Continue reading <span>→</span></a>
stringdist 0.9.4.2 was accepted on CRAN at the end of last week. This release just fixes a few bugs affecting the stringdistmatrix function, when called with a single argument. From the NEWS file: bugfix in stringdistmatrix(a): value of p, for … <a href="http://www.markvanderloo.eu/yaRb/2016/09/11/stringdist-0-9-4-2-released/">Continue reading <span>→</span></a>
This blog post is about clustering and specifically about my recently released package on CRAN, ClusterR. The following notes and examples are based mainly on the package Vignette.
Cluster analysis or clustering is the task of grouping a set...
Recently together with my friend <a href="https://www.linkedin.com/in/jakuczunwit">Wit Jakuczun</a> we have discussed about a <a href="http://blog.revolutionanalytics.com/2016/08/telco-customer-churn-with-r-in-sql-server-2016.html">blog post</a> on Revolution showing application of SQL Server R services to build and run telco churn model. It is a very nice analysis and we thought that it would be interesting to compare the results to <a href="http://www.h2o.ai/">H2O</a>, which is a great tool for automated building of prediction models.<br><br>Today Wit has published on Github his codes performing the analysis. You can check them out <a href="https://github.com/WLOGSolutions/telco-customer-churn-in-r-and-h2o">here</a>. The obtained model is pretty good, with AUC equal to 0.947 which is better than any of the models presented in Revolution blog. But, in my opinion the key value of the Wit's work is to show how simple it is to build models with H2O using R. From his <a href="https://github.com/WLOGSolutions/telco-customer-churn-in-r-and-h2o">Github reposity</a> you can download a ready to run RStudio project and all the instructions needed to build the model in just a few lines of code.
The pbapply R package adds progress bar to vectorized functions, like lapply. A feature request regarding progress bar for parallel functions has been sitting at the development GitHub repository for a few months. More recently, the author of the pbmcapply package dropped a note about his implementation of forking functionality with progress bar for Unix/Linux computers, which got me thinking. How should we add progress bar to snow type clusters? Which led to more important questions: what is the real cost of the progress bar and how can we reduce overhead on process times?
I’ve had this book on pre-order since spring and it finally arrived on Friday. I subsequently devoured it over the weekend. The book lays out a clear and compelling case for how data-driven algorithms can become — in contrast to their promise of amoral objectivism — efficient means for reproducing and even exacerbating social inequalities […]
After high school I made my way from Johannesburg, situated in the northern part of South Africa, to the famous wine country known as Stellenbosch in the south. Here for the first time I got a ton of exposure to wine and the countless varietals that ma...
Share This: This article is originally published in Capital of Statistic by Chinese [link] and I would like to thank He Tong for lots of great suggestions. All code in this post can be found on GitHub [link]. Data scientists are already very familiar with statistical software like R, SAS, SPSS, MATLAB; however, some of them are relatively […]
Introduction This is a post I have been wanting to write for several months, but had to put it off for one reason or another. In this post I use my R package cricketr to analyze the performance of All-rounder greats namely Kapil Dev, Ian Botham, Imran Khan and Richard Hadlee. All these players had … <a href="https://gigadom.wordpress.com/2016/09/10/cricketr-sizes-up-legendary-all-rounders-of-yesteryear/">Continue reading <span>cricketr sizes up legendary All-rounders of yesteryear</span></a>
gettz is now on CRAN in its initial release 0.0.1.
It provides a possible fallback in situations where Sys.timezone() fails to determine the system timezone. That can happen when e.g. the file /etc/localtime somehow is not a link into the correspondi...
The new issue of The R Journal is now available! You may Download the complete issue, or choose your pleasure from the following links: Table of contents Editorial Michael Lawrence Contributed Research Articles metaplus: An R Package for the Analysis of Robust Meta-Analysis and Meta-Regression Ken J. Beath Gender Prediction Methods Based on First Names with genderizeR Kamil Wais Conditional Fractional Gaussian Fields with the Package FieldSim Lambert-Lacroix Sophie, Alexandre Brouste, Jacques Istas rTableICC: An R Package for Random Generation of 2*2*K and R*C Contingency Tables Haydar Demirhan Maps, Coordinate Reference Systems and Visualising Geographic Data with mapmisc Patrick Brown Variable Clustering in High-Dimensional Linear Regression: The R Package clere Julien Jacques, Loïc Yengo, Christophe Biernacki, Mickael Canouil Stylometry with R: A Package for Computational Text Analysis Maciej Eder, Jan Rybicki, Mike Kestemont quickpsy: An R Package to Fit Psychometric Functions for Multiple Groups Daniel Linares, Joan López-Moliner FWDselect: An R Package for Variable Selection in Regression Models Marta Sestelo, Nora M. Villanueva, Luís Meira-Machado, Javier Roca-Pardiñas An Interactive Survey Application for Validating Social Network Analysis Techniques Mitchell Joblin, Wolfgang Mauerer Exploring Interaction Effects in Factorial Studies using the hiddenf Package in R Christopher T. Franck, Jason A. Osborne Heteroscedastic Censored and Truncated […]
Inspired by this article about sentiment analysis and this guide to webscraping, I have decided to get my hands dirty by scraping and analyzing a sample of reviews on the website Goodreads. The goal of this project is to demonstrate a complete example, going from data collection to machine learning analysis, and to illustrate a […]<ul></ul><strong><p>Related Post</p></strong><ol><li><a title="Permanent Link toEuro 2016 analytics: Who’s playing the toughest game?" rel="bookmark" href="http://datascienceplus.com/euro-2016-analytics-whos-playing-the-toughest-game/">Euro 2016 analytics: Who’s playing the toughest game?</a></li><li><a title="Permanent Link toIntegrating R with Apache Hadoop" rel="bookmark" href="http://datascienceplus.com/integrating-r-with-apache-hadoop/">Integrating R with Apache Hadoop</a></li><li><a title="Permanent Link toPlotting App for ggplot2" rel="bookmark" href="http://datascienceplus.com/plotting-app-for-ggplot2/">Plotting App for ggplot2</a></li><li><a title="Permanent Link toPerforming SQL selects on R data frames" rel="bookmark" href="http://datascienceplus.com/performing-sql-selects-on-r-data-frames/">Performing SQL selects on R data frames</a></li><li><a title="Permanent Link toDeveloping an R Tutorial shiny dashboard app" rel="bookmark" href="http://datascienceplus.com/developing-a-r-tutorial-shiny-dashboard-app/">Developing an R Tutorial shiny dashboard app</a></li></ol>
Practical example of how to leverage humans and artificial intelligence strengths at a fashion company.
<div>Five years ago I wrote a post titled, &quot;<a href="http://davegiles.blogspot.ca/2011/08/beware-of-econometricians-bearing.html">Beware of Econometricians Bearing Spreadsheets</a>&quot;. </div><div><br></div><div>The take-away message from that post was simple: there's considerable, well-documented, evidence that spreadsheets are very, very, dangerous when it comes to statistical calculations. That is, if you care about getting the right answers!</div><div><br></div><div>Read that post, and the associated references, and you'll see what I mean.</div><div><br></div><div>(You might also ask yourself, why would I pay big bugs for commercial software that is of questionable quality when I can use high-quality statistical software such as R, for free?)</div><div><br></div><div>This week, a piece in <i><a href="http://www.economist.com/blogs/graphicdetail/2016/09/daily-chart-3?fsrc=scn/fb/te/bl/ed/excelerrorsandsciencepapers">The Economist</a> </i>looks at the shocking record of publications in genomics that fall prey to spreadsheet errors. It's a sorry tale, to be sure. I strongly recommend that you take a look.</div><div><br></div><div>Yes, any software can be mis-used. Anyone can make a mistake. We all know that. However, it's not a good situation when a careful and well-informed researcher ends up making blunders just because the software they trust simply isn't up to snuff!  </div><div><br></div><br>© 2016, David E. Giles
<p>A new version of the ropensci <a href="https://cran.r-project.org/package=magick">magick</a> package has been released to CRAN. Magick is a package for Advanced Image-Processing in R. It wraps the ImageMagick STL which is perhaps the most comprehensive open-source image processing library available today. Our <a href="https://ropensci.org/blog/2016/08/23/z-magick-release">original announcement</a> has more details.</p>
<h1>New features</h1>
<p>This new version now includes a beautiful <a href="https://cloud.r-project.org/web/packages/magick/vignettes/intro.html">vignette</a> which gives an overview of the main functionality to get you started! It lists the various formats, transformations, effects, operations and much more.</p>
<p><a href="https://cloud.r-project.org/web/packages/magick/vignettes/intro.html"></a></p>
<p>The vignette also explains how to use magick with <a href="https://cloud.r-project.org/web/packages/magick/vignettes/intro.html#raster_graphics">raster graphics</a>. This new functionality makes it possible to overlay images on R's graphics device.</p>
<h1>Installing and Updating</h1>
<p>The package is most easily installed from CRAN:</p>
<div><pre><code>install.packages<span>(</span><span>&quot;magick&quot;</span><span>)</span>
</code></pre></div>
<p>When installing from source make sure the Magick++ library is available on your system.</p>
Speaker of the [R] Kenntnis-Tage 2016: Julia Gleixner | TRUMPF Laser GmbH The solid-state lasers of TRUMPF Laser are used for machining materials in various areas – from welding car bodies to cutting stents to drilling minute holes for the production of solar cells. These are just some of the diverse application possibilities. In any … <a href="http://blog.eoda.de/2016/09/08/application-possibilities-of-data-science-in-laser-technology/"><span>„Application possibilities of data science in laser technology“ </span>weiterlesen</a>
When conducting meta-analysis, you most likely have to calculate or convert effects sizes into an effect size with common measure. There are various tools to do this – one easy to use tool is the Practical Meta-Analysis Effect Size Calculator from David B. Wilson. This online-tool is now implemented as an R-package: esc: Effect Size […]
The Problem In two previous posts, I discussed visualizing your turnaround times (TATs). These posts are here and here. One other nice way to visualize your TAT is by means of a heatmap. In particular, we would like to look at the TAT for every hour of the week in a single figure. This manner … <a href="http://labrtorian.com/2016/09/08/make-easy-heatmaps-to-visualize-your-turnaround-times/">Continue reading <span>Make Easy Heatmaps to Visualize your Turnaround Times</span> <span>→</span></a>
The apply() function is an alternative to writing loops, via applying a function to columns, rows, or individual values of an array or matrix. The structure of the apply() function is: apply(X, MARGIN, FUN, ...) The matrix variable used for the exercises is: dataset1 &lt;- cbind(observationA = 16:8, observationB = c(20:19, 6:12)) Answers to the […]
The R package development is a complex process of creating (mostly) a useful software, that will (probably) be used by other users. This means the provided tool should be resistant, immune, well tested and properly documented. Developers from many dif...
googleformr
For the past few months I’ve been toying with the idea of using googleformr to empower data science-oriented instructors to use data science in their workflow. In this 2-part series of posts, I’ll show how to AppScripts and R to turn...
Joe Thorley at Poisson Consulting has released a new R package, rtide, (on which I am listed as a co-author) that provides the ability to predict future (and past) tide heights for 637 different tide stations in the United States and associated territories. The underlying data, consisting of tide harmonic constituents, are collected and released […]
<p>Spark 2.0 has been released since last July but, despite the numerous improvements and new features, several annoyances still remain and can cause headaches, especially in the Spark machine learning APIs. Today we’ll have a look at some of them, inspired by a recent answer of mine in a Stack Overflow question (the question was about Spark 1.6 but, as ...
</p><div><a href="http://www.nodalpoint.com/spark-classification/">Read More</a></div>
<p>The post <a rel="nofollow" href="http://www.nodalpoint.com/spark-classification/">Classification in Spark 2.0: “Input validation failed” and other wondrous tales</a> appeared first on <a rel="nofollow" href="http://www.nodalpoint.com/">Nodalpoint</a>.</p>
This animated gif  shows changes in annual mean temperature at 12 East Coast USA airports that had continuous daily data for the 1950 – 2015 period. The data was retrieved from Weather Underground using the R weatherData package . 11 … <a href="https://rclimate.wordpress.com/2016/09/06/annual-mean-temperature-trends-12-airports/">Continue reading <span>→</span></a>
One of the first steps of data analysis is the descriptive analysis; this helps to understand how the data is distributed and provides important information for further steps. This set of exercises will include functions useful for one variable descriptive analysis, including graphs. Before proceeding, it might be helpful to look over the help pages […]
Tell me, which side of the earth does this nose come from? Ha! (ALF) Reading about strange attractors I came across with this book, where I discovered a way to generate two dimensional chaotic maps. The generic equation is pretty simple: I used it to generate these chaotic galaxies: Changing the vector of parameters you … <a href="https://aschinchon.wordpress.com/2016/09/06/chaotic-galaxies/">Continue reading <span>Chaotic Galaxies</span> <span>→</span></a>
First satRdays in Budapest September 03, 2016 event is completed. This one day, community driven event with regional for very affordable prices, good for networking, getting latest from R community event is over. And it was a blast! Great time, nice atmosphere, lots of interesting people and where there is a good energy, there is … <a href="https://tomaztsql.wordpress.com/2016/09/06/sharing-thoughts-on-satrdays-r-conference-budapest-2016-satrdays/">Continue reading <span>Sharing thoughts on satRdays R Conference, Budapest 2016 #satRdays</span></a>
We released googleVis version 0.6.1 on CRAN last week. The update fixes issues with setting certain options, following the switch from RJSONIO to jsonlite. Screen shot of some of the Google ChartsNew to googleVis? The package provides an interface betw...
Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. – Martin Golding In previous posts, I had discussed how Rperform can be used to obtain and visualize package performance data. However, real-world software development is a collaborative process. Thus, automating performance testing for your package is not only a good idea, it’s a critical one; testing projects locally might not be good enough. This post will cover usage of Rperform with Travis CI for automated performance testing. More importantly, we will be able to assess performance impact …
The pretty pictures from my empirical software engineering book are now online, along with the 210 data sets and R code (330M). Plotting the number of data sets in each year shows that empirical software engineering has really taken off in the last 10 years (code+data). Around dozen or so confidential data sets are not […]