<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/a-look-back-at-the-year-in-r-and-microsoft/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://blog.revolutionanalytics.com/2017/02/the-year-in-r.html"> Revolutions</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div>
<p><em><a target="_blank" rel="nofollow" href="https://thomaswdinsmore.com/author/thomaswdinsmore/">Thomas Dinsmore's ML/DL blog</a> recently concluded a look back on significant advancements in data science, machine learning and deep learning — many of which involved R and/or Microsoft. Here are those highlights (reproduced with permission):</em></p>
<h2>The R Project</h2>
<p>R and Python maintained their leadership as primary tools for open data science. The Python versus R debate <a target="_blank" rel="nofollow" href="https://opensource.com/article/16/11/python-vs-r-machine-learning-data-analysis">continued</a> amid an emerging <a target="_blank" rel="nofollow" href="http://www.activestate.com/blog/2016/02/r-vs-python-false-dichotomy">consensus</a> that data scientists should consider learning <a target="_blank" rel="nofollow" href="http://www.kdnuggets.com/2016/03/r-python-learning-both-datacamp.html">both</a>. R has a stronger library of statistics and machine learning techniques and is agiler when working with small data. Python is better suited to developing applications, and the Python open source license is less restrictive for commercial application development.</p>
<ul>
<li>The R user community continued to expand in 2016. It ranked second only to SQL in the <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/09/2016-data-science-salary-survey.html">2016 O’Reilly Data Science Salary Survey</a>; first in the <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/06/r-holds-top-ranking-in-kdnuggets-software-poll.html">KDNuggets</a> poll; and first in the <a target="_blank" rel="nofollow" href="http://www.rexeranalytics.com/data-science-survey.html">Rexer</a> survey. R ranked fifth in the <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/07/r-moves-up-to-5th-place-in-ieee-language-rankings.html">IEEE Spectrum</a> ranking.</li>
<li>R functionality grew at a rapid pace. In April, Microsoft’s Andrie de Vries <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/04/cran-package-growth.html">reported</a> that there were more than 8,000 packages in CRAN, R’s primary repository for contributed packages. As of mid-December, there are <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/">9,737</a> packages.  Machine learning <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/views/MachineLearning.html">packages</a> in CRAN continued to grow in number and functionality.</li>
<li>The <a target="_blank" rel="nofollow" href="https://www.r-consortium.org/">R Consortium</a>, a Collaborative <a target="_blank" rel="nofollow" href="https://www.linuxfoundation.org/news-media/announcements/2015/06/linux-foundation-announces-r-consortium-support-millions-users">Project</a> of the Linux Foundation, made some progress in 2016. <a target="_blank" rel="nofollow" href="https://www.r-consortium.org/announcement/2016/06/06/ibm-invests-in-r-programming-language-for-data-science-joins-r-consortium">IBM</a> and <a target="_blank" rel="nofollow" href="https://www.r-consortium.org/news/2016/09/26/esri-joins-the-r-consortium">ESRI</a> joined the Consortium, whose membership now also <a target="_blank" rel="nofollow" href="https://www.r-consortium.org/members">includes</a> Alteryx, Avant, DataCamp, Google, Ketchum Trading, Mango Solutions, Microsoft, Oracle, RStudio, and TIBCO. There are now three working groups and eight funded <a target="_blank" rel="nofollow" href="https://www.r-consortium.org/projects/awarded-projects">projects</a>.</li>
<li>Hadley Wickham had a good year. One of the top contributors to the R project, Wickham co-wrote <a target="_blank" rel="nofollow" href="http://amzn.to/2i8eHyB">R for Data Science</a> and <a target="_blank" rel="nofollow" href="https://blog.rstudio.org/2016/09/15/tidyverse-1-0-0/">released</a> tidyverse 1.0.0 in September. In <em>The tidy tools manifesto</em>, Wickham <a target="_blank" rel="nofollow" href="https://mran.microsoft.com/web/packages/tidyverse/vignettes/manifesto.html">explained</a> the four basic principles to a tidy API.</li>
<li>Max Kuhn, the author of <a target="_blank" rel="nofollow" href="http://amzn.to/2i8BGcS">Applied Predictive Modeling</a> and developer of the <a target="_blank" rel="nofollow" href="http://topepo.github.io/caret/index.html">caret</a> package for machine learning, <a target="_blank" rel="nofollow" href="https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwiXuK2Pr4HRAhUB7oMKHTixBqIQFggaMAA&url=http%3A%2F%2Fappliedpredictivemodeling.com%2Fblog%2F2016%2F11%2F28%2Fworking-at-rstudio&usg=AFQjCNHuP_gkwQQZtBAjf3md0w8zjE_Dsw&sig2=YdiVS4E1UNfTjfzSc1c7dw&bvm=bv.142059868,d.eWE">joined</a> RStudio in November. RStudio previously hired <a target="_blank" rel="nofollow" href="https://www.rstudio.com/rviews/author/josephrickert/">Joseph Rickert</a> away from Microsoft.</li>
<li>AT&amp;T Labs is doing some impressive work with R, including the development of a distributed back-end for out-of-core processing with Hadoop and other data platforms. At the UseR! Conference, Simon Urbanek <a target="_blank" rel="nofollow" href="https://channel9.msdn.com/Events/useR-international-R-User-conference/useR2016/RCloud-Collaborative-Environment-for-Visualization-and-Big-Data-Analytics">presented</a> a summary.</li>
<li>It is impossible to enumerate all of the interesting analysis performed in R this year. David Robinson’s <a target="_blank" rel="nofollow" href="http://varianceexplained.org/r/trump-tweets/">analysis</a> of Donald Trump’s tweets resonated; using tidyverse, <a target="_blank" rel="nofollow" href="https://github.com/juliasilge/tidytext">tidytext</a>, and <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/twitteR/">twitteR</a>, Robinson was able to distinguish between the candidate’s “voice” and that of his staffers on the same account.</li>
<li>On the Revolutions blog, Microsoft’s David Smith <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/09/all-the-r-ladies.html">surveyed</a> the growing role of women in the R community.</li>
</ul>
<h2>Microsoft</h2>
<p>Microsoft had quite a year in machine learning and deep learning. As I noted in Parts One and Two, in 2016 MSFT launched cognitive APIs in Azure for vision, speech, language, knowledge, and search; a managed service for Spark in Azure HDInsight; enhancements to Azure Machine Learning and Version 2.0 of its deep learning framework, rebranded as Microsoft Cognitive Toolkit.</p>
<p>That’s just for starters.</p>
<ul>
<li>In January, Microsoft <a target="_blank" rel="nofollow" href="https://www.petri.com/microsoft-announces-r-server-and-r-server-developer-edition">announced</a> Microsoft R Server, a rebranding of the product it <a target="_blank" rel="nofollow" href="http://www.zdnet.com/article/microsoft-finalizes-its-revolution-analytics-acquisition/">acquired</a> with Revolution Analytics in 2015. Microsoft R Server includes an enhanced R distribution, a scalable back-end, and integration tools. During the year, Microsoft two major releases for R Server. In Release 8, the company <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/06/apache-spark-integrated-with-mrs-for-hadoop.html">added</a> push-down integration with Spark. Release 9 <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/12/microsoft-r-server-90-now-available.html">updated</a> the Spark integration for Spark 2.0, and added <a target="_blank" rel="nofollow" href="https://msdn.microsoft.com/en-us/microsoft-r/microsoftml-introduction">MicrosoftML</a>, a new R package for machine learning.</li>
<li>Microsoft announced SQL Server 2016 in March with embedded <a target="_blank" rel="nofollow" href="https://msdn.microsoft.com/en-us/library/mt604845.aspx">SQL Server R Services</a>. On the Revolutions blog, David Smith <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/03/sql-server-2016-launch.html">reports</a> on the launch. Tomaž Kaštrun <a target="_blank" rel="nofollow" href="http://www.sqlservercentral.com/articles/Microsoft/145393/">explains</a> what you can do with R services in SQL Server.</li>
<li>In November, after an extended preview, Microsoft <a target="_blank" rel="nofollow" href="https://azure.microsoft.com/en-us/blog/r-server-for-hdinsight-now-generally-available/">announced</a> the general availability of R Server for Azure HDInsight, a scale-out implementation of R integrated with Spark clusters created from HDInsight.</li>
<li>Also in Azure, Microsoft <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/04/microsoft-ds-vm-linux.html">added</a> a Linux version of the Data Science Virtual Machine (DSVM). Previously available as a <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2015/12/microsoft-data-science.html">Windows</a> instance, DSVM includes Revolution R Open, Anaconda, Visual Studio Community Edition, PowerBI Desktop, SQL Server Express and the Azure SDK.</li>
<li>PowerBI, Microsoft’s powerful visualization tool, added R support in August. In ComputerWorld, Sharon Machlis, an R user, <a target="_blank" rel="nofollow" href="http://www.computerworld.com/article/3109533/business-intelligence/microsoft-ratchets-up-its-r-enthusiasm.html">enthused</a>. More <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/08/powerbi-and-r.html">here</a>, on the Revolutions blog.</li>
<li><a target="_blank" rel="nofollow" href="https://www.visualstudio.com/vs/rtvs/">R Tools for Visual Studio</a> <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/03/rtvs-preview.html">launched</a> to public preview in March, and to general availability in September. Also in September, Microsoft <a target="_blank" rel="nofollow" href="https://msdn.microsoft.com/en-us/microsoft-r/r-client">released</a> the Microsoft R Client, a free data science tool that works with Microsoft R Open and the ScaleR distributed back end.</li>
<li>Microsoft data scientists Gopi Krishna Kumar, Hang Zhang and Jacob Spoelstra developed a methodology for data science, which they <a target="_blank" rel="nofollow" href="https://channel9.msdn.com/Events/Machine-Learning-and-Data-Sciences-Conference/Data-Science-Summit-2016/MSDSS23">presented</a> at the Microsoft Machine Learning and Data Science Summit 2016 in September. David Smith <a target="_blank" rel="nofollow" href="http://blog.revolutionanalytics.com/2016/10/the-team-data-science-process.html">reports</a>. The method, which the authors call <a target="_blank" rel="nofollow" href="https://github.com/Azure/Microsoft-TDSP">Team Data Science Process</a>, includes a standard directory structure for managing project artifacts using a system such as Git. It also <a target="_blank" rel="nofollow" href="https://github.com/Azure/Azure-TDSP-Utilities/tree/master/DataScienceUtilities/Modeling">includes</a> open source utilities to support the process.</li>
<li>A Microsoft team <a target="_blank" rel="nofollow" href="http://blogs.microsoft.com/next/2016/10/18/historic-achievement-microsoft-researchers-reach-human-parity-conversational-speech-recognition/#sm.00001c26p6xvebfkhujxs063q9t28">developed</a> a system that recognizes conversational speech as well as humans do. The team <a target="_blank" rel="nofollow" href="https://arxiv.org/abs/1610.05256">used</a> convolutional and long short-term memory (LSTM) neural networks built with Microsoft Cognitive Toolkit (CNTK).</li>
<li>Released in 2015 as CNTK, Microsoft rebranded its deep learning framework as <a target="_blank" rel="nofollow" href="https://www.microsoft.com/en-us/research/product/cognitive-toolkit/">Microsoft Cognitive Toolkit</a> (MCT) and <a target="_blank" rel="nofollow" href="http://blogs.microsoft.com/next/2016/10/25/microsoft-releases-beta-microsoft-cognitive-toolkit-deep-learning-advances/#sm.00001c26p6xvebfkhujxs063q9t28">released</a> Version 2.0, with a new Python API and many other enhancements. In VentureBeat, Jordan Novet <a target="_blank" rel="nofollow" href="http://venturebeat.com/2016/10/25/microsoft-launches-cognitive-toolkit-2-0-beta-with-python-support/">reports</a>.</li>
<li>The company also <a target="_blank" rel="nofollow" href="https://www.microsoft.com/cognitive-services/en-us/news">launched</a> 22 cognitive APIs in Azure for vision, speech, language, knowledge, and search. Separately, MSFT <a target="_blank" rel="nofollow" href="http://www.businesscloudnews.com/2016/06/07/microsoft-launches-spark-for-azure-hdinsight-and-pushes-into-consumer-ai/">released</a> its managed service for Spark in Azure HDInsight and continued to <a target="_blank" rel="nofollow" href="https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-whats-new">enhance</a> Azure Machine Learning.</li>
<li>MSFT announced the <a target="_blank" rel="nofollow" href="https://azure.microsoft.com/en-us/blog/azure-n-series-general-availability-on-december-1/">Azure N-Series compute instances</a> powered by NVIDIA GPUs for general availability in December.</li>
</ul>
<p>Other than that, it was a quiet year in Redmond. [<em>ha! — ed.</em>]</p>
<p><em>Catch up on the other roudups from the ML/DL blog here:</em></p>
<ul>
<li><a target="_blank" rel="nofollow" href="https://thomaswdinsmore.com/2016/12/19/the-year-in-machine-learning-part-one/">Part 1</a>: General trends in machine learning and deep learning</li>
<li><a target="_blank" rel="nofollow" href="https://thomaswdinsmore.com/2017/01/02/the-year-in-machine-learning-part-two/">Part 2</a>: Open source machine learning and deep learning projects, including  R, Python, Spark, H20 and Tensorflow</li>
<li><a target="_blank" rel="nofollow" href="https://thomaswdinsmore.com/2017/01/09/the-year-in-machine-learning-part-three/">Part 3</a>: Machine learning and deep learning initiatives from major vendors, including SAS, IBM, Microsoft, Oracle, SAP and Teradata</li>
<li><a target="_blank" rel="nofollow" href="https://thomaswdinsmore.com/2017/01/16/the-year-in-machine-learning-part-four/">Part 4</a>: Startups involved in machine learning and deep learning, including Alpine, Continuum, Databricks, DataRobot, KNIME, and RapidMiner</li>
</ul>
</div>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://blog.revolutionanalytics.com/2017/02/the-year-in-r.html"> Revolutions</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/-EOSM5kl2ss"></a></div>