<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="http://www.r-bloggers.com/setting-up-a-datascience-server/"></div></div><div><a data-url="http://www.r-bloggers.com/setting-up-a-datascience-server/" data-counturl="http://www.r-bloggers.com/setting-up-a-datascience-server/" data-count="horizontal" data-text="Setting up a DataScience Server" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.networkx.nl/data-science/settingup-datascience-server/"> R – Networkx</a></strong>, and kindly contributed to <a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>After installing multiple software, servers etc. on my  laptop it was overloaded with different tools and running services. When I get a new laptop or it will crash I can start over again installing everything, at home on my iMac I had the same tools and servers. So I decided to setup a DataScience Server with a necessary software and servers.</p>
<p>I know there are a lot easier and faster projects for setting up a datascience server, but I will only install one server with all the necessary software that I can connect to in my home network. Besides that it’s a lot of fun for doing this <img alt="😉" class="wp-smiley" src="https://s.w.org/images/core/emoji/72x72/1f609.png"></p>
<p>In this post I will install a minimal CentOS 7 server with the containing software and servers to start with datascience:</p>
<ul>
<li>Anaconda Python</li>
<li>The Jupyter Notebook</li>
<li>R and Rstudio Server</li>
<li>MongoDB</li>
<li>Splunk®</li>
</ul>
<h3>Minimal install CentOS 7 for setting up a datascience server</h3>
<p>I will not explain it in detail, if you are not familiar with a CentOS installation, there are a lot of manuals to find.<br>
Get a fresh “Minimal ISO” copy of the CentOS 7 image from https://www.centos.org/download/.<br>
Burn it with your favorite software or mount it in your new virtual machine and boot it. I have changed some things like root password, timezone, disk layout etc.</p>
<p>If you have finished the minimal installation we need to install some needed packages.</p>
<pre># yum -y install net-tools ntp wget bzip2</pre>
<p><h4>Configure NTPD</h4>
<p>You can edit your configuration and servers with vi /etc/ntp.conf, the default is good enough for me.</p>
<pre># systemctl start ntpd
# systemctl enable ntpd
# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+dns02.wsrs.net  193.79.237.14    2 u   15   64    1    9.496   -8.290   0.259
*84-245-30-184.d 193.79.237.14    2 u   14   64    1   12.822    3.127   0.173
 ntp4.bit.nl     .PPS.            1 u   13   64    1   11.873    2.484   0.303
 ran.as65342.net 192.36.144.23    2 u   12   64    1   11.692    2.915   0.174</pre>
<p><h3>Installing Anaconda Python</h3>
<p>Anaconda is the leading open datascience platform powered by Python. The open source version of Anaconda is a high performance distribution of Python and R and includes over 100 of the most popular Python, R and Scala packages for datascience. (source: <a target="_blank" shape="rect" href="https://www.continuum.io/">https://www.continuum.io</a>)</p>
<p>Get the latest Linux version from <a target="_blank" shape="rect" href="https://www.continuum.io/downloads">https://www.continuum.io/downloads</a><br>
This package has a total size of 392M</p>
<p>Follow the instructions, I changed the install location:</p>
<pre># wget http://repo.continuum.io/archive/Anaconda2-4.0.0-Linux-x86_64.sh
# bash Anaconda2-4.0.0-Linux-x86_64.sh
....
Anaconda2 will now be installed into this location:
/root/anaconda2

  - Press ENTER to confirm the location
  - Press CTRL-C to abort the installation
  - Or specify a different location below

[/root/anaconda2] >>> /usr/local/bin/anaconda2</pre>
<p>
And I have updated my .bashrc</p>
<pre>Do you wish the installer to prepend the Anaconda2 install location
to PATH in your /root/.bashrc ? [yes|no]

[no] >>> yes</pre>
<p>
After the installation is completed check your path and reinitialise it.</p>
<pre># cat .bashrc
....
# added by Anaconda2 4.0.0 installer 
export PATH=&quot;/usr/local/bin/anaconda2/bin:$PATH&quot;
# echo $PATH 
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin 
# which python 
/usr/bin/python</pre>
<p>
This is the default installed python with CentOS, we need the anaconda python to be default.</p>
<pre># cd ~
# . .bashrc 
# echo $PATH 
/usr/local/bin/anaconda2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin 
# which python 
/usr/local/bin/anaconda2/bin/python 
# python 
Python 2.7.11 |Anaconda 4.0.0 (64-bit)| (default, Dec  6 2015, 18:08:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2 
Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information. 
Anaconda is brought to you by Continuum Analytics. 
Please check out: http://continuum.io/thanks and https://anaconda.org 
>>> quit() 
#</pre>
<p><h3>Installing The Jupyter Notebook</h3>
<p>The Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, machine learning and much more for doing datascience. (source: http://jupyter.org/)</p>
<pre># conda install jupyter</pre>
<p>
The notebook will default run on localhost and you need to start it by hand. I’ve created a systemd unit file to start it automaticly and runs on a different user.</p>
<p>First create the user:</p>
<pre># useradd jupyternb
# su - jupyternb
# mkdir notebooks
# exit</pre>
<p>
Create the system unit file for automatically start on boot.</p>
<pre># vi /usr/lib/systemd/system jupyter-notebook.service</pre>
<p>
In vi press i to enable insert and copy paste below, at the end press &lt;esc> :wq</p>
<pre>[Unit]
Description=The Jupyter HTML Notebook

[Service]
Type=simple
PIDFile=/var/run/jupyter-notebook.pid
ExecStart=/usr/local/bin/anaconda2/bin/jupyter notebook --no-browser --ip=*
User=jupyternb
Group=jupyternb
WorkingDirectory=/home/jupyternb/notebooks

[Install]
WantedBy=multi-user.target</pre>
<p>
Now we created the unit file we only need to reload the inits and enable the system unit file fo the Jupyter Notebook.</p>
<pre># systemctl daemon-reload
# systemctl enable jupyter-notebook
Created symlink from /etc/systemd/system/multi-user.target.wants/jupyter-notebook.service to /usr/lib/systemd/system/jupyter-notebook.service.
# systemctl start jupyter-notebook
# systemctl status jupyter-notebook
● jupyter-notebook.service - The Jupyter HTML Notebook
Loaded: loaded (/usr/lib/systemd/system/jupyter-notebook.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2016-05-16 16:23:41 CEST; 2s ago
Main PID: 2347 (jupyter-noteboo)
CGroup: /system.slice/jupyter-notebook.service
└─2347 /usr/local/bin/anaconda2/bin/python /usr/local/bin/anaconda2/bin/jupyter-notebook --no-browser --ip=*
May 16 16:23:41 dss.home systemd[1]: Starting The Jupyter HTML Notebook...
May 16 16:23:41 dss.home systemd[1]: Started The Jupyter HTML Notebook.
May 16 16:23:42 dss.home jupyter[2347]: [W 16:23:42.101 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
May 16 16:23:42 dss.home jupyter[2347]: [W 16:23:42.101 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using authentication. This is highly insecure... recommended.
May 16 16:23:42 dss.home jupyter[2347]: [I 16:23:42.108 NotebookApp] Serving notebooks from local directory: /home/jupyternb/notebooks
May 16 16:23:42 dss.home jupyter[2347]: [I 16:23:42.109 NotebookApp] 0 active kernels
May 16 16:23:42 dss.home jupyter[2347]: [I 16:23:42.109 NotebookApp] The Jupyter Notebook is running at: http://[all ip addresses on your system]:8888/
May 16 16:23:42 dss.home jupyter[2347]: [I 16:23:42.109 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
Hint: Some lines were ellipsized, use -l to show in full.</pre>
<p>
If the daemon started you can connect to http://your-server-name-here:8888 and shows your home screen.</p>
<p><img sizes="(max-width: 1159px) 100vw, 1159px" srcset="http://i2.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-16-at-16.24.42.png?resize=456%2C85 1159w, http://www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-16-at-16.24.42-300x56.png 300w, http://www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-16-at-16.24.42-768x142.png 768w, http://www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-16-at-16.24.42-1024x190.png 1024w" alt="datascience - jupyter" class="wp-image-1293" src="http://i2.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-16-at-16.24.42.png?resize=456%2C85" data-recalc-dims="1"></p>
<p>For detail configuration, like encryption and authentication you can check the official Jupiter documentation here <a target="_blank" href="http://jupyter-notebook.readthedocs.io/en/latest/">http://jupyter-notebook.readthedocs.io/en/latest/</a></p>
<h3>Installing R and Rstudio</h3>
<p><strong>R</strong> is a language and environment for statistical computing and graphics. R provides a wide variety of statistical (linear and nonlinear modelling, classical statistical tests, time-series analysis, classification, clustering, …) and graphical techniques, and is highly extensible. (source: https://www.r-project.org/)</p>
<p><strong>RStudio</strong> is an integrated development environment (IDE) for R. It includes a console, syntax-highlighting editor that supports direct code execution, as well as tools for plotting, history, debugging and workspace management. (source: https://www.rstudio.com/)</p>
<p>For installing R we need to install the Extra Packages for Enterprise Linux (EPEL) repo.</p>
<pre># yum -y install epel-release</pre>
<p>
Refresh the repo</p>
<pre># yum repolist</pre>
<p>
Now we can install R</p>
<pre># yum -y install R</pre>
<p>
This will install about 390 packages, so get a cup of coffee <img alt="🙂" class="wp-smiley" src="https://s.w.org/images/core/emoji/72x72/1f642.png"></p>
<pre>Install  1 Package (+389 Dependent packages) 
Total download size: 337 M 
Installed size: 819 M</pre>
<p>
If R is installed we can install Rstudio-server<br>
I have used the instructions from <a target="_blank" shape="rect" href="https://www.rstudio.com/products/rstudio/download-server-2/">https://www.rstudio.com/products/rstudio/download-server-2/</a> This package has a total size of 280M</p>
<pre># yum install --nogpgcheck https://download2.rstudio.org/rstudio-server-rhel-0.99.896-x86_64.rpm</pre>
<p>
If everything went fine you can connect to your server with the following URL,  and you will see a sign in screen.</p>
<p>http://your-server-name-here:8787</p>
<p><img sizes="(max-width: 311px) 100vw, 311px" srcset="http://i1.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-16-at-15.45.51.png?resize=311%2C271 311w, http://www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-16-at-15.45.51-300x261.png 300w" alt="datascience - rstudio" class="wp-image-1294" src="http://i1.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-16-at-15.45.51.png?resize=311%2C271" data-recalc-dims="1"></p>
<p>See the official <a title="Getting Started" target="_blank" shape="rect" href="http://www.rstudio.com/ide/docs/server/getting_started">Getting Started document</a> for information configuring and managing the server.</p>
<h3>Installing MongoDB</h3>
<p>MongoDB is an open-source document database that provides high performance, high availability, and automatic scaling. (source: https://www.mongodb.com/)</p>
<p>I have MongoDB installed as described on the MongoDB site you can find it <a target="_blank" href="http://docs.mongodb.org/manual/installation/">here</a>. Why should I write it again :). I installed CentOS 7, and therefore I used the Red Hat installation guide. To determine which platform you run, check it with the following command on CentOS.</p>
<pre># getconf LONG_BIT
64
#</pre>
<p>
As you can see we are running on a 64-bit platform, thats fine because the installation guide only supports 64-bit systems <img alt="😀" class="wp-smiley" src="https://s.w.org/images/core/emoji/72x72/1f600.png"></p>
<p>Disable SELinux by setting the SELINUX setting to disabled in /etc/selinux/config.</p>
<pre>SELINUX=disabled</pre>
<p>
You can check if MongoDB is running and listen on the tcp port.</p>
<pre># ps aux | grep -i mongo
mongod    1138  0.5  8.3 1502560 243764 ?      Sl   Jul04  88:26 /usr/bin/mongod -f /etc/mongod.conf
# netstat -atnp | grep -i mongo
tcp        0      0 0.0.0.0:27017           0.0.0.0:*               LISTEN      1138/mongod</pre>
<p><h3>Installing Splunk®</h3>
<p>You see servers and devices, apps and logs, traffic and clouds. We see data—everywhere. Splunk<sup>®</sup>offers the leading platform for Operational Intelligence. It enables the curious to look closely at what others ignore—machine data—and find what others never see: insights that can help make your company more productive, profitable, competitive and secure. What can you do with Splunk?<br>
Just ask.</p>
<p><img alt="stock_notes-1" class="wp-image-1449" src="http://i1.wp.com/www.networkx.nl/wp-content/uploads/2016/05/stock_notes-1.png?resize=48%2C48" data-recalc-dims="1"><strong>For downloading Splunk® you need to create an account on www.splunk.com.</strong></p>
<p>You can get your free Splunk® Enterprise here: https://www.splunk.com/en_us/download/splunk-enterprise.html</p>
<p>We need to choose Linux, than the 64-bits, stop the download because we are going it to download with the wget command. On the right side we can find “Got wget?”, press that and copy the URL into your linux console to download the rpm package.</p>
<p><a target="_blank" href="http://i1.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-20-at-21.48.32.png"><img sizes="(max-width: 306px) 100vw, 306px" srcset="http://i1.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-20-at-21.48.32.png?resize=306%2C239 306w, http://www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-20-at-21.48.32-300x234.png 300w" alt="datascience-splunk-download" class="wp-image-1443" src="http://i1.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-20-at-21.48.32.png?resize=306%2C239" data-recalc-dims="1"></a></p>
<p>If Splunk® is downloaded than install it with rpm</p>
<pre># rpm -Uvh splunk-6.4.1-debde650d26e-linux-2.6-x86_64.rpm
warning: splunk-6.4.1-debde650d26e-linux-2.6-x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 653fb112: NOKEY
Preparing...                          ################################# [100%]
useradd: cannot create directory /opt/splunk
Updating / installing...
   1:splunk-6.4.1-debde650d26e        ################################# [100%]
complete
#</pre>
<p>
After install go the the directory where Splunk® is installed and start it. We accept the license directly with the start.</p>
<pre># cd /opt/splunk/bin
# ./splunk start --accept-license

This appears to be your first time running this version of Splunk.
Copying '/opt/splunk/etc/openldap/ldap.conf.default' to '/opt/splunk/etc/openldap/ldap.conf'.
Generating RSA private key, 1024 bit long modulus
.......++++++
..++++++
e is 65537 (0x10001)
writing RSA key

Generating RSA private key, 1024 bit long modulus
........++++++
.........++++++
e is 65537 (0x10001)
writing RSA key

Moving '/opt/splunk/share/splunk/search_mrsparkle/modules.new' to '/opt/splunk/share/splunk/search_mrsparkle/modules'.

Splunk> Winning the War on Error

Checking prerequisites...
	Checking http port [8000]: open
	Checking mgmt port [8089]: open
	Checking appserver port [127.0.0.1:8065]: open
	Checking kvstore port [8191]: open
	Checking configuration...  Done.
		Creating: /opt/splunk/var/lib/splunk
		Creating: /opt/splunk/var/run/splunk
		Creating: /opt/splunk/var/run/splunk/appserver/i18n
		Creating: /opt/splunk/var/run/splunk/appserver/modules/static/css
		Creating: /opt/splunk/var/run/splunk/upload
		Creating: /opt/splunk/var/spool/splunk
		Creating: /opt/splunk/var/spool/dirmoncache
		Creating: /opt/splunk/var/lib/splunk/authDb
		Creating: /opt/splunk/var/lib/splunk/hashDb
	Checking critical directories...	Done
	Checking indexes...
		Validated: _audit _internal _introspection _thefishbucket history main summary
	Done
New certs have been generated in '/opt/splunk/etc/auth'.
	Checking filesystem compatibility...  Done
	Checking conf files for problems...
	Done
	Checking default conf files for edits...
	Validating installed files against hashes from '/opt/splunk/splunk-6.4.1-debde650d26e-linux-2.6-x86_64-manifest'
	All installed files intact.
	Done
All preliminary checks passed.

Starting splunk server daemon (splunkd)...  
Generating a 1024 bit RSA private key
........++++++
....++++++
writing new private key to 'privKeySecure.pem'
-----
Signature ok
subject=/CN=dss.home/O=SplunkUser
Getting CA Private Key
writing RSA key
Done
                                                           [  OK  ]

Waiting for web server at http://127.0.0.1:8000 to be available.... Done


If you get stuck, we're here to help.  
Look for answers here: http://docs.splunk.com

The Splunk web interface is at http://your-server-name-here:8000</pre>
<p>
You can now connect to http://your-server-name-here:8000</p>
<p><a target="_blank" href="http://i1.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-20-at-22.03.35.png"><img sizes="(max-width: 559px) 100vw, 559px" srcset="http://i1.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-20-at-22.03.35.png?resize=456%2C299 559w, http://www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-20-at-22.03.35-300x197.png 300w" alt="datascience-splunk" class="wp-image-1445" src="http://i1.wp.com/www.networkx.nl/wp-content/uploads/2016/05/Screen-Shot-2016-05-20-at-22.03.35.png?resize=456%2C299" data-recalc-dims="1"></a></p>
<p>Configure Splunk® to start automatically</p>
<pre># cd /opt/splunk/bin
# ./splunk enable boot-start</pre>
<p>
Well thats it, you we can now start to gather some data and doing some datascience.</p>
<p>If you have some questions, follow me on Twitter or mail me, in the footer you can find my contact information.</p>
<p>Good luck!</p>
<p>The post <a target="_blank" rel="nofollow" href="http://www.networkx.nl/data-science/settingup-datascience-server/">Setting up a DataScience Server</a> appeared first on <a target="_blank" rel="nofollow" href="http://www.networkx.nl/">Networkx</a>.</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="http://www.r-bloggers.com/setting-up-a-datascience-server/"></div></div><div><a data-url="http://www.r-bloggers.com/setting-up-a-datascience-server/" data-counturl="http://www.r-bloggers.com/setting-up-a-datascience-server/" data-count="vertical" data-text="Setting up a DataScience Server" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.networkx.nl/data-science/settingup-datascience-server/"> R – Networkx</a></strong>.</div>
<hr>
<a rel="nofollow" href="http://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="http://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="http://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="http://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="http://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="http://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="http://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="http://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="http://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="http://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="http://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="http://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="http://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="http://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="http://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="http://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="http://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="http://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="http://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/EPCZQL02Tr0"></a></div>