Pages

Showing posts with label AWS. Show all posts
Showing posts with label AWS. Show all posts

2016-02-03

R on RHEL 7 in AWS

Tux, the Linux penguin

R on RHEL 7 in AWS


If I have start to run some software, and it ends up being less than straightforward, I always try to make a checklist of the procedure for how to build it. 

Recently I was testing the performance of some R code. 


I launched an AWS server Red Hat Linux 7 (RHEL 7) and found R was not installed. 

Ok, so this is a simple yum install, right? 

Not so fast. 

Here is everything I needed to do, in order to get R and rstudio running on my server: 

sudo su -c 'rpm -Uvh http://download.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm'
sudo yum update
wget http://mirror.centos.org/centos/7/os/x86_64/Packages/lapack-devel-3.4.2-5.el7.x86_64.rpm
sudo yum locallinstall lapack-devel-3.4.2-5.el7.x86_64.rpm
wget http://mirror.centos.org/centos/7/os/x86_64/Packages/blas-devel-3.2.2-5.el7.x86_64.rpm
sudo yum locallinstall blas-devel-3.2.2-5.el7.x86_64.rpm
wget http://mirror.centos.org/centos/7/os/x86_64/Packages/texlive-epsf-svn21461.2.7.4-38.el7.noarch.rpm
sudo yum locallinstall texlive-epsf-svn21461.2.7.4-38.el7.noarch.rpm
wget http://mirror.centos.org/centos/7/os/x86_64/Packages/texinfo-tex-5.1-4.el7.x86_64.rpm
sudo yum locallinstall texinfo-tex-5.1-4.el7.x86_64.rpm
sudo yum install R

wget https://download2.rstudio.org/rstudio-server-rhel-0.99.491-x86_64.rpm
yum localinstall --nogpgcheck rstudio-server-rhel-0.99.491-x86_64.rpm
service rstudio-server
service rstudio-server status

All of this came about from google searches on stackoverflow, and other places where I found hiccup after hiccup just getting everything installed. 

These commands will save you some time. 


2015-05-07

PloyGlot Management

PloyGlot Data Management

 

What is the data stored in? 


A traditional database administrator is familiar with a set of utilities, SQL, and a stack of tools specific to the RDBMS that they are supporting.


SQL Server, SQL Server Analysis Service, SQL Server Integration Services, SQL Server Reporting Services. SQL Server Management Studio.

Not to mention the whole disk partitioning, SAN layout, RAID level and such that they need to know.

Oracle has it's own set of peculiarities as does DB2, MySQL, and others.

There is a new kid on the block.

NoSQL.

Martin Fowler does a phenomenal job introducing the topic of NoSQL in this talk 

While most people do not get into too many of the details about how their data is stored and structured, the world of #DataOps the nature of how and where the data is stored becomes incredibly important.

In any given day managing a PloyGlot environment the command line tools used could be sqlplus, bcp, sqlplus,hdfs,spark-shell, xd-shell.

Each of these data storage engines have different requirements. The manner in which databases are clustered when moving between RDBMS, and the NoSQL platforms are quite different.

Putting Hadoop or Cassandra on a SAN should require due thought before doing so.

Likewise creating an environment with isolated disks for an Oracle Cluster may not be the correct solution.


Managing a PolyGlot environment, is by itself a challenge. Sometimes this requires a team,
sometimes this requires a lightly shaded purple squirrel that is equally at home at the command line, the SQL prompt,a REPL, a console, a white board, an AWS management browser, or a management console like Grid control, Ops Center, or Cloudera Manager.

Working with this variety of data, and the variety of the types of teams, and people that need access to this data, or even a subset of the data requires it's own level of understanding of the data, data management, and how to make the data itself work to contribute to the bottom line of the organization.

Are your purple squirrels only on the data science team? Probably not.