Pages

2015-04-24

#DataOps

#DataOps: Oh, that's what they call what I do?

I just read a few amazing articles:

Agile DataOps
DataOps a New Discipline

Trying to explain to people what I do becomes tedious at times.


Yes, I can build a Cassandra Cluster, or an Oracle Cluster, or a Cloudera Cluster. Each of those clusters has its own challenges and rewards. But once the Cluster is built and we start putting data into it, what do you want to do with that data?  How do you want the data organized? Is your data model correct? Do you have a Data model? Do you know how big the cluster or database is going to get? Have you done a Volumetric calculation? Is your budget big enough to allow for fail-over? Downtime? Maintenance? Disaster recover practice?

Data Scientist, Business Analyst, Executive Analyst, Business User, Project Manager.


Each of these specialties have their own unique challenges. Being The Data Guy in an organization, requires the ability to at least communicate effectively to all of these specialists.

Here are some of the things we do:

Oracle Certifications, Microsoft Certifications, Red Hat Certifications. Statistics, ETL, Informatica, Data Stage, Pentaho,  Data mining, R, Python, Scala, Spark, Data modeling, SQL, CQL, Hive, Hadoop, Impala, Map Reduce, Spring, Data Movement, Data Plumbing.


Backups, Restores, Performance checks, SQL tuning, Code tuning to match the data platform. These are all the day in and day out life of Data Operations. Data modeling, ERWin, ERStudio, JSON, XML, Column stores, Document stores, Text mining, Text processing and storage. RAID, SAN, NAS, local storage, spindles, SATA drives. Jobs, batches, Schedulers, 3:00 a.m. wake-up calls, alerts, on-call troubleshooting.


Now it is all summed up in a new hashtag: #DataOps

What is it that you do?

Just as DevOps is important to make our organizations Agile and responsive to the needs of the business users, so to does DataOps have it's unique and peculiar take on impacting the business.

Data Scientist on the left, Business Analyst on the right, Developers behind us, and Project management ahead of us. Standing on the infrastructure that we work together with DevOps to create, implement, and manage.

This is #DataOps.

Are you up for it?


No comments:

Post a Comment