Becoming a Multiplatform DBA

-
Image result for databases landscape
I wonder if I had started my career as a PostgreSQL DBA or MongoDB DBA, would it be easier for me to acceptthe rapid growth and the variety of data storage solutions and our new reality of the Polyglot persistence?  

Polyglot is the term that came from the Ancient Greek meaning speaking many languages.  Polyglot persistence is about storing your data not only inside SQL Server but in multiple data storage technologies. Whatever suits better your application needs or sometimes even single application component.

Be prepared that tomorrow or next month one of developers will come up with some "other database" which (they will be 100% confident) will serve their application needs better. In  some situations they might be right. My own first natural reaction to those situations is to immediately start searching for "why not" arguments. To keep them in SQL Server. These days I try my best to hold this reaction and allow them to try. If it will work out - everyone will be happy. If not, everyone will learn.

If you push yourself to this change, you soon will realize that the learning curve is not that huge and you are not starting from zero level. Data can be modeled in various forms. Relational form, document form or key and value form but you already understand how to deal with data and how to take care of it. And believe me or not, the concepts are quite similar. 

Of course, your developer is now convinced that he will not need a DBA with his brand new NoSQL technology. Surprise him. You, as a DBA, and Mr. Developer have a strong focus and interest on an entirely different yet equally important aspects of the application infrastructure.
You know that  data must be backed up on some agreed schedule and from time to time it is beneficial to try and restore the backup to make sure it is useful.  Of course if data is important and needs to be persistent. 
After digging for a while you learn that the many open schema systems need someone to manage the schema anyway. Correct data types impact queries in ElasticSearch, Cassandra and some other systems that I have worked with.
Data modelling can be tricky and experienced DBA has a lot of added value there. Some queries might benefit from the pre-aggregations.
Monitoring your server is another huge topic. Monitoring free space, monitoring queries per second, events per second - trust me, Mr. Developer didn’t think about all this. He will be more than grateful to have you as a part of his project. He will re-learn to value DBA.

DBA stays DBA no matter which technology he is managing. Your experience is very valuable and you can append it to any technology you meet on your way.

Comments

Post a Comment

Popular posts from this blog

Unlocking Microsoft Fabric: A Simple Guide when you only have a personal account.

-
Image
" You cannot create a Microsoft capacity using a personal account. Use your organizational account instead." What a disappointment and frustration! I've been trying to set up Microsoft Fabric for a while now and figure out how to work around this error. I use my personal Gmail for an MVP subscription to learn and experiment with Azure services. When I visit app.fabric.microsoft.com and try to use my private Gmail address, I keep getting a similar message: It's frustrating because I'm eager to dive into Fabric and explore its capabilities. I would get the same error trying to set up Azure Data Catalog, PowerBI embedded, and more. What are the different account types and how do they differ? A work or school account is created through Active Directory or other cloud directory services, such as Microsoft 365. On the other hand, a personal account is one that's manually created for individual use, consisting simply of a username and password. After digging into ...
Read more

ETL to ELT journey: Break free your Transformations and discover Happiness and Data Zen

-
Every data integration pipeline consists of 3 stages: Data Extraction (E), Data Transformation (T) and Data Load (L) During the  Data Extraction stage, the source data is read from its origins: transactional databases, CRM or ERP systems or through data scraping from web pages. During the  Data Transformation stage, the necessary modifications are applied to the source data. This includes data filtering, enrichment or merging with existing or other source datasets, data obfuscation, dataset structure alignment or validation, fields renaming and data structuring, according to the canonical data warehouse model.  During the Data Loading stage, the data is stored in the pipeline destination, which could be a staging area, data lake or data warehouse. There are two principal methods for the data integration process: transferring it from where it originated to the destination, where the data will be used for analysis, ETL and ELT. The difference between ETL and ELT pipelines...
Read more

Coding is a rollercoaster of efficiency and eyebrow-raising discoveries.

-
Image
Data Engineers or Developers - many of us love to be gourmet chefs in the kitchen. When it comes to planning and design, we would rather throw all ingredients in the pot and see how it comes out.  Coding without a plan is like assembling a puzzle in a dark room. The result will most probably be unexpected and off the canvas. Whether you follow Waterfall or Agile development strategy, planning and design phases are non-negotiable and are essential to reduce development cycles and redo work. Once upon a time, one data engineer created an amazing piece-of-cake automation pipeline. This masterpiece had very complex logic, pulling data from multiple sources, and merging and persisting the data in a complex, incremental way. When the pipeline started to run successfully and automation flows worked, the data engineer got very excited and considered this development done. A few days later QA engineer found out that the result dataset was never created in the destination. Why has that ...
Read more