Maria's journey in the data fields

Imagine having a dancing party for your data. Everyone in harmony waltzing, stepping on their partner's toes from time to time.  Distributed data management is no less amusing. It's chaotic, occasionally hair-raising but with the right approach can even be perfect.  A central huge data warehouse is a struggle to scale efficiently and hard to innovate. There is no clear ownership of the data domains and it is a single point of failure. During peak usage times, data access and processing can be slow. Even implementing updates or upgrades can be quite complex and time-consuming. Centralized databases are attractive targets for cyberattacks and successful breaches can compromise a large amount of sensitive data.  As an alternative to a centralized Data Warehouse, data can be owned and managed by the domains, producing it. When considering a decentralized approach, we need to make sure there is a self-serve data infrastructure platform that allows different domains or teams to...

When developing a modern Data Platform Layer, one of the main decisions is whether to opt for centralized or decentralized data access architecture.  There is no “one-fits-all” solution, both have advantages and disadvantages. A Centralized Data Access Architecture would usually mean duplicating data from the operational layer into the analytical layer and applying various transformations to data to support and speed up data analytics. Operational online transaction processing Layer , where all microservices and their operational databases are located. Analytical Data Layer , where we would have data lakes that support data Scientists' work and a data warehouse, that supports Business Intelligence. Transformations, ETL or ELT data pipelines , which are moving data from the operational layer into the analytical layer. I f we opt for a Centralized Data Access architecture, what would be the benefits and the drawbacks? CONSISTENCY : consolidating data into a central location can...

  Azure CosmosDB is a modern distributed data store that can handle any data volume, any data velocity ( data arrival speed) and any data variety (different types of data). CosmosDb requires minimal setup and management efforts. It is very easy to integrate CosmosDB into your existing data infrastructure using various APIs that can either mimic your existing data management systems, like MongoDB, PostgreSQL or Cassandra and provide you with under 10s latency from anywhere, 99.999% availability and instant scalability. From the cost perspective, storage costs and utilization costs are almost the same regardless of which API you are planning to use. There is neither an autoscale nor serverless option for PostgreSQL API. Serverless NoSql API,  Serverless   Gremlin API,  Serverless  MongoDB API,  Serverless  Cassandra API and Serverless Table API are available as only as Single Region write architecture. If you are interested in Multi-region write clu...