Data Quake
Dave Wells have just gave this great definition that clearly describes what's happening in the data management world during the recent years.
I am greatly enjoying Dave’s session today at Enterprise Data World summit and couldn't resist writing down the summary.
Everything that we did in the last decade becomes wrong now. We have used to believe that application logic can run faster and do better if it sits inside the database layer. Now this architecture is being considered a wrong choice. Same goes for data normalization or strong schema. Some people even say that data warehouses are dead.
We need to rethink everything. Data schema used to be defined during the design phase. Now we define schema-on-read, after the data have been persisted. Good news - I have always believed that and Dave have just mentioned - there is no schema-less data. Despite the fact that we do not get to design the schema anymore, for Big Data we need to understand the schema from the existing data and define a separate schema models for each use case. Same goes to Data Quality rules that aren't generic anymore. Need to figure out the data quality rules for each use case. Even data governance is changing. We cannot govern the data anymore. It's getting out of the boundaries, out of the control. We can only govern what people do with the data.
In the modern data world we have more data sources and more types of data, we have much more ways to organize and store data, more uses for data and more data consumers that require fast and on-demand data delivery.
Data management world is changing.
The age of Data Warehousing and BI have came to an end.
We are now in the age of Big Data and Data Lakes but they are slowly going away as well.
And our future is approaching fast, bringing with it Data Catalogs, Data Hubs and Data Fabric concepts.
I am myself on the way to figure out what is it and how it all makes sense.
What a great speaker, awesome session and tons of learning ahead of me.
Yours,
Maria
Comments
Post a Comment