Understanding the Pillars of Data Quality

-

Imagine, you are baking a cake. You have all the ingredients except eggs. Of course, you could improvise but most probably instead of the moist chocolate cake, you end up with a dry science experiment. Incomplete data leaves everyone unsatisfied.

During the decision-making party, every data quality dimension is an important guest with a unique vibe:

Good Data should be Complete, when all data attributes, that describe data in its fullness, are present as a part of your data. This guest keeps the party snacks stock full and makes sure no one gets hungry. Incomplete data leads to half-baked insights.

Good Data should be Accurate. This means that the data correctly describes its objects and accurately reflects the reality. This party guest is a perfectionist, checking that the playlist is perfectly chosen.

Good Data should be Timely. This means that the data is fresh. No one wants to eat last week's sushi. It's not only unappetizing; it is downright risky. This guest makes sure everyone at the party shows up on time - no fashionably late nonsense.

Good Data should be Consistent. This means that data in the different datasets is the same. Having inconsistent data chaos is like measuring weight in kilograms and pounds interchangeably. This party guest makes sure that everyone sticks to a dress code - no pyjama rebels allowed. 

Good Data should be Relevant. This means that the data is useful for the specific use case. Imagine making spaghetti but reading a pancake recipe. Irrelevant data makes noise, not providing any value to your analysis.

Good Data should be Valid. Validity is about following the rules. This means that data conforms to the required format and standards, and fits within defined rules and constraints. The invalid data format is like pouring orange juice into your soup.

Good Data should be Unique, ensure no duplicates exist and that every record is one-of-a-kind. It's like two guests at the party saying exactly the same thing, which is pointless and annoying.

Data Party, where guests don't vibe together will lead to incorrect conclusions and bad strategic choices.





Comments

Post a Comment

Popular posts from this blog

Unlocking Microsoft Fabric: A Simple Guide when you only have a personal account.

-
Image
" You cannot create a Microsoft capacity using a personal account. Use your organizational account instead." What a disappointment and frustration! I've been trying to set up Microsoft Fabric for a while now and figure out how to work around this error. I use my personal Gmail for an MVP subscription to learn and experiment with Azure services. When I visit app.fabric.microsoft.com and try to use my private Gmail address, I keep getting a similar message: It's frustrating because I'm eager to dive into Fabric and explore its capabilities. I would get the same error trying to set up Azure Data Catalog, PowerBI embedded, and more. What are the different account types and how do they differ? A work or school account is created through Active Directory or other cloud directory services, such as Microsoft 365. On the other hand, a personal account is one that's manually created for individual use, consisting simply of a username and password. After digging into ...
Read more

The backbone your data pipelines have been waiting for.

-
Image
Kafka isn’t just a buzzword—it’s the backbone your data pipelines have been waiting for. Who new that messaging system, like Apache Kafka should hold a central place in a Data Engineer toolbelt.  Apache Kafka is a low-latency distributed data streaming platform for real-time data processing. Kafka can handle large volumes of data and is very helpful for distributed data integration projects. Top 2 reasons why you might need Kafka in your Data Integration architecture 1. Support multiple destinations by decoupling data producers and data consumers. Data in source will be processed only once, which lowers an overall cost in consumption-based data producer databases and we can add new/change existing destinations without changing the extraction components. 2. Ability to deal with massive amounts of data , supports high throughput and scalability. Decoupling pipeline extract and load stages is an important Data Integration principle and can improve pipeline flexibility, extract and loa...
Read more

Replication fun: setting NFR on all objects

-
If you ever tried to set up replication on the large environment with numerous servers, hundreds of databases then you, for sure, spend some time searching/writing scripts to set flag NOT FOR REPLICATION on thousands of identities, constraints and triggers. I want to share here my scripts and I hope it will save some readers hours of headache. Feel free to use them, just add comments below - there is nothing better than feedback. Here we go. ******************************************************************************** 1. Identities. The resutset is a list of identity columns on all tables in all databases where NOT FOR REPLICATION option is not set and fixing script: DROP TABLE #IdentityNFR CREATE TABLE #IdentityNFR (ServerName varchar(250), Dbname varchar(250),TableName varchar(250), IdentityColName varchar(250), IsNFR varchar(10),FixingScript varchar(4000)) exec sp_MSforeachdb ' IF ''?'' NOT IN (''tempdb'',''master'',''mo...
Read more