Changes for page The data quality dimensions

Summary

• Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

...	...	@@ -4,38 +4,5 @@
4	4
5	5	[[image:image-20220705171127-1.png||height="350" width="541"]]
6	6
7		-DAMA Data quality dimensions (Lean-Data, 2022)
	7	+**Figure 16:** DAMA Data quality dimensions (Lean-Data, 2022)
8	8
9		-==== 1. Accuracy ====
10		-
11		-Accuracy is achieved when data reflects reality. Stored data must represent real information. A likely place for errors to occur is during data collection. Simple checks can be carried out to see if data looks accurate, for example if the height of a person recorded as 15 cm it is clearly wrong. To this end, the units of data should be checked carefully.
12		-
13		-Data cleaning process need to be included in this dimension. Data cleaning is the process of fixing or removing incorrect, corrupted, incorrectly formatted, duplicate, or incomplete data within a dataset. It is particularly important when combining multiple data sources.  In those cases, data can be highly likely be duplicated or mislabelled.
14		-
15		-Furthermore, real-world information can change over time – for example a person’s phone number or home address. This makes accuracy quite challenging to monitor. This type of data should be regularly checked and updated where necessary. High data accuracy produces trustworthy analytics. It also leads to correct reporting and confident decision-making.
16		-
17		-==== 2. Completeness ====
18		-
19		-Data is considered complete when all the data required for a particular use is present and available to be used. It’s not about ensuring 100% of data fields are complete, but rather is the determination of what data is critical and what is optional. It needs to be identified if the data gap is crucial for the analysis or not.
20		-
21		-For example, if a carbon assessment data needs to be collected, a piece of data required to make the assessment complete would be the total annual carbon emissions caused by the transport sector. Without this, the assessment could not occur, and the data is incomplete. However, if the city were unable to provide data about the total average rainfall, it would not matter as this will not have an impact on a carbon assessment and the dataset could still be considered complete.
22		-
23		-Completeness applies not only at the data item level but also at the record level in terms of assessing whether a data set is comprehensive. This aids the understanding of whether missing data will have an impact on the reliability of insights which are gathered from it.
24		-
25		-Completeness is not the same as accuracy as a full data set may still have incorrect values. Information can be complete but inaccurate.
26		-
27		-==== 3. Uniqueness ====
28		-
29		-Uniqueness measures the number of duplicates. Data is unique if it appears only once in a data set. A record can be a duplicate even if it has some fields that are different. For example, two air pollution sensor records may have different addresses and ID numbers, but if they both refer to the same sensor then there is duplication. Duplication is a particular risk when combining data sets. Unique records build trust in the data.
30		-
31		-==== 4. Consistency ====
32		-
33		-Consistency is achieved when data values do not conflict with other values within a record or across different data sets. This involves both formatting consistency and consistency of underlying information.  For example, the inconsistency formatting (e.g., format of date such as 1st of May and 01/05) can be solved relatively easily. However, if an event record puts the date of event as May 1st, and another record shows it as June 1st, you may first need to assess the accuracy of data from both sources. It may require verification with another source. Consistent data improves the ability to link data from multiple sources, supplementing and increasing the utility of both data sets.
34		-
35		-==== 5. Timeliness ====
36		-
37		-Timeliness indicates whether the data is available when expected and needed. Timeliness is important as it adds value to information that is particularly time sensitive. Data quality may diminish over time. For example, someone might provide a correct home address when the data is captured, but if the same individual changes their address these data items will become outdated.
38		-
39		-==== 6. Validity ====
40		-
41		-Validity is defined as the extent to which the data conforms to the expected format, type, and range. For example, an email address must have an ‘@’ symbol and a month should be between one and twelve. Having valid data means that it can be used with other sources. It also helps to promote the smooth running of automated data processes. Important to note is that valid values may not necessarily be correct; in a system storing the colour of people’s eyes, a value of ‘pale’ is invalid. A value of ‘blue’ for a person’s eye colour would be valid but incorrect if it is brown.