Help and Support

Back to Help and Support Topics

Data Quality

Data Quality Framework

Introduction

This overview of the Data Quality Framework is intended to provide high level support and direction to better understand the intent and purpose of the data quality framework developed by the Road Efficiency Group .
Any relevant current industry guidance and case studies have been referenced, where they provide more detailed assistance.

What are the Data Quality Framework and Reports

The data quality of each Road Controlling Authority (RCA) is assessed annually against a suite of data quality metrics interrogating a cross-section of the data used to support asset management activities.

Each RCA’s results are compared against an expected standard and the distribution of all RCAs across New Zealand. RCAs can compare themselves against the rest of the RCAs. Individual State Highways regions can also be compared against the other State Highway regions. The results of the annual data quality reports have several uses. The main ones are:

  • At a network level to support the identification of data quality improvement opportunities enabling evidence-based decision-making
  • Annual monitoring and reporting on data quality at a national level supporting the identification and development of reference and guidance material to support the sector
  • For Technical Auditors to reference and for Investment Advisors to consider alongside funding requests.

The intent is for the results to flag potential issues with how the data is being collected, managed and maintained. Further analysis is required on those results not at the expected standard to determine what action (if any) is required.

ANY IMPROVEMENT ACTION SHOULD BE TO ADDRESS THE ROOT CAUSE, RATHER THAN SIMPLY CORRECTING THE RECORD(S).

Why have the metrics been selected and why are they important to me?

Good quality data is necessary to effectively and efficiently deliver services and manage our assets. The intention is for the results to identify opportunities for improvement in the way both an individual network and
the sector collects, manages and uses data to support our investment planning and decision-making processes.

Each metric has been selected to test a cross-section of the data looking at data quality in terms of accuracy, completeness and timeliness. The metrics have been selected as they:

  1. Have been considered an important input into our asset management decision-making processes, and
  2. Interrogate data that underpins the ONF performance measures.

It is necessary to have confidence in the data when considering the results at an individual level and in the comparative reporting against peers.

  

 

Key Points

  • Good quality data is necessary to effectively and efficiently deliver services and manage our assets.
  •  The data quality of each RCA is assessed annually against a suite of data quality metrics interrogating a cross-section of the data used to support our asset management activities.
  • The intent is for the results to flag potential issues with how the data is being collected, managed and maintained.
  • Each metric has been selected to test a cross-section of the data looking at data quality in terms of accuracy, completeness and timeliness.
  • The result for each metric is considered against grade thresholds providing an indication of the level of quality.
  • The framework should not be used to replace quality control/assurance processes required with updates to the asset information system.
  • The focus of any improvement action by an RCA should be at the root cause, and not simply correcting the record(s).
  • The annual national and RCA results are used to inform data improvement strategies and develop supporting programmes.
 

The data quality framework and metrics interrogate a targeted sample of the data that supports our asset management decision making.

 

The Data Quality Metric Framework

The framework interrogates a TARGETED SAMPLE of each RCA's asset data. It is not a complete data quality check or validation of data managed and maintained by each RCA. IT SHOULD NOT BE USED TO REPLACE QUALITY CONTROL/ASSURANCE CHECKS required as part of adding, updating and deleting records in your asset database.

Each metric is associated with a cross-section of the data that supports our asset management processes, including a subset that underpins the ONF performance measure results. The structure of the report is to group the metrics into data categories, level of importance and the data quality dimensions tested. The reported results at a metric level are aggregated up to these category and dimension levels.

Score

The data quality report has an overall Score. This is a weighted calculated score based on the importance of each metric applicable to an individual RCA. The maximum Score of 100 is achievable by having all metrics at the expected standard level. The current minimum achievable Score is 14. More detail on the SCORE CALCULATION (PDF) can be found in the Relevant Links below and in the resources on the REG website.

Quality Dimensions

The metrics test the quality of a specific set of data against the quality dimensions of accuracy, completeness or timeliness. The table below defines each of these dimensions.

Dimension Definition
Accuracy The data reflects the real-world object or event
Completeness  The data is comprehensive for where it is intended to be used
Timeliness The data is available when expected and needed

 

The DATA QUALITY DIMENSIONS OVERVIEW (PDF) provides a more detailed view of the three data quality dimensions.

  

Importance

Each metric has been assigned a level of importance based on its likely impact on our asset management decision-making processes. There are three categories; high, moderate and low.

Expected Standard

The results for each metric is considered against grade thresholds proving and an indication of the level of quality. There are three grades; at the expected standard, minor issues and major issues.

Data Categories

There are six categories, each with one or multiple sub-categories interrogating a specific data set via a varying number of metrics:

  • Network
  • Asset Inventory
  • Maintenance Activity
  • Condition
  • Demand/Use
  • Crash

Metrics Not Relevant to a Network

Some metrics will not apply to all networks. For example, any metrics relating to the rural network on an urban only network. These metrics have a result of "NA" when not applicable and do not contribute to the aggregated results at a dimension or category level or the Score.

What action should be taken for results not at the expected standard?

The results are intended to flag potential quality or process issues with an RCA's data process or practices. As you read the reports, think about why the result may be as it is. Consider what can be done to improve the data for those metrics where the result is not in the "green zone" (the expected standard) and is low compared to the sector.

 

Conclusion

The data quality framework and metrics interrogate a targeted sample of the data that supports our asset management decision-making. Each metric has been selected to test a cross-section of the data looking at data quality in terms of accuracy, completeness and timeliness. It should not be used to replace quality control/assurance processes required with updates to the asset information system.

 

 

Relevant Links

REG Data Quality project (external link)

Data quality dimensions (help page)

Understanding data quality results (help page)

Calculating the data quality score (help page)

Metric Library