Font Size

Evaluating Data Quality


Computer/Internet/Software Articles
Submit Articles   Back to Articles

Poor data quality costs businesses vast amounts of money and leads to breakdowns in the supply chain, poor business decisions, and inferior customer relationship management. Defective data also hampers efforts to meet regulatory compliance responsibilities. Data quality encompasses more than finding and fixing missing or inaccurate data. It means delivering comprehensive, consistent, relevant, fit-for-purpose, and timely data to the business regardless of its application, use, or origin.

Ensuring data quality is a challenge for most organizations partly because they may not be fully aware of their own data quality levels. Without this information, they cannot know the full business impact of poor or unknown data quality, or how to begin addressing it. Poor data quality is a costly issue. According to Pricewaterhouse Coopers Global Data Management Survey, 75 percent of those surveyed reported significant problems as a result of defective and fragmented data, more than 50 percent had incurred extra costs due to the need for internal reconciliations, 33 percent had been forced to delay or scrap new systems, 33 percent had failed to bill or collect receivables, and 20 percent had failed to meet a contractual or service-level agreement.

Put simply, to realize the full benefits of investments in enterprise computing systems, your organization must have a detailed understanding of the quality of its data, where quality is poor, how to clean it, and how to keep it clean---and must be able to deliver it to the business on the front lines with data integration. By making data quality and data integration a strategic priority, your organization better positions itself to streamline operations, grow revenue, keep costs in check, and achieve long-term competitive advantage.

What is Data Quality? Data quality is a broad umbrella term for the accuracy, completeness, consistency, conformity, and timeliness of a particular piece or set of data and for how data enters and flows through the enterprise. Different organizations will have different definitions and requirements for data quality, but it ultimately boils down to data that is "fit for purpose". The data needs to be of high enough quality to do the job at hand. If your business is risk management, then your data will probably need to be much higher quality than data for a mass marketing business. Organizations may not be aware of the full business impact of poor or unknown data quality if they define the term too narrowly.

To evaluate your organization's data quality, consider the following metrics: * Completeness--- Is all necessary data present? *Conformity--- What data is stored in a nonstandard format? *Consistency--- What data values give conflicting information? *Accuracy--- Does the data accurately represent reality or a verifiable source? * Duplication--- What data records are repeated? *Integrity--- What data is missing important relationship linkages? *Timeliness--- Does the age of the data meet user requirements?

In the past, data quality was primarily associated with cleansing and matching name and address data with a focus on contact efficiency and the identification of duplicates and relationships for marketing and customer service applications. However, the more pressing business issues of today, such as regulatory compliance, corporate accountability, and streamlining the supply chain, require cracking the much harder nut of non-customer-oriented data quality.

Large organizations have a wide variety of data in addition to customer name and address information. Product, material, inventory, asset, financial, and vendor data are abundant in the enterprise and are an ever-present source of information quality problems. Increasingly, the initiatives of businesses and government agencies call for access to a comprehensive enterprise-level information source that draws data from a wide range of internal systems as well as from external sources. This enterprise-level data is being used to support strategies beyond CRM, to include regulatory compliance, supply chain automation, asset management and strategic decision support, or business intelligence. This represents a significant change in how data acquired through operational applications is used and places a new emphasis on data quality for information beyond customer names and addresses.

To achieve any of these strategies, the data quality team also needs to think about data quality analysis, including upfront data profiling or data quality audits, as well as ongoing monitoring.

Your organization can achieve high-quality data. But with systems and applications frequently receiving new data and undergoing incremental changes, ensuring data quality must be more than a one-time event. All organizations need to manage data quality in a phased, iterative, and ongoing process that includes data quality assessment, planning, and strategy selection and implementation.

Traditionally, data quality was addressed as part of data warehousing projects, because moving data out of operational systems into the warehouse offers an opportunity to refine the data as it is moved. As the data integration technology that powers this process has increasingly been adopted to support other initiatives--- such as data migration, data consolidation, data synchronization, master data management, and outsourcing--- these initiatives now offer opportunities to improve data quality.

Selecting a strategy to address data quality in the long term requires balancing the cost of each data quality initiative against its impact. Regardless of what strategy you use, the evaluation of your data quality and its effectiveness will be of benefit to your organizations structure in more ways than one.

About the Author

Stephen J. Richards has 25 years experience in Data Management and Information Technology. This information is provided as a public service by Neon Enterprise Software, a leading provider of IMS outsourcing. For more information, please visit

Follow us @Scopulus_News

Article Published/Sorted/Amended on Scopulus 2008-05-16 23:58:02 in Computer Articles

All Articles

Copyright © 2004-2021 Scopulus Limited. All rights reserved.