Evaluating Data Quality
Computer/Internet/Software Articles
Submit Articles Back to Articles
Poor data quality costs businesses vast amounts of money and leads to
breakdowns in the supply chain, poor business decisions, and inferior customer
relationship management. Defective data also hampers efforts to meet regulatory
compliance responsibilities. Data quality encompasses more than finding and
fixing missing or inaccurate data. It means delivering comprehensive,
consistent, relevant, fit-for-purpose, and timely data to the business
regardless of its application, use, or origin.
Ensuring data quality is a challenge for most organizations partly because
they may not be fully aware of their own data quality levels. Without this
information, they cannot know the full business impact of poor or unknown data
quality, or how to begin addressing it. Poor data quality is a costly issue.
According to Pricewaterhouse Coopers Global Data Management Survey, 75 percent
of those surveyed reported significant problems as a result of defective and
fragmented data, more than 50 percent had incurred extra costs due to the need
for internal reconciliations, 33 percent had been forced to delay or scrap new
systems, 33 percent had failed to bill or collect receivables, and 20 percent
had failed to meet a contractual or service-level agreement.
Put simply, to realize the full benefits of investments in enterprise
computing systems, your organization must have a detailed understanding of the
quality of its data, where quality is poor, how to clean it, and how to keep it
clean---and must be able to deliver it to the business on the front lines with
data integration. By making data quality and data integration a strategic
priority, your organization better positions itself to streamline operations,
grow revenue, keep costs in check, and achieve long-term competitive advantage.
What is Data Quality? Data quality is a broad umbrella term for the accuracy,
completeness, consistency, conformity, and timeliness of a particular piece or
set of data and for how data enters and flows through the enterprise. Different
organizations will have different definitions and requirements for data quality,
but it ultimately boils down to data that is "fit for purpose". The data needs
to be of high enough quality to do the job at hand. If your business is risk
management, then your data will probably need to be much higher quality than
data for a mass marketing business. Organizations may not be aware of the full
business impact of poor or unknown data quality if they define the term too
narrowly.
To evaluate your organization's data quality, consider the following metrics:
* Completeness--- Is all necessary data present? *Conformity--- What data is
stored in a nonstandard format? *Consistency--- What data values give
conflicting information? *Accuracy--- Does the data accurately represent reality
or a verifiable source? * Duplication--- What data records are repeated?
*Integrity--- What data is missing important relationship linkages?
*Timeliness--- Does the age of the data meet user requirements?
In the past, data quality was primarily associated with cleansing and
matching name and address data with a focus on contact efficiency and the
identification of duplicates and relationships for marketing and customer
service applications. However, the more pressing business issues of today, such
as regulatory compliance, corporate accountability, and streamlining the supply
chain, require cracking the much harder nut of non-customer-oriented data
quality.
Large organizations have a wide variety of data in addition to customer name
and address information. Product, material, inventory, asset, financial, and
vendor data are abundant in the enterprise and are an ever-present source of
information quality problems. Increasingly, the initiatives of businesses and
government agencies call for access to a comprehensive enterprise-level
information source that draws data from a wide range of internal systems as well
as from external sources. This enterprise-level data is being used to support
strategies beyond CRM, to include regulatory compliance, supply chain
automation, asset management and strategic decision support, or business
intelligence. This represents a significant change in how data acquired through
operational applications is used and places a new emphasis on data quality for
information beyond customer names and addresses.
To achieve any of these strategies, the data quality team also needs to think
about data quality analysis, including upfront data profiling or data quality
audits, as well as ongoing monitoring.
Your organization can achieve high-quality data. But with systems and
applications frequently receiving new data and undergoing incremental changes,
ensuring data quality must be more than a one-time event. All organizations need
to manage data quality in a phased, iterative, and ongoing process that includes
data quality assessment, planning, and strategy selection and implementation.
Traditionally, data quality was addressed as part of data warehousing
projects, because moving data out of operational systems into the warehouse
offers an opportunity to refine the data as it is moved. As the data integration
technology that powers this process has increasingly been adopted to support
other initiatives--- such as data migration, data consolidation, data
synchronization, master data management, and outsourcing--- these initiatives
now offer opportunities to improve data quality.
Selecting a strategy to address data quality in the long term requires
balancing the cost of each data quality initiative against its impact.
Regardless of what strategy you use, the evaluation of your data quality and its
effectiveness will be of benefit to your organizations structure in more ways
than one.
About the Author
Stephen J. Richards has 25 years experience in Data Management and
Information Technology. This information is provided as a public service by Neon
Enterprise Software, a leading provider of
IMS outsourcing.
For more information, please visit
http://www.neonesoft.com.
Follow us @Scopulus_News
Article Published/Sorted/Amended on Scopulus 2008-05-16 23:58:02 in Computer Articles
