Perhaps no subject is more important, or bedeviling, to those who labor to leverage their data warehouses than data quality. When populated with high-quality, trusted data, warehouses—including the associated decision support, business intelligence (BI) and data mining tools—are powerful enablers. They can help businesses see customers and markets in new ways, create new products and services, and improve operations. In so doing, these organizations distance themselves from competitors.

Decision makers are smart. They instinctively know that their decisions are no better than the data on which they are based. And when they suspect the data isn't good, they don't trust the data warehouse—for the really important decisions, anyway. So all of the potential to develop more complete views of customers, uncover new niches and better manage risk lies fallow.

Fortunately, over the last 20 years leading-edge companies have learned how to systematically manage and improve data quality. Now, the 10 habits of enterprises with the best data are available to anyone.

A closer look
It's important to realize that these habits aren't adopted with a cookie-cutter approach. Not all enterprises with high-quality data follow each habit in equal measure or in the same exact manner. They emphasize those habits that work best for them, tailoring each to their cultures. By doing so, they develop an overall data quality program that permeates the entire organization. Let's look at the 10 habits in greater detail:

1. CUSTOMER FOCUS
Treating decision makers who use data warehouses as customers is essential. Without understanding customer needs, it is impossible to satisfy them. One way to follow this habit is by sitting with customers, learning what decisions they make and how they make them, and determining what data they need and the level of quality they will accept. Documenting these needs using "Customer Data Requirements" is an important step. In addition, enterprises must determine which customers are most important. Failing to focus on these users is likely to dilute efforts to improve data quality.

2. PROCESS MANAGEMENT
With requirements established, top organizations work backward to the business processes that create data and actively manage those processes end to end. Often those departments or individuals who create and input data have no idea why they are being asked to do so. Good process managers address this issue by sharing the requirements document. Immediate benefits usually result as employees figure out how to improve their work.

3. SUPPLIER MANAGEMENT
Data-quality leaders aren't deterred if the business process that creates the needed data lies outside the organization, in the supplier base. They know how to manage data suppliers. Isn't it odd that companies issue detailed specifications and employ service level agreements for physical goods they purchase but ignore the data? The methods of supplier management for physical goods, however, are indeed just as effective for data.

4. MEASUREMENT
Measurement at the source is critical. The old management adage that "you can't manage what you don't measure" is as applicable to data quality as it is to expense budgets. Top companies often publish their data quality statistics. If the data is good and getting better, publishing the statistics builds decision makers' confidence. If the data is not so good or not getting better, publishing the statistics increases the urgency for improvement.

5. CONTINUOUS IMPROVEMENT
Measurement feeds into continuous improvement. It is the means by which much of the heavy lifting is accomplished. An improvement project, in and of itself, involves investigating the pattern of errors, selecting a category to work on, identifying the root cause and changing the business process to eliminate that cause. Enterprises with the best data have a knack for starting and completing improvement projects.

6. CONTROL
Control is the "managerial act of comparing actual performance against requirements, and acting on the difference," according to quality guru Joseph Juran. It is crucial to employ controls on many levels. Leading organizations stop errors in their tracks by establishing business rules to identify errors, and they correct them before allowing the data downstream. They employ statistical controls to distinguish special and common causes of process errors from one another.

7. TARGETS FOR IMPROVEMENT
Intimately related to measurement, improvement and control is the habit of setting and achieving aggressive targets for improvement. For example, the head of a business unit remarked that her goal was to cut in half the error rate in all key processes every year. This is impressive, but it is typical of organizations that maintain quality data.

8. CLEAR MANAGEMENT ACCOUNTABILITIES
In a perfect world, middle managers could develop and implement all of the above habits on their own. After all, almost all managers buy in, conceptually at least, to the notion of preventing errors at their sources. But the world is far from perfect. So it's important to recognize the importance of leadership. One step in doing so is maintaining clear management accountabilities. Specifically, the people who create data are accountable for quality. This applies to operational data, financial data, reporting data, metadata—all data.

9. MANAGING SOFT ISSUES
Most data people are surprisingly naïve about organizational politics, even though it is the soft issues that most often derail them. For example, many data people think it obvious that developing a 360-degree view of customers is a good idea. But for a senior salesperson with extensive knowledge of a big customer, this information is a source of power and commissions. As a result, the salesperson may be reluctant to contribute. Resolving such political issues takes astute leadership.

10. BROAD, SENIOR GROUP LEADERSHIP
Finally, data quality programs usually extend just as far as the influence of the most senior managers perceived to be leading the effort. Top organizations secure broad, senior leadership and support—the higher and broader the better.

Data quality and the business community
Data quality must be the responsibility of the business community, not IT. The proven way to reap the benefits of high-quality data is to create it correctly the first time. The effort to achieve the highest quality, as evidenced by the 10 habits, occurs in an enterprise's business community. While poor data quality can lead to mistrust of a data warehouse or harm the IT department's reputation, the costs to the business side are more direct. They involve both initial investment and ongoing operational dollars. Conversely, if the data is of high quality and the data warehouse trusted, the business may reap millions from a single, better-conceived and well-executed decision. For IT, the aim should be the same as a good referee at a soccer match—invisible to the crowd and the outcome. T

Thomas C. Redman, "the Data Doc," is president of Navesink Consulting Group, Little Silver, N.J. His fourth book, "Data Driven: Profiting From Your Most Important Business Asset," was published by Harvard Business Press in September 2008. He can be reached at tomredman@dataqualitysolutions.com.

Photography by Michael Denora

Teradata Magazine-December 2008