How to Avoid “Bad Data Days”

When a large insurer wanted to create a more integrated, customer-focused organization, it began by moving some 4 terabytes of premium, loss, customer, policy, web traffic, and external data from disparate legacy systems “owned” by individual business units into an enterprise data warehouse (EDW).

As the project evolved, myriad data-quality issues emerged: repetition, different formats, different metrics, and multiple business definitions. Rather than deal with these problems piecemeal, the company recognized data quality as an ongoing concern that could make or break the success of its initiative. In response, it instituted a “data stewardship” program that drew on the expertise of high-level business and IT executives across the organization.

Four years later, by using data in new ways, the warehouse has helped maintain the product innovation, customer responsiveness, regulatory compliance, and smart underwriting that characterize today’s most successful insurers. At the same time, it has reduced costs through labor force optimization, replacement of legacy applications, and faster retirement of maintenance hardware. Without a data stewardship program, the initiative might never have achieved those results.

The single-enterprise view that a data warehouse enables has long promised breakthrough insights, especially for businesses like insurance that rely heavily on data. Nevertheless, many data consolidation projects underperform, and quite often bad data is the cause.

In March 2002, National Underwriter concluded that a midsized insurer (approximately two million claims transactions per month) can lose $10 million per year due to the direct costs of analyzing and correcting data errors. (The estimate was based on a Data Warehouse Institute study which reported that data-quality issues cost U.S. industries more than $600 billion per year.) In reaching its conclusion, National Underwriter assumed that only .001% of a company’s data is bad and that companies would attempt to fix only the 10% of errors that are critical to the business.

But there would likely be indirect costs as well. When errors become exposed to customers and regulators, fines follow and the backlash can force an avalanche of expensive changes in how a company conducts its business. Even worse, sources for bad data, both inside and outside of a company, are proliferating. External sources include government organizations, credit and claims bureaus, broker channels, and online consumers.

Internally, as many companies have recognized the business potential of making data accessible beyond the traditional set of high-level users, a wider variety of staff, such as sales and service personnel, captive agents, and back-office operations, has begun to gather and input data. The Data Warehousing Institute has found that employees make 76% of the data-entry errors that account for bad data.

Equally important, good-quality data has always offered significant business advantages. Customer and market data are useful in developing new products and services and pricing them appropriately. All staff can use it to provide a good customer experience and grow and maintain customer relationships.

A data stewardship program can maximize the accessibility, reusability, and quality of a company’s data as an essential survival tool. A number of best practices for such programs have now emerged.

First, as with any major corporate initiative, senior management must be fully engaged. Second, the business and technical sides must work closely together in cross-functional teams. Cross-functional teams ensure that everyone understands their role in maintaining the quality of the data. Moreover, by increasing their awareness of what data exists and where they can find it, they are more likely to actively use the data. From the other side, technical staff gain insights that allow them to fully align their work priorities with the company’s overall business strategy.

Unfortunately, fewer than half of all companies nationwide have a formal data stewardship program. If a company has a working program, evaluating that program to ensure it is in line with best practices is worthwhile. An evaluation can help a company measure data-quality costs and benefits, understand the data-value chain, readily view where company data resides, and prioritize data-quality efforts.

Evaluations can include an examination of overall data quality, a gap analysis that identifies organizational holes that lead to bad data, and a process review of existing data management programs.

Given the number of people who touch data throughout an organization, companies should also evaluate if they have an effective communication plan so that everyone knows their role in ensuring high-quality data and the importance of doing so. Finally, companies might spend some time identifying additional business users to increase the value of any data-related projects and realize a greater return on investment.

Creating a visible and vigilant process to guard against the “garbage in, garbage out” syndrome is one key to achieving the kind of return on data and technology investments that every company seeks.

The CPA Journal is broadly recognized as an outstanding, technical-refereed publication aimed at public practitioners, management, educators, and other accounting professionals. It is edited by CPAs for CPAs. Our goal is to provide CPAs and other accounting professionals with the information and news to enable them to be successful accountants, managers, and executives in today's practice environments.