TDWI's Operational Business Intelligence report shows a trend of forging beyond traditional business intelligence.
by Wayne Eckerson
Colonists had settled in North America for nearly 200 years before they pushed beyond the Appalachian Mountains in the
great westward migration that led to the settlement of the rest of the vast continent. In the same way, organizations that
today use business intelligence (BI) primarily to analyze historical data to ascertain trends and assist with strategic planning
decisions are now pioneering the use of BI to manage operational processes.
By so doing, they are opening an enormous new territory for the application of BI that promises
to empower workers, both inside and outside the organization, with information and insights to
work more proactively and productively.
Operational BI represents a turning point. Traditionally, BI has been the province of business analysts
who spend many hours with sophisticated tools analyzing trends and patterns in large volumes of historical
data to improve the effectiveness of strategic and tactical decisions. But operational BI changes this
equation. It intertwines BI with operational processes and applications that drive thousands of daily decisions—in
essence, merging analytical and operational processes into a unified whole.
Operational business intelligence (BI) includes a variety of approaches and
architectures. Most organizations implement different levels of operational BI over time.
Most BI solutions provide an aggregate or snapshot view of business performance by department or
product. However, operational BI can track the activity of each step in a multi-step business process
that may cut across departmental boundaries. It can also anticipate activity as well as facilitate responses
or actions in well-known business processes.
Operational BI delivers information and insights on demand to all workers—from the shipping clerk to
the CEO—so they can work smarter and faster to achieve critical business objectives.
In its sophisticated form, operational BI encapsulates business insights into rules and models
that organizations can use to automate decisions and responses, eliminating the need for human
intervention. Automating decisions not only streamlines processes and reduces costs but also improves
service and gives organizations a competitive advantage in the marketplace.
Operational BI encompasses many different approaches, architectures and technologies, and it can be
used to support a variety of decisions and accomplish multiple tasks. While most organizations already
support some form of operational reporting, many have yet to embrace more complex types of operational
BI that generate greater business value. Thus operational BI gives organizations a chance to reap greater
dividends from their BI investments.
Figure 1 represents TDWI's framework for understanding the levels that organizations can ascend
as they gain experience and sophistication in the use of BI techniques and technologies. As
organizations reach new levels of operational BI, the data latency—the time between an event
and when data about that event is presented to a user or application—shortens, sometimes to zero
in event-driven systems that automate decisions and actions.
Operational reports allow users to analyze operational processes using traditional reports. They
can be implemented in many ways, but two major approaches to operational BI stand out. Companies can
build operational reports by querying transaction systems directly, or they can offload the transaction
data into an operational data store or real-time data warehouse and query the data there.
An event-driven analytic engine captures and correlates a continuous stream of business events
to provide real-time insights into operational processes and automate responses
through alerts and business workflows.
Operational dashboards involve monitoring business processes using performance dashboards. An operational
dashboard is, in effect, a graphical report tailored to each user that focuses on a few
metrics that represent the performance of key operational processes populated with near-real time
data. Also, dashboards can proactively alert users to exception conditions through e-mail or wireless devices.
Composite applications facilitate processes. Companies can reach this level by embedding metrics
or reports within operational applications or portals. In other words, instead of requiring users to use
two different systems—one to run the process and another to analyze it—organizations can embed the analysis
directly into the process and the application that drives it. The many approaches to this include embedding
source code or remote function calls within applications and using a service-oriented architecture to stitch
together application components across different platforms and systems.
Event-driven analytic platforms are perhaps the most exciting stage of operational BI when an organization
uses analytics as an engine to execute processes and workflows. Here, BI does not just provide insights
into the process; it is the process. These event-driven analytic platforms (also known by the terms business
activity monitoring [BAM], complex event processing, and business process management [BPM]) enable companies
to capture business events and apply rules to assist or automate the execution of business processes. (See figure 2.)
These platforms are like intelligent sensors that organizations can attach to their transaction
streams. The sensors take a continuous reading of the health of various business processes and trigger
actions based on prior experience. In other words, these engines can execute processes and automate decisions. More
sophisticated engines use analytical models to improve the sophistication of their rules.
Despite its promise, operational BI introduces several challenges. It stretches the architectural boundaries
of current solutions, forcing BI professionals to rethink the way they design and build systems. Queries
must return in seconds rather than minutes or hours, and reports must update dynamically. Operational BI systems
must capture large volumes of data in near-real time without degrading the performance of existing processes and
jobs on source or target systems. The systems also have less time to recover from a server outage, making it imperative
for BI professionals to build resilient, highly available systems with sufficient backup and recovery.
Perhaps the biggest challenge is simply deciding whether to adapt an existing data warehousing architecture to
deliver just-in-time data to support operational BI. Many architects believe it is critical to do
so. "The big showstopper is whether you are going to apply the same business rules to integrate, cleanse and validate operational
data streams as the rest of the data in your data warehouse," says John O'Brien, a BI consultant and former data
warehousing architect at several telecommunications companies. Pulling operational streams out of the data warehousing
process undermines data quality and creates divergent data sets that may not reconcile, he claims.
But some disagree, believing that a data warehouse becomes a bottleneck if you try to load all data into it that
users may possibly want to query. BI vendors that support federated query believe their tools provide an easy, low-cost way to
capture real-time data and deliver it. Likewise, vendors of embedded BI, event-driven analytic platforms, composite
applications and in-memory analytics believe their offerings provide the most suitable way to meet high-end operational BI
requirements. These vendors say a data warehouse is critical for applying historical context to real-time data but not necessary
for managing the real-time data itself.
Nonetheless, data warehousing is a well-established IT practice in corporate environments, and many organizations seek to
protect their data warehouse investments by adapting the architecture to support just-in-time data and operational processes.
To create a real-time data warehouse, you need a database platform that provides robust support for mixed workload
processing. A relational database management system (RDBMS) that supports mixed workload processing lets organizations
load current and historical data in the same database and optimize performance across all types of queries. In this
way, an RDBMS that supports mixed workload enables organizations to use a data warehouse for operational BI. Organizations
should be sure the mixed workload capabilities will meet their needs because the capabilities can vary depending on the vendor.
Operational BI opens a vast new territory that will make BI a more mission-critical resource for driving
operations. Despite the promise, organizations have many technical and business challenges to hurdle. Retooling
a data warehousing architecture for just-in-time data delivery is a significant undertaking, while educating users
about the need and nature of operational data is another. BI teams that surmount these challenges are already finding
that the payoffs are truly worth the effort. T
Currently, only a small percentage of organizations have implemented a mature
operational BI environment, according to a recent report by TDWI Research
titled "Best Practices in Operational Business Intelligence: Converging Analytical and
Operational Processes." Yet this activity will greatly increase, according to the
response from 423 respondents to the question "What's the status of your operational BI
environment?" (See figure 3, above.)
The TDWI report also shows that many organizations are adapting their data warehouse
architecture to support operational BI. About half of organizations (51%) run both operational
and analytical reporting from the same environment, according to 225 respondents to the
question "Is your operational BI environment the same as your regular BI environment?" (See figure 4, above.)
Note: Because of rounding, percentages do not add up to 100.
For more information, download the 35-page TDWI report free from http://www.teradata.com/t/page/171360/index.html
Wayne Eckerson is the Director of TDWI Research, an author and a noted speaker and consultant.
Teradata Magazine-December 2007