«Real-time Business Intelligence: Best Practices at Continental Airlines1 Written by Hugh J. Watson (University of Georgia), Barbara H. Wixom ...»
Real-time Business Intelligence: Best Practices at Continental Airlines1
Written by Hugh J. Watson (University of Georgia), Barbara H. Wixom (University of Virginia),
Jeffrey A. Hoffer (University of Dayton), Ron Anderson-Lehman (Continental Airlines), and
Anne Marie Reynolds(Continental Airlines)
Data management for decision support has moved through three generations, with the
latest being real-time data warehousing. This latest generation is significant because of its
potential for affecting tactical decision making and business processes. Continental Airlines is a leader in real-time business intelligence and much can be learned from how they have implemented it.
The movement to real-time is the latest development in business intelligence (BI) and data warehousing. Real-time data warehousing provides the data that is required to implement realtime BI. By moving to real-time, firms can use BI to affect current decision making and business processes. This capability is especially important for customer-facing applications, such as those found in call centers and check-in processes, and helps firms become more customer-centric.
Terms such as the “real-time enterprise” and the “zero latency organization” are often used to describe firms that use real-time BI.
The purpose of real-time BI is to increase revenues and decrease costs. Companies that successfully implement real-time BI can dramatically improve their profitability. For example, Continental Airlines, which is discussed later, has taken a $30M investment in hardware, software, and personnel to generate over $500M in revenue enhancements and cost savings, resulting in a ROI of over 1,000 percent.
To be successful with real-time BI, organizations must overcome both organizational and technical challenges. On the organizational side, there must be executive sponsorship and support, initial and on-going financial support, governance processes put in place, BI and data warehousing personnel with the requisite skills, changes in business processes and acceptance of use of real-time data by organizational personnel. On the technical side, new hardware and software must be acquired and implemented, processes and procedures for supporting and managing real-time data feeds from source systems must be established, data must be quickly transformed and loaded into the warehouse, and the data must be analyzed and made available to operational systems and personnel.
In this article, we provide frameworks and discuss key issues that are helpful in understanding real-time BI. We then use Continental Airlines as a case study of highly successful real-time BI.
In 2004, Continental won The Data Warehousing Institute’s prestigious Best Practices and Leadership Awards. We briefly discuss Continental and their business strategy that led to implementing real-time BI, describe an application that illustrates Continental’s use of real-time BI, and focus on the technical issues associated with Continental’s implementation of real-time BI.
We gratefully acknowledge Teradata, a division of NCR, who provided funding for this research.
This is an electronic version of an article published in Information Systems Management© 2006 Copyright Taylor & Francis; Information Systems Management is available online at www.informaworld.com.
Putting Real-time BI in Perspective Before we explore real-time BI at Continental, it is useful to put real-time BI and data warehousing in context. In particular, real-time data warehousing is the latest of three generations of data management for decision support. It is also useful to sort out what real-time really means as it applies to BI. And finally, we discuss latency, its impact on the value of data, and how it requires both technical and organizational solutions.
Three Generations of Data Management in Decision Support
The use of data for decision support can be conceptualized as moving through three generations.
The first generation emerged with decision support systems (DSS) in the early 1970s. It was recognized that DSS applications required a repository of data, some of which was sourced from operational systems, but also other data, such as external data. The data was customized for the specific DSS that was developed. This was a very application-centric approach, with the data supporting a single or a few related applications. It did, however, show the critical role of data in decision support. In his seminal work on decision support, Sprague provided the Data-DialogModels (DDM) paradigm, which recognizes data as one of the cornerstones of DSS.1 The second generation emerged in the late 1980s. Firms in the telecommunications, retailing, and financial services industries built data warehouses to store vast amounts of customer and sales-related data. Companies in these industries remain leaders in terms of the size of their warehouses and how the warehouses are used. Unlike DSS in the first generation, data warehouses tend to be data-centric. While a single or a few applications may be used to help make the business case for the warehouse, the data is modeled to support a variety of applications. The term “single version of the truth” is commonly used to describe the official repository of data that applications are supposed to use.2 In 2000, the third generation began with the movement to real-time data warehousing. The major reason this development is significant and worthy of being a new generation is the changes in the way that warehouse data is used. Previously, the data was primarily employed to understand what had already happened and to predict what would happen in the future. Its use for influencing real-time decisions and current operations was limited. With real-time data, current decisions and critical business processes, such as customer-facing and supply chain applications, can be significantly enhanced.
Real or Right Time?
For many people, the “real-time” term is synonymous with “instantaneous.” This interpretation, however, is incorrect when applied to data warehousing. While some warehouse data may be captured and entered into the warehouse in seconds or minutes, much of it is not. For example, some source systems, such as a legacy COBOL program that is updated once a month, can never be more real-time than when last updated. Some data may be prohibitively expensive or difficult to make real-time. Most importantly, there may not be a business need for real-time data. Data only needs to be as fresh as the business requirements. For these reasons, some people prefer the “right time” term.3 We use them simultaneously and recognize that real-time does not always mean instantaneous.
The Latency and Value of Data In most cases, the value of data decreases rapidly as it ages. Stating it differently, low latency (i.e., fresh) data has more value than high latency data. This is why the movement to real-time BI is appealing.
Richard Hackathorn provides a useful perspective on latency as applied to data warehousing.4 He identifies three kinds of latency; see Figure 1. Data latency is the length of time between when an event occurs and when the associated data is stored in the data warehouse. Analysis latency is the time between when the data is stored and when it is analyzed and made available to applications and users. Decision latency is the time from when the information is available until some action is taken on it. These three sources of latency are additive and result in total latency.
Reducing data and analysis latency depends primarily on technical solutions. Recent developments in real-time data warehousing provide help in this regard. However, reducing decision latency requires changes in business processes and how people use information in performing their jobs. Providing fresher data does not create business value unless it is used in a timely manner. Dealing with decision latency is usually more challenging than data and analysis latency.
Continental Airlines is a leader in real-time BI. It has received numerous awards for its work, including in 2004, The Data Warehousing Institute’s Best Practices and Leadership Awards.6 Continental’s experiences with real-time BI illustrate the challenges, solutions, and business value associated with real-time BI.
About Continental Airlines
Continental Airlines was founded in 1934 with a single-engine Lockheed aircraft on dusty runways in the American Southwest. Over the years, Continental has grown and successfully weathered the storms associated with the highly volatile, competitive airline industry. With headquarters in Houston, Texas, Continental is currently the USA’s fifth largest airline and the seventh largest in the world. It carries approximately 50 million passengers a year to five continents (North and South America, Europe, Asia, and Australia), with over 2,300 daily departures, to more than 227 destinations. Continental, along with Continental Express and Continental Connection, now serves more destinations than any other airline in the world.
Continental’s Business Strategy
Continental was in trouble eleven years ago when Gordon Bethune became CEO.7 There were ten major US airlines, and Continental consistently ranked tenth in the Department of Transportation metrics used to monitor the industry’s performance: on-time arrivals, baggage handling, customer complaints, and denied boardings because of overbooking. Not surprisingly, with this kind of service, Continental was in financial trouble.
Bethune and Greg Brenneman, who was a Continental consultant at the time, conceived and sold the Board of Directors on the Go Forward Plan. It had four interrelated parts that had to be executed simultaneously.
• Fly to Win. Continental needed to better understand what products customers wanted and were willing to pay for.
• Fund the Future. It needed to change its costs and cash flow so that the airline could continue to operate.
• Make Reliability a Reality. It had to be an airline that got its customers to their destinations safely, on-time, and with their luggage.
• Working Together. Continental needed to create a culture where people wanted to come to work.
Under Bethune’s leadership, the Go Forward Plan, along with a re-energized workforce, has helped Continental make rapid strides. Within two years, it moved from “worst to first” in many airline performance metrics, including on-time performance, lost baggage claims, and customer satisfaction.
After this success, Gordon Bethune and his management team raised the bar with a new vision.
Instead of merely performing best, they wanted Continental to be their customers’ favorite airline. The First to Favorite strategy built on Continental’s operational success and focused on treating customers extremely well, especially the high-value customers.
The Role of Information Technology
The movement from “worst to first” was only partially supported by information technology.
Historically, Continental had outsourced its operational systems to EDS. These included mainframe systems that provided a limited set of scheduled reports and no support for ad hoc queries. The airline lacked the corporate data infrastructure that a broad range of employees could use for quick access to key insights about the business.
In 1998, the decision was made to develop an enterprise data warehouse that all employees could use for quick access to key information about the business and its customers. The CIO at the time, Janet Wejman, recognized that the warehouse was a strategic project and brought the development and the subsequent maintenance and support in-house. She believed that the warehouse was core to Continental’s business strategy and should not be outsourced.
The data warehouse provided a variety of early, big “wins” for the business. The initial applications for pricing and revenue management were followed by the integration of customer information, finance, flight information, and security. They created significant financial lift in all areas of the Go Forward Plan.
However, when Continental moved ahead with the First to Favorite strategy, it became increasingly important for the warehouse to provide real-time, actionable information to support tactical decision making and business processes. Fortunately, the warehouse team had anticipated and prepared for the ultimate move to real-time. Real-time meant that the warehouse team had to introduce real-time feeds of data into the warehouse, extract data that the warehouse produced and incorporate them back into legacy systems, and open the warehouse to tactical queries with sub-second response time requirements. In preparation, the team had developed a warehouse architecture that could grow and scale to meet these new real-time and operational needs. While not all applications required real-time data, many did. In 2001, real-time data became available in the warehouse.
Real-time BI Applications
Continental’s real-time applications fall into the following categories:
• Revenue management and accounting
• Customer relationship management
• Crew operations and payroll
• Security and fraud
• Flight operations The objective of revenue management is to maximize revenue given a finite set of resources. An airline seat is a perishable good, and an unfilled seat has no value once a plane takes off. The revenue accounting area seeks to quickly and accurately record the revenues that Continental generates, including estimating the revenues from any flight as soon after “the wheels are up.” The Marketing group employs customer relationship management (CRM) in order to increase revenues, profits, and customer service by knowing customers exceptionally well (e.g., customer value, flying preferences) and giving them great service. Continental’s Marketing group uses the warehouse for customer segmentation and target marketing, loyalty/retention management, customer acquisition, channel optimization, and campaign management.
The Crew Operations group is concerned with issues related to pilots and flight attendants. It is involved in crew pay, crew scheduling, crew performance, and crew efficiency. The data warehouse is used in conjunction with all of these activities at varying levels.