Evolving From Information to Insight

Business leaders often believe their organizations are swamped with business process information collected through their own enterprise systems or those of their business partners, most of which is trapped in data warehouses. They seek to harness it and use it to inform better decision making, financial management and customer service.1 What they do not realize is that, even if they were able to meet this goal, they would merely have caught up with yesterday. The information frontier is moving forward quickly, opening up incredibly rich streams of new information sources and formats and dramatically increasing the technical ability to manage them. Forward-looking companies must stay ahead of that curve.

Consider an everyday example. More than 30 million packages are delivered globally by courier services every business day. All too often, the recipient isn’t there to receive the package when it arrives, which is frustrating for him, the sender and the courier. The problem is that packages are directed to a physical location, which is static, but the intended recipient is mobile. Yet with information and technology available today, the delivery “address” could be a person who could be found in real time, anywhere.

How would that work? Imagine that a delivery company had access to your electronic appointments calendar and could locate you using the Global Positioning System via your vehicle or cell phone — with your permission, of course. The company could notify you of an imminent delivery and ask if a courier should bring your package directly to you right away, leave it at another location, or reschedule delivery for a more convenient time and place. Depending on your choice, you might “sign” for the package, by verifying your identity through a smart card or a thumbprint. This kind of delivery would be a highly personalized service, made possible by combining several different types of information not commonly or economically available until recently.

So far none of the major package delivery services has adopted usercentric addressing. But the technical capabilities do exist.2 In the near future, leaders in any industry will be able to draw on not only enterprise data but also information from a wealth of new sources that will create a richer, real-time picture of the world. And those leaders will be able to use that information not just to understand the past, but also to anticipate the future, gleaning novel insights and innovative products, services and business models. To understand this assertion, it first is important to look at what is happening in the realm of information. (See “About the Research.”)

Information Reaches Critical Mass

In just the past few years, both information itself and the ability to manage it have reached a critical turning point. The relative trickle of information that businesses had access to in the past has become a torrent made up of many converging streams. Consider the types of information that are available, or potentially available, to business today:

Business process information.

This is the kind of data that comes to mind when executives discuss the need for better business intelligence. It includes all the transaction data now gathered at the point of sale, whether at a bricks-and-mortar outlet or through e-commerce. It also encompasses the wealth of information that corporations gain about their own operations, as well as those of business partners and allies, through their enterprise resource planning, supply chain management and customer relationship management systems. This is a significant and still expanding source of information. IDC, a division of International Data Group, estimates that e-business and other enterprise technology trends such as ERP, CRM, and data warehousing are doubling the amount of corporate data every six months.3

Physical-world observations.

These are the result of technologies such as radio frequency identification (RFID) sensors, miniature cameras, wireless access, GPS and sensor telemetry, which reveal where any person or thing is located, what it is doing, and under what conditions it is performing. These technologies are being rapidly advanced and will engender vast amounts of new information. Indeed, RFID tags have been made mandatory by purchasing giants such as Wal-Mart Stores Inc. and the U.S. Department of Defense.4 An increasing number of cell phones and automobiles come with technology that can locate a user to within 50 meters or less.5 Miniature cameras are not only becoming ubiquitous in our streets and public buildings but are also being incorporated into the newest generation of mobile phones. By 2010, the number of deployed sensors will dwarf the number of personal computers by a thousandfold.6

Biological data.

This kind of information is also increasingly available. Fingerprinting is hardly a new form of identification, but fingerprints today can be more easily taken and shared. Biometric devices such as those that scan retinas provide a more advanced form of identification, as does voice scanning and facial recognition software. The biometric profile for a single individual from various recognition technologies can easily exceed 6 kb, not including DNA data.7 Genetic information is increasingly advanced and available, making it possible to identify and know a good deal about a human being. Such information is being collected now mostly for security or fraud protection, but it offers new potential to tailor products and services.

Public data.

This includes the electronic traces that anyone leaves when, for example, posting to the Internet or joining in a chat session, as well as information in data repositories that are free and readily accessible. How much public data is available? In 2003, the size of the Surface Web (that portion of the Internet accessible to any user) was estimated at 167 terabytes. (By comparison, the 19 million books and other print collections in the Library of Congress contain about 10 terabytes of information.) The Deep Web (which consists of sources accessible for a fee or with a password) is about 550 times larger, estimated at nearly 92,000 terabytes. E-mail originals add about 441,000 terabytes, and instant messages add another 274 terabytes.8 That brings the size of the Internet to more than 533,000 terabytes. All this public data is increasingly being stored, shared or sold.

Data that indicate personal preferences or intentions.

One example of this kind of data is shared calendar data within an organization — recorded intentions to travel to a location, participate in a conference call or take a day off. In other settings, people may be asked to indicatewhere they like to vacation, what colors or styles they prefer, or how they like to be contacted. More broadly, e-shoppers leave a trail of information that goes beyond transaction data and can reveal personal likes and dislikes, plans and aspirations. To get some indication of the volume of such data, consider that in January 2004, nearly 84 million people — more than half of all U.S. Internet users — visited retail Web sites for either eBay Inc. or Amazon.com Inc.9 eBay has already begun selling blinded transaction data to interested third parties.10 (Collecting and using such information clearly raises concerns about privacy and confidentiality, much of which governments will address through legislation. In this article we will address the narrower issue of how corporations can fulfill their responsibility with regard to sensitive data.)

New Computing Muscle to Process the Data

The sheer quantity of all this information is unprecedented, but so is the complexity of working with it, because it is fragmented. That is, it arrives in both structured and unstructured formats, often in granular increments and from numerous sources — and because it takes a variety of forms, including textual and numerical, audio and video. Yet, despite its volume and disparate nature, this data is potentially useful to business because the computing power necessary to merge, manage and make sense of it has also been advancing.

Massive computing power has become more affordable than ever before. Consider the cost of a teraflop of processing power (defined as a trillion floating point operations per second). The Earth Simulation Center in Yokohama, Japan, which opened in 2002, boasted 35.6 teraflops and cost $350 million; not long after, Virginia Polytechnic Institute and State University opened a 10 teraflop computing center based on Apple Computer Inc.’s Power Mac G5 computers for $5.2 million.11 Data storage costs also have plummeted: The storage cost per gigabyte of data was $2.25 in 2002, and is projected to drop to 38 cents by 2008.12 With such computing power available and increasingly affordable, it becomes possible to gather data at minute levels, store it in massive quantities, and use it in virtual real time.

Furthermore, computing systems are increasingly able to communicate with one another, as standards and protocols create a universal language that enables data to be shared. This is essentially the promise of Web services, which, in theory, utilize an evolving set of industry-determined technical standards to enable the integration of applications, making it increasingly feasible for businesses to link their legacy systems with new systems and to share data across corporate boundaries. More broadly, Web services should eventually enable data from newer information sources, such as sensors, to communicate with computer applications (or with other sensors in ad hoc networks).13

Machine learning — the ability of a computer to improve its own performance over time — also has advanced. In the past decade, researchers in artificial intelligence have improved their basic understanding of the learning process itself, and from that have been able to build algorithms that help computers make sense of content without the need for detailed, domain-specific instructions. With these advances, it becomes possible to make use of unstructured data, which includes all text and other information that is not formatted or labeled in such a way that it can be tapped through conventional data mining. Unstructured data includes essentially all of the Internet — Web pages, e-mail, instant messages — as well as most written material such as magazine articles and press releases.14 Techniques have been developed that will search through data that users have left on many Internet services and identify, categorize, distill and aggregate relevant opinions on any topic.15

Gaining Insight: What Companies Can Do

As organizations’ ability to take full advantage of this combination of new information sources and increased computing power becomes increasingly feasible and economical, the very nature of business insight will evolve. As data itself becomes more available and usable, competitive advantage will go to organizations that can use this data either to respond in (near) real time to the present or to predict and direct the course of events, as appropriate for different specific business situations. Forward-looking companies are already positioning themselves for information insight by moving quickly and simultaneously down two parallel tracks: one aimed at getting information and the permission to use it, the other aimed at developing the organization’s ability to convert that information into insight. (See “The Evolution of Insight.”)

Increase the Ability to Gather and Access Data

A company can only benefit from these streams of information if it can bring the information in-house or otherwise gain access to it. Because gathering much of this information will require an investment in technology and effort, time is not an ally. What can a forward-thinking company be doing now?

Capture physical-world data.

The technology for gathering physical-world observations is advancing so rapidly that just about every company should be looking toward making infrastructure investments. This is particularly true for RFID and sensors, which can deliver value now while also positioning an organization on a learning curve to greater value.

A good example comes from Australia’s sugar cane mills. Cane plantations cover over 1,000 miles of the tropical coastline of northeastern Australia. During the annual “crush,” cane is cut, loaded onto wagons holding several tons and hauled to the mill for processing via an extensive private narrow-gauge railroad system. Cane degrades from the moment it is cut, so speed counts. A rail accident could block the line, causing the shutdown of mills that would otherwise be generating revenue at around A$10,000 per hour. And rail accidents occurred regularly, in large part because of failed wheel bearings.

Engineers knew that axle overheating was a sign of imminent failure, but had not been able to develop a viable early warning system until an industry supplier, AMSKAN Ltd., brought together two advances in information technology. The first was a composite optical sensor that was inexpensive and would work in the blazing sunlight of tropical Australia, detecting any overheating axle as a rail car passed by. The second was RFID technology, which made it possible to identify and track those specific rail cars so that they could be pulled offline for repair before causing an accident. Mills that installed AMSKAN’s new Hot Axle Box Detection System were able to anticipate and prevent rail car failure, which gave them an immediate competitive edge. Equally important, they then had in place sensor and RFID technology. These data can be put to use in the future in other creative and unforeseen ways. Meanwhile AMSKAN itself is looking for extensions of this technology into other industries, such as mining.16

Invest in technology to explore public data.

Just as RFID and sensor technology is rapidly reaching the point of maturity and payback, so is the availability of public data, especially on the Internet. It is not too early to begin developing or buying whatever tools will help make this publicly available data (as well as internal enterprise data) more accessible and actionable.

In addition to efforts to explore public data, there have been a number of recent initiatives in “information fusion”: the combining of multiple sources of data to achieve clarity through context. For instance, the U.S. Department of Defense has been combining terrestrial maps with war theater data to provide rich command-and-control environments —recently achieving the capabilities that the novels of Tom Clancy have been alluding to for years. Commercially, companies like Axiom Manufacturing Inc., a maker of microcontrollers, and Fair Isaac Corp., a provider of predictive analytics, have combined data to support marketing and credit applications, while startups like Semagix Ltd., which specializes in content aggregation and risk assessment, have begun to offer toolsets for knowledge discovery.

Accenture developed one such tool that is currently employed by pharmaceutical companies. The tool probes current and pertinent data repositories (such as research journal postings, human genome database entries and the pharmaceutical company’s own inventory of potential treatment compounds), displays knowledge in new integrated ways and puts researchers into contact with other researchers having common interests via e-mail or chat rooms.17 Essentially, it brings together numerous diverse sources of data for rapid analysis and insight, enabling the discovery of previously unknown links that might lead to research breakthroughs.

Similarly, a number of tools are being developed to monitor sentiment on the Internet. One prototype developed by Accenture, for example, gives companies the ability to search through the more than 550 billion unstructured, ever changing and fragmented pieces of information on the Internet to find meaningful patterns in individual opinions, such as a perspective on their corporate brand. One automobile company is using that insight to identify and prioritize features and enhancements for its future models.18 Imagine how incorporating physical-world data could further enrich this knowledge. Sensors and telemetry could gather information on actual usage after purchase, comparing how people actually use a product with how they believe it delivers value and yielding new insights for product development and advertising.

Begin building trust.

Real advances or breakthrough business models will almost certainly depend on accessing new types of information, often very sensitive in nature. Recall, for example, the dynamic package delivery example offered earlier. That model works only if the package recipient is willing to reveal his or her physical location, calendar, identification authorization, billing information and perhaps more — information that the package delivery company will not only use internally but likely also share with corporate partners.

A 2003 Accenture survey indicated that trust or the lack of it is a significant issue for consumers and companies alike.19 Thus far, most of the furor has focused on individuals’ concerns about privacy, which has built into a customer backlash against invasive information requests or the misuse of that information, such as through sale to third parties. Trust issues also arise between corporations, especially as business becomes an extended network of alliances and partnerships that may not be exclusive.

The basic components of a trust-based relationship are the same for individual consumers and for business partners:security (Is my information protected?);data and access control (Can I control who accesses my information? Who contacts me and when?);accountability (If my information is abused, will someone take corrective action?);benefit (Will I gain from having provided my information?).

How can trust be built? At every step and in every transaction, a few key principles need to be applied.

First,spell out the benefits. Trust builders understand that information is not given as a gift but as a price paid for something of value. When consumers or business partners offer up information, it is to help themselves — a distinction that companies often overlook. For example, the OnStar system, first used in the Cadillac division of General Motors Corp. and now available in more than a dozen vehicle lines, delivers many of its benefits precisely because GPS pinpoints where a vehicle is at any moment. That could be seen as an invasion of privacy, yet users are focused instead on the additional services they receive, such as help with directions and assistance in emergencies. In our survey, 69% of consumers said that, despite privacy concerns, they would surrender personal information in exchange for rewards such as bonus points or greater convenience.

Second,ask for information in context. This means asking for only the information that you clearly need to deliver the benefits. This is not the time to probe for ancillary information, however interesting it may be. For example, if your mortgage lender asks for your net worth, you provide it; if your doctor makes the same request, you get nervous.

Third,give the data owner control over its use. Especially in the early stages of trust building, owners need to be given clear explanations of how information will be used. Simply having a comprehensive, highly visible privacy policy is not enough. A more constructive step is to give data owners an easy, straightforward way to say yes or no to the use of their data. When dealing with individual consumers, an opt-in policy may build more goodwill than an opt-out policy.

Fourth,demonstrate that sensitive information is secure. Keeping information safe from misuse is a challenge for any organization, especially when it shares information with suppliers, channels, retailers, allies and partners. Methods may need to be developed specifically to safeguard information in particular situations. For example, the Canada Revenue Agency, the nation’s taxing authority, has access to a wealth of citizens’ personal data, which allows it to perform compliance work, for example, tracking whether someone’s tax bracket squares with their extensive number of first-class flights to Europe. Much of this data is also valuable to other government entities for projecting demographic trends and likely revenue streams. CRA has developed an approach to “blinding” the data so that other agencies can create statistical summaries for their own purposes without accessing restricted information on any single person.20

Build Organizational Capability to Use Data for Insight

Having a wealth of information is of little value unless it can be used for competitive advantage. This is where many organizations fall especially short. As some of our colleagues have said with reference to traditional transaction data: “Firms have purchased the technologies that enable transformation of data into knowledge, but the human component necessary for this process has been given short shrift.”21 Attention to human capabilities is even more critical when the challenge is to spot opportunities that lie within the information itself.

Any number of good studies have been written on the broad topic of creating an innovative organizational culture,22 and our goal is not to repeat or duplicate those ideas. Rather, we want to focus on three keys to creating an organizational ability to find and develop competitive opportunities within nascent streams of knowledge.

Equip everyone with question marks.

“Mere access to information does not automatically bestow power,” says John Kao, former academic director of the Managing Innovation program at Stanford University. “The crucial variable in the process of turning knowledge into value is creativity. Always more important than the tool itself is the use to which the hand directs it.”23

Stated simply, data holds innumerable answers — but are the right questions being asked? The truly exciting applications of new information will come when some human mind asks: “Given the data we’re getting, couldn’t we find out … ?” or “With what we know today, would it be possible to … ?” The package delivery model cited earlier, for example, came about because a human mind formulated this question: “Since GPS and other technologies make it possible to locate a person in real time and to reroute vehicles in transit, why are we still delivering packages to someone’s front door?”

Many companies are stalled because their key players are each waiting for someone else. Executives and managers are waiting for IT to bring them some interesting new insight; IT specialists are expecting business leaders to formulate hypotheses for them to explore; and most everyone else within the company feels no expectations on them whatsoever. For innovation to occur, everyone must be formulating questions.24 This is true in part because of Hamel’s “Law of Innovation,” which essentially states that for every 1,000 ideas formulated only one or two will prove to be both feasible and of significant market value.25 It is also true because provocative questions and new ideas can bubble up from anywhere — from management ranks but also from the front line, analysts in IT, suppliers, customers or consultants. Organizations need to create a “what if” culture, in which idea generation is part of the corporate vocabulary and on the agenda of every meeting.

Corporations also must put as much information as possible in front of as many people as possible. This may well require breaking down traditional isolated data fiefdoms so that information belongs to the enterprise, not to any one department, and insight can be gained by looking at data across traditional boundaries. Some education may be needed as well, so that everyone can understand the different types of information now available and what those information streams will enable.

Bring on the “quants.”

Asking all the right questions is only part of the equation. The questions then have to be answered, and that is the role of quantitative analysts. Drawing meaning from new information sources will require a robust analytic capability, because each provocative new idea needs to be subject to a feasibility test. Analytical rigor leads to verifying a pattern of past events or behaviors that has predictive value for the future.

Companies in search of an insight advantage should establish a department or a role dedicated to quantitative analysis. With the right tools and a deep understanding of the various information sources, these quants can apply intellectual rigor to ideas that surface throughout the company, forming hypotheses and testing them. Equally important, they can delve into the data themselves to find patterns and generate their own information insights.

The day is coming, though not yet at hand, when analytical capabilities will be so advanced and easy to use that they can be widely shared, allowing everyone in the organization to not only pose questions but also to answer them. Quants will still be needed, of course, but primarily to address the most complex or innovative questions.

Push information outward.

Successful companies have always tried to push their products or services in new directions to open up additional opportunities for value. Consider Sealed Air Corp., the company best known for making Bubble Wrap packaging, invented in 1960. In recent years, Sealed Air asked itself what else it could do to make bubble-sealed air more valuable. The result was the creation of an inflatable packaging system, dispensed on demand at the client site from a compact piece of equipment and wrap-coated with an oxygen scavenging film that keeps food products fresher.

Companies should apply this same kind of innovative thinking process to use of their information resources. Once an information-based product or service is in place, the addition of more information or a wider vision often can take that idea to new levels. Everyone in the organization should think of information itself as a new and barely tapped resource in its own right.

For example, a major truck manufacturer is equipping its fleet with sensor telemetry (including GPS) that enables the location of each truck in the fleet to be tracked and its fuel consumption monitored.26 The ability to gather that information is a real benefit to fleet owners. For the manufacturer, though, it opens up possibilities for additional business services. Could the manufacturer work with the fleet owners to institute new training procedures and incentives to encourage drivers to operate their vehicles more efficiently? Or could the cumbersome process of fuel-tax reporting be automated?

Here’s another example of how companies think about pushing information outward. A prominent supplier to the energy industry, realizing that intelligent meters can acquire and transmit data on energy usage hourly or sometimes minute by minute to a central repository, has begun to think about how that information could be usefully shared, and with whom. Allowing customers to access the information in real time, for instance, via the Internet, telephone messaging, e-mail or some other existing medium, would allow them to see the impact of every appliance used and every light turned on or off, thereby motivating them to modify their energy usage. Further, an econometric model could be used to predict how the community of customers would respond to changes in price, thereby allowing utilities and marketers to predict the relationship between price and load. They ideally could avoid activating peaking plants at high cost or buying power on the spot market. Bringing customers into the energy information network could significantly change the way the entire industry operates.27

The New Competitive Frontier

Information insights such as these are the new tools of competitive advantage. Of late, much of the business buzz has focused on “continuous innovation” (which sounds exciting but can be exhausting), on providing better products and services through in-depth knowledge of the customer and on providing services that are rich in context using technology advances such as Web services. Innovation and the move into services will continue to be important. But future advantage will ride on an ability to make your offerings more valued through the beneficial use of a variety of kinds of data.

If used well, these data will enable you to meet business challenges by responding in essentially real time or by predicting the course of events and responding in order to change the outcome, as each aspect of the business requires. As information becomes more abundant, the edge will go to those who can acquire it wisely and use it creatively. It’s ironic that after accounting for all the technological advances of the past and future, sustainable competitive advantage will rely on two very human characteristics: insight and trust.