Sign up FAST! Login

How ‘Big Data’ Is Different | MIT Sloan Management Review

How Big Data Is Different MIT Sloan Management Review


Organizations that capitalize on big data stand apart from traditional data analysis environments in three key ways:

  • They pay attention to data flows as opposed to stocks.
  • They rely on data scientists and product and process developers rather than data analysts.
  • They are moving analytics away from the IT function and into core business, operational and production functions.

There are several types of big data applications. The first type supports customer-facing processes to do things like identify fraud in real time or score medical patients for health risk. A second type involves continuous process monitoring to detect such things as changes in consumer sentiment or the need for service on a jet engine. Yet another type uses big data to explore network relationships like suggested friends on LinkedIn and Facebook. In all these applications, the data is not the “stock” in a data warehouse but a continuous flow. This represents a substantial change from the past, when data analysts performed multiple analyses to find meaning in a fixed supply of data.

The behavior of credit card companies offers a good illustration of this dynamic. In the past, direct marketing groups at credit card companies created models to select the most likely customer prospects from a large data warehouse. The process of data extraction, preparation and analysis took weeks to prepare — and weeks more to execute. However, credit card companies, frustrated by their inability to act quickly, determined that there was a much faster way to meet most of their requirements. In fact, they were able to create a “ready-to-market” database and system that allows a marketer to analyze, select and issue offers in a single day. Through frequent iterations and monitoring of website and call-center activities, companies can make personalized offers in milliseconds, then optimize the offers over time by tracking responses.

Early users of big data are also rethinking their organizational structures for data scientists. Traditionally, analytical professionals were often part of internal consulting organizations advising managers or executives on internal decisions. However, in some industries, such as online social networks, gaming and pharmaceuticals, data scientists are part of the product development organization, developing new products and product features. At Merck & Co. Inc, for example, data scientists (whom the company calls statistical genetics scientists) are members of the drug discovery and development organization.

Another approach to managing big data is leaving the data where it is. So-called “virtual data marts” allow data scientists to share existing data without replicating it. EBay Inc., for example, used to have an enormous data replication problem, with between 20- and 50-fold versions of the same data scattered throughout its various data marts. Now, thanks to its virtual data marts, the company’s replication problem has been dramatically reduced. EBay has also established a “data hub” — an internal website to make it easier for managers and analysts to serve themselves and share data and analyses across the organization. In effect, eBay has built a social network around analytics and data.

Coming to terms with big data is prompting organizations to rethink their basic assumptions about the relationship between business and IT — and their respective roles. The traditional role of IT— automating business processes — imposes precise requirements, adherence to standards and controls on changes. Analytics has been more of an afterthought for monitoring processes and notifying management about the anomalies. Big data flips this approach on its head. A key tenet of big data is that the world and the data that describe it are constantly changing, and organizations that can recognize the changes and react quickly and intelligently will have the upper hand. Whereas the most vaunted business and IT capabilities used to be stability and scale, the new advantages are based on discovery and agility — the ability to mine existing and new data sources continuously for patterns, events and opportunities.

Stashed in: Big Data!, MIT TR

To save this post, select a stash from drop-down menu or type in a new one:

You May Also Like: