Big Answers

In a recent Economist story two striking facts extracted from Big Data were reported:

Best Buy, a retailer, found that 7% of its customers accounted for 43% of its sales.

and

In Britain the Royal Shakespeare Company (RSC) sifted through seven years of sales data for a marketing campaign that increased regular visitors by 70%.

And it's not hard to find other stories indicating that enormous gains in revenue can be extracted from the data that companies already have. For example, the web site NapaStyle.com reports a 10% lift in revenue per visitor after two months of mining customer clicks to understand behavior. World Kitchen reported an average order value increase of 20% using similar technology.

Anecdotal evidence says that Amazon.com is getting an additional 10% sales revenue by mining the books that its customers buy to recommend other titles they might like. And Netflix gets 60% of its rentals from people clicking through on similarly mined data guiding its recommendations.

Startup FlightCaster is taking a Big Data approach mining airline records, weather and FAA real-time traffic information to provide accurate predictions of flight delays well ahead of the airlines themselves. Their application provides flight delay information 6 hours before the airlines tell their own customers.

Fox Audience Network, which handles advertising across all Fox properties (including MySpace), is using a large cluster of machines crunching Big Data to obtain an order of magnitude increase in advertising margins.

And Europe's largest ad targeting network, nugg.ad is using Big Data to drive everything they do.

Users of Big Data shouldn't be surprised by Big Answers: Big Data represents a massive untapped resource that all companies already have. It's just a question of integrating the right software to obtain Big Answers.

John.

The Economist catches the Big Data wave

At Causata we spend all our time with Big Data and so it's nice to see the Economist (current issue) produce a special report (written by Kenneth Cukier) about the explosion of data and the technologies needed to process it.

The business of information management—helping organisations to make sense of their proliferating data—is growing by leaps and bounds. In recent years Oracle, IBM, Microsoft and SAP between them have spent more than $15 billion on buying software firms specialising in data management and analytics. This industry is estimated to be worth more than $100 billion and growing at almost 10% a year, roughly twice as fast as the software business as a whole.

In addition to the special report there's an interview with the report's author:

Part of the Economist article was driven by a report from O'Reilly called Big Data: Technologies and Techniques for Large-Scale Data.

O'Reilly produced a set of interviews to accompany that report. Here's the first:

The rest can be found here.

John.