When organisations are failing to operationalise the insight from small data, it leads me to wonder how big data avoids becoming another management panacea.

http://www.forethought.com.au/ArticleDocuments/86/Forethought%20White%20Paper%20-%20Big%20Data%20Pandemic.pdf.aspx?Embed=Y

I often wonder if herd instinct is a core subject taught at business schools. Recently, Forethought has been witnessing the adoption velocity of big data as vendors go house to house evangelising the next managerial panacea. Although, for a long time there has been an abundance of data, the big data pandemic is only now in full swing.

Anderson argues that we can stop looking at marketing models. ‘We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where [marketing] science cannot.’Is he right? Not yet. Examples of errors in big data can be readily sourced. For example, Naturevirecently reported that Google Flu Trends had markedly overestimated the proportion of North Americans to get the fl u in 2012/13 (See Figure 1). Nature reported that for big data based Google Flu Trends ‘The latest US fl u season seems to have confounded its algorithms. Its estimate for the Christmas national peak of fl u is almost double the estimate from the USA Center for Disease Control and some of its state data show even larger discrepancies.’

Ideally it is the integration of big data, software and small data’s great marketing scientists that brings about the best outcome. Too often in the big data sales literature the human component to the successful application of big data is understated.

Perhaps it is not about big data versus small data but rather, big data and small data combining to produce synergistic insight.