http://elsevierconnect.com/harnessing-the-power-of-big-data-is-no-small-feat/

Big data is not new, citing the financial industry, telecommunications industry and astronomy as just a few examples of areas in which large amounts of data have been routinely processed for a long time. “What is new is that any one of us can access big data now, and so today there are many data enthusiasts,” she said.But easy access doesn’t mean easy analysis. “Data analysis can’t just be done with computation; you need to have the human in the loop,” she said. “Getting data from multiple sources, integrating and exploring the data to actually get insights from it — that’s hard.”The path from data to knowledge requires the use of methodologies such as machine learning, visualization, statistics and algorithms before anyone can begin to make sense of the input. What’s missing are tools and techniques that  automate as many of the tedious tasks as possible, enabling enthusiasts and experts alike to more rapidly draw useful conclusions.

Dr. Ramakrishnan went on to describe how algorithms developed from big data underpin virtually all Web search results, aggregating data so the user doesn’t just see specific web pages but also suggestions for refining a query (“semantic refinement”), maps if the query is about a specific place or entity, and relevant images, news and so forth. Those algorithms also underlie content optimization and recommendations based on what users are most interested in (“popularity”), users’ habits (“National Football League news up front over coffee in the morning, political news at night”) as well as users’ expressed preferences. Big data also enables the analysis of behavior when people are engaged in applications such as Microsoft’s Office 365, which provides cloud-based delivery of email, calendars, teleconferencing and the like; by analyzing user activity patterns, developers can make common functions easier to access — one click away versus those that take several clicks —  thereby optimizing the underlying process to make users more productive.

As another example, big data analysis of data from applications like Microsoft Kinect, which is mainly used for game playing, can help determine whether someone has a hip or knee problem that should be looked into by the way they move (gait analysis), Dr. Ramakrishnan said. And on a larger scale, big data will eventually enable responses to such questions as: What would we do if a new disease hits wheat? Will forests accelerate or slow climate change? How many species are there on Earth, and how can we predict them?

Those are some of the positive sides of big data. At the same time, a host of social, legal and regulatory issues need to be worked out, including privacy concerns and ethical considerations. “These will take longer to understand and resolve,” Ramakrishnan acknowledged. Moreover, as the other speakers noted, there is still a large gap between the availability of big data and people qualified to interpret it; nearly 200,000 people in the United States alone are needed to work with various aspects of big data to help maximize its potential, he said, citing a recent McKinsey & Co. report.

- See more at: http://elsevierconnect.com/harnessing-the-power-of-big-data-is-no-small-feat/#sthash.fv2w6Tjs.dpuf

A myth about big data is that it’s all about volume and size, Dr. Weikum noted. Many applications of big data deal with terabytes and gigabytes or less, he pointed out, but agreed with Dr. Freire that even those relatively smaller volumes can be daunting to handle. Using an analogy from mathematics, Dr. Weikum explained that in standard geometry, the term “volume” encompasses three dimensions: length, width and depth.Moving from the back end of big data management to current and potential applications, Dr. Ramakrishnan said big data “is not a specific technological innovation; it’s about being able to do things that would have been pipe dreams just a few years ago. Data is literally the new gold — data mining, the new Klondike.”(Read the full post, it's good, had to post a bit more here though)Dr. Ramakrishnan went on to describe how algorithms developed from big data underpin virtually all Web search results, aggregating data so the user doesn’t just see specific web pages but also suggestions for refining a query (“semantic refinement”), maps if the query is about a specific place or entity, and relevant images, news and so forth. Those algorithms also underlie content optimization and recommendations based on what users are most interested in (“popularity”), users’ habits (“National Football League news up front over coffee in the morning, political news at night”) as well as users’ expressed preferences.Big data also enables the analysis of behavior when people are engaged in applications such as Microsoft’s Office 365, which provides cloud-based delivery of email, calendars, teleconferencing and the like; by analyzing user activity patterns, developers can make common functions easier to access — one click away versus those that take several clicks —  thereby optimizing the underlying process to make users more productive.As another example, big data analysis of data from applications like Microsoft Kinect, which is mainly used for game playing, can help determine whether someone has a hip or knee problem that should be looked into by the way they move (gait analysis), Dr. Ramakrishnan said. And on a larger scale, big data will eventually enable responses to such questions as: What would we do if a new disease hits wheat? Will forests accelerate or slow climate change? How many species are there on Earth, and how can we predict them?Those are some of the positive sides of big data. At the same time, a host of social, legal and regulatory issues need to be worked out, including privacy concerns and ethical considerations. “These will take longer to understand and resolve,” Ramakrishnan acknowledged. Moreover, as the other speakers noted, there is still a large gap between the availability of big data and people qualified to interpret it; nearly 200,000 people in the United States alone are needed to work with various aspects of big data to help maximize its potential, he said, citing a recent McKinsey & Co. report.

Harnessing the power of big data is no small feat Applications can yield insights into science, business and where to catch a taxi — but better tools and more analysts are needed, experts say - See more at: http://elsevierconnect.com/harnessing-the-power-of-big-data-is-no-small-feat/#sthash.fv2w6Tjs.dpuf

Harnessing the power of big data is no small feat Applications can yield insights into science, business and where to catch a taxi — but better tools and more analysts are needed, experts say - See more at: http://elsevierconnect.com/harnessing-the-power-of-big-data-is-no-small-feat/#sthash.fv2w6Tjs.dpuf