Runaway Complexity in #bigdata and a plan to stop it - can you afford not to read this?
Mo Data stashed this in Big Data Philosophy
That's actually pretty old. Since then, Nathan published an awesome book: http://www.amazon.com/Big-Data-Principles-practices-scalable/dp/1617290343
which incorporates these thoughts.
This book, defines lambda architecture, which seem to be quickly becoming a buzz word.
It now being questioned by linkedin guys here: http://radar.oreilly.com/2014/07/questioning-the-lambda-architecture.html
What do you think, Sergey? Is lambda as good as Nathan says?
I guess there's no silver bullet.
Keep the design as simple as possible, and don't try to use a tool just because it's available.
I do think lambda got a lot of good ideas, but it's not specs like J2EE. It's high level architecture and it's up to the project team to take these ideas and implement as needed for specific project. The book is very good. Unlike typical <<Product X Bible>> book, it actually sets the process for reasoning about building big data systems. It does not just pushes certain technologies, but sets the operational requirements and explains why these technologies are chosen to meet these requirements.
Unlike typical <<Patterns and Principles>> book it offers some solutions and demonstrates how to use these.