Wednesday, September 03, 2014


The rise of BigData is now prompting engineering teams around the world to graple the last mile that comes along with huge data sets: how do you turn them to actionable insights? A picture is worth a thousand numbers, naturally, data visualization is the preferred medium.

If you start separating the tiers required for managing tons of data for visualization, the need for a fast searchable data store with analytical capabilities will emerge. This is where elasticsearch comes into the picture. Its a swiss army knife stack that can be used as a regular lucene backed data store, or call in the power of aggregations (facet replacement) and you have a decent analytical engine.

The question is where does Hadoop/Teradata maps into this in the overall scheme of Big Data. I'd say if your data set is only in the few GBs a clustered Elasticsearch is all that you need. If your data is running in terrabytes then its probably worth to have that in a Hadoop cluster and run your jobs there that either feeds in back to Elasticsearch or use Elasticsearch as one of the inputs to your favorite Hadoop script.

On one end, I can't believe this prompted me to write a three paragraph blog post!


Mathew Stephen said...

There are lots of information about latest technology, like Hadoop cluster is a special type of computational cluster designed specifically for storing and analyzing huge amounts of unstructured data in a distributed computing environment. This information seems to be more unique and interesting. Thanks for sharing.
Big Data Training | Big Data Hadoop Training in Chennai | Big Data Course in Chennai

Andrew Son said...

Excellent!blog of information and very interesting.Hadoop is especially used for Big Data maintenance. It uses Hadoop distributed file system . Its operating system is in cross platform. Its framework mostly written in java programming language. The other languages which are used by hadoop framework are c, c++ (c with classes) and sometimes in shell scripting. Thank you for your information.
Hadoop Training in Chennai | Big Data Training in Chennai