Deep dive hadoop

75 %
25 %
Information about Deep dive hadoop

Published on May 31, 2014

Author: jaksky


“Deep dive”

Key takeaways • NameNode is critical to cluster • NameNode doesn’t equal to SecondaryNameNode, no back up etc. • Client access cluster nodes • NameNode doesn’t take part in Data transfer


YARN – Map-Reduce

Key takeaways • YARN – promotes hadoop cluster to “universal computational cluster” • Map-Reduce is just one application running on cluster • Hadoop is not just a Map- Reduce since Hadoop 2.0

High Availability • Issue for Hadoop 1.x – NameNode SPOF – Problems with cluster maintenance – “Split the brain scenario” – “Shoot me in the HEAD” • Solutions: – NFS – Facebook’s “Avatar Node” – Hadoop 2.0 • Things to consider – Cold, Warm or Hot stand by – Manual, Semi-automated, Automated failover

Hadoop 2.0 HA – Key points • Hadoop HA doesn’t influence just HDFS • Provides semi-automated or automatic failover • Simplifies cluster maintenance • Complicates node installations • Cluster operations more complicated

Cluster processes Hadoop • NameNode • SecondaryNameNode • DataNode • ResourceManager • NodeManager • ZooKeeper • JournalNode • ZKFailOverControler • History server Hbase • HMaster • RegionServer • ZooKeeper

Service Profiles – Node Roles

Takeaway • HDFS should be “SAFE” – Possibility to protect data • Resource-Manager is now SPOF – We may not be able to process data in cluster

Add a comment

Related pages

Download the Hadoop Deep Dive | InfoWorld

Businesses are using Hadoop across low-cost hardware clusters to find meaningful patterns in unstructured data. In this in-depth PDF, InfoWorld explains ...
Read more

A Deep Dive Into Hadoop - TechWise Episode 1 Transcript

Takeaway: Host Eric Kavanagh discusses Hadoop, where it's been and where it's going with industry insiders.
Read more

Hadoop Training 2 : Deep Dive In HDFS (What is Hadoop ...

Want to watch this again later? Sign in to add this video to a playlist. Full Hadoop Training is in Just $69/3500INR visit : ...
Read more

Hadoop: A Deep Dive | WANdisco

Hadoop: A Deep Dive. This webinar will provide a holistic view of Hadoop and the components that make up its ecosystem. This webinar will also cover how to ...
Read more

TSM - Hadoop MapReduce deep diving and tuning

Hadoop MapReduce deep diving and tuning. ... Soon Apache Hadoop 1 reached a very strong community and big players, ... MapReduce deep dive.
Read more

Apache Hadoop - A Deep Dive (Part 1 - HDFS) - YouTube

This is our next tech talk in the series where we dive deep into the Apache Hadoop framework. Hadoop, undoubtedly is the current industry leader ...
Read more

DeepDive Infotech, Providing Trainings on Latest ...

Do you know that 90% of current data is generated in last 2 years and in next 3 years all these data will move on hadoop....So this is the best opportunity ...
Read more

Hadoop Syllabus - DeepDive Infotech

Introduction of hadoop. Hadoop: Basic concepts; Why Hadoop? What is Hadoop; History of Hadoop; Hadoop comparison with traditional systems; Hadoop 2.x core ...
Read more