HADOOP Administration (Cloudera & Apache)

Hadoop admin is responsible for capacity planning and estimating the requirements for lowering or increasing the capacity of the Hadoop cluster. Hadoop admin is also responsible for deciding the size of the Hadoop cluster based on the data to be stored in HDFS.

Course Overview

Course Objective

What will you learn?

Who Should Learn

Who should go for this training?
The following professionals can go for this course:

Prerequisite

What are the pre-requisites for this Course?

Course Content

1. Introduction to Big Data & Hadoop

  • Hadoop Introduction
  • Big Data usage & benefits
  • Hadoop Architecture
  • Mapreduce 1.0 & 2.0
  • Hadoop cluster setup
  • File read / write operations

3. Understanding & Installing Cloudera Hadoop

  • Creating Yum Repository
  • .repo files
  • Host inspectors
  • Block Size
  • Mount directories for DataNodes , NameNode, Secondary Namenode

5. Working with Metadata

  • Fsimage
  • Edit logs
  • Core-site.xml
  • Hdfs-site.xml
  • Secondary Namenode

7. Setup Hadoop0 - Hadoop High Availability (HANamenode’s & Zookeeper)

  • Cloudera Manager
  • Setup Zookeeper services Cloudera Manager
  • Setup Journal Nodes using Cloudera Manager
  • Active & Standby Name Nodes
  • Understanding Zookeeper & Journal Nodes
  • High availability configurations (core-site.xml / hdfs-site.xml)
  • Understanding cdhns

9. Setup YARN-Map Reduce0 and Resource Manager & Node Managers

  • Add YARN service
  • YARN Architecture
  • Application Masters
  • Resource Managers
  • Containers
  • Cluster Metrics

11. Hadoop Cluster Planning

  • Typical Configuration, HA Configuration, Federation Configurations
  • Hardware consideration for slaves and servers
  • Awareness of hyperthreading, blade servers
  • Operating system considerations
  • Workload, CPU, memory, i/o, storage, disks, JBOD, RAID’s etc
  • Recommendations by Cloudera
  • Storage Intensive, Compute intensive, Balanced configurations
  • Network designs for Hadoop

13. PIG on Hadoop cluster

  • Understanding Pig latin
  • Grunt shell, local mode and MR mode
  • Testing & Executing PIG jobs
  • Executing Big Data analysis using PIG

15. Adding and Removing Hadoop nodes

  • Decommission/Recommission nodes
  • Add/remove Hadoop nodes
  • Working with cloudera-scm-agent
  • Running balancer
  • Creating and applying host templates
  • Deploy Client Configuration

17. Troubleshooting - Hadoop administration

  • Hadoop Damemon logs
  • Understanding common error & troubleshooting them

19. Installing a MultiNode Apache Hadoop Cluster

  • Setup Environment Variables
  • Setup Core-site.xml
  • Hdfs-site.xml
  • Make datanode directories
  • dfsadmin

21. 2Exploring HDFS

  • File system commands (fs, jar, fsck)
  • HDFS Balancer
  • Quota
  • HDFS health checks

23. Exploring YARN / MRv2

  • Configure yarn-site.xml & mapred-site.xml
  • Running MRv2 jobs

4. Exploring HDFS

  • HDFS logs directories
  • Observing charts
  • Setup root user
  • Dfs.blocksize, Data.dir etc properties
  • property
  • Fault tolerance
  • Checksum
  • Working with HDFS Blocksize
  • HDFS replication
  • Observing Namenode UI
  • Block pool

6. Setup Hadoop0 - Namenode Recovery & Secondary Namenode

  • Dfsadmin
  • Safemode
  • saveNamespace
  • fsimage / edits log
  • Namenode recovery after server crash process
  • Datanode registrations

8. HDFS Commands

  • File system commands (fs, jar, fsck)
  • Fs – appendToFile,
  • put, copyToLocal,
  • chown, chgrp,
  • df, du
  • moveFromLocal, moveToLocal
  • stat etc

12. 1Setup external databases – MYSQL

  • mysql-connector-java
  • mysql_secure_installation

14. Setup HIVE on Hadoop cluster

  • Setup Metastore on MySql DB
  • HIVE architecture
  • Understanding HIVE warehouse directory
  • Hive logs
  • Creating HIVE tables & testing HIVE
  • Executing Big Data analysis using HIVE

16. Backup’s and Migrations

  • Data backup using SNAPSHOTS
  • Data migrations using ‘distcp’ utility

18. Apache Hadoop Installing a Single Node Apache Hadoop (HDFS& YARN Cluster)

  • Getting Ready
  • Setup Environment Variables
  • Test HDFS

20. Installing Gateway / Edge Node

  • Understanding Gateway node functionality
  • Setup hdfs-site.xml

22. 2Rack Awareness Setup

  • Understanding rack awareness
  • Setup topology.script
  • Topology.data

24. Configure High Availability for HDFS

  • Zookeeper configuration
  • Setup Journal Nodes
  • Setup automatic failover
  • Setup folder structures for HA
  • About QuorumPeerMain

Modes of Training

Classroom Training

Live interactive sessions delivered in our classroom by our expert trainers with real-time scenarios.

Online Training

Learn from anywhere over the internet, joining the live sessions delivered by our expert trainers.

Self-Pace Training

Learn through pre-recorded video sessions delivered by experts at your own pace and timing.

Frequently Asked Questions

Who are the trainers?

Our trainer is an OCP & OCM certified consultant and has a significant amount of experience in working with the technology, having 18yrs of experience.

How do I join the live sessions?

Once you get registered, our back-end team will share you the details to join the session live over an online portal which can be accessed through a browser.

What if I miss a class?

Each of our live sessions is recorded. In case if you miss any, you can request us to share the link to that particular session.

How will I execute the practical?

For practical execution, our trainer/technical team will provide server access details to the student

Will you help us in building our own Lab Setup in our local Desktop/Laptop?

Yes. We do provide the step-by-step document which you can follow and if required our technical team will assist you.

What is the difference between live online and pre-recorded video training?

Live-Online training is where you can have a live session with the trainer and clarify queries parallelly.

Pre-recorded sessions are the recorded videos that will be provided to you that you can see, listen, and learn anytime at ur feasible place. For doubts in the videos, you can mail the trainer regarding the same.

What if I have queries after completion of the class?

You can contact our support team, or just drop an email to help@sarinfotechindia.com with your queries.

Is the course material accessible after completion of the course?

The course material and recorded videos which are provided during the course period. You can download it anytime.

Any offer/discount that I can avail?

Visit our website regularly to check discounts offers from time to time. However, we provide a discount for single participants & special discounts for 2 or more participants.

Will I get the refund if I cancel my enrollment?

* If the request for cancelation is made within 2 days of enrolment for class, 100% refunded.

* If the request made after 2 days, then Refund is made after deduction of the administration fee.