Single Node to Distributed System

  • By
  • on Feb 24, 2017
  • in Linux

Are the concepts of Distributed System Unique or Are they borrowed from Single Node System

The important resources at disposal on a single node are

  1. CPU
  2. RAM
  3. Network
  4. File-system

 

Single Node and their Counter component on a Distributed System(YARN + HDFS )

Single Node Distributed Systems
Example  Linux Box Hadoop Stack
File-system  extfs HDFS
CPU CPU and cores CPU and Cores across multiple machine(YARN)
RAM RAM on a single machine RAM across multiple machines(YARN)
Process Process Process across multiple machine(YARN)

 

[addToAppearHere]

 

  1. File-system
Single Node Distributed Systems
File-system extFS HDFS
File File is broken in tracks and sectors and saved on same machine File is divided  HDFS blocks . The blocks are saved across machines
Command to know where a file is saved stat fileName hadoop fsck /path/to/file -files -blocks
To enable Data recovery in case of failure extfs provide journaling maintains Replica of blocks

 

 

 

Tags: , ,