hadoop YARN

YARN architecture:

1)ResourceManager:RM

There is only one RM that provides services for the entire cluster at the same time, which is responsible for the unified management and scheduling of cluster resources.

Process client requests: submit jobs, kill jobs

 

2)NodeManager:NM

There are several in the entire cluster, responsible for the management and use of their own node resources.

Periodically report the resource usage of the node to the RM.

Receive and process various commands from RM: start Container

 

3)ApplicationMaster:AM

Each application corresponds to one: spark, MR... Responsible for application management

Request resources (core, memory) from RM for the application to allocate to internal tasks

Need to communicate with NM: start/stop task

 

4)Container

 A container that encapsulates resources such as CPU and Memory

is an abstraction of the task execution environment

 

5)Client

submit homework

Query job running progress

kill job

 

YARN environment construction:

Version used: hadoop-2.6.0-cdh5.7.0

yean-site.xml

mapred-site.xml

Start and stop YARN

 

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=325238727&siteId=291194637