YARN architecture:
1)ResourceManager:RM
There is only one RM that provides services for the entire cluster at the same time, which is responsible for the unified management and scheduling of cluster resources.
Process client requests: submit jobs, kill jobs
2)NodeManager:NM
There are several in the entire cluster, responsible for the management and use of their own node resources.
Periodically report the resource usage of the node to the RM.
Receive and process various commands from RM: start Container
3)ApplicationMaster:AM
Each application corresponds to one: spark, MR... Responsible for application management
Request resources (core, memory) from RM for the application to allocate to internal tasks
Need to communicate with NM: start/stop task
4)Container
A container that encapsulates resources such as CPU and Memory
is an abstraction of the task execution environment
5)Client
submit homework
Query job running progress
kill job
YARN environment construction:
Version used: hadoop-2.6.0-cdh5.7.0
yean-site.xml
mapred-site.xml
Start and stop YARN