YARN framework: Resource Scheduling
In the execution job.waitforcompletion (true) when it will start a process Runjar, equivalent MapReduce programs submitted by the client, with the cluster rely on this communication
1.Runjar to apply for a job ResourceManager
2.ResourceManager returns a resource to job submission Runjar path (staging- dir) and generates a job-based jobID
3.Runjar submit up resources to HDFS
4.Runjar submit the results to the ResourceManager report
5.ResourceManager this job to join the task queue
6.NodeManager task to get through the heartbeat mechanism ResourceManager
Container 7.NodeManage generation, allocation of resources to run the container, including CPU, memory and other resources; resource files will also be loaded into the container
At this time code, configuration files, have the resources needed to perform in the container in MapReduce, which then run on a single machine, each node performs several map, several reduce, these YARN do not know, only YARN Resource Management it can not go start mapreduce
MRAPPMaster package implemented by MapReduce framework is mapreduce manager application, which is initiated by the frame YARN
8. Start MRAPPMaster, this task is substantially complete frame YARN
9.MRAPPMaster registered with the ResourceManager, which gets run at the node
map task process called yarnChild
10.MRAPPMaster start the process task map task
11.MRAPPMaster start the process reduce task task
executing the map, MRAPPMaster recovery will be notified YARN
After the completion of MRAPPMaster 12.job to deregister its ResourceManager
YARN framework, the main role is ResoucceManager -------- management NodeManaer
MapReduce framework summary, the main role is MRAPPMaster ----------- management yarnChild
Source: webmaster platform