hadoop start
This script is Deprecated. Instead use start-dfs.cmd and start-yarn.cmd
starting yarn daemons
What is yarn used for?
Manage resources
The operating mechanism of yarn:
YARN provides its core services through two types of long-running daemons:
- A resource manager that manages the use of resources on the cluster (resource manager)
- A node manager that runs on all nodes in the cluster and can start and monitor containers
Containers are used to execute processes of specific applications , and each container has resource limits (memory, CPU, etc.).
A container can be a Unix process or a Linux cgroup, depending on the YARN configuration .
The following figure describes how YARN runs an application.
Process:
1. The client submits a job to yarn and first finds ResourceManager to allocate resources
2. ResourceManager opens a Container and runs an Application manager in the Container
3. The Application manager finds a nodemanager to start a Container , runs the Application master in the Container, and the Application master calculates how many resources are needed
4. The Application master applies to the Application manager (Yarn) for the resources needed to run the task
5.Resource scheduler sends the resource package to the Application master
6. The application master allocates the acquired resources to each nodemanager , including the nodemanager currently running the Application master
7. Each nodemanager gets tasks and resources and starts to execute map task
8. After the execution of the map task is over, start to execute the reduce task
9. The map task and reduce task feedback the execution results to the Application master
10. The application master feeds back the results of task execution to the application manager
