Yarn’s task submission process

When jobclient submits an application to Yarn, Yarn will run the application in two stages:

• One is to start AM (ApplicationMaster);

• The second is for AM to create the application, apply for resources for it, and monitor the operation until the end.

Specific steps are as follows:

1. The user submits an application to Yarn and specifies the AM program, the command to start AM, and the user program.
2. RM (ResourceManager) allocates the first Container to this application and communicates with its corresponding NM (NodeManager), asking it to start the application AM in this Container.
3. AM registers with RM, then splits it into internal subtasks, applies for resources for each internal task, and monitors the running of these tasks until the end.
4. AM applies for and receives resources from RM in a polling manner.
5. RM allocates resources to AM and returns them in the form of Container.
6. After the AM applies for resources, it communicates with the corresponding NM and requires the NM to start the task.
7. NM sets up the running environment for the task, writes the task startup command into a script, and starts the task by running this script.
8. Each task reports its status and progress to AM so that the task can be restarted when it fails.
9. After the application completes, AM logs out to RM and shuts down itself.

---

See you next time, bye!