The difference and connection between YARN and Zookeeper
Distributed systems have many problems, two of them
- Coordination
- Resource Management
Zookeeper focuses on solving the former and
Yarn focuses on solving the latter. For example, if there is something to be done now, we need to notify yarn if we want this system to do this. For example, the masternode of yarn looks at the many nodes in the system, and sees who is free and who has the ability (cpu, memory, etc.) to do this. Then assign the task to the appropriate node to do this.
YARN is equivalent to the government, responsible for managing the allocation of machine resources. If you have difficulties, you can find the government. The government arranges manpower and material resources to solve your problems. This is the management node you are talking about.
Zookeeper is equivalent to the legislative committee, the goal is to provide authoritative information to all people in this country. Zookeeper focuses on the consistency of information, and manages some information that needs to be consistent in everyone's minds. Although it is often used to manage information such as which node is alive, which node is dead, or which node is responsible for what, it is still the department mainly responsible for managing information (files, law), not the people's livelihood department...
The functions of Yarn and Zookeeper are different. For a complete system, both need to be used.