(1) Comparison of hadoop1.x and hadoop2.x
hadoop1.x：
- JobTracker does resource management and task scheduling, and there is a single point of problem. If the cluster expands and there are more TaskTrackers, the pressure on JobTracker will increase (because it is one-to-many, there is only one JobTracker); if the JobTracker hangs up, it cannot be submitted.
- Only supports mr jobs, not other types of jobs (spark, etc.)
- Static resource allocation, poor resource utilization
hadoop2.x：
- Support other jobs than mr
- You can improve resource utilization by configuring the priority of the task queue, running various jobs, etc.
- 1 resourmanager corresponds to multiple nodemanagers, and resourcemanager supports HA (but there are problems, and additional operations are required to achieve true HA)
(2) What are the responsibilities of ApplicationMaster?
When the job is submitted to the ResourceManager, it will start a Container, and start the ApplicationMaster corresponding to the job on the Container (MR job is the MR Application Master, which may also be Spark). The ApplicationMaster applies to the ResourceManager for computing resources, and starts the Container on the corresponding NodeManager. Program, ApplicationMaster completes the communication request with ResourceManager voluntarily, communicates with NodeManager to complete task distribution, start and stop, job fault tolerance, etc.