HADOOP task scheduler

  • FIFO scheduler

    • FIFO scheduling is also FIFO (First In First Out) scheduler. FIFO scheduling strategy scheduler is a first application of Hadoop is that the cluster can only have one job while running. All the Application filed in the order of time to perform only after the completion of the implementation of a Job Job will turn back to be executed in the order queue. FIFO scheduler to cluster resources exclusive way to run a job, this benefit is a job can make full use of all cluster resources, but for the running time is short, high importance or MR will work interactively query like waiting in platoon sequence before the job is completed in order to be implemented, which also led to a very big if job is running, the back of the job will be blocked

  • Capacity Scheduler

    • Capacity Scheduler i.e. container scheduler. It can be appreciated that resource into one queue. This resource queue is assigned to the user's own

  • Fair Scheduler

    • Fair is fair scheduler scheduler. Fair scheduler queue is a resource allocation scheme, the entire timeline, all Job average access to resources. By default, Fair scheduler just do a fair scheduling and allocation of memory resources. When the cluster has only one task at run time, then the task will occupy resources of the entire cluster. When other tasks submitted, the release of those resources will be allocated to the new Job, so each task can eventually get nearly as much resources

  • If the business logic is simple recommends using FIFO scheduler

  • If you need to control the case of the priority part of the application at the same time wants to make full use of cluster resources, it is recommended to use the scheduler Capacity

  • If you want a multi-user or multi-queue fair share cluster resources, then choose Fair Scheduler

Guess you like

Origin www.cnblogs.com/xiangyuguan/p/11367362.html