Yarn Scheduler Load Simulator (SLS)-调度算法选择源码解析

如何为SLS选择调度算法。这里谈下本人的理解

sls通过slsrun.sh脚本来运行，其中要加入配置文件sls-runner.xml。在该配置文件中，有关于调度算法的配置：

  </property>  
  <property>  
    <name>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler</name>  
    <value>org.apache.hadoop.yarn.sls.scheduler.FifoSchedulerMetrics</value>  
  </property>  
  <property>  
    <name>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</name>  
    <value>org.apache.hadoop.yarn.sls.scheduler.FairSchedulerMetrics</value>  
  </property>  
  <property>  
    <name>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</name>  
    <value>org.apache.hadoop.yarn.sls.scheduler.CapacitySchedulerMetrics</value>  
  </property>  

这是配置了三种调度算法的具体实现在哪。

但没有选择用哪种算法。

算法的选择在yarn配置时就确定好了，在yarn-site.xml文件中进行配置。

Property===>yarn.resourcemanager.scheduler.class

Value=====>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler

在sls源码中也可以看出：

/hadoop-trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java

  private void startRM() throws ClassNotFoundException, YarnException {
    Configuration rmConf = new YarnConfiguration(getConf());
    String schedulerClass = rmConf.get(YarnConfiguration.RM_SCHEDULER);

    if (Class.forName(schedulerClass) == CapacityScheduler.class) {
      rmConf.set(YarnConfiguration.RM_SCHEDULER,
          SLSCapacityScheduler.class.getName());
      rmConf.setBoolean(YarnConfiguration.RM_SCHEDULER_ENABLE_MONITORS, true);
      rmConf.set(YarnConfiguration.RM_SCHEDULER_MONITOR_POLICIES,
          ProportionalCapacityPreemptionPolicy.class.getName());
    } else if (Class.forName(schedulerClass) == FairScheduler.class) {
      rmConf.set(YarnConfiguration.RM_SCHEDULER,
          SLSFairScheduler.class.getName());
    } else if (Class.forName(schedulerClass) == FifoScheduler.class) {
      // TODO add support for FifoScheduler
      throw new YarnException("Fifo Scheduler is not supported yet.");
    }

    rmConf.set(SLSConfiguration.METRICS_OUTPUT_DIR, metricsOutputDir);

    final SLSRunner se = this;
    rm = new ResourceManager() {
      @Override
      protected ApplicationMasterLauncher createAMLauncher() {
        return new MockAMLauncher(se, this.rmContext, amMap);
      }
    };

    // Across runs of parametrized tests, the JvmMetrics objects is retained,
    // but is not registered correctly
    JvmMetrics jvmMetrics = JvmMetrics.initSingleton("ResourceManager", null);
    jvmMetrics.registerIfNeeded();

    // Init and start the actual ResourceManager
    rm.init(rmConf);
    rm.start();
  }

在start RM开始时，首先从YARN configuration（我猜想是yarn-site.xml）中读取Scheduler的类别，也就是三种中的一种，然后与sls的sls-runner.xml的配置值相比，找到name相对应的配置值。到此，选择好了算法。