Spark task execution

 

Four nodes, Driver and Worker is to start on the node processes, processes running in the JVM.

  • Driver frequent communications between the cluster nodes
  • Driver is responsible for tasks (tasks) the distribution and recovery results. Scheduling tasks. If the result of a very large task, do not recovered. Cause oom
  • Worker node from inside the Standalone Resource Scheduling Resource Management framework. It is the JVM process
  • Master is the master node which Standalone resource scheduling resource management framework. It is the JVM process

Guess you like

Origin www.cnblogs.com/xiangyuguan/p/11203162.html