Tuning of HIVE JVM reuse

  • The default configuration of Hadoop is usually derived using JVM to perform map and Reduce tasks. At this point the JVM startup process can cause considerable overhead, especially job execution contains hundreds task task situation. JVM JVM instances such reuse may be reused in the same N-th job in

  • This shortcoming function is turned on JVM reuse will always take up the task to use slots for reuse, until the task is completed to release. If an "unbalanced" job there are a few reduce task execution time of other much more time consuming than the Reduce task, then the reserved slots would have been idling can not be used by other job, until all of the task was over before release

Guess you like

Origin www.cnblogs.com/xiangyuguan/p/11416341.html