Spark foreach和foreachPartition的区别

object foreach&foreachPartition{
  def main(args: Array[String]): Unit = {
    val config = new SparkConf().setAppName("demo").setMaster("local[*]")
    val sc = new SparkContext(config)
    sc.setLogLevel("WARN")
    sc.makeRDD(1 to 4 ,2).foreach(x=>println(x+5))
    // 6 7 8 9
    sc.makeRDD(1 to 12,3).foreachPartition(x=>println(x.size))
    //x指的是分区  4  4  4
  }
}

猜你喜欢

转载自blog.csdn.net/bx1215/article/details/105383918
今日推荐