Spark从入门到精通 -Transformations与 Actions 算子

Transformations

map(func) 、filter(func) 、flatMap(func)、mapPartitions(func)、mapPartitionsWithIndex(func)、sample(withReplacement, fraction, seed)、union(otherDataset)、intersection(otherDataset)、distinct([numPartitions]))、groupByKey([numPartitions])、reduceByKey(func, [numPartitions])、aggregateByKey(zeroValue)(seqOp, combOp, [numPartitions])、sortByKey([ascending], [numPartitions])、join(otherDataset, [numPartitions])、cogroup(otherDataset, [numPartitions])、cartesian(otherDataset)、pipe(command, [envVars])、coalesce(numPartitions)、repartition(numPartitions)、repartitionAndSortWithinPartitions(partitioner)

Actions

reduce(func)、collect()、count()、first()、take(n)、takeSample(withReplacement, num, [seed])、takeOrdered(n, [ordering])、saveAsTextFile(path)、saveAsSequenceFile(path)
(Java and Scala)、saveAsObjectFile(path)、countByKey()、foreach(func)

猜你喜欢

转载自blog.csdn.net/m0_49447718/article/details/115461953