1.mapValues[Pair]
def mapValues[U](f: V => U): RDD[(K, U)]
说明：将RDD[(K, V)] --> RDD[(K, U)]，对Value做(f: V => U)操作
val a = sc.parallelize(List("dog", "tiger", "lion", "cat", "panther", "eagle"), 2)
val b = a.map(x => (x.length, x))
b.mapValues("x" + _ + "x").collect
res5: Array[(Int, String)] = Array((3,xdogx), (5,xtigerx), (4,xlionx), (3,xcatx), (7,xpantherx), (5,xeaglex))
2.flatMapValues[Pair]
def flatMapValues[U](f: V => TraversableOnce[U]): RDD[(K, U)]
val a = sc.parallelize(List("dog", "tiger", "lion", "cat", "panther", "eagle"), 2)
val b = a.map(x => (x.length, x))
b.flatMapValues("x" + _ + "x").collect
res6: Array[(Int, Char)] = Array((3,x), (3,d), (3,o), (3,g), (3,x), (5,x), (5,t), (5,i), (5,g), (5,e), (5,r), (5,x), (4,x), (4,l), (4,i), (4,o), (4,n), (4,x), (3,x), (3,c), (3,a), (3,t), (3,x), (7,x), (7,p), (7,a), (7,n), (7,t), (7,h), (7,e), (7,r), (7,x), (5,x), (5,e), (5,a), (5,g), (5,l), (5,e), (5,x))
3.subtractByKey[Pair]
def subtractByKey[W: ClassTag](other: RDD[(K, W)]): RDD[(K, V)]
说明：删掉RDD 中键与other RDD 中的键相同的元素
val a = sc.parallelize(List("dog", "tiger", "lion", "cat", "spider", "eagle"), 2)
val b = a.keyBy(_.length)
val c = sc.parallelize(List("ant", "falcon", "squid"), 2)
val d = c.keyBy(_.length)
b.subtractByKey(d).collect
res15: Array[(Int, String)] = Array((4,lion))
4.combineByKey[Pair]
def combineByKey[C](createCombiner: V => C, mergeValue: (C, V) => C, mergeCombiners: (C, C) => C): RDD[(K, C)]
说明：createCombiner：当分区中遇到第一次出现的键时，触发此函数
mergeValue：当分区中再次出现的键时，触发此函数
mergeCombiners：处理不同区当中相同Key的Value，执行此函数
案例：
RDD为一个分区时：
scala> var rdd1 = sc.makeRDD(Array(("A",1),("A",2),("B",1),("B",2),("C",1)))
rdd1: org.apache.spark.rdd.RDD[(String, Int)] = ParallelCollectionRDD[0] at makeRDD at <console>:24
scala> rdd1.combineByKey(x=>x+"_",(x:String,y:Int)=>x+"@"+y,(x:String,y:String)=>x+"$"+y)
res0: org.apache.spark.rdd.RDD[(String, String)] = ShuffledRDD[1] at combineByKey at <console>:27
scala> res0.collect
res1: Array[(String, String)] = Array((B,1_@2), (A,1_@2), (C,1_))
RDD为两个分区时：
scala> val rdd2 = rdd1.repartition(2)
rdd2: org.apache.spark.rdd.RDD[(String, Int)] = MapPartitionsRDD[5] at repartition at <console>:26
scala> rdd2.partitions.size
res2: Int = 2
scala> rdd2.glom.collect
res3: Array[Array[(String, Int)]] = Array(Array((A,1), (B,1), (C,1)), Array((A,2), (B,2)))
scala> rdd2.combineByKey(x=>x+"_",(x:String,y:Int)=>x+"@"+y,(x:String,y:String)=>x+"$"+y)
res4: org.apache.spark.rdd.RDD[(String, String)] = ShuffledRDD[7] at combineByKey at <console>:29
scala> res4.collect
res6: Array[(String, String)] = Array((B,1_$2_), (A,1_$2_), (C,1_))
RDD为三个分区时：
scala> val rdd3 = rdd1.partitionBy(new org.apache.spark.HashPartitioner(3))
rdd3: org.apache.spark.rdd.RDD[(String, Int)] = ShuffledRDD[8] at partitionBy at <console>:26
scala> rdd3.partitions.size
res7: Int = 3
scala> rdd3.glom.collect
res8: Array[Array[(String, Int)]] = Array(Array((B,1), (B,2)), Array((C,1)), Array((A,1), (A,2)))
scala> rdd3.combineByKey(x=>x+"_",(x:String,y:Int)=>x+"@"+y,(x:String,y:String)=>x+"$"+y)
res9: org.apache.spark.rdd.RDD[(String, String)] = MapPartitionsRDD[10] at combineByKey at <console>:29
scala> res9.collect
res10: Array[(String, String)] = Array((B,1_@2), (C,1_), (A,1_@2))
val a = sc.parallelize(List("dog","cat","gnu","salmon","rabbit","turkey","wolf","bear","bee"), 3)
val b = sc.parallelize(List(1,1,2,2,2,1,2,2,2), 3)
val c = b.zip(a)
val d = c.combineByKey(List(_), (x:List[String], y:String) => y :: x, (x:List[String], y:List[String]) => x ::: y)
d.collect
res16: Array[(Int, List[String])] = Array((1,List(cat, dog, turkey)), (2,List(gnu, rabbit, salmon, bee, bear, wolf)))
5.foldByKey[Pair]
def foldByKey(zeroValue: V)(func: (V, V) => V): RDD[(K, V)]
说明：与reduceByKey作用类似，但通过柯里化函数，首先要初始化zeroValue
val a = sc.parallelize(List("dog", "cat", "owl", "gnu", "ant"), 2)
val b = a.map(x => (x.length, x))
b.foldByKey("")(_ + _).collect
res84: Array[(Int, String)] = Array((3,dogcatowlgnuant)
val a = sc.parallelize(List("dog", "tiger", "lion", "cat", "panther", "eagle"), 2)
val b = a.map(x => (x.length, x))
b.foldByKey("")(_ + _).collect
res85: Array[(Int, String)] = Array((4,lion), (3,dogcat), (7,panther), (5,tigereagle))
6.reduceByKeyLocally：行动操作
def reduceByKeyLocally(func: (V, V) => V): Map[K, V]
scala> val a = sc.parallelize(List("dog", "cat", "owl", "gnu", "ant"), 2)
a: org.apache.spark.rdd.RDD[String] = ParallelCollectionRDD[0] at parallelize at <console>:24
scala> val b = a.map(x => (x.length, x))
b: org.apache.spark.rdd.RDD[(Int, String)] = MapPartitionsRDD[1] at map at <console>:26
scala> b.reduceByKeyLocally(_+_)
res0: scala.collection.Map[Int,String] = Map(3 -> dogcatowlgnuant)
7.join
def join[W](other: RDD[(K, W)]): RDD[(K, (V, W))]
说明：将两个RDD进行内连接
val a = sc.parallelize(List("dog", "salmon", "salmon", "rat", "elephant"), 3)
val b = a.keyBy(_.length)
val c = sc.parallelize(List("dog","cat","gnu","salmon","rabbit","turkey","wolf","bear","bee"), 3)
val d = c.keyBy(_.length)
b.join(d).collect
8.rightOuterJoin
说明：对两个RDD 进行连接操作，确保第一个RDD 的键必须存在（右外连接）
9.leftOuterJoin
说明：对两个RDD 进行连接操作，确保第二个RDD 的键必须存在（左外连接）
10.cogroup
说明：将两个RDD 中拥有相同键的数据分组到一起，全连