Scala的集合算子

集合算子

Scala集合提供了丰富的计算算⼦，⽤于实现集合/数组的计算，这些计算⼦⼀般针对于List、Array、Set、Map、Range、Vector、Iterator等都可以适⽤。

排序

① sorted

scala> var a = Array(1,3,4,2,5)
a: Array[Int] = Array(1, 3, 4, 2, 5)
scala> a.sorted
//指定Int的泛型（根据数组的类型而变动），内部提供基本数据类型的隐式值 Ordering[Int]
def sorted[B >: Int](implicit ord: scala.math.Ordering[B]): Array[Int]
Array[Int] = Array(1, 2, 3, 4, 5)

因为系统已经提供了相应的隐式值 Ordering[String] ,所以⽤户在使⽤的时候⼀般⽆需提供，如果⽤户需要⾃定义排序规则，⽤户可以⾃⼰提供参数，不适⽤默认排序规则。

//自定义隐式值
val order = new Ordering[Int]{
    override def compare(x: Int, y: Int): Int = {
        x.compareTo(y) * -1
    }
}
a.sorted(order)

case class User(id:Int,name:String,salary:Double)
var users = Array(User(1,"aaa",1000),User(2,"bbb",3000),User(1,"ccc",2000))
scala> users.sorted
//系统报错，没有找到对应User的隐式值用来排序
<console>:13: error: value sorted is not a member of (User, User, User)
implicit val order = new Ordering[User]{
    override def compare(x: User, y: User): Int = {
        x.salary.compareTo(y.salary) * -1
    }
}
users.sorted

② sortby 基于单⼀属性排序

def sortBy[B](f: Int => B)(implicit ord: scala.math.Ordering[B]): Array[Int]

scala> users.sortBy(u=>u.salary)
res13: Array[User] = Array(User(3,wangwu,800.0), User(1,张三,1000.0),User(2,lisi,1500.0))

③ sortWith

def sortWith(lt: ((String, Int), (String, Int)) => Boolean): List[(String, Int)]

var tuples = Array(("a",2),("c",3),("b",1),("a",1),("d",4))
tuples.sortWith((t1,t2)=> {
    if(t1._1.equals(t2._1)){
        (t1._2.compareTo(t2._2) * -1) > 0
    }else{
        t1._1.compareTo(t2._1) > 0
    }
})
res19: List[(String, Int)] = List((d,4), (c,3), (b,1), (a,1), (a,2))

flatten

⽤于展开集合中的元素，主要作⽤于降维。

def flatten[B](implicit asTraversable: String =>scala.collection.GenTraversableOnce[B]): List[B]

scala> var list=List(List("a","b","c"),List("d","e"))
list: List[List[String]] = List(List(a, b, c), List(d, e))
scala> list.flatten
res20: List[String] = List(a, b, c, d, e)

scala> var lines=List("hello word","ni hao")
lines: List[String] = List(hello word, ni hao)
scala> lines.flatten
res26: List[Char] = List(h, e, l, l, o, , w, o, r, d, n, i, , h, a, o)
scala> lines.flatten(line => line.split("\\s+"))
res28: List[String] = List(hello, word, ni, hao)
scala> lines.flatten(line => line.split(" "))
res29: List[String] = List(hello, word, ni, hao)
scala> lines.flatten(_.split(" "))			    //简洁写法，参数只有一个时使用
res37: List[String] = List(hello, word, ni, hao)

默认降维，但集合元素是String时会将集合中元素当成char数组，一个个拆分

Map

该算⼦可以操作集合的每⼀个元素，并且对集合中的每⼀个元素做映射（转换）

def map[B](f: Int => B): scala.collection.TraversableOnce[B]

scala> var list=List(1,2,4,5)
list: List[Int] = List(1, 2, 4, 5)
scala> list.map(item => item *2 )
res35: List[Int] = List(2, 4, 8, 10)
scala> list.map(_ * 2)				
res36: List[Int] = List(2, 4, 8, 10)
scala> var lines=List("Hello World","good good study")
lines: List[String] = List(Hello World, good good study)
scala> lines.flatten(_.split("\\s+")).map(w=>(w.toLowerCase,1))
res47: List[(String, Int)] = List((hello,1), (world,1), (good,1), (good,1), (study,1))
scala> lines.flatten(_.split("\\s+")).map(w=>(w.toLowerCase,1))
res47: List[(String, Int)] = List((hello,1), (world,1), (good,1), (good,1), (study,1))

flatMap

对集合元素先进⾏转换，然后执⾏flatten展开降维。

def flatMap[B](f: String => scala.collection.GenTraversableOnce[B]): scala.collection.TraversableOnce[B]

scala> var lines=List("Hello World","good good study")
lines: List[String] = List(Hello World, good good study)
scala> lines.map(line=> line.split(" ")).flatten
res55: List[String] = List(Hello, World, good, good, study)
scala> lines.flatMap(line=> line.split("\\s+"))         // lines.flattern(_.split(" "))
res56: List[String] = List(Hello, World, good, good, study)

filter/filterNot

过滤掉集合中不满⾜条件的 元素

def filter(p: ((Int, String, Double)) => Boolean): List[(Int, String, Double)]

scala> var list=List((1,"zhangsan",1000.0),(2,"lisi",800.0))
list: List[(Int, String, Double)] = List((1,zhangsan,1000.0), (2,lisi,800.0))
scala> list.filter(_._3 >= 1000)			//元组数据取值从1开始  _看成 t=>t
res59: List[(Int, String, Double)] = List((1,zhangsan,1000.0))
scala> list.filterNot(_._3 >= 1000)
res60: List[(Int, String, Double)] = List((2,lisi,800.0))

groupBy

通常⽤户统计分析,将List或者Array转换为⼀个Map

def groupBy[K](f: String => K): scala.collection.immutable.Map[K,List[String]]

scala> var list=List("a","b","a","c")
scala> list.groupBy(w=>w)
res61: scala.collection.immutable.Map[String,List[String]] = Map(b -> List(b), a ->
List(a, a), c -> List(c))
scala> list.groupBy(w=>w).map(t=>(t._1,t._2.size))
res63: scala.collection.immutable.Map[String,Int] = Map(b -> 1, a -> 2, c -> 1)

groupby() —括号里面选择拿什么分组

示例练习：计算各部门的总工资

def groupBy[K](f: Employee => K): scala.collection.immutable.Map[K,List[Employee]]

scala> var emps=List("1,001,zhangsan,1000.0","2,002,lisi,1000.0","3,001,王五,800.0")
scala> case class Employee(id:Int,deptNo:String,name:String,salary:Double)
defined class Employee
scala>
emps.map(_.split(",")).map(ts=>Employee(ts(0).toInt,ts(1),ts(2),ts(3).toDouble)).group By(emp=>emp.deptNo).map(t=>(t._1,t._2.map(e=>e.salary).sum))
res69: scala.collection.immutable.Map[String,Double] = Map(002 -> 1000.0, 001 ->
1800.0)
//根据工资进行降序排序，可以先使用sortBy()升序排，再使用reverse将排序后的数据倒置
scala>
emps.map(_.split(",")).map(ts=>Employee(ts(0).toInt,ts(1),ts(2),ts(3).toDouble)).group
By(emp=>emp.deptNo).map(t=>(t._1,(for(e<- t._2) yield
e.salary).sum)).toList.sortBy(t=>t._2).reverse
//emps.map(_.split(",")).map(i=>(i(1),i(3).toDouble)).groupBy(i=>i._1).map(i=>(i._1,i._2.map(t=>t._2).sum)).toList.sortBy(i=>i._2).reverse
res1: List[(String, Double)] = List((001,1800.0), (002,1000.0))

max|min

def max[B >: Int](implicit cmp: Ordering[B]): Int
def min[B >: Int](implicit cmp: Ordering[B]): Int

scala> list.max
res5: Int = 5
scala> list.min
res6: Int = 1
scala> list.sorted
res7: List[Int] = List(1, 2, 3, 4, 5)
scala> list.sorted.head
res8: Int = 1
scala> list.sorted.last
res9: Int = 5

maxBy|minBy

计算含有最⼤值或者最⼩值的记录,按照特定条件求最⼤或最⼩

def maxBy[B](f: ((Int, String, Int)) => B)(implicit cmp: Ordering[B]): (Int, String,Int)
def minBy[B](f: ((Int, String, Int)) => B)(implicit cmp: Ordering[B]): (Int, String,Int)

scala> var list=List((1,"zhangsan",28),(2,"lisi",20),(3,"wangwu",18))
list: List[(Int, String, Int)] = List((1,zhangsan,28), (2,lisi,20), (3,wangwu,18))
scala> list.maxBy(t=>t._3)				//参数是寻找最大最小值的依据
res12: (Int, String, Int) = (1,zhangsan,28)
scala> list.maxBy(t=>t._1)
res13: (Int, String, Int) = (3,wangwu,18)

scala> var emps=List("1,001,zhangsan,1000.0","2,002,lisi,1000.0","3,001,王五,800.0")
emps: List[String] = List(1,001,zhangsan,1000.0, 2,002,lisi,1000.0, 3,001,王五,800.0)
scala> emps.map(_.split(",")).map(w=>(w(1),w(3))).groupBy(_._1).map(t=>t._2.maxBy(i=>i._2))
res24: scala.collection.immutable.Map[String,String] = Map(002 -> 1000.0, 001 ->800.0)
scala> emps.map(line=>line.split(",")).map(ts=>(ts(1),ts(3).toDouble)).groupBy(_._1).map(_._2).map(_.maxBy(_._2))
//emps.map(line=>line.split(",")).map(ts=>(ts(1),ts(3).toDouble)).groupBy(i=>i._1).map(t=>(t._1,t._2.map(i=>i._2))).toList.map(i=>(i._1,i._2.max))
res4: scala.collection.immutable.Iterable[(String, Double)] = List((002,1000.0),(001,1000.0))

reduce | reduceLeft | reduceRight

 def reduce[A1 >: Int](op: (A1, A1) => A1): A1

scala> var list=List(1,5,3,4,2)
list: List[Int] = List(1, 5, 3, 4, 2)
scala> list.reduce((v1,v2)=>v1+v2)
res7: Int = 15
scala> list.reduceLeft((v1,v2)=>v1+v2)
res8: Int = 15
scala> list.reduceRight((v1,v2)=>v1+v2)
res9: Int = 15
scala> list.reduceRight(_+_)
res17: Int = 15

如果集合为空（没有数据），系统报错

scala> var list=List[Int]()
list: List[Int] = List()
scala> list.reduce((v1,v2)=>v1+v2)
java.lang.UnsupportedOperationException: empty.reduceLeft at
scala.collection.LinearSeqOptimized$class.reduceLeft(LinearSeqOptimized.scala:137)
 at scala.collection.immutable.List.reduceLeft(List.scala:84)
 at scala.collection.TraversableOnce$class.reduce(TraversableOnce.scala:208)
 at scala.collection.AbstractTraversable.reduce(Traversable.scala:104)
 ... 32 elided

fold | foldLeft | foldRight

def fold[A1 >: Int](z: A1)(op: (A1, A1) => A1): A1

scala> var list=List(1,5,3,4,2)
list: List[Int] = List(1, 5, 3, 4, 2)
scala> list.fold(0)((z,v)=> z+v)
res12: Int = 15
scala> var list=List[Int]()
list: List[Int] = List()
scala> list.fold(0)((z,v)=> z+v)
res13: Int = 0
scala> list.fold(0)(_+_)
res19: Int = 15

aggregate

def aggregate[B](z: => B)(seqop: (B, Int) => B,combop: (B, B) => B): B

scala> var list=List(1,5,3,4,2)
list: List[Int] = List(1, 5, 3, 4, 2)
scala> list.aggregate(0)((z,v)=>z+v,(b1,b2)=>b1+b2)
res29: Int = 15
scala> list.aggregate(0)(_+_,_+_)
res33: Int = 15
scala> var list=List[Int]()
list: List[Int] = List()
scala> list.aggregate(0)((z,v)=>z+v,(b1,b2)=>b1+b2)
res31: Int = 0

我们reduce和fold计算要求计算结果类型必须和集合元素类型⼀致，⼀般⽤于求和性质的计算。由于
aggregate计算对类型⽆要求，因此可以使⽤aggregate完成更复杂的计算逻辑，例如：计算均值

scala> emps.map(t=>t.split(",")).map(t=>(t(1),t(3).toDouble)).groupBy(t=>t._1).map(t=>(t._1,t._2.map(i=>i._2))).map(i=>(i._1,i._2.aggregate(0,0.0)((z,v)=>(z._1+1,z._2+v),(b1,b2)=>(b1._1+b2._1,b1._2+b2._2)))).map(i=>(i._1,i._2._2/i._2._1))
res34: scala.collection.immutable.Map[String,Double] = Map(002 -> 1000.0, 001 -> 900.0)

scala> var list=List(1,5,3,4,2)
list: List[Int] = List(1, 5, 3, 4, 2)
scala> list.aggregate((0,0.0))((z,v)=>(z._1+1,z._2+v),(b1,b2)=>(b1._1+b2._1,b1._2+b2._2))
res34: (Int, Double) = (5,15.0)

按照部⻔计算员⼯平均薪资

scala> var emps=List("1,001,zhangsan,1000.0","2,002,lisi,1000.0","3,001,王五,800.0")
emps: List[String] = List(1,001,zhangsan,1000.0, 2,002,lisi,1000.0, 3,001,王五,800.0)
scala> emps.map(line=>line.split(",")).map(ts=>(ts(1),ts(3).toDouble)).groupBy(_._1).map(t=>(t._1,t._2.map(_._2))).map(t=>(t._1,t._2.aggregate((0,0.0))((z,v)=>(z._1+1,z._2+v),(b1,b2)=>(b1._1+b2._1,b1._2+b2._2)))).map(t=>(t._1,t._2._2/t._2._1))
res63: scala.collection.immutable.Map[String,Double] = Map(002 -> 1000.0, 001 ->900.0)
scala> emps.map(line=>line.split(",")).map(ts=>(ts(1),ts(3).toDouble)).groupBy(_._1).map(t=>(t._1,t._2.map(_._2))).map(t=>(t._1,t._2.aggregate((0,0.0))((z,v)=>(z._1+1,z._2+v),(b1,b2)=>
(b1._1+b2._1,b1._2+b2._2)))).map(t=>(t._1,t._2._2/t._2._1))
res63: scala.collection.immutable.Map[String,Double] = Map(002 -> 1000.0, 001 ->900.0)

group

可以对⼀维度数据进⾏升维度

def grouped(size: Int): Iterator[List[Int]]

scala> var list=List(1,5,3,4,2)
list: List[Int] = List(1, 5, 3, 4, 2)
scala> list.grouped(2)
res73: Iterator[List[Int]] = non-empty iterator
scala> list.grouped(2).toList
res74: List[List[Int]] = List(List(1, 5), List(3, 4), List(2))

zip

将两个⼀维的集合合并⼀个⼀维度的集合

scala> var list1=List(1,5,3,4,2)
list1: List[Int] = List(1, 5, 3, 4, 2)
scala> var list2=List("a","b","c")
list2: List[String] = List(a, b, c)
scala> list2.zip(list1)
res75: List[(String, Int)] = List((a,1), (b,5), (c,3))
scala> list1.zip(list2)
res76: List[(Int, String)] = List((1,a), (5,b), (3,c))

unizp

将⼀个元组分解成多个⼀维度集合

scala> var v=List(("a",1),("b",2),("c",3))
v: List[(String, Int)] = List((a,1), (b,2), (c,3))
scala> v.unzip
res90: (List[String], List[Int]) = (List(a, b, c),List(1, 2, 3))

diff | intersect | union

计算差集合、交集、并集

scala> var v=List(1,2,3)
v: List[Int] = List(1, 2, 3)
scala> v.diff(List(2,3,5))
res54: List[Int] = List(1)
scala> var v=List(1,2,3,5)
v: List[Int] = List(1, 2, 3, 5)
scala> v.intersect(List(2,4,6))
res55: List[Int] = List(2)
scala> var v=List(1,2,3,5)
v: List[Int] = List(1, 2, 3, 5)
scala> v.union(List(2,4,6))
res56: List[Int] = List(1, 2, 3, 5, 2, 4, 6)

Sliding

滑动产⽣新的数组元素

scala> val list=List(1,2,3,4,5,6)
list: List[Int] = List(1, 2, 3, 4, 5, 6)
scala> list.sliding(3,3)
res0: Iterator[List[Int]] = non-empty iterator
scala> list.sliding(3,3).toList
res1: List[List[Int]] = List(List(1, 2, 3), List(4, 5, 6))
scala> list.sliding(3,1).toList
res2: List[List[Int]] = List(List(1, 2, 3), List(2, 3, 4), List(3, 4, 5), List(4, 5,6))

slice

截取数组⼦集

scala> val list=List(1,2,3,4,5,6)
list: List[Int] = List(1, 2, 3, 4, 5, 6)
scala> list.slice(0,3)
res3: List[Int] = List(1, 2, 3)
scala> list.slice(3,5)
res5: List[Int] = List(4, 5)

案例剖析

① 有如下数组

var arrs=Array("this is a demo","good good study","day day up")

请统计字符出现的次数，并按照次数降序排列

arrs.map(t=>t.split(" ")).flatten.groupBy(t=>t).map(t=>(t._1,t._2.size)).toList.sortBy(i=>i._2).reverse
res11: List[(String, Int)] = List((day,2), (good,2), (study,1), (a,1), (up,1), (is,1),
(demo,1), (this,1))

② 读取⼀个⽂本⽂件，计算字符出现的个数

var source=Source.fromFile("/Users/admin/IdeaProjects/20200203/scalalang/src/main/resources/t_word")
var array=ListBuffer[String]()
val reader = source.bufferedReader()
var line = reader.readLine()
while(line!=null){
    array+=line
    line = reader.readLine()
}
array.flatMap(_.split(" "))
.map((_,1))
.groupBy(_._1)
.map(x=> (x._1,x._2.size))
.toList
.sortBy(_._2)
.reverse
.foreach(println)
reader.close()