Scala functions
1. Function declaration
Scala functions are defined through the def keyword, and def can have modifiers in front of it, and its access rights can be controlled through private and protected.
Note: If there is no public, it is public by default if it is not written. In addition, it can also keep up with keyword modifications such as override and final.
1. Format
[private/protected] def function name (parameter list): return value declaration = {function body}
2. the return value of the function
1) The return keyword in the function body can often be omitted. Once omitted, the function will return the value of the expression in the last line of the entire function body, which also requires that the last line of the entire function body must be an expression of the correct type of value .
2) Most of the time, scala can automatically infer the type of the return value through the = symbol, so usually the return value type declaration can be omitted.
But note: if ambiguity is caused by omitting the return value type, be sure to write the return value declaration.
3) If the function body has only one line, the curly brackets surrounding the function body can be omitted.
4) If the return value type is UNIT, another way to write it is to remove the return value type and the equal sign, and write the method body in curly braces. At this time, no matter what is returned in the method, the return value is UNIT. Equivalent to void in Java.
Example:
//方法的返回值为空
private def f1(){}
protected def f2():String={"hello"}
def f3()={"hello"}
//如果函数体只一行内容，可以省了花括号
def f4()="hello"
//定义方法参数类型，返回值类型，及返回值
def f5(a:Int,b:Int)={a+b}3. default parameters
Default values can be set for the parameters of the function.
Example:
//默认参数的使用
def f8(a:String,b:String="[",c:String="]")={
b+a+c
}4. Placeholder
Placeholder: A placeholder refers to the underscore _ in scala, which can be used as one or more parameters.
Prerequisites for using _ placeholders: each parameter appears only once in the function.
When using underscore, the type is not declared if the type can be inferred automatically. If the type cannot be automatically deduced, you can display the declared type yourself after the underscore.
Example:
//要求通过reduceLeft函数计算阶乘结果
val a2=Array(1,2,3,4)
a2.reduceLeft{(a:Int,b:Int)=>{a*b}}
a2.reduceLeft{_*_}2. Types of functions
Functions in Scala are divided into member functions, local functions (functions embedded in functions), function values (anonymous functions), and higher-order functions.
1. member function
Member function: The function is used inside the class, as a member of the class, called the member function of the class.
Example:
class Person {
//eat方法是Person的成员方法
def eat() {
println("eat")
}
}2. local function
Local functions: Functions embedded in functions are called local functions, and such functions cannot be accessed by the outside world.
Example:
class Person {
//eat方法是Person的成员方法
def eat() {
println("eat")
//本地函数：内嵌在函数内的函数。对象不能直接调用本地函数。
def cook() {
println("cook")
}
}
}3. anonymous function
Function value (anonymous function):
1. An anonymous function does not have a function name.
2. The role of anonymous functions is to use with higher-order functions, and anonymous functions can be passed as parameters of functions.
Example:
(a:Int,b:Int)=>{a+b}
(a:Int,b:Int)=>a+b
val a1=Array(1,2,3,4)
//a=1 b=2 a+b=3
//a=3 b=3 a+b=6
//a=6 b=4 a+b=10
a1.reduceLeft{(a:Int,b:Int)=>a+b}
a1.reduceLeft{(a,b)=>a+b}
a1.foreach{(x:Int)=>{println(x)}}
a1.foreach{x=>println(x)}4. Higher order functions
Higher-order functions: Functions can be passed and called as arguments to methods.
Example:
//定义一个高阶函数，可以将函数当作参数传递
def f2(a:Int,b:Int,f:(Int,Int)=>Int)={
f(a,b)
}
f2(2,3,(a:Int,b:Int)=>{a+b})
f2(2,3,(a,b)=>a+b)
f2(2,3,(a,b)=>a*b)
f2(2,3,_*_)
//定义一个高阶函数，要求：传入一个String类型的参数，以及一个处理String类型的匿名函数
//此高阶函数的返回值就是匿名函数的返回值
def f3(a:String,f:(String)=>Array[String])={
f(a)
}
//注意：匿名函数一定要和指定返回值类型匹配
f3("hello,world",(a:String)=>{a.split(",")})
f3("hello,world",a=>a.split(","))
f3("hello,world",a=>a split ",")
f3("hello,world",_.split(","))
//要求通过reduceLeft函数计算阶乘结果
val a2=Array(1,2,3,4)
a2.reduceLeft{(a:Int,b:Int)=>{a*b}}3. Recursion
To implement a recursive method, there are two elements that can be implemented quickly.
Element 1: Find out the conditions under which the recursion ends.
Element 2: Find out the mapping relationship of functions.
In scala, if the last line of the function body is guaranteed to be called recursively during recursion, such recursion is called tail recursion. Scala will optimize for tail recursion, so it is recommended to write tail recursion when writing recursion.
example:
Fibonacci sequence: 1 1 2 3 5 8 13 ?
Element 1: Find the condition for the end of the recursion: f(n)=f(n-1)+f(n-2)
Element 2: Find the mapping relationship of the function: f(0)=1; f(1)=1
So the code can be:
def f1(n:Int):Int={
if(n==0)return 1
if(n==1)return 1
else f1(n-1)+f1(n-2)
}Example:
//从1开始做加法，只加偶数，当加和累计超过50时，结束递归
//示意：2+4+6……
def f1(num: Int, sum: Int): Int = {
if (sum > 50) return sum;
if (num % 2 == 0) { f1(num + 1, sum + num) }
else { f1(num + 1, sum) }
}
//2 3 4 9 8 27 16 ?
//f(n)= f(n-2)*2
//f(n)=f(n-2)*3
//f(0)=2;f(1)=3
def f2(n: Int): Int = {
if (n == 0) return 2
if (n == 1) return 3
if (n % 2 == 0) return f2(n - 2) * 2
else f2(n - 2) * 3
}
//2 3 4 9 16 81 ?
// n的取值:f(0) f(1) f(2) f(3) f(4) f(5)
//当n为偶数时,f(n)=f(n-2)*f(n-2)
//当n为奇数是,f(n)=f(n-2)*f(n-2)
//fn=f(n-2)*f(n-2)
def f3(n: Int): Int = {
if (n == 0) return 2
if (n == 1) return 3
else f3(n - 2) * f3(n - 2)
}
//求 1~n的数字之和
//1 3 6 10 15
//f(0) f(1) f(2) f(3) f(4)
//f(n)=f(n-1)+n+1
def f5(n: Int): Int = {
if (n == 0) return 1
else f5(n - 1) + n + 1
}
//给定一个初始值n，并设定sum的初始值为0，当求和sum>12时结束递归
//0 1 3 6 10 15
//f(0,0)——n=0,sum=0
//f(1,1)——n=1,sum=1
//f(2,3)——n=2,sum=3
//f(3,6)——n=3,sum=6
//f(n+1,sum+n)
def f6(n: Int, sum: Int): Int = {
if (sum > 12) return sum
else f6(n + 1, sum + n)
}
//给定一个scope范围，计算0~scope范围的整数之和，
//当和>12或者达到scope边界时，结束递归
def f7(n: Int, sum: Int, scope: Int): Int = {
if (sum > 12) return sum
if (n - 1 == scope) return sum
else f7(n + 1, sum + n, scope)
}4. Variable length parameters
In scala, it is possible to specify that the last parameter of a function is repeated. This allows clients to pass a list of variadic arguments to the function.
To mark a duplicate parameter, place an asterisk after the parameter's type. The type of a repeated parameter (variadic parameter) is an array of the declared parameter type.
Example:
//定义变长参数
def f1(num:Int*)={}
def f2(num:Int*)={
for(i<-num)println(i)
}
f2(1,2,3,4,5)
def f21(a:Int,str:String*)={}5. Currying
1. introduce
The role of scala's currying is to combine scala's higher-order functions, thereby allowing users to build their own control structures.
Currying (Currying) technique Christopher Strachey named after the logician Haskell Curry (though it was invented by Moses Schnfinkel and Gottlob Frege). It is the technique of transforming a function that accepts multiple parameters into a function that accepts a single parameter, and returns a new function that accepts the remaining parameters and returns the result.
case
//首先我们定义一个函数:
def f1(a:Int,b:Int):Int={a+b}
//现在我们把这个函数变一下形：
def f2(a:Int)(b:Int)={a+b}
//那么我们应用的时候，应该是这样用：f2(2)(3),最后结果都一样是5，这种方式（过程）就叫柯里化。
val r1=f2(2)(3)
//柯里化实质上会演变成这样一个函数：
//接收一个参数a，返回一个匿名函数,
//该匿名函数又接收一个参数b,函数体为a+b
def f3(a:Int)=(b:Int)=>a+b
val f4=f3(2)
//请思考 f4(3)的值是多少？答案是：5
f4(3)Example
//柯里化
def f3(a:Int,b:Int)={a+b}
def f31(a:Int)(b:Int)={a+b}
f3(2,3)
f31(2)(3)
def f4(a:Int,b:Int,c:Int)={a+b+c}
def f41(a:Int)(b:Int)(c:Int)={a+b+c}
def f42(a:Int,b:Int)(c:Int)={a+b+c}
def f43(a:Int)(b:Int,c:Int)={a+b+c}
def f5(a:Int,b:Int,f:(Int,Int)=>Int)={f(a,b)}
def f51(a:Int)(b:Int,f:(Int,Int)=>Int)={f(a,b)}
def f52(a:Int,b:Int)(f:(Int,Int)=>Int)={f(a,b)}
def f53(a:Int)(b:Int)(f:(Int,Int)=>Int)={f(a,b)}
f5(2,3,_+_)
f52(2,3)(_+_) 4. effect
Currying technology plays an important role in improving applicability, delaying execution, or fixing volatile factors. In addition, the scala language itself advocates concise coding, which makes functions with the same function more flexible and diverse when they are defined and converted. In addition, there is a lot of use of scala currying technology in Spark's source code. It is necessary to master this technology to understand the relevant source code.
Closures also play an important role in scala currying. The so-called closure is that the variable can still function in other code blocks outside the scope of the function. Such a situation is called a closure. As far as the case discussed above is concerned, if there is no closure function, then the anonymous function returned by the converted function cannot be combined with the first parameter a, and naturally there is no guarantee that the function it implements is the same as the original one. of.
6. Built-in higher-order functions
Applies to all collections.
1．partition
Split, divide a set into two sets according to a Boolean value, one set that satisfies the condition, and the other set.
Split according to the specified principle and return a binary Tuple.
val l1=List(1,2,3,4,5,6)
l1.partition{x=> x%2==0}
//> res0: (List[Int], List[Int]) = (List(2, 4, 6),List(1, 3, 5))2．map
Mapping, which converts one collection to another. The number of elements in the set does not change.
Change the type of the collection, the form or data of the elements, and return a new collection. This method does not change the number of elements in the collection, only the value and form.
val l2=List("hadoop","world","hello","hello")
l2.map(x=>(x,1))
//> res1: List[(String, Int)] = List((hadoop,1), (world,1), (hello,1), (hello,1))
val l3=List("hello word","hello hadoop")
l3.map{x=>x.split(" ")}
//> res2: List[Array[String]] = List(Array(hello, word), Array(hello, hadoop))3．flatMap
Flattening the map will take out the elements of the collection. Note that this method changes the number of elements in the collection.
General reference scenario: After reading the file, process the file and divide each line of data according to the specified delimiter.
l3.flatMap{x=>x.split(" ")}
//> res3: List[String] = List(hello, word, hello, hadoop)
//要求：操作l3将其变成(word,1)的形式
l3.flatMap{x=>x.split(" ")}.map(x=>(x,1))
//> res4: List[(String, Int)] = List((hello,1), (word,1), (hello,1), (hadoop,1))4．filter
filter.
val l4=List(1,2,3,4,5)
l4.filter(x=>x>3)
//> res5: List[Int] = List(4, 5)5．reduce
Reduction, the process of reduce: operate the result of the last operation and the next value.
The function takes two arguments => returns a value.
Equivalent to reduceLeft.
l4.reduce{_+_}
//> res6: Int = 156．groupBy
Aggregate according to the specified rules, and finally return the result to a map.
Group according to the specified principle, and return the Map type. The key of the Map is the grouping key, and the value is the corresponding List collection.
val l5=List(("bj",1),("sh",2),("bj",3),("sh",4),("sz",5))
l5.groupBy{x=>x._1}
//> res7: scala.collection.immutable.Map[String,List[(String, Int)]] = Map(bj ->List((bj,1), (bj,3)), sz -> List((sz,5)), sh -> List((sh,2), (sh,4)))
l5.groupBy{case(addr,count)=>addr}
//> res8: scala.collection.immutable.Map[String,List[(String, Int)]] = Map(bj ->List((bj,1), (bj,3)), sz -> List((sz,5)), sh -> List((sh,2), (sh,4)))7．mapValues
This method is to operate on the value of the Map type. This method is only applicable to the Map type.
val m1=Map("rose"->23,"tom"->25,"jary"->30)
m1.mapValues {x=>x+10}
//> res9: scala.collection.immutable.Map[String,Int] = Map(rose -> 33, tom -> 35, jary -> 40)8．sortBy
sort.
val l6=List((2,"aaa"),(1,"bbb"),(4,"ddd"),(3,"ccc"))
l6.sortBy{x=>x._1}
//> res10: List[(Int, String)] = List((1,bbb), (2,aaa), (3,ccc), (4,ddd))
l6.sortBy{x=>x._2}
//> res11: List[(Int, String)] = List((2,aaa), (1,bbb), (3,ccc), (4,ddd))
l6.sortBy{case(num,str)=>num}
//> res12: List[(Int, String)] = List((1,bbb), (2,aaa), (3,ccc), (4,ddd))Example: Counting Word Frequency
Count the frequency of occurrence of each word. The final result is of the form: (hello,5)(hadoop,2)…
val l7=List("hello hadoop","hello world","hello spark","hello hadoop","hello hive")
//方法一：
l7.flatMap{line=>line.split(" ")}.groupBy{word=>word}.mapValues { list => list.size }.foreach{println(_)}
//方法二：
l7.flatMap { line => line.split(" ") }.map { word => (word,1) }.groupBy(x=>x._1).mapValues{list=>list.size}.foreach{println(_)}
//方法三：
l7.flatMap { line => line.split(" ") }.map { word => (word,1) }.groupBy(x=>x._1).mapValues{list=>list.map(x=>x._2).reduce(_+_)}.foreach{println(_)}
//简化写法：
l7.flatMap{_.split(" ")}.map{(_,1)}.groupBy(_._1).mapValues{_.map(_._2).reduce(_+_)}.foreach{println(_)}
//要求统计单词频次，然后返回频次最高的前2项结果
l7.flatMap { line => line.split(" ") }.map { word => (word,1) }.groupBy(x=>x._1).mapValues{list=>list.map(x=>x._2).reduce(_+_)}.toList.sortBy(x=> -x._2).take(2)
//> res13: List[(String, Int)] = List((hello,5), (hadoop,2))