Several hvie commonly used functions

cast

cast(value as type)
# 将int类型的id转化为了String类型
cast(id as string)

Cast converts the displayed value of a certain column to a certain type. Generally, it is necessary to force the conversion of a large type to a small type because there is a risk of data overflow.

concat

concat(string s1, string s2, string s3,...)
# 输出1-2
concat(1,"-",2)

Concat is the same as MySQL's concat function, which is the connection string, which is equivalent to Java +

concat_ws

concat_ws(seperator, string s1, string s2,...)

Concat_ws is the same as MySQL's concat_ws function, using a specified separator to connect multiple strings

collect_set

collect_set needs to be used in conjunction with group by

collect_set regards the grouped data as possible, and the grouped values ​​are spliced ​​into a set

For example, there is table idname:

id name
1  a
2  a
1  b

sql:

select id,collect_set(name) as cname from table_name group by id;

result:

id  cname
1   [a,b]
2   [a]

Collect_set and concat_ws are used together to realize the MySQL group_concat function:

select id,group_concat(name) as cname from idname group by id;

Equivalent to hive:

select id,concat_ws(",",collect_set(name)) as cname from idname group by id;

collect_list

collect_list and collect_set are basically the same, but you can guess one difference from the name, that is:
collect_set will de- duplicate the value after grouping , while collect_list will not.