Hive function encyclopedia
1 Math functions
ceil round up the integer floor round down the integer abs absolute value function bin binary function hex hexadecimal function bround (DOUBLE a, INT d) banker's rounding method, retain d decimal place rand () random value round () retain d decimal2.Date function
from_unixtime Convert timestamp to date unix_timestamp Get timestamp current_date Current date to_date Convert date datediff(string enddate, string startdate) Return the end date minus the number of days from the start date Return the number of days year Get the year date_format(current_date(),'MM.dd') Press Specify the format to return the date, such as: date_format("2016-06-22", "MM-dd")=06-223. Conditional function
if (>,1,2) case (>,when 1 then 2) isnull returns boolean isnotnull returns boolean nvl (null,1) ===14.Character functions
regexp_extract regular expression parsing function regexp_replace('hello,world','o|l','e'); regular expression replacement substr, substring string interception reverse string flip function trim remove spaces split string split function instr( xx,x) returns a number without returning 05. Aggregation function
max maximum value min minimum value count number sum sum avg average6. Table generation function
explode(ARRAY) Each row corresponds to an element in the array7. Type conversion function
cast(expr as) Convert expr to type type such as: cast("1" as BIGINT) Convert string 1 to BIGINT type. If the conversion fails, NULL binary will be returned. Convert to binary8. Aggregate functions
size(Array) Gets the number of sets map_keys(Map<KV>) Returns all keys in the map array_contains(Array, value) Returns true if the array Array contains value. Otherwise return false sort_array(Array) sorting function
Data cleaning
Group reset
kafka-consumer-groups.sh --bootstrap-server 192.168.64.128:9092 --group cm --reset-offsets --all-topics --to-earliest
Browser to see yarn
http://192.168.64.128:8088/cluster#/
row to column
max(case when then else end) group by
Column to row
max( as ) group by union all
Regular expression collection