Hive built-in functions and custom functions and basic example

Hive function

1. Built-in functions

1) Check the system comes with a function - hive> show functions;

2) the use of the display function comes - hive> desc function upper;

3) the usage of functions shown in detail comes - hive> desc function extended upper;

2.Hive custom function

1) Hive comes with some functions, such as: max / min the like, but a limited number of their own custom UDF can be conveniently extended.

2) When the Hive provides built-in functions can not meet the processing needs of your business, then you can consider using user-defined functions

Custom functions into three categories:

（1）UDF（User-Defined-Function）

Entered a

(2) outward (User-Defined Aggregation Function)

Aggregate function, into a plurality

Similar to: count / max / min

（3）UDTF（User-Defined Table-Generating Functions）

A multiple-out

The lateral view explore ()

Programming steps:

(1) inherited org.apache.hadoop.hive.ql.UDF

(2) the need to achieve evaluate function; evaluate support function overloading;

Precautions:

(1) UDF must have a return type, you can return null, but can not be void return type;

(2) UDF commonly Text / LongWritable other types, type java not recommended;

3. Examples of simple UDF

The first step in the configuration file pom.xml

cloudera https://repository.cloudera.com/artifactory/cloudera-repos/ org.apache.hadoop hadoop-common 2.6.0-cdh5.14.0 org.apache.hive hive-exec 1.1.0-cdh5.14.0 org.apache.maven.plugins maven-compiler-plugin 3.0 1.8 1.8 UTF-8 org.apache.maven.plugins maven-shade-plugin 2.2 package shade *:* META-INF/*.SF META-INF/*.DSA META-INF/*/RSA

The second step **: **** develop java class inherits UDF, and reload evaluate ** method

class public ItcastUDF the extends the UDF {
public the Text the evaluate ( Final the Text S) {
IF ( null == S) {
return null ;
}
// returns uppercase return new new the Text (s.toString () the toUpperCase ().);

​ }
}

The third step: the written project package upload to the desktop

Step 4: Add the jar package written to the linux Hive

Renaming our jar package name

cd /export/servers/hive-1.1.0-cdh5.14.0/lib

mv original-day_06_hive_udf-1.0-SNAPSHOT.jar udf.jar

hive add our client jar package

add jar /export/servers/hive-1.1.0-cdh5.14.0/lib/udf.jar;

The fifth step **: Set the function associated with our custom function **

Creating a temporary function

create temporary function tolowercase as ‘cn.itcast.udf.ItcastUDF’;

Delete the temporary function

drop temporary function tolowercase

Create a permanent function

create function tolowercase1 as ‘cn.itcast.udf.ItcastUDF’;

Delete function permanently

drop function tolowercase1;

Step Six **: Use a custom function **

select tolowercase(‘abc’);