hive udf usage example

 In the process of use, the partition needs to be dynamically partitioned according to certain rules in the process of executing sql:

import org.apache.hadoop.hive.ql.exec.UDF;
public class SharedRouter extends UDF {
	private static SolrCloudPainRouter cloudPainRouter;

	public String evaluate(final String shardValue
, final String collection, final String runtime) {
		return getRouter(collection, runtime).getShardIndex(shardValue);
	}

	private SolrCloudPainRouter getRouter(String collection, String runtime) {

		// Build cloudPainRouter
		return cloudPainRouter;

	}
}

 

 

Create a temporary function:

create temporary function tisshare as 'com.dfire.tis.dump.hive.SharedRouter';

 

Invoke udf test in hive:

select tisshare('hello','search4totalpay','daily');

Function description, this function has a total of three parameters, the description is as follows:

• Parameter 1: a column attribute
• Parameter 2: An index name in the corresponding partition tis
• Parameter 3: Operating environment, daily use 'daily', online production environment 'online'

When this udf is executed, it needs to depend on some jar packages, so when starting hive, you need to add the jar to the classpath:

hive start

hive --auxpath /root/standalone/commons-io-2.3.jar,/root/standalone/commons-logging-1.2.jar,/root/standalone/fastjson-1.1.41.jar,/root/standalone/global-diamond-config-1.0.3-SNAPSHOT.jar,/root/standalone/hive-exec-1.1.0.jar,/root/standalone/noggit-0.6.jar,/root/standalone/slf4j-api-1.7.5.jar,/root/standalone/solr-solrj-5.3.0.jar,/root/standalone/spring-2.5.6.jar,/root/standalone/terminator-manage-pojo-1.0.3-SNAPSHOT.jar,/root/standalone/tis-client-standalone-1.0.3-SNAPSHOT.jar,/root/standalone/tis-solrj-client-1.0.3-SNAPSHOT.jar,/root/standalone/zookeeper-3.4.5.jar

Use example in hive script

ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/commons-io-2.3.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/commons-logging-1.2.jar;
ADD jar hdfs: //cluster-cdh/zeus/hdfs-upload-dir/jar/fastjson-1.1.41.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/global-diamond-config.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/hive-exec-1.1.0.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/noggit-0.6.jar;
ADD jar hdfs: //cluster-cdh/zeus/hdfs-upload-dir/jar/slf4j-api-1.7.5.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/solr-solrj-5.3.0.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/spring-2.5.6.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/terminator-manage-pojo-1.0.3-SNAPSHOT.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/tis-client-standalone-1.0.3-SNAPSHOT.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/tis-solrj-client-1.0.3-SNAPSHOT.jar;
ADD jar hdfs://cluster-cdh/zeus/hdfs-upload-dir/jar/zookeeper-3.4.5.jar;
CREATE TEMPORARY FUNCTION tisshare as 'com.dfire.tis.dump.hive.SharedRouter';


select tisshare('123','search4totalpay','online')