hive explode & lateral view

1.explode


hive wiki对于expolde的解释如下:

explode() takes in an array (or a map) as an input and outputs the elements of the array (map) as separate rows. UDTFs can be used in the SELECT expression list and as a part of LATERAL VIEW.

As an example of using explode() in the SELECT expression list, consider a table named myTable that has a single column (myCol) and two rows:

Then running the query:

SELECT explode(myCol) AS myNewCol FROM myTable;


will produce:

这里写图片描述

2、Lateral View

Lateral View是Hive中提供给UDTF的conjunction,它可以解决UDTF不能添加额外的select列的问题。

Why we need Lateral View?
当我们想对hive表中某一列进行split之后,想对其转换成1 to N的模式,即一行转多列。
hive不允许我们在UDTF函数之外,再添加其它select语句。
如下,我们想将登录某个游戏的用户id放在一个字段user_ids里,对每一行数据用UDTF后输出多行。

select game_id, explode(split(user_ids,'\\[\\[\\[')) as user_id   from login_game_log  where dt='2014-05-15' 
FAILED: Error in semantic analysis: UDTF's are not supported outside the SELECT clause, nor nested in expressions。

具体的操作 参考:https://blog.csdn.net/oopsoom/article/details/26001307
参考:

https://blog.csdn.net/bitcarmanlee/article/details/51926530

https://blog.csdn.net/oopsoom/article/details/26001307

发布了42 篇原创文章 · 获赞 6 · 访问量 2万+

猜你喜欢

转载自blog.csdn.net/zkyxgs518/article/details/103977381