Spark Sql merge multiple lines into one line

Used functions and definitions

concat_ws(sep, [str | array(str)]+) - Returns the concatenation of the strings separated by sep.
Examples:

SELECT concat_ws(’ ', ‘Spark’, ‘SQL’);
Spark SQL

collect_set(expr) - Collects and returns a set of unique elements.

Example data

Here Insert Picture Description
We string the fourth column should be combined into one line

select 
	Asset_a
	, concat_ws(',',collect_set(NT_Login)) as ntlogin 
from (
	select 
	Asset
	,Platform
	,UserOrBatch
	,NT_Login
	from eip_rewards_usage  
	where Platform='hercules' 
	and UserOrBatch='User'
	)
group by 1
order by 1 asc

First we need to increase the limit for the data group by, and within a group of data items collect_set NT_Login together, CONCAT_WS merge, '' as a separator

sparksql Function Documentation:
https://docs.databricks.com/spark/latest/spark-sql/language-manual/functions.html#concat_ws

Released five original articles · won praise 2 · Views 171

Guess you like

Origin blog.csdn.net/DUTwangtaiyu/article/details/103661526