SaadH :
I have the following two data frames:
id websites
-- ---
0 1 [cnn.com, bbc.com]
1 2 [ebay.com, facebook.com]
________________
id websites
-- ---
0 2 [google.com, facebook.com]
1 3 [amazon.com, youtube.com]
I want to outer join them on the id column by aggregating unique websites for matched rows. The output should be as below:
id websites
-- ---
0 1 [cnn.com, bbc.com]
1 2 [ebay.com, facebook.com, google.com]
2 3 [amazon.com, youtube.com]
I have tried the following so far:
import pandas as pd
df_a = pd.DataFrame({'id':[1,2],'websites':[['cnn.com','bbc.com'],['ebay.com','facebook.com']]})
df_b = pd.DataFrame({'id':[2,3],'websites':[['google.com','facebook.com'],['amazon.com','youtube.com']]})
df_a.merge(df_b, on='id', how='outer')
which is giving me the following output:
id websites_x websites_y
-- --- ---
0 1 [cnn.com, bbc.com] NaN
1 2 [ebay.com, facebook.com] [google.com, facebook.com]
2 3 NaN [amazon.com, youtube.com]
anky_91 :
You can concat them and then groupby on id column:
df_a = pd.DataFrame({'id':[1,2],'websites':[['cnn.com','bbc.com'],
['ebay.com','facebook.com']]})
df_b = pd.DataFrame({'id':[2,3],'websites':[['google.com','facebook.com'],
['amazon.com','youtube.com']]})
Solution:
Method1:
a = df_a.explode('websites') #requires pandas version 0.25+
b = df_b.explode('websites') #requires pandas version 0.25+
out = pd.concat((a,b)).groupby('id')['websites'].apply(pd.unique).reset_index()
#or out = pd.concat((a,b)).groupby('id')['websites'].agg(set).reset_index()
print(out)
Method2:
Another solution using itertools.chain.from_iterable which doesnot need the exploded dataframes:
from itertools import chain
out = (pd.concat((df_a,df_b)).groupby('id')['websites']
.apply(lambda x : dict.fromkeys(chain.from_iterable(x)).keys()).reset_index())
print (out)
id websites
0 1 [cnn.com, bbc.com]
1 2 [ebay.com, facebook.com, google.com]
2 3 [amazon.com, youtube.com]