insert dataframe into rows for each group in another dataframe

Ambleu :

I've created MRE for clarity.

df = pd.DataFrame({
    "region": ["Canada", "Korea", "Norway", "China", "Canada", "Korea", "Norway", "China", "Canada", "Korea", "Norway", "China"],
    "type" :["A", "B", "C", "D", "A", "C", "C", "A", "B", "B", "B", "B"],
    "actual fees": [1235, 422, 333, 111, 1233, 555, 23, 3, 3,4, 1, 2],
    "total fee": [2222, 444, 67, 711, 4873, 785, 453, 7, 7,9, 11, 352]
})

df_to_insert = pd.DataFrame({
    "region":["Canada", "Korea", "Norway", "China"],
    "users" :[55, 36, 87, 250]
})

so my df would look like:

              actual fees   total fee
region  type        
Canada    A      2             2
          B      1             1
China     A      1             1
          B      1             1
          D      1             1

and df_to_insert looks like below:

    region  users
0   Canada  55
1   Korea   36
2   Norway  87
3   China   250

now what I want to do is at end of each region in column "type" insert "users" and user values under "actual fees" column and under "total fee" column its regional sum.

So my desired dataframe would look like something below:

              actual fees   total fee
region  type        
Canada    A      2             2
          B      1             1
        Users   55             3

China     A      1             1
          B      1             1
          D      1             1
        Users   250            3

I hope this was clear enough. Let me know if something is not clear.

Thanks in advance!

anky_91 :

You can melt the df_to_insert first , then concat and set_index for MultiIndex, lastly for total fee , groupby region and map back to mlt dataframe

mlt = df_to_insert.melt('region',var_name='type',value_name='actual fees')
mlt['total fee'] = mlt['region'].map(df.groupby('region')['total fee'].sum())

out = pd.concat((df,mlt),sort=False).set_index(['region','type']).sort_index(0)

print(out)
              actual fees  total fee
region type                         
Canada A             1235       2222
       A             1233       4873
       B                3          7
       users           55       7102
China  A                3          7
       B                2        352
       D              111        711
       users          250       1070
Korea  B              422        444
       B                4          9
       C              555        785
       users           36       1238
Norway B                1         11
       C              333         67
       C               23        453
       users           87        531

You can see how the melt work and helps in concating :

print(df_to_insert.melt('region',var_name='type',value_name='actual fees'))

   region   type  actual fees
0  Canada  users           55
1   Korea  users           36
2  Norway  users           87
3   China  users          250

insert dataframe into rows for each group in another dataframe

Guess you like