pandas splicing operation of the data analysis

 

pandas splicing operation

pandas splicing of two kinds:

• Cascade: pd.concat, pd.append
• Merge: pd.merge, pd.join

1. pd.concat () cascaded

pandas use pd.concat function, and np.concatenate function similar, but more than a few parameters:

objs
axis=0
keys
join='outer' / 'inner':表示的是级联的方式，outer会将所有的项进行级联（忽略匹配和不匹配），而inner只会将匹配的项级联到一起，不匹配的不级联
ignore_index=False

1.1 匹配级联

import numpy as np
import pandas as pd
from pandas import Series,DataFrame
df1 = DataFrame(data=np.random.randint(0,100,size=(3,4)),index=['A','B','C'],columns=['a','b','c','d'])
df2 = DataFrame(data=np.random.randint(0,100,size=(3,4)),index=['A','D','C'],columns=['a','b','e','d'])
display(df1,df2)

 

 

pd.concat ((df1, df1, df1), Axis =. 1, the Join = ' Inner ' ) # the splicing line df1

 

2 cascade mismatch

Does not match the index finger is inconsistent dimension cascade. E.g. inconsistent longitudinal cascade column index, a transverse row index inconsistent cascaded

There are two kinds of connections:

• External connection: complement NaN (default mode)

• En: Only connect matching items

pd.concat ((DF1, DF2), Axis = 0, the Join = ' Inner ' ) # column spliced within the connector

 

Use pd.merge () merge

Concat merge with the difference that, merge need to be merged based on a common column

Use pd.merge () merge, both automatically according to the row same column name as a key to merge.

Note that the order of elements in each column are not consistent with the requirements

parameter:

• how: out and set to take inner intersected

• on: When a plurality of the same column may be used to specify the row on merge, on a list of values

1) one of the merging

Creating a Data

df1 = DataFrame({'employee':['Bob','Jake','Lisa'],
                'group':['Accounting','Engineering','Engineering'],
                })

 

 

df2 = DataFrame({'employee':['Lisa','Bob','Jake'],
                'hire_date':[2004,2008,2012],
                })

 

 

pd.merge (df1, df2) # default combiner combined according to the same field

 

 

2) Many-merger

# 创建数据
df3 = DataFrame({
    'employee':['Lisa','Jake'],
    'group':['Accounting','Engineering'],
    'hire_date':[2004,2016]})

 

df4 = DataFrame({'group':['Accounting','Engineering','Engineering'],
                       'supervisor':['Carly','Guido','Steve']
                })

pd.merge(df3,df4) # 合并数据

3) 多对多合并

df1 = DataFrame({'employee':['Bob','Jake','Lisa'],
                 'group':['Accounting','Engineering','Engineering']})

df5 = DataFrame({'group':['Engineering','Engineering','HR'],
                'supervisor':['Carly','Guido','Steve']
                })

pd.merge(df1,df5,how='outer') # 合并外连接

4) key的规范化

• 当列冲突时，即有多个列名称相同时，需要使用on=来指定哪一个列作为key，配合suffixes指定冲突列名

 

df1 = DataFrame({'employee':['Jack',"Summer","Steve"],
                 'group':['Accounting','Finance','Marketing']})

df2 = DataFrame({'employee':['Jack','Bob',"Jake"],
                 'hire_date':[2003,2009,2012],
                'group':['Accounting','sell','ceo']})

pd.merge(df1,df2,on='employee') #合并 指定固定的列

• 当两张表没有可进行连接的列时，可使用left_on和right_on手动指定merge中左右两边的哪一列列作为连接的列

df1 = DataFrame({'employee':['Bobs','Linda','Bill'],
                'group':['Accounting','Product','Marketing'],
               'hire_date':[1998,2017,2018]})

df5 = DataFrame({'name':['Lisa','Bobs','Bill'],
                'hire_dates':[1998,2016,2007]})

pd.merge(df1,df5,left_on='employee',right_on='name') # 指定左右合并的列

5) 内合并与外合并:out取并集 inner取交集

df6 = DataFrame({'name':['Peter','Paul','Mary'],
               'food':['fish','beans','bread']}
               )
df7 = DataFrame({'name':['Mary','Joseph'],
                'drink':['wine','beer']})
外合并 how='outer'：补NaN

df6 = DataFrame({'name':['Peter','Paul','Mary'],
               'food':['fish','beans','bread']}
               )
df7 = DataFrame({'name':['Mary','Joseph'],
                'drink':['wine','beer']})