版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/u013317445/article/details/85268877
文章目录
groupby功能:分组
groupby + agg(聚集函数们): 分组后,对各组应用一些函数,如’sum’,‘mean’,‘max’,‘min’…
groupby默认纵方向上分组,axis=0
DataFrame
import pandas as pd
import numpy as np
df = pd.DataFrame({'key1':['a', 'a', 'b', 'b', 'a'],
'key2':['one', 'two', 'one', 'two', 'one'],
'data1':np.random.randn(5),
'data2':np.random.randn(5)})
print(df)
data1 data2 key1 key2
0 -0.410122 0.247895 a one
1 -0.627470 -0.989268 a two
2 0.179488 -0.054570 b one
3 -0.299878 -1.640494 b two
4 -0.297191 0.954447 a one
分组,并对分组进行迭代
list(df.groupby(['key1']))#list后得到:[(group1),(group2),......]
[('a', data1 data2 key1 key2
0 -0.410122 0.247895 a one
1 -0.627470 -0.989268 a two
4 -0.297191 0.954447 a one), ('b', data1 data2 key1 key2
2 0.179488 -0.054570 b one
3 -0.299878 -1.640494 b two)]
list后得到:[(group1),(group2),…]
每个数据片(group)格式: (name,group)元组
1. 按key1(一个列)分组,其实是按key1的值
groupby对象支持迭代,产生一组二元元组:(分组名,数据块),(分组名,数据块)…
for name,group in df.groupby(['key1']):
print(name)
print(group)
a
data1 data2 key1 key2
0 -0.410122 0.247895 a one
1 -0.627470 -0.989268 a two
4 -0.297191 0.954447 a one
b
data1 data2 key1 key2
2 0.179488 -0.054570 b one
3 -0.299878 -1.640494 b two
2. 按[key1, key2](多个列)分组
对于多重键,产生的一组二元元组:((k1,k2),数据块),((k1,k2),数据块)…
第一个元素是由键值组成的元组
for name,group in df.groupby(['key1','key2']):
print(name) #name=(k1,k2)
print(group)
('a', 'one')
data1 data2 key1 key2
0 -0.410122 0.247895 a one
4 -0.297191 0.954447 a one
('a', 'two')
data1 data2 key1 key2
1 -0.62747 -0.989268 a two
('b', 'one')
data1 data2 key1 key2
2 0.179488 -0.05457 b one
('b', 'two')
data1 data2 key1 key2
3 -0.299878 -1.640494 b two
3. 按函数分组
4. 按字典分组
5. 按索引级别分组
6.将函数跟数组、列表、字典、Series混合使用也不是问题,因为任何东西最终都会被转换为数组
将这些数据片段做成字典
dict(list(df.groupby(['key1'])))#dict(list())
{'a': data1 data2 key1 key2
0 -0.410122 0.247895 a one
1 -0.627470 -0.989268 a two
4 -0.297191 0.954447 a one, 'b': data1 data2 key1 key2
2 0.179488 -0.054570 b one
3 -0.299878 -1.640494 b two}
分组后进行一些统计、计算等
1. 分组后,返回一个含有分组大小的Series
按key1分组
df.groupby(['key1']).size()
key1
a 3
b 2
dtype: int64
dict(['a1','x2','e3'])
{'a': '1', 'e': '3', 'x': '2'}
按[key1,key2]分组
df.groupby(['key1','key2']).size()
key1 key2
a one 2
two 1
b one 1
two 1
dtype: int64
2. 对data1按key1进行分组,并计算data1列的平均值
df['data1'].groupby(df['key1']).mean()
#groupby没有进行任何的计算。它只是进行了一个分组
key1
a -0.444928
b -0.060195
Name: data1, dtype: float64
df.groupby(['key1'])['data1'].mean()#理解:对df按key1分组,并计算分组后df['data1']的均值
#等价于:df.groupby(['key1']).data1.mean()
key1
a -0.444928
b -0.060195
Name: data1, dtype: float64
说明:
groupby没有进行任何的计算。它只是进行了一个分组。
数据(Series)根据分组键进行了聚合,产生了一个新的Series,其索引为key1列中的唯一值。
这种索引操作所返回的对象是一个已分组的DataFrame(如果传入的是列表或数组)或已分组的Series
df.groupby(['key1'])['data1'].size()
key1
a 3
b 2
Name: data1, dtype: int64
3.对data1按[key1,key2]进行分组,并计算data1的平均值
df['data1'].groupby([df['key1'],df['key2']]).mean()
key1 key2
a one -0.353657
two -0.627470
b one 0.179488
two -0.299878
Name: data1, dtype: float64
df.groupby(['key1','key2'])['data1'].mean()
#等价于:df.groupby(['key1','key2']).data1'.mean()
key1 key2
a one -0.353657
two -0.627470
b one 0.179488
two -0.299878
Name: data1, dtype: float64
通过两个键对数据进行了分组,得到的Series具有一个层次化索引(由唯一的键对组成):
df.groupby(['key1','key2'])['data1'].mean().unstack()
key2 | one | two |
---|---|---|
key1 | ||
a | -0.353657 | -0.627470 |
b | 0.179488 | -0.299878 |
在上面这些示例中,分组键均为Series。实际上,分组键可以是任何长度适当的数组。非常灵活。
横方向上
按列的数据类型(df.dtypes)来分
df共两种数据类型:float64和object,所以会分为两组(dtype(‘float64’),数据片),(dtype(‘O’), 数据片)
list(df.groupby(df.dtypes, axis=1))
[(dtype('float64'), data1 data2
0 -0.410122 0.247895
1 -0.627470 -0.989268
2 0.179488 -0.054570
3 -0.299878 -1.640494
4 -0.297191 0.954447), (dtype('O'), key1 key2
0 a one
1 a two
2 b one
3 b two
4 a one)]
agg的应用
groupby+agg 可以对groupby的结果同时应用多个函数
SeriesGroupBy的方法agg()参数:
aggregate(self, func_or_funcs, * args, ** kwargs)
func: function, string, dictionary, or list of string/functions
返回:aggregated的Series
s= pd.Series([10,20,30,40])
s
0 10
1 20
2 30
3 40
dtype: int64
for n,g in s.groupby([1,1,2,2]):
print(n)
print(g)
1
0 10
1 20
dtype: int64
2
2 30
3 40
dtype: int64
s.groupby([1,1,2,2]).min()
1 10
2 30
dtype: int64
#等价于这个:
s.groupby([1,1,2,2]).agg('min')
1 10
2 30
dtype: int64
s.groupby([1,1,2,2]).agg(['min','max'])#加[],func仅接受一个参数
min | max | |
---|---|---|
1 | 10 | 20 |
2 | 30 | 40 |
常常这样用:
df
data1 | data2 | key1 | key2 | |
---|---|---|---|---|
0 | -0.410122 | 0.247895 | a | one |
1 | -0.627470 | -0.989268 | a | two |
2 | 0.179488 | -0.054570 | b | one |
3 | -0.299878 | -1.640494 | b | two |
4 | -0.297191 | 0.954447 | a | one |
比较下面,可以看出agg的用处:
df.groupby(['key1'])['data1'].min()
key1
a -0.627470
b -0.299878
Name: data1, dtype: float64
df.groupby(['key1'])['data1'].agg({'min'})
min | |
---|---|
key1 | |
a | -0.627470 |
b | -0.299878 |
#推荐用这个√
df.groupby(['key1']).agg({'data1':'min'})#对data1列,取各组的最小值,名字还是data1
data1 | |
---|---|
key1 | |
a | -0.627470 |
b | -0.299878 |
#按key1分组后,aggregate各组data1的最小值和最大值:
df.groupby(['key1'])['data1'].agg({'min','max'})
max | min | |
---|---|---|
key1 | ||
a | -0.297191 | -0.627470 |
b | 0.179488 | -0.299878 |
#推荐用这个√
df.groupby(['key1']).agg({'data1':['min','max']})
data1 | ||
---|---|---|
min | max | |
key1 | ||
a | -0.627470 | -0.297191 |
b | -0.299878 | 0.179488 |
可以对groupby的结果更正列名(不推荐用这个,哪怕在后面单独更改列名)
# 对data1,把min更名为a,max更名为b
df.groupby(['key1'])['data1'].agg({'a':'min','b':'max'})#这里的'min' 'max'为两个函数名
d:\python27\lib\site-packages\ipykernel_launcher.py:2: FutureWarning: using a dict on a Series for aggregation
is deprecated and will be removed in a future version
a | b | |
---|---|---|
key1 | ||
a | -0.627470 | -0.297191 |
b | -0.299878 | 0.179488 |