前导
更多文章代码详情可查看博主个人网站：https://www.iwtmbtly.com/
导入需要使用的库和文件：
>>> import pandas as pd
>>> import numpy as np
一、时序的创建
（一）四类时间变量
现在理解可能关于③和④有些困惑，后面会作出一些说明：
（二）时间点的创建
1. to_datetime方法
Pandas在时间点建立的输入格式规定上给了很大的自由度，下面的语句都能正确建立同一时间点：
>>> pd.to_datetime('2020.1.1')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('2020 1.1')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('2020 1 1')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('2020 1-1')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('2020-1 1')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('2020-1-1')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('2020/1/1')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('1.1.2020')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('1.1 2020')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('1 1 2020')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('1 1-2020')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('1-1 2020')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('1-1-2020')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('1/1/2020')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('20200101')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('2020.0101')
Timestamp('2020-01-01 00:00:00')
下面的语句都会报错:
# pd.to_datetime('2020\\1\\1')
# pd.to_datetime('2020`1`1')
# pd.to_datetime('2020.1 1')
# pd.to_datetime('1 1.2020')
此时可利用format参数强制匹配:
>>> pd.to_datetime('2020\\1\\1',format='%Y\\%m\\%d')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('2020`1`1',format='%Y`%m`%d')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('2020.1 1',format='%Y.%m %d')
Timestamp('2020-01-01 00:00:00')
>>> pd.to_datetime('1 1.2020',format='%d %m.%Y')
Timestamp('2020-01-01 00:00:00')
同时，使用列表可以将其转为时间点索引:
>>> pd.Series(range(2),index=pd.to_datetime(['2020/1/1','2020/1/2']))
2020-01-01 0
2020-01-02 1
dtype: int64
>>> type(pd.to_datetime(['2020/1/1','2020/1/2']))
<class 'pandas.core.indexes.datetimes.DatetimeIndex'>
对于DataFrame而言，如果列已经按照时间顺序排好，则利用to_datetime可自动转换:
>>> df = pd.DataFrame({
'year': [2020, 2020],'month': [1, 1], 'day': [1, 2]})
>>> pd.to_datetime(df)
0 2020-01-01
1 2020-01-02
dtype: datetime64[ns]
2. 时间精度与范围限制
扫描二维码关注公众号，回复：
15619325 查看本文章
事实上，Timestamp的精度远远不止day，可以最小到纳秒(ns)：
>>> pd.to_datetime('2020/1/1 00:00:00.123456789')
Timestamp('2020-01-01 00:00:00.123456789')
同时，它带来范围的代价就是只有大约584年的时间点是可用的:
>>> pd.Timestamp.min
Timestamp('1677-09-21 00:12:43.145224193')
>>> pd.Timestamp.max
Timestamp('2262-04-11 23:47:16.854775807')
3. date_range方法
一般来说，start/end/periods（时间点个数）/freq（间隔方法）是该方法最重要的参数，给定了其中的3个，剩下的一个就会被确定
>>> pd.date_range(start='2020/1/1',end='2020/1/10',periods=3)
DatetimeIndex(['2020-01-01 00:00:00', '2020-01-05 12:00:00',
'2020-01-10 00:00:00'],
dtype='datetime64[ns]', freq=None)
>>> pd.date_range(start='2020/1/1',end='2020/1/10',freq='D')
DatetimeIndex(['2020-01-01', '2020-01-02', '2020-01-03', '2020-01-04',
'2020-01-05', '2020-01-06', '2020-01-07', '2020-01-08',
'2020-01-09', '2020-01-10'],
dtype='datetime64[ns]', freq='D')
>>> pd.date_range(start='2020/1/1',periods=3,freq='D')
DatetimeIndex(['2020-01-01', '2020-01-02', '2020-01-03'], dtype='datetime64[ns]', freq='D')
>>> pd.date_range(end='2020/1/3',periods=3,freq='D')
DatetimeIndex(['2020-01-01', '2020-01-02', '2020-01-03'], dtype='datetime64[ns]', freq='D')
其中freq参数有许多选项，下面将常用部分罗列如下，更多选项可看这里
>>> pd.date_range(start='2020/1/1',periods=3,freq='T')
DatetimeIndex(['2020-01-01 00:00:00', '2020-01-01 00:01:00',
'2020-01-01 00:02:00'],
dtype='datetime64[ns]', freq='T')
>>> pd.date_range(start='2020/1/1',periods=3,freq='M')
DatetimeIndex(['2020-01-31', '2020-02-29', '2020-03-31'], dtype='datetime64[ns]', freq='M')
>>> pd.date_range(start='2020/1/1',periods=3,freq='BYS')
DatetimeIndex(['2020-01-01', '2021-01-01', '2022-01-03'], dtype='datetime64[ns]', freq='BAS-JAN')
bdate_range是一个类似与date_range的方法，特点在于可以在自带的工作日间隔设置上，再选择weekmask参数和holidays参数
它的freq中有一个特殊的’C’/‘CBM’/'CBMS’选项，表示定制，需要联合weekmask参数和holidays参数使用
例如现在需要将工作日中的周一、周二、周五3天保留，并将部分holidays剔除:
>>> weekmask = 'Mon Tue Fri'
>>> holidays = [pd.Timestamp('2020/1/%s'%i) for i in range(7,13)]
>>> #注意holidays
>>> pd.bdate_range(start='2020-1-1',end='2020-1-15',freq='C',weekmask=weekmask,holidays=holidays)
DatetimeIndex(['2020-01-03', '2020-01-06', '2020-01-13', '2020-01-14'], dtype='datetime64[ns]', freq='C')
（三）DateOffset对象
1. DataOffset与Timedelta的区别
Timedelta绝对时间差的特点指无论是冬令时还是夏令时，增减1day都只计算24小时
DataOffset相对时间差指，无论一天是23\24\25小时，增减1day都与当天相同的时间保持一致
例如，英国当地时间 2020年03月29日，01:00:00 时钟向前调整 1 小时 变为 2020年03月29日，02:00:00，开始夏令时：
>>> ts = pd.Timestamp('2020-3-29 01:00:00', tz='Europe/Helsinki')
>>> ts + pd.Timedelta(days=1)
Timestamp('2020-03-30 02:00:00+0300', tz='Europe/Helsinki')
>>>
>>> ts + pd.DateOffset(days=1)
Timestamp('2020-03-30 01:00:00+0300', tz='Europe/Helsinki')
这似乎有些令人头大，但只要把tz（time zone）去除就可以不用管它了，两者保持一致，除非要使用到时区变换:
>>> ts = pd.Timestamp('2020-3-29 01:00:00')
>>> ts + pd.Timedelta(days=1)
Timestamp('2020-03-30 01:00:00')
>>> ts + pd.DateOffset(days=1)
Timestamp('2020-03-30 01:00:00')
2. 增减一段时间
DateOffset的可选参数包括years/months/weeks/days/hours/minutes/seconds
>>> pd.Timestamp('2020-01-01') + pd.DateOffset(minutes=20) - pd.DateOffset(weeks=2)
Timestamp('2019-12-18 00:20:00')
3. 各类常用offset对象
>>> pd.Timestamp('2020-01-01') + pd.offsets.Week(2)
Timestamp('2020-01-15 00:00:00')
>>> pd.Timestamp('2020-01-01') + pd.offsets.BQuarterBegin(1)
Timestamp('2020-03-02 00:00:00')
4. 序列的offset操作
利用apply函数:
>>> pd.Series(pd.offsets.BYearBegin(3).apply(i) for i in pd.date_range('20200101',periods=3,freq='Y'))
0 2023-01-02
1 2024-01-01
2 2025-01-01
dtype: datetime64[ns]
直接使用对象加减:
>>> pd.date_range('20200101',periods=3,freq='Y') + pd.offsets.BYearBegin(3)
DatetimeIndex(['2023-01-02', '2024-01-01', '2025-01-01'], dtype='datetime64[ns]', freq=None)
定制offset，可以指定weekmask和holidays参数（思考为什么三个都是一个值）
>>> pd.Series(pd.offsets.CDay(3,weekmask='Wed Fri',holidays='2020010').apply(i)
... for i in pd.date_range('20200105',periods=3,freq='D'))
0 2020-01-15
1 2020-01-15
2 2020-01-15
dtype: datetime64[ns]
二、时序的索引及属性
（一）索引切片
>>> rng = pd.date_range('2020','2021', freq='W')
>>> ts = pd.Series(np.random.randn(len(rng)), index=rng)
>>> ts.head()
2020-01-05 -0.748400
2020-01-12 0.486114
2020-01-19 0.510675
2020-01-26 0.757519
2020-02-02 -0.839067
Freq: W-SUN, dtype: float64
>>> ts['2020-01-26']
0.757519483225889
合法字符自动转换为时间点：
>>> ts['2020-01-26':'20200726'].head()
2020-01-26 0.757519
2020-02-02 -0.839067
2020-02-09 0.448796
2020-02-16 0.420513
2020-02-23 -1.340417
Freq: W-SUN, dtype: float64
（二）子集索引
>>> ts['2020-7'].head()
2020-07-05 -0.887375
2020-07-12 0.068180
2020-07-19 -0.000156
2020-07-26 1.562112
Freq: W-SUN, dtype: float64
支持混合形态索引：
>>> ts['2011-1':'20200726'].head()
2020-01-05 -0.748400
2020-01-12 0.486114
2020-01-19 0.510675
2020-01-26 0.757519
2020-02-02 -0.839067
Freq: W-SUN, dtype: float64
（三）时间点的属性
采用dt对象可以轻松获得关于时间的信息：
>>> pd.Series(ts.index).dt.isocalendar().week.head()
0 1
1 2
2 3
3 4
4 5
Name: week, dtype: UInt32
>>> pd.Series(ts.index).dt.day.head()
0 5
1 12
2 19
3 26
4 2
dtype: int64
利用strftime可重新修改时间格式：
>>> pd.Series(ts.index).dt.strftime('%Y-间隔1-%m-间隔2-%d').head()
0 2020-间隔1-01-间隔2-05
1 2020-间隔1-01-间隔2-12
2 2020-间隔1-01-间隔2-19
3 2020-间隔1-01-间隔2-26
4 2020-间隔1-02-间隔2-02
dtype: object
对于datetime对象可以直接通过属性获取信息：
>>> pd.date_range('2020','2021', freq='W').month
Int64Index([ 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 4, 4, 4, 4,
5, 5, 5, 5, 5, 6, 6, 6, 6, 7, 7, 7, 7, 8, 8, 8, 8,
8, 9, 9, 9, 9, 10, 10, 10, 10, 11, 11, 11, 11, 11, 12, 12, 12,
12],
dtype='int64')
>>> pd.date_range('2020','2021', freq='W').weekday
Int64Index([6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
6, 6, 6, 6, 6, 6, 6, 6],
dtype='int64')
三、重采样
所谓重采样，就是指resample函数，它可以看做时序版本的groupby函数
（一）resample对象的基本操作
采样频率一般设置为上面提到的offset字符：
>>> df_r = pd.DataFrame(np.random.randn(1000, 3),index=pd.date_range('1/1/2020', freq='S', periods=1000),
... columns=['A', 'B', 'C'])
>>> r = df_r.resample('3min')
>>> r
<pandas.core.resample.DatetimeIndexResampler object at 0x7f82387e4340>
>>> r.sum()
A B C
2020-01-01 00:00:00 -7.516439 -27.783036 -11.448831
2020-01-01 00:03:00 -9.991624 7.390296 8.338640
2020-01-01 00:06:00 7.468198 -22.687593 10.293133
2020-01-01 00:09:00 -26.955084 -23.255671 -10.254862
2020-01-01 00:12:00 9.351612 -16.941258 9.323046
2020-01-01 00:15:00 -5.380861 -0.258748 -9.376369
>>> df_r2 = pd.DataFrame(np.random.randn(200, 3),index=pd.date_range('1/1/2020', freq='D', periods=200),
... columns=['A', 'B', 'C'])
>>> r = df_r2.resample('CBMS')
>>> r.sum()
A B C
2020-01-01 -2.941740 5.320574 -6.844297
2020-02-03 5.239486 -8.492715 3.398018
2020-03-02 5.122721 -6.177475 1.329978
2020-04-01 -3.582743 0.851905 -2.708295
2020-05-01 1.538799 0.209188 7.031907
2020-06-01 8.507732 -0.766705 -1.486927
2020-07-01 -2.576345 2.197384 -3.776819
（二）采样集合
>>> r = df_r.resample('3T')
>>> r['A'].mean()
2020-01-01 00:00:00 -0.041758
2020-01-01 00:03:00 -0.055509
2020-01-01 00:06:00 0.041490
2020-01-01 00:09:00 -0.149750
2020-01-01 00:12:00 0.051953
2020-01-01 00:15:00 -0.053809
Freq: 3T, Name: A, dtype: float64
>>> r['A'].agg([np.sum, np.mean, np.std])
sum mean std
2020-01-01 00:00:00 -7.516439 -0.041758 1.031633
2020-01-01 00:03:00 -9.991624 -0.055509 1.058948
2020-01-01 00:06:00 7.468198 0.041490 0.985695
2020-01-01 00:09:00 -26.955084 -0.149750 0.942381
2020-01-01 00:12:00 9.351612 0.051953 0.933944
2020-01-01 00:15:00 -5.380861 -0.053809 1.033877
类似地，可以使用函数/lambda表达式：
>>> r.agg({
'A': np.sum,'B': lambda x: max(x)-min(x)})
A B
2020-01-01 00:00:00 -7.516439 5.848965
2020-01-01 00:03:00 -9.991624 5.735483
2020-01-01 00:06:00 7.468198 5.503003
2020-01-01 00:09:00 -26.955084 5.264593
2020-01-01 00:12:00 9.351612 5.774718
2020-01-01 00:15:00 -5.380861 4.630647
（三）采样组的迭代
采样组的迭代和groupby迭代完全类似，对于每一个组都可以分别做相应操作：
>>> small = pd.Series(range(6),index=pd.to_datetime(['2020-01-01 00:00:00', '2020-01-01 00:30:00'
... , '2020-01-01 00:31:00','2020-01-01 01:00:00'
... ,'2020-01-01 03:00:00','2020-01-01 03:05:00']))
>>> resampled = small.resample('H')
>>> for name, group in resampled:
... print("Group: ", name)
... print("-" * 27)
... print(group, end="\n\n")
...
Group: 2020-01-01 00:00:00
---------------------------
2020-01-01 00:00:00 0
2020-01-01 00:30:00 1
2020-01-01 00:31:00 2
dtype: int64
Group: 2020-01-01 01:00:00
---------------------------
2020-01-01 01:00:00 3
dtype: int64
Group: 2020-01-01 02:00:00
---------------------------
Series([], dtype: int64)
Group: 2020-01-01 03:00:00
---------------------------
2020-01-01 03:00:00 4
2020-01-01 03:05:00 5
dtype: int64
四、窗口函数
下面主要介绍pandas中两类主要的窗口(window)函数:rolling/expanding：
>>> s = pd.Series(np.random.randn(1000),index=pd.date_range('1/1/2020', periods=1000))
>>> s.head()
2020-01-01 -0.504213
2020-01-02 -0.481141
2020-01-03 -0.799043
2020-01-04 0.382436
2020-01-05 -1.933380
Freq: D, dtype: float64
（一）Rolling
1. 常用聚合
所谓rolling方法，就是规定一个窗口，它和groupby对象一样，本身不会进行操作，需要配合聚合函数才能计算结果：
>>> s.rolling(window=50)
Rolling [window=50,center=False,axis=0,method=single]
>>> s.rolling(window=50).mean()
2020-01-01 NaN
2020-01-02 NaN
2020-01-03 NaN
2020-01-04 NaN
2020-01-05 NaN
...
2022-09-22 0.061305
2022-09-23 0.006119
2022-09-24 0.020960
2022-09-25 -0.004617
2022-09-26 -0.000460
Freq: D, Length: 1000, dtype: float64
min_periods参数是指需要的非缺失数据点数量阀值：
>>> s.rolling(window=50,min_periods=3).mean().head()
2020-01-01 NaN
2020-01-02 NaN
2020-01-03 -0.594799
2020-01-04 -0.350490
2020-01-05 -0.667068
Freq: D, dtype: float64
count/sum/mean/median/min/max/std/var/skew/kurt/quantile/cov/corr都是常用的聚合函数
2. rolling的apply聚合
使用apply聚合时，只需记住传入的是window大小的Series，输出的必须是标量即可，比如如下计算变异系数:
>>> s.rolling(window=50,min_periods=3).apply(lambda x:x.std()/x.mean()).head()
2020-01-01 NaN
2020-01-02 NaN
2020-01-03 -0.298010
2020-01-04 -1.453968
2020-01-05 -1.250537
Freq: D, dtype: float64
3. 基于时间的rolling
>>> s.rolling('15D').mean().head()
2020-01-01 -0.504213
2020-01-02 -0.492677
2020-01-03 -0.594799
2020-01-04 -0.350490
2020-01-05 -0.667068
Freq: D, dtype: float64
可选closed=‘right’（默认）‘left’‘both’'neither’参数，决定端点的包含情况
>>> s.rolling('15D', closed='right').sum().head()
2020-01-01 -0.504213
2020-01-02 -0.985354
2020-01-03 -1.784397
2020-01-04 -1.401961
2020-01-05 -3.335340
Freq: D, dtype: float64
（二）Expanding
1. expanding函数
普通的expanding函数等价与rolling(window=len(s),min_periods=1)，是对序列的累计计算：
>>> s.rolling(window=len(s),min_periods=1).sum().head()
2020-01-01 -0.504213
2020-01-02 -0.985354
2020-01-03 -1.784397
2020-01-04 -1.401961
2020-01-05 -3.335340
Freq: D, dtype: float64
>>> s.expanding().sum().head()
2020-01-01 -0.504213
2020-01-02 -0.985354
2020-01-03 -1.784397
2020-01-04 -1.401961
2020-01-05 -3.335340
Freq: D, dtype: float64
apply方法也是同样可用的：
>>> s.expanding().apply(lambda x:sum(x)).head()
2020-01-01 -0.504213
2020-01-02 -0.985354
2020-01-03 -1.784397
2020-01-04 -1.401961
2020-01-05 -3.335340
Freq: D, dtype: float64
2. 几个特别的Expanding类型函数
>>> s.cumsum().head()
2020-01-01 -0.504213
2020-01-02 -0.985354
2020-01-03 -1.784397
2020-01-04 -1.401961
2020-01-05 -3.335340
Freq: D, dtype: float64
>>> s.cumprod().head()
2020-01-01 -0.504213
2020-01-02 0.242598
2020-01-03 -0.193846
2020-01-04 -0.074134
2020-01-05 0.143329
Freq: D, dtype: float64
shift/diff/pct_change都是涉及到了元素关系：
-
shift是指序列索引不变，但值向后移动
-
diff是指前后元素的差，period参数表示间隔，默认为1，并且可以为负
-
pct_change是值前后元素的变化百分比，period参数与diff类似
>>> s.shift(2).head()
2020-01-01 NaN
2020-01-02 NaN
2020-01-03 -0.504213
2020-01-04 -0.481141
2020-01-05 -0.799043
Freq: D, dtype: float64
>>> s.diff(3).head()
2020-01-01 NaN
2020-01-02 NaN
2020-01-03 NaN
2020-01-04 0.886649
2020-01-05 -1.452239
Freq: D, dtype: float64
>>> s.pct_change(3).head()
2020-01-01 NaN
2020-01-02 NaN
2020-01-03 NaN
2020-01-04 -1.758481
2020-01-05 3.018323
Freq: D, dtype: float64