pandas（一）pandas的数据结构

Logout 1-pandas_objects Last Checkpoint: 06/13/2018 (unsaved changes)

Python 3 

Trusted

• File
• Edit
• View
• Insert
• Cell
• Kernel
• Widgets
• Help

CodeMarkdownRaw NBConvertHeading-

Pandas的数据结构

pandas是数据分析的一个核心框架，集成了数据结构化和数据清洗以及分析的一些方法。pandas在numpy的基础上新增了三个数据类型，Series、DataFrame、Panel

In [1]:

import numpy as np

import pandas as pd

​

1、Series

Series是一种类似与一维数组的对象，由下面两个部分组成：

• values：一组数据（ndarray类型）
• index：相关的数据索引标签

In [2]:

# 引入Series

from pandas import Series

In [ ]:

​

1）Series的创建

两种创建方式：

(1) 由列表或numpy数组创建

默认索引为0到N-1的整数型索引

In [3]:

nd = np.array([1,2,3,4])

nd

Out[3]:

array([1, 2, 3, 4])

In [4]:

s = Series(nd) # 没有指定索引默认0~N-1

s

Out[4]:

0    1
1    2
2    3
3    4
dtype: int32

In [5]:

s = Series([1,2,3,4,5],index=list("abcde"))

s

Out[5]:

a    1
b    2
c    3
d    4
e    5
dtype: int64

In [6]:

s["a"]

Out[6]:

1

In [7]:

s = Series([1,2,3,4,5,6],index=["A","A","B","B","A","C"])

s

Out[7]:

A    1
A    2
B    3
B    4
A    5
C    6
dtype: int64

In [8]:

s["B"]

Out[8]:

B    3
B    4
dtype: int64

(2) 由字典创建

In [9]:

s = Series({"a":1,"b":2,"c":3})

s

Out[9]:

a    1
b    2
c    3
dtype: int64

In [10]:

s1=Series({"a":123,"b":431},index=list("ac"))

s1

Out[10]:

a    123.0
c      NaN
dtype: float64

============================================

练习1：

使用多种方法创建以下Series，命名为s1：
语文 150
数学 150
英语 150
理综 300

============================================

In [11]:

#数组

nd = np.array([150,150,150,300])

s1 = Series(nd,index=["语文","数学","英语","理综"])

s1

Out[11]:

语文    150
数学    150
英语    150
理综    300
dtype: int32

In [12]:

dic = {"语文":150,"数学":150,"英语":150,"理综":300}

s2 = Series(dic)

s2

Out[12]:

数学    150
理综    300
英语    150
语文    150
dtype: int64

In [13]:

nd[0] = 1000

nd

Out[13]:

array([1000,  150,  150,  300])

In [14]:

s1 #由数组和列表创建Series是一个浅拷贝（只拷贝引用地址，不拷贝对象本身）

Out[14]:

语文    1000
数学     150
英语     150
理综     300
dtype: int32

In [15]:

dic["数学"] = 120

dic

Out[15]:

{'数学': 120, '理综': 300, '英语': 150, '语文': 150}

In [16]:

s2 # 由字典创建Series是一个创建副本的过程（也叫深拷贝）

Out[16]:

数学    150
理综    300
英语    150
语文    150
dtype: int64

2）Series的索引和切片

可以使用中括号取单个索引（此时返回的是元素类型），或者中括号里一个列表取多个索引（此时返回的仍然是一个Series类型）。分为显示索引和隐式索引：

(1) 显式索引：

- 使用index中的元素作为索引值
- 使用.loc[]（推荐）

注意，此时是闭区间

In [17]:

s

Out[17]:

a    1
b    2
c    3
dtype: int64

In [18]:

s.values

Out[18]:

array([1, 2, 3], dtype=int64)

In [19]:

s.index # index的值是显示索引

Out[19]:

Index(['a', 'b', 'c'], dtype='object')

In [20]:

#方式一

s["a"]

Out[20]:

1

In [21]:

# 方式二（推荐）

s.loc["a"] 

Out[21]:

1

In [22]:

s.loc["a","b"] # 不能写成这种形式

---------------------------------------------------------------------------
IndexingError                             Traceback (most recent call last)
<ipython-input-22-1c72e1f53463> in <module>()
----> 1 s.loc["a","b"] # 不能写成这种形式

d:\Anaconda\lib\site-packages\pandas\core\indexing.py in __getitem__(self, key)
   1323             except (KeyError, IndexError):
   1324                 pass
-> 1325             return self._getitem_tuple(key)
   1326         else:
   1327             key = com._apply_if_callable(key, self.obj)

d:\Anaconda\lib\site-packages\pandas\core\indexing.py in _getitem_tuple(self, tup)
    839 
    840         # no multi-index, so validate all of the indexers
--> 841         self._has_valid_tuple(tup)
    842 
    843         # ugly hack for GH #836

d:\Anaconda\lib\site-packages\pandas\core\indexing.py in _has_valid_tuple(self, key)
    186         for i, k in enumerate(key):
    187             if i >= self.obj.ndim:
--> 188                 raise IndexingError('Too many indexers')
    189             if not self._has_valid_type(k, i):
    190                 raise ValueError("Location based indexing can only have [%s] "

IndexingError: Too many indexers

In [ ]:

s.loc[["a","b","a"]] # 通过列表来查找，实际上就是从s中截取子series

(2) 隐式索引：

- 使用整数作为索引值
- 使用.iloc[]（推荐）

注意，此时是半开区间

In [ ]:

s.iloc[0]

In [23]:

s2.iloc[0]

Out[23]:

150

In [24]:

s.iloc[0,1]

---------------------------------------------------------------------------
IndexingError                             Traceback (most recent call last)
<ipython-input-24-61ca01bee2d4> in <module>()
----> 1 s.iloc[0,1]

d:\Anaconda\lib\site-packages\pandas\core\indexing.py in __getitem__(self, key)
   1323             except (KeyError, IndexError):
   1324                 pass
-> 1325             return self._getitem_tuple(key)
   1326         else:
   1327             key = com._apply_if_callable(key, self.obj)

d:\Anaconda\lib\site-packages\pandas\core\indexing.py in _getitem_tuple(self, tup)
   1660     def _getitem_tuple(self, tup):
   1661 
-> 1662         self._has_valid_tuple(tup)
   1663         try:
   1664             return self._getitem_lowerdim(tup)

d:\Anaconda\lib\site-packages\pandas\core\indexing.py in _has_valid_tuple(self, key)
    186         for i, k in enumerate(key):
    187             if i >= self.obj.ndim:
--> 188                 raise IndexingError('Too many indexers')
    189             if not self._has_valid_type(k, i):
    190                 raise ValueError("Location based indexing can only have [%s] "

IndexingError: Too many indexers

In [25]:

s.iloc[[0,1]]

Out[25]:

a    1
b    2
dtype: int64

(3)切片

In [26]:

# 显示

s.loc["a":"c"] # 闭区间

Out[26]:

a    1
b    2
c    3
dtype: int64

In [27]:

# 隐式

s.iloc[0:2] # 前闭后开

Out[27]:

a    1
b    2
dtype: int64

In [28]:

s = Series([1,2,3,4,5,6],index=["A","A","B","C","B","C"])

s

Out[28]:

A    1
A    2
B    3
C    4
B    5
C    6
dtype: int64

In [29]:

s.loc["A":"C"]

# 如果显式索引中有重复的不建议用显示索引来切片

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-29-dd4f4933695c> in <module>()
----> 1 s.loc["A":"C"]
      2 # 如果显式索引中有重复的不建议用显示索引来切片

d:\Anaconda\lib\site-packages\pandas\core\indexing.py in __getitem__(self, key)
   1326         else:
   1327             key = com._apply_if_callable(key, self.obj)
-> 1328             return self._getitem_axis(key, axis=0)
   1329 
   1330     def _is_scalar_access(self, key):

d:\Anaconda\lib\site-packages\pandas\core\indexing.py in _getitem_axis(self, key, axis)
   1504         if isinstance(key, slice):
   1505             self._has_valid_type(key, axis)
-> 1506             return self._get_slice_axis(key, axis=axis)
   1507         elif is_bool_indexer(key):
   1508             return self._getbool_axis(key, axis=axis)

d:\Anaconda\lib\site-packages\pandas\core\indexing.py in _get_slice_axis(self, slice_obj, axis)
   1354         labels = obj._get_axis(axis)
   1355         indexer = labels.slice_indexer(slice_obj.start, slice_obj.stop,
-> 1356                                        slice_obj.step, kind=self.name)
   1357 
   1358         if isinstance(indexer, slice):

d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in slice_indexer(self, start, end, step, kind)
   3348         """
   3349         start_slice, end_slice = self.slice_locs(start, end, step=step,
-> 3350                                                  kind=kind)
   3351 
   3352         # return a slice

d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in slice_locs(self, start, end, step, kind)
   3542         end_slice = None
   3543         if end is not None:
-> 3544             end_slice = self.get_slice_bound(end, 'right', kind)
   3545         if end_slice is None:
   3546             end_slice = len(self)

d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in get_slice_bound(self, label, side, kind)
   3496             if isinstance(slc, np.ndarray):
   3497                 raise KeyError("Cannot get %s slice bound for non-unique "
-> 3498                                "label: %r" % (side, original_label))
   3499 
   3500         if isinstance(slc, slice):

KeyError: "Cannot get right slice bound for non-unique label: 'C'"

============================================

练习2：

使用多种方法对练习1创建的Series s1进行索引和切片：

索引： 数学 150

切片： 语文 150 数学 150 英语 150

============================================

In [30]:

s1[[1]]

s1.loc[["数学"]]

Out[30]:

数学    150
dtype: int32

In [31]:

s1.loc["语文":"英语"]

Out[31]:

语文    1000
数学     150
英语     150
dtype: int32

In [32]:

s1.iloc[0:3]

Out[32]:

语文    1000
数学     150
英语     150
dtype: int32

3）Series的基本概念

可以把Series看成一个定长的有序字典

可以通过shape，size，index,values等得到series的属性

In [33]:

s.shape

Out[33]:

(6,)

In [34]:

s.reshape((3,2)) # 一般不对Series进行reshape操作，会改变原来的数据形式

d:\Anaconda\lib\site-packages\ipykernel_launcher.py:1: FutureWarning: reshape is deprecated and will raise in a subsequent release. Please use .values.reshape(...) instead
  """Entry point for launching an IPython kernel.

Out[34]:

array([[1, 2],
       [3, 4],
       [5, 6]], dtype=int64)

In [35]:

s.size

Out[35]:

6

In [36]:

s.index

Out[36]:

Index(['A', 'A', 'B', 'C', 'B', 'C'], dtype='object')

In [37]:

s.index[[0,1]]

Out[37]:

Index(['A', 'A'], dtype='object')

In [38]:

s.index[0:2]

Out[38]:

Index(['A', 'A'], dtype='object')

In [39]:

s.index = list("abcdef")

s

Out[39]:

a    1
b    2
c    3
d    4
e    5
f    6
dtype: int64

In [40]:

s.index[0] = "A" # index的值不允许单个修改

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-40-7c5a7b9320c3> in <module>()
----> 1 s.index[0] = "A" # index的值不允许单个修改

d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in __setitem__(self, key, value)
   1668 
   1669     def __setitem__(self, key, value):
-> 1670         raise TypeError("Index does not support mutable operations")
   1671 
   1672     def __getitem__(self, key):

TypeError: Index does not support mutable operations

可以通过head(),tail()快速查看Series对象的样式

In [41]:

# 把数据读入

data = pd.read_csv("./titanic.txt")

data

Out[41]:

	row.names	pclass	survived	name	age	embarked	home.dest	room	ticket	boat	sex
0	1	1st	1	Allen, Miss Elisabeth Walton	29.0000	Southampton	St Louis, MO	B-5	24160 L221	2	female
1	2	1st	0	Allison, Miss Helen Loraine	2.0000	Southampton	Montreal, PQ / Chesterville, ON	C26	NaN	NaN	female
2	3	1st	0	Allison, Mr Hudson Joshua Creighton	30.0000	Southampton	Montreal, PQ / Chesterville, ON	C26	NaN	(135)	male
3	4	1st	0	Allison, Mrs Hudson J.C. (Bessie Waldo Daniels)	25.0000	Southampton	Montreal, PQ / Chesterville, ON	C26	NaN	NaN	female
4	5	1st	1	Allison, Master Hudson Trevor	0.9167	Southampton	Montreal, PQ / Chesterville, ON	C22	NaN	11	male
5	6	1st	1	Anderson, Mr Harry	47.0000	Southampton	New York, NY	E-12	NaN	3	male
6	7	1st	1	Andrews, Miss Kornelia Theodosia	63.0000	Southampton	Hudson, NY	D-7	13502 L77	10	female
7	8	1st	0	Andrews, Mr Thomas, jr	39.0000	Southampton	Belfast, NI	A-36	NaN	NaN	male
8	9	1st	1	Appleton, Mrs Edward Dale (Charlotte Lamson)	58.0000	Southampton	Bayside, Queens, NY	C-101	NaN	2	female
9	10	1st	0	Artagaveytia, Mr Ramon	71.0000	Cherbourg	Montevideo, Uruguay	NaN	NaN	(22)	male
10	11	1st	0	Astor, Colonel John Jacob	47.0000	Cherbourg	New York, NY	NaN	17754 L224 10s 6d	(124)	male
11	12	1st	1	Astor, Mrs John Jacob (Madeleine Talmadge Force)	19.0000	Cherbourg	New York, NY	NaN	17754 L224 10s 6d	4	female
12	13	1st	1	Aubert, Mrs Leontine Pauline	NaN	Cherbourg	Paris, France	B-35	17477 L69 6s	9	female
13	14	1st	1	Barkworth, Mr Algernon H.	NaN	Southampton	Hessle, Yorks	A-23	NaN	B	male
14	15	1st	0	Baumann, Mr John D.	NaN	Southampton	New York, NY	NaN	NaN	NaN	male
15	16	1st	1	Baxter, Mrs James (Helene DeLaudeniere Chaput)	50.0000	Cherbourg	Montreal, PQ	B-58/60	NaN	6	female
16	17	1st	0	Baxter, Mr Quigg Edmond	24.0000	Cherbourg	Montreal, PQ	B-58/60	NaN	NaN	male
17	18	1st	0	Beattie, Mr Thomson	36.0000	Cherbourg	Winnipeg, MN	C-6	NaN	NaN	male
18	19	1st	1	Beckwith, Mr Richard Leonard	37.0000	Southampton	New York, NY	D-35	NaN	5	male
19	20	1st	1	Beckwith, Mrs Richard Leonard (Sallie Monypeny)	47.0000	Southampton	New York, NY	D-35	NaN	5	female
20	21	1st	1	Behr, Mr Karl Howell	26.0000	Cherbourg	New York, NY	C-148	NaN	5	male
21	22	1st	0	Birnbaum, Mr Jakob	25.0000	Cherbourg	San Francisco, CA	NaN	NaN	(148)	male
22	23	1st	1	Bishop, Mr Dickinson H.	25.0000	Cherbourg	Dowagiac, MI	B-49	NaN	7	male
23	24	1st	1	Bishop, Mrs Dickinson H. (Helen Walton)	19.0000	Cherbourg	Dowagiac, MI	B-49	NaN	7	female
24	25	1st	1	Bjornstrm-Steffansson, Mr Mauritz Hakan	28.0000	Southampton	Stockholm, Sweden / Washington, DC	NaN		D	male
25	26	1st	0	Blackwell, Mr Stephen Weart	45.0000	Southampton	Trenton, NJ	NaN	NaN	(241)	male
26	27	1st	1	Blank, Mr Henry	39.0000	Cherbourg	Glen Ridge, NJ	A-31	NaN	7	male
27	28	1st	1	Bonnell, Miss Caroline	30.0000	Southampton	Youngstown, OH	C-7	NaN	8	female
28	29	1st	1	Bonnell, Miss Elizabeth	58.0000	Southampton	Birkdale, England Cleveland, Ohio	C-103	NaN	8	female
29	30	1st	0	Borebank, Mr John James	NaN	Southampton	London / Winnipeg, MB	D-21/2	NaN	NaN	male
...	...	...	...	...	...	...	...	...	...	...	...
1283	1284	3rd	0	Vestrom, Miss Hulda Amanda Adolfina	NaN	NaN	NaN	NaN	NaN	NaN	female
1284	1285	3rd	0	Vonk, Mr Jenko	NaN	NaN	NaN	NaN	NaN	NaN	male
1285	1286	3rd	0	Ware, Mr Frederick	NaN	NaN	NaN	NaN	NaN	NaN	male
1286	1287	3rd	0	Warren, Mr Charles William	NaN	NaN	NaN	NaN	NaN	NaN	male
1287	1288	3rd	0	Wazli, Mr Yousif	NaN	NaN	NaN	NaN	NaN	NaN	male
1288	1289	3rd	0	Webber, Mr James	NaN	NaN	NaN	NaN	NaN	NaN	male
1289	1290	3rd	1	Wennerstrom, Mr August Edvard	NaN	NaN	NaN	NaN	NaN	NaN	male
1290	1291	3rd	0	Wenzel, Mr Linhart	NaN	NaN	NaN	NaN	NaN	NaN	male
1291	1292	3rd	0	Widegren, Mr Charles Peter	NaN	NaN	NaN	NaN	NaN	NaN	male
1292	1293	3rd	0	Wiklund, Mr Jacob Alfred	NaN	NaN	NaN	NaN	NaN	NaN	male
1293	1294	3rd	1	Wilkes, Mrs Ellen	NaN	NaN	NaN	NaN	NaN	NaN	female
1294	1295	3rd	0	Willer, Mr Aaron	NaN	NaN	NaN	NaN	NaN	NaN	male
1295	1296	3rd	0	Willey, Mr Edward	NaN	NaN	NaN	NaN	NaN	NaN	male
1296	1297	3rd	0	Williams, Mr Howard Hugh	NaN	NaN	NaN	NaN	NaN	NaN	male
1297	1298	3rd	0	Williams, Mr Leslie	NaN	NaN	NaN	NaN	NaN	NaN	male
1298	1299	3rd	0	Windelov, Mr Einar	NaN	NaN	NaN	NaN	NaN	NaN	male
1299	1300	3rd	0	Wirz, Mr Albert	NaN	NaN	NaN	NaN	NaN	NaN	male
1300	1301	3rd	0	Wiseman, Mr Phillippe	NaN	NaN	NaN	NaN	NaN	NaN	male
1301	1302	3rd	0	Wittevrongel, Mr Camiel	NaN	NaN	NaN	NaN	NaN	NaN	male
1302	1303	3rd	1	Yalsevac, Mr Ivan	NaN	NaN	NaN	NaN	NaN	NaN	male
1303	1304	3rd	0	Yasbeck, Mr Antoni	NaN	NaN	NaN	NaN	NaN	NaN	male
1304	1305	3rd	1	Yasbeck, Mrs Antoni	NaN	NaN	NaN	NaN	NaN	NaN	female
1305	1306	3rd	0	Youssef, Mr Gerios	NaN	NaN	NaN	NaN	NaN	NaN	male
1306	1307	3rd	0	Zabour, Miss Hileni	NaN	NaN	NaN	NaN	NaN	NaN	female
1307	1308	3rd	0	Zabour, Miss Tamini	NaN	NaN	NaN	NaN	NaN	NaN	female
1308	1309	3rd	0	Zakarian, Mr Artun	NaN	NaN	NaN	NaN	NaN	NaN	male
1309	1310	3rd	0	Zakarian, Mr Maprieder	NaN	NaN	NaN	NaN	NaN	NaN	male
1310	1311	3rd	0	Zenn, Mr Philip	NaN	NaN	NaN	NaN	NaN	NaN	male
1311	1312	3rd	0	Zievens, Rene	NaN	NaN	NaN	NaN	NaN	NaN	female
1312	1313	3rd	0	Zimmerman, Leo	NaN	NaN	NaN	NaN	NaN	NaN	male

1313 rows × 11 columns

In [42]:

data.head(3)

Out[42]:

	row.names	pclass	survived	name	age	embarked	home.dest	room	ticket	boat	sex
0	1	1st	1	Allen, Miss Elisabeth Walton	29.0	Southampton	St Louis, MO	B-5	24160 L221	2	female
1	2	1st	0	Allison, Miss Helen Loraine	2.0	Southampton	Montreal, PQ / Chesterville, ON	C26	NaN	NaN	female
2	3	1st	0	Allison, Mr Hudson Joshua Creighton	30.0	Southampton	Montreal, PQ / Chesterville, ON	C26	NaN	(135)	male

In [43]:

data.tail(10)

Out[43]:

	row.names	pclass	survived	name	age	embarked	home.dest	room	ticket	boat	sex
1303	1304	3rd	0	Yasbeck, Mr Antoni	NaN	NaN	NaN	NaN	NaN	NaN	male
1304	1305	3rd	1	Yasbeck, Mrs Antoni	NaN	NaN	NaN	NaN	NaN	NaN	female
1305	1306	3rd	0	Youssef, Mr Gerios	NaN	NaN	NaN	NaN	NaN	NaN	male
1306	1307	3rd	0	Zabour, Miss Hileni	NaN	NaN	NaN	NaN	NaN	NaN	female
1307	1308	3rd	0	Zabour, Miss Tamini	NaN	NaN	NaN	NaN	NaN	NaN	female
1308	1309	3rd	0	Zakarian, Mr Artun	NaN	NaN	NaN	NaN	NaN	NaN	male
1309	1310	3rd	0	Zakarian, Mr Maprieder	NaN	NaN	NaN	NaN	NaN	NaN	male
1310	1311	3rd	0	Zenn, Mr Philip	NaN	NaN	NaN	NaN	NaN	NaN	male
1311	1312	3rd	0	Zievens, Rene	NaN	NaN	NaN	NaN	NaN	NaN	female
1312	1313	3rd	0	Zimmerman, Leo	NaN	NaN	NaN	NaN	NaN	NaN	male

DataFrame是由Series构成的

In [44]:

data["age"]

Out[44]:

0       29.0000
1        2.0000
2       30.0000
3       25.0000
4        0.9167
5       47.0000
6       63.0000
7       39.0000
8       58.0000
9       71.0000
10      47.0000
11      19.0000
12          NaN
13          NaN
14          NaN
15      50.0000
16      24.0000
17      36.0000
18      37.0000
19      47.0000
20      26.0000
21      25.0000
22      25.0000
23      19.0000
24      28.0000
25      45.0000
26      39.0000
27      30.0000
28      58.0000
29          NaN
         ...   
1283        NaN
1284        NaN
1285        NaN
1286        NaN
1287        NaN
1288        NaN
1289        NaN
1290        NaN
1291        NaN
1292        NaN
1293        NaN
1294        NaN
1295        NaN
1296        NaN
1297        NaN
1298        NaN
1299        NaN
1300        NaN
1301        NaN
1302        NaN
1303        NaN
1304        NaN
1305        NaN
1306        NaN
1307        NaN
1308        NaN
1309        NaN
1310        NaN
1311        NaN
1312        NaN
Name: age, Length: 1313, dtype: float64

当索引没有对应的值时，可能出现缺失数据显示NaN（not a number）的情况

In [45]:

s = Series({"a":123,"b":345,"c":685},index=list("abcdef"))

s

Out[45]:

a    123.0
b    345.0
c    685.0
d      NaN
e      NaN
f      NaN
dtype: float64

In [46]:

s.index = list("abcdnm")

s

Out[46]:

a    123.0
b    345.0
c    685.0
d      NaN
n      NaN
m      NaN
dtype: float64

可以使用pd.isnull()，pd.notnull()，或自带isnull(),notnull()函数检测缺失数据

In [47]:

pd.isnull(s)

Out[47]:

a    False
b    False
c    False
d     True
n     True
m     True
dtype: bool

In [48]:

s.isnull()

Out[48]:

a    False
b    False
c    False
d     True
n     True
m     True
dtype: bool

In [49]:

ind = s.isnull()

ind

Out[49]:

a    False
b    False
c    False
d     True
n     True
m     True
dtype: bool

In [50]:

s[ind] # 索引对应的值为True则输出

# 输出所有的缺失的数据

Out[50]:

d   NaN
n   NaN
m   NaN
dtype: float64

In [51]:

s.notnull()

Out[51]:

a     True
b     True
c     True
d    False
n    False
m    False
dtype: bool

In [52]:

s[s.notnull()]

Out[52]:

a    123.0
b    345.0
c    685.0
dtype: float64

In [53]:

s1 = Series({"a":True,"b":False,"c":True,"d":False,"n":True,"m":True})

s1

Out[53]:

a     True
b    False
c     True
d    False
m     True
n     True
dtype: bool

In [54]:

s[s1] 

# 一个Series如果索引和另外一个完全一致，值是bool类型，这个Series是可以作为另外

# 一个Series的索引来查找对应元素的，查找的结果就是所有bool为True那些索引对应的元素

Out[54]:

a    123.0
c    685.0
n      NaN
m      NaN
dtype: float64

In [55]:

s

Out[55]:

a    123.0
b    345.0
c    685.0
d      NaN
n      NaN
m      NaN
dtype: float64

In [56]:

s[s.isnull()] = 1000

s

Out[56]:

a     123.0
b     345.0
c     685.0
d    1000.0
n    1000.0
m    1000.0
dtype: float64

Series对象本身及其实例都有一个name属性

In [57]:

s.name = "python"

s # name属性是series在DataFrame中的表头信息

Out[57]:

a     123.0
b     345.0
c     685.0
d    1000.0
n    1000.0
m    1000.0
Name: python, dtype: float64

【拓展】过滤条件

In [58]:

s = Series({"a":1,"b":2,"c":3})

s > 2

Out[58]:

a    False
b    False
c     True
dtype: bool

In [59]:

nd = np.random.randint(0,20,size=10)

nd

Out[59]:

array([12, 15, 13,  6,  5,  3,  1,  3,  2,  6])

In [60]:

a = nd > 10

a

Out[60]:

array([ True,  True,  True, False, False, False, False, False, False, False], dtype=bool)

In [61]:

nd[a]

Out[61]:

array([12, 15, 13])

19、将一个一维数组转化为二进制表示矩阵。例如

[1,2,3]

转化为

[[0,0,1],

[0,1,0],

[0,1,1]]

In [62]:

1 and 0

Out[62]:

0

In [63]:

1 or 0

Out[63]:

1

In [64]:

1 & 2

Out[64]:

0

In [65]:

# 60 = 1*2^5 + 1*2^4 + 1*2^3 +0*2^2+0*2^1+ 1*2^0

# 111001 & 2^3

​

#2 = 010

# 3 = 011

# 011

# 010

1 & 2**0

1 & 2**1

1 & 2**2

3 & 2**0

3 & 2**1

3 & 2**2

​

​

​

​

Out[65]:

0

In [66]:

I = np.array([0,1,2,3,15,17,32,64,128])

I

Out[66]:

array([  0,   1,   2,   3,  15,  17,  32,  64, 128])

In [67]:

A = I.reshape((-1,1))

A

Out[67]:

array([[  0],
       [  1],
       [  2],
       [  3],
       [ 15],
       [ 17],
       [ 32],
       [ 64],
       [128]])

In [68]:

B = 2**np.arange(8)

B

Out[68]:

array([  1,   2,   4,   8,  16,  32,  64, 128], dtype=int32)

In [69]:

M = A & B

M

Out[69]:

array([[  0,   0,   0,   0,   0,   0,   0,   0],
       [  1,   0,   0,   0,   0,   0,   0,   0],
       [  0,   2,   0,   0,   0,   0,   0,   0],
       [  1,   2,   0,   0,   0,   0,   0,   0],
       [  1,   2,   4,   8,   0,   0,   0,   0],
       [  1,   0,   0,   0,  16,   0,   0,   0],
       [  0,   0,   0,   0,   0,  32,   0,   0],
       [  0,   0,   0,   0,   0,   0,  64,   0],
       [  0,   0,   0,   0,   0,   0,   0, 128]], dtype=int32)

  [[  0,0,0,0,0,0,0,0],        1,2,4,8,16,32,64,128
   [  1,1,1,1,1,11,1,1],
   [  2], 
   [  3],                 &     
   [ 15],
   [ 17],
   [ 32],
   [ 64],
   [128]]

In [70]:

M != 0

Out[70]:

array([[False, False, False, False, False, False, False, False],
       [ True, False, False, False, False, False, False, False],
       [False,  True, False, False, False, False, False, False],
       [ True,  True, False, False, False, False, False, False],
       [ True,  True,  True,  True, False, False, False, False],
       [ True, False, False, False,  True, False, False, False],
       [False, False, False, False, False,  True, False, False],
       [False, False, False, False, False, False,  True, False],
       [False, False, False, False, False, False, False,  True]], dtype=bool)

In [71]:

M[M!=0] = 1

M

Out[71]:

array([[0, 0, 0, 0, 0, 0, 0, 0],
       [1, 0, 0, 0, 0, 0, 0, 0],
       [0, 1, 0, 0, 0, 0, 0, 0],
       [1, 1, 0, 0, 0, 0, 0, 0],
       [1, 1, 1, 1, 0, 0, 0, 0],
       [1, 0, 0, 0, 1, 0, 0, 0],
       [0, 0, 0, 0, 0, 1, 0, 0],
       [0, 0, 0, 0, 0, 0, 1, 0],
       [0, 0, 0, 0, 0, 0, 0, 1]], dtype=int32)

In [72]:

M[:,::-1]

Out[72]:

array([[0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 1],
       [0, 0, 0, 0, 0, 0, 1, 0],
       [0, 0, 0, 0, 0, 0, 1, 1],
       [0, 0, 0, 0, 1, 1, 1, 1],
       [0, 0, 0, 1, 0, 0, 0, 1],
       [0, 0, 1, 0, 0, 0, 0, 0],
       [0, 1, 0, 0, 0, 0, 0, 0],
       [1, 0, 0, 0, 0, 0, 0, 0]], dtype=int32)

4）Series的运算

(1) 适用于numpy的数组运算也适用于Series

In [73]:

s

Out[73]:

a    1
b    2
c    3
dtype: int64

In [74]:

s + 3

Out[74]:

a    4
b    5
c    6
dtype: int64

In [75]:

s * 3

Out[75]:

a    3
b    6
c    9
dtype: int64

(2) Series之间的运算

• 在运算中自动对齐不同索引的数据
• 如果索引不对应，则补NaN

• 注意：要想保留所有的index，则需要使用.add()函数

In [76]:

s1 = Series(np.random.randint(0,150,size=4),index=list("ABCD"),name="数学")

s1

Out[76]:

A    121
B     97
C     76
D     27
Name: 数学, dtype: int32

In [77]:

s2 = Series(np.random.randint(0,150,size=4),index=list("ABmn"),name="python")

s2

Out[77]:

A     64
B     19
m     17
n    137
Name: python, dtype: int32

In [78]:

np.nan + 3

Out[78]:

nan

In [79]:

s1 + s2

Out[79]:

A    185.0
B    116.0
C      NaN
D      NaN
m      NaN
n      NaN
dtype: float64

A 142 A 132 B 0 + B 107 C 6 m 50 D 67 n 5 m nan C nan n nan D nan

In [80]:

s1 * s2

Out[80]:

A    7744.0
B    1843.0
C       NaN
D       NaN
m       NaN
n       NaN
dtype: float64

In [81]:

s1 / s2

Out[81]:

A    1.890625
B    5.105263
C         NaN
D         NaN
m         NaN
n         NaN
dtype: float64

In [82]:

s1.add(s2)

Out[82]:

A    185.0
B    116.0
C      NaN
D      NaN
m      NaN
n      NaN
dtype: float64

In [83]:

s1.subtract(s2)

Out[83]:

A    57.0
B    78.0
C     NaN
D     NaN
m     NaN
n     NaN
dtype: float64

In [84]:

s1.pow(2)

Out[84]:

A    14641
B     9409
C     5776
D      729
Name: 数学, dtype: int32

============================================

练习3：

1. 想一想Series运算和ndarray运算的规则有什么不同？

2. 新建另一个索引包含“文综”的Series s2，并与s2进行多种算术操作。思考如何保存所有数据。

============================================

2、DataFrame

DataFrame是一个【表格型】的数据结构，可以看做是【由Series组成的字典】（共用同一个索引）。DataFrame由按一定顺序排列的多列数据组成。设计初衷是将Series的使用场景从一维拓展到多维。DataFrame既有行索引，也有列索引。

• 行索引：index
• 列索引：columns
• 值：values（numpy的二维数组）

1）DataFrame的创建

最常用的方法是传递一个字典来创建。DataFrame以字典的键作为每一【列】的名称，以字典的值（一个数组）作为每一列。

此外，DataFrame会自动加上每一行的索引（和Series一样）。

同Series一样，若传入的列与字典的键不匹配，则相应的值为NaN。

In [85]:

from pandas import DataFrame

In [86]:

dic = {

    "姓名":["狗蛋","刘德华","鹿晗","TF","张柏芝","谢霆锋"],

    "语文":[120,130,140,110,110,100],

    "数学":[130,110,100,100,100,120],

    "外语":[120,130,123,125,112,143],

    "综合":[230,240,254,234,100,100] 

}

dic

Out[86]:

{'外语': [120, 130, 123, 125, 112, 143],
 '姓名': ['狗蛋', '刘德华', '鹿晗', 'TF', '张柏芝', '谢霆锋'],
 '数学': [130, 110, 100, 100, 100, 120],
 '综合': [230, 240, 254, 234, 100, 100],
 '语文': [120, 130, 140, 110, 110, 100]}

In [87]:

df = DataFrame(dic,index=list("abcdef")) 

# 如果不指定索引，默认是把当前行的位置序列做为索引

df

Out[87]:

	外语	姓名	数学	综合	语文
a	120	狗蛋	130	230	120
b	130	刘德华	110	240	130
c	123	鹿晗	100	254	140
d	125	TF	100	234	110
e	112	张柏芝	100	100	110
f	143	谢霆锋	120	100	100

In [ ]:

​

DataFrame属性：values、columns、index、shape

In [88]:

df.values # 二维数组

Out[88]:

array([[120, '狗蛋', 130, 230, 120],
       [130, '刘德华', 110, 240, 130],
       [123, '鹿晗', 100, 254, 140],
       [125, 'TF', 100, 234, 110],
       [112, '张柏芝', 100, 100, 110],
       [143, '谢霆锋', 120, 100, 100]], dtype=object)

In [89]:

df.index # index索引对象

Out[89]:

Index(['a', 'b', 'c', 'd', 'e', 'f'], dtype='object')

In [90]:

df.index[0] = "q" # 索引不能单独赋值

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-90-2b136e7ea00f> in <module>()
----> 1 df.index[0] = "q" # 索引不能单独赋值

d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in __setitem__(self, key, value)
   1668 
   1669     def __setitem__(self, key, value):
-> 1670         raise TypeError("Index does not support mutable operations")
   1671 
   1672     def __getitem__(self, key):

TypeError: Index does not support mutable operations

In [91]:

df.index = list("ABCDEF") # index只能整体赋值

In [92]:

df

Out[92]:

	外语	姓名	数学	综合	语文
A	120	狗蛋	130	230	120
B	130	刘德华	110	240	130
C	123	鹿晗	100	254	140
D	125	TF	100	234	110
E	112	张柏芝	100	100	110
F	143	谢霆锋	120	100	100

In [93]:

df.columns  # index对象

Out[93]:

Index(['外语', '姓名', '数学', '综合', '语文'], dtype='object')

In [94]:

df.columns[0] = "python" # 不能单独赋值

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-94-82c06599cb92> in <module>()
----> 1 df.columns[0] = "python" # 不能单独赋值

d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in __setitem__(self, key, value)
   1668 
   1669     def __setitem__(self, key, value):
-> 1670         raise TypeError("Index does not support mutable operations")
   1671 
   1672     def __getitem__(self, key):

TypeError: Index does not support mutable operations

In [95]:

df.columns = ["python","java","h5","ruby","javascript"]

In [96]:

df

Out[96]:

	python	java	h5	ruby	javascript
A	120	狗蛋	130	230	120
B	130	刘德华	110	240	130
C	123	鹿晗	100	254	140
D	125	TF	100	234	110
E	112	张柏芝	100	100	110
F	143	谢霆锋	120	100	100

In [97]:

df1 = DataFrame(dic,columns=["姓名","java","h5","ruby","javascript"])

df1 # 字典创建的时候，如果属性不对应，不对应那些就会缺失

Out[97]:

	姓名	java	h5	ruby	javascript
0	狗蛋	NaN	NaN	NaN	NaN
1	刘德华	NaN	NaN	NaN	NaN
2	鹿晗	NaN	NaN	NaN	NaN
3	TF	NaN	NaN	NaN	NaN
4	张柏芝	NaN	NaN	NaN	NaN
5	谢霆锋	NaN	NaN	NaN	NaN

In [98]:

df2 = DataFrame(dic,columns=["姓名","语文","数学","外语","综合"])

df2

Out[98]:

	姓名	语文	数学	外语	综合
0	狗蛋	120	130	120	230
1	刘德华	130	110	130	240
2	鹿晗	140	100	123	254
3	TF	110	100	125	234
4	张柏芝	110	100	112	100
5	谢霆锋	100	120	143	100

In [99]:

nd = np.random.randint(0,150,size=(4,4))

In [100]:

df3 = DataFrame(nd,index=list("abcd"),columns=list("ABCD"))

df3

Out[100]:

	A	B	C	D
a	84	37	48	114
b	122	31	4	35
c	100	40	73	0
d	138	111	101	110

In [101]:

df2["数学"][0] = 1000

d:\Anaconda\lib\site-packages\ipykernel_launcher.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  """Entry point for launching an IPython kernel.

In [102]:

df2

Out[102]:

	姓名	语文	数学	外语	综合
0	狗蛋	120	1000	120	230
1	刘德华	130	110	130	240
2	鹿晗	140	100	123	254
3	TF	110	100	125	234
4	张柏芝	110	100	112	100
5	谢霆锋	100	120	143	100

In [103]:

dic

Out[103]:

{'外语': [120, 130, 123, 125, 112, 143],
 '姓名': ['狗蛋', '刘德华', '鹿晗', 'TF', '张柏芝', '谢霆锋'],
 '数学': [130, 110, 100, 100, 100, 120],
 '综合': [230, 240, 254, 234, 100, 100],
 '语文': [120, 130, 140, 110, 110, 100]}

In [104]:

df3["A"]["a"] = 2000

df3

Out[104]:

	A	B	C	D
a	2000	37	48	114
b	122	31	4	35
c	100	40	73	0
d	138	111	101	110

In [105]:

nd

Out[105]:

array([[2000,   37,   48,  114],
       [ 122,   31,    4,   35],
       [ 100,   40,   73,    0],
       [ 138,  111,  101,  110]])

【注意】用字典创建是副本拷贝，用数组创建是索引的拷贝

============================================

练习4：

根据以下考试成绩表，创建一个DataFrame，命名为df：

    张三  李四
语文 150  0
数学 150  0
英语 150  0
理综 300  0

============================================

2）DataFrame的索引

(1) 对列进行索引

- 通过类似字典的方式
- 通过属性的方式

可以将DataFrame的列获取为一个Series。返回的Series拥有原DataFrame相同的索引，且name属性也已经设置好了，就是相应的列名。

In [106]:

df

Out[106]:

	python	java	h5	ruby	javascript
A	120	狗蛋	130	230	120
B	130	刘德华	110	240	130
C	123	鹿晗	100	254	140
D	125	TF	100	234	110
E	112	张柏芝	100	100	110
F	143	谢霆锋	120	100	100

In [107]:

# 键值形式

df["java"]

Out[107]:

A     狗蛋
B    刘德华
C     鹿晗
D     TF
E    张柏芝
F    谢霆锋
Name: java, dtype: object

In [108]:

#属性形式

df.java

df2.数学 

Out[108]:

0    1000
1     110
2     100
3     100
4     100
5     120
Name: 数学, dtype: int64

In [109]:

# 索引多列

# df.java.python # 属性不能索引多列

df[["python","java"]] # 索引多累得到一个DataFrame

Out[109]:

	python	java
A	120	狗蛋
B	130	刘德华
C	123	鹿晗
D	125	TF
E	112	张柏芝
F	143	谢霆锋

In [110]:

df[["python"]]

Out[110]:

	python
A	120
B	130
C	123
D	125
E	112
F	143

In [111]:

df["python":"java"] # 对列进行切片没有意义

Out[111]:

(2) 对行进行索引

- 使用.ix[]来进行行索引
- 使用.loc[]加index来进行行索引
- 使用.iloc[]加整数来进行行索引

同样返回一个Series，index为原来的columns。

In [112]:

df

Out[112]:

	python	java	h5	ruby	javascript
A	120	狗蛋	130	230	120
B	130	刘德华	110	240	130
C	123	鹿晗	100	254	140
D	125	TF	100	234	110
E	112	张柏芝	100	100	110
F	143	谢霆锋	120	100	100

In [113]:

df.loc["A"]

Out[113]:

python        120
java           狗蛋
h5            130
ruby          230
javascript    120
Name: A, dtype: object

In [114]:

df.iloc[1]

Out[114]:

python        130
java          刘德华
h5            110
ruby          240
javascript    130
Name: B, dtype: object

In [115]:

# 切片

df.loc["A":"A"]

Out[115]:

	python	java	h5	ruby	javascript
A	120	狗蛋	130	230	120

In [116]:

df.loc[["A","B"]]

Out[116]:

	python	java	h5	ruby	javascript
A	120	狗蛋	130	230	120
B	130	刘德华	110	240	130

In [117]:

df.iloc[0:3]

Out[117]:

	python	java	h5	ruby	javascript
A	120	狗蛋	130	230	120
B	130	刘德华	110	240	130
C	123	鹿晗	100	254	140

In [118]:

df.loc["A","python"]

Out[118]:

120

In [119]:

df.iloc[1,1] # 隐式索引相当于索引数组

Out[119]:

'刘德华'

In [120]:

df["python","A"]  # 不能先列后行

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
   2441             try:
-> 2442                 return self._engine.get_loc(key)
   2443             except KeyError:

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: ('python', 'A')

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-120-d9617f00040f> in <module>()
----> 1 df["python","A"]  # 不能先列后行

d:\Anaconda\lib\site-packages\pandas\core\frame.py in __getitem__(self, key)
   1962             return self._getitem_multilevel(key)
   1963         else:
-> 1964             return self._getitem_column(key)
   1965 
   1966     def _getitem_column(self, key):

d:\Anaconda\lib\site-packages\pandas\core\frame.py in _getitem_column(self, key)
   1969         # get column
   1970         if self.columns.is_unique:
-> 1971             return self._get_item_cache(key)
   1972 
   1973         # duplicate columns & possible reduce dimensionality

d:\Anaconda\lib\site-packages\pandas\core\generic.py in _get_item_cache(self, item)
   1643         res = cache.get(item)
   1644         if res is None:
-> 1645             values = self._data.get(item)
   1646             res = self._box_item_values(item, values)
   1647             cache[item] = res

d:\Anaconda\lib\site-packages\pandas\core\internals.py in get(self, item, fastpath)
   3588 
   3589             if not isnull(item):
-> 3590                 loc = self.items.get_loc(item)
   3591             else:
   3592                 indexer = np.arange(len(self.items))[isnull(self.items)]

d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
   2442                 return self._engine.get_loc(key)
   2443             except KeyError:
-> 2444                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2445 
   2446         indexer = self.get_indexer([key], method=method, tolerance=tolerance)

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: ('python', 'A')

(3) 对元素索引的方法

- 使用列索引
- 使用行索引(iloc[3,1]相当于两个参数;iloc[[3,3]] 里面的[3,3]看做一个参数)
- 使用values属性（二维numpy数组）

In [ ]:

# 两次索引

df["python"]["A"]

In [ ]:

df.loc["A"]["java"]

In [121]:

df.loc["A":"C"]["java"]

Out[121]:

A     狗蛋
B    刘德华
C     鹿晗
Name: java, dtype: object

In [122]:

df["python":"java"]["A"] # 对列切片得到了空

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
   2441             try:
-> 2442                 return self._engine.get_loc(key)
   2443             except KeyError:

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'A'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-122-f6746d39a074> in <module>()
----> 1 df["python":"java"]["A"] # 对列切片得到了空

d:\Anaconda\lib\site-packages\pandas\core\frame.py in __getitem__(self, key)
   1962             return self._getitem_multilevel(key)
   1963         else:
-> 1964             return self._getitem_column(key)
   1965 
   1966     def _getitem_column(self, key):

d:\Anaconda\lib\site-packages\pandas\core\frame.py in _getitem_column(self, key)
   1969         # get column
   1970         if self.columns.is_unique:
-> 1971             return self._get_item_cache(key)
   1972 
   1973         # duplicate columns & possible reduce dimensionality

d:\Anaconda\lib\site-packages\pandas\core\generic.py in _get_item_cache(self, item)
   1643         res = cache.get(item)
   1644         if res is None:
-> 1645             values = self._data.get(item)
   1646             res = self._box_item_values(item, values)
   1647             cache[item] = res

d:\Anaconda\lib\site-packages\pandas\core\internals.py in get(self, item, fastpath)
   3588 
   3589             if not isnull(item):
-> 3590                 loc = self.items.get_loc(item)
   3591             else:
   3592                 indexer = np.arange(len(self.items))[isnull(self.items)]

d:\Anaconda\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
   2442                 return self._engine.get_loc(key)
   2443             except KeyError:
-> 2444                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2445 
   2446         indexer = self.get_indexer([key], method=method, tolerance=tolerance)

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'A'

In [123]:

df.iloc[0:3,4] # 隐式切片的时候第二个也要隐式

Out[123]:

A    120
B    130
C    140
Name: javascript, dtype: int64

In [124]:

df.iloc[0:3]["h5"]

Out[124]:

A    130
B    110
C    100
Name: h5, dtype: int64

【注意】 直接用中括号时：

• 索引表示的是列索引
• 切片表示的是行切片

In [ ]:

​

============================================

练习5：

使用多种方法对ddd进行索引和切片，并比较其中的区别

============================================

3）DataFrame的运算

（1） DataFrame之间的运算

同Series一样：

• 在运算中自动对齐不同索引的数据
• 如果索引不对应，则补NaN

创建DataFrame df1 不同人员的各科目成绩，月考一

In [125]:

df1 = DataFrame(np.random.randint(0,150,size=(4,4)),

                columns=["python","线性代数","微积分","数理统计"],

                index=list("abcd"))

df1

Out[125]:

	python	线性代数	微积分	数理统计
a	101	33	31	87
b	98	69	139	104
c	12	59	40	136
d	8	120	92	91

创建DataFrame df2 不同人员的各科目成绩，月考二
有新学生转入

In [126]:

df2 = DataFrame(np.random.randint(0,150,size=(6,4)),

                columns=["python","java","ruby","数理统计"],

                index=list("abcdef"))

df2

Out[126]:

	python	java	ruby	数理统计
a	47	51	134	116
b	91	27	133	79
c	123	83	0	54
d	50	10	31	143
e	84	117	17	132
f	103	87	62	23

In [127]:

df1 + df2

Out[127]:

	java	python	ruby	微积分	数理统计	线性代数
a	NaN	148.0	NaN	NaN	203.0	NaN
b	NaN	189.0	NaN	NaN	183.0	NaN
c	NaN	135.0	NaN	NaN	190.0	NaN
d	NaN	58.0	NaN	NaN	234.0	NaN
e	NaN	NaN	NaN	NaN	NaN	NaN
f	NaN	NaN	NaN	NaN	NaN	NaN

In [128]:

df1.add(df2,fill_value=0)

Out[128]:

	java	python	ruby	微积分	数理统计	线性代数
a	51.0	148.0	134.0	31.0	203.0	33.0
b	27.0	189.0	133.0	139.0	183.0	69.0
c	83.0	135.0	0.0	40.0	190.0	59.0
d	10.0	58.0	31.0	92.0	234.0	120.0
e	117.0	84.0	17.0	NaN	132.0	NaN
f	87.0	103.0	62.0	NaN	23.0	NaN

In [129]:

df2.add(df1,fill_value=0) # fill_value参数在补用它的值去填补缺失的地方

#（注意）原来既缺行又缺列的仍然补nan

Out[129]:

	java	python	ruby	微积分	数理统计	线性代数
a	51.0	148.0	134.0	31.0	203.0	33.0
b	27.0	189.0	133.0	139.0	183.0	69.0
c	83.0	135.0	0.0	40.0	190.0	59.0
d	10.0	58.0	31.0	92.0	234.0	120.0
e	117.0	84.0	17.0	NaN	132.0	NaN
f	87.0	103.0	62.0	NaN	23.0	NaN

java  数学 python       java  python 数学

a 0 + a 0 b 0 b 0 c 0 c 0 d 0 0 nan d 0

下面是Python 操作符与pandas操作函数的对应表：

Python Operator	Pandas Method(s)
+	add()
-	sub(), subtract()
*	mul(), multiply()
/	truediv(), div(), divide()
//	floordiv()
%	mod()
**	pow()

In [ ]:

​

（2） Series与DataFrame之间的运算

【重要】

• 使用Python操作符：以行为单位操作（参数必须是行），对所有行都有效。（类似于numpy中二维数组与一维数组的运算，但可能出现NaN）

• 使用pandas操作函数：

    axis=0：以列为单位操作（参数必须是列），对所有列都有效。
    axis=1：以行为单位操作（参数必须是行），对所有行都有效。
  

In [130]:

df2

Out[130]:

	python	java	ruby	数理统计
a	47	51	134	116
b	91	27	133	79
c	123	83	0	54
d	50	10	31	143
e	84	117	17	132
f	103	87	62	23

In [131]:

s_row = df2.loc["c"]

s_row

Out[131]:

python    123
java       83
ruby        0
数理统计       54
Name: c, dtype: int32

In [132]:

s_col = df2["python"]

s_col

Out[132]:

a     47
b     91
c    123
d     50
e     84
f    103
Name: python, dtype: int32

In [133]:

df2

Out[133]:

	python	java	ruby	数理统计
a	47	51	134	116
b	91	27	133	79
c	123	83	0	54
d	50	10	31	143
e	84	117	17	132
f	103	87	62	23

In [134]:

df2.add(s_row,axis=1) # 按照列标进行加（把s加到所有的行中）

Out[134]:

	python	java	ruby	数理统计
a	170	134	134	170
b	214	110	133	133
c	246	166	0	108
d	173	93	31	197
e	207	200	17	186
f	226	170	62	77

In [135]:

df2.add(s_col,axis=0) # 按行标来加（把s加到所有的列中）

Out[135]:

	python	java	ruby	数理统计
a	94	98	181	163
b	182	118	224	170
c	246	206	123	177
d	100	60	81	193
e	168	201	101	216
f	206	190	165	126

============================================

练习6：

1. 假设ddd是期中考试成绩，ddd2是期末考试成绩，请自由创建ddd2，并将其与ddd相加，求期中期末平均值。

2. 假设张三期中考试数学被发现作弊，要记为0分，如何实现？

3. 李四因为举报张三作弊立功，期中考试所有科目加100分，如何实现？

4. 后来老师发现有一道题出错了，为了安抚学生情绪，给每位学生每个科目都加10分，如何实现？

============================================

In [136]:

dic = {

    "姓名":["狗蛋","刘德华","鹿晗","TF","张三","李四"],

    "语文":[120,130,140,110,110,100],

    "数学":[130,110,100,100,100,120],

    "外语":[120,130,123,125,112,143],

    "综合":[230,240,254,234,100,100] 

}

ddd = DataFrame(dic,columns=["姓名","语文","数学","外语","综合"])

ddd

Out[136]:

	姓名	语文	数学	外语	综合
0	狗蛋	120	130	120	230
1	刘德华	130	110	130	240
2	鹿晗	140	100	123	254
3	TF	110	100	125	234
4	张三	110	100	112	100
5	李四	100	120	143	100

In [137]:

dic = {

    "姓名":["狗蛋","刘德华","鹿晗","TF","张三","李四"],

    "语文":[121,131,140,110,110,100],

    "数学":[130,110,100,120,100,120],

    "外语":[120,30,123,125,132,143],

    "综合":[230,40,254,34,100,60] 

}

ddd2 = DataFrame(dic,columns=["姓名","语文","数学","外语","综合"])

ddd2

Out[137]:

	姓名	语文	数学	外语	综合
0	狗蛋	121	130	120	230
1	刘德华	131	110	30	40
2	鹿晗	140	100	123	254
3	TF	110	120	125	34
4	张三	110	100	132	100
5	李四	100	120	143	60

In [138]:

d = ddd[["语文","数学","外语","综合"]] + ddd2[["语文","数学","外语","综合"]]

d

Out[138]:

	语文	数学	外语	综合
0	241	260	240	460
1	261	220	160	280
2	280	200	246	508
3	220	220	250	268
4	220	200	244	200
5	200	240	286	160

In [139]:

d = d/2

In [140]:

d["姓名"] = ddd["姓名"] # 添加上姓名这一列

In [141]:

d

Out[141]:

	语文	数学	外语	综合	姓名
0	120.5	130.0	120.0	230.0	狗蛋
1	130.5	110.0	80.0	140.0	刘德华
2	140.0	100.0	123.0	254.0	鹿晗
3	110.0	110.0	125.0	134.0	TF
4	110.0	100.0	122.0	100.0	张三
5	100.0	120.0	143.0	80.0	李四

In [142]:

ddd.loc[4,"数学"] = 0 # 如果要修改不能用二次索引

In [143]:

ddd

Out[143]:

	姓名	语文	数学	外语	综合
0	狗蛋	120	130	120	230
1	刘德华	130	110	130	240
2	鹿晗	140	100	123	254
3	TF	110	100	125	234
4	张三	110	0	112	100
5	李四	100	120	143	100

In [144]:

ddd.loc[5,["语文","数学","外语","综合"]] += 100

In [145]:

ddd

Out[145]:

	姓名	语文	数学	外语	综合
0	狗蛋	120	130	120	230
1	刘德华	130	110	130	240
2	鹿晗	140	100	123	254
3	TF	110	100	125	234
4	张三	110	0	112	100
5	李四	200	220	243	200

In [146]:

ddd[["语文","数学","外语","综合"]] += 10

In [147]:

ddd

Out[147]:

	姓名	语文	数学	外语	综合
0	狗蛋	130	140	130	240
1	刘德华	140	120	140	250
2	鹿晗	150	110	133	264
3	TF	120	110	135	244
4	张三	120	10	122	110
5	李四	210	230	253	210

In [148]:

ddd.iloc[:,1:5] +=10 # 用数组的方法

In [149]:

ddd

Out[149]:

	姓名	语文	数学	外语	综合
0	狗蛋	140	150	140	250
1	刘德华	150	130	150	260
2	鹿晗	160	120	143	274
3	TF	130	120	145	254
4	张三	130	20	132	120
5	李四	220	240	263	220