机器学习之数据预处理，Pandas读取excel数据

Python读写excel的工具库很多，比如最耳熟能详的xlrd、xlwt，xlutils，openpyxl等。其中xlrd和xlwt库通常配合使用，一个用于读，一个用于写excel。xlutils结合xlrd可以达到修改excel文件目的。openpyxl可以对excel文件同时进行读写操作。

而说到数据预处理，pandas就体现除了它的强大之处，并且它还支持可读写多种文档格式，其中就包括对excel的读写。本文重点就是介绍pandas对excel数据集的预处理。

机器学习常用的模型对数据输入都是有要求的，多数机器学习算法最基本的要求是训练数据要转换成数值格式。当然，也有像决策树算法这种不需要转换为数值的算法，这里不做特例讨论。

pandas读取excel文件的函数是pandas.read_excel()，主要参数包括：

io : 读取的excel文档地址，

string, path object (pathlib.Path or py._path.local.LocalPath),

file-like object, pandas ExcelFile, or xlrd workbook. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. For instance, a local file could be file://localhost/path/to/workbook.xlsx

sheet_name : 读取的excel指定的sheet页

string, int, mixed list of strings/ints, or None, default 0

Strings are used for sheet names, Integers are used in zero-indexed sheet positions.

Lists of strings/integers are used to request multiple sheets.

Specify None to get all sheets.

str|int -> DataFrame is returned. list|None -> Dict of DataFrames is returned, with keys representing sheets.

Available Cases

Defaults to 0 -> 1st sheet as a DataFrame

1 -> 2nd sheet as a DataFrame

“Sheet1” -> 1st sheet as a DataFrame

[0,1,”Sheet5”] -> 1st, 2nd & 5th sheet as a dictionary of DataFrames

None -> All sheets as a dictionary of DataFrames

header : 设置读取的excel第一行是否作为列名称

int, list of ints, default 0

Row (0-indexed) to use for the column labels of the parsed DataFrame. If a list of integers is passed those row positions will be combined into a MultiIndex. Use None if there is no header.

names :设置每列的名称，数组形式参数

　　　array-like, default None

List of column names to use. If file contains no header row, then you should explicitly pass header=None

index_col :设置读取的excel第一列是否作为行名称

　　　int, list of ints, default None

Column (0-indexed) to use as the row labels of the DataFrame. Pass None if there is no such column. If a list is passed, those columns will be combined into a MultiIndex. If a subset of data is selected with usecols, index_col is based on the subset.

usecols :执行需要读取的数据列，通常载入的excel包含不需要的列

　　　　int or list, default None

If None then parse all columns,

If int then indicates last column to be parsed

If list of ints then indicates list of column numbers to be parsed

If string then indicates comma separated list of Excel column letters and column ranges (e.g. “A:E” or “A,C,E:F”). Ranges are inclusive of both sides.

下满是一些pandas读取excel数据的示例：

将数据集写入excel文件：

 
  >>> df_out = pd.DataFrame([('string1', 1), ... ('string2', 2), ... ('string3', 3)], ... columns=['Name', 'Value']) >>> df_out  Name Value 0 string1 1 1 string2 2 2 string3 3 >>> df_out.to_excel('tmp.xlsx')  
 

读取excel文件：

 
  >>> pd.read_excel('tmp.xlsx')  Name Value 0 string1 1 1 string2 2 2 string3 3 
 

参数index_col and header 都设置为None表示不读取excel的第一行和第一列作为标题和默认索引：

 
  >>> pd.read_excel('tmp.xlsx', index_col=None, header=None)  0 1 2 0 NaN Name Value 1 0.0 string1 1 2 1.0 string2 2 3 2.0 string3 3  
 

甚至可以专门制定列的格式：

 
  >>> pd.read_excel('tmp.xlsx', dtype={'Name':str, 'Value':float})  Name Value 0 string1 1.0 1 string2 2.0 2 string3 3.0  
 

下面是综合示例：读取text.xlsx文件的sheet1页，仅载入D:F列的数据。这里F列是类别标签，需要类别1和类别2转换为数字，应用于机器学习的输入建模。

import pandas as pd

def reader(path,sheet):
    return pd.read_excel(path, sheet_name=sheet, usecols='D:F')
    
trainrd = reader('text.xlsx','sheet1')
trainrd.head(5)  #查看前5行数据
trainrd['x']=0  #新建一列x
trainrd.loc[trainrd['类别']=='类别1','x']=0 #将类别列的文字转换为数字
trainrd.loc[trainrd['类别']=='类别2','x']=1

机器学习之数据预处理，Pandas读取excel数据

猜你喜欢