Python读取大文件的方法

1. 使用生成器读取（指定读取长度）

def read_file(file_obj, size=1024):
    while True:
        data = file_obj.read(size)
        yield data
        if not data:
            break


def deal_data(data):
    pass


with open('z.txt', 'r', encoding='utf-8') as f:
    for data in read_file(f, 2048):
        deal_data(data)

2. 逐行迭代读取

def deal_data(data):
    pass


with open('z.txt', 'r', encoding='utf-8') as f:
    for line in f:
        deal_data(line.strip('\n'))