原文作者:唐僧不爱八戒
原文地址:https://blog.csdn.net/python36/article/details/83029661
对词库去重
def dict():
f = open(‘sougou.txt’,‘r’)
out=open(“sougou_etl.txt”,“w”)
input = f.readlines()
dict=[]
for i in input:
a,b=i.split(" ")
b=b.strip()
if b not in dict:
dict.append(b)
out.write(str(b)+’\n’)
out.close()
dict()
文章地址:https://blog.csdn.net/leitouguan8655/article/details/82834517