关于Python BeautifulSoup 爬取网页信息中文乱码 解决方法

爬取中国基金网数据时中文部分出现乱码

原code如下:

url=r'http://data.chinafund.cn/' 

urlString= urlopen(url)

soup= BeautifulSoup(urlString, 'html.parser')

nameList= soup.findAll('div',{'id':'content'})  #print(nameList)

for name in nameList:

    nameString= name.getText(',')  #get raw data

nameString= nameString.replace('--','0')

#'--' means NA on this website. replaced as '0' to easy the next steps like float()

nameString= nameString.splitlines() # split lines by '\r'(the default method)

#print(nameString)

data=[]  #data empty list

for line in nameString:

    lines= line.split(',')  #split text by ','

    data+=[lines[1:5]]  # the 2nd, 3rd, 4th, 5th data in each line

colnames=['Date','Symbol','Fundname','NAV'] # assign col names

dataFrame=pd.DataFrame(data,columns=colnames)

dataFrame=dataFrame[4:len(dataFrame)-2]  #delete the first 4 lines and the last 2 lines which are invalid

##print(dataFrame)

dataFrame['Symbol']= dataFrame['Symbol'].astype(float)  #transfer NAVs to float 

dataFrame['NAV']= dataFrame['NAV'].astype(float)

# transfer Symbol to float(I need to do that to remove first few zeroes for easy work)

filePath= ('D:/MS/Allprice.csv')   #My  filepath 

dataFrame.to_csv(filePath,encoding='utf8',index=False) #encoding=utf-8

如果使用以上代码，最后column['Fundname']下是乱码，即使我在最后已经使用了 encoding=‘utf8’了。为了解决这个问题，我改变了encoding，最后一行代码变为dataFrame.to_csv(filePath,encoding='utf_8_sig',index=False) 。这样跑出来的数据显示是正常的，而且也可以用作后续的读取与计算。

最后结果：