Python实现统计一篇英文文章内每个单词的出现频率的两种很好解法

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/Jerry_1126/article/details/85404005

有一道Python面试题: 用python实现统计一篇英文文章内每个单词的出现频率，并返回出现频率最高的前10个单词及其出现次数。文件的内容，就拷贝import this模块中的内容，文件名为: this.txt

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

这里想到两种很不错的方法

方法一：常规方法

# coding = utf -8
import re

with open("this.txt", "r", encoding="utf-8") as fd:
    word_list = []     # 存放所有单词，全部小写，并去除,.!等后缀，并去除空格字符串
    word_dict = {}     # 保留{word: count}键值对
    for line in fd.readlines():
        for word in line.strip().split(" "):
            word_list.append(re.sub(r"[.|!|,]", "", word.lower()))
    word_sets = list(set(word_list))   # 确保唯一
    word_dict = {word: word_list.count(word) for word in word_sets if word}
result = sorted(word_dict.items(), key=lambda d: d[1], reverse=True)[:10]
print(result)

备注: 遍历文件，用word_list保留所有的单词,用word_sets保存唯一的单词，方便word_dict来作为键。最后对字典排序，取出前10个，非常巧妙.

方法二： 借助collections模块

# coding = utf -8
import re
from collections import Counter

with open("this.txt", "r", encoding="utf-8") as fd:
    texts = fd.read()                         # 将文件的内容全部读取成一个字符串
    count = Counter(re.split(r"\W+", texts))  # 以单词为分隔

result = count.most_common(10)                # 统计最常使用的前10个
print(result)

备注: 上面的一种方法可谓非常简单，借助collections的Counter.most_common方法来求出出现频率最大的前10位

最后输出:

[('is', 10), ('better', 8), ('than', 8), ('to', 5), ('the', 5), ('of', 3), ('Although', 3), ('never', 3), ('be', 3), ('one', 3)]