贝叶斯
贝叶斯公式:
理论部分知识我这里就不啰嗦了,直接上代码:
使用贝叶斯算法实现一个拼写检查器:
import re, collections
# 正则匹配
def words(text):
return re.findall('[a-z]+', text.lower())
def train(features):
model = collections.defaultdict(lambda: 1)
for f in features:
model[f] += 1
return model
# 读取数据集
NWORDS = train(words(open('big.txt').read()))
alphabet = 'abcdefghijklmnopqrstuvwxyz'
def edits1(word):
n = len(word)
return set([word[0:i]+word[i+1:] for i in range(n)] + # deletion
[word[0:i]+word[i+1]+word[i]+word[i+2:] for i in range(n-1)] + # transposition
[word[0:i]+c+word[i+1:] for i in range(n) for c in alphabet] + # alteration
[word[0:i]+c+word[i:] for i in range(n+1) for c in alphabet]) # insertion
def known_edits2(word):
return set(e2 for e1 in edits1(word) for e2 in edits1(e1) if e2 in NWORDS)
def known(words):
return set(w for w in words if w in NWORDS)
def correct(word):
candidates = known([word]) or known(edits1(word)) or known_edits2(word) or [word]
return max(candidates, key=lambda w: NWORDS[w])
#appl #appla #learw #tess #morw
这里调用correct()方法验证,传入的参数:morw
输出:‘more’
换个传入的参数: ‘learw’
结果输出:‘learn’
小结:
比如用户输入tlp ,那到底是 top 还是 tip ?这个时候,当最大似然不能作出决定性的判断时,先验概率就可以插手进来给出指示——“既然你无法决定,那么我告诉你,一般来说 top 出现的程度要高许多,所以更可能他想打的是 top ”