samuelbrody1249 :
Is there a listing or library that has all punctuations that we might commonly come across?
Normally I use string.punctuation, but some punctuation characters are not included in it, for example:
>>> "'" in string.punctuation
True
>>> "’" in string.punctuation
False
wim :
You might do better with this check:
>>> import unicodedata
>>> unicodedata.category("'").startswith("P")
True
>>> unicodedata.category("’").startswith("P")
True
The Unicode categories P* are specifically for Punctuation:
connector (Pc), dash (Pd), initial quote (Pi), final quote (Pf), open (Ps), close (Pe), other (Po)
To prepare the exhaustive collection, which you can subsequently use for fast membership checks, use a set comprehension:
>>> import sys
>>> from unicodedata import category
>>> codepoints = range(sys.maxunicode + 1)
>>> punctuation = {c for i in codepoints if category(c := chr(i)).startswith("P")}
>>> "'" in punctuation
True
>>> "’" in punctuation
True
Assignment expression here requires Python 3.8+, equivalent for older Python versions:
chrs = (chr(i) for i in range(sys.maxunicode + 1))
punctuation = set(c for c in chrs if category(c).startswith("P"))
Beware that some of the other characters in string.punctuation are actually in Unicode category Symbol. It's easy to add those in also if you want.
Guess you like
Origin http://10.200.1.11:23101/article/api/json?id=401423&siteId=1