5-n-matching the acquired data

4, the regular match - data acquisition

https://docs.python.org/zh-cn/3/library/re.html

A regular expression is a logical formula of string operations, some combinations of certain pre-defined characters, and these particular character, composed of a "regular character", the "rules of character" A filtering logic to express the character

Commonly used pattern matching

\w      匹配字母数字及下划线
\W      匹配f非字母数字下划线
\s      匹配任意空白字符，等价于[\t\n\r\f]
\S      匹配任意非空字符
\d      匹配任意数字
\D      匹配任意非数字
\A      匹配字符串开始
\Z      匹配字符串结束，如果存在换行，只匹配换行前的结束字符串
\z      匹配字符串结束
\G      匹配最后匹配完成的位置
\n      匹配一个换行符
\t      匹配一个制表符
^       匹配字符串的开头
$       匹配字符串的末尾
.       匹配任意字符，除了换行符，re.DOTALL标记被指定时，则可以匹配包括换行符的任意字符
[....]  用来表示一组字符，单独列出：[amk]匹配a,m或k
[^...]  不在[]中的字符：[^abc]匹配除了a,b,c之外的字符
*       匹配0个或多个的表达式
+       匹配1个或者多个的表达式
?       匹配0个或1个由前面的正则表达式定义的片段，非贪婪方式
{n}     精确匹配n前面的表示
{m,m}   匹配n到m次由前面的正则表达式定义片段，贪婪模式
a|b     匹配a或者b
()      匹配括号内的表达式，也表示一个组

re.match()

Try to match a pattern from a starting position of the string, if not matched, then the starting position, match () returns None
syntax:
re.match (pattern, String, the flags = 0)

import re

content= "hello 123 4567 World_This is a regex Demo"
result = re.match('^hello\s\d\d\d\s\d{4}\s\w{10}.*Demo$',content)
print(result)
print(result.group())
print(result.span())

result.group () Gets the matching results
result.span () is eligible to match the length of the string

Pan match

import re

content= "hello 123 4567 World_This is a regex Demo"
result = re.match("^hello.*Demo$",content)
print(result)
print(result.group())
print(result.span())

Match goal

If the string to match specific target is required by () enclosed

import re
content= "hello 1234567 World_This is a regex Demo"
result = re.match('^hello\s(\d+)\sWorld.*Demo$',content)
print(result)
print(result.group())
print(result.group(1))
print(result.span())

After obtaining the results re.group (), if the regular expression in parentheses, then re.group (1) obtained is the result of the first match in parentheses

Greed match

import re

content= "hello 1234567 World_This is a regex Demo"
result= re.match('^hello.*(\d+).*Demo',content)
print(result)
print(result.group(1))

The reason only match to 7, and there is no match to 1234567, this happens in front of. * To match out. * Will match here much content as possible, what we call the greedy match.

#如果我们想要匹配到1234567则需要将正则表达式改为：

result= re.match('^he.*?(\d+).*Demo',content)

Match mode

Many times the content is matching wrap problem of existence, this time we need to use pattern matching to match the content re.S wrap

import re


content = """hello 123456 world_this
my name is zhaofan
"""

result =re.match('^he.*?(\d+).*?zhaofan$',content,re.S)
print(result)
print(result.group())
print(result.group(1))

Escape

import re

content= "price is $5.00"

result = re.match('price is \$5\.00',content)
print(result)
print(result.group())

A summary of the above:
to make use of pan matching parentheses is matched target, try to use non-greedy, there is a newline re.S
emphasized re.match match from a starting position of a character string pattern

re.search

re.search scan the entire string returns the result of a successful match

import re

content = "extra things hello 123455 world_this is a Re Extra things"

result = re.search("hello.*?(\d+).*?Re",content)
print(result)
print(result.group())
print(result.group(1))

This time we do not need to write ^ and $, because the search is to scan the whole string
Note: So in order to facilitate the match, we will be more of a search, do not match, match must match head, so many times it is not particularly convenient

Matching exercise

import re

html = '''<div id="songs-list">
    <h2 class="title">经典老歌</h2>
    <p class="introduction">
        经典老歌列表
    </p>
    <ul id="list" class="list-group">
        <li data-view="2">一路上有你</li>
        <li data-view="7">
            <a href="/2.mp3" singer="任贤齐">沧海一声笑</a>
        </li>
        <li data-view="4" class="active">
            <a href="/3.mp3" singer="齐秦">往事随风</a>
        </li>
        <li data-view="6"><a href="/4.mp3" singer="beyond">光辉岁月</a></li>
        <li data-view="5"><a href="/5.mp3" singer="陈慧琳">记事本</a></li>
        <li data-view="5">
            <a href="/6.mp3" singer="邓丽君">但愿人长久</a>
        </li>
    </ul>
</div>'''

result = re.search('<li.*?active.*?singer="(.*?)">(.*?)</a>',html,re.S)
print(result)
print(result.groups())
print(result.group(1))
print(result.group(2))

re.findall

Search string, returns substrings of all matches in the form of a list

import re

html = '''<div id="songs-list">
    <h2 class="title">经典老歌</h2>
    <p class="introduction">
        经典老歌列表
    </p>
    <ul id="list" class="list-group">
        <li data-view="2">一路上有你</li>
        <li data-view="7">
            <a href="/2.mp3" singer="任贤齐">沧海一声笑</a>
        </li>
        <li data-view="4" class="active">
            <a href="/3.mp3" singer="齐秦">往事随风</a>
        </li>
        <li data-view="6"><a href="/4.mp3" singer="beyond">光辉岁月</a></li>
        <li data-view="5"><a href="/5.mp3" singer="陈慧琳">记事本</a></li>
        <li data-view="5">
            <a href="/6.mp3" singer="邓丽君">但愿人长久</a>
        </li>
    </ul>
</div>'''

results = re.findall('<li.*?href="(.*?)".*?singer="(.*?)">(.*?)</a>', html, re.S)
print(results)
print(type(results))
for result in results:
    print(result)
    print(result[0], result[1], result[2])

Example 2:

import re

html = '''<div id="songs-list">
    <h2 class="title">经典老歌</h2>
    <p class="introduction">
        经典老歌列表
    </p>
    <ul id="list" class="list-group">
        <li data-view="2">一路上有你</li>
        <li data-view="7">
            <a href="/2.mp3" singer="任贤齐">沧海一声笑</a>
        </li>
        <li data-view="4" class="active">
            <a href="/3.mp3" singer="齐秦">往事随风</a>
        </li>
        <li data-view="6"><a href="/4.mp3" singer="beyond">光辉岁月</a></li>
        <li data-view="5"><a href="/5.mp3" singer="陈慧琳">记事本</a></li>
        <li data-view="5">
            <a href="/6.mp3" singer="邓丽君">但愿人长久</a>
        </li>
    </ul>
</div>'''

results = re.findall('<li.*?>\s*?(<a.*?>)?(\w+)(</a>)?\s*?</li>',html,re.S)
print(results)
for result in results:
    print(result[1])

to sum up:

\s*? 这种用法其实就是为了解决有的有换行，有的没有换行的问题

(<a.*?>)? 这种用法是因为html中有的有a标签，有的没有的，？表示匹配一个或0个，正好可以用于匹配

re.sub

import re

content = "Extra things hello 123455 World_this is a regex Demo extra things"

content = re.sub('\d+','',content)
print(content)

Example 2:
In some cases we replaced the character, we would like to get a string of matches, then add something behind

import re

content = "Extra things hello 123455 World_this is a regex Demo extra things"

content = re.sub('(\d+)',r'\1 7890',content)
print(content)

#\1是获取第一个匹配的结果，为了防止转义字符的问题，我们需要在前面加上r

re.compile

The regular expression compiled into a regular expression object, to facilitate reuse of the regular expression

import re
content= """hello 12345 world_this
fan
"""

pattern =re.compile("hello.*fan",re.S)

result = re.match(pattern,content)
print(result)
print(result.group())

experiment:

import requests
import re
html = requests.get("http://www.iphai.com/free/wg")
find_tr = re.compile('<tr>(.*?)</tr>', re.S)
trs = find_tr.findall(html.text)  #需要获取的是网页的文本string   trs为列表
print(trs[2])  #出现不规则的一行
protocol = re.compile('<td>\s+HTTPS\s+</td>', re.S)
pp = protocol.findall(trs[3]) #匹配的是具有HTTPS的一行
['<td>\n                            HTTPS                        </td>']

protocol = re.compile('<td>\s+(HTTPS)\s+</td>', re.S)  #仅仅匹配分组,获取分组
pp = protocol.findall(trs[3])  
print(pp)
['HTTPS']