1. The love and hatred between the verification code and the crawler?
Anti-climbing mechanism: Verification code. Identify the data in the verification code picture to simulate the login operation.
2. The operation of identifying the verification code:
- Artificial naked eye recognition. (Not recommended)
- Third-party automatic identification (recommended)
-Super Eagle coding: https://www.chaojiying.com/
The use process of Super Eagle coding platform:
1. Register:
2. Login:
-
Query whether the user has remaining question points
-
Create a software: Super Eagle Home>User Center>Software ID>Generate a Software ID>Enter the software name>Submit (software id and secret key)
-
Download the sample code: Development document>Click here to download the python sample The
sample code is as follows:
#!/usr/bin/env python
# coding:utf-8
import requests
from hashlib import md5
class Chaojiying_Client(object):
def __init__(self, username, password, soft_id):
self.username = username
password = password.encode('utf8')
self.password = md5(password).hexdigest()
self.soft_id = soft_id
self.base_params = {
'user': self.username,
'pass2': self.password,
'softid': self.soft_id,
}
self.headers = {
'Connection': 'Keep-Alive',
'User-Agent': 'Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)',
}
def PostPic(self, im, codetype):
"""
im: 图片字节
codetype: 题目类型 参考 http://www.chaojiying.com/price.html
"""
params = {
'codetype': codetype,
}
params.update(self.base_params)
files = {
'userfile': ('ccc.jpg', im)}
r = requests.post('http://upload.chaojiying.net/Upload/Processing.php', data=params, files=files, headers=self.headers)
return r.json()
def ReportError(self, im_id):
"""
im_id:报错题目的图片ID
"""
params = {
'id': im_id,
}
params.update(self.base_params)
r = requests.post('http://upload.chaojiying.net/Upload/ReportError.php', data=params, headers=self.headers)
return r.json()
if __name__ == '__main__':
chaojiying = Chaojiying_Client('超级鹰用户名', '超级鹰用户名的密码', '96001') #用户中心>>软件ID 生成一个替换 96001
im = open('a.jpg', 'rb').read() #本地图片文件路径 来替换 a.jpg 有时WIN系统须要//
print chaojiying.PostPic(im, 1902) #1902 验证码类型 官方网站>>价格体系 3.4+版 print 后要加()
Just add parentheses () after the last line of print
The parameters required for the last three lines:
Super Eagle Username and Password: Register this website
Software ID: Get it from the user center.
Picture path: Pay attention to relative path and absolute path. A picture a.jpg is attached in the case.
Verification code type: check http://www.chaojiying.com/price.html in the price system
Operation result (data in json format can be known from the program):
Return description:
err_no, (numerical value) return code
err_str, (string) Chinese description of the return information
pic_id, (string) picture identification number, or picture id number
pic_str, (String) the identified result
md5, (string) md5 check value, used to verify whether this piece of data returned is true and valid
Third, actual: identifying ancient poetry network login page code.
The coding process of using the coding platform to identify the verification code:
-Download the verification code image locally
-Call the sample code provided by the platform for image data recognition
import requests
from lxml import etree
from hashlib import md5
# 封装识别验证码图片的函数
def getCodeText(userName, password, appId, imgUrl):
class Chaojiying_Client(object):
def __init__(self, username, password, soft_id):
self.username = username
password = password.encode('utf8')
self.password = md5(password).hexdigest()
self.soft_id = soft_id
self.base_params = {
'user': self.username,
'pass2': self.password,
'softid': self.soft_id,
}
self.headers = {
'Connection': 'Keep-Alive',
'User-Agent': 'Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)',
}
def PostPic(self, im, codetype):
"""
im: 图片字节
codetype: 题目类型 参考 http://www.chaojiying.com/price.html
"""
params = {
'codetype': codetype,
}
params.update(self.base_params)
files = {
'userfile': ('ccc.jpg', im)}
r = requests.post('http://upload.chaojiying.net/Upload/Processing.php', data=params, files=files,
headers=self.headers)
return r.json()
def ReportError(self, im_id):
"""
im_id:报错题目的图片ID
"""
params = {
'id': im_id,
}
params.update(self.base_params)
r = requests.post('http://upload.chaojiying.net/Upload/ReportError.php', data=params, headers=self.headers)
return r.json()
if __name__ == '__main__':
chaojiying = Chaojiying_Client(userName, password, appId) # 用户中心>>软件ID 生成一个替换 96001
im = open(imgUrl, 'rb').read() # 本地图片文件路径 来替换 a.jpg 有时WIN系统须要//
# print(chaojiying.PostPic(im, 1902)) # 1902 验证码类型 官方网站>>价格体系 3.4+版 print 后要加()
return chaojiying.PostPic(im, 1902)
# 将验证码图片下载到本地
url = 'https://so.gushiwen.cn/user/login.aspx?from=http://so.gushiwen.cn/user/collect.aspx'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36'
}
page_text = requests.get(url=url,headers=headers).text
# 解析验证码图片img中src属性值
tree = etree.HTML(page_text)
img_url = tree.xpath('//img[@id="imgCode"]/@src')[0]
img_url = 'https://so.gushiwen.cn/' + img_url
# print(img_url)
img_data = requests.get(url=img_url,headers=headers).content
# 将验证码图片存到本地
with open('./code.jpg','wb') as fp:
fp.write(img_data)
# 调用打码平台的示例程序进行验证码图片数据识别
result = getCodeText('你的超级鹰用户名','你的超级鹰密码', '你的超级鹰appid', '需要识别的验证码图片在本地的路径')
print(result)
