Selenium module: You can open the browser, and then operate the browser like a human. Programmers can directly extract various information of web pages from selenium.
Environment installation
1.pip install selenium
2. Download the browser driver: https://npm.taobao.org/mirrors/chromedriver
Put the decompressed browser driver chromedriver in the folder where the python interpreter is located
Let selenium start google chrome
from selenium.webdriver import Chrome
# 1.创建浏览器对象
web = Chrome()
# 2.打开网址
web.get("XXXXX")
selenium syntax
1. Find an element and simulate the process of clicking on the element
el = web.find_element_by_xpath("XXXX")
el.click()
2. Simulate in the input box, enter python, and then enter Enter
from selenium.webdriver.common.keys import keys
web.find_element_by_xpath('XXXXX').send_keys("python",Keys.ENTER)
3. Find the location where the data is stored and extract the data
li_list = web.find_elements_by_xpath("XXXXX")
for li in li_list:
name = li.find_element_by_tag_name("XXXXX").text
Selenium window switching
el = web.find_element_by_xpath("XXXX")
el.click() #产生了新窗口
#在selenium视角中新窗口默认不切换
web.switch_to.window(web.window_handles[-1])
#关掉子窗口
web.close()
web.switch_to.window(web.window_handles[0])
#变更selenium的窗口视角，回到原来窗口中
handle iframes
To deal with iframe, you must get the iframe first, then switch the viewing angle to the iframe, and then get the data
iframe = web.find_element_by_xpath("XXXXX")
web.switch_to.frame(iframe)
web.switch_to.default_content() #切换回元页面
Selenium handles dropdown list
from selenium.webdriver.support.select import Select
#定位到下拉列表
sel_el = web.find_element_by_xpath("XXX")
#对元素进行包装，包装成下拉菜单
sel = Select(sel_el)
#让浏览器进行调整选项
for i in range(len(sel.options)):
sel.select_by_index(i)
headless browser
A headless browser refers to using selenium to operate without automatically popping up web pages.
from selenium.webdriver.chrome.option import Options
opt = Options()
opt.add_argument("--headless")
opt.add_argument("--disable-gpu")
web = Chrome(options=opt)
Selenium event chain processing picture verification code
from selenium.webdriver.common.cation_chains import ActionChains
ActionChains(web).move_to_element_with_offset(verify_img_element,x,y).click().perform()