The most complete network, summary of selenium automated testing, the road to advanced testing...

Contents: Guide

foreword

Start the browser with selenium

You can use the following code to start a Chrome browser in python, and then control the browser's behavior or read data.

from selenium import webdriver

# 启动Chrome浏览器，要求chromedriver驱动程序已经配置到环境变量
# 将驱动程序和当前脚本放在同一个文件夹也可以
driver = webdriver.Chrome()

# 手动指定驱动程序路径
driver = webdriver.Chrome(r'D:/python/tools/chromedriver.exe')

driver = webdriver.Ie()        # Internet Explorer浏览器
driver = webdriver.Edge()      # Edge浏览器
driver = webdriver.Opera()     # Opera浏览器
driver = webdriver.PhantomJS()   # PhantomJS

driver.get('http://xxxxxx.com')  # 打开指定路径的页面

You can also set startup parameters during startup. For example, the following code implements adding a proxy at startup and ignores the https certificate verification.

from selenium import webdriver

# 创建chrome启动选项对象
options = webdriver.ChromeOptions()


options.add_argument("--proxy-server=127.0.0.1:16666")  # 设置代理
options.add_argument("---ignore-certificate-errors")  # 设置忽略https证书校验
options.add_experimental_option("excludeSwitches", ["enable-logging"])  # 启用日志

# 设置浏览器下载文件时保存的默认路径
prefs = {
    
    "download.default_directory": get_download_dir()}
options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(options=options)

Some very useful startup options, options = webdriver.ChromeOptions() used below:

options.add_argument("--proxy-server=127.0.0.1:16666")	#设置代理，可以结合mitmproxy进行抓包等
option.add_experimental_option('excludeSwitches', ['enable-automation'])	#设置绕过selenium检测
options.add_argument("---ignore-certificate-errors")	#设置忽略https证书校验
options.add_experimental_option("prefs", {
    
    "profile.managed_default_content_settings.images": 2})	#设置不请求图片模式加快页面加载速度
chrome_options.add_argument('--headless')	#设置无头浏览器

selenium page load wait and detection

After using selenium to open the page, it cannot be operated immediately. It needs to wait until the loading of the pending page elements is completed. At this time, it is necessary to detect and wait for the loading of the page elements.

Use time.sleep() to wait:

The easiest way is to use time.sleep() to wait for a certain period of time after opening the page. This method can only set a fixed time to wait. If the page is loaded in advance, it will be blocked in vain.

from time import sleep
from selenium import webdriver

driver = webdriver.Chrome()
driver.get('www://baidu.com')
time.sleep(10)
print('load finish')

Use implicitly_wait to set the maximum waiting time:

In addition, you can use implicitly_wait to set the maximum waiting time. If the page is loaded within a given time or has timed out, the next step will be executed.

This method will wait until all resources are loaded, that is, the loading chart in the browser tab bar does not turn before executing the next step. It is possible that the page elements have been loaded, but resources such as js or pictures have not been loaded yet, and you need to wait at this time.

It should also be noted that using implicitly_wait only needs to be set once, and it works for the entire driver life cycle. Whenever a page is loading, it will be blocked.

from selenium import webdriver

driver = webdriver.Chrome()
driver.implicitly_wait(30)   # 设置最长等30秒
driver.get('http://baidu.com')
print(driver.current_url)

driver.get('http://baidu.com')
print(driver.current_url)

Use WebDriverWait to set the wait condition:

Using WebDriverWait (selenium.webdriver.support.wait.WebDriverWait) can set the waiting time more accurately and flexibly. WebDriverWait can check whether a certain condition is met at intervals within the set time. If the condition is met, the next step will be performed. If the set time is not met, TimeoutException will be thrown. The method statement is as follows:

WebDriverWait(driver, timeout, poll_frequency=0.5, ignored_exceptions=None)

The meanings of the parameters are as follows:

driver: browser driver
timeout: the longest timeout time, the default is in seconds
poll_frequency: the detection interval (step) time, the default is 0.5 seconds
ignored_exceptions: ignored exceptions, even if a given exception is thrown during the call to until() or until_not()

WebDriverWait() is generally used with the until() or until_not() method, which means waiting to block until the return value is True or False. It should be noted that the parameters of these two methods must be callable objects, that is, the method name. You can use the methods in the expected_conditions module or the methods encapsulated by yourself.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions

driver = webdriver.Chrome()
driver.get("http://baidu.com")

# 判断id为`input`的元素是否被加到了dom树里，并不代表该元素一定可见，如果定位到就返回WebElement
element = WebDriverWait(driver, 5, 0.5).until(expected_conditions.presence_of_element_located((By.ID, "s_btn_wr")))

# implicitly_wait和WebDriverWait都设置时，取二者中最大的等待时间
driver.implicitly_wait(5)

# 判断某个元素是否被添加到了dom里并且可见，可见代表元素可显示且宽和高都大于0
WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.ID, 'su')))

# 判断元素是否可见，如果可见就返回这个元素
WebDriverWait(driver,10).until(EC.visibility_of(driver.find_element(by=By.ID, value='kw')))

Selenium element positioning and reading

Finding elements
Selenium provides a series of APIs to facilitate access to elements in chrome. These APIs all return WebElement objects or their lists, such as:

find_element_by_id(id): Find the first element matching id
find_element_by_class_name(): Find the first element matching class
find_elements_by_xpath(): Find all elements matching xpath
find_elements_by_css_selector(): Find all elements matching css selector

In fact, you can look at the implementation source code in the WebDriver class. Its core implementation calls two basic functions:

find_element(self, by=By.ID, value=None): Find the first element of the matching strategy
find_elements(self, by=By.ID, value=None): Find all elements of the matching strategy
where the by parameter can be ID, CSS_SELECTOR, CLASS_NAME, XPATH, etc.

Here are a few simple examples:

Query the first element containing the text login via xpath:

find_element_by_xpath("//*[contains(text(),'登录')]")

Query all elements containing the class name refresh:

find_elements_by_class_name('refresh')

Query the second row of the table table:

find_element_by_css_selector('table tbody > tr:nth(2)')

Dom element interaction
The element search result described above is a WebElement object. The commonly used APIs are:

element.text: Return the text content of the element (including the content of all descendant nodes), note that if the element display=none, it will return an empty string element.screenshot_as_png: Screenshot of the element element.send_keys(“input”): Enter the input string in the element input box element.get_attribute('data-v'): Get the data-v name attribute value. In addition to custom node attributes, you can also get attributes such as textContent ele ment.is_displayed(): Determine whether the element is visible to the user element.clear(): Clear the element text element.click(): Click the element, if the element is not clickable, an ElementNotInteractableException will be thrown
element.submit
(
)
:
Simulate
form
submission

Find element failure handling:

If the specified element cannot be found, NoSuchElementException will be thrown, and it should be noted that the element with display=none can be obtained, and all elements in the dom node can be obtained.

And when you actually use it, you should pay attention to some elements dynamically created by js code, which may need to be polled or monitored.
A method that checks for the existence of a specified element is as follows:

def check_element_exists(xpath):
    try:
        driver.find_element_by_xpath(xpath)
    except NoSuchElementException:
        return False
    return True

selenium interactive control

ActionChains action chain:
webdriver simulates user operations through the ActionChains object, which represents an action chain queue, and all operations will enter the queue in turn but will not be executed immediately until the perform() method is called. Its commonly used methods are as follows:

click(on_element=None): 单击鼠标左键
click_and_hold(on_element=None): 点击鼠标左键，不松开
context_click(on_element=None): 点击鼠标右键
double_click(on_element=None): 双击鼠标左键
send_keys(*keys_to_send): 发送某个键到当前焦点的元素
send_keys_to_element(element, *keys_to_send): 发送某个键到指定元素
key_down(value, element=None): 按下某个键盘上的键
key_up(value, element=None): 松开某个键
drag_and_drop(source, target): 拖拽到某个元素然后松开
drag_and_drop_by_offset(source, xoffset, yoffset): 拖拽到某个坐标然后松开
move_by_offset(xoffset, yoffset): 鼠标从当前位置移动到某个坐标
move_to_element(to_element): 鼠标移动到某个元素
move_to_element_with_offset(to_element, xoffset, yoffset): 移动到距某个元素（左上角坐标）多少距离的位置
perform(): 执行链中的所有动作
release(on_element=None): 在某个元素位置松开鼠标左键

Simulate mouse events:

The following code simulates mouse movement, click, drag and other operations. Note that you need to wait for a certain period of time during the operation, otherwise the page will not have time to render.

from time import sleep
from selenium import webdriver
# 引入 ActionChains 类
from selenium.webdriver.common.action_chains import ActionChains

driver = webdriver.Chrome()
driver.get("https://www.baidu.cn")
action_chains = ActionChains(driver)

target = driver.find_element_by_link_text("搜索")
# 移动鼠标到指定元素然后点击
action_chains.move_to_element(target).click(target).perform()
time.sleep(2)

# 也可以直接调用元素的点击方法
target.click()
time.sleep(2)

# 鼠标移动到(10, 50)坐标处
action_chains.move_by_offset(10, 50).perform()
time.sleep(2)

# 鼠标移动到距离元素target(10, 50)处
action_chains.move_to_element_with_offset(target, 10, 50).perform()
time.sleep(2)

# 鼠标拖拽，将一个元素拖动到另一个元素
dragger = driver.find_element_by_id('dragger')
action.drag_and_drop(dragger, target).perform()
time.sleep(2)

# 也可以使用点击 -> 移动来实现拖拽
action.click_and_hold(dragger).release(target).perform()
time.sleep(2)
action.click_and_hold(dragger).move_to_element(target).release().perform()

Simulate keyboard input events:

Simulate keyboard events through send_keys, commonly used are:

send_keys(Keys.BACK_SPACE): 删除键（BackSpace）
send_keys(Keys.SPACE): 空格键(Space)
send_keys(Keys.TAB): 制表键(Tab)
send_keys(Keys.ESCAPE): 回退键（Esc）
send_keys(Keys.ENTER): 回车键（Enter）
send_keys(Keys.F1): 键盘 F1
send_keys(Keys.CONTROL,'a'): 全选（Ctrl+A）
send_keys(Keys.CONTROL,'c'): 复制（Ctrl+C）
send_keys(Keys.CONTROL,'x'): 剪切（Ctrl+X）
send_keys(Keys.CONTROL,'v'): 粘贴（Ctrl+V）

Example: Locate the input box and enter content

# 输入框输入内容
driver.find_element_by_id("kw").send_keys("NBA")

# 模拟回车删除多输入的一个字符a
driver.find_element_by_id("kw").send_keys(Keys.BACK_SPACE)

Alert box handling:

It is used to handle the alert box popped up by calling alert.
driver.switch_to_alert(): switch to the alert box
text: return the text information in alert/confirm/prompt, for example, js calling alert('failed') will get the failed string accept(): accept the
existing alert box
dismiss(): close the existing alert box
send_keys(keysToSend): send the text to the alert box

selenium browser control

Basic common api

Some very useful browser control APIs are listed below:

driver.current_url: 获取当前活动窗口的url
driver.switch_to_window("windowName"): 移动到指定的标签窗口
driver.switch_to_frame("frameName"): 移动到指定名称的iframe
driver.switch_to_default_content(): 移动到默认文本内容区
driver.maximize_window(): 将浏览器最大化显示
driver.set_window_size(480, 800): 设置浏览器宽480、高800显示
driver.forword(), driver.back(): 浏览器前进和后退
driver.refresh(): 刷新页面
driver.close(): 关闭当前标签页
driver.quiit(): 关闭整个浏览器
driver.save_screenshot('screen.png'): 保存页面截图
driver.maximize_window(): 将浏览器最大化显示
browser.execute_script('return document.readyState;'): 执行js脚本

Selenium reads and loads cookies:

Use get_cookies and add_cookie to cache cookies locally, and then load them at startup, so that the login status can be preserved.
The implementation is as follows:

import os
import json
from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.baidu.cn")

# 读取所有cookie并保存到文件
cookies = driver.get_cookies()
cookie_save_path = 'cookie.json'
with open(cookie_save_path, 'w', encoding='utf-8') as file_handle:
    json.dump(cookies, file_handle, ensure_ascii=False, indent=4)

# 从文件读取cookie并加载到浏览器
with open(cookie_save_path, 'r', encoding='utf-8') as file_handle:
    cookies = json.load(file_handle)
    for cookie in cookies:
        driver.add_cookie(cookie)

Selenium opens a new tab window:

Using driver.get(url) will open the specified connection in the first tab window by default, and a new tab window will also be opened when clicking the _blank link in the page.

You can also use the following method to manually open a tab window of a specified page. Note that after opening a new window or closing it, you need to manually call switch_to.window to switch the currently active tab window, otherwise NoSuchWindowException will be thrown.

from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.baidu.cn")

new_tab_url = 'https://www.baidu.c'
driver.execute_script(f'window.open("{
      
      new_tab_url}", "_blank");')
time.sleep(1)

# 注意：必须调用switch_to.window手动切换window，否则会找不到tab view
# 聚焦到新打开的tab页面，然后关闭
driver.switch_to.window(driver.window_handles[1])
time.sleep(2)
driver.close()   # 关闭当前窗口

# 手动回到原来的tab页面
driver.switch_to.window(driver.window_handles[0])
time.sleep(1)

In addition to using execute_script, you can also create a new tab window by simulating the button to open a new tab page:

driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + 't')
ActionChains(driver).key_down(Keys.CONTROL).send_keys('t').key_up(Keys.CONTROL).perform()

Browser crash WebDriverException exception handling:

For example, if a page runs for a long time in Chrome, an Out Of Memory error will occur. At this time, WebDriver will throw a WebDriverException. Basically all APIs will throw this exception. At this time, it needs to be caught and handled specially.

The processing method is to record some basic information of the page, such as url, cookie, etc., and write it to the file regularly. If the abnormality is detected, restart the browser and load data such as url and cookie.

The following is the most complete software test engineer learning knowledge architecture system diagram in 2023 that I compiled