[Automated testing] Selenium's three major waiting methods and manual termination of page loading

There are three ways to wait for elements in selenium, namely mandatory waiting, implicit waiting and display waiting

1. Forced waiting

Simple, rough and easy to use, give me a few seconds to sleep no matter what, it is often used for debugging, but it will affect the speed when actually running production, use it according to the situation

import time
time.sleep(3)

2. Implicit waiting

Set a global implicit waiting time, if it is found within the waiting time, it will execute the next step, if it is not found, it will report an error. Implicit wait only needs to be set once to take effect for all elements. If an element we need has already appeared, we still need to wait for other elements to be loaded before proceeding to the next step. This is its shortcoming. In this case, display waiting is required.

Look at it every once in a while. If the condition is met, execute the next step. If the maximum time set is not met, a timeout exception will be thrown. Display waiting needs to use webdriverwait class, cooperate with the until() and until_not() methods of this class to judge whether to execute the next step according to the conditions, and use it in combination with expected_conditions

from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.jd.com")
driver.implicitly_wait(3)  # 设置隐式等待时间
driver.quit()

3. Explicit wait

from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get("https://www.jd.com")
WebDriverWait(driver, 10,2).until(EC.presence_of_element_located((By.id, 'key')))
#每隔2s判断元素是否出现，2可以不设置，默认为0.5s轮询一次，最长10s，超时未出现则报错

until_not(self, method, message: str = "") function, the effect of until_not is opposite to until, when it returns false, the judgment succeeds, and when it returns true, the judgment fails

The expected_conditions module provides 17 judgment methods

When both implicit wait and explicit wait are set, the longer time shall prevail

2. Manually terminate page loading

The three waiting methods are based on the method that can only be executed after the loading of the webpage is completed and the element is not loaded. There is a situation that the webpage has not been loaded yet, the desired elements have already appeared, and the page can also be operated. At this time, it will not work if you set these waiting methods, because it has to wait for the webpage to load first. For example, in the website below, the input box element and the entire page have already been loaded when it is opened, and can be operated, but because a script is still loading, the script needs to wait for the page to be loaded before it can be obtained, and It takes more than 20 seconds for this page to load

from selenium import webdriver
import time

input = '//*[@id="RA-root"]/div/div[1]/div[1]/div[2]/span/span/span[1]/input1'

driver.get("https://www.geeksforgeeks.org/")

driver.find_element(By.XPATH, input).send_keys("python")
#input_ele = WebDriverWait(driver,10,5).until(EC.presence_of_element_located((By.XPATH, input )))
#driver.implicitly_wait(10)
#time.sleep(10)
#这个时候不管设置哪些等待方式都没用，页面加载完成需要二十多秒，才会执行下一步

In this case, we can manually terminate the page loading after setting the page loading time through set_page_load_timeout(), and continue to the next step

from selenium import webdriver

input = '//*[@id="RA-root"]/div/div[1]/div[1]/div[2]/span/span/span[1]/input'
try:
    driver.set_page_load_timeout(10) #设置页面加载时间，超时没完成则捕获异常
    #driver.set_script_timeout(10)
    driver.get("https://www.geeksforgeeks.org/")
except Exception as e:
    print(e)
    #driver.execute_script('window.stop()') #手动停止页面加载
    print("超时，直接进入下一步")

driver.find_element(By.XPATH, input).send_keys("python")

Note that the exception should be caught. If the program is not caught, a timeout exception will be thrown directly and the program will be terminated.

Extended knowledge points: page loading strategy page loading strategy

The web page attribute document.readyState describes the loading state of the current page. This attribute has three values: loading (loading), interactive (interactive), and complete (completed).
By default, WebDriver will delay the response of driver.get() or the call of driver.navigate().to() until document.readyState is COMPLETE.

In a single-page application (such as Angular, React, Ember), once the dynamic content is loaded (that is, the pageLoadStrategy state is COMPLETE), the behavior of clicking a link or performing some operation within the page will not make a new request to the server, because Content is loaded dynamically on the client side without refreshing the page. A single page application can dynamically load many views without any server request, so the page load strategy will always show a status of COMPLETE until we do a new driver.get() or driver.navigate().to().

The three page loading strategies supported by WebDriver
pageLoadStrategy have three values:

normal: Waiting for the entire page to load, Selenium WebDriver keeps waiting until the load event is returned. By default, if no page load policy is set, normal is set as the initial policy.
eager : Selenium WebDriver keeps waiting until the HTML document is fully loaded and parsed. This strategy has nothing to do with the loading of style sheets, images, and subframes. When set to eager, Selenium WebDriver keeps waiting until the DOMContentLoaded event is returned.
none : Selenium WebDriver only waits until the initial page download is complete.
By default, when Selenium WebDriver loads a page, it follows the normal page loading strategy. It is always recommended that you stop downloading other resources (eg images, css, js) when your page is loading slowly.
How to set page load policy

from selenium import webdriver
options = Options()
options.page_load_strategy = 'normal' # normal eager none 默认normal
driver = webdriver.Chrome(options=options)
driver.get("https://www.baidu.com")
print(driver.capabilities)