[Python crawler] selenium4 new version user guide

Selenium is a tool for web application testing. Selenium tests run directly in the browser, just like real users. Supported browsers include IE（7, 8, 9, 10, 11）, Mozilla Firefox, Safari, Google Chrome, Opera, Edgeetc. The main functions of this tool include: Test compatibility with browsers - test applications to see if they can work well on different browsers and operating systems.

With the development of the times, when using python for automation, there have been great changes from before. These changes are the places where an old programmer must step on the pit. It is also because he has already mastered the old version and is using the new version. When you need to explore many times, this article starts from the latest version of Selenium and takes the Chrome driver as an example to summarize the usage of the new version, so as not to search everywhere and step on pits.

01. Set the driver

Current Selenium version: 4.3.0, please check whether your version is above 4.0

Note:
Please check your browser version, and then download the corresponding driver. Generally speaking, the version number is the same. If you don’t have your version number, then choose the version that is closest to your browser’s version number.

1. selenium4 recommended method

Because the traditional way of setting the driver is too cumbersome, GitHub boss Sergey Pirogov wrote a browser driver manager, which can make setting the driver more convenient.

First, you need to install the manager, use the following command

pip install webdriver-manager

Then import the package

from webdriver_manager.chrome import ChromeDriverManager

Set up the service and get the driver through ChromeDriverManager().install(), which saves the steps of downloading the driver and setting the driver

service = ChromeService(executable_path=ChromeDriverManager().install())

Then the driver loads the service

driver = webdriver.Chrome(service=service)

The complete code is posted here for reference

from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
service = ChromeService(executable_path=ChromeDriverManager().install())
driver = webdriver.Chrome(service=service)
driver.quit()

2. Traditional method

Here we use the traditional method to set the driver of Selenium, that is, import webdriver, and then set the path of the executable program.

First of all, in Selenium4, setting the driver location is no longer directly set in webdriver.Chrome, but introduces Service.

So first import the Service package

from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService

Then set the driver through Service, replace CHROMEDRIVER_PATH with your driver location

service = ChromeService(executable_path=CHROMEDRIVER_PATH)

Finally call this service in webdriver.Chrome

driver = webdriver.Chrome(service=service)

Give the complete code for reference

from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option("useAutomationExtension", False)
service = ChromeService(executable_path=CHROMEDRIVER_PATH)
driver = webdriver.Chrome(service=service, options=options)

 

现在我也找了很多测试的朋友，做了一个分享技术的交流群，共享了很多我们收集的技术文档和视频教程。
如果你不想再体验自学时找不到资源，没人解答问题，坚持几天便放弃的感受
可以加入我们一起交流。而且还有很多在自动化，性能，安全，测试开发等等方面有一定建树的技术大牛
分享他们的经验，还会分享很多直播讲座和技术沙龙
可以免费学习！划重点！开源的！！！
qq群号：110685036

02. Find elements

In selenium4, a series of findElement methods such as findElementByClassName, findElementByIdetc. are integrated into one method - findElement. And By.methodchoose your method of finding elements by, for example, as follows.

If you want to find element based on class name you can use following method

driver.findElement(By.className("className"));

If you want to find elements by css selector, you can use the following method

driver.findElement(By.cssSelector(".className"));

Here are all the search element methods corresponding to the past, before the update

driver.findElementByClassName("className");

driver.findElementByCssSelector(".className");
driver.findElementById("elementId");
driver.findElementByLinkText("linkText");
driver.findElementByName("elementName");
driver.findElementByPartialLinkText("partialText");
driver.findElementByTagName("elementTagName");
driver.findElementByXPath("xPath");

Updated

driver.find_element(By.XPATH,'XPATH')
driver.find_element(By.CLASS_NAME,'CLASS_NAME')
driver.find_element(By.CSS_SELECTOR,'CSS_SELECTOR')
driver.find_element(By.ID,'ID')
driver.find_element(By.LINK_TEXT,'LINK_TEXT')
driver.find_element(By.PARTIAL_LINK_TEXT,'PARTIAL_LINK_TEXT')
driver.find_element(By.TAG_NAME,'TAG_NAME')

If you are looking for multiple elements, just replace find_element with find_elements.

wait for element to appear

Sometimes an element does not appear directly. If you do not make a judgment, it will cause the program to crash. Therefore, exception handling can generally be done. There is also a method to wait for the element to appear.

First, you need to import the waiting package

from selenium.webdriver.support.ui import WebDriverWait

Then use the following method to wait for the element to appear. The driver is the browser driver, and the timeout is the waiting time. After until, it is judged whether the element appears.

el = WebDriverWait(driver, timeout=3).until(lambda d: d.find_element_by_tag_name("p"))

03. Action API

The action API has relatively little information on the Internet, because the previous search for elements, calling click, etc. can solve many problems. When looking through the official documents, I found that selenium also supports the action API to simulate actions.

Action API is divided into four parts, namely 键盘, 鼠标, 笔, 滚轮. Here we first explain the following common parts.

pause

During cursor movement and scrolling, there will be some time gaps, which can be realized by pausing here, which supports chain calls, and the official examples are posted here

clickable = driver.find_element(By.ID, "clickable")
ActionChains(driver)\
        .move_to_element(clickable)\
        .pause(1)\
        .click_and_hold()\
        .pause(1)\
        .send_keys("abc")\
        .perform()

release all actions

When actions are currently being executed, these actions can be stopped using the following methods

ActionBuilder(driver).clear_actions()

keyboard

keyboard code table

If you are interested, you can take a look, the address is here, you can see the complete list.

Press a key, take input shift+abc as an example

    ActionChains(driver)\
        .key_down(Keys.SHIFT)\
        .send_keys("abc")\
        .perform()

Pop up a key, take input shift+a和shift+bas an example

    ActionChains(driver)\
        .key_down(Keys.SHIFT)\
        .send_keys("a")\
        .key_up(Keys.SHIFT)\
        .send_keys("b")\
        .perform()

The browser enters a string of characters (without specifying elements)

    ActionChains(driver)\
        .send_keys("abc")\
        .perform()

Specifies the element input string

    text_input = driver.find_element(By.ID, "textInput")
    ActionChains(driver)\
        .send_keys_to_element(text_input, "abc")\
        .perform()

copy and paste

cmd_ctrl = Keys.COMMAND if sys.platform == 'darwin' else Keys.CONTROL
ActionChains(driver)\
        .send_keys("Selenium!")\
        .send_keys(Keys.ARROW_LEFT)\
        .key_down(Keys.SHIFT)\
        .send_keys(Keys.ARROW_UP)\
        .key_up(Keys.SHIFT)\
        .key_down(cmd_ctrl)\
        .send_keys("xvv")\
        .key_up(cmd_ctrl)\
        .perform()

mouse

On mouse click hold, this method combines moving the mouse to the center of the element with pressing the left mouse button.

This helps to focus specific elements:

    clickable = driver.find_element(By.ID, "clickable")
    ActionChains(driver)\
        .click_and_hold(clickable)\
        .perform()

mouse click release

    clickable = driver.find_element(By.ID, "click")
    ActionChains(driver)\
        .click(clickable)\
        .perform()

5 kinds of buttons defined by the mouse

• 0 - left mouse button
• 1 - middle mouse button
• 2 - Right mouse button
• 3——X1 (Back key)
• 4——X2 (forward key)

right mouse click

    clickable = driver.find_element(By.ID, "clickable")
    ActionChains(driver)\
        .context_click(clickable)\
        .perform()

press mouse button 3

    action = ActionBuilder(driver)
    action.pointer_action.pointer_down(MouseButton.BACK)
    action.pointer_action.pointer_up(MouseButton.BACK)
    action.perform()

press mouse button 4

    action = ActionBuilder(driver)
    action.pointer_action.pointer_down(MouseButton.FORWARD)
    action.pointer_action.pointer_up(MouseButton.FORWARD)
    action.perform()

mouse double click

    clickable = driver.find_element(By.ID, "clickable")
    ActionChains(driver)\
        .double_click(clickable)\
        .perform()

mouse over element

    hoverable = driver.find_element(By.ID, "hover")
    ActionChains(driver)\
        .move_to_element(hoverable)\
        .perform()

mouse displacement

It is to perform displacement operation through pixel points

offset from the top left edge of the element

    mouse_tracker = driver.find_element(By.ID, "mouse-tracker")
    ActionChains(driver)\
        .move_to_element_with_offset(mouse_tracker, 8, 11)\
        .perform()

Displacement from the center of the element (please expect)

Shift from the upper left corner of the current window

    action = ActionBuilder(driver)
    action.pointer_action.move_to_location(8, 12)
    action.perform()

Displace from current mouse position

    ActionChains(driver)\
        .move_by_offset( 13, 15)\
        .perform()

Drag and drop elements

The method first clicks and holds the source element, moves to the location of the target element, and releases the mouse.

    draggable = driver.find_element(By.ID, "draggable")
    droppable = driver.find_element(By.ID, "droppable")
    ActionChains(driver)\
        .drag_and_drop(draggable, droppable)\
        .perform()

drag by displacement

    draggable = driver.find_element(By.ID, "draggable")
    start = draggable.location
    finish = driver.find_element(By.ID, "droppable").location
    ActionChains(driver)\
        .drag_and_drop_by_offset(draggable, finish['x'] - start['x'], finish['y'] - start['y'])\
        .perform()

Pen (valid for some browsers)

Since the pen only works in some browsers, I won’t write it here. If you are interested or have a need, you can go to the official document to view it. The official document address is posted here.

Scroll wheel (only valid for Google kernel browser)

Scroll to an element position

iframe = driver.find_element(By.TAG_NAME, "iframe")

ActionChains(driver)\
    .scroll_to_element(iframe)\
    .perform()

Quantitative scrolling

    footer = driver.find_element(By.TAG_NAME, "footer")
    delta_y = footer.rect['y']
    ActionChains(driver)\
        .scroll_by_amount(0, delta_y)\
        .perform()

Scroll the specified amount from an element

    iframe = driver.find_element(By.TAG_NAME, "iframe")
    scroll_origin = ScrollOrigin.from_element(iframe)
    ActionChains(driver)\
        .scroll_from_origin(scroll_origin, 0, 200)\
        .perform()

scroll from an element, and specify the offset

    footer = driver.find_element(By.TAG_NAME, "footer")
    scroll_origin = ScrollOrigin.from_element(footer, 0, -50)
    ActionChains(driver)\
        .scroll_from_origin(scroll_origin, 0, 200)\
        .perform()

displacement from an element's origin

    ActionChains(driver)\
        .scroll_from_origin(scroll_origin, 0, 200)\
        .perform()

Summarize

The above is all the content of this article, summarizing some common pits in the use of selenium4, and the official recommended way

The following are supporting learning materials. For friends who do [software testing], it should be the most comprehensive and complete preparation warehouse. This warehouse also accompanied me through the most difficult journey. I hope it can help you too!

Software testing interview applet

The software test question bank maxed out by millions of people! ! ! Who is who knows! ! ! The most comprehensive quiz mini program on the whole network, you can use your mobile phone to do the quizzes, on the subway or on the bus, roll it up!

The following interview question sections are covered:

1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux

6. web, app, interface automation, 7. performance testing, 8. programming basics, 9. hr interview questions, 10. open test questions, 11. security testing, 12. computer basics

Information acquisition method: