[Python crawler framework] Getting started with Selenium library: How to use Python to automate web page testing?

Are you still troubled by webpage testing? Are you still tired of repeated clicking and waiting? Try Mighty Selenium! Make your web page automation testing easy and fun!

1. What exactly is the Selenium library?

Selenium is a powerful automated testing tool that allows you to directly manipulate the browser to complete various tasks that interact with web pages. By using Python's Selenium library, you can efficiently automate web page testing, saving a lot of time and effort.

1.1 The main functions of the Selenium library

  • Automated testing: It can realize the automated functional testing of the website, such as clicking buttons, entering text, selecting drop-down menus, etc.

  • Web crawler: For some websites that require login or have an anti-crawling mechanism, Selenium can be used to easily deal with them.

  • Automated tasks: such as regularly visiting websites, automatically submitting forms, etc.

1.2 Advantages of Selenium library

  1. Supports multiple programming languages: Python, Java, C#, Ruby, etc.

  2. Supports multiple browsers: Chrome, Firefox, Edge, Safari, etc.

  3. Cross-platform: Windows, macOS, Linux.

  4. The community is active and continuously updated and maintained.

Second, the installation and configuration of the Selenium library

2.1 Install the Selenium library

Before installing the Selenium library, make sure you have Python installed. Next, simply open a terminal or command prompt and enter the following command to install the Selenium library:

pip install selenium

2.2 Configure browser driver

To use Selenium to control the browser, you also need to install the corresponding browser driver. The Chrome browser is used as an example here, and the installation methods for other browser drivers are similar.

  1. Download Chrome Driver: Search ChromeDriver and select the driver that matches your Chrome browser version on the download page.

  2. Decompress the downloaded compressed package, and place the decompressed  chromedriver file in an executable path, such as  /usr/local/bin(macOS, Linux) or  C:\Windows(Windows).

3. Use the Selenium library for basic operations

3.1 Start the browser

Import the Selenium  webdriver module, and then instantiate a Chrome driver object to start the Chrome browser:

from selenium import webdriver

driver = webdriver.Chrome()

3.2 Open the web page

In  get the method of use, pass in the target URL to let the browser open the URL:

driver.get("https://www.baidu.com")

3.3 Positioning page elements

Selenium provides a variety of methods to locate page elements, such as  find_element_by_id, find_element_by_name, find_element_by_class_name and so on. Here, take the Baidu search box as an example, use  find_element_by_id the method to locate:

search_box = driver.find_element_by_id("kw")

3.4 Operating page elements

After finding the page element, you can perform various operations on it, such as entering text, clicking buttons, etc. Here is an example of entering search keywords:

search_box.send_keys("Selenium")

3.5 Submit the form

After entering keywords in the search box, you also need to click the "Baidu click" button to submit the form. Here,  find_element_by_id the method is used to locate the button, and then  click the method is used to click:

search_button = driver.find_element_by_id("su")
search_button.click()

3.6 Waiting for the page to load

When doing automated testing, it is often necessary to wait for the page to load. Selenium provides two wait methods: explicit wait and implicit wait.

3.6.1 Explicit waits

An explicit wait will keep trying to find the target element for the specified time until it is found. Here is an example of waiting for the search results to appear:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

try:
    element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "content_left"))
    )
except TimeoutException:
    print("等待超时!")

3.6.2 Implicit waits

Implicit wait waits for a specified amount of time before throwing an exception while looking for an element. Here is an example of waiting for the search results to appear:

driver.implicitly_wait(10)
content_left = driver.find_element_by_id("content_left")

3.7 Get page information

Get the title, URL, source code and other information of the page:

title = driver.title
url = driver.current_url
source = driver.page_source

print("标题:", title)
print("URL:", url)

3.8 Closing the browser

Don't forget to close your browser when everything is done:

driver.quit()

4. Advanced application of Selenium library

4.1 Switch windows

When doing automated testing, it is sometimes necessary to switch to a newly opened window. Here is an example of clicking the "About Baidu" link at the bottom of Baidu's homepage to demonstrate how to switch windows:

from selenium.webdriver.common.keys import Keys

about_link = driver.find_element_by_link_text("关于百度")
about_link.send_keys(Keys.CONTROL + Keys.RETURN)

driver.switch_to.window(driver.window_handles[-1])

4.2 Switch Frame

Some webpages use Frame to nest pages. In this case, you need to switch to the corresponding Frame to operate the elements. You can use  switch_to.frame the method to switch Frame:

driver.switch_to.frame("frame_name")

To switch back to the main page, you can use  switch_to.default_content the following methods:

driver.switch_to.default_content()

4.3 Execute JavaScript code

Sometimes, you may need to execute some JavaScript codes to achieve specific functions, such as scrolling pages, modifying element attributes, and so on. Here is an example of scrolling the page:

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

5. Technical summary

Through the introduction of this article, I believe you have mastered the basic knowledge of Selenium library, including automated testing, browser driver, page interaction, etc. Now, you can easily automate web page testing with Python + Selenium!

Do you have any thoughts or questions about this article? Welcome to leave a message to discuss in the comment area!

Finally:  The complete software testing video learning tutorial below has been sorted out and uploaded, and friends can get it for free if they need it【保证100%免费】

insert image description here

 These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey. I hope it can help you too!

Guess you like

Origin blog.csdn.net/m0_75277660/article/details/130589581