[Python crawler 01] Simple understanding of Selenium

In modern web development, automated testing is an integral part. And Selenium is a powerful Python library for automating the operation and testing of web browsers. This blog post will introduce you to the basic knowledge and usage of Selenium, and illustrate how to conduct web automation testing through cases.

What is Selenium?

Selenium is an open source automated testing framework that is primarily used to simulate user interaction with a web browser. It can perform operations in different browsers, such as clicking buttons, filling forms, extracting data, etc., thereby realizing automated web testing.

Selenium supports several programming languages, of which Python is the most commonly used one. Python provides a wealth of libraries and tools that make web automation testing with Selenium easier and more efficient.

Install Selenium

Before we start, we need to install the Selenium library. It can be installed through the pip command:

pip install selenium

In addition, you need to download the corresponding browser driver, such as ChromeDriver or GeckoDriver, so that Selenium can interact with the browser. According to the type and version of the browser used, download the corresponding driver and add it to the system path.

How to download and install chromedriver

Web Automation Testing with Selenium

1. Import the necessary libraries

First, we need to import the Selenium library along with other necessary libraries:

from selenium import webdriver

2. Create a browser driver object

Next, we need to create a browser driver object for controlling and operating the browser. Take Chrome browser as an example:

driver = webdriver.Chrome()

3. Open the web page

Open the desired web page using the drive object:

driver.get("https://www.example.com")

4. Find elements and operations

There are different ways to find and manipulate elements on a web page. For example, to locate elements by ID, class name, or XPath:

element = driver.find_element(By.ID,"element_id")
element.click()

5. Fill out the form and submit

For cases where you need to fill out a form, you can use send_keys()the method to enter text, and submit()the method to submit the form:

input_field = driver.find_element(By.ID,"input_field_id")
input_field.send_keys("Hello, World!")
input_field.submit()

6. Close the browser

Remember to close your browser when you're done testing:

driver.quit()

The above is the basic flow of a simple Selenium test. Through steps such as finding elements, manipulating forms, and submitting, you can implement various automated web tests.

Example: Automatically log in to a website

let's go through a case

An example to demonstrate how to use Selenium to automatically log in to a website.

from selenium import webdriver
from selenium.webdriver.common.by import By
import time
# 创建浏览器驱动对象
driver = webdriver.Chrome()
driver.maximize_window() # 窗口最大化

# 打开登录页面
driver.get("https://portrait.gitee.com/login")

# 填写用户名和密码
try:
    username_field = driver.find_element(By.NAME, 'user[login]')
    username_field.send_keys("这里输入你的账号")
    password_field = driver.find_element(By.NAME, 'user[password]')
    password_field.send_keys("这里输入你的密码")

    remember_me_field = driver.find_element(By.XPATH, '//*[@id="new_user"]/div/div/div/div[3]/div[1]/div/label')
    remember_me_field.click()
except Exception as e:
    print(f"[+] error:{
      
      e[:20]}")
    
time.sleep(2)
# 登录
commit_field = driver.find_element(By.NAME, 'commit').click()

time.sleep(10)
# 执行其他操作，如点击链接、提取数据等

# 关闭浏览器
driver.quit()

Through the above code, you can simulate the user logging in to the website, enter the user name and password, and click Login.

Demo video:

gitee login automatic login demo

Summary:
This blog post introduces the basics and methods of using Selenium for Web automation testing. You learned how to install Selenium libraries, create browser-driven objects, and perform various operations to simulate user behavior. Through the demonstration of specific cases, you can better understand and apply the role of Selenium in Web automation testing. I hope this blog post can help you get started with Selenium and improve your automated testing skills.