Table of contents
1. Client and driver (environment) installation
2. Simple example (Python automation development)
Browser homepage showing antivirus reset settings
Selenium is a program automation operation solution for a set of Web sites.
Through it, we can write automated programs and operate the web interface in the browser like a human. For example, click the interface button, enter text in the text box and other operations.
But also from the web interface to get information. For example, obtain train and bus ticket information, job information on recruitment websites, stock price information on financial websites, etc., and then use programs to analyze and process them.
Selenium's automation principle is like this
As can be seen from the figure above:
The automation programs we write require the use of client libraries .
The automation requests of our program are sent to the browser through the programming interface in this library.
For example, if we want to simulate the user clicking the interface button, the corresponding function of the client library should be called in the automation program, and a request to click on the element will be sent to the lower browser driver . Then, the browser driver forwards the request to the browser.
The requests that this automated program sends to the browser driver are HTTP requests.
Where did the client library come from? It is provided by the Selenium organization.
The Selenium organization provides Selenium client libraries in a variety of programming languages, including java, python, js, ruby, etc., for developers of different programming languages to use.
We only need to install the client libraries, call these libraries, and then send automated requests to the browser.
The browser driver is also an independent program provided by the browser manufacturer, and different browsers require different browser drivers. For example, Chrome browser and Firefox browser have different drivers.
After the browser driver receives the interface operation request sent by our automation program, it will forward the request to the browser and let the browser perform the corresponding automation operation.
After the browser completes the operation, it will return the automation result to the browser driver, and the browser driver will return it to the client library of our automation program through the HTTP response message.
After receiving the response, the automation program's client library converts the result into 数据对象a code that is returned to us.
Our program can know the result of this automated operation.
Let's summarize again, the selenium automation process is as follows:
-
Automated program calls Selenium client library functions (such as clicking button elements)
-
The client library sends Selenium commands to the browser driver
-
After the browser driver receives the command, it drives the browser to execute the command
-
The browser executes the command
-
The browser driver gets the result of the command execution and returns it to our automation program
-
The automated program processes the returned results
1. Client and driver (environment) installation
The installation of the Selenium environment is mainly to install two things: 客户端库and 浏览器 驱动.
1. Install the client library
Different programming languages choose different Selenium client libraries.
Corresponding to our Python language, the installation of the Selenium client library is very simple, just use the pip command.
Open the command line program and run the following command
pip install selenium
If it cannot be installed, it may be a network problem, you can specify to use the domestic Douban source
pip install selenium -i https://pypi.douban.com/simple/2. Install browser and driver
The browser driver corresponds to the browser. Different browsers need to choose different browser drivers.
Among the current mainstream browsers, Google Chrome browser has more mature support for Selenium automation.
It is recommended that you use the Chrome browser.
You can click here to download and install Google Chrome
After ensuring that the Chrome browser is installed, please open the link below to visit the driver download page of the Chrome browser
Chrome browser driver download address
Note that the browser driver must match the browser version. The version number in the red circle in the figure below corresponds to the browser version number.
For example: if the current Chrome browser version is 98, you usually need to download the driver in the directory starting with 98.
Note: The closer the version numbers of the driver and browser are, the better, but there is usually no problem if there is a slight difference, such as 98 and 97.
Check the chrome browser version:
Find the corresponding version
Open the directory, there are 3 zip packages in it, corresponding to Linux, Mac, and Windows platforms.
If we are a computer on Windows platform, download chromedriver_win32.zip
This is a zip package. After downloading, unzip the program file chromedriver.exe in it to a certain directory. Note that the path of this directory should preferably have no Chinese names and spaces.
For example, extract it to d:\toolsthe directory below.
That is to ensure that our Chrome browser drive path isD:\tools\Chrome_plugins_driver\chrome_driver\chromedriver.exe
Some friends may not be able to open the above URL, we provide a newer driver for you to download on the following Baidu network disk
Link: https://pan.baidu.com/s/1qMzJ1n-KknqaHHd5rhDfEA Extraction code: 1111
If you choose Microsoft Edge browser, click here to download the driver
These are the constructions of the web automation environment based on selenium, which is relatively simple.
2. Simple example (Python automation development)
Click here to watch the video explanation and learn the following content
The following code can automatically open the Chrome browser, and automatically open the Baidu website, you can run it and have a look.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
# 创建 WebDriver 对象，指明使用chrome浏览器驱动
wd = webdriver.Chrome(service=Service(r'd:\tools\chromedriver.exe'))
# 调用WebDriver 对象的get方法 可以让浏览器打开指定网址
wd.get('https://www.baidu.com')
Among them, the following line of code will run the browser driver and run the Chrome browser
wd = webdriver.Chrome(service=Service(r'd:\tools\chromedriver.exe'))
Note that what is returned on the right side of the equal sign is an object of type WebDriver, through which we can manipulate the browser, such as opening URLs, selecting interface elements, and so on.
And the following line of code is to use the get method of WebDriver to open the URL Baidu
wd.get('https://www.baidu.com')
When the above line of code is executed, the automation program initiates the opening of the Baidu website 请求消息 , and sends it to the Chrome browser through the browser driver.
After the Chome browser receives the request, it will open the Baidu website, and through the browser driver, tell the automation program to open successfully.
After executing the automation code, if you want to close the browser window, you can call the quit method of the WebDriver object, like this wd.quit()
omit browser driver path
Earlier, when our code created the WebDriver object, we needed to specify the browser driver path, such as
from selenium.webdriver.chrome.service import Service
wd = webdriver.Chrome(service=Service(r'd:\tools\chromedriver.exe'))
There are several problems with writing this way:
One is that it is more troublesome, and the path must be specified every time the automation code is written.
Second, if your code is run by someone else, the path where the browser driver is stored on his computer may not be the same as yours (for example, his computer is an Apple Mac computer), and the script must be changed.
What's the best way?
We can add the browser driver 所在目录 to the environment variable Path . When writing code, there is no need to specify the browser driver path, as follows
wd = webdriver.Chrome()
Because, Selenium will automatically look for a file named chromedriver.exe in those directories specified by the environment variable Path.
It must be noted that, to add the environment variable Path,
Not the full path of the browser driver, such as D:\tools\Chrome_plugins_driver\chrome_driver\chromedriver.exe
It is the directory where the browser driver is located , such as D:\tools\Chrome_plugins_driver\chrome_driver
And after setting the environment variables, don't forget to restart the IDE (such as PyCharm) for the new environment variables to take effect.
Then, how selenium automatically clicks, enters, and obtains information on web pages will be learned in the next chapters.
3. Frequently asked questions
close chromedriver log
By default, after chromedriver is started, it will output a lot of log information on the screen, as follows
DevTools listening on ws://127.0.0.1:19727/devtools/browser/c19306ca-e512-4f5f-b9c7-f13aec506ab7
[21564:14044:0228/160456.334:ERROR:device_event_log_impl.cc(211)] [16:04:56.333] Bluetooth: bluetooth_adapter_winrt.cc:1072 Getting Default Adapter failed.
can be closed like this
from selenium import webdriver
# 加上参数，禁止 chromedriver 日志写屏
options = webdriver.ChromeOptions()
options.add_experimental_option(
'excludeSwitches', ['enable-logging'])
wd = webdriver.Chrome(options=options # 这里指定 options 参数
)
Browser homepage showing antivirus reset settings
When Selenium is automated on some friends' computers, the browser starts to display the following
It can be solved like this:
-
Command line input
regedit, run Registry Editor -
In the directory tree on the left find
HKEY_CURRENT_USER\Software\Google\Chrome -
TriggeredResetdelete subkeys under it -
close regedit
expand knowledge
The interface between the browser and the driver is private to each browser vendor, and we usually don't need to care about it.
Friends who like to get to the bottom of it can refer to this link to learn about the interface between Chrome browser and Chrome