Selenium configuration
First find out what selenium does. This stuff is actually used for automated web testing. It is to simulate human behaviors like mouse and keyboard operations. The difference from the previous use of requests, etc. is that this thing is a dynamically acquired behavior. It is not simply a one-time acquisition of the source code of the webpage, but it can be continuously acquired later. The simplest application in the crawler is to verify by moving the slider.
To download selenium, you can directly use pip to download the installation instructions.
Configure the browser and download the corresponding webdriver version.
Take chrome as an example
. 1. View the chrome version
2. Download the chromedriver
download URL
3. Put the downloaded chromedriver.exe into the file.
There are two options; 1. Place and Python script directory
Or placed in another designated directory (in English path)
Verify successful installation
from selenium import webdriver
drive=webdriver.Chrome()
#drive=webdriver.Chrome(executable_path=指定路径)
drive.get('https://www.baidu.com/')
pop-up window
Basic operation
The first thing to declare is that some methods of selenium are very long at first glance, but they are actually simple duck skins.
1. Open a web page
2. Position an element
3. Implement simple click operations
open the Web page
drive.get('https://www.baidu.com/')
Positioning element
There are more operating methods for positioning elements.
The most important thing is to first divide into two categories, one is to obtain an object, and the other is to obtain multiple objects.
such as;
InputTag=drive.find_element_by_name('wd')
Get one (first one)
InputTag=drive.find_elements_by_name('wd')
Get all
In the code, _by_name() means to locate by the value of name.
There are many other methods. As shown in the figure,
if necessary, locate directly through xpath syntax
Object operation
It should be noted here that we only obtained the corresponding elements through positioning,
so the only operations we can perform are click, send and other operations. For more complicated operations, only the mouse can be used. For example; single click, double click, right click, drag. . .
send_keys() send the corresponding value
click() click
clear() to clear
Example operation (demo)
Goal enter python on the Baidu page and click search
Now we already know the specific element location.
The following are respectively used by name positioning and xpath positioning
from selenium import webdriver
drive=webdriver.Chrome()
drive.get('https://www.baidu.com/')
############
InputTag=drive.find_element_by_name('wd')
InputTag.send_keys('python')
############
SubmitBut=drive.find_element_by_xpath('//input[@type="submit" and @value="百度一下"]')
SubmitBut.click()
The effect is as follows
Shorthand
Using the code like the above is more verbose, this can be considered to reduce the writing, and this writing will have a small surprise later.
The difference; it's nothing, just use another method to wrap it
1. Import
from selenium.webdriver.common.by import By
2. Reduce the code
from selenium import webdriver
from selenium.webdriver.common.by import By
drive=webdriver.Chrome()
drive.get('https://www.baidu.com/')
############
InputTag=drive.find_element(By.NAME,'wd')
InputTag.send_keys('python')
############
SubmitBut=drive.find_element(By.XPATH,'//input[@type="submit" and @value="百度一下"]')
SubmitBut.click()
Page close and browser
1. Just close the previous page
drive.close()
2Close the browser
drive.quit()