Selenium各种操作解析
Article Directory
Selenium is essentially an automated testing tool, mainly used to test Web applications. For dynamic web pages (web pages rendered with JavaScript), this crawling method is very effective.
One, install Selenium
Enter the start menu, search for cdm, and pip install selenium after entering
Two, install WebDriver
WebDriver is a W3C specification that defines the API to control the browser. As long as a certain browser implements the WebDriver API, you can use Selenium to control the browser. So install EebDriver to distinguish which browser WebDriver is.
Example: Install ChromeDriver, which is the official download page of ChromeDriver (https://npm.taobao.org/mirrors/chromedriver/).
Chrome is divided into three versions: Windows, Linux, and Mac. Download the version that matches yours. If the version of Chrome is 73.0.3683.86, the version of chromedriver should also be 73.0.3683.86.
After downloading, you get an executable file
You can put chromedriver.exe in the PATH environment variable so that any path can be executed. You can also put chromedriver in any directory and specify its directory at runtime.
Three, use chromedriver
1 from selenium import webdriver
2 browser = webdriver.Chrome('E://python/chromedriver.exe')
3 #设置环境变量可能出错，但放入路径一定无误。但路径需将'\'改为'/'
Fourth, the use of Selenium
(1) Open the browser
(2) Get specific content of the browser
(3) Control the controls on the browser page, for example, enter a string into the text box
(4) Close the browser
The main function:
1 from selenium import webdriver
2 browser = webdriver.Chrome('E://python/chromedriver.exe')
3 #设置环境变量可能出错，但放入路径一定无误。但路径需将'\'改为'/'
4 browser.get('https://www.jd.com/')
5 #打开京东首页
6 input = browser.find_element_by_id('key')
7 #用id的属性值查找搜索框
8 input.send_keys('Python')
9 #用send_keys向搜索框中输入Python文本
10 browser.close()
11 #关闭浏览器
Five, find nodes
1. The method at the beginning of find_element (up to only the first eligible node is returned), such as Xpath, CSS selector, class attribute, id attribute, tag name, etc.
2. The method at the beginning of find_elements returns a list, the usage is the same as the former
Note: These two methods are the first parameter selection search method
Six, node interaction
Selenium can interact with nodes, that is, simulate the actions of the browser. For example, clicking a certain button on the page and entering a certain text in the text input box belong to node interaction
input.click()
Action chain
The previous interactive actions have specific execution objects, and there is another type of interactive actions that have no specific execution objects, such as mouse drag and keyboard buttons. These need to be executed in another way, this is the action chain
1. Simulate mouse movement (move_to_element)
1 action = ActionChains(browser)
2 #创建ActionChains对象
3 input = browser.find_elements_by_class_name('cate_menu_item')
4 for i in input:
5 action.move_to_element(i).perform()
6 #一定要用perfrom方法才能生效
7 time.sleep(1)
2. Drag a node to another node (drag_and_drop)
1 first = browser.find_element_by_class_name('navitems-lk')
2 second = browser.find_element_by_class_name('text')
3 action.drag_and_drop(first,second)
4 #用drap_and_drop方法拖到节点
5 action.perform()
6 #要调用peform方法才有用
8. Execute JavaScript code
For some operations, selenium does not provide corresponding APIs, such as drop-down pages, but you can use selenium's execute_script method to run JavaScript code directly
1 browser.execute_script('window.scrollTo(0,document.body.scrollHeight)')
2 #将页面滚动到最低端
3 browser.execute_async_script('alert("已到达页面低端")')
4 #弹出对话框
Nine, get node information
1 input = browser.find_elements_by_class_name('cate_menu_lk')
2 for i in input:
3 print(i.text)
4 #获取节点文本
5 print(i.id)
6 #输出节点内部使用的id，注意：不是id属性值
7 print(i.location)
8 #输出节点的位置
9 print(i.tag_name)
10 #输出节点名称
11 print(i.size)
12 #输出节点的尺寸
13 browser.close()
10. Manage cookies
Using selenium, you can easily manage cookies, such as obtaining cookies, adding and deleting cookies, etc.
1 print(browser.get_cookies())
2 #获取cookies列表
3 browser.add_cookie({
'name':'name','attrs':'attrs'})
4 #添加新的cookie
5 browser.delete_all_cookies()
6 #删除所有cookie
11. Change the attribute value of the node
Selenium itself does not provide an API to modify node attributes, but node attributes can be set by executing JavaScript code, and nodes obtained through selenium can be used directly as DOM, which means that the found nodes can be used directly in JavaScript code. The first parameter of the execute_script method is used to specify the JavaScript code, and the following variable parameters can pass parameters for the JavaScript code. Obtain the value of each parameter through the arguments variable. For example, arguments[0] represents the first parameter value.
1 a1 = browser.find_element_by_class_name('navitems-lk')
2 a2 = browser.find_element_by_class_name('cate_menu_lk')
3 js = '''
4 arguments[0].text = '标题'
5 arguments[0].href = 'https://i.cnblogs.com/articles/edit'
6 arguments[1].text = '时间'
7 arguments[1].href = 'https://www.hao123.com/'
8 '''
9 browser.execute_script(js,a1,a2)
10 #用js代码改变a1和a2的属性值