[python crawler] using selenium and Chrome browser for automated web search and browsing

Open a portal : https://www.cnblogs.com/liangxuehui/p/5797185.html

Thanks for sharing, the original text is attached below.

Function introduction: Use selenium and Chrome browsers to automatically open Baidu pages, and set it to display 50 items per page, then enter selenium in Baidu's search box to query. Then select "Selenium - Open Source Chinese Community" in the opened page, and open the page
Knowledge introduction:
The role of selenium:
1). Originally used for website automation testing, and in recent years, used to obtain accurate website snapshots.
2). It can run directly on the browser, let the browser automatically load the page, obtain the required data, and take screenshots of the page, or determine whether certain actions on the website occur.
Project steps:
1. Install the chromedriver driver when using Google's Chrome browser.
download link:
After downloading, it is best to put it together with your python file for easy subsequent calls.
2. Install selenium (if already installed, please ignore this step)
Windows users install directly with the pip install selenium command.
3. Code display:
copy code
from selenium import webdriver
from time import sleep

#The back is your browser driver location, remember to add r'' in front, 'r' is the 
driver to prevent character escape = webdriver.Chrome(r ' C:\ Python34 \chromedriver_x64.exe ' )
 #Open Baidu page with get 
driver.get( " http://www.baidu.com " )
 #Find the "Settings" option on the page and click 
driver.find_elements_by_link_text( ' Settings ' )[0].click() #Open
 the settings and find the "Search" "Settings" option, set to display 50 per page 
driver.find_elements_by_link_text( ' Search settings ' )[0].click()
sleep(2)
m = driver.find_element_by_id('nr')
sleep(2)
m.find_element_by_xpath('//*[@id="nr"]/option[3]').click()
sleep( 2 )
 #Process the pop-up warning page 
driver.find_element_by_class_name( " prefpanelgo " ).click()
sleep(2)
driver.switch_to_alert().accept()
sleep( 2 )
 #Find Baidu's input box and enter "selenium" 
driver.find_element_by_id( ' kw ' ).send_keys( ' selenium ' )
sleep( 2 )
 #Click the search button 
driver.find_element_by_id( ' su ' ).click()
sleep( 2 )
 #Find "Selenium - Open Source Chinese Community" in the opened page, and open this page 
driver.find_elements_by_link_text( ' Selenium - Open Source Chinese Community ' )[0].click()
copy code

 4. The following page operations are completed automatically

 

 

 


Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=326180121&siteId=291194637