from selenium import webdriver
from scrapy.selector import Selector
from time import sleep
chrome_opt = webdriver.ChromeOptions();
prefs ={"profile.managed_default_content_settings.images":2}
chrome_opt.add_experimental_option("prefs",prefs)
browser = webdriver.Chrome(executable_path="D:/Program Files/selenium_brower/chromedriver.exe",chrome_options=chrome_opt)
browser.get("https://www.taobao.com")
profile.managed_default_content_settings.images means do not load the pictures in the web page
In the windows environment, pay attention to the path must use / this
browser.find_element_by_css_selector(".Login-content input[name='username']").send_keys("13460688542")
browser.find_element_by_css_selector(".Login-content input[name='password']").send_keys("3989441qwe")
browser.find_element_by_css_selector(".Button.SignFlow-submitButton.Button--primary.Button--blue").click()
You can select the corresponding css to get the element, send_keys assigns the element
js = "window.scrollTo(0,document.body.scrollHeight);var lenOfpage=document.body.scrollHeight;return lenOfpage " browser.execute_script(js)
Scroll down scrollbar in js
Selenium support for PhantomJS has been deprecated,
please use headless versions of Chrome or Firefox instead
PhantomJS and Chrome officially recommend using Chrome for browsing
t_selector = Selector(text=browser.page_source)
t_selector.css(".tm-promo-price .tm-price::text").extract()
Parsing CSS with scrapy feels great
http://selenium-python.readthedocs.io/installation.html
The official talk is the most detailed, the others are secondary