Python Crawler(4)Selenium

Python Crawler(4)Selenium

>pip install selenium

Simple open_url.py
from selenium import webdriver
import os

path2phantom = '/opt/phantomjs/bin/phantomjs'
browser = webdriver.PhantomJS(path2phantom)
browser.get('https://www.walmart.com/search/?grid=false&page=2&query=computer#searchProductResult')

links = browser.find_elements_by_css_selector('a.product-title-link')

count = 0

for link in links:
   count = count + 1
print str(count) + '    ' + link.get_attribute('href')

browser.quit()

>python open_url.py
1    https://www.walmart.com/ip/Dell-Inspiron-15-6-Laptop-AMD-A9-8GB-AMD-Radeon-R5-Graphics-1TB-HD-Red/54527141
2    https://www.walmart.com/ip/Dell-Inspiron-15-6-Laptop-AMD-A9-8GB-AMD-Radeon-R5-Graphics-1TB-HD-Red/54527141
3    https://www.walmart.com/ip/Refurbished-HP-15-f387wm-15-6-Laptop-Touchscreen-Windows-10-Home-AMD-Quad-Core-A8-7410-APU-Processor-4GB-RAM-500GB-Hard-Drive/54476821


References:
http://www.marinamele.com/selenium-tutorial-web-scraping-with-selenium-and-python
https://gxnotes.com/article/52544.html
http://selenium-python.readthedocs.io/locating-elements.html
https://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webdriver.WebDriver.find_element_by_class_name