python 模仿点击事件爬取数据

1. 首先要安装 selenium

2.   options = webdriver.ChromeOptions()
           options.set_headless()
           options.add_argument('--disable - gpu')
           driver = webdriver.Chrome(options=options)
           driver.get(url)
           driver.switch_to.frame(0)
           # driver.find_element_by_xpath("//*[@id='root']/div/div[1]/div/div/div/button[4]").click()
           true_page = driver.page_source
           # print(driver.page_source)
           html = HtmlResponse(url=self.start_urls[0], body=true_page, encoding='utf-8')

   获取到了html下面就可以写爬取了

3.   content = html.selector.xpath("//*[@id='root']/div/div[1]/div/div/div/button[2]/span[1]/text()").extract()[0]

   有什么问题可以入群 602640939 一起交流