When programming meets AI, enjoy silky smoothness

Table of contents

foreword

1. Putting forward demands

 2. Check the code

3. Further requirements

Summarize 

---

foreword

Since CHATGPT became popular, I found that people around me are no longer afraid of writing reports. Personal summaries, report materials, and annual summaries can be reached at any time (no one reads any reports anyway), except for some highly targeted articles. (Here we are talking about pertinence, not technicality.) Basically, artificial intelligence can say one, two, three. In the past, it used to copy a lot of articles in the world. It depends on whether you can copy or not. Now it depends on whether you can ask or not. As long as The question is accurate, the answer should be accurate. Next, according to a crawler note I took before, we used artificial intelligence to find the answer. At that time, there were still some small mistakes. Let's see if artificial intelligence can solve it.

1. Putting forward demands

In "Programming Xiaobai's Self-study Notes 12 (Python Crawler Introduction 4 Selenium Usage Example 2)", we found the advertisement subpage of Baidu Tieba, and then clicked on the first advertisement. Let's try to ask:

Please use python code to write a crawler to find the advertisement subpage of Baidu Tieba and click on the first advertisement

 

 We found that it seems that the keyword crawler cannot be mentioned, so we modify the question: please write the code with python code to find the advertisement subpage of Baidu Tieba and click on the first advertisement. This time it's ok

 

 2. Check the code

 Let's take a look at the full answer of artificial intelligence:

Here is a sample code of Python+Selenium:

from selenium import webdriver

# 打开chrome浏览器
driver = webdriver.Chrome()
# 打开百度贴吧
driver.get('https://tieba.baidu.com/')
# 等待页面加载完成
driver.implicitly_wait(10)
# 找到广告iframe并切换到它
iframe = driver.find_element_by_xpath('//iframe[@class="iframe_wrapper"]')
driver.switch_to.frame(iframe)
# 找到第一个广告并点击
ad = driver.find_element_by_xpath('//div[@class="ad-list"]//a')
ad.click()

This code will first open the Chrome browser and enter the homepage of Baidu Tieba. It then waits for the page to finish loading, finds the ad's iframe and switches to it. Next, it finds the first ad and clicks on it. Note that this code assumes that the first ad is in a ad-listdiv with a class name. If the actual situation is different, the xpath expression needs to be modified.

 We check the code, there is no logical problem, and the comments are very detailed, I can't do this, the only possible problem is that the xpath may be wrong, in line with the principle of rigor, let's run a test try it.

Not surprisingly, there was an accident, and an error was reported. 

 

 AttributeError: 'WebDriver' object has no attribute 'find_element_by_xpath', this error is due to the version, the new Selenium no longer uses this format, let's try to modify the code slightly:

from selenium import webdriver
from selenium.webdriver.common.by import By

# 打开chrome浏览器
driver = webdriver.Chrome()
# 打开百度贴吧
driver.get('https://tieba.baidu.com/')
# 等待页面加载完成
driver.implicitly_wait(10)
# 找到广告iframe并切换到它
iframe = driver.find_element(By.XPATH,'//iframe[@class="iframe_wrapper"]')
driver.switch_to.frame(iframe)
# 找到第一个广告并点击
ad = driver.find_element(By.XPATH,'//div[@class="ad-list"]//a')
ad.click()

It can be seen that we imported a new method, changed some syntax, and made an error after running

 This time, the XPATH should be wrong. Regardless, it can be seen that the code generated by artificial intelligence still needs to be continuously improved.

3. Further requirements

 If we find that the answer given by artificial intelligence is not what we need, or we are not satisfied with the answer, we can continue to raise the demand, such as the above example, I think it is not well written, we can let it answer again, click here Just regenerate.

 

 Here is the regenerated answer:

 For this question, I need to understand your specific needs and situation first. If you are using selenium to simulate browser behavior, you can follow the steps below:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# 初始化浏览器
driver = webdriver.Chrome()
driver.maximize_window()  # 最大化浏览器窗口

# 打开百度贴吧
driver.get("https://tieba.baidu.com/")

# 等待广告出现
wait = WebDriverWait(driver, 10)
ad_element = wait.until(EC.presence_of_element_located((By.CLASS_NAME, 'ad_item')))

# 点击第一个广告
ad_element.click()

# 关闭浏览器
driver.quit()

In the above code, we use the WebDriverWait class of the selenium library to wait for the advertisement button element ( ad_item) on the page, and use click()the method to click on the first advertisement. Finally, we quit()close the browser using the method.

It should be noted that if you want to use an automated program in a production environment, you need to confirm whether such an automated operation complies with the terms of service of the relevant website. In addition, the selectors in the code may need to be adjusted accordingly according to the actual situation.

 It can be seen with the naked eye that the second code is very different from the first one. Let's run it and report the same error.

 

 We can use artificial intelligence to query what is wrong, we ask the question: "File "D:/Users/Administrator/PycharmProjects/pythonProject/venv/crawler/AI.py", line 15, in <module> ad_element =
    wait.until (EC.presence_of_element_located((By.CLASS_NAME, 'ad_item'))) what is wrong", the artificial intelligence gave the answer

 

 Here's how the AI ​​responded: "

This error means that your Python code has a problem at runtime on line 15. Specifically, this error indicates that WebDriver was unable to find the specified element while waiting.

According to the code, on line 15, you are using Selenium's wait method to wait for the ad element on the page to load. If this element is not present on the page, then this error will occur. Therefore, you need to check if the page loads successfully and there is an element called 'ad_item'.

”

 In fact, it is still a positioning error. We found that the artificial intelligence uses the method of the selenium library, and this method was used twice. Our white notes use the requests method, so can I make a request to let the artificial intelligence use requests? How to achieve it, the answer is that it must be possible. The reason why AI can only be called artificial intelligence, not Baidu or Google, is because it is not a simple input query, it can communicate with other people in daily life. At this time, we are like Party A. Now we are talking to the programmers of Party B. The requirement is raised: please use the requests method to achieve it.

 

 I don’t know if our questions are not accurate enough. The artificial intelligence did not give targeted answers according to our previous questions. It seems to have introduced the requests method in detail. Let’s adjust the way of asking questions. We ask again: "Please use python code to write the code to find the advertisement subpage of Baidu Tieba, click on the first advertisement", and then we continue to ask: "Please use the requests method to realize the question I just raised", this time for A reliable answer came out.

 

 Ok, the following is a sample code for obtaining Baidu search results using the requests method:

import requests

query = "python"  # 搜索关键词
url = f"http://www.baidu.com/s?wd={query}"
response = requests.get(url)

if response.status_code == 200:
    # 解析响应内容，获取所有搜索结果的标题和链接
    search_results = []
    for item in response.html.find(".t a"):
        title = item.text
        href = item.attrs['href']
        if href.startswith("http"):
            search_results.append((title, href))
    
    # 打印搜索结果
    for i, result in enumerate(search_results):
        print(f"{i+1}. {result[0]}")
        print(f"   {result[1]}")
else:
    print("获取搜索结果失败")

 This sample code uses the requests library to send a search "python" request to Baidu, then parses the response content, obtains the titles and links of all search results, and finally prints them out. Note: A third-party library is also used here pyqueryto parse the html. If you do not have this library installed, you can use the pip command to install it (pip install pyquery).

 The problem is not big, but it is estimated that the operation is still wrong, so I will not demonstrate it.

Summarize 

 Next, write a poem with artificial intelligence to see how great our artificial intelligence is.

Artificial intelligence, with endless magical uses
, changing life, becoming more and more obvious
Smart home, everything is convenient
Smart security, home protection is more stable

Machine translation, language is not difficult
Smart customer service, can answer questions
Automated driving, smooth roads
Medical assistance, disease prevention

Smart city, faster transportation
Smart education, better learning
Smart manufacturing, more efficient production
Artificial intelligence, continuous innovation

Advances in science and technology, human beings move forward
Artificial intelligence, the direction is right
A bright future, more expectations
Smart life, still need to work hard!