Resolved IndexError: list index out of range

Solved (Python crawler traverses the list error) IndexError: list index out of range

---

---

error code

When a small friend in the fan group wanted to use the requests crawler and then traverse the data list located by Xpath, an error occurred (at that time, he felt a lot of cold in his heart, and came to me for help, and then successfully helped him solve it, by the way Make a record and hope to help more friends who encounter this bug and will not solve it), the error code is as follows: :

import requests
from lxml import etree

text = requests.get("https://www.baidu.com").content.decode()


html = etree.HTML(text)

div_divst = html.xpath("//div")
for div in div_divst:
    title = div.xpath("@title")[0]
    url = div.xpath("@url")[0]

The error message is as follows :

IndexError: list index out of range

error translation

Error message translation :

index error: list index out of range

Reason for error

There are two main reasons for the list index out of range error:

• One possibility is that the subscript is out of range
• One may be that the list is empty and does not have an element
  
  

The fan's crawler code is obviously because Xpath does not locate the data, so the list is empty, and list[0] will report an error.Friends, please follow the steps below to solve it! ! !

Solution

Use exception to capture Xpath positioning, when an error is reported, capture the exception and assign the title and url to None:

import requests
from lxml import etree

text = requests.get("https://www.baidu.com").content.decode()

html = etree.HTML(text)

div_divst = html.xpath("//div")
for div in div_divst:
    try:
        title = div.xpath("@title")[0]
        url = div.xpath("@url")[0]
    except:
        title = None
        url = None

help

This article has been included in: "Farewell to Bug" column

This column is used to record various difficult bugs encountered in study and work, as well as various problems raised by small partners in the fan group. Article format: error code + error translation + error reason + solution, including program installation, operation If you encounter other problems in the process of the program, after subscribing to the column + following the blogger, if you encounter other problems, you can privately chat to help solve them! ! !