Preface
This article is the fourth article of this column. I will continue to share useful information on python crawler cases later, so remember to pay attention.
When working on a crawler project, sometimes the captured platform target data starts with &#x , as shown in the following figure:
The browser displays normal data, but the web page source code data obtained through the crawler protocol is hidden data starting with &#x . When encountering this situation, what should the crawler do?
When the crawler encounters hidden data starting with &#x , it can be solved with one line of code. Follow the author directly to read the detailed solution in the text. (complete code attached)
text
Address : aHR0cHM6Ly93d3cuYnRoaG90ZWxzLmNvbS9saXN0L3NoYW5naGFp
Goal : During the crawler process, the crawled target data is data starting with &#x
1. Problem description
Author or above