Beautiful Soup is a Python library that can extract data from HTML or XML files. It can use your favorite converter to achieve the usual way of document navigation, search, and modification of documents. Beautiful Soup will save you hours or even days. operating hours.
Hyperlink on the page
In HTML, the hyperlink is represented by a, and the link address is written as href=. . .
<a href='http://www.baidu.com'>baidu</a>
Publish to the browser is:
Click this hyperlink to link to the corresponding webpage:,
Use bs4 to grab hyperlinks
Crawl all hyperlinks of Douban homepage
from urllib.request import urlopen#用于获取网页
from bs4 import BeautifulSoup#用于解析网页
html = urlopen('https://www.douban.com/')
bsObj = BeautifulSoup(html, 'html.parser')
t1 = bsObj.find_all('a')
for t2 in t1:
t3 = t2.get('href')
print(t3)The console prints the result:
https://book.douban.com
https://movie.douban.com
https://music.douban.com
https://www.douban.com/group/
https://www.douban.com/location/
https://douban.fm
https://www.douban.com/time/?dt_time_source=douban-web_anonymous_index_top_nav
https://market.douban.com?
Etc., etc. . . . . (Too much, I won’t paste it)
Grab picture link
from urllib.request import urlopen#用于获取网页
from bs4 import BeautifulSoup#用于解析网页
html = urlopen('https://www.douban.com/')
bsObj = BeautifulSoup(html, 'html.parser')
t1 = bsObj.find_all('img')
for t2 in t1:
t3 = t2.get('src')
print(t3)
Console print result:
https://www.douban.com/pics/blank.gif
https://img3.doubanio.com/f/sns/0c708de69ce692883c1310053c5748c538938cb0/pics/sns/anony_home/icon_qrcode_green.png
https://img3 .doubanio.com / F / SNS / 1cad523e614ec4ecb6bf91b054436bb79098a958 / pics / SNS / anony_home / doubanapp_qrcode.png
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com /f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https: //img3fce5174fe96fe96fe96/dee /blank.gif
https://img1.doubanio.com/dae/niffler/niffler/images/513f432a-6545-11e7-a372-0242ac110018.png
https://img1.doubanio.com/dae/niffler/niffler/images/416add14-0254-11e7-8292-0242ac11002c.jpg
https://img1.doubanio.com/dae/niffler/niffler/images/7bd08eba-0496-11e7-a4c9-0242ac110038.png
https://img1.doubanio.com/dae/niffler/niffler/images/54d34448-1dea-11e7-afa9-0242ac11002c.jpg
https://img3.doubanio.com/dae/niffler/niffler/images/6db31128-39dd-11e7-895e-0242ac110016.png
https://img3.doubanio.com/dae/niffler/niffler/images/2621f522-50ad-11e7-a3d6-0242ac110041.png
https://img3.doubanio.com/dae/niffler/niffler/images/c3ab871e-2401-11e7-9fe8-0242ac110046.png
https://img1.doubanio.com/dae/niffler/niffler/images/7bea8578-3f69-11e7-8261-0242ac11001b.png
https://img3.doubanio.com/dae/niffler/niffler/images/5d4dd498-5676-11e7-bb3d-0242ac11002d.jpg
https://img3.doubanio.com/pics/new_menu.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/pics/new_menu.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/img/songlist/large/476033-1.jpg
https://img3.doubanio.com/img/songlist/large/7610723-3.jpg
https://img3.doubanio.com/img/songlist/large/1321480-2.jpg
https://img3.doubanio.com/img/songlist/large/16075-2.jpg
https://img3.doubanio.com/img/songlist/large/449323-1.jpg
https://img3.doubanio.com/img/songlist/large/464547-1.jpg
https://img3.doubanio.com/view/dianpu_shop_icon/medium/public/2e35f89d2c7990d.jpg
https://img3.doubanio.com/view/dianpu_shop_icon/medium/public/9d03efbf705887d.jpg
https://img3.doubanio.com/view/dianpu_shop_icon/medium/public/169b1b76356f636.jpg
https://img3.doubanio.com/view/dianpu_shop_icon/medium/public/7f37444b0a6a0e2.jpg
https://img3.doubanio.com/view/dianpu_product_item/medium/public/p1971380.jpg
https://img3.doubanio.com/view/dianpu_product_item/medium/public/p458880.jpg
https://img3.doubanio.com/view/dianpu_product_item/medium/public/p270364.jpg
https://img3.doubanio.com/view/dianpu_product_item/medium/public/p521646.jpg
https://img1.doubanio.com/view/dianpu_product_item/medium/public/p509169.jpg
https://img3.doubanio.com/view/dianpu_product_item/medium/public/p377790.jpg
https://img3.doubanio.com/view/dianpu_product_item/medium/public/p1971380.jpg
https://img3.doubanio.com/view/dianpu_product_item/medium/public/p377790.jpg
https://img3.doubanio.com/view/dianpu_product_item/medium/public/p458880.jpg
https://img3.doubanio.com/view/dianpu_product_item/medium/public/p270364.jpg
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/pics/biaoshi.gif
https://img3.doubanio.com/img/files/file-1423193113.png
https://img3.doubanio.com/pics/icon/jubao.png
https://img3.doubanio.com/img/files/file-1489464722.jpg
