background:
Recent department heads to author a crawling task Baidu keyword ranking. Written the basic functions, you can not use it properly? Ever since, with this article, crawling some of the beautiful pictures, one can examine the next reptile effect; and secondly, it can also Yangyang Yan, work to increase the point is not fun, ha ha. Without further ado, this is to grab a picture, it is very seductive. Directly on the code address: http://www.win4000.com/meitu.html
Environmental :( reader is configurable)
Python3
urllib3
BeautifulSoup
requests
Please review the reader to see their own elements to determine the crawl target, entirely mechanically, it may be a problem
Source:
download_meinv.py
import os
urllib.parse Import urlparse from # urllib3 module should be brought, if not, then use the process in accordance with the error message resolve it
from bs4 import BeautifulSoup
import requests
'' 'When first introduced import module system libraries, the library of the third party' ''
'' 'Crawling all photos of beauty Home' ''
r = requests.get('http://www.win4000.com/meitu.html')
soup = BeautifulSoup(r.text,'html.parser')
img_list = []
for img in soup.select('img'):
if img.has_attr('alt'):
if img.has_attr('data-original'):
img_list.append((img.attrs['alt'],img.attrs['data-original']))
else:
img_list.append((img.attrs['alt'],img.attrs['src']))
image_dir = os.path.join (os.curdir 'Meinv')
if not os.path.isdir(image_dir):
os.mkdir (image_dir)
for img in img_list:
name = img[0] + '.' + 'jpg'
a = urlparam (img [1])
filepath = os.path.join(image_dir,name)
= URL '% S: S //% / S%'% (o.scheme, o.netloc, o.path [. 1:]. Replace ( '_ 250_350', '')) # download picture
print(url)
resp = requests.get(url)
with open(filepath,'wb') as f:
chunk in resp.iter_content for (1024): # If the picture is too large to download 1024 bytes
f.write(chunk)