Python爬虫实战，简单的爬虫案例，以及爬取百度贴吧网页原码和360翻译

一、爬取网页上的图片

import requests

response = requests.get("http://file.elecfans.com/web1/M00/8B/33/o4YBAFyTVqGAOgJFAAA5RKs3FWg313.jpg")
with open("pclogo.png", "wb") as f:
    f.write(response.content)

运行结果：

其中的地址是图片的地址，不是网页的地址，我们可以在网页上打开图片，然后点击右键，复制链接地址，就可以啦。

二、爬取百度贴吧

import requests

class TieBa(object):
    def __init__(self, tieba_name):
        self.tieba_name = tieba_name
        self.start_url = "https://tieba.baidu.com/f?kw="+tieba_name+"&ie=utf-8&pn={}"
        self.header = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) "
                                     "Chrome/63.0.3239.132 Safari/537.36 QIHU 360SE/10.0.2032.0"}

    def get_url_list(self):
        url_list = []
        for i in range(3):
            url = self.start_url.format(i * 50)
            url_list.append(url)
        return url_list

    def parse_url(self, url):
        print(url)
        response = requests.get(url, headers=self.header)
        return response.content.decode()

    def save_html(self, html, num):
        with open(self.tieba_name + "-" + "第" + str(num) + "页.html", "w", encoding="utf-8") as f:
            f.write(html)

    def run(self):
        url_list = self.get_url_list()
        for url in url_list:
            html = self.parse_url(url)
            page_num = url_list.index(url) + 1
            self.save_html(html, page_num)
        print("爬取成功！")


if __name__ == '__main__':
    print("爬取信息开始")
    tieba = TieBa('穿越火线')
    tieba.run()

结果如下：

因为有些网页是不允许特殊爬虫爬取的，所以我们需要更改头信息。

获取头信息header，我们可以在网页中按F12,然后点击Network,找到User-Agent，将头信息修改为这个就可以了。

三、爬取360翻译，实现英译中

import requests
import json

header = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/"
                        "63.0.3239.132 Safari/537.36 QIHU 360SE/10.0.2032.0"}
query = input("英文:")
data = {
    "eng": "1",
    "ignore_trans": "0",
    "query": query
}
post_url = "https://fanyi.so.com/index/search?eng=1&validate=&ignore_trans=0&query=hello"
response = requests.post(post_url, data=data, headers=header)
result = response.content.decode()
result = json.loads(result)
print("中文:"+result["data"]["fanyi"])

结果：