I. Introduction
1.1 Project blog address
https://www.cnblogs.com/lmc7318/p/12050055.html
1.2 Functions and Features completion of the project
Reptile crawling historical weather Xiamen newspaper website after weather to save the crawl to Xiamen historical weather to csv file, and then extract data csv file maximum and minimum temperature, and data visualization analysis with maximum and minimum temperature , then draw a line chart and a bar graph.
Temperature can make a person through line graphs and histograms more intuitive to see the magnitude of temperature changes.
1.3 project uses technology stack
python
Address project draws 1.4 source code
https://blog.csdn.net/qq_29721419/article/details/71638912
https://blog.csdn.net/mei1092670355/article/details/79699846
https://www.cnblogs.com/djw12333/p/11627573.html
1.5 team member task allocation table
| Liang Ying |
Data crawling |
| Liang Ying |
Data visualization analysis |
Second, the project's needs analysis
Can be obtained by reptiles data you want the site, you can save the data to your computer, and then extract the data you want to visualize data analysis. Data visualization analysis may draw various graphics, can visually observe a change in terms of specific gravity between the data, and various data through the graphics. This can greatly reduce your intuitive look at these tables and then the time it takes to analyze.
Third, the project functional architecture diagram, primarily functional flowchart
1. Item Function Chart
2. The main functional flowchart
Fourth, the described system modules
4.1 System Module List
| Requests，
|
BeautifulSoup， |
matplotlib |
4.2 detailed description of each module (name, function, operation theme, the key source code)
Requests 是用Python语言编写，基于 urllib，采用 Apache2 Licensed 开源协议的 HTTP 库。它比 urllib 更加方便，可以节约我们大量的工作，完全满足 HTTP 测试需求。Requests 的哲学是以 PEP 20 的习语为中心开发的，所以它比 urllib 更加 Pythoner。
Beautiful Soup提供一些简单的、python式的函数用来处理导航、搜索、修改分析树等功能。它是一个工具箱，通过解析文档为用户提供需要抓取的数据，因为简单，所以不需要多少代码就可以写出一个完整的应用程序。
Matplotlib是一个Python 2D绘图库，它可以在各种平台上以各种硬拷贝格式和交互式环境生成出具有出版品质的图形。 Matplotlib可用于Python脚本，Python和IPython shell，Jupyter笔记本，Web应用程序服务器和四个图形用户界面工具包
五、项目总结
5.1 特点
爬虫领域，涉及的知识点比较多，不仅要熟悉web开发，有时候还涉及机器学习等知识，不过在python里一切变的简单，有许多第三方库来帮助我们实现。使用python编写爬虫首先要选择合适的抓取模块，最简单的功能就是能发送和处理请求，这次的项目主要是爬取历史天气，通过爬虫爬出厦门天气的数据进行数据可视化绘制出的各种图形，能够更好的观测这些气温的变化幅度。
5.2 不足之处
功能比较简单，绘制的图不够齐全。