Preface: Use python crawlers to crawl weather data and then use echarts data visualization to draw and analyze future weather trends
Process from data acquisition to visualization
(Familiar with the process of crawling data and then visualizing data)
1. Python crawls weather data for the next 8-15 days
1. Import some python third-party libraries used
import requests
import re
from bs4 import BeautifulSoup
import json
import pandas as pd
2. Crawl the source code of the web page, and get the source code of the web page through utf-8 conversion
url='http://www.weather.com.cn/weather15d/101290101.shtml'
rqg=requests.get(url)
html=rqg.content.decode('utf-8')
soup=BeautifulSoup(html,'lxml')
print(soup.prettify())
3. Use the BeautifulSoup library to extract time information
tt=soup.find_all("span",class_="time")
tt
4. Use list comprehension to filter and extract time
time=[i.string for i in tt]
time 
5. Use the BeautifulSoup library to extract weather information
hh=soup.find_all("span",class_="tem")
hh 
6. Use list comprehension to filter and extract weather
tem=[i.get_text() for i in hh]
tem 
7. Use the re library for "/" separator to divide the temperature
tem1=[re.split('/',i) for i in tem]
tem1 
8. Use list comprehension to extract the highest temperature and the lowest temperature
h=[i[0] for i in tem1]
h
l=[i[1] for i in tem1]
l 
9. Use list comprehension to remove the sign of Celsius
[int(re.sub('℃','',i)) for i in h]
[int(re.sub('℃','',i)) for i in l] 
2. Using echarts data visualization analysis
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>气温变化分析</title>
<script src="js/echarts.js"></script>
</head>
<body>
<div id="main" style="width: 800px; height: 600px"></div>
<script type="text/javascript">
var myChart = echarts.init(document.getElementById("main"));
mytextStyle = {
color: 'blue',
fontStyle: 'normal',
fontWeight: 1,
fontFamily: '黑体',
fontSize: 20,
};
var option = {
grid: {
show: true,
x: 50, y: 66,
borderColor: '#FA8072',
},
title: {
show: true,
text: '未来8-15天气温变化',
subtext: '折线图',
target: 'blank',
subtarget: 'blank',
textAlign: 'center',
textBaseline: 'top',
textStyle: mytextStyle,
padding: 5,
itemGap: 10,
zlevel: 0,
z: 2,
left: '20%',
top: '10',
right: 'auto',
bottom: 'anto',
backgroundColor: 'yellow',
borderColor: '#ccc',
borderWidth: 2,
shadowColor: 'red',
showOffsetX: 0,
showOffsetY: 0,
shadowBlur: 10,
},
tooltip: { trigger: 'axis' },
legend: { data: ['最高气温', '最低气温'] },
toolbox: {
show: true,
feature: {
mark: { show: true },
dataView: { show: true, readOnly: false },
magicType: { show: true, type: ['line', 'bar'] },
restore: { show: true },
saveAsImage: { show: true }
}
},
calculable: true,
xAxis: [
{
show: true, type: 'category',
boundaryGap: false,
data: ['周六（30日）', '周日（1日）', '周一（2日）', '周二（3日）', '周三（4日）', '周四（5日）', '周五（6日）']
}
],
yAxis: [
{
show: true, type: 'value',
axisLabel: { formatter: '{value} °C' }
}
],
series: [
{
name: '最高气温',
smooth: true, type: 'line',
data: [28, 28, 19, 16, 23, 29, 27, 20],
markPoint: {
data: [
{
type: 'max', name: '最大值', symbol: 'diamond', symbolSize: 25,
itemStyle: {
normal: { color: 'red' }
},
},
]
},
markLine: {
data: [
{
type: 'average', name: '平均值',
itemStyle:
{
normal: { borderType: 'dotted', color: 'darkred' }
},
}],
},
},
{
name: '最低气温',
smooth: true, type: 'line', data: [15, 11, 8, 8, 12, 13, 13, 13],
markPoint: {
data: [
{
type: 'max', name: '最大值', symbol: 'diamond', symbolSize: 25,
itemStyle: {
normal: { color: 'red' }
},
},
]
},
markLine: {
data: [
{
type: 'average', name: '平均值',
itemStyle:
{
normal: { borderType: 'dotted', color: 'darkred' }
},
}],
},
},
],
};
myChart.setOption(option);
</script>
</body>
</html>
The implemented functions are relatively simple, and the processing flow is relatively complete. Data analysis has been automated as much as possible.