Use chatGPT to develop and obtain grid weather data

1. Grid Weather

1.1. Grid weather

Global high-precision, kilometer-level, grid-based weather forecast products based on latitude and longitude, including real-time weather and weather forecasts at any latitude and longitude. Among them, the high-precision weather at any coordinate is accurate to a range of 3-5 kilometers, including: temperature, humidity, atmospheric pressure, weather conditions, wind force, wind direction, etc.

Note: Grid weather is a numerical forecast, which is calculated from satellite, weather radar and other data through model algorithms. It may be affected by many factors, such as mountains, buildings, human activities, atmospheric changes, etc. Grid weather and observation station data will be different and cannot be directly compared.

1.1. Grid weather forecast

Grid point weather forecasting is a technology based on spatial interpolation method to forecast various physical quantities of the atmosphere on grid points with a certain spatial resolution. It predicts future meteorological changes with a high spatial resolution by gridding atmospheric physical quantities. Specifically, grid-point weather forecasting usually adopts a numerical weather prediction model, which divides the earth into a series of grids, uses physical equations to calculate the weather parameters in each grid, and then predicts future weather through time integration.

The characteristic of grid weather forecast is that it can provide higher precision weather forecast, because it can better capture small-scale weather phenomena and local meteorological changes. In addition, the grid weather forecast can also provide different forecast results for different application scenarios. For example, providing different forecast data for different fields such as aviation, ocean, agriculture, and urban meteorology can help related fields make more accurate decisions.

1.2. Grid real-time weather

Grid real-time weather refers to the technology of using near-Earth observation data and grid weather models to generate various meteorological elements in real time through interpolation algorithms. It can quickly generate the distribution status of meteorological elements, and forecast the change trend in the next few hours through the forecast model. This technology can be used for real-time monitoring and early warning of weather disasters, meteorological services in aviation, ocean and other fields.

Grid real-time weather is characterized by real-time and accuracy. By using near-Earth observation data, it can more accurately reflect actual weather conditions and provide timely weather warning services. In addition, because the real-time forecast is shorter, its resolution is higher, which can better reflect the changes of small-scale weather phenomena.

2. How to obtain weather data service

This article takes Hefeng Weather as an example, registered users (free/paid), use its API interface, see " Getting Started " for details.

2.1. Registered users

The first step is to register an account.
First of all, you need an account of Hefeng Weather. For the documentation of the account, please refer to Account Management.

Click to register!

The second step is to create a project and KEY.
In the left navigation of the console, select project management, and click the "Create Project" button in the upper right corner. Reference project and KEY.

When creating a project, you need to choose a subscription at the same time, for example: free subscription, standard subscription or premium subscription. Here we take free subscription as an example. See subscription.

The third step to get data
Enter the following API address in the browser (please replace the last KEY with the KEY you created in the previous step):

https://devapi.qweather.com/v7/weather/now?location=101010100&key=Your KEY
If you selected a standard subscription in the previous step, please ensure that your account’s available credit is ≥ 10, and enter the following in the browser API address:

https://api.qweather.com/v7/weather/now?location=101010100&key=your KEY
Now you should be able to get the real-time weather data of Beijing. However, please note that the free subscription only supports limited data and request volume, please refer to subscription comparison.

Step 4 More data
Refer to the development documentation for more data.

2.2. API

For the grid weather of Hefeng Weather Network, the corresponding API and parameters are as follows:

Lattice 24-hour weather forecast API:

• url=“https://devapi.qweather.com/v7/grid-weather/24h?”

• location='103.59984,36.18057', replace it with your actual latitude and longitude

• key='**************' #My own wind weather key

Lattice real-time weather data API:

• url=“https://devapi.qweather.com/v7/grid-weather/now?”

• location='103.59984,36.18057', replace it with your actual latitude and longitude

• key='**************' #My own wind weather key

Return the result set:

• code API status code, please refer to Status Code for specific meaning
• updateTime The last update time of the current API
• Responsive pages for fxLink current data for easy embedding in websites or applications
• now.obsTime data observation time
• now.temp temperature, default unit: Celsius
• now.icon Code for weather conditions and icons, icons can be downloaded through weather conditions and icons
• now.text Text description of weather conditions, including descriptions of weather conditions such as cloudy, rainy and snowy
• now.wind360 wind direction 360 angle
• now.windDir wind direction
• now.windScale wind scale
• now.windSpeed ​​wind speed, km/h
• now.humidity relative humidity, percentage value
• now.precip Accumulated precipitation in the current hour, default unit: mm
• now.pressure Atmospheric pressure, default unit: hPa
• now.cloud cloud amount, percentage value. may be empty
• now.dew Dew point temperature. may be empty
• refer.sources Original data source, or data source description, may be empty
• refer.license data license or copyright notice, may be empty

3. Use chatGPT to develop python to obtain grid weather data

The development and acquisition of grid weather data, combined with power generation data (electricity) to build a machine learning dataset requires a background service. In order to achieve fast and high-quality implementation, I chose chatGPT to assist in the implementation.

3.1. Code written by chatGPT

import requests
import csv
import time
import datetime
import keyboard

# API接口
url = "https://devapi.qweather.com/v7/grid-weather/now"

# 请求参数，location为经纬度坐标，key为API密钥
params = {
    
    
    "location": "103.59984,36.18057",
    "key": "**********"
}

fieldnames = ['obsTime', 'temp', 'icon', 'text', 'wind360', 'windDir', 'windScale', 'windSpeed', 'humidity', 'precip', 'pressure', 'cloud', 'dew']

# csv文件名
filename = 'weather_data.csv'

# 获取天气数据
while True:
    # 按下Esc键退出
    if keyboard.is_pressed('Esc'):
        print('程序已退出')
        break
    
    # 获取当前时间
    current_time = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime())

    # 判断是否为整点或半点
    if current_time.split(":")[1] in ["00", "30"]:     
        # 发送API请求并获取返回结果
        response = requests.get(url, params=params)
        result = response.json()

        # 提取关键数据并写入csv文件
        now_data = result['now']
        with open(filename, mode='a', newline='') as file:
            writer = csv.DictWriter(file, fieldnames=fieldnames)
            if file.tell() == 0:
                writer.writeheader()
           
            writer.writerow(now_data)

        # 记录日志
        with open('log.txt', mode='a') as log:
            log.write(f'{
      
      current_time}：获取天气数据成功\n')

    # 程序等待60秒钟后再次执行
    time.sleep(60)  

3.2. Communicate with chatGPT, clearly and accurately describe the requirements

Here's how I communicate with it:

Me:
Use the python language to develop an automatic hourly weather data service, and enter the Esc key to exit.
Use the following API interface to get the current weather data,
https://devapi.qweather.com/v7/grid-weather/now?location=103.59984,36.18057&key=************The
data is saved in csv format , and record the log of each data fetch.

chatGPT:

import requests
import csv
import time

# API接口地址
url = "https://devapi.qweather.com/v7/grid-weather/now"

# 请求参数，根据需要修改
params = {
    
    
    "location": "103.59984,36.18057",
    "key": "***********"
}

# csv文件名
filename = "weather_data.csv"

# 日志文件名
log_filename = "weather_log.txt"

# 逐时获取天气数据
with open(filename, mode="a", newline="") as csv_file, open(log_filename, mode="a") as log_file:
    writer = csv.writer(csv_file)
    while True:
        try:
            # 发送请求
            response = requests.get(url, params=params)
            # 解析响应数据
            data = response.json()
            # 提取需要的数据
            obs_time = data["updateTime"]
            temp = data["gridTemperature"]
            humidity = data["humidity"]
            wind_speed = data["windSpeed"]
            # 写入csv文件
            writer.writerow([obs_time, temp, humidity, wind_speed])
            # 写入日志文件
            log_file.write(f"{
      
      obs_time}: success\n")
        except Exception as e:
            # 发生异常时写入日志文件
            log_file.write(f"{
      
      time.strftime('%Y-%m-%d %H:%M:%S', time.localtime())}: error {
      
      str(e)}\n")
        # 暂停1小时后继续下一次循环
        time.sleep(3600)

Among them, you need to modify the API interface address, request parameters, file name and other information to the actual situation, and use your own API key for authorization and authentication. The program will continuously obtain weather data hour by hour, save the data into a csv file, and record the log of each data fetch. Press the Esc key to exit the program.

Me:
I didn’t make it clear, the time to take the data is the hour, and help to adjust the program.

chatGPT:

import requests
import csv
import time
import datetime

# API接口
url = "https://devapi.qweather.com/v7/grid-weather/now"

# 请求参数，location为经纬度坐标，key为API密钥
params = {
    
    
    "location": "103.59984,36.18057",
    "key": "***********"
}

# 循环获取数据
while True:
    # 获取当前时间
    now = datetime.datetime.now()
    
    # 判断是否为整点
    if now.minute == 0 and now.second == 0:
        # 发送请求获取数据
        response = requests.get(url, params=params)
        data = response.json()
        
        # 打印数据
        print(data)
        
        # 保存数据到csv文件
        with open("weather.csv", mode="a", newline="") as file:
            writer = csv.writer(file)
            writer.writerow([now.strftime("%Y-%m-%d %H:%M:%S"), data["now"]["temp"], data["now"]["humidity"]])
        
        # 记录日志
        with open("log.txt", mode="a") as file:
            file.write(f"{
      
      now.strftime('%Y-%m-%d %H:%M:%S')}：获取数据成功\n")
        
    # 等待1分钟
    time.sleep(60)
    
    # 监听键盘，按下Esc键退出循环
    if msvcrt.kbhit():
        if ord(msvcrt.getch()) == 27:
            print("退出程序")
            break

Here, the datetime module in the Python standard library is used to obtain the current time, determine whether it is an hour, and format the time into a string. At the same time, each time data is acquired, record the current time to the log file.

Me:
Please add that in the csv data, the name of the data field needs to be supplemented, for example, the data content is as follows: 'now': {'obsTime': '2023-04-27T09:00+00:00', 'temp': '10', 'icon': '100', 'text': 'Sunny', 'wind360': '63', 'windDir': 'Northeast wind', 'windScale': '5', 'windSpeed': ' 30', 'humidity': '46', 'precipe': '0.0', 'pressure': '820', 'cloud': '25', 'dew': '-1'}

chatGPT:

fieldnames = ['obsTime', 'temp', 'icon', 'text', 'wind360', 'windDir', 'windScale', 'windSpeed', 'humidity', 'precip', 'pressure', 'cloud', 'dew']

with open('weather.csv', mode='a', newline='') as file:
    writer = csv.DictWriter(file, fieldnames=fieldnames)
    if file.tell() == 0:
        writer.writeheader()
    writer.writerow(data)

Where fieldnames is a list containing the name of each data field, writer.writeheader() can automatically add the field name line when writing data for the first time.

Me:
Very good, thank you!
Trouble again, I want to fetch the data once every 30 minutes, and help modify the code.

chatGPT:

import requests
import csv
import time
import keyboard

# API接口和参数
url = 'https://devapi.qweather.com/v7/grid-weather/now'
location = '103.59984,36.18057'
key = '***********'
params = {
    
    'location': location, 'key': key}

# 字段名
fieldnames = ['obsTime', 'temp', 'icon', 'text', 'wind360', 'windDir', 'windScale', 'windSpeed', 'humidity', 'precip', 'pressure', 'cloud', 'dew']

# csv文件名
filename = 'weather_data.csv'

# 打开文件并写入字段名
with open(filename, mode='a', newline='') as file:
    writer = csv.DictWriter(file, fieldnames=fieldnames)
    writer.writeheader()

# 获取天气数据
while True:
    # 按下Esc键退出
    if keyboard.is_pressed('Esc'):
        print('程序已退出')
        break
    
    # 获取当前时间
    current_time = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime())
    
    # 发送API请求并获取返回结果
    response = requests.get(url, params=params)
    result = response.json()
    
    # 提取关键数据并写入csv文件
    now_data = result['now']
    with open(filename, mode='a', newline='') as file:
        writer = csv.DictWriter(file, fieldnames=fieldnames)
        writer.writerow(now_data)
    
    # 记录日志
    with open('log.txt', mode='a') as log:
        log.write(f'{
      
      current_time}：获取天气数据成功\n')
    
    # 程序等待30分钟后再次执行
    time.sleep(1800)

This code is similar to the previous code, except that time.sleep(3600) is changed to time.sleep(1800), that is, the program waits for 30 minutes and executes again. At the same time, the content of the log records has also been modified, and the record of successfully obtaining weather data has been added.

Me:
It’s great, I missed it, and I’ll add the full and half points.

chatGPT:
The code is detailed below, omitted.
This version of the code fetches data every hour and half hour and saves the data to a CSV file. At the same time, the log of each data fetch will be recorded and saved in the text file weather_log.txt.

3.3. Running the program

The content of the saved csv data file is intercepted as follows:

Note: The time difference between the time and Beijing time is 8 hours, please pay attention and deal with it when using the subsequent data.

There was a forced disconnection in the middle, the reason is unknown.

requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054,
 '远程主机强迫关闭了一个现有的连接。', None, 10054, None))

4. Summary

Gridpoint weather data service content is very comprehensive, and the functions are very easy to use.

Developed using chatGPT, for specific and versatile functions, its ability to write code is quite powerful, especially for continuous communication and improvement of the process, which is even more commendable.

Do you write code with chatGPT?