2023 "Huawei Cup" Research Competition Topic F | Strong convective precipitation nowcasting, modeling analysis, senior Xiaolu leads the team to guide the full code articles and ideas

Problem restatement

1. Design a model that uses dual-polarization radar data (Z_H, Z_DR and K_DP) to extract microphysical feature information and conduct nowcasting of severe convective precipitation. The input of the model is the radar data of the previous hour, and the output is the Z_H forecast of the next hour.

2. On the basis of question 1, the model is designed to alleviate the "regression to the mean" problem and make the forecast results more realistic and sufficient.

3. Use Z_H and Z_DR design models to estimate precipitation, K_DP is not available.

4. Design a model to evaluate the contribution of dual-polarization radar data in nowcasting of severe convective precipitation and optimize the data fusion strategy.

Question one

For question 1, to establish a model for extracting microphysical characteristics from dual-polarization radar data and conducting nowcasting of severe convective precipitation, the following ideas can be considered:

1. Data preprocessing: Perform necessary preprocessing such as filtering and calibration on dual-polarization radar data to obtain Z_H, Z_DR, and K_DP data with good quality.

2. Feature engineering: Extract statistics that describe the microphysical characteristics of strong convection systems, such as the average value, standard deviation, extreme value, etc. of each variable, as well as the correlation between variables, etc. Use these statistics as model input features.

3. Modeling: You can consider using convolutional neural network (CNN) or recurrent neural network (RNN). The input is the radar characteristics of the previous 1 hour, and the output is the Z_H forecast of the next 1 hour. The loss function is the error between predicted Z_H and true Z_H.

4. Training: Use historical radar data and corresponding Z_H labels to train the model to minimize the loss function.

5. Testing: Use an independent test set to evaluate model forecasting capabilities.

The corresponding mathematical expression can be expressed as:

Configuration number collection$\{(X_i,{AND}_i)\}$, inside$X_i$Display number$i$Customer import special expedition,${AND}_i$Represents the corresponding Z_H tag.

Model display:
$${WITH}_{H_{pred}}=f_{\theta }(X)$$

in that,$f_{\theta }$Display number 为$Neural\; network\; model\; of\; \theta$,${WITH}_{H_{pred}}$Represents predicted Z_H.

The training goal is to minimize the loss function:
$$L(\theta )=\frac{1}{N}\sum _{i=1}^{N}l(Z_{H_{pred}},{AND}_i)$$

Naka,$N$为样本Quantity,$l$ represents the loss function, common ones include MSE, MAE, etc. Through training, $L\left(\theta \right)$Since the minimization Excellent number$i^{\ast }$。

The model effect is evaluated by calculating evaluation indicators on the test set, such as RMSE, CORR, etc.

# 导入需要的库
import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader
#定义数据集类
class RadarModel(nn.Module):
    
    def __init__(self):
        super().__init__()
        
        # 输入通道数
        in_channels = 3 
        
        # 卷积层
        self.conv1 = nn.Conv2d(in_channels, 64, kernel_size=3, padding=1)
        self.bn1 = nn.BatchNorm2d(64)
        self.conv2 = nn.Conv2d(64, 128, kernel_size=3, padding=1)
        self.bn2 = nn.BatchNorm2d(128)
        self.conv3 = nn.Conv2d(128, 256, kernel_size=3, padding=1)
        self.bn3 = nn.BatchNorm2d(256)

        # 全连接层
        self.fc1 = nn.Linear(256 * 4 * 4, 1024) 
        self.fc2 = nn.Linear(1024, 1)
        
    def forward(self, x):
        
        # 输入大小 (batch_size, 3, 32, 32)
        
        x = self.conv1(x)
        x = self.bn1(x)
        x = nn.functional.relu(x)
        

Question 2

1. Using Generative Adversarial Networks (GAN)

• Add a discriminator D to the model, whose purpose is to distinguish between the real Z_H and the generated Z_H_pred.

• The loss function of the discriminator encourages it to judge the real Z_H as a positive example and Z_H_pred as a negative example.

• Adding the misclassification loss of the discriminator to the generator loss function makes the generated Z_H_pred more realistic and can deceive the discriminator.

• Through the adversarial training of the generator and the discriminator, a more detailed Z_H_pred can be obtained.

2. Add perceptual loss

• Define a feature extraction function φ, the input is Z_H, and the output is high-dimensional features.

• Add the difference between real and generated Z_H features in the loss function to make it a better match at the feature level.

• This can enhance the generation of details because high-dimensional features contain richer detailed information.

3. Auxiliary wind field prediction

• At the same time, the near-term wind field is predicted as auxiliary input information for the generator.

• The wind field provides flow information, which can help the generator synthesize a more realistic Z_H distribution.

• The final generated Z_H not only conforms to the physical constraints of precipitation, but also matches accurate wind field information.

The corresponding mathematical expression:

Original loss function:

$L_{MSE}=\frac{1}{N}{\sum }_{i=1}^N{\left|{WITH}_{H_{pred}}-{WITH}_{H_{true}}\right|}^2$

Add perceptual loss:

$L_{perceptual}=\frac{1}{N}{\sum }_{i=1}^N{\left|\varphi (Z_{H_{pred}})-\varphi \left(Z_{H_{true}}\right)\right|}^2$

inside$\varphi$ represents the high-dimensional feature extraction function.

Then the new loss function is:

$L=L_{MSE}+\lambda L_{perceptual}$

Add discriminator loss:

$L_D=-\frac{1}{N}{\sum }_{i=1}^N[\log D(Z_{H_{true}})+\log (1-D(Z_{H_{pred}}))]$

Final loss function:

$L_{total}=L+\lambda L_D$

import torch
import torch.nn as nn
from torch.nn import functional as F

# 生成器 
class Generator(nn.Module):

  def __init__(self, in_channels, out_channels):
    super().__init__()
    self.net = nn.Sequential(
        nn.Conv2d(in_channels, 64, kernel_size=3),
        nn.BatchNorm2d(64),
        nn.ReLU(),
        nn.Conv2d(64, 128, kernel_size=3),
        nn.BatchNorm2d(128),
        nn.ReLU(),
        nn.ConvTranspose2d(128, out_channels, kernel_size=4, stride=2, padding=1),
        nn.Tanh()
    )

  def forward(self, x):
    return self.net(x)

# 判别器  
class Discriminator(nn.Module):

  def __init__(self, in_channels):
    super().__init__()
    self.net = nn.Sequential(
        nn.Conv2d(in_channels, 64, kernel_size=4, stride=2, padding=1),
        nn.LeakyReLU(0.2),
        nn.Conv2d(64, 128, kernel_size=4, stride=2, padding=1),
        nn.BatchNorm2d(128),
        nn.LeakyReLU(0.2),
        nn.Conv2d(128, 1, kernel_size=4, stride=1, padding=0),
        nn.Sigmoid()
    )

  def forward(self, x):
    return self.net(x)

# 特征提取网络
class FeatureExtractor(nn.Module):

For the full version of the code, please pay attention to my column:
2023 "Huawei Cup" Research Competition Topic F | Strong convective precipitation nowcasting, modeling analysis, senior Xiaolu leads the team to guide the whole process Code articles and ideas