Brief description: mainly a few papers on how to use GNN to model resource allocation, [1] [2] are all about wireless link modeling, [3] is more informative, [4] is biased towards RL, and for GNNs are vague.
Ideas for modeling networks with GNN:
1. Is Graph a directed graph or an undirected graph?
2. What does Node mean? What are the features?
3. What does Edge mean? What are the features?
4. How is GNN updated? At present, the general update is to aggregate the messages of neighbor nodes first, and then combine all the information.
There will be different function choices for aggregate AGGREGATE and composite COMBINE .
[1]
Z. He, L. Wang, H. Ye, G. Y. Li, and B.-H. F. Juang, ‘Resource Allocation based on Graph Neural Networks in Vehicular Communications’, in 2020 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM), in IEEE Global Communications Conference. New York: IEEE, 2020. doi: 10.1109/GLOBECOM42002.2020.9322537.
Abstract: This paper develops a distributed GNN-enhanced RL spectrum sharing scheme for V2X networks. In our proposed method, the V2V network is represented as a graph. The local observations of V2V pairs and the channel gain of interfering links are regarded as the information of nodes and edges, respectively . We use GNNs to learn low-dimensional features of each node corresponding to V2V pairs based on graph information.
Model：
Graph is modeled as a directed graph
Node：each of the V2V pairs is regarded as a node 
Edge：interference links between V2V pairs as edges.
Node features：node observation contains the VUEs channel gain and its corresponding transmit power 
Edge features： edge weights are represented by the interference channel gain.
Use GNN to extract node features
The way the Node feature used in this article transmits messages 
|| Indicates the splicing of matrices, which is equivalent to the observation at node v, the mu value of the previous round, the sum of all neighbor edges now, and the sum of all mu in the previous round to form a matrix multiplied by a weight.
Why can the dimension of be set? ?
Algorithm framework
The general idea is a two-layer loop, the outer loop is t, and the RL network is trained to make decisions, but the input state s(t) of the RL network is not directly input observation observation, but uses GNN to first extract features from observation , and then as part of s(t), the GNN feature needs to iteratively train a network, which is the inner loop.
Feature extraction is to extract what feature? ? Why O means observation and the extracted feature should be used as state together. Doesn’t feature already contain observation?
There is very little data in the simulation part, and it is only compared with random, which is about 83% of the performance of RL-GNN, indicating that this GNN is not very good.
[2]
T. Chen, X. Zhang, M. You, G. Zheng, and S. Lambotharan, ‘A GNN-Based Supervised Learning Framework for Resource Allocation in Wireless IoT Networks’, IEEE Internet of Things Journal, vol. 9, no. 3, pp. 1712–1724, Feb. 2022, doi: 10.1109/JIOT.2021.3091551.
Abstract: A Graph Neural Network (GNN) based framework is proposed to address this challenge in a supervised manner. Specifically, a wireless network is modeled as a directed graph, where desired communication links are modeled as nodes and harmful interfering links are modeled as edges.
GAP: Some previous work on GNN [23][24] was limited to homogeneous wireless systems and may not be compatible with heterogeneous IoT systems. Furthermore, these works only studied continuous optimization problems, and the proposed methods may not be able to handle discrete optimization problems
GNN Modeling: The whole is modeled as a directed graph.
Node： the communication link between a transceiver pair can be treated as a node, and the Edge： interference link between two nodes can be treated as an edge.
Node features： The properties, such as the distance, channel information weight and priority that are related to communication links can be taken as node features.
Edge features: The properties, such as the distance and channel information that are related to interference links can be treated as edge features.
The way to update GNN:
[3]
Z. Sun, Y. Mo, and C. Yu, ‘Graph-Reinforcement-Learning-Based Task Offloading for Multiaccess Edge Computing’, IEEE Internet of Things Journal, vol. 10, no. 4, pp. 3138–3150, Feb. 2023, doi: 10.1109/JIOT.2021.3123822.
Abstract: The heuristic algorithm relies heavily on the precise mathematical model of the MEC system, and DRL does not make reasonable use of the relationship between devices in the MEC graph. In response to this problem, this paper proposes a task offloading mechanism based on Graph Neural Network (GNN), which can learn directly on graph data with message passing and aggregation.
GAP: Heuristic algorithms require expert knowledge and are difficult to make real-time decisions in dynamic MEC scenarios, and environmental changes often need to be resolved. Deep reinforcement learning (DRL), which combines reinforcement learning with deep neural networks (DNNs), offers a promising solution to the above challenges. Researchers have studied the application of DRL to various MEC task offloading problems. The MEC system consisting of wireless access devices, wireless channels and MEC servers is considered as an RL environment. The powerful modeling ability of DNN is used for comprehensive representation, and the offloading strategy is learned by interacting with the environment. However, DNN cannot represent graph data well, and the topological relationship between components in MEC with wireless communication as the edge is ignored. Therefore, traditional DRL methods are less adaptable and inapplicable when the MEC topology becomes complex.
In fact, this paper mainly optimizes the offloading variable
Model：
N wireless devices A server Each device can be connected to 1 or more MEC servers
the system time is divided into consecutive time frames of equal length t.
Dynamic: Tasks can be generated in any slot. The task can be offloaded and can be used in local computing.
GNN Modeling: Modeling the System as an Undirected Graph
Node: network entities in the MEC system (there are two types of WD and server)
Edge：WD and MEC communication connections
Node features: WD node update according to , that is, the task information generated by each slot
Time varying ：Applications on the WD randomly generate tasks, the wireless channel gain and background noise of the MEC system are also time varying.
The topology of devices may change over time, such as nodes addition or reduction, edges establishment, and disconnection.
Other parameters (e.g., the maximum computational performance of a particular server node and the total communication bandwidth of a wireless AP) are fixed.
Goal： Our goal is to minimize the total task-weighted response time of the MEC system over time by designing a reasonable computation offloading policy and resource scheduling policy.
For RL, its task is to find the optimal unloading strategy under the condition of given graph and task .
The main framework of the algorithm:
1. Model MEC and WD as Graph, use GNN to extract the characteristics of Graph, and predict x, where x represents the offloading variable. Note that the predicted x here is that each element is a continuous variable.
2. After obtaining the prediction vector , use a quantization method (a modified order preserving algorithm) to quantize it to 0 or 1. The quantization method depends on the exploration space K, and the selection of quantization results can be based on the Q value.
Suppose that GNN has prediction ˆ xt at the time slot t, then we quantize ˆ xt and extend K binarized offloading actions. In general, K is related to the size of the search space of the extended algorithm. When the search space is constant, the larger K is, the better the extended algorithm is, but accordingly, the computational complexity increases.
Graph Node Embedding
The neighborhood information aggregation process as follows:
Edges embedding:
[4]
K. Li, W. Ni, X. Yuan, A. Noor, and A. Jamalipour, ‘Deep-Graph-Based Reinforcement Learning for Joint Cruise Control and Task Offloading for Aerial Edge Internet of Things (EdgeIoT)’, IEEE Internet of Things Journal, vol. 9, no. 21, pp. 21676–21686, Nov. 2022, doi: 10.1109/JIOT.2022.3182119.
Abstract: This paper investigates a novel joint optimization of UAV cruise control and task offload assignment to maximize offloading to UAVs given the constraints of computing power and battery budget of IoT devices and the speed of UAVs. task. Since the optimization involves a large solution space and the instantaneous network state of the UAV is unknown, we propose a novel depth map-based reinforcement learning framework. An advantage actor-critic (A2C) structure is developed to train real-time continuous actions of UAVs in terms of flight speed, heading, and unloading scheduling.
In fact, it is based on the current state of the network to decide where the UAV will fly and which WD task to choose for processing.
Model：
N ground devices
1 UAV
GNN modeling:
This part is very vague, explaining the use of GNN to infer actions based on the state of the network, that is, trajectory and resource allocation.
The definition of state and action are given, but it does not specifically give how to define each vertex and edge in GNN, which features are included, and message passing only gives the definition, and does not explain how this article is done. , and which type of activation function is used.
State：
Action：
Algorithm framework:
