Application of Fourier Transform in Graph Neural Networks

       Fourier transform is commonly used in graph neural networks to transform node features from the time domain (or space domain) to the frequency domain, so as to perform convolution operations on node features in the frequency domain. Specifically, the graph signal can be regarded as a time-series signal, and the Fourier transform is applied to the time-series signal to transform the node features into a frequency-domain signal. In this way, in the frequency domain, the convolution operation can be expressed as a simple dot product operation.

       For GNN based on spectral decomposition of Laplacian matrix, such as GCN, Fourier transform can be used to represent the eigenvector of Laplacian matrix as a frequency domain signal, and then the node features are convolved by frequency domain convolution operation. Specifically, let $X$ is$n\times$Node feature matrix of$f$$A$为$n\times$Adjacency matrix of$n$$D$ is$n\times The\; degree\; matrix\; of\; n$ , then the Laplacian matrix$L=D-A.$ _ to$L$ performs eigenvalue decomposition to obtain eigenvalues$l_0\le l_1\le ...\le l_{n-1}$Summation quantity $u_0,u_1,...,u_{n-1}$. Using the Fourier transform, the eigenvector can be expressed as a Fourier basis function $e_k(x)=\sqrt{\frac{2}{n}}cos(\frac{\pi kx}{n})$ , where$k=0,1,2,...,n-1$ . Representing the feature vector as a frequency-domain signal, the frequency-domain convolution operation can be performed, thereby realizing the convolution operation on the node features.

        It should be noted that in practical applications, due to the limitation of computational efficiency and other factors, the complete Fourier transform and inverse Fourier transform are usually not used, but methods such as fast Fourier transform based on Chebyshev polynomials or Krylov subspaces are used to perform frequency domain convolution operations to improve computational efficiency.

        The frequency domain convolution operation is a commonly used convolution operation in graph neural networks. Its basic idea is to perform convolution operations on node features in the frequency domain after Fourier transform. Specifically, let $X$ is$n\times$Node feature matrix of$f$$A$为$n\times$Adjacency matrix of$n$$D$ is$n\times The\; degree\; matrix\; of\; n$ , then the Laplacian matrix$L=D-A.$ _ to$L$ performs eigenvalue decomposition to obtain eigenvalues$l_0\le l_1\le ...\le l_{n-1}$Summation quantity $u_0,u_1,...,u_{n-1}$, then the Fourier transform can express the node feature matrix as:

$$\tilde{X}=U^{T}X$$

       任何$U=(u_0,u_1,...,u_{n-1})$ is a matrix composed of eigenvectors of the Laplacian matrix,$\tilde{X}$ is the frequency domain representation of the node feature matrix after Fourier transform. In the frequency domain, the convolution operation can be expressed as a simple point multiplication operation, so the node features can be convolved in the frequency domain to obtain a new frequency domain feature representation$\tilde{Y}$：

$$\tilde{Y}=Th\tilde{X}$$

       Θ $\Theta$ is the convolution kernel in the frequency domain, usually a set of learnable parameters. In practical applications, in order to improve computational efficiency, some fast Fourier transform methods, such as the method based on Chebyshev polynomials or the method based on Krylov subspace, are usually used to perform frequency domain convolution operations.

       Finally, through the inverse Fourier transform, the frequency domain features can be expressed as $\tilde{Y}$ Convert back to the node feature representation in the spatial domain$Y$：

$$Y=U\tilde{Y}$$

Get a new representation YY        of node features$Y.$ _ This Fourier transform-based frequency domain convolution operation is widely used in graph neural networks, such as Laplacian matrix-based spectral convolution networks (such as GCN), GraphSAGE, etc.