Reading time: 2023-11-5
1 Introduction
Year: 2022
Author: Jacob Morra, Mark Daley Western University
Journal: 2022 International Joint Conference on Neural Networks (IJCNN) The idea was inspired by the paper [A Connectome of the Adult Drosophila Central Brain] This algorithm replaces the reservoir layer of the classic ESN with the connection matrix obtained from the Drosophila connectome. Topological constraints are imposed on the ESN reservoir by forcing a connection graph with a specific connection structure. The impact of the topology of the Drosophila connection graph on the performance of the Echo State Network (ESN) in chaotic time series prediction was studied. The paper discusses the development of a class of models, called "Drosophila ESNs" (FFESNs), which replace the reservoir layers of classic ESNs with connectivity matrices derived from Drosophila connectivity graphs. The authors trained and verified the performance of FFESN in chaotic time series prediction tasks and compared it with the control model ESN. The results show that FFESN is either significantly better than ESN or has lower variance.
Citations: 3
2 innovation points
(1) The fruit fly-based neural connection structure (Fruit Fly Connectome) is introduced as the network topology of the Echo State Network (ESN). A class of models called "Fruit Fly ESNs" (FFESNs) is proposed by replacing the reservoir layers of classic ESNs with connection matrices based on the Drosophila neural connection structure.
(2) FFESNs were trained and verified on the chaotic time series prediction task, and the performance was compared with the control model ESN. The results show that FFESN is either significantly better than ESN or has lower variance.
(3) The method of generating a connection matrix based on the Drosophila neural connection structure is described in detail, and a formal description of the ESN network dynamics is given.
3 Algorithms
Topological constraints are imposed on the ESN reservoir by forcing a connection graph with a specific connection structure. The connection graph adopts Janelia's hemibrain connection graph matrix. The cell values in this matrix correspond to the number of synapse-to-synapse connections (weights) between neurons. ). The diagonal of the weight matrix is filled with 1, as shown below after normalization.
4 Experimental analysis
(1) Parameter adjustment
Leakying reate α determines the impact of the previous and current reservoir activation on the current reservoir activation. The regularization coefficient λ determines the size of the penalty term of ridge regression. Other hyperparameters such as spectral radius were not chosen as these would change the experimental and comparative reservoir topology.
(2) MSE distribution of multiple iterations
The mean square error of prediction for 50 iterations
shows the training input size When it is 900, the mean square error (MSE) of FFESN is significantly lower than that of ESN, and the error distribution is more concentrated.
(2) Forecasting MSE variance
Comparison results of forecasting performance and error variance in the case of Variant A and Variant B
For a specific training input size A, Variant A needs to predict values further into the future, in contrast to Variant B, which needs shorter-term predictions. In time series forecasting tasks, FFESN models either significantly outperform ESN models with lower error variance, or perform comparably to ESN models while maintaining significantly reduced error variance.
5 thoughts
This paper has the same idea as [Investigating Echo State Network Performance with Biologically-Inspired Hierarchical Network Structure], and the idea is the same. It’s just that the source of the idea for the paper is different. This article comes from [A Connectome of the Adult Drosophila Central Brain], and the other article comes from [Community detection and classification in hierarchical stochastic blockmodels.]. I haven't figured out the specific difference yet, but my understanding is roughly the same.