0. Paper:
《MOABB: trustworthy algorithm benchmarking for BCIs》
On September 25, 2018, in "Journal of Neural Engineering"Journal publication
1. MOABB data set
《MOABB: trustworthy algorithm benchmarking for BCIs》2018th
1. Reasons for establishing the mother of BCI benchmarks:
Brain-computer interface builds a bridge of communication between humans and computers. Research on EEG signals is quite active, but reproducibility research in the field of BCI still has a long way to go. While many BCI datasets are freely available, researchers do not release the code, and reproducing the results needed to benchmark new algorithms is much harder than imagined. The parameters of the preprocessing step, the toolbox used and the implementation "tricks" can have a significant impact on performance, and these are almost never reported in the literature. Therefore, there is no comprehensive benchmark for BCI algorithms, and novices need to spend a lot of time browsing the literature to understand which algorithm is most effective on which data set.
2. The purpose of establishing the mother of BCI benchmarks:
Algorithms can be ranked and promoted on the website, making the different solutions available in the field visible at a glance.
When we read in the abstract of the paper that "...the proposed method achieved a score of 89% on MOABB (Mother of All BCI Benchmarks), which is higher than the state-of-the-art methods If you pay 5%...", the project will be successful.
3、Github地址：https://github.com/NeuroTechX/moabb
4,Indication BCI III/IV Numerical deficiencies:
Written over a thousand journal and conference papers on the BCI Competition III and IV datasets in the past year and a half . While it is impossible to deny the impact of these two datasets on the field, given that these datasets have been publicly available for over a decade, relying so heavily on such a small number of datasets—fewer than 50 subjects in total—makes the The field exposed several important issues. In particular, models published in papers that rely solely on this data set may be overfitting.
5, Point out the comparison set in the BCI innovation algorithm model paperSOTAQuestion:
The most difficult problem is that the open source code of the new BCI algorithm is scarce, which makes each laboratory responsible for open source its ownsource code, so that the public can verify that their models are comparable to the "state-of-the-art" (SOTA). Therefore, the vast majority of novel BCI algorithm papers are compared either with other work from the same laboratory, or with old, easy-to-implement models with SOTA accuracy in previous years, such as CSP or channel level. classifier is combined with the selectedvariance
6. Download MOABB data:https://neurotechx.github.io/moabb/datasets.html
Finally, I will send out introductions about brain-computer interfaces, especially algorithm models, from time to time every week. I will explain the BCI model from the perspective of scientific research and engineering implementation. Please pay attention and don’t lose it~