The benchmarking datasets are the basis of fair comparison and validation of computational methods.
What are benchmark datasets
Benchmark datasets (benchmark database) are the basis for fair comparison and verification of computing methods. Benchmark datasets are used for benchmark testing and are a benchmark for measuring the performance of an algorithm model. Many specific machine learning problems require benchmark data.
Running an algorithm on a data set can be understood as a running score. The benchmark data set is a data set used as an evaluation standard. Many people run on it to evaluate the quality of an algorithm model. Otherwise, everyone is running on different data sets, how to evaluate whose algorithm is good and whose algorithm is bad?
What is the difference between benchmark datasets and baseline
Benchmark is a process, and baseline is an instance of the benchmark process.
Baseline can be understood as the lowest standard, that is, it is definitely not acceptable to be lower than this standard; while benchmark can be understood as "a certain scale", which can be high or low, so the benchmark itself is not good or bad, and it is simply used for comparison.
The baseline is objective, and the baseline is subtracted from the original data to obtain the final value. The benchmark is relatively subjective, it is a standard value, depending on the basis, and is mainly used for parallel comparison.
personal understanding
We have learned in high school that an important principle of scientific experiments is to control variables, and benchmarks and baselines are actually just control variables.
As far as I understand, benchmark data is generally used for experiments, ABCD four models are tested, and then the model with the worst result, assuming C, is used as the baseline to see how much other models have improved relative to C.
The purpose of the baseline is to compare the performance of the proposed algorithm or to compare and highlight the advantages of the proposed algorithm.
some benchmark databases
- 1 UCL Machine Learning Knowledge Base
- 2 Amazon AWS Public Dataset
- 3 Kaggle
- 4 KD nuggets
- 5 US Federal Government Datasets
- 6 Datasets from infochimps
- 7 million song database
- 8 Protein Information Resource Dataset
- 9 Handwritten Digit Image Dataset
- 10 face recognition database
- 11 Some Bayesian Nets
- 12 kdd99 datasets
- 13 Dataset index
- 14 University of Washington
- 15 Sogou Data Resources
reference article
What is the difference between baseline and benchmark?
Special Issue : Benchmarking Datasets in Bioinformatics - MDPI