The current understanding of Bayesian is to use a probabilistic proxy model to fit the probability distribution of the actual model. The actual model can be a complex optimization problem with unknown objective function expression, non-convex, multimodal and expensive evaluation. Use the probabilistic proxy model to continuously evaluate the objective function value with noise (noise is because this is a proxy model rather than the actual model), and then continuously update the new set to obtain new evaluation points (the evaluation point is the parameter to be optimized), The objective function value is then evaluated using the new evaluation points and the surrogate model.
Bayesian optimization algorithm framework:
Common probabilistic proxy functions:
Common collection functions:
reference:
[1] Cui Jiaxu, Yang Bo. Bayesian optimization method and application review [J]. Journal of Software, 2018, 29(10): 3068-3090. DOI: 10.13328/j.cnki.jos.005607.
[2] Three implementations of Bayesian optimization - Search (bing.com)