Talk:Bayesian optimization

"does not assume any functional forms"

I'll admit that this may be a pedantic point, but is the statement "does not assume any functional forms" accurate? Later in the article it talks about using Kriging as one option for the underlying surrogate. Kriging is quite general, but it's not able to capture all possible functional forms. For example, it's limited to continuous functions (at least for the typical form of Kriging I've seen used). Bayesian Optimization can use essentially any surrogate form on which the confidence interval can be calculated, which is good for generality, but means the user must make a decision about what surrogate to use at the start of the process. If that surrogate is not well-suited to the problem then this approach may give poor results.

My point in all this is I think this statement in the opening sentence is inaccurate and may give readers the wrong idea. - EndingPop (talk) 14:30, 28 August 2024 (UTC)

Pareto Optimization

Bayesian optimization inherently deals with the exploration exploitation tradeoff (of suggesting new points) and is therefore closely related to the concept of pareto optima. Could somebody please discuss this in the article? Biggerj1 (talk) 21:49, 6 October 2022 (UTC)

Good observation! These are indeed conflicting goals that could be handled using multiobjectie optimization. However, this is rather a current research topic than state of the art. BumbleMath (talk) 13:33, 22 January 2024 (UTC)

Adding pseudo code in wikipedia

Hi there,

how do I add Pseudo Code in wikipedia?

Regards Ethan R. Erdabravest (talk) 14:58, 20 June 2024 (UTC)

Link from "Bayesian optimiser"

Needs a link adding as using British spelling gets no results. 212.100.0.227 (talk) 10:37, 23 July 2025 (UTC)

Proposed broad rewrite of Bayesian optimization article

I think this article would benefit from a broad rewrite rather than a set of small additions. The current version gives a useful starting definition, but it over-centres the basic Gaussian-process/expected-improvement presentation and does not clearly map several major parts of the field. This would also address points already raised above about the phrase "does not assume any functional forms" and about Pareto/multi-objective optimization.

Main issues:

• The lead says Bayesian optimization "does not assume any functional forms", which is misleading. BO does not require a closed-form objective or derivatives, but it does make modelling assumptions through the surrogate, prior, kernel/covariance function, likelihood/noise model, and acquisition policy.
• The article does not clearly distinguish noiseless and noisy settings, even though this affects the definition of the incumbent, the acquisition function, and the optimization target.
• Constrained Bayesian optimization and multi-objective Bayesian optimization are underdeveloped, despite being central extensions.
• The acquisition-function section is too thin. It should distinguish improvement-based, confidence-bound, sampling-based, value-of-information, and information-theoretic methods.
• The information-theoretic branch should include Villemonteix, Vazquez, and Walter's IAGO / minimizer-entropy work, as well as entropy search, predictive entropy search, max-value entropy search, and knowledge-gradient methods.
• The multi-objective branch should not start with recent software-oriented work only; it should mention scalarization-based methods such as ParEGO and hypervolume-based methods such as expected hypervolume improvement.
• The current references mix foundational sources with weak or overly narrow citations. A rewrite should rely more heavily on broad reviews, tutorials, books, and field-defining papers.

Suggested structure:

• Lead
• Problem setting
• Basic method
• Surrogate models
• Acquisition functions
• Variants and extensions
• Applications
• History
• Limitations
• See also
• References

Core sources that would support the rewrite include:

• Mockus, Bayesian Approach to Global Optimization.
• Jones, Schonlau, and Welch, "Efficient Global Optimization of Expensive Black-Box Functions".
• Rasmussen and Williams, Gaussian Processes for Machine Learning.
• Brochu, Cora, and de Freitas, "A Tutorial on Bayesian Optimization of Expensive Cost Functions".
• Shahriari et al., "Taking the Human Out of the Loop: A Review of Bayesian Optimization".
• Frazier, "A Tutorial on Bayesian Optimization".
• Garnett, Bayesian Optimization.
• Villemonteix, Vazquez, and Walter, "An informational approach to the global optimization of expensive-to-evaluate functions".
• Hennig and Schuler, "Entropy Search for Information-Efficient Global Optimization".
• Hernandez-Lobato, Hoffman, and Ghahramani, "Predictive Entropy Search for Efficient Global Optimization of Black-box Functions".
• Frazier, Powell, and Dayanik, "The knowledge-gradient policy for correlated normal beliefs".
• Gelbart, Snoek, and Adams, "Bayesian Optimization with Unknown Constraints".
• Picheny, Gramacy, Wild, and Le Digabel, "Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian".
• Feliot, Bect, and Vazquez, "A Bayesian approach to constrained single- and multi-objective optimization".
• Knowles, "ParEGO: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems".
• Daulton, Balandat, and Bakshy, "Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization".

I have prepared a draft rewrite in sandbox-style wikitext. It is intended as a replacement starting point rather than final text; individual sections can be merged incrementally if a full replacement is considered too disruptive. Antro5ae (talk) 15:51, 22 June 2026 (UTC)