Convergence Rates of Constrained Expected Improvement (original) (raw)
Abstract:Constrained Bayesian optimization (CBO) methods have seen significant success in black-box optimization with constraints. One of the most commonly used CBO methods is the constrained expected improvement (CEI) algorithm. CEI is a natural extension of expected improvement (EI) when constraints are incorporated. However, the theoretical convergence rate of CEI has not been established. In this work, we study the convergence rate of CEI by analyzing its simple regret upper bound. First, we show that when the objective function fff and constraint function ccc are assumed to each lie in a reproducing kernel Hilbert space (RKHS), CEI achieves the convergence rates of mathcalOleft(t−frac12logfracd+12(t)right)textandmathcalOleft(tfrac−nu2nu+dlogfracnu2nu+d(t)right)\mathcal{O} \left(t^{-\frac{1}{2}}\log^{\frac{d+1}{2}}(t) \right) \ \text{and }\ \mathcal{O}\left(t^{\frac{-\nu}{2\nu+d}} \log^{\frac{\nu}{2\nu+d}}(t)\right)mathcalOleft(t−frac12logfracd+12(t)right)textandmathcalOleft(tfrac−nu2nu+dlogfracnu2nu+d(t)right) for the commonly used squared exponential and Matérn kernels ($\nu>\frac{1}{2}$), respectively. Second, we show that when fff is assumed to be sampled from Gaussian processes (GPs), CEI achieves similar convergence rates with a high probability. Numerical experiments are performed to validate the theoretical analysis.
Submission history
From: Haowei Wang [view email]
[v1] Fri, 16 May 2025 14:44:13 UTC (1,499 KB)
[v2] Sat, 10 Jan 2026 08:43:38 UTC (1,689 KB)