[FIXED] Bayesian Optimization for a multi-dimensional vector

Issue

I get difficulties when trying to use Bayesian Optimization package (https://github.com/fmfn/BayesianOptimization) for parameter optimization. The target function is of input vector, which is a 10-dimensional vector and the constrain that each entry is between (-2,2), is as following.

def black_box_function(vector):
    s = 0
    for i in range (0, 10):
        s = s + vector[i]/(vector[i+1] + vector[i+2])

    return s

from bayes_opt import BayesianOptimization
pbounds = {'vector': (-2*np.ones(10), 2*np.ones(10))}

optimizer = BayesianOptimization(
    f=black_box_function,
    pbounds=pbounds,
    verbose=2, 
    random_state=1,
)

optimizer.maximize(
    init_points=2,
    n_iter=3,
)

But it gets the error:

|  iter    |  target   |  vector   |
-------------------------------------
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_4116\2141365244.py in <module>
      8 )
      9 
---> 10 optimizer.maximize(
     11     init_points=2,
     12     n_iter=3,

D:\Code\bayes_opt\bayesian_optimization.py in maximize(self, init_points, n_iter, acquisition_function, acq, kappa, kappa_decay, kappa_decay_delay, xi, **gp_params)
    283         self._prime_subscriptions()
    284         self.dispatch(Events.OPTIMIZATION_START)
--> 285         self._prime_queue(init_points)
    286 
    287         old_params_used = any([param is not None for param in [acq, kappa, kappa_decay, kappa_decay_delay, xi]])

D:\Code\bayes_opt\bayesian_optimization.py in _prime_queue(self, init_points)
    240 
    241         for _ in range(init_points):
--> 242             self._queue.add(self._space.random_sample())
    243 
    244     def _prime_subscriptions(self):

D:\Code\bayes_opt\target_space.py in random_sample(self)
    261         data = np.empty((1, self.dim))
    262         for col, (lower, upper) in enumerate(self._bounds):
--> 263             data.T[col] = self.random_state.uniform(lower, upper, size=1)
    264         return data.ravel()
    265 

mtrand.pyx in numpy.random.mtrand.RandomState.uniform()

_common.pyx in numpy.random._common.cont()

_common.pyx in numpy.random._common.cont_broadcast_2()

_common.pyx in numpy.random._common.validate_output_shape()

ValueError: Output size (1,) is not compatible with broadcast dimensions of inputs (10,).

I don't know how to solve this problem. Thank you very much for your help.

Solution

Bayesian optimization here works on multiple scalar arguments and this package and won't support your vector implementation. Similarly, bounds are defined on each argument rather than the vector. You could instead go about it like this:

labels = ["x0", "x1", "x2", ...]
pbounds = {"x0": (-2, 2), "x1": (-2, 2), ... "x9": (-2, 2)}

Also, you would need to modify your function to take in keyword arguments rather than a vector parameter.

def black_box_func(x0, x1, x2, ...)

To make this less verbose you could use **kwargs in your function and some dict to map the labels to their indeces.

Answered By - Matt Prodani