[FIXED] 'TPESampler' object has no attribute '_group'

Issue

Today the following error appeared in my study when I try to optimize it after loading on Colab on Optuna 2.8.0. I save a study with joblib each time a trial begins or ends in a separate file for each trial. Never have I had this problem before and not sure what actually causes it.

Colab shows the following trace:

/usr/local/lib/python3.7/dist-packages/optuna/study.py in optimize(self, func, n_trials, timeout, n_jobs, catch, callbacks, gc_after_trial, show_progress_bar)
    408             callbacks=callbacks,
    409             gc_after_trial=gc_after_trial,
--> 410             show_progress_bar=show_progress_bar,
    411         )
    412 

/usr/local/lib/python3.7/dist-packages/optuna/_optimize.py in _optimize(study, func, n_trials, timeout, n_jobs, catch, callbacks, gc_after_trial, show_progress_bar)
     73                 reseed_sampler_rng=False,
     74                 time_start=None,
---> 75                 progress_bar=progress_bar,
     76             )
     77         else:

/usr/local/lib/python3.7/dist-packages/optuna/_optimize.py in _optimize_sequential(study, func, n_trials, timeout, catch, callbacks, gc_after_trial, reseed_sampler_rng, time_start, progress_bar)
    160 
    161         try:
--> 162             trial = _run_trial(study, func, catch)
    163         except Exception:
    164             raise

/usr/local/lib/python3.7/dist-packages/optuna/_optimize.py in _run_trial(study, func, catch)
    195                 failed_trial_callback(study, failed_trial)
    196 
--> 197     trial = study.ask()
    198 
    199     state: Optional[TrialState] = None

/usr/local/lib/python3.7/dist-packages/optuna/study.py in ask(self, fixed_distributions)
    485         if trial_id is None:
    486             trial_id = self._storage.create_new_trial(self._study_id)
--> 487         trial = trial_module.Trial(self, trial_id)
    488 
    489         for name, param in fixed_distributions.items():

/usr/local/lib/python3.7/dist-packages/optuna/trial/_trial.py in __init__(self, study, trial_id)
     55         self.storage = self.study._storage
     56 
---> 57         self._init_relative_params()
     58 
     59     def _init_relative_params(self) -> None:

/usr/local/lib/python3.7/dist-packages/optuna/trial/_trial.py in _init_relative_params(self)
     65         self.relative_search_space = self.study.sampler.infer_relative_search_space(study, trial)
     66         self.relative_params = self.study.sampler.sample_relative(
---> 67             study, trial, self.relative_search_space
     68         )
     69 

/usr/local/lib/python3.7/dist-packages/optuna/samplers/_tpe/sampler.py in sample_relative(self, study, trial, search_space)
    327         self._raise_error_if_multi_objective(study)
    328 
--> 329         if self._group:
    330             assert self._search_space_group is not None
    331             params = {}

AttributeError: 'TPESampler' object has no attribute '_group'

PS. Interestingly, the problem with the study doesn't seem to exist on my local machine. I use version 2.5.0.

Solution

Just got the answer from Optuna developers:

A (private) attribute was introduced to the TPESampler in v2.8 and if you've pickled or serialized the sampler before this introduction, you'll encounter this AttributeError when unpickling it with v2.8.

The solution to this problem is re-instantiating the sampler object and substitute it to the study.sampler:

study = optuna.load_study(...) 
sampler = optuna.samplers.TPESampler() 
study.sampler = sampler
study.optimize(...)

PS. The same kind of problem may appear with a pruner. If so, re-instantiate also the pruner!

Answered By - Krzysztof Maliszewski