[FIXED] Error in search space while performing optuna hyperparameter optimization

Issue

the code most likely containing the bug->

import optuna

def objective(trial):
    

    
    criterion = trial.suggest_categorical("criterion", ["gini", "entropy"]),
    min_samples_split = trial.suggest_float('min_samples_split', 0, 1),
    max_depth = trial.suggest_int('max_depth', 3, 20, step=3),
    min_samples_leaf = trial.suggest_float('min_samples_leaf', 0, 1),
    min_impurity_decrease = trial.suggest_float('min_impurity_decrease', 0, 0.1),
    splitter = trial.suggest_categorical('splitter', ['best', 'random']),
    class_weight = trial.suggest_categorical('class_weight', [None, 'balanced'])

    clf = DecisionTreeClassifier(splitter=splitter, criterion=criterion, min_samples_split=min_samples_split, max_depth=max_depth, min_samples_leaf=min_samples_leaf, min_impurity_decrease=min_impurity_decrease, class_weight=class_weight)

    return cross_val_score(clf, X_train, y_train, cv=5, scoring='accuracy', n_jobs=-1).mean()

study = optuna.create_study(direction='maximize', sampler=optuna.samplers.TPESampler())
study.optimize(objective, n_trials=60, n_jobs=-1)

trial = study.best_trial

print('Accuracy: ', trial.value)
print('Best params: ', trial.params)

the error log->

[I 2023-08-13 23:29:24,693] A new study created in memory with name: no-name-9880947b-61e8-4f48-9439-9598b5855f1e
[W 2023-08-13 23:29:24,949] Trial 0 failed with parameters: {'criterion': 'gini', 'min_samples_split': 0.9672704152904984, 'max_depth': 3, 'min_samples_leaf': 0.9028727486308868, 'min_impurity_decrease': 0.03168240940817766, 'splitter': 'best', 'class_weight': 'balanced'} because of the following error: ValueError('\nAll the 5 fits failed.\nIt is very likely that your model is misconfigured.\nYou can try to debug the error by setting error_score=\'raise\'.\n\nBelow are more details about the failures:\n--------------------------------------------------------
  File "/home/mist/anaconda3/envs/ml/lib/python3.8/site-packages/sklearn/utils/_param_validation.py", line 97, in validate_parameter_constraints
    raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'criterion' parameter of DecisionTreeClassifier must be a str among {'log_loss', 'gini', 'entropy'}. Got ('gini',) instead.

What am i I doing wrong?

I tried commenting out the trial.suggest_categorical part for criterion variable but nothing changed.

Solution

The commas at the end of some of your statements are converting parameters to tuples. Removing the commas should fix it.

Answered By - some3128