[FIXED] How to determine param_grid in GridSearchCV if there are some parameters can't be used with others?

Issue

In GridSearchCV, I want to try different combinations of parameters to tune hyperparameter but some can't be use with another such as lbfgs can be used with only l2 in logistic regression.

Below is common way that I use currently,

clf = LogisticRegression(random_state=0)
param_grid = {
   "C": [0.0001, 0.001, 0.01, 0.1, 1],
   "penalty": ["l1", "l2"],
   "solver": ["liblinear", "lbfgs"]
}

but this gives me an error because the lbfgs solver can't be used with l2.

Solution

You can use list of dict of parameter combinations instead of a dict.

For example if you want to tune C, penalty, and solver by separating the solvers to different combination, you can do it by this way:

param_grid = [
    {
        "C": [0.0001, 0.001, 0.01, 0.1, 1],
        "penalty": ["l1"], 
        "solver": ["liblinear"]
    },
    {
        "C": [0.0001, 0.001, 0.01, 0.1, 1],
        "penalty": ["l2"],
        "solver": ["lbfgs"]
    }
]

Answered By - JayPeerachai