[FIXED] Using GridSearchCV for MLPRegressor: How to fit multiple hidden_layer_sizes

Issue

I'm using GridsearchCV for hyperparameter tuning. Im having problems fitting a variation of "hidden_layer_sizes", as they have to be tuples.

Is there a way to check multiple hidden layer sizes, without manually typing them in?

My code looks like this:

param_list = {"hidden_layer_sizes": [(200,70), (150,100,50), (120,80,40), (100,50,30)], "activation": ["identity", "logistic", "tanh", "relu"], "solver": ["lbfgs", "sgd", "adam"], "alpha": [0.0001,0.0005]}
MLP_gridCV = GridSearchCV(
    estimator=MLPRegressor(max_iter=10000, n_iter_no_change=30),
    param_grid=param_list,
    n_jobs=-1,
    cv=3,
    verbose=5,
)
MLP_gridCV.fit(X_train, y_train.ravel())

# Prediction
y_pred8 = MLP_gridCV.predict(X_test)

For my SVR I used:

gamma = np.arange(0.001, 0.1, 0.001).tolist()

Is there any possibility to use the same logic for the MLPRegressor so that I dont have to type in (150,100,50) for example?

I tried something like this, but it doesnt work:

learning_rate_init= [0.0001,0.0002]
first_layer_neurons= np.arange(10, 200, 10).tolist()
second_layer_neurons= np.arange(10, 200, 10).tolist()
hidden_layer_sizes = [first_layer_neurons,second_layer_neurons]
activation= ['identity', 'tanh', 'relu']
   

params_grid = {
                'hidden_layer_sizes':hidden_layer_sizes,
                'learning_rate_init':learning_rate_init,
                'activation':activation}

print(params_grid)
{'hidden_layer_sizes': [[10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190],
[10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190]],
'learning_rate_init': [0.0001, 0.0002], 
'activation': ['identity', 'tanh', 'relu']}

Solution

You can use product from itertools:

from itertools import product

first_layer_neurons = np.arange(10, 200, 10)
second_layer_neurons = np.arange(10, 200, 10)
hidden_layer_sizes = list(product(first_layer_neurons, second_layer_neurons))

Output:

>>> hidden_layer_sizes
[(10, 10),
 (10, 20),
 (10, 30),
 (10, 40),
 (10, 50),
 ...
 (190, 150),
 (190, 160),
 (190, 170),
 (190, 180),
 (190, 190)]

Answered By - Corralien