[FIXED] Sklearn MLP Classifier Hyperparameter Optimization (RandomizedSearchCV)

Issue

I have the following parameters set up :

parameter_space = {
    'hidden_layer_sizes': [(sp_randint(100,600),sp_randint(100,600),), (sp_randint(100,600),)],
    'activation': ['tanh', 'relu', 'logistic'],
    'solver': ['sgd', 'adam', 'lbfgs'],
    'alpha': stats.uniform(0.0001, 0.9),
    'learning_rate': ['constant','adaptive']}

All the parameters except the hidden_layer_sizes is working as expected. However, fitting this RandomizedSearchCV model and displaying it's verbose text shows that it treats hidden_layer_sizes as :

hidden_layer_sizes=(<scipy.stats._distn_infrastructure.rv_frozen object and goes on to throw : TypeError: '<=' not supported between instances of 'rv_frozen' and 'int'

This result is obtained instead of the expected 1 or 2 layer MLP with hidden layer neurons between 100 and 600. Any ideas / other related tips?

Solution

sp_randint returns an instance of the rv_discrete class / a randint object, in order to generate random numbers, the correct syntax should be sp_randint.rvs(low, high, size). In order to make it work you need to define the parameter_space as below:

parameter_space = { 'hidden_layer_sizes': [(sp_randint.rvs(100,600,1),sp_randint.rvs(100,600,1),), (sp_randint.rvs(100,600,1),)], 'activation': ['tanh', 'relu', 'logistic'], 'solver': ['sgd', 'adam', 'lbfgs'], 'alpha': uniform(0.0001, 0.9), 'learning_rate': ['constant','adaptive']}

Answered By - Sandipan Dey