[FIXED] cant print sfs features selected in pipeline

Issue

I am selecting best features and then doing grid search. When finished, I want to print the best features that have been selected.

knn = KNeighborsRegressor()
sfs = SequentialFeatureSelector(knn,
                                scoring=custom_scorer,
                                n_features_to_select='auto',
                                tol=.01,
                                n_jobs=-1)

pipe = Pipeline([('sfs', sfs),
                    ('knn', knn)])

param_grid = {
    'sfs__estimator__n_neighbors': [4, 5, 6],
    'sfs__estimator__weights': ['uniform', 'distance'],
    'sfs__estimator__algorithm': ['ball_tree', 'kd_tree', 'brute'],
    'sfs__estimator__leaf_size': [15,30,60],
  }

gs = GridSearchCV(estimator=pipe,
                  param_grid=param_grid,
                  scoring=custom_scorer,
                  n_jobs=-1,
                  cv=cv,
                  refit=False)
gs = gs.fit(X, y)

When trying to print with

pipe.named_steps["sfs"].support_

I get the following error

'SequentialFeatureSelector' object has no attribute 'support_'

Ive also tried

pipe[1].support_

but have gotten an error.

Solution

The grid search clones its estimator before fitting, so your pipe itself remains unfitted. You can access the refitted-to-best-hyperparameters best_estimator_:

gs.best_estimator_.named_steps["sfs"].support_

Answered By - Ben Reiniger