[FIXED] GridSearchCV does not report scores on verbose mode

Issue

I am running a parameter grid with GridSearchCV on python 3.8.5 and sklearn 0.24.1:

grid_search = GridSearchCV(estimator=xg_clf, scoring=make_scorer(matthews_corrcoef), param_grid=param_grid, n_jobs=args.n_jobs, verbose = 3)

according to the documentation,

 |  verbose : int
 |      Controls the verbosity: the higher, the more messages.
 |  
 |      - >1 : the computation time for each fold and parameter candidate is
 |        displayed;
 |      - >2 : the score is also displayed;
 |      - >3 : the fold and candidate parameter indexes are also displayed
 |        together with the starting time of the computation.

setting verbose = 3, which I did, should print the Matthews Correlation Coefficient for each run.

However, the output is

Fitting 5 folds for each of 480 candidates, totalling 2400 fits
[CV 1/5] END colsample_bytree=0.8, gamma=0, learning_rate=0.7, max_depth=3, n_estimators=200, subsample=0.9; total time=   0.2s
[CV 2/5] END colsample_bytree=0.8, gamma=0, learning_rate=0.7, max_depth=3, n_estimators=200, subsample=0.9; total time=   0.2s
[CV 3/5] END colsample_bytree=0.8, gamma=0, learning_rate=0.7, max_depth=3, n_estimators=200, subsample=0.9; total time=   0.2s
[CV 4/5] END colsample_bytree=0.8, gamma=0, learning_rate=0.7, max_depth=3, n_estimators=200, subsample=0.9; total time=   0.2s
[CV 5/5] END colsample_bytree=0.8, gamma=0, learning_rate=0.7, max_depth=3, n_estimators=200, subsample=0.9; total time=   0.2s
[CV 1/5] END colsample_bytree=0.8, gamma=0, learning_rate=0.7, max_depth=3, n_estimators=200, subsample=0.95; total time=   0.2s

why isn't GridSearchCV printing the MCC for each run?

Maybe this is because I'm using a non-standard scorer?

Solution

I tried something similar to your code with a few different sklearn versions. As it turns out, version 0.24.1 does not print out the scores when verbose=3.

Here's my code and output with sklearn version 0.22.2.post1:

clf = XGBClassifier()
search = GridSearchCV(estimator=clf, scoring=make_scorer(matthews_corrcoef),
                      param_grid={'max_depth':[3, 4, 5]}, verbose=3)
search.fit(X, y)

> Fitting 5 folds for each of 3 candidates, totalling 15 fits
  [CV] max_depth=3 .....................................................
  [CV] ......................... max_depth=3, score=0.959, total=   0.2s

Here's my code and output with sklearn version 0.24.1:

clf = XGBClassifier()
search = GridSearchCV(estimator=clf, scoring=make_scorer(matthews_corrcoef),
                      param_grid={'max_depth':[3, 4, 5]}, verbose=3)
search.fit(X, y)

> Fitting 5 folds for each of 3 candidates, totalling 15 fits
  [CV 1/5] END ....................................max_depth=3; total time=   0.2s

In conclusion, you found a bug. Normally, I would recommend opening an issue on GitHub, but you'll be happy to know that version 0.24.2 does print the scores for each fold.

You can try pip install scikit-learn --upgrade or pip install scikit-learn==0.24.2 to fix this error.

Answered By - Arturo Sbr