[FIXED] xgboost.cv gives TypeError: 'StratifiedKFold' object is not iterable

Issue

I have been trying to implement this code in python 2.7. It gives me this error. I would appreciate help. I have latest version of sklearn(0.18.1) and xgboost(0.6)

import xgboost as xgb
from sklearn.model_selection import StratifiedKFold
from sklearn.metrics import f1_score, roc_auc_score, confusion_matrix

nfold = 3
kf = StratifiedKFold(nfold, shuffle=True)

dtrain = xgb.DMatrix(x_train, label=y_train)
dtest = xgb.DMatrix(x_test)

params = {
    'objective' : 'binary:logistic',
    'eval_metric': 'auc',
    'min_child_weight':10,
    'scale_pos_weight':scale,
}
hist = xgb.cv(params, dtrain, num_boost_round=10000, folds=kf, early_stopping_rounds=50, as_pandas=True, verbose_eval=100, show_stdv=True, seed=0)

I get this error:

TypeErrorTraceback (most recent call last)
<ipython-input-52-41c415e116d7> in <module>()
      5     'scale_pos_weight':scale,
      6 }
----> 7 hist = xgb.cv(params, dtrain, num_boost_round=10000, folds=kf, early_stopping_rounds=50, as_pandas=True, verbose_eval=100, show_stdv=True, seed=0)
      8 
      9 

/opt/conda/lib/python2.7/site-packages/xgboost/training.pyc in cv(params, dtrain, num_boost_round, nfold, stratified, folds, metrics, obj, feval, maximize, early_stopping_rounds, fpreproc, as_pandas, verbose_eval, show_stdv, seed, callbacks)
    369 
    370     results = {}
--> 371     cvfolds = mknfold(dtrain, nfold, params, seed, metrics, fpreproc, stratified, folds)
    372 
    373     # setup callbacks

/opt/conda/lib/python2.7/site-packages/xgboost/training.pyc in mknfold(dall, nfold, param, seed, evals, fpreproc, stratified, folds)
    236         idset = [randidx[(i * kstep): min(len(randidx), (i + 1) * kstep)] for i in range(nfold)]
    237     elif folds is not None:
--> 238         idset = [x[1] for x in folds]
    239         nfold = len(idset)
    240     else:

TypeError: 'StratifiedKFold' object is not iterable

Solution

Inside the xgb.cv function, try to replace

folds=kf

with

folds=list(kf.split(x_train,y_train))

The split method is applied in order to get the split into training and validation. We then convert it into a list so that it would be an iterable object.

If that doesn't work, try without the list. That is:

folds=kf.split(x_train,y_train)

Answered By - Miriam Farber