[FIXED] Sklearn: Is there a way to define a specific score type to pipeline?

Issue

I can do this:

model=linear_model.LogisticRegression(solver='lbfgs',max_iter=10000)
kfold = model_selection.KFold(n_splits=number_splits,shuffle=True, random_state=random_state)
scalar = StandardScaler()
pipeline = Pipeline([('transformer', scalar), ('estimator', model)])
results = model_selection.cross_validate(pipeline, X, y, cv=kfold, scoring=score_list,return_train_score=True)

where score_list can be something like ['accuracy','balanced_accuracy','precision','recall','f1'].

I also can do this:

kfold = model_selection.KFold(n_splits=number_splits,shuffle=True, random_state=random_state)
scalar = StandardScaler()
pipeline = Pipeline([('transformer', scalar), ('estimator', model)])
for i, (train, test) in enumerate(kfold.split(X, y)):
    pipeline.fit(self.X[train], self.y[train])
    pipeline.score(self.X[test], self.y[test])

However, I am not able to change the score type for pipeline in the last line. How can I do that?

Solution

score method is always accuracy for classification and r2 score for regression. There is no parameter to change that. It comes from the Classifiermixin and RegressorMixin.

Instead, when we need other scoring options, we have to import it from sklearn.metrics like the following.

from sklearn.metrics import balanced_accuracy

y_pred = pipeline.predict(self.X[test])
balanced_accuracy(self.y_test, y_pred)

Answered By - Venkatachalam