[FIXED] How to compute different metric values in cross validation?

Issue

I am using scikit-lear for classification. I am using cross validation for finding effectiveness of the classification algorithm. I want to compute accuracy, precision, recall, F1 measure. Currently I am using following code.

           dt  =  DecisionTreeClassifier(max_depth=dt_est)
           dt_acc = cross_validation.cross_val_score(dt,x_data_tfidf.toarray(), target_arr, cv=cv,                  scoring='accuracy')
           dt_f1 = cross_validation.cross_val_score(dt,x_data_tfidf.toarray(), target_arr, cv=cv,scoring='f1')
           dt_pre = cross_validation.cross_val_score(dt,x_data_tfidf.toarray(), target_arr, cv=cv, scoring='precision')     
           dt_re = cross_validation.cross_val_score(dt,x_data_tfidf.toarray(), target_arr, cv=cv, scoring='recall')    
 

Is there any way using which I can get all (accuracy, precision, recall, f1) in single computation? Currently I have to compute all the metric individually.

Solution

Unfortunately this is not currently possible out of the box, but we are working on it. You can define your own scoring object that computes all of them and prints them / stores them somewhere.

FYI an unfinished PR is here.

Answered By - Andreas Mueller