[FIXED] using cross validation for calculating specificity

Issue

I want to use cross-validation for calculating specificity. I found code for calculating accuracy, really, f1-score, and precision. but I couldn't found for specificity. for example, the code for f1-score is like:

cross_val_score(SVC, X, y, scoring="f1", cv = 7)

or for precision is like:

cross_val_score(SVC, X, y, scoring="precision", cv = 7)

Solution

The specifity is basically the True Negative Rate which is the same as the True Positive Rate (Recall) but for the negative class

If you have a binary class, you should do the following

• Import the metric recall_score from metrics (details here), and make_scorer function

  from sklearn.metrics import recall_score
  from sklearn.metrics import make_scorer
  

• Then you generate your new scorer, defining for which class you are calculating recall (by default, the recall is calculated on the label=1)

  specificity = make_scorer(recall_score, pos_label=0)
  

The label 0 is usually the negative class in a binary problem.

print(cross_val_score(classifier, X_train, y_train, cv=10, specificity))

if you want the recall (True positive rate) you can do the same changing the class

sensitivity = make_scorer(recall_score, pos_label=1)
print(cross_val_score(classifier, X_train, y_train, cv=10, sensitivity))

Anyway you can make your custom scorer, if you need something more complex

make_scorer

Answered By - Nikaido