[FIXED] Is Scikit-Learn's classification report supposed to show samples with 0 support?

Issue

I'm running a text classification task and am using sklearn.metrics.classification_report. I've noticed that there are many cases where labels with supports of 0 are also shown, but is this expected behavior?

To give a specific example, let's say that the task I have at hand has five labels 0, 1, 2, 3, and 4. I wanted to make sure that my model is able to train properly on label 0 and so I created a separate dataset containing only those labels. The classification report after the first epoch looks like this:

              precision    recall  f1-score   support

           0     1.0000    0.8929    0.9434      9713
           1     0.0000    0.0000    0.0000         0
           2     0.0000    0.0000    0.0000         0
           3     0.0000    0.0000    0.0000         0

    accuracy                         0.8929      9713
   macro avg     0.2500    0.2232    0.2359      9713
weighted avg     1.0000    0.8929    0.9434      9713

As far as I know (and please correct me if I'm wrong), the labels shown on the left column are labels contained in y_true. So not only is the report not supposed to show 1, 2, and 3, but it's also not showing 4 which it should if the behavior's supposed to be consistent.

I've checked my data and made sure that it only contains 0 so it's not a data issue, and I've also double checked to make sure I'm not swapping y_true and y_pred when calling the function.

Why does this behavior happen? And is there a way to fix it?

Solution

The labels on the left are the predicted labels. In other words, your model still predicts classes 1, 2 and 3 (but not 4) in some (about 11%) cases where it should ideally predict 0.

Edit: to be precise, classification_report() runs sklearn.utils.multiclass.unique_labels() on both y_true and y_pred (unless you specify a label list explicitly), which is the same as predicted labels in this case.

The rest just wraps precision_recall_fscore_support(). The support is the number of occurrences of each class in y_true.

Answered By - dx2-66