[FIXED] How to evaluate my MLPClassifier model? Is ROC curve needed?

Issue

How to evaluate my MLPClassifier model? Is confusion matrix, accuracy, classification report enough? Do i need ROC for evaluating my MLPClassifier result? And aside from that how can i plot loss for test and training set, i used loss_curve function but it only show the loss plot for training set.

Ps. I'm dealing with multi-class classification problem.

Solution

This is a very open question and with no code, so I will answer you with what I think is best. Usually for multi-label classification problem it is standard to use accuracy as a measure to track training. Another good measure is called f1-score. Sklearn's classification_report is a very good method to track training.

Confusion matrices come after you train the model. They are used to check where the model is failing by evaluating which classes are harder to predict.

ROC curves are, usually, for binary classification problems. They can be adapted to multi-class by doing a one class vs the rest approach.

For the losses, it seems to me you might be confusing things. Training takes place over epochs, testing does not. If you train over 100 epochs, then you have 100 values for the loss to plot. Testing does not use epochs, at most it uses batches, therefore plotting the loss does not make sense. If instead you are talking about validation data, then yes you can plot the loss just like with the training data.

Answered By - DPM