[FIXED] loss function with pytorch

Issue

I'm new to pytorch, when I see tutorials with MNIST dataset the target is a scalar (a digit from 0 to 9) and the output of the model is a layer is a vector (the code of the last layer is nn.Linear(32,10)) and they calculte the loss with (loss=nn.CrossEntropyLoss() loss = loss(output,target) ) are they compareing digit with a vector ?

Solution

I think that according to the PyTorch documentation torch.nn.functional.cross_entropy() that the output is like you mentioned a tensor with shape (N,C) (N is batch size) and C is the number of classes, where the target is either shape (N) when containing only class indices and shape (N,C) when containing also class probabilities. Details about how they compute the actual cross entropy is mentioned here (pytorch docs). So yes they are comparing a digit to a tensor in the sense that the digit tries to show which index should be 1 and therefore the other ones 0.

Answered By - Jack897