Issue
I have created a binary classification model. During training the model I am getting both loss and accuracy as constant. I tried the same code for other datasets and it is giving the same thing in every case. How to fix this? Here is the code
model=Model()
lossfn=nn.BCEWithLogitsLoss()
optimizer=torch.optim.Adam(model.parameters(),lr=0.01)
#training model
epochs=10
train_acc=[]
test_acc=[]
losses=torch.zeros(epochs)
#training over loops
for i in range(epochs):
model.train()
batchAcc=[]
batchLoss=[]
for X,y in train_loader:
#forward pass
yHat=model(X)
#lossfunction
loss=lossfn(y,yHat)
#backprop
optimizer.zero_grad()
loss.backward()
optimizer.step()
batchLoss.append(loss.item())
# predictions=(torch.sigmoid(yHat)>0.5).float()
predictions=yHat>0
batchLoss.append(loss.item())
batchAcc.append(100*torch.mean(((yHat>0) == y).float()))
losses[i]=np.mean(batchLoss)
train_acc.append(np.mean(batchAcc))
#Evaluation mode
model.eval()
X,y=next(iter(test_loader))
with torch.no_grad():
yHat=model(X)
test_acc.append( 100*torch.mean(((yHat>0) == y).float()))
print(f'Epoch:{i+1} loss:{loss.item()} train accuracy:{train_acc[-1]} test accuracy:{test_acc[-1]}')
Output of the code
Epoch:1 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Epoch:2 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Epoch:3 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Epoch:4 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Epoch:5 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Epoch:6 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Epoch:7 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Epoch:8 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Epoch:9 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Epoch:10 loss:0.6931471824645996 train accuracy:33.33646011352539 test accuracy:0.0
Solution
The other answers are wrong, the order of
optimizer.zero_grad() # Clears the gradient for all parameters
loss.backward() # Populates the gradient for all parameters
optimizer.step() # Uses the gradient to update the parameters
is correct!
Likely culprit is this line:
loss=lossfn(y,yHat)
In your case, y
is the ground truth and yHat
is the prediction (which is confusing in the usual convention, hence the mistake).
However, PyTorch losses expect the prediction first and the target as the second argument; the gradient is not propagated correctly through the targets. Therefore, you should change it to
loss=lossfn(yHat,y)
Or even better, call yHat
the ground truth target and y
the prediction.
Answered By - burzan
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.