[FIXED] Label Smoothing in PyTorch - Using BCE loss -> doing it with the data itself

Issue

i am doing a classification task (binary) in PyTorch, so with labels 0 und 1. No I want introduce label smoothing as another regularization technique. Because I Use the ice loss, there is no such function to use label smoothing as in the cross entropy loss (for man than 0,1). Now I am considering to implement it not in the loss but in the data itself. Would it be right to just replace my y_true to for example 0->0.1 and 1->0.9 before they go into the loss?

Solution

You can replace the 0 with 0.1 and 1 with 0.9 if label smoothing is 0.1

criterion(disc_fake_pred, torch.zeros_like(disc_fake_pred)+0.1) #0.1
criterion(disc_real_pred, torch.ones_like(disc_real_pred)-0.1)  #0.9

Answered By - Mohamed Fathallah