[FIXED] Cannot verify manually the calculations by tf.keras.losses.BinaryCrossentropy

Issue

https://www.tensorflow.org/api_docs/python/tf/keras/losses/BinaryCrossentropy

The example in tensorflow site.

y_true = [0, 1, 0, 0]
y_pred = [-18.6, 0.51, 2.94, -12.8]
bce = tf.keras.losses.BinaryCrossentropy(from_logits=True)
bce(y_true, y_pred).numpy()
0.865

as far as i know BinaryCrossentropy is

loss = y_true * log(y_pred) + (1 - y_true) * log(1 - y_pred)

0 * log(2.94) + (1 - 0) * log(1 - 2.94)

log(1 - 2.94)

log(-1.94) ????

it is impossible to calculate

Solution

Yes, you are right, but notice in the source code, y_pred or output in that case is clipped to be between a very small epsilon value and 1 minus this epsilon value 1e-7:

...
  output = tf.clip_by_value(output, epsilon_, 1. - epsilon_)

  # Compute cross entropy from probabilities.
  bce = target * tf.math.log(output + epsilon())
  bce += (1 - target) * tf.math.log(1 - output + epsilon())
...

Answered By - AloneTogether