Issue
I am trying to define a custom rmse loss function for Keras. I wrote the function below to penalize the loss when the value of the data is less than 0.15 and otherwise.
import keras.backend as K
def custom_rmse(y_true, y_pred):
loss = K.square(y_pred - y_true)
for i in range(len(y_true)):
for j in range(y_true.shape[1]):
tmp = float(y_true[i][j])
if (tmp < 0.15):
loss[i][j] *= 0.2
else:
loss[i][j] *=0.8
loss = K.sqrt(K.sum(loss, axis=1))
return loss
But when I ran the model and attempted to fix it, I kept getting this error
/usr/local/lib/python3.7/dist-packages/keras/engine/training.py:853 train_function *
return step_function(self, iterator)
<ipython-input-95-efab27dd2563>:8 custom_rmse *
if (tmp < 0.15):
/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:1172 if_stmt
_tf_if_stmt(cond, body, orelse, get_state, set_state, symbol_names, nouts)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:1219 _tf_if_stmt
cond, aug_body, aug_orelse, strict=True)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:206 wrapper
return target(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py:549 new_func
return func(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/control_flow_ops.py:1254 cond
return cond_v2.cond_v2(pred, true_fn, false_fn, name)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/cond_v2.py:88 cond_v2
op_return_value=pred)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py:1007 func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:1197 aug_body
set_state(init_vars)
/tmp/tmp_3e6lmrw.py:35 set_state
(loss[i][j],) = vars_
TypeError: 'Tensor' object does not support item assignment
I will appreciate suggestions on how to fix this. Thanks.
Solution
If-Else statements are usually not the way to go for loss functions. Most of the time, it is better to do a "soft" way of what you are trying to achieve. This can be done by (for example) using a steep logistic function on your loss values in the following way:
def custom_rmse(y_true, y_pred):
loss = K.square(y_pred - y_true)
logistic_values = tf.sigmoid(1000 * (y_true - 0.15))
loss = logistic_values * loss * 0.8 + (1-logistic_values * loss * 0.2)
loss = K.sqrt(K.sum(loss, axis=1))
return loss
This code will do the following:
- We subtract 0.15 (your threshold) from your y_true so that the threshold for the new values now is at 0.
- We multiply the result by a high number (I selected 1000 here, the higher the number the steeper will the "soft threshold" be. This means, that all values higher than your threshold are now very high positive values and all values lower than your threshold will now be high negative values.
- We apply the sigmoid function to the resulting values. This function will be 1 for all high positive values and -0 for all high negative values (with a soft transition in between).
- Now, we can just multiply our loss by logistic_values or 1-logistic_values, which basically acts as a mask that masks out all values that are 0 or 1 respectively. All the values that are not masked out can now be multiplied by their respective factor of 0.8 or 0.2.
Answered By - Marc Felix
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.