[FIXED] tensorflow use input in loss function

Issue

i am using tensorflow/keras and i would like to use the input in the loss function

as per this answer here

Custom loss function in Keras based on the input data

I have created my loss function thusly

def custom_Loss_with_input(inp_1):
    def loss(y_true, y_pred):
        b = K.mean(inp_1)
        return y_true - b
    return loss
    

and set up the model with the layers and all ending like this

    model = Model(inp_1, x)    
    model.compile(loss=custom_Loss_with_input(inp_1), optimizer= Ada)   
    return model

Nevertheless, i get the following error:

TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.

Any advice on how to eliminate this error? Thanks in advance

Solution

You can use add_loss to pass external layers to your loss, in your case the input tensor.

Here an example:

def CustomLoss(y_true, y_pred, input_tensor):
    b = K.mean(input_tensor)
    return K.mean(K.square(y_true - y_pred)) + b

X = np.random.uniform(0,1, (1000,10))
y = np.random.uniform(0,1, (1000,1))

inp = Input(shape=(10,))
hidden = Dense(32, activation='relu')(inp)
out = Dense(1)(hidden)
target = Input((1,))
model = Model([inp,target], out)

model.add_loss( CustomLoss( target, out, inp ) )
model.compile(loss=None, optimizer='adam')
model.fit(x=[X,y], y=None, epochs=3)

If your loss is composed of different parts and you want to track them you can add different losses corresponding to the loss parts. In this way, the losses are printed at the end of each epoch and are stored in model.history.history. Remember that the final loss minimized during training is the sum of the various loss parts.

def ALoss(y_true, y_pred):
    return K.mean(K.square(y_true - y_pred))

def BLoss(input_tensor):
    b = K.mean(input_tensor)
    return b

X = np.random.uniform(0,1, (1000,10))
y = np.random.uniform(0,1, (1000,1))

inp = Input(shape=(10,))
hidden = Dense(32, activation='relu')(inp)
out = Dense(1)(hidden)
target = Input((1,))
model = Model([inp,target], out)

model.add_loss(ALoss( target, out ))
model.add_metric(ALoss( target, out ), name='a_loss')
model.add_loss(BLoss( inp ))
model.add_metric(BLoss( inp ), name='b_loss')
model.compile(loss=None, optimizer='adam')
model.fit(x=[X,y], y=None, epochs=3) 

To use the model in inference mode (removing the target from inputs):

final_model = Model(model.input[0], model.output)
final_model.predict(X)

Answered By - Marco Cerliani