[FIXED] TypeError: object of type 'Tensor' has no len() when using a custom metric in Tensorflow

Issue

I am developing a model for multi-class classification problem ( 4 classes) using Keras with Tensorflow backend. The values of y_test have 2D format:

0 1 0 0
0 0 1 0
0 0 1 0

This is the function that I use to calculate a balanced accuracy:

def my_metric(targ, predict):
    val_predict = predict
    val_targ = tf.math.argmax(targ, axis=1)
    return metrics.balanced_accuracy_score(val_targ, val_predict)

And this is the model:

hidden_neurons = 50
timestamps = 20
nb_features = 18

model = Sequential()

model.add(LSTM(
                units=hidden_neurons,
                return_sequences=True, 
                input_shape=(timestamps,nb_features),
                dropout=0.15
                #recurrent_dropout=0.2
              )
         )

model.add(TimeDistributed(Dense(units=round(timestamps/2),activation='sigmoid')))

model.add(Dense(units=hidden_neurons,
               activation='sigmoid'))


model.add(Flatten())

model.add(Dense(units=nb_classes,
               activation='softmax'))

model.compile(loss="categorical_crossentropy",
              metrics = [my_metric],
              optimizer='adadelta')

When I run this code, I get this error:

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) in () 30 model.compile(loss="categorical_crossentropy", 31 metrics = [my_metric], #'accuracy', ---> 32 optimizer='adadelta')

~/anaconda3/lib/python3.6/site-packages/keras/engine/training.py in compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, weighted_metrics, target_tensors, **kwargs) 449 output_metrics = nested_metrics[i] 450 output_weighted_metrics = nested_weighted_metrics[i] --> 451 handle_metrics(output_metrics) 452 handle_metrics(output_weighted_metrics, weights=weights) 453

~/anaconda3/lib/python3.6/site-packages/keras/engine/training.py in handle_metrics(metrics, weights) 418 metric_result = weighted_metric_fn(y_true, y_pred, 419 weights=weights, --> 420 mask=masks[i]) 421 422 # Append to self.metrics_names, self.metric_tensors,

~/anaconda3/lib/python3.6/site-packages/keras/engine/training_utils.py in weighted(y_true, y_pred, weights, mask) 402 """ 403 # score_array has ndim >= 2 --> 404 score_array = fn(y_true, y_pred) 405 if mask is not None: 406 # Cast the mask to floatX to avoid float64 upcasting in Theano

in my_metric(targ, predict) 22 val_predict = predict 23 val_targ = tf.math.argmax(targ, axis=1) ---> 24 return metrics.balanced_accuracy_score(val_targ, val_predict) 25 #return 5 26

~/anaconda3/lib/python3.6/site-packages/sklearn/metrics/classification.py in balanced_accuracy_score(y_true, y_pred, sample_weight, adjusted)
1431 1432 """ -> 1433 C = confusion_matrix(y_true, y_pred, sample_weight=sample_weight) 1434 with np.errstate(divide='ignore', invalid='ignore'): 1435
per_class = np.diag(C) / C.sum(axis=1)

~/anaconda3/lib/python3.6/site-packages/sklearn/metrics/classification.py in confusion_matrix(y_true, y_pred, labels, sample_weight) 251 252 """ --> 253 y_type, y_true, y_pred = _check_targets(y_true, y_pred) 254 if y_type not in ("binary", "multiclass"): 255 raise ValueError("%s is not supported" % y_type)

~/anaconda3/lib/python3.6/site-packages/sklearn/metrics/classification.py in _check_targets(y_true, y_pred) 69 y_pred : array or indicator matrix 70 """ ---> 71 check_consistent_length(y_true, y_pred) 72 type_true = type_of_target(y_true) 73 type_pred = type_of_target(y_pred)

~/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py in check_consistent_length(*arrays) 229 """ 230 --> 231 lengths = [_num_samples(X) for X in arrays if X is not None] 232 uniques = np.unique(lengths) 233 if len(uniques) > 1:

~/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py in (.0) 229 """ 230 --> 231 lengths = [_num_samples(X) for X in arrays if X is not None] 232 uniques = np.unique(lengths) 233 if len(uniques) > 1:

~/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py in _num_samples(x) 146 return x.shape[0] 147 else: --> 148 return len(x) 149 else: 150 return len(x)

TypeError: object of type 'Tensor' has no len()

Solution

You cannot call a sklearn function on a Keras tensor. You'll need to implement the functionality yourself using Keras' backend functions, or TensorFlow functions if you are using the TF backend.

The balanced_accuracy_score is defined as the average of the recall obtained in each column. Check this link for implementations of precision and recall. As for the balanced_accuracy_score, you can implement it as follows:

import keras.backend as K

def balanced_recall(y_true, y_pred):
    """
    Computes the average per-column recall metric
    for a multi-class classification problem
    """ 
    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)), axis=0)  
    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)), axis=0)   
    recall = true_positives / (possible_positives + K.epsilon())    
    balanced_recall = K.mean(recall)
    return balanced_recall

Answered By - sdcbr