[FIXED] Output of model after serving different with keras model output

Issue

I have a model with input that has a shape like (145, 24, 24, 3)

With model load by tensorflow keras output will have a shape like (145, 4)

But when I convert input from tensor to list and POST it into model serving.

Output return (,4)

I was using tensorflow-serving with docker

My code:

        img_boxes = get_image_boxes(bboxes, img, height, width, num_boxes, size=24)
        img_in = tf.make_tensor_proto(img_boxes)
        img_in = tf.make_ndarray(img_in)
        print(img_in.shape) # (145, 24, 24, 3)

        # probs, offsets = self.rnet(img_boxes)
        payload = {'instances': img_in.tolist()}
        res = requests.post('http://localhost:8501/v1/models/r_net:predict', data=payload)
        res = res.json()['predictions'][0]
        probs = tf.convert_to_tensor(res['softmax'])
        offsets = tf.convert_to_tensor(res['dense5_2'])
        print(probs.shape) # (, 4)

And my model:

    __________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_1 (InputLayer)           [(None, 24, 24, 3)]  0           []                               
                                                                                                  
 permute (Permute)              (None, 24, 24, 3)    0           ['input_1[0][0]']                
                                                                                                  
 conv1 (Conv2D)                 (None, 22, 22, 28)   784         ['permute[0][0]']                
                                                                                                  
 prelu1 (PReLU)                 (None, 22, 22, 28)   28          ['conv1[0][0]']                  
                                                                                                  
 pool1 (MaxPooling2D)           (None, 11, 11, 28)   0           ['prelu1[0][0]']                 
                                                                                                  
 conv2 (Conv2D)                 (None, 9, 9, 48)     12144       ['pool1[0][0]']                  
                                                                                                  
 prelu2 (PReLU)                 (None, 9, 9, 48)     48          ['conv2[0][0]']                  
                                                                                                  
 pool2 (MaxPooling2D)           (None, 4, 4, 48)     0           ['prelu2[0][0]']                 
                                                                                                  
 conv3 (Conv2D)                 (None, 3, 3, 64)     12352       ['pool2[0][0]']                  
                                                                                                  
 prelu3 (PReLU)                 (None, 3, 3, 64)     64          ['conv3[0][0]']                  
                                                                                                  
 flatten (Flatten)              (None, 576)          0           ['prelu3[0][0]']                 
                                                                                                  
 dense4 (Dense)                 (None, 128)          73856       ['flatten[0][0]']                
                                                                                                  
 prelu4 (PReLU)                 (None, 128)          128         ['dense4[0][0]']                 
                                                                                                  
 dense5_1 (Dense)               (None, 2)            258         ['prelu4[0][0]']                 
                                                                                                  
 softmax (Softmax)              (None, 2)            0           ['dense5_1[0][0]']               
                                                                                                  
 dense5_2 (Dense)               (None, 4)            516         ['prelu4[0][0]']                 
                                                                                                  
==================================================================================================
Total params: 100,178
Trainable params: 100,178
Non-trainable params: 0
__________________________________________________________________________________________________

Solution

My mistake was I did not get all predictions in output

res = res.json()['predictions'][0]

it should be:

res = res.json()['predictions']

, output is an array dictionary I need to concat them by the same keys I will get output as I want. {'predictions': [{'softmax': [0.210848287, 0.789151728], 'dense5_2': [0.182407588, 0.159706563, 0.113356635, 0.401812047]}, {'softmax': [0.993124425, 0.0068756], 'dense5_2': [0.138127923, -0.0996344835, -0.020740509, -0.0633568913]}, {'softmax': [0.958264828, 0.0417351499], 'dense5_2': [0.0111884885, 0.0999818072, -0.130534053, 0.0480072]}, {'softmax': [0.983752847, 0.0162471626], 'dense5_2': [-0.0930862129, -0.224962771, 0.0652338713, 0.205662221]}, {'softmax': [0.947032034, 0.0529679693], 'dense5_2': [0.242067963, 0.0784763768, 0.0628610402, 0.109089941]}, {'softmax': [0.964857459, 0.0351425521], 'dense5_2': [0.210765839, 0.110972457, 0.0283354521, 0.210187465]},...]}

Answered By - Dat Le