[FIXED] How to translate a conv2D in keras or tensorflow which is already implemented in PyTorch?

Issue

I have the following function in pytorch implementation for replacing a conv2D layer with 3 different layers:

first_layer = torch.nn.Conv2d(in_channels=3, \
            out_channels=3, kernel_size=1,
            stride=1, padding=0, dilation = (1,1), bias=False)

core_layer = torch.nn.Conv2d(in_channels=3, \
            out_channels=16, kernel_size=(3,3),
            stride=(1,1), padding=(1,1), dilation=(1,1),
            bias=False)

last_layer = torch.nn.Conv2d(in_channels=16, \
                                 out_channels=64], kernel_size=1, stride=1,
                                 padding=0, dilation=(1,1), bias=True)

last_layer.bias.data = layer.bias.data

first_layer.weight.data = \
        torch.transpose(first, 1, 0).unsqueeze(-1).unsqueeze(-1)
last_layer.weight.data = last.unsqueeze(-1).unsqueeze(-1)
core_layer.weight.data = core

new_layers = [first_layer, core_layer, last_layer]
y = nn.Sequential(*new_layers)

where, 'first' represents a random 3 by 3 matrix. 'core' represents a tensor of shape [16,3,3,3] 'last' represents another random matrix of size (64,16).

When I tried to translate this into keras, I have the following :

first_layer = tf.keras.layers.SeparableConv2D(3, kernel_size=1, strides = (1,1), padding = 'same',  dilation_rate = (1,1), use_bias = False )

core_layer = tf.keras.layers.Conv2D(16, kernel_size=3, strides = (1,1), padding = (1,1), dilation_rate = (1,1), use_bias = False)

last_layer = tf.keras.layers.SeparableConv2D(64, kernel_size=1, strides = (1,1), \
                           padding = 'same', dilation_rate = (1,1), use_bias =True )

first_layer = tf.expand_dims(tf.expand_dims(tf.transpose(first, perm = [1,0]),0),0)
    
last_layer = tf.expand_dims(tf.expand_dims(last, 0),0)
core_layer = core
new_layers = [first_layer, core_layer, last_layer]

when I tried to get back the weights of the model in keras, I am getting a list with no weights at all. The convolution is not performed. Any idea on how to proceed further/ any other approached of transforming the above pytorch implementation to keras or tensorflow?

Solution

You are missing exactly the last step in Keras transformation.

There is also a Sequential() class in TensorFlow(Keras) that you can use to instantiate the model.

I haven't checked for the exact match between TF and PyTorch, but this should be your starting point to solve your problem.

model = tf.keras.Sequential([first_layer, core_layer, last_layer])
y = model(x)

Answered By - Timbus Calin