[FIXED] custom padding in convolution layer of tensorflow 2.0

Issue

In Pytorch, nn.conv2d()'s padding parameter allows a user to enter the padding size of choice(like p=n). There is no such equivalent for TensorFlow. How can we achieve similar customization?. Would be much appreciated if a small network is designed, using the usual CNN layers like pooling and FC, to demonstrate how to go about it, starting from the input layer.

Solution

You can use tf.pad followed by a convolution with no ("valid") padding. Here's a simple example:

inp = tf.keras.Input((32, 32, 3)) # e.g. CIFAR10 images
custom_padded = tf.pad(inp, ((0, 0), (2, 0), (2, 0), (0, 0)))
conv = tf.keras.layers.Conv2D(16, 3)(custom_padded)  # default padding is "valid"

model = tf.keras.Model(inp, conv)

The syntax for padding can take some getting used to, but basically each 2-tuple stands for a dimension (batch, width, height, filters), and within each 2-tuple the first number is how many elements to pad in front, and the second one how many to pad in the back. So in this case:

• no padding in the batch axis
• 2 elements on the left, 0 elements on the right for the width axis
• 2 elements on top, 0 elements on the bottom for the height axis
• no padding in the channel axis

In this example, we are using 16 filters with a filter size of 3. Normally this would require a padding of 1 element on each side to achieve "same" padding, but here we decide to pad 2 elements on one side and 0 on the other. This can of course be adapted to any other scheme you want/need.

This uses 0-padding by default, but you can change that in the pad function. See the docs: https://www.tensorflow.org/api_docs/python/tf/pad

Note that I left out pooling or other layers because this should be simple to add. The basic recipe is just to replace the convolution layer by pad plus convolution with no padding.

Answered By - xdurch0