[FIXED] Fine-tune Pytorch I3D model on a custom dataset

Issue

I want to fine-tune the I3D model from torch hub, which is pre-trained on Kinetics 400 classes, on a custom dataset, where I have 4 possible output classes.

I'm loading the model by:

model = torch.hub.load("facebookresearch/pytorchvideo", i3d_r50, pretrained=True)

I printed it, and saw this layer:

(6): ResNetBasicHead(
      (pool): AvgPool3d(kernel_size=(4, 7, 7), stride=(1, 1, 1), padding=(0, 0, 0))
      (dropout): Dropout(p=0.5, inplace=False)
      (proj): Linear(in_features=2048, out_features=400, bias=True)
      (output_pool): AdaptiveAvgPool3d(output_size=1)

So, I tried:

model = torch.hub.load("facebookresearch/pytorchvideo", i3d_r50, pretrained=True)
num_classes = 4
model.ResNetBasicHead.proj = torch.nn.Linear(model.ResNetBasicHead.proj.in_features, num_classes)

but I'm getting the error:

AttributeError: 'Net' object has no attribute 'ResNetBasicHead'

What's the proper way to do this?

Solution

If I understand you correctly, you want to change the blocks of your net by appending a new block, namely a linear layer.

You can do this by appending a new block i.e.

model.blocks.add_module("linear", torch.nn.Linear(model.blocks[6].proj.in_features, num_classes))

The model then looks like this:

(6): ResNetBasicHead(
  (pool): AvgPool3d(kernel_size=(4, 7, 7), stride=(1, 1, 1), padding=(0, 0, 0))
  (dropout): Dropout(p=0.5, inplace=False)
  (proj): Linear(in_features=2048, out_features=400, bias=True)
  (output_pool): AdaptiveAvgPool3d(output_size=1)
)
(7): Linear(in_features=2048, out_features=4, bias=True)

Or if you want to add it to your ResNetBasicHead you can do it like this:

model.blocks[6].add_module("linear", torch.nn.Linear(model.blocks[6].proj.in_features, num_classes))

which yields:

  (6): ResNetBasicHead(
  (pool): AvgPool3d(kernel_size=(4, 7, 7), stride=(1, 1, 1), padding=(0, 0, 0))
  (dropout): Dropout(p=0.5, inplace=False)
  (proj): Linear(in_features=2048, out_features=400, bias=True)
  (output_pool): AdaptiveAvgPool3d(output_size=1)
  (linear): Linear(in_features=2048, out_features=4, bias=True)
)

Answered By - rot8