[FIXED] Meaning of **kwargs attribute put in classes of Machine Learning Models

Issue

I am wondering about the meaning of the attribute **kwargs that I've found typically added in the constructor of some machine learning models classes. For example considering a neural network in PyTorch:

class Model(nn.Module):

def __init__(self, input_dim, hidden_dim, output_dim, **kwargs)

is **kwargs associated with extra parameters that are defined later?

Solution

This is not specific to machine learning models classes, it is rather a Python feature.

You are indeed right, it corresponds to additional keyword arguments. It will essentially collect the remaining passed named argument, that are not defined in the function header, and add them to the dictionary variable kwargs. This variable can actually be renamed to any name, it is customary to keep 'args' for iterable unnamed arguments (*args) and 'kwargs' for keyword arguments (**kwargs).

This adds the flexibility to allow for additional arguments to be defined and passed to the function without having to specifically state their names in the header. One common use case is when extending a class. Here we are implementing a dummy 3x3 2D convolution layer named Conv3x3, which will extends the base nn.Conv2d module:

class Conv3x3(nn.Conv2d):
    def __init__(self, **kwargs):
       super().__init__(kernel_size=3, **kwargs)

As you can see, we didn't need to name all arguments and we still keep the same interface as nn.Conv2d in our Conv3x3 class initializer:

>>> Conv3x3(in_channels=3, out_channels=16)
Conv3x3(3, 16, kernel_size=(3, 3), stride=(1, 1))

---

There are a lot of nice things you can do with these two constructs. Much of which you can find on here.

Answered By - Ivan