[FIXED] Create a simple PyTorch neural network with a normalized weights

Issue

I want to create a simple PyTorch neural network with the sum of its weights equal to 1. To understand my question here is a to give an example:

Solution

You can simply normalize by the sum of all initialized weights:

>>> layer = nn.Linear(4, 1, bias=False)
>>> layer.weight
Parameter containing:
tensor([[-0.2565,  0.4753, -0.1129,  0.2327]], requires_grad=True)

Normalize layer.weight:

>>> layer.weight.data /= layer.weight.data.sum()

Then:

>>> layer.weight
Parameter containing:
tensor([[-0.7573,  1.4034, -0.3333,  0.6872]], requires_grad=True)

Answered By - Ivan