[FIXED] How to make the sum of output to 1

Issue

My (PyTorch) sum of model’s output isn’t 1. And this is the structure of model.

LSTM(4433, 64)
LSTM(64, 64)
Linear(64, 4433)
Sigmoid()

And this is the predicted output of the model. Input

[1, 0, 0, …, 0, 0]

Output

[.7842, .5, .5, …, .5, .5]

Do you know any function that can make its sum 1?

Solution

Sigmoid activation function maps every input to a value between [0, 1], without taking into account other elements in the input vector. However, Softmax does a similar transformation but the output vector sums 1.

TL;DR: use softmax instead of sigmoid.

Answered By - Ignacio Aristimuño