[FIXED] Using the embedding layer as the input for an encoder

Issue

I want to use the embedding layer as the input for encoder, however I got an error as follow. My input y is a time series data with shape of 1*84. Could you please help me with that?

import numpy 
import torch.nn as nn

r_input    = torch.nn.Embedding(84, 10)  
activation  = nn.functional.relu
mu_r      = nn.Linear(10, 6)
log_var_r = nn.Linear(10, 6)

y = np.random.rand(1,84) 

def encode_r(y):
    y         = torch.reshape(y, (-1, 1, 84)) # torch.Size([batch_size, 1, 84])
    hidden    = torch.flatten(activation(r_input(y)), start_dim = 1)       
    z_mu      = mu_r(hidden)
    z_log_var = log_var_r(hidden)
    return z_mu, z_log_var```

Error: RuntimeError: Expected tensor for argument #1 'indices' to have one of the following scalar types: Long, Int; but got torch.cuda.FloatTensor instead (while checking arguments for embedding)

Solution

According to this thread: https://discuss.pytorch.org/t/expected-tensor-for-argument-1-indices-to-have-scalar-type-long-but-got-cpufloattensor-instead-while-checking-arguments-for-embedding/32441/4, it seems that one possible solution would be to ensure embeddings have integer values and not float values in them (by embeddings we mean the lookup table not an actual embedding vector).

Answered By - Timbus Calin