Issue
This is a toy example as I'm learning PyTorch and using it on one-dimensional time series, in this case a sine wave.
I'm trying to use Conv1d, but I get the following error:
RuntimeError: Given groups=1, weight of size [5, 1, 2], expected input[1, 994, 5] to have 1 channels, but got 994 channels instead
My 'lookback' is 5 time steps, and the shape of my data batch is [994, 5]
.
What am I doing wrong?
import torch;from torch.utils.data import Dataset, DataLoader
import torch.nn.functional as F;import pytorch_lightning as pl
from torch import nn, tensor
class TsDs(torch.utils.data.Dataset):
def __init__(self, s, l=5): super().__init__();self.l,self.s=l,s
def __len__(self): return self.s.shape[0] - 1 - self.l
def __getitem__(self, i): return self.s[i:i+self.l], torch.log(self.s[i+self.l+1]/self.s[i+self.l])
def plt(self): plt.plot(self.s)
class TsDm(pl.LightningDataModule):
def __init__(self, length=5000, batch_size=1000): super().__init__();self.batch_size=batch_size;self.s = torch.sin(torch.arange(length)*0.2) + 5
def train_dataloader(self): return DataLoader(TsDs(self.s[:3999]), batch_size=self.batch_size, shuffle=False)
def val_dataloader(self): return DataLoader(TsDs(self.s[4000:]), batch_size=self.batch_size)
dm = TsDm()
class MyModel(pl.LightningModule):
def __init__(self, learning_rate=0.01):
super().__init__();self.learning_rate = learning_rate
super().__init__();self.learning_rate = learning_rate
self.network = nn.Sequential(nn.Conv1d(1,5,2),nn.ReLU(),nn.Linear(5,3),nn.ReLU(),nn.Linear(3,1), nn.Tanh())
# self.network = nn.Sequential(nn.Linear(5,5),nn.ReLU(),nn.Linear(5,3),nn.ReLU(),nn.Linear(3,1), nn.Tanh())
def forward(self, x): return self.network(x)
def step(self, batch, batch_idx, stage):
x, y = batch
loss = -torch.mean(self(x)*y)
print(loss)
return loss
def training_step(self, batch, batch_idx): return self.step(batch, batch_idx, "train")
def validation_step(self, batch, batch_idx): return self.step(batch, batch_idx, "val")
def configure_optimizers(self): return torch.optim.SGD(self.parameters(), lr=self.learning_rate)
mm = MyModel(0.01);trainer = pl.Trainer(max_epochs=10)
trainer.fit(mm, datamodule=dm)
Solution
There are two issues in your code:
Looking at the documentation of
nn.Conv1d
, your input shape should be(B, C, L)
. In your default case, you haveL=5
, the sequence length, but you need to create that extra dimension representing the feature size of a sequence element, hereC=1
. You can do so by changingTsDs
's__getitem__
function to:def __getitem__(self, i): x = self.s[i:i+self.l] # minibatch x shaped (1, self.l) y = torch.log(self.s[i+self.l+1]/self.s[i+self.l]) # minibatch y shaped (1,) return x, y
Your convolutional layer has a stride of
1
and a size of2
, this means its output will be shaped(B, 5, L-1=4)
. The following layer is a fully connected layer instantiated asnn.Linear(5, 3)
, which means it expects(*, H_in=5)
and will output(*, H_out)
. You can eitherYou can flatten the conv1d output with
nn.Flatten
and feed it to a bigger fully connected layer (for instancenn.Linear(20, 3)
.You can use a convolutional layer with a wider kernel, if you use a kernel of
5
(your sequence length you will end up with a tensor of(B, 5, 1)
which you feed to ann.Linear(5, 3)
. Although this approach doesn't really scale whenL
is changed.You could apply a
nn.AvgPool1d
to get an average representation of the sequence after the convolutional layers have been applied.
Those are just a few directions...
Answered By - Ivan
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.