[FIXED] Simple Neural Network in Pytorch with 3 inputs (Numerical Values)

Issue

Having a hard time setting up a neural network most of the examples are images. My problem has 3 inputs each of size N X M where N are the samples and M are the features. I have a separate file (CSV) with 1 x N binary target (0,1).

The network i'm trying to configure should have two hidden layers with 100 and 50 neurons, respectively. Sigmoid activation function and cross-entropy to check performance. The result should just be a single probability output.

Please help?

EDIT:

import torch
import torch.nn as nn
import torch.optim as optim
import torch.autograd as autograd 
import torch.nn.functional as F
#from torch.autograd import Variable
import pandas as pd

# Import Data
Input1 = pd.read_csv(r'...')
Input2 = pd.read_csv(r'...')
Input3 = pd.read_csv(r'...')
Target = pd.read_csv(r'...' )

# Convert to Tensor
Input1_tensor = torch.tensor(Input1.to_numpy()).float()
Input2_tensor = torch.tensor(Input2.to_numpy()).float()
Input3_tensor = torch.tensor(Input3.to_numpy()).float()
Target_tensor = torch.tensor(Target.to_numpy()).float()

# Transpose to have signal as columns instead of rows
input1 = Input1_tensor
input2 = Input2_tensor
input3 = Input3_tensor
y = Target_tensor

# Define the model
class Net(nn.Module):
    def __init__(self, num_inputs, hidden1_size, hidden2_size, num_classes):
        # Initialize super class
        super(Net, self).__init__()
        #self.criterion = nn.CrossEntropyLoss()
        
        # Add hidden layer 
        self.layer1 = nn.Linear(num_inputs,hidden1_size)
        # Activation
        self.sigmoid = torch.nn.Sigmoid()
        # Add output layer
        self.layer2 = nn.Linear(hidden1_size,hidden2_size)
        # Activation
        self.sigmoid2 = torch.nn.Sigmoid()
        self.layer3 = nn.Linear(hidden2_size, num_classes)

    def forward(self, x1, x2, x3):
        # implement the forward pass
     
        in1 = self.layer1(x1)
        in2 = self.layer1(x2)
        in3 = self.layer1(x3)
                      
        xyz = torch.cat((in1,in2,in3),1)

        return xyz

# Define loss function
loss_function = nn.CrossEntropyLoss()

# Define optimizer
optimizer = optim.SGD(model.parameters(), lr=1e-4)

for t in range(num_epochs):

    # Forward pass: Compute predicted y by passing x to the model
    y_pred = model(input1, input2, input3)

    # Compute and print loss
    loss = loss_function(y_pred, y)
    print(t, loss.item())

    # Zero gradients, perform a backward pass, and update the weights.
    optimizer.zero_grad()

    # Calculate gradient using backward pass
    loss.backward()

    # Update model parameters (weights)
    optimizer.step()

Here I am getting an error of " RuntimeError: 0D or 1D target tensor expected, multi-target not supported"

for line "loss = loss_function(y_pred, y)"

Where y_pred is [20000,375] and y is [20000,1]

Solution

you can refer to pytorch, a python library for deep learning and neural networks.

and you can use code that defines network below:

from torch import nn
import torch.nn.functional as F

def network(nn.Module):
    def __init__(self, M):
        # M is the dimension of input feature
        super(network, self).__init__()
        self.layer1 = nn.Linear(M, 100)
        self.layer2 = nn.Linear(100, 50)
        self.out = nn.Linear(50,1)

    def forward(self,x):
        return F.sigmoid(self.out(self.layer2(self.layer1(x))))

----------

You can then refer to the pytorch documentation and finish the rest training code.

Edit:

As for RuntimeError, you can squeeze the target tensor by y.squeeze(). This will remove redundant dimension in your tensor, e.g. [20000,1] -> [20000]

Answered By - Richardson