[FIXED] Best way to output prediction result for a test set from a model with 1 output (binary classification)

Issue

This seems simple, but I'm looking for the best way to output the prediction results for a model with 1 output (binary classification model). My labels are 0 and 1 in this example. Now I can just say if model output > 0.5 label is 1. But I'm sort of guessing this is correct. I am hence wondering if there is a better approach, like

prediction = np.argmax(y_hat.detach().numpy, axis=0) for multiple classes. 

Currently I'm doing this to test:

test = [[0,0],[0,1],[1,1],[1,0]]

for trial in test: 
  Xtest = torch.Tensor(trial)
  y_hat = logreg_clf(Xtest)

  if y_hat > 0.5:
    prediction = 1
  else: 
    prediction = 0
  # prediction = np.argmax(y_hat.detach().numpy, axis=0)
  print("{0} xor {1} = {2}".format(int(Xtest[0]), int(Xtest[1]), prediction))

Was also wondering if there is a function to automatically get the accuracy or confusion matrix for test set 'test'.

FYI my model:

import torch.nn as nn
import torch.nn.functional as F

class LogisticRegression(nn.Module):
  # input_size: Dimensionality of input feature vector.
  # num_classes: The number of classes in the classification problem.
  def __init__(self, input_size, num_classes):
    # Always call the superclass (nn.Module) constructor first!
    super(LogisticRegression, self).__init__()
    # Set up the linear transform
    self.linear = nn.Linear(input_size, num_classes)
    # I do not yet include the sigmoid activation after the linear 
    # layer because our loss function will include this as you will see later

  # Forward's sole argument is the input.
  # input is of shape (batch_size, input_size)
  def forward(self, x):
    # Apply the linear transform.
    # out is of shape (batch_size, num_classes). 
    out = self.linear(x)
    out = F.sigmoid(out)
    # Softmax the out tensor to get a log-probability distribution
    # over classes for each example.
    return out

# Binary classifiation
num_outputs = 1
num_input_features = 2

# Create the logistic regression model
logreg_clf = LogisticRegression(num_input_features, num_outputs)

print(logreg_clf)

import torch 
lr_rate = 0.001  # alpha

# training set of input X and labels Y
X = torch.Tensor([[0,0],[0,1], [1,0], [1,1]])
Y = torch.Tensor([0,1,1,0]).view(-1,1) #view is similar to numpy.reshape() here it makes it into a column

# Run the forward pass of the logistic regression model
sample_output = logreg_clf(X) #completely random at the moment
print(X)

loss_function = nn.BCELoss() 
# SGD: stochastic gradient descent is used to train/fit the model
optimizer = torch.optim.SGD(logreg_clf.parameters(), lr=lr_rate)


import numpy as np 
# from torch.autograd import Variable

#training loop:

epochs = 2001 #how many times we go through the training set
steps = X.size(0) #steps = 4; we have 4 training examples (I know, tiny training set :)

for i in range(epochs):
    for j in range(steps):
        # randomly sample from the training set:
        data_point = np.random.randint(X.size(0))
        # store the retrieved datapoint into 2 separate variables of the right shape
        x_var = torch.Tensor(X[data_point]).unsqueeze(0)
        y_var = torch.Tensor(Y[data_point])

        optimizer.zero_grad() # empty (zero) the gradient buffers
        y_hat = logreg_clf(x_var) #get the output from the model
        print(y_hat)
        print(y_var)
        loss = loss_function(y_hat, y_var) #calculate the loss
        loss.backward() #backprop
        optimizer.step() #does the update

    if i % 500 == 0:
        print ("Epoch: {0}, Loss: {1}, ".format(i, loss.data.numpy()))

Solution

test_data = torch.Tensor(test)
raw_preds = logreg_clf(test_data)
preds = (raw_preds > 0.5).long()

To get the predictions for all test data at once, we first convert the test data to a tensor, then we can make a forward pass with that tensor. The raw predictions are something like tensor([[0.4795], [0.4749], [0.4960], [0.5006]]. Then we apply the "risk neutral strategy" i.e. 0.5 as the threshold to get the results (which will be of type torch.bool so we convert it to long).

Equivalently in one line:

preds = (logreg_clf(torch.Tensor(test)) > 0.5).long()    

So the predictions are:

tensor([[0], [0], [0], [1]])

wondering if there is a function to automatically get the accuracy or confusion matrix

You can use respective functions from sklearn.metrics:

from sklearn.metrics import accuracy_score, confusion_matrix

# the ground truth for the given test data
truth = torch.Tensor([0, 1, 0, 1]).view(-1, 1) 

conf_mat = confusion_matrix(truth, preds)
acc = accuracy_score(truth, preds)

which gives the confusion matrix as

array([[2, 0],
       [1, 1]], dtype=int64)

and the accuracy as

0.75

So you have one false negative i.e. the second sample in the test data.

Answered By - Mustafa Aydın