[FIXED] PyTorch - Getting the 'TypeError: pic should be PIL Image or ndarray. Got <class 'numpy.ndarray'>' error

Issue

I am getting the error TypeError: pic should be PIL Image or ndarray. Got <class 'numpy.ndarray'> when I try to load a non-image dataset through the DataLoader. The versions of torch and torchvision are 1.0.1, and 0.2.2.post3, respectively. Python's version is 3.7.1 on a Windows 10 machine.

Here is the code:

class AndroDataset(Dataset):
    def __init__(self, csv_path):
        self.transform = transforms.Compose([transforms.ToTensor()])

        csv_data = pd.read_csv(csv_path)

        self.csv_path = csv_path
        self.features = []
        self.classes = []

        self.features.append(csv_data.iloc[:, :-1].values)
        self.classes.append(csv_data.iloc[:, -1].values)

    def __getitem__(self, index):
        # the error occurs here
        return self.transform(self.features[index]), self.transform(self.classes[index]) 

    def __len__(self):
        return len(self.features)

And I set the loader:

training_data = AndroDataset('android.csv')
train_loader = DataLoader(dataset=training_data, batch_size=batch_size, shuffle=True)

Here is the full error stack trace:

Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm 2018.1.2\helpers\pydev\pydevd.py", line 1758, in <module>
    main()
  File "C:\Program Files\JetBrains\PyCharm 2018.1.2\helpers\pydev\pydevd.py", line 1752, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Program Files\JetBrains\PyCharm 2018.1.2\helpers\pydev\pydevd.py", line 1147, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2018.1.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/talha/Documents/PyCharmProjects/DeepAndroid/deep_test_conv1d.py", line 231, in <module>
    main()
  File "C:/Users/talha/Documents/PyCharmProjects/DeepAndroid/deep_test_conv1d.py", line 149, in main
    for i, (images, labels) in enumerate(train_loader):
  File "C:\Users\talha\Documents\PyCharmProjects\DeepAndroid\venv\lib\site-packages\torch\utils\data\dataloader.py", line 615, in __next__
    batch = self.collate_fn([self.dataset[i] for i in indices])
  File "C:\Users\talha\Documents\PyCharmProjects\DeepAndroid\venv\lib\site-packages\torch\utils\data\dataloader.py", line 615, in <listcomp>
    batch = self.collate_fn([self.dataset[i] for i in indices])
  File "C:/Users/talha/Documents/PyCharmProjects/DeepAndroid/deep_test_conv1d.py", line 102, in __getitem__
    return self.transform(self.features[index]), self.transform(self.classes[index])
  File "C:\Users\talha\Documents\PyCharmProjects\DeepAndroid\venv\lib\site-packages\torchvision\transforms\transforms.py", line 60, in __call__
    img = t(img)
  File "C:\Users\talha\Documents\PyCharmProjects\DeepAndroid\venv\lib\site-packages\torchvision\transforms\transforms.py", line 91, in __call__
    return F.to_tensor(pic)
  File "C:\Users\talha\Documents\PyCharmProjects\DeepAndroid\venv\lib\site-packages\torchvision\transforms\functional.py", line 50, in to_tensor
    raise TypeError('pic should be PIL Image or ndarray. Got {}'.format(type(pic)))
TypeError: pic should be PIL Image or ndarray. Got <class 'numpy.ndarray'>

Solution

Expanding on @MiriamFarber's answer, you cannot use transforms.ToTensor() on numpy.ndarray objects. You can convert numpy arrays to torch tensors using torch.from_numpy() and then cast your tensor to the required datatype.

---

Eg:

>>> import numpy as np
>>> import torch
>>> np_arr = np.ones((5289, 38))
>>> torch_tensor = torch.from_numpy(np_arr).long()
>>> type(np_arr)
<class 'numpy.ndarray'>
>>> type(torch_tensor)
<class 'torch.Tensor'>

Answered By - Vishnu Dasu