[FIXED] What is the role of pytorch dataset.imagefolder's loader option?

Issue

Loader option

What is the role of PyTorch dataset.imagefolder's loader option?

Actually, now I have a 10,000 images in a folder, but I want to use only 100 images. Now I load all the image and subset dataset by index.

from torch.utils.data import Subset
a = dset.ImageFolder(root=path_F,transform=transform)
index = list(range(0,100))
b = Subset(a,index)

Can loader option make my code simple?

Solution

loader is responsible for loading the image given path. By default PyTorch uses pillow and it's Image.open(path) functionality see docs.

You can specify custom loading like this (load and rotate by 45 degrees):

import torchvision
from PIL import Image


def loader(path):
    return Image.open(path).rotate(45)


dataset = torchvision.datasets.ImageFolder("images", loader=loader)

So no, you shouldn't use it for choosing images.

You could, in principle, use is_valid_file argument to do it like this though:

class Chooser:
    def __init__(self):
        self._counter = -1

    def __call__(self, path):
        self._counter += 1
        return self._counter < 100


dataset = torchvision.datasets.ImageFolder("images", is_valid_file=Chooser())

I would highly discourage this approach though as the intent isn't immediately clear and it will go through whole dataset checking each image. Your current way is the way to go.

Answered By - Szymon Maszke