[FIXED] Does image_dataset_from_directory load all the images into memory at once?

Issue

I'm new to machine learning, and I am trying to create an image classifier, I want to load the dataset, but I want to do it in a way such that it does not take up all of my memory. Reading the tensorflow documentation, it says that iteration of a dataset happens in streaming fashion, and I am wondering if tf.keras.preprocessing.image_dataset_from_directory will load the images at once or "stream" it a batch at a time. If not I was thinking of making a generator to read file names one at a time and load them when the batches are ready with keras.utils.Sequence.

Solution

tf.keras.preprocessing.image_dataset_from_directory(
    directory, labels='inferred', label_mode='int', class_names=None,
    color_mode='rgb', batch_size=32, image_size=(256, 256), shuffle=True, seed=None,
    validation_split=None, subset=None, interpolation='bilinear', follow_links=False
)

If you define batch size it will generate data according to batch size otherwise default batch size is 32. It is never possible to load the whole data in a single batch in normal computer. For more details read documentation.

Answered By - ashraful16