[FIXED] Tensorflow : Manually selecting the batch when training deep neural networks

Issue

# x_train.shape[0] = 54000

model.fit(
    x_train, y_train,
    batch_size = 128,
    epochs = 12,
    validation_data = (x_val, y_val)
)

When I am using this fit() method to train a neural network:

• batch_size = 128 means that I randomly pick 54000 // 128 batches of size 128 in my training dataset every epoch.

Are those batches chosen with replacement? I suspect from the docs they're not but I'd like confirmation.

Can I manually choose my batches? I would like to focus on specific images and not others for a given batch, by choosing them personally instead of letting randomness choose for me.

Solution

Are those batches chosen with replacement?

In each individual epoch, no. Of course the entire dataset is used again in the next epoch.

Can I manually choose my batches? I would like to focus on specific images and not others for a given batch, by choosing them personally instead of letting randomness choose for me.

You should create a custom dataset for this, and leave the rest of the training loop (data loader, model etc.) unchanged.

But be aware that the samples in a minibatch are supposed to be random.

Answered By - Nils Werner