[FIXED] keras_tuner does not work in time series forecasting

Issue

I tried to tune hyperparameters of my gru model using keras_tuner, but it did not work. It worked when I run the code without validation data here:tuner.search(x_train, epochs=5, validation_data=x_val). But the 'optimal' hyperparameters I got showed a ridiculously underfitting when I checked it although the loss values were very good during running keras_tuner. How can I correctly tune the parameters?

'''

def windowed_dataset_test(series, window_size, batch_size):
    dataset = tf.data.Dataset.from_tensor_slices(series)
    dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)
    dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))
    dataset = dataset.map(lambda window: (window[:-1], window[-1]))
    dataset = dataset.batch(batch_size).prefetch(1)
    return dataset
class MyHyperModel(HyperModel):
    def build(self, hp):
        strategy = tf.distribute.MirroredStrategy()
        window_size = hp.Int('window_size', min_value=5, max_value=60, step=3)
        with strategy.scope():
            model = tf.keras.models.Sequential()
            for i in range(hp.Int("num_layers", min_value=1, max_value=5, step=1)):  # tuning the num of layers
                model.add(tf.keras.layers.GRU(
                    units=hp.Int("units", min_value=10, max_value=500, step=10),  # tuning the num of units
                    input_shape=[window_size, 1],  # use self.window_size here
                    return_sequences=True,
                ))
            model.add(tf.keras.layers.Dense(1))
            learning_rate = hp.Float("lr", min_value=1e-8, max_value=1e-2, sampling="log")  # tuning the learning late
            model.compile(
                optimizer=tf.optimizers.SGD(learning_rate=learning_rate, momentum=hp.Float('momentum', min_value=0, max_value=0.9, step=0.1)),
                # tuning the momentum
                loss="mse",
                metrics="mse")
        return model

    def fit(self, hp: object, model: object, x: object, **kwargs: object) -> object:
        batch_size = hp.Choice('batch_size', [8, 16, 32, 64, 128])
        x = windowed_dataset_test(x_train, hp.get('window_size'), batch_size)

        return model.fit(x, **kwargs)

tuner = keras_tuner.BayesianOptimization(
    MyHyperModel(),
    objective=Objective('val_loss', direction='min'),
    max_trials=50,
    overwrite=True,
    directory="hyperprameter_tuning",
    project_name="GRU_only_price_baysian",
)

series_price = df['nationwide_price']
x_train = series_price['2010-03-13':'2021-12-31'].to_numpy()
x_test = series_price['2022-01-01':].to_numpy()
val_split = int(len(x_train) * 0.9)  # or any other split ratio
x_val = x_train[val_split:]
x_train = x_train[:val_split]

tuner.search(x_train, epochs=5, validation_data=x_val)
with tf.device('/device:GPU:0'):
    tuner.search(x_train, epochs=3)

hypermodel = MyHyperModel()
best_hp = tuner.get_best_hyperparameters()[0:10]
print(best_hp)

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

'''

Solution

This typically happens when you use an array where a single boolean value is expected.

In your code, the error is likely coming from the line:

tuner.search(x_train, epochs=5, validation_data=x_val)

To fix this error, you should modify your code as follows:

• Check the data type and structure of x_train and x_val to ensure they are NumPy arrays.

• Ensure that the validation_data argument of tuner.search() is
  correctly formatted. It should be a tuple containing the validation
  data and labels (if applicable).

• Make sure that the shapes of x_train and x_val are compatible with
  the input requirements of your model. They should have the same
  number of features and compatible dimensions.

Assuming x_train and x_val are NumPy arrays, ensure they have compatible shapes.

tuner.search(x=x_train, epochs=5, validation_data=(x_val, y_val))

In the code above, replace y_val with your validation labels if you have them. By ensuring that your data is correctly formatted and providing the validation data as a tuple, you should be able to resolve the "truth value of an array" error.

Answered By - user3433050