[FIXED] Keras LSTM: how to predict beyond validation vs predictions?

Issue

When dealing with time series forecasting, I've seen most people follow these steps when using an LSTM model:

1. Obtain, clean, and pre-process data
2. Take out validation dataset for future comparison with model predictions
3. Initialise and train LSTM model
4. Use a copy of validation dataset to be pre-processed exactly like the training data
5. Use trained model to make predictions on the transformed validation data
6. Evaluate results: predictions vs validation

However, if the model is accurate, how do you make predictions that go beyond the end of the validation period?

The following only accepts data that have been transformed in the same way as the training data, but for predictions that go beyond the validation period, you don't have any input data to feed to the model. So, how do people do this?

# Predictions vs validation
predictions = model.predict(transformed_validation)

# Future predictions
future_predictions = model.predict(?)

Solution

To predict the ith value, your LSTM model need last N values. So if you want to forecast, you should use each prediction to predict the next one.

In other terms you have to loop over something like

prediction = model.predict(X[-N:])
X.append(prediction)

As you can guess, you add your output in your input that's why your predictions can diverge and amplify uncertainty.

Other model are more stable to predict far future.

Answered By - Harold G