[FIXED] How to set sizes of LSTM vector inputs and outputs

Issue

My question is about TensorFlow LSTMs, specifically, how the shape of various inputs and outputs of the LSTM cell are specified in the LSTM constructor. My understanding is that an LSTM cell has the following six inputs and outputs:

1 External input/output
2 Hidden State input/output
3 Context State input/output

I also believe that each of these is a vector with a specified length. My question is: when I'm adding an LSTM to my TensorFlow model, how do I specify the various vector lengths?

Solution

An LSTM layer has several weight vectors but their size is determined from two main quantities: the number of units in the layer and the dimensionality of the input (data dimensions/features, as determined for example from the number of units in the previous layer).

Let's see how those two determine the each of the dimensions that you asked about:

• The external input is, of course, determined by the dimensionality of the input data.
• The external output is determined by the number of units. If the layer has 100 units, then that's the output dimension.
• Both the hidden and context state are fed back (think of how things look if you unroll the network), so they have the same input and output dimensions.
• Also, each unit creates a scalar hidden and context value, so there are many of them as units, meaning that their dimensionality is defined by the number of units in the layer.

Answered By - ATony