Issue
I am trying to feed stock data to Conv2D. But ran into dimension problem. I have no idea how to solve it and need help. Below are detailed steps that I have implemented.
I have attached data and code in the following link: https://drive.google.com/drive/folders/1snsQ-96AeRn521oc0aQVlTTd9nHbtyjO?usp=sharing
by the code itself should run. It will download the data automatically. but ive taken out the featuers to simplify the run. So it will have 5 features in the attached code.
but to give you quick glance of The Problem I had-----------------------
1. Got stock data and generated some features, it looks like:
2. Add time step to it by using:
def reshape_data(X, y, period=28):
n_past = period # number of days to look back in the past and compile into a time series
trainX = []
trainY = np.array(y.iloc[n_past:])
trainY = trainY[..., np.newaxis]
for i in range(n_past, len(X)):
trainX.append(X[i - n_past:i, 0:X.shape[1]])
trainX = np.array(trainX)
return trainX, trainY
Note:
data can be found here
https://drive.google.com/drive/folders/1snsQ-96AeRn521oc0aQVlTTd9nHbtyjO?usp=sharing
I have applied pca on it. But simply convert it into numpy and apply reshap_data() on trainX should work
trainX, trainY = reshape_data(X_train_pca, y_train, period=30)
3. shape
trainX (5768, 30, 30) # 5768-rows, 30- time steps, 30- # of features
trainY (5768,1)
4. Add 1 axis after train X
trainX = trainX[...,np.newaxis]
trainX is now (5768, 30, 30, 1)
5. Build model
6. fit and run
model.compile(optimizer=Adam(learning_rate=0.01) , metrics="mse", loss='binary_crossentropy')
reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss',factor=0.5,patience=10,verbose=0,mode='auto',min_delta=0.0002,cooldown=0,min_lr=0.0001)
early_stop = tf.keras.callbacks.EarlyStopping(monitor="val_loss", patience=80, mode="min", restore_best_weights = True)
history = model.fit(trainX, trainY, epochs=300,
batch_size= 512, shuffle=False, verbose = 1,
# validation_data=(testX, testY),
validation_split=0.2,
callbacks=[early_stop, reduce_lr] )
7. ERROR
I thought since I have convered the stock into 30,30,1 should looks like a image dataset, which would enable tensorflow to work. But somehow it doesn't
Solution
Add two layers after your convolution layer:
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))
And do not mix up tensorflow.keras
and keras
. Rather just use tensorflow.keras
for everything.
Answered By - AloneTogether
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.