[FIXED] Tensorflow predict the class of output

Issue

I have tried the example with keras but was not with LSTM. My model is with LSTM in Tensorflow and I am willing to predict the output in the form of classes as the keras model thus with predict_classes.
The Tensorflow model I am trying is something like this:

seq_len=10
n_steps = seq_len-1 
n_inputs = x_train.shape[2]
n_neurons = 50
n_outputs = y_train.shape[1]
n_layers = 2
learning_rate = 0.0001
batch_size =100
n_epochs = 1000
train_set_size = x_train.shape[0]
test_set_size = x_test.shape[0]

tf.reset_default_graph()
X = tf.placeholder(tf.float32, [None, n_steps, n_inputs])
y = tf.placeholder(tf.float32, [None, n_outputs])
layers = [tf.contrib.rnn.LSTMCell(num_units=n_neurons,activation=tf.nn.sigmoid, use_peepholes = True)  for layer in range(n_layers)]

multi_layer_cell = tf.contrib.rnn.MultiRNNCell(layers)
rnn_outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)

stacked_rnn_outputs = tf.reshape(rnn_outputs, [-1, n_neurons]) 
stacked_outputs = tf.layers.dense(stacked_rnn_outputs, n_outputs)
outputs = tf.reshape(stacked_outputs, [-1, n_steps, n_outputs])
outputs = outputs[:,n_steps-1,:]                                       
loss = tf.reduce_mean(tf.square(outputs - y)) 
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate) 
training_op = optimizer.minimize(loss)

I am encoding the with sklearn LabelEncoder as:

encoder_train = LabelEncoder()
encoder_train.fit(y_train)
encoded_Y_train = encoder_train.transform(y_train)
y_train = np_utils.to_categorical(encoded_Y_train)

The data is converted to sparse matrix kinda thing in binary format.
When I tried to predict the output I got the following:

actual==>  [[0. 0. 1.]
 [1. 0. 0.]
 [1. 0. 0.]
 [0. 0. 1.]
 [1. 0. 0.]
 [1. 0. 0.]
 [1. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]] 
predicted==>  [[0.3112209  0.3690182  0.31357136]
 [0.31085992 0.36959863 0.31448898]
 [0.31073445 0.3703295  0.31469804]
 [0.31177694 0.37011752 0.3145326 ]
 [0.31220382 0.3692756  0.31515726]
 [0.31232828 0.36947766 0.3149037 ]
 [0.31190437 0.36756667 0.31323162]
 [0.31339088 0.36542615 0.310322  ]
 [0.31598282 0.36328828 0.30711085]] 

What I was expecting for the label based on the encoding done. As the Keras model thus. See the following:

predictions = model.predict_classes(X_test, verbose=True)
print("REAL VALUES:",reverse_category(Y_test,axis=1))
print("PRED VALUES:",predictions)
print("REAL COLORS:")
print(encoder.inverse_transform(reverse_category(Y_test,axis=1)))
print("PREDICTED COLORS:")
print(encoder.inverse_transform(predictions))

The output is something like the following:

REAL VALUES: [1 1 1 ... 1 2 1]
PRED VALUES: [2 1 1 ... 1 2 2]
REAL COLORS:
['ball' 'ball' 'ball' ... 'ball' 'bat' 'ball']
PREDICTED COLORS:
['bat' 'ball' 'ball' ... 'ball' 'bat' 'bat']

Kindly, let me know what I can do in the tensorflow model that will get me the result with respect to the encoding done.
I am using Tensorflow 1.12.0 and Windows 10

Solution

You are trying to map the predicted class probabilities back to class labels. Each row in the list of output predictions contains the three predicted class probabilities. Use np.argmax to obtain the one with the highest predicted probability in order to map to the predicted class label:

import numpy as np

predictions = [[0.3112209,  0.3690182,  0.31357136],
 [0.31085992, 0.36959863, 0.31448898],
 [0.31073445, 0.3703295, 0.31469804],
 [0.31177694, 0.37011752, 0.3145326 ],
 [0.31220382, 0.3692756, 0.31515726],
 [0.31232828, 0.36947766, 0.3149037 ],
 [0.31190437, 0.36756667, 0.31323162],
 [0.31339088, 0.36542615, 0.310322  ],
 [0.31598282, 0.36328828, 0.30711085]] 

np.argmax(predictions, axis=1) 

Gives:

array([1, 1, 1, 1, 1, 1, 1, 1, 1])

In this case, class 1 is predicted 9 times.

As noted in the comments: this is exactly what Keras does under the hood, as you'll see in the source code.

Answered By - sdcbr