[FIXED] Plot the SHAP values for LSTM model in python

Issue

I have the following code that is working.

import numpy as np
import shap
from tensorflow import keras

X = np.array([[(1,2,3,3,1),(3,2,1,3,2),(3,2,2,3,3),(2,2,1,1,2),(2,1,1,1,1)],
              [(4,5,6,4,4),(5,6,4,3,2),(5,5,6,1,3),(3,3,3,2,2),(2,3,3,2,1)],
              [(7,8,9,4,7),(7,7,6,7,8),(5,8,7,8,8),(6,7,6,7,8),(5,7,6,6,6)],
              [(7,8,9,8,6),(6,6,7,8,6),(8,7,8,8,8),(8,6,7,8,7),(8,6,7,8,8)],
              [(4,5,6,5,5),(5,5,5,6,4),(6,5,5,5,6),(4,4,3,3,3),(5,5,4,4,5)],
              [(4,5,6,5,5),(5,5,5,6,4),(6,5,5,5,6),(4,4,3,3,3),(5,5,4,4,5)],
              [(1,2,3,3,1),(3,2,1,3,2),(3,2,2,3,3),(2,2,1,1,2),(2,1,1,1,1)]])
y = np.array([0, 1, 2, 2, 1, 1, 0])

# Updated model with correct input shape
model = keras.Sequential([
    keras.layers.LSTM(128, return_sequences=True, input_shape=(5, 5)),  # LSTM layer with return sequences
    keras.layers.LSTM(128, return_sequences=False),  # Another LSTM layer
    keras.layers.Flatten(),
    keras.layers.Dense(128, activation='relu'),
    keras.layers.Dense(3, activation='softmax')  # 3 output classes
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Train the model
model.fit(X, y, epochs=10)

# Use GradientExplainer with the model itself
explainer = shap.GradientExplainer(model, X)
shap_values = explainer.shap_values(X)
print(shap_values)

I want to display a nice plot of SHAP values.

I tryied the following line of code shap.summary_plot(shap_values, X, feature_names=['Feature 1', 'Feature 2', 'Feature 3', 'Feature 4', 'Feature 5']) but is not working

Solution

With your LSTM model, specify the class and datapoint you want to explain, and you'll satisfy summary_plot expectations of data shapes, i.e.:

cls = 0
idx = 0
shap.summary_plot(shap_values[cls][:,idx,:], X[:,idx,:])

Answered By - Sergey Bushmanov