[FIXED] Error in plotting the decision boundary for SVC Laplace kernel

Issue

I'm trying to plot the decision boundary of the SVM classifier using a precomputed Laplace kernel (code below) on the similar lines of this scikit-learn post. I'm taking test points as mesh grid values (xx, yy) just like as mentioned in the post and train points as X and y. I'm able to fit the pre-computed kernel using train points.

import numpy as np
#from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.metrics.pairwise import laplacian_kernel

#Load the iris data
iris_data = load_iris()

#Split the data and target
X = iris_data.data[:, :2]
y = iris_data.target

#Step size in mesh plot
h = 0.02

#Convert X and y to a numpy array
X = np.array(X)
y = np.array(y)

#Using Laplacian kernel - https://scikit-learn.org/stable/modules/metrics.html#laplacian-kernel
K = np.array(laplacian_kernel(X, gamma=.5))
svm = SVC(kernel='precomputed').fit(K, np.ravel(y))

# create a mesh to plot in
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                     np.arange(y_min, y_max, h))

# Plot the decision boundary. For that, we will assign a color to each
# point in the mesh [x_min, x_max]x[y_min, y_max].
#plt.subplot(2, 2, i + 1)
#plt.subplots_adjust(wspace=0.4, hspace=0.4)

# Calculate the gram matrix for test points. Here is where the error is coming. xx- test, X-train.
K_test = np.array(laplacian_kernel(xx, X,  gamma=.5)) 

#Predict using the gram matrix for test
Z = svm.predict(np.c_[K_test])

# Put the result into a color plot
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, cmap=plt.cm.coolwarm, alpha=0.8)

# Plot also the training points
plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.coolwarm)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.xlim(xx.min(), xx.max())
plt.ylim(yy.min(), yy.max())
plt.xticks(())
plt.yticks(())
plt.title('SVC with Laplace kernel')

plt.show()

However, when I try to plot the decision boundary on graph for grid points, I get the below error.

Traceback (most recent call last):
  File "/home/user/Src/laplce.py", line 37, in <module>
    K_test = np.array(laplacian_kernel(xx, X,  gamma=.5)) 
  File "/home/user/.local/lib/python3.9/site-packages/sklearn/metrics/pairwise.py", line 1136, in laplacian_kernel
    X, Y = check_pairwise_arrays(X, Y)
  File "/home/user/.local/lib/python3.9/site-packages/sklearn/utils/validation.py", line 63, in inner_f
    return f(*args, **kwargs)
  File "/home/user/.local/lib/python3.9/site-packages/sklearn/metrics/pairwise.py", line 160, in check_pairwise_arrays
    raise ValueError("Incompatible dimension for X and Y matrices: "
ValueError: Incompatible dimension for X and Y matrices: X.shape[1] == 280 while Y.shape[1] == 2

So, how do I resolve the error and plot the decision boundary for iris data ? Thanks in advance

Solution

The issue is getting your meshgrid into the same dimensions as the training matrix, before applying the laplacian. So if we run the code below to fit the svm :

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.metrics.pairwise import laplacian_kernel

iris_data = load_iris()

X = iris_data.data[:, :2]
y = iris_data.target
h = 0.02

K = laplacian_kernel(X,gamma=.5)
svm = SVC(kernel='precomputed').fit(K, y)

Create the grid like you did:

x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
x_test = np.meshgrid(np.arange(x_min, x_max, h),
                     np.arange(y_min, y_max, h))

xx,yy = np.meshgrid(np.arange(x_min, x_max, h),np.arange(y_min, y_max, h))

Your original input into the laplacian was (150,2) so you need to basically put your xx,yy into 2 columns:

x_test = np.vstack([xx.ravel(),yy.ravel()]).T

K_test = laplacian_kernel(x_test, X,  gamma=.5)
Z = svm.predict(K_test)
Z = Z.reshape(xx.shape)

Then plot:

plt.contourf(xx, yy, Z, cmap=plt.cm.coolwarm, alpha=0.8)
plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.coolwarm)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.xlim(xx.min(), xx.max())
plt.ylim(yy.min(), yy.max())

The points are more or less correct, you can see it does not resolve 1,2 very well:

pd.crosstab(y,svm.predict(K))

col_0   0   1   2
row_0           
0   49  1   0
1   0   35  15
2   0   11  39

Answered By - StupidWolf