[FIXED] Need to force overlapping for seaborn's heatmap and kdeplot

Issue

I'm trying to combine seaborn's heatmap and kdeplot in one figure, but so far the result is not very promising since I cannot find a way to make them overlap. As a result, the heatmap is just squeezed to the left side of the figure.

I think the reason is that seaborn doesn't seem to recognize the x-axis as the same one in two charts (see picture below), although the data points are exactly the same. The only difference is that for heatmap I needed to pivot them, while for the kdeplot pivoting is not needed.

Therefore, data for the axis are coming from the same dataset, but in the different forms as it can be seen in the code below.

The dataset sample looks something like this:

X           Y       Z
7,75        280     52,73
3,25        340     54,19
5,75        340     53,61
2,5         180     54,67
3           340     53,66
1,75        340     54,81
4,5         380     55,18
4           240     56,49
4,75        380     55,17
4,25        180     55,40
2           420     56,42
2,25        380     54,90

My code:

import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

f, ax = plt.subplots(figsize=(11, 9), dpi=300)
plt.tick_params(bottom='on')

# dataset is just a pandas frame with data
X1 = dataset.iloc[:, :3].pivot("X", "Y", "Z")
X2 = dataset.iloc[:, :2]

ax = sns.heatmap(X1, cmap="Spectral")
ax.invert_yaxis()

ax2 = plt.twinx()

sns.kdeplot(X2.iloc[:, 1], X2.iloc[:, 0], ax=ax2, zorder=2)
ax.axis('tight')

plt.show()

Please help me with placing kdeplot on top of the heatmap. Ideally, I would like my final plot to look something like this:

Any tips or hints will be greatly appreciated!

Solution

The question can be a bit hard to understand, because the dataset can't be "just some data". The X and Y values need to lie on a very regular grid. No X,Y combination can be repeated, but not all values appear. The kdeplot will then show where the used values of X,Y are concentrated.

Such a dataset can be simulated by first generating dummy data for a full grid, and then take a subset.

Now, a seaborn heatmap uses categorical X and Y axes. Such axes are very hard to align with the kdeplot. To obtain a similar heatmap with numerical axes, ax.pcolor() can be used.

from matplotlib import pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np

xs = np.arange(2, 10, 0.25)
ys = np.arange(150, 400, 10)
# first create a dummy dataset over a full grid
dataset = pd.DataFrame({'X': np.repeat(xs, len(ys)),
                        'Y': np.tile(ys, len(xs)),
                        'Z': np.random.uniform(50, 60, len(xs) * len(ys))})
# take a random subset of the rows
dataset = dataset.sample(200)

fig, ax = plt.subplots(figsize=(11, 9), dpi=300)

X1 = dataset.pivot("X", "Y", "Z")
collection = ax.pcolor(X1.columns, X1.index, X1, shading='nearest', cmap="Spectral")
plt.colorbar(collection, ax=ax, pad=0.02)
# default, cut=3, which causes a lot of surrounding whitespace
sns.kdeplot(x=dataset["Y"], y=dataset["X"], cut=1.5, ax=ax)
fig.tight_layout()
plt.show()

Answered By - JohanC