[FIXED] Pandas merge dataframes based on closest match

Issue

I have the following 2 dataframes (df_a,df_b):

df_a

    N0_YLDF
0   11.79
1   7.86
2   5.78
3   5.35
4   6.32
5   11.79
6   6.89
7   10.74


df_b
    N0_YLDF N0_DWOC
0   6.29    4
1   2.32    4
2   9.10    4
3   4.89    4
4   10.22   4
5   3.80    3
6   5.55    3
7   6.36    3

I would like to add a column N0_DWOC in df_a, such that the value in that column is from the row where df_a['N0_YLDF'] is closest to df_b['N0_YLDF'].

Right now, I am doing a simple merge but that does not do what I want

Solution

You could find the cutoff values which are midway between the (sorted) values in df_b['N0_YLDF']. Then call pd.cut to categorize the values in df_a['N0_YLDF'], with the cutoff values being the bin edges:

import numpy as np
import pandas as pd

df_a = pd.DataFrame({ 'N0_YLDF': [11.79, 7.86, 5.78, 5.35, 6.32, 11.79, 6.89, 10.74]})
df_b = pd.DataFrame({ 'N0_YLDF':[6.29, 2.32, 9.10, 4.89, 10.22, 3.80, 5.55, 6.36] })

edges, labels = np.unique(df_b['N0_YLDF'], return_index=True)
edges = np.r_[-np.inf, edges + np.ediff1d(edges, to_end=np.inf)/2]
df_a['N0_DWOC'] = pd.cut(df_a['N0_YLDF'], bins=edges, labels=df_b.index[labels])
print(df_a)

yields

In [293]: df_a
Out[293]: 
   N0_YLDF N0_DWOC
0    11.79       4
1     7.86       2
2     5.78       6
3     5.35       6
4     6.32       0
5    11.79       4
6     6.89       7
7    10.74       4

To join the two DataFrames on N0_DWOC you could use:

print(df_a.join(df_b, on='N0_DWOC', rsuffix='_b'))

which yields

   N0_YLDF N0_DWOC  N0_YLDF_b
0    11.79       4      10.22
1     7.86       2       9.10
2     5.78       6       5.55
3     5.35       6       5.55
4     6.32       0       6.29
5    11.79       4      10.22
6     6.89       7       6.36
7    10.74       4      10.22

Answered By - unutbu