[FIXED] Python Scipy Interpolate Splrep Interpolation does not fit to the points correctly and creates fake maximums and minimums

Issue

I am trying to do a plot of some experimental data and find a smooth interpolation for it. I have tried scipy.interpolate.splrep and it creates weird ups and downs between my known data (effectively creating fake maximums and minimums). I've also tried UnivariateSpline and savgol_filter, but they all yield similar results. I have also tried varying the weight factors, although I don't really know what to put for those (I tried to change the weights for the 4th and 5th points in the dataset to be more similar, but it did not change at all). The value of k=2 works just a little better.

import matplotlib.pyplot as plt
import numpy as np
from scipy.interpolate import splrep, splev
import matplotlib.pyplot as plt 
 
x =  [0,0.001226, 0.002454, 0.003696, 0.01,0.027,0.058, 0.068]
y =  [0, 101.85916358, 203.71832716, 305.57749074, 356.50707253, 407.43665432, 458.3662361,  470.58933573]

plt.plot(x,y)

itp_spline = splrep(x,y)
x_new = np.linspace(min(x), max(x), num=100)
y_new = splev(x_new, tck=itp_spline)

plt.plot(x_new, y_new)

weights = [1,1,4,100,200,40,30,2]

itp_spline_2 = splrep(x, y, w=weights, k=2, s=1)
x_new = np.linspace(min(x), max(x), num=100)
y_new = splev(x_new, tck=itp_spline_2)

plt.plot(x_new, y_new, linestyle="dashed")

The blue line is the actual data of the experiment. The orange line is a simple spline from splrep, and the green dashed line is after editing the w, k, and s parameters. The pink line (drawn in photoshop) is more similar to what I want.

What am I doing wrong? Can I get it to work with this method but varying the weights so the 4th and 5th values are more important to the spline? Does anyone know another algorithm that may yield closer results to what I want?

Solution

You don't have very much data and the points are not spaced closely, so the cubic spline interpolation (the default when using something like scipy.interpolate.splrep and scipy.interpolate.make_interp_spline) will have overshoots and undershoots in between the points.

I can think of 4 options for what you can do.

1. Accept the results as they are.
2. Collect more data so it isn't as coarse.
3. Use a first-order fit (i.e. linear interpolation).
4. Curve fit based on the known underlying function (if one exists).

For now, here is how to make a first-order fit using scipy.make_interp_spline:

from scipy.interpolate import make_interp_spline

spl = make_interp_spline(x, y, k=1)
x_new = np.linspace(min(x), max(x), num=100)
y_new = spl(x_new)

Edit: @ev-br suggested using scipy.interpolate.PchipInterpolator, which is a monotonic interpolator. It does well on this example.

from scipy.interpolate import PchipInterpolator

spl = PchipInterpolator(x, y, k=1)
x_new = np.linspace(min(x), max(x), num=100)
y_new = spl(x_new)

Answered By - jared