[FIXED] Curve fitting sticks too much to the initial parameters

Issue

I´ve been trying to fit a curve, using scipy.curve_fit, but it seems, that it sticks to some local minimum (very close to the initial parameter guess) and doesn´t try any other combinations. Usually stops after 100-200 iterations. As you can see, the fits are often terrible. The expected shape is: short constant line, slope line, (optional constant line), slope line down, short constant line

def piecewise_linear(X, x1, x2, x22, x3, y1, y2, y3):
    return  np.piecewise(X,  [(X < x1), (x1 <= X) & (X < x2), (x2 <= X) & (X <= x22), (x22 < X) & (X < x3), X >= x3],
                  [lambda X: y1, lambda X: y1 + (y2 - y1) / (x2 - x1) * (X - x1),
                  lambda X: y2, lambda X: y2 + (y3 - y2) / (x3 - x22) * (X - x22), lambda X: y3])  

for i in range(num of vectors): 
    bounds = ([0, global_min_left_x, global_max_x, global_max_x,  -np.inf, -np.inf,-np.inf],
              [global_max_x, global_max_x, global_min_right_x, vector_length, np.inf,np.inf, np.inf])
    additional_kwargs = {'method': 'dogbox', 'ftol': 1e-15, 'xtol': 1e-15, 'maxfev': 100000}
    params, _ = curve_fit(piecewise_linear, X, Z, bounds=bounds,
                         p0=(global_min_left_x, global_max_x,global_max_x, global_min_right_x,
                                          global_min_left_z, global_max_z, global_min_right_z),
                                      **additional_kwargs)

    fitted_curve = [piecewise_linear(x, *params) for x in X]

Fit example

Maybe the solution is in setting the right kwargs (tbh, I am not really sure if I understand it correctly). I have also tried different p0, but the problem was the same - almost no fitting, basically just connected the p0 points.

Maybe it could be useful to look at the data I am trying to fit, so there is one example https://pastebin.com/f0xF9fsd

Solution

You have a scaling issue, the data you posted is on the scale of 1e-8 which confuses the optimization algorithm.

You have to rescale the data first, and then as this optimization can stuck in local minima, you should try to restart it from random initial points. I tried the following code on the data you posted (using your notation, first column is X, the second is Z, and function piecewise_linear is left as you defined):

ntrials = 1000
nparams = 7  # number of parameters to optimize

scale = max(max(abs(X)), max(abs(Z)))
print(f'scale: {scale}')
X = X / scale
Z = Z / scale

min_err = np.inf
best_params = None
for i in range(ntrials):
    try:
        params, stats = curve_fit(piecewise_linear, X, Z,
                                  p0=np.random.randn(nparams))
    except:
        # Ignores RuntimeError: Optimal parameters not found:
        # Number of calls to function has reached maxfev
        continue
    fitted_curve = [piecewise_linear(x, *params) for x in X]
    err = np.mean(np.square(Z - fitted_curve))
    # print(f'i: {i}: {err}')
    if err < min_err:
        min_err = err
        best_params = params

Printing scale shows around 6e-8 and you can multiply the parameters with this to scale them back if you need to. The best parameters of a 1000 trials gives me a mean squared error around 4e-6 on the rescaled data, and good curve:

Answered By - gabalz