[FIXED] Can scipy.optimize minimize functions of complex variables at all and how?

Issue

I am trying to minimize a function of a complex (vector) variable using scipy.optimize. My results so far indicate that it may not be possible. To investigate the problem, I have implemented a simple example - minimize the 2-norm of a complex vector with an offset:

import numpy as np
from scipy.optimize import fmin

def fun(x):
    return np.linalg.norm(x - 1j * np.ones(2), 2)

sol = fmin(fun, x0=np.ones(2) + 0j)

The output is

Optimization terminated successfully.
         Current function value: 2.000000
         Iterations: 38
         Function evaluations: 69

>>> sol
array([-2.10235293e-05,  2.54845649e-05])

Clearly, the solution should be

array([0.+1.j, 0.+1.j])

Disappointed with this outcome, I have also tried scipy.optimize.minimize:

from scipy.optimize import minimize

def fun(x):
    return np.linalg.norm(x - 1j * np.ones(2), 1)

sol = minimize(fun, x0=np.ones(2) + 0j)

The output is

>>> sol
      fun: 2.0
 hess_inv: array([[ 9.99997339e-01, -2.66135332e-06],
       [-2.66135332e-06,  9.99997339e-01]])
      jac: array([0., 0.])
  message: 'Optimization terminated successfully.'
     nfev: 24
      nit: 5
     njev: 6
   status: 0
  success: True
        x: array([6.18479071e-09+0.j, 6.18479071e-09+0.j])

Not good either. I have tried specifying all of the possible methods for minimize (supplying the Jacobian and Hessian as necessary), but none of them reach the correct result. Most of them cause ComplexWarning: Casting complex values to real discards the imaginary part, indicating that they cannot handle complex numbers correctly.

Is this possible at all using scipy.optimize?

If so, I would very much appreciate if someone can tell me what I am doing wrong.

If not, do you perhaps have suggestions for alternative optimization tools (for Python) that allow this?

Solution

The minimization methods of SciPy work with real arguments only. But minimization on the complex space Cⁿ amounts to minimization on R²ⁿ, the algebra of complex numbers never enters the consideration. Thus, adding two wrappers for conversion from Cⁿ to R²ⁿ and back, you can optimize over complex numbers.

def real_to_complex(z):      # real vector of length 2n -> complex of length n
    return z[:len(z)//2] + 1j * z[len(z)//2:]

def complex_to_real(z):      # complex vector of length n -> real of length 2n
    return np.concatenate((np.real(z), np.imag(z)))

sol = minimize(lambda z: fun(real_to_complex(z)), x0=complex_to_real(np.ones(2) + 0j))
print(real_to_complex(sol.x))   # [-7.40376620e-09+1.j -8.77719406e-09+1.j]

You mention Jacobian and Hessian... but minimization only makes sense for real-valued functions, and those are never differentiable with respect to complex variables. The Jacobian and Hessian would have to be computed over R²ⁿ anyway, treating the real and imaginary parts as separate variables.

Answered By - user6655984