[FIXED] Overflow Error in Python's numpy.exp function

Issue

I want to use numpy.exp like this:

cc = np.array([
    [0.120,0.34,-1234.1]
])

print 1/(1+np.exp(-cc))

But this gives me error:

/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:5: RuntimeWarning: overflow encountered in exp

I can't understand why? How can I fix this? It seems the problem is with third number (-1234.1)

Solution

As fuglede says, the issue here is that np.float64 can't handle a number as large as exp(1234.1). Try using np.float128 instead:

>>> cc = np.array([[0.120,0.34,-1234.1]], dtype=np.float128)
>>> cc
array([[ 0.12,  0.34, -1234.1]], dtype=float128)
>>> 1 / (1 + np.exp(-cc))
array([[ 0.52996405,  0.58419052,  1.0893812e-536]], dtype=float128)

Note however, that there are certain quirks with using extended precision. It may not work on Windows; you don't actually get the full 128 bits of precision; and you might lose the precision whenever the number passes through pure python. You can read more about the details here.

For most practical purposes, you can probably approximate 1 / (1 + <a large number>) to zero. That is to say, just ignore the warning and move on. Numpy takes care of the approximation for you (when using np.float64):

>>> 1 / (1 + np.exp(-cc))
/usr/local/bin/ipython3:1: RuntimeWarning: overflow encountered in exp
  #!/usr/local/bin/python3.4
array([[ 0.52996405,  0.58419052,  0.        ]])

If you want to suppress the warning, you could use scipy.special.expit, as suggested by WarrenWeckesser in a comment to the question:

>>> from scipy.special import expit
>>> expit(cc)
array([[ 0.52996405,  0.58419052,  0.        ]])

Answered By - Praveen