[FIXED] Softmax and Hinge functions in Python

Issue

I'm trying to implement the Hinge loss function in Python and faced with some misleadings.

In some sources that I used to read (for example, "Regression Analysis in Python"under Luca Massoron) states that Hinge sometimes calls as Softmax function.

But for me it is kind of strange because, Hinge:

and Softmax is just exponential function like:

I made that function in Python (for Softmax) this way:

def softmax(x):
   e_x = np.exp(x - np.max(x))
   return e_x/e_x.sum(axis=0)

Have two questions:

1. Can I use that softmax function like an equivalent to hinge function?
2. If not, how can hinge be implemented in Python?

Thanks.

Solution

Can I use that softmax function like an equivalent to hinge function?

no - they are not equivalent.

a hinge function is a loss function and do not provide well-calibrated probabilities, whereas softmax is a mapping function (one that maps a set of scores into a distribution, one that sums to one).

If not, how can hinge be implemented in Python?

this following snippet captures the essence of hinge loss functions:

import numpy as np
import matplotlib.pyplot as plt

xmin, xmax = -1, 2
xx = np.linspace(xmin, xmax, 100)
plt.plot(xx, np.where(xx < 1, 1 - xx, 0), label="Hinge loss")

Answered By - mynameisvinn